Image captioning via hierarchical attention mechanism and policy gradient optimization.

•We propose a hierarchical attention mechanism to reason on the global features and the local object features for image captioning.•The policy gradient algorithm combined with the GAN is proposed for the training and optimization of the language model, with improvements over MLE training scheme.•Through comprehensive experiments, we validate the proposed algorithm and improved results are achieved on the MSCOCO image captioning dataset.