Weapon-Target Assignment by Reinforcement Learning with Pointer Network

No AccessTechnical NotesWeapon–Target Assignment by Reinforcement Learning with Pointer NetworkHyungho Na, Jaemyung Ahn and Il-Chul MoonHyungho Na https://orcid.org/0000-0002-7687-2513Korea Advanced Institute of Science and Technology, Daejeon 34141, Republic of Korea*Ph.D. Candidate, Department of Aerospace Engineering.Search for more papers by this author, Jaemyung Ahn https://orcid.org/0000-0003-4971-5130Korea Advanced Institute of Science and Technology, Daejeon 34141, Republic of Korea†Associate Professor, Department of Aerospace Engineering. Senior Member AIAA.Search for more papers by this author and Il-Chul Moon https://orcid.org/0000-0002-1798-1306Korea Advanced Institute of Science and Technology, Daejeon 34141, Republic of Korea‡Professor, Department of Industrial and Systems Engineering.Search for more papers by this authorPublished Online:27 Nov 2022https://doi.org/10.2514/1.I011150SectionsRead Now ToolsAdd to favoritesDownload citationTrack citations ShareShare onFacebookTwitterLinked InRedditEmail About References [1] Ponda S. S., Johnson L. B., Geramifard A. and How J. P., “Cooperative Mission Planning for Multi-UAV Teams,” Handbook of Unmanned Aerial Vehicles, Vol. 2, Aug. 2015, pp. 1447–1490. https://doi.org/10.1007/978-90-481-9707-1_16 CrossrefGoogle Scholar[2] Johnson L. B., Choi H.-L., Ponda S. S. and How J. P., “Decentralized Task Allocation Using Local Information Consistency Assumptions,” Journal of Aerospace Information Systems, Vol. 14, No. 2, 2017, pp. 103–122. https://doi.org/10.2514/1.I010461 LinkGoogle Scholar[3] Wang Z., Delahaye D., Farges J.-L. and Alam S., “Air Traffic Assignment for Intensive Urban Air Mobility Operations,” Journal of Aerospace Information Systems, Vol. 18, No. 11, 2021, pp. 860–875. https://doi.org/10.2514/1.I010954 LinkGoogle Scholar[4] Sheu J.-B., “A Novel Dynamic Resource Allocation Model for Demand-Responsive City Logistics Distribution Operations,” Transportation Research Part E: Logistics and Transportation Review, Vol. 42, No. 6, 2006, pp. 445–472. https://doi.org/10.1016/j.tre.2005.05.004 CrossrefGoogle Scholar[5] Krichman M., Ghose D., Speyer J. L. and Shamma J. S., “Theater Level Campaign Resource Allocation,” Proceedings of the 2001 American Control Conference (Cat. No. 01CH37148), Inst. of Electrical and Electronics Engineers, New York, Vol. 6, 2001, pp. 4716–4721. https://doi.org/10.1109/acc.2001.945727 Google Scholar[6] Ahuja R. K., Kumar A., Jha K. C. and Orlin J. B., “Exact and Heuristic Algorithms for the Weapon-Target Assignment Problem,” Operations Research, Vol. 55, No. 6, 2007, pp. 1136–1146. https://doi.org/10.1287/opre.1070.0440 CrossrefGoogle Scholar[7] Lloyd S. P. and Witsenhausen H. S., “Weapons Allocation is NP-Complete,” 1986 Summer Computer Simulation Conference, Soc. for Modelling and Simulation International (SCS), San Diego, CA, 1986, pp. 1054–1058. Google Scholar[8] Lee Z.-J., Lee C.-Y. and Su S.-F., “An Immunity-Based Ant Colony Optimization Algorithm for Solving Weapon–Target Assignment Problem,” Applied Soft Computing, Vol. 2, No. 1, 2002, pp. 39–47. https://doi.org/10.1016/S1568-4946(02)00027-3 CrossrefGoogle Scholar[9] Bo Z., Feng-xing Z. and Jia-hua W., “A Novel Approach to Solving Weapon-Target Assignment Problem Based on Hybrid Particle Swarm Optimization Algorithm,” Proceedings of 2011 International Conference on Electronic & Mechanical Engineering and Information Technology, Inst. of Electrical and Electronics Engineers, New York, Vol. 3, 2011, pp. 1385–1387. https://doi.org/10.1109/EMEIT.2011.6023352 Google Scholar[10] Wang S. and Chen W., “Solving Weapon-Target Assignment Problems by Cultural Particle Swarm Optimization,” 2012 4th International Conference on Intelligent Human-Machine Systems and Cybernetics, Inst. of Electrical and Electronics Engineers, New York, Vol. 1, 2012, pp. 141–144. https://doi.org/10.1109/IHMSC.2012.41 Google Scholar[11] Lee Z.-J., Su S.-F. and Lee C.-Y., “Efficiently Solving General Weapon-Target Assignment Problem by Genetic Algorithms with Greedy Eugenics,” IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), Vol. 33, No. 1, 2003, pp. 113–121. https://doi.org/10.1109/TSMCB.2003.808174 CrossrefGoogle Scholar[12] Cho D.-H. and Choi H.-L., “Greedy Maximization for Asset-Based Weapon-Target Assignment with Timedependent Rewards,” Cooperative Control of Multi-Agent Systems: Theory and Applications, Wiley, Hoboken, NJ, 2017, pp. 115–139. https://doi.org/10.1002/9781119266235.ch5 CrossrefGoogle Scholar[13] Shin M.-K., Lee D. and Choi H.-L., “Weapon-Target Assignment Problem with Interference Constraints Using Mixed-Integer Linear Programming,” arXiv preprint arXiv:1911.12567, 2019. https://doi.org/10.48550/arXiv.1911.12567 Google Scholar[14] Lee D., Shin M. K. and Choi H.-L., “Weapon Target Assignment Problem with Interference Constraints,” AIAA Scitech 2020 Forum, AIAA Paper 2020-0388, 2020. https://doi.org/10.2514/6.2020-0388 Google Scholar[15] Bello I., Pham H., Le Q. V., Norouzi M. and Bengio S., “Neural Combinatorial Optimization with Reinforcement Learning,” arXiv preprint arXiv:1611.09940, 2016. https://doi.org/10.48550/arXiv.1611.09940 Google Scholar[16] Nazari M., Oroojlooy A., Snyder L. and Takac M., “Reinforcement Learning for Solving the Vehicle Routing Problem,” Advances in Neural Information Processing Systems, Vol. 31, Curran Associates, Red Hook, NY, Dec. 2018, pp. 9861–9871. https://doi.org/10.48550/arXiv.1802.04240 Google Scholar[17] Grondman I., Busoniu L., Lopes G. A. and Babuska R., “A Survey of Actor-Critic Reinforcement Learning: Standard and Natural Policy Gradients,” IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), Vol. 42, No. 6, 2012, pp. 1291–1307. https://doi.org/10.1109/TSMCC.2012.2218595 CrossrefGoogle Scholar[18] Vinyals O., Fortunato M. and Jaitly N., “Pointer Networks,” Advances in Neural Information Processing Systems, Vol. 28, Curran Associates, Red Hook, NY, Dec. 2015, pp. 2692–2700. https://doi.org/10.48550/arXiv.1506.03134 Google Scholar[19] Vaswani A., Shazeer N., Parmar N., Uszkoreit J., Jones L., Gomez A. N., Kaiser Ł. and Polosukhin I., “Attention is All You Need,” Advances in Neural Information Processing Systems, Vol. 30, Curran Assoc., Red Hook, NY, Dec. 2017, pp. 6000–6010. https://doi.org/10.48550/arXiv.1706.03762 Google Scholar[20] Bello I., Zoph B., Vaswani A., Shlens J. and Le Q. V., “Attention Augmented Convolutional Networks,” Proceedings of the IEEE/CVF International Conference on Computer Vision, Oct. 2019, pp. 3286–3295. https://doi.org/10.48550/arXiv.1904.09925 Google Scholar[21] Kool W., van Hoof H. and Welling M., “Attention, Learn to Solve Routing Problems!” International Conference on Learning Representations, May 2019. https://doi.org/10.48550/arXiv.1803.08475 Google Scholar[22] Sutskever I., Vinyals O. and Le Q. V., “Sequence to Sequence Learning with Neural Networks,” Advances in Neural Information Processing Systems, Vol. 27, Dec. 2014, pp. 3104–3112. https://doi.org/10.48550/arXiv.1409.3215 Google Scholar[23] Sutton R. S. and Barto A. G., Reinforcement Learning: An Introduction, MIT Press, Cambridge, MA, 2018, pp. 329–332, Chap. 13. https://doi.org/0.1109/TNN.1998.712192 Google Scholar Previous article FiguresReferencesRelatedDetails What's Popular Volume 20, Number 1January 2023 Metrics CrossmarkInformationCopyright © 2022 by the American Institute of Aeronautics and Astronautics, Inc. All rights reserved. All requests for copying and permission to reprint should be submitted to CCC at www.copyright.com; employ the eISSN 2327-3097 to initiate your request. See also AIAA Rights and Permissions www.aiaa.org/randp. TopicsAlgorithms and Data StructuresArtificial IntelligenceArtificial Neural NetworkComputing SystemComputing and InformaticsComputing, Information, and CommunicationControl SystemsData ScienceEvolutionary AlgorithmGenetic AlgorithmGuidance, Navigation, and Control SystemsMachine LearningMilitary ScienceMilitary TechnologyMissile Systems, Dynamics and TechnologyRoboticsRobotics SystemsWeapon Systems KeywordsWeapon Target AssignmentReinforcement LearningMarkov Decision ProcessArtificial Neural NetworkProbability DistributionParticle Swarm OptimizationStochastic Gradient DescentGenetic AlgorithmMixed Integer Linear ProgrammingMulti Agent SystemAcknowledgmentThis work was supported by Theater Defense Research Center funded by Defense Acquisition Program Administration under Grant UD200043CD.PDF Received17 May 2022Accepted7 November 2022Published online27 November 2022