Joint Multi-Agent Reinforcement Learning and Message-Passing for Distributed Multi-Uav Network Management using Conflict Graphs

Yeryeong Cho; Hyunsoo Lee; Soohyun Park; Joongheon Kim

doi:10.1109/NOMS57970.2025.11073688

Joint Multi-Agent Reinforcement Learning and Message-Passing for Distributed Multi-Uav Network Management using Conflict Graphs

Yeryeong Cho
, Hyunsoo Lee
, Soohyun Park^*
, Joongheon Kim

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

This paper proposes a novel algorithm for distributed multi unmanned aerial vehicles (UAVs) cooperation in dynamic and unstable network environments by employing joint multi-agent reinforcement learning (MARL) and message-passing. To realize MARL, our proposed algorithm utilizes a centralized training with distributed execution (CTDE) framework. However, CTDE-based algorithms should be able to recognize the communications between UAVs and centralized server, which is not possible in every single time step. Therefore, after conducting centralized training for MARL, the distribution of the model for distributed execution should be re-designed. For this objective, a conflict graph-based approach is used, which enables graph-edge if two UAVs can talk to each other. Based on this conflict graph construction, message-passing is used to select UAVs for communication with the server. The non-selected UAVs can receive their models from conflict graph-connected UAVs.

Original language	English
Title of host publication	Proceedings of IEEE/IFIP Network Operations and Management Symposium 2025, NOMS 2025
Editors	Doug Zuckerman, Mehmet Ulema, Noura Limam, Young-Tak Kim, Lisandro Zambenedetti Granville, Vinicius Fulber-Garcia
Publisher	Institute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)	9798331531638
DOIs	https://doi.org/10.1109/NOMS57970.2025.11073688
Publication status	Published - 2025
Event	38th IEEE/IFIP Network Operations and Management Symposium, NOMS 2025 - Honolulu, United States Duration: 2025 May 12 → 2025 May 16

Publication series

Name	Proceedings of IEEE/IFIP Network Operations and Management Symposium 2025, NOMS 2025

Conference

Conference	38th IEEE/IFIP Network Operations and Management Symposium, NOMS 2025
Country/Territory	United States
City	Honolulu
Period	25/5/12 → 25/5/16

Bibliographical note

Publisher Copyright:
© 2025 IEEE.

Keywords

Centralized Training with Distributed Execution
Conflict Graph
Multi-Agent Reinforcement Learning

ASJC Scopus subject areas

Artificial Intelligence
Computer Networks and Communications
Information Systems and Management
Modelling and Simulation

Access to Document

10.1109/NOMS57970.2025.11073688

Cite this

Cho, Y., Lee, H., Park, S., & Kim, J. (2025). Joint Multi-Agent Reinforcement Learning and Message-Passing for Distributed Multi-Uav Network Management using Conflict Graphs. In D. Zuckerman, M. Ulema, N. Limam, Y.-T. Kim, L. Z. Granville, & V. Fulber-Garcia (Eds.), Proceedings of IEEE/IFIP Network Operations and Management Symposium 2025, NOMS 2025 (Proceedings of IEEE/IFIP Network Operations and Management Symposium 2025, NOMS 2025). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/NOMS57970.2025.11073688

Joint Multi-Agent Reinforcement Learning and Message-Passing for Distributed Multi-Uav Network Management using Conflict Graphs. / Cho, Yeryeong; Lee, Hyunsoo; Park, Soohyun et al.
Proceedings of IEEE/IFIP Network Operations and Management Symposium 2025, NOMS 2025. ed. / Doug Zuckerman; Mehmet Ulema; Noura Limam; Young-Tak Kim; Lisandro Zambenedetti Granville; Vinicius Fulber-Garcia. Institute of Electrical and Electronics Engineers Inc., 2025. (Proceedings of IEEE/IFIP Network Operations and Management Symposium 2025, NOMS 2025).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Cho, Y, Lee, H, Park, S & Kim, J 2025, Joint Multi-Agent Reinforcement Learning and Message-Passing for Distributed Multi-Uav Network Management using Conflict Graphs. in D Zuckerman, M Ulema, N Limam, Y-T Kim, LZ Granville & V Fulber-Garcia (eds), Proceedings of IEEE/IFIP Network Operations and Management Symposium 2025, NOMS 2025. Proceedings of IEEE/IFIP Network Operations and Management Symposium 2025, NOMS 2025, Institute of Electrical and Electronics Engineers Inc., 38th IEEE/IFIP Network Operations and Management Symposium, NOMS 2025, Honolulu, United States, 25/5/12. https://doi.org/10.1109/NOMS57970.2025.11073688

Cho Y, Lee H, Park S, Kim J. Joint Multi-Agent Reinforcement Learning and Message-Passing for Distributed Multi-Uav Network Management using Conflict Graphs. In Zuckerman D, Ulema M, Limam N, Kim YT, Granville LZ, Fulber-Garcia V, editors, Proceedings of IEEE/IFIP Network Operations and Management Symposium 2025, NOMS 2025. Institute of Electrical and Electronics Engineers Inc. 2025. (Proceedings of IEEE/IFIP Network Operations and Management Symposium 2025, NOMS 2025). doi: 10.1109/NOMS57970.2025.11073688

Cho, Yeryeong ; Lee, Hyunsoo ; Park, Soohyun et al. / Joint Multi-Agent Reinforcement Learning and Message-Passing for Distributed Multi-Uav Network Management using Conflict Graphs. Proceedings of IEEE/IFIP Network Operations and Management Symposium 2025, NOMS 2025. editor / Doug Zuckerman ; Mehmet Ulema ; Noura Limam ; Young-Tak Kim ; Lisandro Zambenedetti Granville ; Vinicius Fulber-Garcia. Institute of Electrical and Electronics Engineers Inc., 2025. (Proceedings of IEEE/IFIP Network Operations and Management Symposium 2025, NOMS 2025).

@inproceedings{a3c5f0b07d7b455eb8004ec0bef50810,

title = "Joint Multi-Agent Reinforcement Learning and Message-Passing for Distributed Multi-Uav Network Management using Conflict Graphs",

abstract = "This paper proposes a novel algorithm for distributed multi unmanned aerial vehicles (UAVs) cooperation in dynamic and unstable network environments by employing joint multi-agent reinforcement learning (MARL) and message-passing. To realize MARL, our proposed algorithm utilizes a centralized training with distributed execution (CTDE) framework. However, CTDE-based algorithms should be able to recognize the communications between UAVs and centralized server, which is not possible in every single time step. Therefore, after conducting centralized training for MARL, the distribution of the model for distributed execution should be re-designed. For this objective, a conflict graph-based approach is used, which enables graph-edge if two UAVs can talk to each other. Based on this conflict graph construction, message-passing is used to select UAVs for communication with the server. The non-selected UAVs can receive their models from conflict graph-connected UAVs.",

keywords = "Centralized Training with Distributed Execution, Conflict Graph, Multi-Agent Reinforcement Learning",

author = "Yeryeong Cho and Hyunsoo Lee and Soohyun Park and Joongheon Kim",

note = "Publisher Copyright: {\textcopyright} 2025 IEEE.; 38th IEEE/IFIP Network Operations and Management Symposium, NOMS 2025 ; Conference date: 12-05-2025 Through 16-05-2025",

year = "2025",

doi = "10.1109/NOMS57970.2025.11073688",

language = "English",

series = "Proceedings of IEEE/IFIP Network Operations and Management Symposium 2025, NOMS 2025",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

editor = "Doug Zuckerman and Mehmet Ulema and Noura Limam and Young-Tak Kim and Granville, \{Lisandro Zambenedetti\} and Vinicius Fulber-Garcia",

booktitle = "Proceedings of IEEE/IFIP Network Operations and Management Symposium 2025, NOMS 2025",

}

TY - GEN

T1 - Joint Multi-Agent Reinforcement Learning and Message-Passing for Distributed Multi-Uav Network Management using Conflict Graphs

AU - Cho, Yeryeong

AU - Lee, Hyunsoo

AU - Park, Soohyun

AU - Kim, Joongheon

PY - 2025

Y1 - 2025

N2 - This paper proposes a novel algorithm for distributed multi unmanned aerial vehicles (UAVs) cooperation in dynamic and unstable network environments by employing joint multi-agent reinforcement learning (MARL) and message-passing. To realize MARL, our proposed algorithm utilizes a centralized training with distributed execution (CTDE) framework. However, CTDE-based algorithms should be able to recognize the communications between UAVs and centralized server, which is not possible in every single time step. Therefore, after conducting centralized training for MARL, the distribution of the model for distributed execution should be re-designed. For this objective, a conflict graph-based approach is used, which enables graph-edge if two UAVs can talk to each other. Based on this conflict graph construction, message-passing is used to select UAVs for communication with the server. The non-selected UAVs can receive their models from conflict graph-connected UAVs.

AB - This paper proposes a novel algorithm for distributed multi unmanned aerial vehicles (UAVs) cooperation in dynamic and unstable network environments by employing joint multi-agent reinforcement learning (MARL) and message-passing. To realize MARL, our proposed algorithm utilizes a centralized training with distributed execution (CTDE) framework. However, CTDE-based algorithms should be able to recognize the communications between UAVs and centralized server, which is not possible in every single time step. Therefore, after conducting centralized training for MARL, the distribution of the model for distributed execution should be re-designed. For this objective, a conflict graph-based approach is used, which enables graph-edge if two UAVs can talk to each other. Based on this conflict graph construction, message-passing is used to select UAVs for communication with the server. The non-selected UAVs can receive their models from conflict graph-connected UAVs.

KW - Centralized Training with Distributed Execution

KW - Conflict Graph

KW - Multi-Agent Reinforcement Learning

UR - https://www.scopus.com/pages/publications/105012224351

U2 - 10.1109/NOMS57970.2025.11073688

DO - 10.1109/NOMS57970.2025.11073688

M3 - Conference contribution

AN - SCOPUS:105012224351

T3 - Proceedings of IEEE/IFIP Network Operations and Management Symposium 2025, NOMS 2025

BT - Proceedings of IEEE/IFIP Network Operations and Management Symposium 2025, NOMS 2025

A2 - Zuckerman, Doug

A2 - Ulema, Mehmet

A2 - Limam, Noura

A2 - Kim, Young-Tak

A2 - Granville, Lisandro Zambenedetti

A2 - Fulber-Garcia, Vinicius

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 38th IEEE/IFIP Network Operations and Management Symposium, NOMS 2025

Y2 - 12 May 2025 through 16 May 2025

ER -

Joint Multi-Agent Reinforcement Learning and Message-Passing for Distributed Multi-Uav Network Management using Conflict Graphs

Abstract

Publication series

Conference

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this