Multi-Agent Reinforcement Learning

by ComputingNotes
Reinforcement Learning

Environment

👁️
Agent 1
{Action, Reward}
👁️
Agent 2
{Action, Reward}
👁️
Agent 3
{Action, Reward}
(Autonomous Agents)
(make their own decisions)

Basics of MARL

Multi-Agent Reinforcement Learning (MARL) extends traditional reinforcement learning by placing multiple autonomous agents into a shared environment. Instead of a single agent learning in isolation, agents in MARL must learn to navigate a dynamic world influenced by the actions of others.

  • Shared Environment The world in which all agents operate. The environment’s state changes based on the combined actions of every agent present.
  • Autonomous Agents Independent decision-makers that observe the environment, take actions, and attempt to maximize their own (or a shared) cumulative reward.
  • Interaction Types Agents can interact in three primary ways: Cooperative (working together for a common goal), Competitive (competing against each other, like in a game of chess), or Mixed (a combination of both).
  • Non-Stationarity A core challenge in MARL. Because multiple agents are learning and updating their behaviors simultaneously, the environment becomes unpredictable (non-stationary) from the perspective of any single agent.

Cooperative Environment

Agents (blue) work together to reach the shared target (green star). They share the same reward function.

Shared Environment Agent 1 Agent 2 Agent n Observation 1 & Reward 1 Observation 2 & Reward 2 Observation n & Reward n action action action Joint Action Integration

Representative Framework of MARL: In this multi-agent architecture, n distinct agents observe the current state of a shared environment and receive individual rewards based on their past behavior. They independently select actions which are aggregated into a single joint action; this combined input loops back to transition the environment into its next state, repeating the cycle.

References

  1. “An introduction to Multi-Agents Reinforcement Learning (MARL) – Hugging Face Deep RL Course,” huggingface.co. https://huggingface.co/learn/deep-rl-course/en/unit7/introduction-to-marl
  2. S. V. Albrecht, F. Christianos, and L. Schäfer, “Multi-Agent Reinforcement Learning: Foundations and Modern Approaches”.
Views: 2
error: