Deep Deterministic Policy Gradient for Nested Parallel Negotiation

In automated negotiation research, achieving effective negotiation strategies using reinforcement learning automatically is an essential topic recently. However, the existing research focuses on naive bilateral negotiations even though negotiations in real life have been done in parallel. Nested parallel negotiation is one of the critical categories of parallel negotiation, such as supply chain management, where an agent is required to simultaneously negotiate in two directions: the supply and demand sides. Especially the existing work of getting an effective negotiation strategy using reinforcement learning has not been achieved for nested parallel negotiation. This paper proposes a new reinforcement learning architecture based on Deep Deterministic Policy Gradient (DDPG) to learn an effective negotiation strategy for nested parallel negotiation. We formulate the environment for reinforcement learning for the nested parallel negotiation and propose a neural network architecture to learn a negotiation strategy efficiently. We evaluate the negotiation strategy based on our proposed architecture in the simulation environment for supply chain management. The results demonstrate that our proposed agent can achieve validation utilities that is comparable to or higher than the existing rule-based and learning-based agent strategies.