Deep reinforcement learning in agent-based simulations for optimal media planning

Abstract

Agent-based models establish a suitable simulation technique to recreate real complex systems, such as those approached in marketing. Reinforcement learning is about learning a behavior policy in order to maximize a long-term reward signal. In this work, we develop a deep reinforcement learning agent that represents a brand in an agent-based model of a market. The goal of the learning agent is to obtain a marketing investment strategy that improves the awareness of its corresponding brand in the marketing scenario. In opposition to conventional marketing investment strategies, the learned strategy is dynamic, so the agent makes its investment decision on-line based on the current state of the market. We choose the Double Deep Q-Network algorithm to train this agent on diverse instances of the model, each of them with different knowledge levels of the brand. First we adjust a subset of the hyperparameters of Double Deep Q-Network on two of the model instances, and then we use the best configuration found to train the agent on all the available instances. The brand agent learns a dynamic policy that optimizes brand’s awareness levels. We perform an expert analysis of the policy obtained, where we observe that the learning brand agent tends to increase investment in media channels with greater awareness impact, but it also invests in other channels according to the situation and the characteristics of the model instance. These results show the benefits of having an on-line dynamic learning environment in a decision support system for media planning in marketing.