Projective simulation for classical learning agents: a comprehensive investigation

Abstract

We study the model of projective simulation (PS), a novel approach to arti cial intelligence based on stochastic processing of episodic memory which was recently introduced [1]. Here we provide a detailed analysis of the model and examine its performance, including its achievable e ciency, its learning times and the way both properties scale with the problems' dimension. In addition, we situate the PS agent in di erent learning scenarios, and study its learning abilities. A variety of new scenarios are being considered, thereby demonstrating the model's exibility. Further more, to put the PS scheme in context, we compare its performance with those of Q-learning and learning classi er systems, two popular models in the eld of reinforcement learning. It is shown that PS is a competitive arti cial intelligence model of unique properties and strengths.