Impulsivity and Compulsivity in Bayesian Reinforcement Learning Models of Addiction: A Computational Critique of the Habit Theory

Addiction is sometimes argued to represent an extreme dominance of habitual behaviour, driven by stimulus–response associations, over goal-directed behaviour, involving planning based on action–outcome contingencies. In this chapter, we formalize a recent elaboration on this “habit theory” of addiction using Bayesian reinforcement algorithms as models of habit and planning. In these models, compulsivity and intertemporal impatience, both considered important elements of addiction, can arise through a dominance of habit over planning, but only on the assumption that the planning system does not overvalue addictive rewards. That is, the habit theory of addiction implicitly assumes that the planning system, in contrast to the habit system, ascribes appropriately high values to non-addictive rewards and appropriately low values to addictive rewards. However, recent evidence suggests that goal-directed overvaluation of addictive rewards is a key driver of addiction, which presents a significant challenge for the habit theory. We discuss whether this challenge will prove to be insurmountable.