Introduction

Sequential learning provides a rigorous framework to address the trade-offs between exploration and exploitation in face of uncertainty, vividly captured in the multi-armed bandit problem. In this chapter, we provide an overview of the sequential learning framework and a taxonomy of the types of problems where the framework applies.