Hybrid Hierarchical Learning for Adaptive Persuasion in Human-Robot Interaction

Adaptive learning is critical to helping robots personalize their interactions with people, particularly when considering skills needed by socially assistive robots, such as persuasion. In this letter, we propose a novel, hybrid hierarchical learning architecture for use in social human-robot interaction (HRI) to adapt robot persuasive behaviors to both the static (e.g., need for cognition) and dynamic (e.g., affect) considerations of a user. A learning hierarchy is introduced that uses a contextual bandit approach in the top level to optimize for a static cognition bias and Q-Learning in the lower level to optimize selection of a robot persuasive strategy to deploy that aligns with a user's affect. We compare the performance of our system with a non-hierarchical learning method in simulated experiments for the task of persuading people to do daily exercises. The results show that our hybrid hierarchical architecture outperforms a non-hierarchical benchmark in learning speed and robustness to both longitudinal user change and noisy observations. Our architecture is the first to: 1) persuasively adapt to different users during social HRI considering both static and dynamic user change, and 2) use user state decomposition in persuasive HRI.