From GOMS to Deep Reinforcement Learning

CHI 2022 Online Course, April 26-27

  • Learn to understand, create, and utilize computational cognitive models in HCI contexts.

  • Use Python notebooks to easily simulate interactive task behaviour in various environments.

  • Topics include computational rationality, deep reinforcement learning, parameter inference, model design

This course introduces computational cognitive modeling for researchers and practitioners in the field of HCI. Cognitive models use computer programs to model how users perceive, think, and act in human–computer interaction. They offer a powerful approach for understanding interactive tasks and improving user interfaces. This course starts with a review of classic architecture based models such as GOMS and ACT-R. It then rapidly progresses to introducing modern modelling approaches powered by machine learning methods, in particular deep learning, reinforcement learning (RL), and deep RL. The course is built around hands-on Python programming using notebooks.

The course consists of four 90 minute modules:

  • Introduction to cognitive modeling. In this session we will critically review the history of cognitive modeling in HCI over the past 50 years. The review will cover modeling approaches that include mathematical models of movement time, procedural models of skill, models of eye movements, computational simulations of human information processing, models inspired by how animals forage for food and models of human learning, rationality and decision making. After completing the section the attendees will understand the role of computational cognitive modeling in HCI, and know how the historical approaches have converged into the state-of-the-art of modeling.

  • Deep learning. The session introduces deep learning as an approach to building cognitive models in HCI. Deep learning has been successfully used in modelling various types of interactive behaviour, such as visual search and mid-air interaction in VR. One of the advantages of deep learning is in its ability to discover features and structure in data that are important for explanation and prediction. Deep learning models al