AI Knows Best, Until It Doesn’t: When Human-AI Interaction Breaks Down

AI Knows Best, Until It Doesn’t: When Human-AI Interaction Breaks Down

AI promises better, faster, and safer decisions – but what happens when that promise outpaces reality? In this talk, I draw on recent work from our HCI group at the University of St. Gallen to show where and why human-AI interaction breaks down, and what this means for designers and users.

I focus on two complementary studies. First, a systematic analysis of “Death by GPS” incidents reveals how over-reliance on navigation systems can lead to strandings, wrong-way driving, and fatal crashes, highlighting concrete design implications for safer routing. Second, a placebo study using Fitts’ Law shows that simply labelling a standard computer mouse as “AI-assisted” significantly inflates user expectations – without improving performance, usability, or workload. Together, these cases expose a central tension in UX today: people often trust systems that either over-promise or fail precisely when their recommendations matter most.

Johannes Schöning is Professor of Computer Science at the University of St. Gallen, where he leads the Human-Computer Interaction (HCI) group. His research focuses on HCI, human-centred AI, and geoinformatics. Before joining St. Gallen, he held positions at the University of Bremen, Hasselt University, UCL, and DFKI. He is an ACM Distinguished Member and recipient of the ACM Eugene L. Lawler Award and two Google Research Awards.