A key skill for social robots in the wild will be to understand the structure and dynamics of conversational groups in order to fluidly participate in them. Social scientists have long studied the rich complexity underlying such focused encounters, or F-formations. However, current state-of-the-art algorithms that robots might use to recognize F-formations are highly heuristic and quite brittle. In this report, we explore a data-driven approach to detect F-formations from sets of tracked human positions and orientations, trained and evaluated on two openly available human-only datasets and a small human-robot dataset that we collected. We also discuss the potential for further computational characterization of F-formations beyond simply detecting their occurrence. 

IRON Lab    IRON Lab Website



  • Hooman HedayatiAnnika MuehlbradtDaniel J. Szafir and Sean Andrist. 2020. REFORM: Recognizing F-formations for Social Robots. (to appear) In Proceedings of the 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2020). (Las Vegas, Nevada–October 25-29, 2020).
  • Hooman HedayatiDaniel Szafir, and James Kennedy. 2020. Comparing F-Formations Between Humans and On-Screen Agents. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (CHI EA ’20). Association for Computing Machinery, New York, NY, USA, 1–9. DOI:https://doi.org/10.1145/3334480.3383015 (Honolulu, Hawaii (virtual) – April 25-30, 2020).
  • Hooman Hedayati, Daniel Szafir, and Sean Andrist. 2019. Recognizing F-Formations in the Open World. In 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI '19). DOI: http://dx.doi.org/10.1109/hri.2019.8673233 (Daegu, South Korea – March 11-14, 2019).


Additional researcher: Sean Andrist (Microsoft Research)