Publications

2025

1. 

   Using Fitts’ Law to Benchmark Assisted Human-Robot Performance

   Jiahe Pan, Jonathan Eden, Denny Oetomo, and Wafa Johal

   In 2025 20th ACM/IEEE International Conference on Human-Robot Interaction (HRI), 2025

   Abs Bib PDF Code Website

   Shared control systems aim to combine human and robot abilities to improve task performance. However, achieving optimal performance requires that the robot’s level of assistance adjusts the operator’s cognitive workload in response to the task difficulty. Understanding and dynamically adjusting this balance is crucial to maximizing efficiency and user satisfaction. In this paper, we propose a novel benchmarking method for shared control systems based on Fitts’ Law to formally parameterize the difficulty level of a target-reaching task. With this we systematically quantify and model the effect of task difficulty (i.e. size and distance of target) and robot autonomy on task performance and operators’ cognitive load and trust levels. Our empirical results (N=24) not only show that both task difficulty and robot autonomy influence task performance, but also that the performance can be modelled using these parameters, which may allow for the generalization of this relationship across more diverse setups. We also found that the users’ perceived cognitive load and trust were influenced by these factors. Given the challenges in directly measuring cognitive load in real-time, our adapted Fitts’ model presents a potential alternative approach to estimate cognitive load through determining the difficulty level of the task, with the assumption that greater task difficulty results in higher cognitive load levels. We hope that these insights and our proposed framework inspire future works to further investigate the generalizability of the method, ultimately enabling the benchmarking and systematic assessment of shared control quality and user impact, which will aid in the development of more effective and adaptable systems.

   @inproceedings{pan2025using,
     title = {Using Fitts' Law to Benchmark Assisted Human-Robot Performance},
     author = {Pan, Jiahe and Eden, Jonathan and Oetomo, Denny and Johal, Wafa},
     year = {2025},
     publisher = {IEEE Press},
     booktitle = {2025 20th ACM/IEEE International Conference on Human-Robot Interaction (HRI)},
     pages = {203-212},
   }

2. 

   OfficeMate: Pilot Evaluation of an Office Assistant Robot

   Jiahe Pan, Sarah Schömbs, Yan Zhang, Ramtin Tabatabaei, Muhammad Bilal, and Wafa Johal

   In 2025 20th ACM/IEEE International Conference on Human-Robot Interaction (HRI), 2025

   Abs Bib PDF Code

   Office Assistant Robots (OARs) offer a promising solution to proactively provide in-situ support to enhance employee well-being and productivity in office spaces. We introduce OfficeMate, a social OAR designed to assist with practical tasks, foster social interaction, and promote health and well-being. Through a pilot evaluation with seven participants in an office environment, we found that users see potential in OARs for reducing stress and promoting healthy habits and value the robot’s ability to provide companionship and physical activity reminders in the office space. However, concerns regarding privacy, communication, and the robot’s interaction timing were also raised. The feedback highlights the need to carefully consider the robot’s appearance and behaviour to ensure it enhances user experience and aligns with office social norms. We believe these insights will better inform the development of adaptive, intelligent OAR systems for future office space integration.

   @inproceedings{pan2025officemate,
     title = {OfficeMate: Pilot Evaluation of an Office Assistant Robot},
     author = {Pan, Jiahe and Sch{\"o}mbs, Sarah and Zhang, Yan and Tabatabaei, Ramtin and Bilal, Muhammad and Johal, Wafa},
     year = {2025},
     publisher = {IEEE Press},
     booktitle = {2025 20th ACM/IEEE International Conference on Human-Robot Interaction (HRI)},
     pages = {1529-1533},
   }

3. 

   Assisting MoCap-Based Teleoperation of Robot Arm using Augmented Reality Visualisations

   Qiushi Zhou, Antony Chacon, Jiahe Pan, and Wafa Johal

   In 2025 20th ACM/IEEE International Conference on Human-Robot Interaction (HRI), 2025

   Abs Bib PDF Code

   Teleoperating a robot arm involves the human operator positioning the robot’s end-effector or programming each joint. Whereas humans can control their own arms easily by integrating visual and proprioceptive feedback, it is challenging to control an external robot arm in the same way, due to its inconsistent orientation and appearance. We explore teleoperating a robot arm through motion-capture (MoCap) of the human operator’s arm with the assistance of augmented reality (AR) visualisations. We investigate how AR helps teleoperation by visualising a virtual reference of the human arm alongside the robot arm to help users understand the movement mapping. We found that the AR overlay of a humanoid arm on the robot in the same orientation helped users learn the control. We discuss findings and future work on MoCap-based robot teleoperation.

   @inproceedings{zhou2025assisting,
     title = {Assisting MoCap-Based Teleoperation of Robot Arm using Augmented Reality Visualisations},
     author = {Zhou, Qiushi and Chacon, Antony and Pan, Jiahe and Johal, Wafa},
     year = {2025},
     publisher = {IEEE Press},
     booktitle = {2025 20th ACM/IEEE International Conference on Human-Robot Interaction (HRI)},
     pages = {1765-1769},
   }

2024

1. 

   Effects of Shared Control on Cognitive Load and Trust in Teleoperated Trajectory Tracking

   Jiahe Pan, Jonathan Eden, Denny Oetomo, and Wafa Johal

   IEEE Robotics and Automation Letters, 2024

   Abs Bib PDF Code Website

   Teleoperation is increasingly recognized as a viable solution for deploying robots in hazardous environments. Controlling a robot to perform a complex or demanding task may overload operators resulting in poor performance. To design a robot controller to assist the human in executing such challenging tasks, a comprehensive understanding of the interplay between the robot’s autonomous behavior and the operator’s internal state is essential. In this letter, we investigate the relationships between robot autonomy and both the human user’s cognitive load and trust levels, and the potential existence of three-way interactions in the robot-assisted execution of the task. Our user study (N = 24) results indicate that while the autonomy level influences the teleoperator’s perceived cognitive load and trust, there is no clear interaction between these factors. Instead, these elements appear to operate independently, thus highlighting the need to consider both cognitive load and trust as distinct but interrelated factors in varying the robot autonomy level in shared-control settings. This insight is crucial for the development of more effective and adaptable assistive robotic systems.

   @article{pan2024effects,
     title = {Effects of Shared Control on Cognitive Load and Trust in Teleoperated Trajectory Tracking},
     author = {Pan, Jiahe and Eden, Jonathan and Oetomo, Denny and Johal, Wafa},
     journal = {IEEE Robotics and Automation Letters},
     year = {2024},
     publisher = {IEEE},
   }

2. 

   FaceVis: Exploring a Robot’s Face for Affective Visualisation Design

   Sarah Schömbs, Jiahe Pan, Yan Zhang, Jorge Goncalves, and Wafa Johal

   In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, 2024

   Abs Bib PDF Code Website

   In this paper we examine the use of a robot’s face as an interface for affective visualisation design, a concept that we name FaceVis. We conducted a design workshop with 9 experts to explore metaphorical ideas on how to leverage a robot’s physicality, appearance and agency to convey data and communicate emotion. We present insights on potential challenges, benefits and pitfalls when using a robot’s face to visualise data. Our results show that this approach has the potential to enhance user engagement, support self-reflection and elicit empathic concern. We contribute three design considerations and provide future research directions to investigate a robot’s face as an interface for visualisation design.

   @inproceedings{schombs2024facevis,
     title = {FaceVis: Exploring a Robot's Face for Affective Visualisation Design},
     author = {Sch{\"o}mbs, Sarah and Pan, Jiahe and Zhang, Yan and Goncalves, Jorge and Johal, Wafa},
     booktitle = {Extended Abstracts of the CHI Conference on Human Factors in Computing Systems},
     pages = {1--10},
     year = {2024},
   }

3. 

   A Review of Differentiable Simulators

   Rhys Newbury, Jack Collins, Kerry He, Jiahe Pan, Ingmar Posner, David Howard, and Akansel Cosgun

   IEEE Access, 2024

   Abs Bib PDF Website

   Differentiable simulators continue to push the state of the art across a range of domains including computational physics, robotics, and machine learning. Their main value is the ability to compute gradients of physical processes, which allows differentiable simulators to be readily integrated into commonly employed gradient-based optimization schemes. To achieve this, a number of design decisions need to be considered representing trade-offs in versatility, computational speed, and accuracy of the gradients obtained. This paper presents an in-depth review of the evolving landscape of differentiable physics simulators. We introduce the foundations and core components of differentiable simulators alongside common design choices. This is followed by a practical guide and overview of open-source differentiable simulators that have been used across past research. Finally, we review and contextualize prominent applications of differentiable simulation. By offering a comprehensive review of the current state-of-the-art in differentiable simulation, this work aims to serve as a resource for researchers and practitioners looking to understand and integrate differentiable physics within their research. We conclude by highlighting current limitations as well as providing insights into future directions for the field.

   @article{newbury2024review,
     title = {A Review of Differentiable Simulators},
     author = {Newbury, Rhys and Collins, Jack and He, Kerry and Pan, Jiahe and Posner, Ingmar and Howard, David and Cosgun, Akansel},
     journal = {IEEE Access},
     year = {2024},
     publisher = {IEEE},
   }

2023

1. 

   Variable Grasp Pose and Commitment for Trajectory Optimization

   Jiahe Pan, Kerry He, Jia Ming Ong, and Akansel Cosgun

   In 2023 5th International Congress on Human-Computer Interaction, Optimization and Robotic Applications (HORA), 2023

   Abs Bib PDF

   We propose enhancing trajectory optimization methods through the incorporation of two key ideas: variable-grasp pose sampling and trajectory commitment. Our iterative approach samples multiple grasp poses, increasing the likelihood of finding a solution while gradually narrowing the optimization horizon towards the goal region for improved computational efficiency. We conduct experiments comparing our approach with sampling-based planning and fixed-goal optimization. In simulated experiments featuring 4 different task scenes, our approach consistently outperforms baselines by generating lower-cost trajectories and achieving higher success rates in challenging constrained and cluttered environments, at the trade-off of longer computation times. Real-world experiments further validate the superiority of our approach in generating lower-cost trajectories and exhibiting enhanced robustness. While we acknowledge the limitations of our experimental design, our proposed approach holds significant potential for enhancing trajectory optimization methods and offers a promising solution for achieving consistent and reliable robotic manipulation.

   @inproceedings{pan2023variable,
     title = {Variable Grasp Pose and Commitment for Trajectory Optimization},
     author = {Pan, Jiahe and He, Kerry and Ong, Jia Ming and Cosgun, Akansel},
     booktitle = {2023 5th International Congress on Human-Computer Interaction, Optimization and Robotic Applications (HORA)},
     pages = {1--6},
     year = {2023},
     organization = {IEEE},
   }