Transforming Surgical Training with AR-Technology

Rethinking Surgical Education Through Technology

Funded by Innosuisse, the PROFICIENCY project aims to rethink the traditional “see one, do one, teach one” approach to surgical education through the integration of Augmented Reality (AR), simulation technologies and data-driven training methods. In collaboration with ETH Zurich, ZHAW and clinical partners, Balgrist University Hospital is developing new educational approaches designed to support surgical residents in an increasingly technology-driven clinical environment.

AR-Guided Surgical Training

A central component of the project is the development of an AR-enabled simulator for total hip arthroplasty, created and tested within the OR-X. The hybrid training platform combines real surgical instruments and specimen-based practice with AR visualizations, allowing trainees to see anatomical structures and potential complications directly during the procedure. Through AR glasses, surgeons receive visual guidance tailored to their level of experience.

The realistic surgical environment of the OR-X enables to bridge the gap between simulation-based learning and operative practice. In parallel, advanced multi-camera systems create photorealistic 3D reconstructions of surgeries combined with AI-supported performance analysis based on tool, hand and eye-tracking data. This allows trainees to replay procedural steps, practice them virtually and continuously assess their technical performance.

Towards Data-Driven Surgical Training

In parallel, advanced multi-camera systems at the OR-X capture procedures to generate photorealistic 3D reconstructions of surgeries. Combined with AI-supported performance analysis, these visualizations create immersive training scenarios and enable objective feedback on surgical precision through tool, hand and eye-tracking data.

By integrating AR, immersive visualization and AI-based evaluation into surgical education, PROFICIENCY contributes to the development of more structured and data-driven training concepts for future surgeons.