A new approach to collaborative AI development

Developing AI models from surgical video recordings faces two major challenges: the videos display highly complex processes and are often several hours long, making them difficult to analyze due to large file sizes, and privacy regulations that limit the exchange of video data between institutions. To address these challenges, the researchers combined two AI approaches for the first time in surgical video analysis: Swarm Learning, which enables decentralized AI training across multiple institutions without exchanging sensitive patient data, and Multiple Instance Learning, a machine learning method that allows models to learn patient-level outcomes from collections of images and data points rather than requiring detailed annotations frame by frame.

This combination allows us to develop AI collaboratively while keeping clinical data at the hospitals where they were generated,” Dr Fiona Kolbinger, one of the senior co-authors of the paper, explains. “At the same time, the model can learn from learn from longer video sections or even entire procedure recordings  instead of relying on labor-intensive frame-by-frame annotations.”

Proof-of-concept using appendectomy videos

For the study, the researchers assembled a multicenter dataset of 397 laparoscopic appendectomy videos collected over approximately one and a half years from six clinical centers in Germany and Portugal. In addition to the surgical videos, the dataset includes patient characteristics (age, sex, body-mass index), clinical information (medical history, symptoms, laboratory test results), treatment details, and expert assessments of disease stage. Using this dataset, the team trained AI models to predict the stage of appendicitis directly from complete surgical videos. The decentralized Swarm Learning approach achieved performance comparable to conventional centralized model training, demonstrating that collaborative AI development is possible without sharing and collecting sensitive clinical data. Predicting appendicitis stage was intentionally chosen as a relatively straightforward clinical task. The study establishes the methodological foundation for applying this approach to more complex and clinically relevant questions.

A valuable resource for the research community

Alongside the proof-of-concept-study, the researchers published the development dataset in Scientific Data. The openly available resource contains 330 real-world appendectomy videos from five German institutions together with matched clinical metadata and expert annotations. Unlike most publicly available surgical video datasets, which contain videos alone, this dataset links surgical recordings with patient-level clinical information and outcomes. The authors hope this resource will accelerate the development and evaluation of clinically meaningful AI methods for surgical care.

 Toward personalized postoperative care

The researchers see their work as an important first step toward AI systems that support personalized surgical care. In the future, similar methods could help identify patients at increased risk of postoperative complications or predict long-term outcomes after complex cancer surgery by combining surgical videos with additional clinical data. In a next step, the researchers plan a follow-up multicenter project led by the Department of Visceral, Thoracic, and Vascular Surgery of Dresden University Hospital focusing on colorectal cancer surgery to further develop and evaluate the methodology in a more complex clinical setting. The study illustrates how privacy-preserving AI can enable collaborative research across institutions while protecting sensitive patient data – an important prerequisite for bringing trustworthy AI into routine surgical practice.

This research is a collaborative effort across several institutions:

• EKFZ for Digital Health, Faculty of Medicine and University Hospital Carl Gustav Carus, TUD Dresden University of Technology
• Department of Translational Surgical Oncology, National Center for Tumor Diseases (NCT/UCC) Dresden
• Department of Visceral, Thoracic, and Vascular Surgery, Department of Pediatric Surgery, Department of Medicine I, University Hospital Carl Gustav Carus, Faculty of Medicine, TUD Dresden University of Technology
• Centre for Tactile Internet with Human-in-the-Loop (CeTI), TUD Dresden University of Technology
• National Center for Tumor Diseases (NCT), Medical Oncology, University Hospital Heidelberg
• Asklepios-ASB Hospital Radeberg
• Dresden-Friedrichstadt General Hospital, Department of General and Visceral Surgery, Dresden
• Elisabeth Hospital Ravensburg, Department of General, Visceral, and Thoracic Surgery, Ravensburg
• Joseph-Stift Hospital Dresden
• Diakonissen Hospital Dresden
• Hospital Prof. Dr. Fernando Fonseca, Lisbon, Portugal

A complete list of involved institutions is available in the original publications.