Basic description of the topic:
The objective analysis of hand movements, instrument use patterns and workflows during surgical procedures can play an important role in surgical skill assessment, operating room workflow analysis and the development of intelligent surgical assistant systems. Hand and instrument tracking based on camera images enables the automatic extraction of features such as motion trajectories, amount of motion, velocity, idle time, hand–instrument interactions, and the sequence of surgical subtasks. Instruments.
The candidate’s task is to develop a machine vision-based system for the analysis of manual surgical training tasks performed in the simulated operating room environment of the Óbuda University MedTech Innovation and Education Center. The work includes the development of an appropriate camera setup, the creation of a custom video database during the execution of predefined surgical subtasks, and the application of AI-based hand detection, instrument tracking and workflow analysis methods. The aim of the developed system is the quantitative characterization of surgical motion patterns and the automatic recognition or segmentation of the main steps of the training tasks.
Detailed tasks:
- Literature overview in the field of surgical motion pattern analysis, hand detection, object tracking, workflow analysis and surgical skill assessment;
- Getting acquainted with the operating room unit, instrument set and relevant manual surgical training tasks of the MedTech Innovation and Education Center;
- Development of a camera setup and video acquisition protocol in a simulated operating room environment;
- Creation of a custom video database based on different surgical subtasks, instrument use patterns and user executions;
- Annotation of the video data according to hand movements, instrument use events and workflow steps;
- Application, and if necessary training or fine-tuning, of hand detection, instrument tracking and event recognition algorithms;
- Estimation of motion trajectories, velocity, amount of motion, idle time and hand–instrument interactions;
- Development of a method suitable for the automatic recognition of surgical subtasks or workflow phases;
- Evaluation of the extracted features and their applicability for surgical workflow analysis or skill assessment;
- Testing the developed system in a simulated operating room environment and evaluating the results.
Over the course of the project, the student will have the opportunity to become familiar with the research infrastructure of the Óbuda University MedTech Innovation and Education Center and the Antal Bejczy Center for Intelligent Robotics, and to get involved in the research and development activities carried out there.