Basic description of the topic:
In modern surgical environments, intelligent assistant systems are playing an increasingly important role in supporting the preparation, monitoring and documentation of operating room workflows. The automatic recognition of surgical instruments and the verification of instrument tray completeness can be an important step toward the objective analysis of surgical workflows, the improvement of patient safety and the development of surgical data science applications.
The candidate’s task is to develop a machine vision-based system for the recognition of surgical hand instruments and the automatic verification of instrument trays in the simulated operating room environment of the Óbuda University MedTech Innovation and Education Center. The work includes the creation of a custom image database under different instrument arrangements, camera positions, lighting conditions and partial occlusions, the annotation of the collected data, and the application of machine learning methods for instrument detection and categorization. The aim of the developed system is to determine whether a given instrument tray contains the predefined set of instruments and to indicate missing or incorrectly placed instruments.
Detailed tasks:
- Literature overview in the field of surgical instrument recognition, object detection and operating room workflow analysis;
- Getting acquainted with the operating room unit, instrument set and relevant surgical layouts of the MedTech Innovation and Education Center;
- Development of an image acquisition protocol for different instrument tray configurations, camera positions and lighting conditions;
- Creation and annotation of a custom image database;
- Training and evaluation of an object detection or image segmentation-based AI model;
- Development of instrument set verification logic for identifying missing or incorrectly placed instruments;
- Testing the developed system in a simulated operating room environment and evaluating the results.
Over the course of the project, the student will have the opportunity to become familiar with the research infrastructure of the Óbuda University MedTech Innovation and Education Center and the Antal Bejczy Center for Intelligent Robotics, and to get involved in the research and development activities carried out there.