Basic description of the topic:
In modern operating room environments, intelligent assistant systems are playing an increasingly important role in supporting surgical workflows, documentation and the control of different devices in a contactless manner. Voice and gesture control can be especially useful in environments where sterility, rapid access to information and preserving the user’s attention are important requirements. Such systems may contribute to more efficient support of operating room workflows, simplified documentation and the development of intelligent surgical assistant systems.
The candidate’s task is to develop a prototype of a voice- or gesture-controlled operating room assistant in the simulated operating room environment of the Óbuda University MedTech Innovation and Education Center. The work includes the definition of a relevant operating room command set and use case scenario, such as navigating a surgical checklist, marking events during data acquisition, querying an instrument list, or displaying the status of simple operating room devices. During the development, the student may apply speech recognition, natural language processing, gesture recognition or multimodal HMI methods, and test the prototype in a simulated operating room environment.
Detailed tasks:
- Literature overview in the field of operating room human-machine interfaces, voice control, gesture control and intelligent assistant systems;
- Getting acquainted with the operating room unit, instrument set and relevant operating room workflows of the MedTech Innovation and Education Center;
- Definition of an operating room command set and relevant use case scenarios;
- Development of a voice-based, gesture-based or combined control concept;
- Application, and if necessary training or fine-tuning, of speech recognition, natural language processing, image processing or gesture recognition methods;
- Implementation of the operating room assistant prototype, complemented by a user interface or simple dashboard;
- Testing the system under different noise, lighting and user interaction conditions;
- Evaluation of recognition accuracy, response time, usability and operating room applicability;
- Investigation of further development possibilities toward intelligent operating room assistant systems.
Over the course of the project, the student will have the opportunity to become familiar with the research infrastructure of the Óbuda University MedTech Innovation and Education Center and the Antal Bejczy Center for Intelligent Robotics, and to get involved in the research and development activities carried out there.