2024
176.
Bashkanov, O; Rak, M; Engelage, L; Hansen, C
Automatic Patient-level Diagnosis of Prostate Disease with Fused 3D MRI and Tabular Clinical Data Proceedings Article
In: Medical Imaging with Deep Learning, pp. 1225–1238, PMLR, 2024, (ISSN: 2640-3498).
@inproceedings{bashkanov_automatic_2024,
title = {Automatic Patient-level Diagnosis of Prostate Disease with Fused 3D MRI and Tabular Clinical Data},
author = {O Bashkanov and M Rak and L Engelage and C Hansen},
url = {https://proceedings.mlr.press/v227/bashkanov24a.html},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
booktitle = {Medical Imaging with Deep Learning},
pages = {1225–1238},
publisher = {PMLR},
abstract = {Computer-aided diagnosis systems for automatic prostate cancer diagnosis can provide radiologists with decision support during image reading. However, in this case, patient-relevant information often remains unexploited due to the greater focus on the image recognition side, with various imaging devices and modalities, while omitting other potentially valuable clinical data. Therefore, our work investigates the performance of recent methods for the fusion of rich image data and heterogeneous tabular data. Those data may include patient demographics as well as laboratory data, e.g., prostate-specific antigen (PSA). Experiments on the large dataset (3800 subjects) indicated that when using the fusion method with demographic data in clinically significant prostate cancer (csPCa) detection tasks, the mean area under the receiver operating characteristic curve (ROC AUC) has improved significantly from 0.736 to 0.765. We also observed that the naïve concatenation performs similarly or even better than the textbackslashmboxstate-of-the-art fusion modules. We also achieved better prediction quality in grading prostate disease by including more samples from longitudinal PSA profiles in the tabular feature set. Thus, by including the three last PSA samples per patient, the best-performing model has reached AUC of 0.794 and a quadratic weighted kappa score (QWK) of 0.464, which constitutes a significant improvement compared with the image-only method, with ROC AUC of 0.736 and QWK of 0.342.},
note = {ISSN: 2640-3498},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Computer-aided diagnosis systems for automatic prostate cancer diagnosis can provide radiologists with decision support during image reading. However, in this case, patient-relevant information often remains unexploited due to the greater focus on the image recognition side, with various imaging devices and modalities, while omitting other potentially valuable clinical data. Therefore, our work investigates the performance of recent methods for the fusion of rich image data and heterogeneous tabular data. Those data may include patient demographics as well as laboratory data, e.g., prostate-specific antigen (PSA). Experiments on the large dataset (3800 subjects) indicated that when using the fusion method with demographic data in clinically significant prostate cancer (csPCa) detection tasks, the mean area under the receiver operating characteristic curve (ROC AUC) has improved significantly from 0.736 to 0.765. We also observed that the naïve concatenation performs similarly or even better than the textbackslashmboxstate-of-the-art fusion modules. We also achieved better prediction quality in grading prostate disease by including more samples from longitudinal PSA profiles in the tabular feature set. Thus, by including the three last PSA samples per patient, the best-performing model has reached AUC of 0.794 and a quadratic weighted kappa score (QWK) of 0.464, which constitutes a significant improvement compared with the image-only method, with ROC AUC of 0.736 and QWK of 0.342.
2023
175.
Heinrich, F; Bornemann, K; Polenz, L; Lawonn, K; Hansen, C
Clutch & Grasp: Activation gestures and grip styles for device-based interaction in medical spatial augmented reality Journal Article
In: International Journal of Human-Computer Studies, vol. 180, pp. 103117, 2023, ISSN: 1071-5819.
@article{heinrich_clutch_2023,
title = {Clutch & Grasp: Activation gestures and grip styles for device-based interaction in medical spatial augmented reality},
author = {F Heinrich and K Bornemann and L Polenz and K Lawonn and C Hansen},
url = {https://www.sciencedirect.com/science/article/pii/S107158192300126X},
doi = {10.1016/j.ijhcs.2023.103117},
issn = {1071-5819},
year = {2023},
date = {2023-12-01},
urldate = {2023-12-01},
journal = {International Journal of Human-Computer Studies},
volume = {180},
pages = {103117},
abstract = {Presenting medical volume data using augmented reality (AR) can facilitate the identification of anatomical structures, the perception of their spatial relations and the development of mental maps compared to more commonly used monitors. However, interaction methods explored in these conventional settings may not be applicable in AR environments, or perform differently. In terms of mode activation, gestural interaction was shown to be a viable, touchless alternative to traditional input devices, which is desirable in sterile medical use cases. Therefore, we present a user study (n = 21) comparing hand and foot gestures with voice commands for the activation of interaction modes within a projector-based, spatial AR prototype to visualize medical volume data. Interaction itself was performed via hand movements captured by a data glove. Consistent, statistically significant results across measured variables suggest advantages of voice commands. In addition, a second experiment (n = 17) compared the hand-based interaction with two motion-sensitive devices held in power and in precision grip respectively. All modes were activated using voice commands. No considerable differences between tested grip styles could be determined. The findings suggest that the choice of preferable interaction devices is user and use case dependent.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Presenting medical volume data using augmented reality (AR) can facilitate the identification of anatomical structures, the perception of their spatial relations and the development of mental maps compared to more commonly used monitors. However, interaction methods explored in these conventional settings may not be applicable in AR environments, or perform differently. In terms of mode activation, gestural interaction was shown to be a viable, touchless alternative to traditional input devices, which is desirable in sterile medical use cases. Therefore, we present a user study (n = 21) comparing hand and foot gestures with voice commands for the activation of interaction modes within a projector-based, spatial AR prototype to visualize medical volume data. Interaction itself was performed via hand movements captured by a data glove. Consistent, statistically significant results across measured variables suggest advantages of voice commands. In addition, a second experiment (n = 17) compared the hand-based interaction with two motion-sensitive devices held in power and in precision grip respectively. All modes were activated using voice commands. No considerable differences between tested grip styles could be determined. The findings suggest that the choice of preferable interaction devices is user and use case dependent.
174.
Schott, D; Heinrich, F; Stallmeister, L; Moritz, J; Hensen, B; Hansen, C
Is this the vReal Life? Manipulating Visual Fidelity of Immersive Environments for Medical Task Simulation Proceedings Article
In: 2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 1171–1180, IEEE, Sydney, Australia, 2023, ISBN: 979-8-3503-2838-7.
@inproceedings{schott_is_2023,
title = {Is this the vReal Life? Manipulating Visual Fidelity of Immersive Environments for Medical Task Simulation},
author = {D Schott and F Heinrich and L Stallmeister and J Moritz and B Hensen and C Hansen},
url = {https://ieeexplore.ieee.org/document/10316533/},
doi = {10.1109/ISMAR59233.2023.00134},
isbn = {979-8-3503-2838-7},
year = {2023},
date = {2023-10-01},
urldate = {2023-10-01},
booktitle = {2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)},
pages = {1171–1180},
publisher = {IEEE},
address = {Sydney, Australia},
abstract = {Recent developments and research advances contribute to an ever-increasing trend towards quality levels close to what we experience in reality. In this work, we investigate how different degrees of these quality characteristics affect user performance, qualia of user experience (UX), and sense of presence in an example medical task. To this end, a two-way within-subjects design user study was conducted, in which three different levels of visual fidelity were compared. In addition, two different interaction modalities were considered: (1) the use of conventional VR controllers and (2) natural hand interaction using 3D-printed, spatially-registered replicas of medical devices, to interact with their virtual representations. Consistent results indicate that higher degrees of visual fidelity evoke a higher sense of presence and UX. However, user performance was less affected. Moreover, no differences were detected between both interaction modalities for the examined task. Future work should investigate the discovered interaction effects between quality levels and interaction modalities in more detail and examine whether these results can be reproduced in tasks that require more precision. This work provides insights into the implications to consider when studying interactions in VR and paves the way for investigations into early phases of medical product development and workflow analysis.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Recent developments and research advances contribute to an ever-increasing trend towards quality levels close to what we experience in reality. In this work, we investigate how different degrees of these quality characteristics affect user performance, qualia of user experience (UX), and sense of presence in an example medical task. To this end, a two-way within-subjects design user study was conducted, in which three different levels of visual fidelity were compared. In addition, two different interaction modalities were considered: (1) the use of conventional VR controllers and (2) natural hand interaction using 3D-printed, spatially-registered replicas of medical devices, to interact with their virtual representations. Consistent results indicate that higher degrees of visual fidelity evoke a higher sense of presence and UX. However, user performance was less affected. Moreover, no differences were detected between both interaction modalities for the examined task. Future work should investigate the discovered interaction effects between quality levels and interaction modalities in more detail and examine whether these results can be reproduced in tasks that require more precision. This work provides insights into the implications to consider when studying interactions in VR and paves the way for investigations into early phases of medical product development and workflow analysis.
173.
Mielke, T; Joeres, F; Schott, D; Hansen, C
Interactive Registration Methods for Augmented Reality in Robotics: A Comparative Evaluation Proceedings Article
In: 2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), pp. 501–506, IEEE, Sydney, Australia, 2023, ISBN: 979-8-3503-2891-2.
@inproceedings{mielke_interactive_2023,
title = {Interactive Registration Methods for Augmented Reality in Robotics: A Comparative Evaluation},
author = {T Mielke and F Joeres and D Schott and C Hansen},
url = {https://ieeexplore.ieee.org/document/10322246/},
doi = {10.1109/ISMAR-Adjunct60411.2023.00109},
isbn = {979-8-3503-2891-2},
year = {2023},
date = {2023-10-01},
urldate = {2023-10-01},
booktitle = {2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)},
pages = {501–506},
publisher = {IEEE},
address = {Sydney, Australia},
abstract = {Augmented Reality (AR) visualization has shown potential for supporting intuitive and efficient human-robot interaction in a range of tasks. Since all these tasks are spatially related to the robot, the precise positioning of the AR content is critical to the applicability. However, most research has primarily focused on developing visualizations rather than exploring methods for aligning AR content in the robotic workspace. This paper aims to bridge this gap by implementing and comparing different interactive registration methods, including two point-based and one manual approach. We comparatively evaluated these registration methods in a user study (n=21), measuring registration accuracy, duration, and subjective user feedback. Our results indicate that the point-based methods outperform the manual approach in terms of both accuracy and perceived workload. Furthermore, participants achieved significantly faster performance with a point-based approach using physically defined registration points compared to a point-based approach using markers attached to the robot.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Augmented Reality (AR) visualization has shown potential for supporting intuitive and efficient human-robot interaction in a range of tasks. Since all these tasks are spatially related to the robot, the precise positioning of the AR content is critical to the applicability. However, most research has primarily focused on developing visualizations rather than exploring methods for aligning AR content in the robotic workspace. This paper aims to bridge this gap by implementing and comparing different interactive registration methods, including two point-based and one manual approach. We comparatively evaluated these registration methods in a user study (n=21), measuring registration accuracy, duration, and subjective user feedback. Our results indicate that the point-based methods outperform the manual approach in terms of both accuracy and perceived workload. Furthermore, participants achieved significantly faster performance with a point-based approach using physically defined registration points compared to a point-based approach using markers attached to the robot.
172.
Allgaier, M; Huettl, F; Hanke, L; Lang, H; Huber, T; Preim, B; Saalfeld, S; Hansen, C
LiVRSono – Virtual Reality Training with Haptics for Intraoperative Ultrasound Proceedings Article
In: 2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 980–989, IEEE, Sydney, Australia, 2023, ISBN: 979-8-3503-2838-7.
@inproceedings{allgaier_livrsono_2023,
title = {LiVRSono - Virtual Reality Training with Haptics for Intraoperative Ultrasound},
author = {M Allgaier and F Huettl and L Hanke and H Lang and T Huber and B Preim and S Saalfeld and C Hansen},
url = {https://ieeexplore.ieee.org/document/10316488/},
doi = {10.1109/ISMAR59233.2023.00114},
isbn = {979-8-3503-2838-7},
year = {2023},
date = {2023-10-01},
urldate = {2023-10-01},
booktitle = {2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)},
pages = {980–989},
publisher = {IEEE},
address = {Sydney, Australia},
abstract = {One of the biggest challenges in using ultrasound (US) is learning to create a spatial mental model of the interior of the scanned object based on the US image and the probe position. As intraoperative ultrasound (IOUS) cannot be easily trained on patients, we present LiVRSono, an immersive VR application to train this skill. The immersive environment, including an US simulation with patientspecific data as well as haptics to support hand-eye coordination, provides a realistic setting. Four clinically relevant training scenarios were identified based on the described learning goal and the workflow of IOUS for liver. The realism of the setting and the training scenarios were evaluated with eleven physicians, of which six participants are experts in IOUS for liver and five participants are potential users of the training system. The setting, handling of the US probe, and US image were considered realistic enough for the learning goal. Regarding the haptic feedback, a limitation is the restricted workspace of the input device. Three of the four training scenarios were rated as meaningful and effective. A pilot study regarding learning outcome shows positive results, especially with respect to confidence and perceived competence. Besides the drawbacks of the input device, our training system provides a realistic learning environment with meaningful scenarios to train the creation of a mental 3D model when performing IOUS. We also identified important improvements to the training scenarios to further enhance the training experience.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
One of the biggest challenges in using ultrasound (US) is learning to create a spatial mental model of the interior of the scanned object based on the US image and the probe position. As intraoperative ultrasound (IOUS) cannot be easily trained on patients, we present LiVRSono, an immersive VR application to train this skill. The immersive environment, including an US simulation with patientspecific data as well as haptics to support hand-eye coordination, provides a realistic setting. Four clinically relevant training scenarios were identified based on the described learning goal and the workflow of IOUS for liver. The realism of the setting and the training scenarios were evaluated with eleven physicians, of which six participants are experts in IOUS for liver and five participants are potential users of the training system. The setting, handling of the US probe, and US image were considered realistic enough for the learning goal. Regarding the haptic feedback, a limitation is the restricted workspace of the input device. Three of the four training scenarios were rated as meaningful and effective. A pilot study regarding learning outcome shows positive results, especially with respect to confidence and perceived competence. Besides the drawbacks of the input device, our training system provides a realistic learning environment with meaningful scenarios to train the creation of a mental 3D model when performing IOUS. We also identified important improvements to the training scenarios to further enhance the training experience.
171.
Schott, D; Moritz, J; Hansen, C; Joeres, F
The UUXR-Framework: A Draft Classification for Using Extended Reality in Usability and User Experience Research Proceedings Article
In: 2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), pp. 460–465, IEEE, Sydney, Australia, 2023, ISBN: 979-8-3503-2891-2.
@inproceedings{schott_uuxr-framework_2023,
title = {The UUXR-Framework: A Draft Classification for Using Extended Reality in Usability and User Experience Research},
author = {D Schott and J Moritz and C Hansen and F Joeres},
url = {https://ieeexplore.ieee.org/document/10322234/},
doi = {10.1109/ISMAR-Adjunct60411.2023.00100},
isbn = {979-8-3503-2891-2},
year = {2023},
date = {2023-10-01},
urldate = {2023-10-01},
booktitle = {2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)},
pages = {460–465},
publisher = {IEEE},
address = {Sydney, Australia},
abstract = {Conducting human-centered evaluations in extended reality (XR) environments is a growing trend in user research and usability engineering. However, there has been little to no systematic investigation of the emerging methods in this field published to date. The motivation behind our work is to explore and classify strategies and methods for utilizing XR technologies in the context of usability and user experience (UUX) activities. This paper proposes a draft classification framework for the use of XR technologies in UUX activities, combining an informal exploration of relevant literature with established UUX methods. Within this framework, we propose 12 dimensions that we consider potentially relevant for determining whether and how the use of XR technologies can benefit product development and user research. To evaluate the structure and phrasing of our proposed dimensions, we conducted an initial evaluation with UUX professionals (N = 11). We believe that our dimensions form an early-stage foundation for future guidelines aimed at UUX researchers. The framework serves as a tool for assessing different levels of virtualization in UUX work and facilitating knowledge transfer between academia and industry.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Conducting human-centered evaluations in extended reality (XR) environments is a growing trend in user research and usability engineering. However, there has been little to no systematic investigation of the emerging methods in this field published to date. The motivation behind our work is to explore and classify strategies and methods for utilizing XR technologies in the context of usability and user experience (UUX) activities. This paper proposes a draft classification framework for the use of XR technologies in UUX activities, combining an informal exploration of relevant literature with established UUX methods. Within this framework, we propose 12 dimensions that we consider potentially relevant for determining whether and how the use of XR technologies can benefit product development and user research. To evaluate the structure and phrasing of our proposed dimensions, we conducted an initial evaluation with UUX professionals (N = 11). We believe that our dimensions form an early-stage foundation for future guidelines aimed at UUX researchers. The framework serves as a tool for assessing different levels of virtualization in UUX work and facilitating knowledge transfer between academia and industry.
170.
Bashkanov, O; Rak, M; Meyer, A; Engelage, L; Lumiani, A; Muschter, R; Hansen, C
Automatic detection of prostate cancer grades and chronic prostatitis in biparametric MRI Journal Article
In: Computer Methods and Programs in Biomedicine, vol. 239, pp. 107624, 2023, ISSN: 0169-2607.
@article{bashkanov_automatic_2023,
title = {Automatic detection of prostate cancer grades and chronic prostatitis in biparametric MRI},
author = {O Bashkanov and M Rak and A Meyer and L Engelage and A Lumiani and R Muschter and C Hansen},
url = {https://www.sciencedirect.com/science/article/pii/S0169260723002894},
doi = {10.1016/j.cmpb.2023.107624},
issn = {0169-2607},
year = {2023},
date = {2023-09-01},
urldate = {2023-09-01},
journal = {Computer Methods and Programs in Biomedicine},
volume = {239},
pages = {107624},
abstract = {Background and objective:With emerging evidence to improve prostate cancer (PCa) screening, multiparametric magnetic prostate imaging is becoming an essential noninvasive component of the diagnostic routine. Computer-aided diagnostic (CAD) tools powered by deep learning can help radiologists interpret multiple volumetric images. In this work, our objective was to examine promising methods recently proposed in the multigrade prostate cancer detection task and to suggest practical considerations regarding model training in this context. Methods:We collected 1647 fine-grained biopsy-confirmed findings, including Gleason scores and prostatitis, to form a training dataset. In our experimental framework for lesion detection, all models utilized 3D nnU-Net architecture that accounts for anisotropy in the MRI data. First, we explore an optimal range of b-values for diffusion-weighted imaging (DWI) modality and its effect on the detection of clinically significant prostate cancer (csPCa) and prostatitis using deep learning, as the optimal range is not yet clearly defined in this domain. Next, we propose a simulated multimodal shift as a data augmentation technique to compensate for the multimodal shift present in the data. Third, we study the effect of incorporating the prostatitis class alongside cancer-related findings at three different granularities of the prostate cancer class (coarse, medium, and fine) and its impact on the detection rate of the target csPCa. Furthermore, ordinal and one-hot encoded (OHE) output formulations were tested. Results: An optimal model configuration with fine class granularity (prostatitis included) and OHE has scored the lesion-wise partial Free-Response Receiver Operating Characteristic (FROC) area under the curve (AUC) of 1.94 (CI 95%: 1.76–2.11) and patient-wise ROC AUC of 0.874 (CI 95%: 0.793–0.938) in the detection of csPCa. Inclusion of the auxiliary prostatitis class has demonstrated a stable relative improvement in specificity at a false positive rate (FPR) of 1.0 per patient, with an increase of 3%, 7%, and 4% for coarse, medium, and fine class granularities. Conclusions: This paper examines several configurations for model training in the biparametric MRI setup and proposes optimal value ranges. It also shows that the fine-grained class configuration, including prostatitis, is beneficial for detecting csPCa. The ability to detect prostatitis in all low-risk cancer lesions suggests the potential to improve the quality of the early diagnosis of prostate diseases. It also implies an improved interpretability of the results by the radiologist.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Background and objective:With emerging evidence to improve prostate cancer (PCa) screening, multiparametric magnetic prostate imaging is becoming an essential noninvasive component of the diagnostic routine. Computer-aided diagnostic (CAD) tools powered by deep learning can help radiologists interpret multiple volumetric images. In this work, our objective was to examine promising methods recently proposed in the multigrade prostate cancer detection task and to suggest practical considerations regarding model training in this context. Methods:We collected 1647 fine-grained biopsy-confirmed findings, including Gleason scores and prostatitis, to form a training dataset. In our experimental framework for lesion detection, all models utilized 3D nnU-Net architecture that accounts for anisotropy in the MRI data. First, we explore an optimal range of b-values for diffusion-weighted imaging (DWI) modality and its effect on the detection of clinically significant prostate cancer (csPCa) and prostatitis using deep learning, as the optimal range is not yet clearly defined in this domain. Next, we propose a simulated multimodal shift as a data augmentation technique to compensate for the multimodal shift present in the data. Third, we study the effect of incorporating the prostatitis class alongside cancer-related findings at three different granularities of the prostate cancer class (coarse, medium, and fine) and its impact on the detection rate of the target csPCa. Furthermore, ordinal and one-hot encoded (OHE) output formulations were tested. Results: An optimal model configuration with fine class granularity (prostatitis included) and OHE has scored the lesion-wise partial Free-Response Receiver Operating Characteristic (FROC) area under the curve (AUC) of 1.94 (CI 95%: 1.76–2.11) and patient-wise ROC AUC of 0.874 (CI 95%: 0.793–0.938) in the detection of csPCa. Inclusion of the auxiliary prostatitis class has demonstrated a stable relative improvement in specificity at a false positive rate (FPR) of 1.0 per patient, with an increase of 3%, 7%, and 4% for coarse, medium, and fine class granularities. Conclusions: This paper examines several configurations for model training in the biparametric MRI setup and proposes optimal value ranges. It also shows that the fine-grained class configuration, including prostatitis, is beneficial for detecting csPCa. The ability to detect prostatitis in all low-risk cancer lesions suggests the potential to improve the quality of the early diagnosis of prostate diseases. It also implies an improved interpretability of the results by the radiologist.
169.
Huettl, F; Heinrich, F; Boedecker, C; Vradelis, L; Ludt, A; Kneist, W; Lang, H; Hansen, C; Huber, T
In: Journal of the American College of Surgeons, vol. 237, no. 2, pp. 292, 2023, ISSN: 1879-1190.
@article{huettl_real-time_2023,
title = {Real-Time Augmented Reality Annotation for Surgical Education during Laparoscopic Surgery: Results from a Single-Center Randomized Controlled Trial and Future Aspects},
author = {F Huettl and F Heinrich and C Boedecker and L Vradelis and A Ludt and W Kneist and H Lang and C Hansen and T Huber},
url = {https://journals.lww.com/journalacs/abstract/2023/08000/real_time_augmented_reality_annotation_for.20.aspx},
doi = {10.1097/XCS.0000000000000712},
issn = {1879-1190},
year = {2023},
date = {2023-08-01},
urldate = {2023-08-01},
journal = {Journal of the American College of Surgeons},
volume = {237},
number = {2},
pages = {292},
abstract = {Background: We developed an interactive augmented reality tool (HoloPointer) that enables real-time annotation on a laparoscopy monitor for intraoperative guidance. This application operates exclusively via verbal commands and head movements to ensure a sterile workflow. Study design: Purpose of this randomized controlled clinical trial was to evaluate the integration of this new technology into the operating room. This prospective single-center study included 32 elective laparoscopic cholecystectomies (29 surgical teams, 15 trainees, 13 trainers). Primary objectives and assessment measures was the HoloPointer's influence on surgical performance (subjective assessment, global operative assessment of laparoscopic skills - GOALS, and Critical View of Safety -CVS). Secondary objectives and outcome variables were its influence on operation time, quality of assistance (5 point likert-scale), and user-friendliness (System Usability Scale - SUS, 0-100 points). Results: Gestural corrections were reduced by 59.4% (4.6 SD 8.1 vs. 1.9 SD 4.7; p > 0.05) and verbal corrections by 36.1% (17.8 SD 12.9 vs. 11.4 SD 8.1; p > 0.05). Subjective surgical performance could be improved by 84,6% of participants. No statistically significant differences were observed for objective parameters GOALS, CVS and operation time. In the SUS, the application achieved an average score of 72.5 SD 16.3 (good user-friendliness). Of the participants, 69.2% wanted to use the HoloPointer more frequently. Conclusion: The majority of trainees had improved their surgical performance using the HoloPointer in elective laparoscopic cholecystectomies, and the rate of classic but potentially misleading corrections was noticeably reduced. The HoloPointer has the potential to improve education in minimally invasive surgery.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Background: We developed an interactive augmented reality tool (HoloPointer) that enables real-time annotation on a laparoscopy monitor for intraoperative guidance. This application operates exclusively via verbal commands and head movements to ensure a sterile workflow. Study design: Purpose of this randomized controlled clinical trial was to evaluate the integration of this new technology into the operating room. This prospective single-center study included 32 elective laparoscopic cholecystectomies (29 surgical teams, 15 trainees, 13 trainers). Primary objectives and assessment measures was the HoloPointer's influence on surgical performance (subjective assessment, global operative assessment of laparoscopic skills – GOALS, and Critical View of Safety -CVS). Secondary objectives and outcome variables were its influence on operation time, quality of assistance (5 point likert-scale), and user-friendliness (System Usability Scale – SUS, 0-100 points). Results: Gestural corrections were reduced by 59.4% (4.6 SD 8.1 vs. 1.9 SD 4.7; p > 0.05) and verbal corrections by 36.1% (17.8 SD 12.9 vs. 11.4 SD 8.1; p > 0.05). Subjective surgical performance could be improved by 84,6% of participants. No statistically significant differences were observed for objective parameters GOALS, CVS and operation time. In the SUS, the application achieved an average score of 72.5 SD 16.3 (good user-friendliness). Of the participants, 69.2% wanted to use the HoloPointer more frequently. Conclusion: The majority of trainees had improved their surgical performance using the HoloPointer in elective laparoscopic cholecystectomies, and the rate of classic but potentially misleading corrections was noticeably reduced. The HoloPointer has the potential to improve education in minimally invasive surgery.
168.
Gulamhussene, G; Rak, M; Bashkanov, O; Joeres, F; Omari, J; Pech, M; Hansen, C
Transfer-learning is a key ingredient to fast deep learning-based 4D liver MRI reconstruction Journal Article
In: Scientific Reports, vol. 13, no. 1, pp. 11227, 2023, ISSN: 2045-2322, (Publisher: Nature Publishing Group).
@article{gulamhussene_transfer-learning_2023,
title = {Transfer-learning is a key ingredient to fast deep learning-based 4D liver MRI reconstruction},
author = {G Gulamhussene and M Rak and O Bashkanov and F Joeres and J Omari and M Pech and C Hansen},
url = {https://www.nature.com/articles/s41598-023-38073-1},
doi = {10.1038/s41598-023-38073-1},
issn = {2045-2322},
year = {2023},
date = {2023-07-01},
urldate = {2023-07-01},
journal = {Scientific Reports},
volume = {13},
number = {1},
pages = {11227},
abstract = {Time-resolved volumetric magnetic resonance imaging (4D MRI) could be used to address organ motion in image-guided interventions like tumor ablation. Current 4D reconstruction techniques are unsuitable for most interventional settings because they are limited to specific breathing phases, lack temporal/spatial resolution, and have long prior acquisitions or reconstruction times. Deep learning-based (DL) 4D MRI approaches promise to overcome these shortcomings but are sensitive to domain shift. This work shows that transfer learning (TL) combined with an ensembling strategy can help alleviate this key challenge. We evaluate four approaches: pre-trained models from the source domain, models directly trained from scratch on target domain data, models fine-tuned from a pre-trained model and an ensemble of fine-tuned models. For that the data base was split into 16 source and 4 target domain subjects. Comparing ensemble of fine-tuned models (N = 10) with directly learned models, we report significant improvements (P < 0.001) of the root mean squared error (RMSE) of up to 12% and the mean displacement (MDISP) of up to 17.5%. The smaller the target domain data amount, the larger the effect. This shows that TL + Ens significantly reduces beforehand acquisition time and improves reconstruction quality, rendering it a key component in making 4D MRI clinically feasible for the first time in the context of 4D organ motion models of the liver and beyond.},
note = {Publisher: Nature Publishing Group},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Time-resolved volumetric magnetic resonance imaging (4D MRI) could be used to address organ motion in image-guided interventions like tumor ablation. Current 4D reconstruction techniques are unsuitable for most interventional settings because they are limited to specific breathing phases, lack temporal/spatial resolution, and have long prior acquisitions or reconstruction times. Deep learning-based (DL) 4D MRI approaches promise to overcome these shortcomings but are sensitive to domain shift. This work shows that transfer learning (TL) combined with an ensembling strategy can help alleviate this key challenge. We evaluate four approaches: pre-trained models from the source domain, models directly trained from scratch on target domain data, models fine-tuned from a pre-trained model and an ensemble of fine-tuned models. For that the data base was split into 16 source and 4 target domain subjects. Comparing ensemble of fine-tuned models (N = 10) with directly learned models, we report significant improvements (P < 0.001) of the root mean squared error (RMSE) of up to 12% and the mean displacement (MDISP) of up to 17.5%. The smaller the target domain data amount, the larger the effect. This shows that TL + Ens significantly reduces beforehand acquisition time and improves reconstruction quality, rendering it a key component in making 4D MRI clinically feasible for the first time in the context of 4D organ motion models of the liver and beyond.
167.
Schwenderling, L; Kleinau, A; Herbrich, W; Kasireddy, H; Heinrich, F; Hansen, C
Activation modes for gesture-based interaction with a magic lens in AR anatomy visualisation Journal Article
In: Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization, vol. 11, no. 4, pp. 1243–1250, 2023, ISSN: 2168-1163, (Publisher: Taylor & Francis).
@article{schwenderling_activation_2023,
title = {Activation modes for gesture-based interaction with a magic lens in AR anatomy visualisation},
author = {L Schwenderling and A Kleinau and W Herbrich and H Kasireddy and F Heinrich and C Hansen},
url = {https://doi.org/10.1080/21681163.2022.2157749},
doi = {10.1080/21681163.2022.2157749},
issn = {2168-1163},
year = {2023},
date = {2023-07-01},
urldate = {2023-07-01},
journal = {Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization},
volume = {11},
number = {4},
pages = {1243–1250},
abstract = {Learning human anatomy is key for health-related education and often requires expensive and time-consuming cadaver dissection courses. Augmented reality (AR) for the representation of spatially registered 3D models can be used as a low-cost and flexible alternative. However, suitable visualisation and interaction approaches are needed to display multilayered anatomy data. This paper features a spherical volumetric AR Magic Lens controlled by mid-air hand gestures to explore the human anatomy on a phantom. Defining how gestures control associated actions is important for intuitive interaction. Therefore, two gesture activation modes were investigated in a user study (n = 24). Performing the gestures once to toggle actions showed a higher interaction count since an additional stop gesture was used. Holding the gestures was favoured in the qualitative feedback. Both modes showed similar performance in terms of accuracy and task completion time. Overall, direct gesture manipulation of a magic lens for anatomy visualisation is, thus, recommended.},
note = {Publisher: Taylor & Francis},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Learning human anatomy is key for health-related education and often requires expensive and time-consuming cadaver dissection courses. Augmented reality (AR) for the representation of spatially registered 3D models can be used as a low-cost and flexible alternative. However, suitable visualisation and interaction approaches are needed to display multilayered anatomy data. This paper features a spherical volumetric AR Magic Lens controlled by mid-air hand gestures to explore the human anatomy on a phantom. Defining how gestures control associated actions is important for intuitive interaction. Therefore, two gesture activation modes were investigated in a user study (n = 24). Performing the gestures once to toggle actions showed a higher interaction count since an additional stop gesture was used. Holding the gestures was favoured in the qualitative feedback. Both modes showed similar performance in terms of accuracy and task completion time. Overall, direct gesture manipulation of a magic lens for anatomy visualisation is, thus, recommended.
166.
Schott, D; Kunz, M; Wunderling, T; Heinrich, F; Braun-Dullaeus, R; Hansen, C
CardioGenesis4D: Interactive Morphological Transitions of Embryonic Heart Development in a Virtual Learning Environment Journal Article
In: IEEE Transactions on Visualization and Computer Graphics, vol. 29, no. 5, pp. 2615–2625, 2023, ISSN: 1941-0506.
@article{schott_cardiogenesis4d_2023,
title = {CardioGenesis4D: Interactive Morphological Transitions of Embryonic Heart Development in a Virtual Learning Environment},
author = {D Schott and M Kunz and T Wunderling and F Heinrich and R Braun-Dullaeus and C Hansen},
url = {https://ieeexplore.ieee.org/document/10049681},
doi = {10.1109/TVCG.2023.3247110},
issn = {1941-0506},
year = {2023},
date = {2023-05-01},
urldate = {2023-05-01},
journal = {IEEE Transactions on Visualization and Computer Graphics},
volume = {29},
number = {5},
pages = {2615–2625},
abstract = {In the embryonic human heart, complex dynamic shape changes take place in a short period of time on a microscopic scale, making this development difficult to visualize. However, spatial understanding of these processes is essential for students and future cardiologists to properly diagnose and treat congenital heart defects. Following a user centered approach, the most crucial embryological stages were identified and translated into a virtual reality learning environment (VRLE) to enable the understanding of the morphological transitions of these stages through advanced interactions. To address individual learning types, we implemented different features and evaluated the application regarding usability, perceived task load, and sense of presence in a user study. We also assessed spatial awareness and knowledge gain, and finally obtained feedback from domain experts. Overall, students and professionals rated the application positively. To minimize distraction from interactive learning content, such VRLEs should consider features for different learning types, allow for gradual habituation, and at the same time provide enough playful stimuli. Our work previews how VR can be integrated into a cardiac embryology education curriculum.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
In the embryonic human heart, complex dynamic shape changes take place in a short period of time on a microscopic scale, making this development difficult to visualize. However, spatial understanding of these processes is essential for students and future cardiologists to properly diagnose and treat congenital heart defects. Following a user centered approach, the most crucial embryological stages were identified and translated into a virtual reality learning environment (VRLE) to enable the understanding of the morphological transitions of these stages through advanced interactions. To address individual learning types, we implemented different features and evaluated the application regarding usability, perceived task load, and sense of presence in a user study. We also assessed spatial awareness and knowledge gain, and finally obtained feedback from domain experts. Overall, students and professionals rated the application positively. To minimize distraction from interactive learning content, such VRLEs should consider features for different learning types, allow for gradual habituation, and at the same time provide enough playful stimuli. Our work previews how VR can be integrated into a cardiac embryology education curriculum.
165.
Chheang, V; Bruggernann, R; Preim, B; Hansen, C
Virtual Resection Planning using Bezier Surface Interactions in Collaborative VR Environments Proceedings Article
In: 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), pp. 166–169, 2023.
@inproceedings{chheang_virtual_2023,
title = {Virtual Resection Planning using Bezier Surface Interactions in Collaborative VR Environments},
author = {V Chheang and R Bruggernann and B Preim and C Hansen},
url = {https://ieeexplore.ieee.org/document/10108900},
doi = {10.1109/VRW58643.2023.00041},
year = {2023},
date = {2023-03-01},
urldate = {2023-03-01},
booktitle = {2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)},
pages = {166–169},
abstract = {The use of virtual reality (VR) has been proposed for collaborative planning or training scenarios to aid surgeons in preparing surgical interventions. In this paper, we explore the usage of cubic Bezier surfaces and their respective interactions in collaborative VR environments to define atypical resection surfaces for planning abdominal organ tumor resections. We conducted a pilot study (n = 10) to evaluate the usability of the Bezier surface interaction compared to a free deformation approach using the example of liver surgery planning. Moreover, we showed the prototype to an expert and conducted an interview to collect clinical feedback. The results show potential benefits of both interaction techniques that could be essential for virtual resection planning and training. The expert highlighted that combining both techniques could speed up the workflow and provide various options for planning.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
The use of virtual reality (VR) has been proposed for collaborative planning or training scenarios to aid surgeons in preparing surgical interventions. In this paper, we explore the usage of cubic Bezier surfaces and their respective interactions in collaborative VR environments to define atypical resection surfaces for planning abdominal organ tumor resections. We conducted a pilot study (n = 10) to evaluate the usability of the Bezier surface interaction compared to a free deformation approach using the example of liver surgery planning. Moreover, we showed the prototype to an expert and conducted an interview to collect clinical feedback. The results show potential benefits of both interaction techniques that could be essential for virtual resection planning and training. The expert highlighted that combining both techniques could speed up the workflow and provide various options for planning.
164.
Sühn, T; Esmaeili, N; Mattepu, S; Spiller, M; Boese, A; Urrutia, R; Poblete, V; Hansen, C; Lohmann, C; Illanes, A; Friebe, M
Vibro-Acoustic Sensing of Instrument Interactions as a Potential Source of Texture-Related Information in Robotic Palpation Journal Article
In: Sensors, vol. 23, no. 6, pp. 3141, 2023, ISSN: 1424-8220, (Publisher: Multidisciplinary Digital Publishing Institute).
@article{suhn_vibro-acoustic_2023,
title = {Vibro-Acoustic Sensing of Instrument Interactions as a Potential Source of Texture-Related Information in Robotic Palpation},
author = {T Sühn and N Esmaeili and S Mattepu and M Spiller and A Boese and R Urrutia and V Poblete and C Hansen and C Lohmann and A Illanes and M Friebe},
url = {https://www.mdpi.com/1424-8220/23/6/3141},
doi = {10.3390/s23063141},
issn = {1424-8220},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Sensors},
volume = {23},
number = {6},
pages = {3141},
abstract = {The direct tactile assessment of surface textures during palpation is an essential component of open surgery that is impeded in minimally invasive and robot-assisted surgery. When indirectly palpating with a surgical instrument, the structural vibrations from this interaction contain tactile information that can be extracted and analysed. This study investigates the influence of the parameters contact angle α and velocity v→ on the vibro-acoustic signals from this indirect palpation. A 7-DOF robotic arm, a standard surgical instrument, and a vibration measurement system were used to palpate three different materials with varying α and v→. The signals were processed based on continuous wavelet transformation. They showed material-specific signatures in the time–frequency domain that retained their general characteristic for varying α and v→. Energy-related and statistical features were extracted, and supervised classification was performed, where the testing data comprised only signals acquired with different palpation parameters than for training data. The classifiers support vector machine and k-nearest neighbours provided 99.67% and 96.00% accuracy for the differentiation of the materials. The results indicate the robustness of the features against variations in the palpation parameters. This is a prerequisite for an application in minimally invasive surgery but needs to be confirmed in realistic experiments with biological tissues.},
note = {Publisher: Multidisciplinary Digital Publishing Institute},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
The direct tactile assessment of surface textures during palpation is an essential component of open surgery that is impeded in minimally invasive and robot-assisted surgery. When indirectly palpating with a surgical instrument, the structural vibrations from this interaction contain tactile information that can be extracted and analysed. This study investigates the influence of the parameters contact angle α and velocity v→ on the vibro-acoustic signals from this indirect palpation. A 7-DOF robotic arm, a standard surgical instrument, and a vibration measurement system were used to palpate three different materials with varying α and v→. The signals were processed based on continuous wavelet transformation. They showed material-specific signatures in the time–frequency domain that retained their general characteristic for varying α and v→. Energy-related and statistical features were extracted, and supervised classification was performed, where the testing data comprised only signals acquired with different palpation parameters than for training data. The classifiers support vector machine and k-nearest neighbours provided 99.67% and 96.00% accuracy for the differentiation of the materials. The results indicate the robustness of the features against variations in the palpation parameters. This is a prerequisite for an application in minimally invasive surgery but needs to be confirmed in realistic experiments with biological tissues.
163.
Gulamhussene, G; Das, A; Spiegel, J; Punzet, D; Rak, M; Hansen, C
Needle Tip Tracking During CT-guided Interventions using Fuzzy Segmentation Proceedings Article
In: Deserno, T; Handels, H; Maier, A; Maier-Hein, K; Palm, C; Tolxdorff, T (Ed.): Bildverarbeitung für die Medizin 2023, pp. 285–291, Springer Fachmedien, Wiesbaden, 2023, ISBN: 978-3-658-41657-7.
@inproceedings{gulamhussene_needle_2023,
title = {Needle Tip Tracking During CT-guided Interventions using Fuzzy Segmentation},
author = {G Gulamhussene and A Das and J Spiegel and D Punzet and M Rak and C Hansen},
editor = {T Deserno and H Handels and A Maier and K Maier-Hein and C Palm and T Tolxdorff},
doi = {10.1007/978-3-658-41657-7_62},
isbn = {978-3-658-41657-7},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Bildverarbeitung für die Medizin 2023},
pages = {285–291},
publisher = {Springer Fachmedien},
address = {Wiesbaden},
abstract = {CT-guided interventions are standard practice for radiologists to treat lesions in various parts of the human body. In this context, accurate tracking of instruments is of paramount importance for the safety of the procedure and helps radiologists avoid unintended damage to adjacent organs. In this work, a novel method for the estimation of 3D needle tip coordinates in a CT volume using only two 2D projections in an interventional setting is proposed. The method applies a deep learning model for the fuzzy segmentation of the region containing the tip on 2D projections and automatically extracts the position of the tip. A simple UNet achieves a Dice score of 0.9906 for the fuzzy segmentation and an average euclidean distance of 2.96 mm for the needle tip regression task.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
CT-guided interventions are standard practice for radiologists to treat lesions in various parts of the human body. In this context, accurate tracking of instruments is of paramount importance for the safety of the procedure and helps radiologists avoid unintended damage to adjacent organs. In this work, a novel method for the estimation of 3D needle tip coordinates in a CT volume using only two 2D projections in an interventional setting is proposed. The method applies a deep learning model for the fuzzy segmentation of the region containing the tip on 2D projections and automatically extracts the position of the tip. A simple UNet achieves a Dice score of 0.9906 for the fuzzy segmentation and an average euclidean distance of 2.96 mm for the needle tip regression task.
162.
Urrutia, R; Espejo, D; Evens, N; Guerra, M; Sühn, T; Boese, A; Hansen, C; Fuentealba, P; Illanes, A; Poblete, V
Clustering Methods for Vibro-Acoustic Sensing Features as a Potential Approach to Tissue Characterisation in Robot-Assisted Interventions Journal Article
In: Sensors, vol. 23, no. 23, pp. 9297, 2023, ISSN: 1424-8220, (Publisher: Multidisciplinary Digital Publishing Institute).
@article{urrutia_clustering_2023,
title = {Clustering Methods for Vibro-Acoustic Sensing Features as a Potential Approach to Tissue Characterisation in Robot-Assisted Interventions},
author = {R Urrutia and D Espejo and N Evens and M Guerra and T Sühn and A Boese and C Hansen and P Fuentealba and A Illanes and V Poblete},
url = {https://www.mdpi.com/1424-8220/23/23/9297},
doi = {10.3390/s23239297},
issn = {1424-8220},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Sensors},
volume = {23},
number = {23},
pages = {9297},
abstract = {This article provides a comprehensive analysis of the feature extraction methods applied to vibro-acoustic signals (VA signals) in the context of robot-assisted interventions. The primary objective is to extract valuable information from these signals to understand tissue behaviour better and build upon prior research. This study is divided into three key stages: feature extraction using the Cepstrum Transform (CT), Mel-Frequency Cepstral Coefficients (MFCCs), and Fast Chirplet Transform (FCT); dimensionality reduction employing techniques such as Principal Component Analysis (PCA), t-Distributed Stochastic Neighbour Embedding (t-SNE), and Uniform Manifold Approximation and Projection (UMAP); and, finally, classification using a nearest neighbours classifier. The results demonstrate that using feature extraction techniques, especially the combination of CT and MFCC with dimensionality reduction algorithms, yields highly efficient outcomes. The classification metrics (Accuracy, Recall, and F1-score) approach 99%, and the clustering metric is 0.61. The performance of the CT–UMAP combination stands out in the evaluation metrics.},
note = {Publisher: Multidisciplinary Digital Publishing Institute},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
This article provides a comprehensive analysis of the feature extraction methods applied to vibro-acoustic signals (VA signals) in the context of robot-assisted interventions. The primary objective is to extract valuable information from these signals to understand tissue behaviour better and build upon prior research. This study is divided into three key stages: feature extraction using the Cepstrum Transform (CT), Mel-Frequency Cepstral Coefficients (MFCCs), and Fast Chirplet Transform (FCT); dimensionality reduction employing techniques such as Principal Component Analysis (PCA), t-Distributed Stochastic Neighbour Embedding (t-SNE), and Uniform Manifold Approximation and Projection (UMAP); and, finally, classification using a nearest neighbours classifier. The results demonstrate that using feature extraction techniques, especially the combination of CT and MFCC with dimensionality reduction algorithms, yields highly efficient outcomes. The classification metrics (Accuracy, Recall, and F1-score) approach 99%, and the clustering metric is 0.61. The performance of the CT–UMAP combination stands out in the evaluation metrics.
161.
Gulamhussene, G; Spiegel, J; Das, A; Rak, M; Hansen, C
Deep Learning-based Marker-less Pose Estimation of Interventional Tools using Surrogate Keypoints Proceedings Article
In: Deserno, T; Handels, H; Maier, A; Maier-Hein, K; Palm, C; Tolxdorff, T (Ed.): Bildverarbeitung für die Medizin 2023, pp. 292–298, Springer Fachmedien, Wiesbaden, 2023, ISBN: 978-3-658-41657-7.
@inproceedings{gulamhussene_deep_2023,
title = {Deep Learning-based Marker-less Pose Estimation of Interventional Tools using Surrogate Keypoints},
author = {G Gulamhussene and J Spiegel and A Das and M Rak and C Hansen},
editor = {T Deserno and H Handels and A Maier and K Maier-Hein and C Palm and T Tolxdorff},
doi = {10.1007/978-3-658-41657-7_63},
isbn = {978-3-658-41657-7},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Bildverarbeitung für die Medizin 2023},
pages = {292–298},
publisher = {Springer Fachmedien},
address = {Wiesbaden},
abstract = {Estimating the position of an intervention needle is an important ability in computer-assisted interventions. Currently, such pose estimations rely either on radiation-intensive CT imaging or need additional optical markers which add overhead to the clinical workflow. We propose a novel deep-learning-based technique for pose estimation of interventional tools which relies on detecting visible features on the tool itself without additional markers.We also propose a novel and fast pipeline for creating vast amounts of robustly labeled and markerless ground truth data for training such neural networks. Initial evaluations suggest that with needle base and needle tip localization errors of about 1 and 4 cm, Our approach can yield a search corridor that can be used to find the needle in a low-dose CT image, reducing radiation exposure.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Estimating the position of an intervention needle is an important ability in computer-assisted interventions. Currently, such pose estimations rely either on radiation-intensive CT imaging or need additional optical markers which add overhead to the clinical workflow. We propose a novel deep-learning-based technique for pose estimation of interventional tools which relies on detecting visible features on the tool itself without additional markers.We also propose a novel and fast pipeline for creating vast amounts of robustly labeled and markerless ground truth data for training such neural networks. Initial evaluations suggest that with needle base and needle tip localization errors of about 1 and 4 cm, Our approach can yield a search corridor that can be used to find the needle in a low-dose CT image, reducing radiation exposure.
160.
Gulamhussene, G; Bashkanov, O; Omari, J; Pech, M; Hansen, C; Rak, M
Using Training Samples as Transitive Information Bridges in Predicted 4D MRI Proceedings Article
In: Xue, Z; Antani, S; Zamzmi, G; Yang, F; Rajaraman, S; Huang, S; Linguraru, M; Liang, Z (Ed.): Medical Image Learning with Limited and Noisy Data, pp. 237–245, Springer Nature Switzerland, Cham, 2023, ISBN: 978-3-031-44917-8.
@inproceedings{gulamhussene_using_2023,
title = {Using Training Samples as Transitive Information Bridges in Predicted 4D MRI},
author = {G Gulamhussene and O Bashkanov and J Omari and M Pech and C Hansen and M Rak},
editor = {Z Xue and S Antani and G Zamzmi and F Yang and S Rajaraman and S Huang and M Linguraru and Z Liang},
doi = {10.1007/978-3-031-44917-8_23},
isbn = {978-3-031-44917-8},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Medical Image Learning with Limited and Noisy Data},
pages = {237–245},
publisher = {Springer Nature Switzerland},
address = {Cham},
abstract = {The lack of real-time techniques for monitoring respiratory motion impairs the development of guidance systems for image-guided interventions. Recent works show that U-Net based real-time 4D MRI prediction methods are promising, but prone to bad image quality when small training data sets and inputs with multiple MR contrast are used. To overcome this problem, we propose a more efficient use of the spare training data and re-utilize 2D training samples as a secondary input for construction of transitive information bridges between the navigator slice primary input and the data slice prediction. We thus remove the need for a separate 3D breath-hold MRI with different MR contrast as the secondary input. Results show that our novel construction leads to improved prediction quality with very sparse training data, with a significant decrease in root mean squared error (RMSE) from 0.3 to 0.27 (p$$<2.2eˆ-16$$<2.2e-16},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
The lack of real-time techniques for monitoring respiratory motion impairs the development of guidance systems for image-guided interventions. Recent works show that U-Net based real-time 4D MRI prediction methods are promising, but prone to bad image quality when small training data sets and inputs with multiple MR contrast are used. To overcome this problem, we propose a more efficient use of the spare training data and re-utilize 2D training samples as a secondary input for construction of transitive information bridges between the navigator slice primary input and the data slice prediction. We thus remove the need for a separate 3D breath-hold MRI with different MR contrast as the secondary input. Results show that our novel construction leads to improved prediction quality with very sparse training data, with a significant decrease in root mean squared error (RMSE) from 0.3 to 0.27 (p$$<2.2eˆ-16$$<2.2e-16
2022
159.
Schreiter, J; Mielke, T; Schott, D; Thormann, M; Omari, J; Pech, M; Hansen, C
A multimodal user interface for touchless control of robotic ultrasound Journal Article
In: International Journal of Computer Assisted Radiology and Surgery, vol. 18, no. 8, pp. 1429–1436, 2022, ISSN: 1861-6429.
@article{schreiter_multimodal_2022,
title = {A multimodal user interface for touchless control of robotic ultrasound},
author = {J Schreiter and T Mielke and D Schott and M Thormann and J Omari and M Pech and C Hansen},
url = {https://link.springer.com/10.1007/s11548-022-02810-0},
doi = {10.1007/s11548-022-02810-0},
issn = {1861-6429},
year = {2022},
date = {2022-12-01},
urldate = {2022-12-01},
journal = {International Journal of Computer Assisted Radiology and Surgery},
volume = {18},
number = {8},
pages = {1429–1436},
abstract = {Purpose Past research contained the investigation and development of robotic ultrasound. In this context, interfaces which allow for interaction with the robotic system are of paramount importance. Few researchers have addressed the issue of developing non-tactile interaction approaches, although they could be beneficial for maintaining sterility during medical procedures. Interaction could be supported by multimodality, which has the potential to enable intuitive and natural interaction. To assess the feasibility of multimodal interaction for non-tactile control of a co-located robotic ultrasound system, a novel human–robot interaction concept was developed.
Methods The medical use case of needle-based interventions under hybrid computed tomography and ultrasound imaging was analyzed by interviewing four radiologists. From the resulting workflow, interaction tasks were derived which include human–robot interaction. Based on this, characteristics of a multimodal, touchless human–robot interface were elaborated, suitable interaction modalities were identified, and a corresponding interface was developed, which was thereafter evaluated in a user study with eight participants.
Results The implemented interface includes voice commands, combined with hand gesture control for discrete control and navigation interaction of the robotic US probe, respectively. The interaction concept was evaluated by the users in the form of a quantitative questionnaire with a average usability. Qualitative analysis of interview results revealed user satisfaction with the implemented interaction methods and potential improvements to the system.
Conclusion A multimodal, touchless interaction concept for a robotic US for the use case of needle-based procedures in interventional radiology was developed, incorporating combined voice and hand gesture control. Future steps will include the integration of a solution for the missing haptic feedback and the evaluation of its clinical suitability.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Purpose Past research contained the investigation and development of robotic ultrasound. In this context, interfaces which allow for interaction with the robotic system are of paramount importance. Few researchers have addressed the issue of developing non-tactile interaction approaches, although they could be beneficial for maintaining sterility during medical procedures. Interaction could be supported by multimodality, which has the potential to enable intuitive and natural interaction. To assess the feasibility of multimodal interaction for non-tactile control of a co-located robotic ultrasound system, a novel human–robot interaction concept was developed.
Methods The medical use case of needle-based interventions under hybrid computed tomography and ultrasound imaging was analyzed by interviewing four radiologists. From the resulting workflow, interaction tasks were derived which include human–robot interaction. Based on this, characteristics of a multimodal, touchless human–robot interface were elaborated, suitable interaction modalities were identified, and a corresponding interface was developed, which was thereafter evaluated in a user study with eight participants.
Results The implemented interface includes voice commands, combined with hand gesture control for discrete control and navigation interaction of the robotic US probe, respectively. The interaction concept was evaluated by the users in the form of a quantitative questionnaire with a average usability. Qualitative analysis of interview results revealed user satisfaction with the implemented interaction methods and potential improvements to the system.
Conclusion A multimodal, touchless interaction concept for a robotic US for the use case of needle-based procedures in interventional radiology was developed, incorporating combined voice and hand gesture control. Future steps will include the integration of a solution for the missing haptic feedback and the evaluation of its clinical suitability.
Methods The medical use case of needle-based interventions under hybrid computed tomography and ultrasound imaging was analyzed by interviewing four radiologists. From the resulting workflow, interaction tasks were derived which include human–robot interaction. Based on this, characteristics of a multimodal, touchless human–robot interface were elaborated, suitable interaction modalities were identified, and a corresponding interface was developed, which was thereafter evaluated in a user study with eight participants.
Results The implemented interface includes voice commands, combined with hand gesture control for discrete control and navigation interaction of the robotic US probe, respectively. The interaction concept was evaluated by the users in the form of a quantitative questionnaire with a average usability. Qualitative analysis of interview results revealed user satisfaction with the implemented interaction methods and potential improvements to the system.
Conclusion A multimodal, touchless interaction concept for a robotic US for the use case of needle-based procedures in interventional radiology was developed, incorporating combined voice and hand gesture control. Future steps will include the integration of a solution for the missing haptic feedback and the evaluation of its clinical suitability.
158.
Alpers, J; Rötzer, M; Gutberlet, M; Wacker, F; Hensen, B; Hansen, C
Adaptive simulation of 3D thermometry maps for interventional MR-guided tumor ablation using Pennes’ bioheat equation and isotherms Journal Article
In: Scientific Reports, vol. 12, no. 1, pp. 20356, 2022, ISSN: 2045-2322.
@article{alpers_adaptive_2022,
title = {Adaptive simulation of 3D thermometry maps for interventional MR-guided tumor ablation using Pennes’ bioheat equation and isotherms},
author = {J Alpers and M Rötzer and M Gutberlet and F Wacker and B Hensen and C Hansen},
url = {https://www.nature.com/articles/s41598-022-24911-1},
doi = {10.1038/s41598-022-24911-1},
issn = {2045-2322},
year = {2022},
date = {2022-11-01},
urldate = {2022-11-01},
journal = {Scientific Reports},
volume = {12},
number = {1},
pages = {20356},
abstract = {Abstract
Minimally-invasive thermal ablation procedures have become clinically accepted treatment options for tumors and metastases. Continuous and reliable monitoring of volumetric heat distribution promises to be an important condition for successful outcomes. In this work, an adaptive bioheat transfer simulation of 3D thermometry maps is presented. Pennes’ equation model is updated according to temperature maps generated by uniformly distributed 2D MR phase images rotated around the main axis of the applicator. The volumetric heat diffusion and the resulting shape of the ablation zone can be modelled accurately without introducing a specific heat source term. Filtering the temperature maps by extracting isotherms reduces artefacts and noise, compresses information of the measured data and adds physical a priori knowledge. The inverse heat transfer for estimating values of the simulated tissue and heating parameters is done by reducing the sum squared error between these isotherms and the 3D simulation. The approach is evaluated on data sets consisting of 13 ex vivo bio protein phantoms, including six perfusion phantoms with simulated heat sink effects. Results show an overall average Dice score of 0.89 ± 0.04 (SEM < 0.01). The optimization of the parameters takes 1.05 ± 0.26 s for each acquired image. Future steps should consider the local optimization of the simulation parameters instead of a global one to better detect heat sinks without a priori knowledge. In addition, the use of a proper Kalman filter might increase robustness and accuracy if combined with our method.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Abstract
Minimally-invasive thermal ablation procedures have become clinically accepted treatment options for tumors and metastases. Continuous and reliable monitoring of volumetric heat distribution promises to be an important condition for successful outcomes. In this work, an adaptive bioheat transfer simulation of 3D thermometry maps is presented. Pennes’ equation model is updated according to temperature maps generated by uniformly distributed 2D MR phase images rotated around the main axis of the applicator. The volumetric heat diffusion and the resulting shape of the ablation zone can be modelled accurately without introducing a specific heat source term. Filtering the temperature maps by extracting isotherms reduces artefacts and noise, compresses information of the measured data and adds physical a priori knowledge. The inverse heat transfer for estimating values of the simulated tissue and heating parameters is done by reducing the sum squared error between these isotherms and the 3D simulation. The approach is evaluated on data sets consisting of 13 ex vivo bio protein phantoms, including six perfusion phantoms with simulated heat sink effects. Results show an overall average Dice score of 0.89 ± 0.04 (SEM < 0.01). The optimization of the parameters takes 1.05 ± 0.26 s for each acquired image. Future steps should consider the local optimization of the simulation parameters instead of a global one to better detect heat sinks without a priori knowledge. In addition, the use of a proper Kalman filter might increase robustness and accuracy if combined with our method.
Minimally-invasive thermal ablation procedures have become clinically accepted treatment options for tumors and metastases. Continuous and reliable monitoring of volumetric heat distribution promises to be an important condition for successful outcomes. In this work, an adaptive bioheat transfer simulation of 3D thermometry maps is presented. Pennes’ equation model is updated according to temperature maps generated by uniformly distributed 2D MR phase images rotated around the main axis of the applicator. The volumetric heat diffusion and the resulting shape of the ablation zone can be modelled accurately without introducing a specific heat source term. Filtering the temperature maps by extracting isotherms reduces artefacts and noise, compresses information of the measured data and adds physical a priori knowledge. The inverse heat transfer for estimating values of the simulated tissue and heating parameters is done by reducing the sum squared error between these isotherms and the 3D simulation. The approach is evaluated on data sets consisting of 13 ex vivo bio protein phantoms, including six perfusion phantoms with simulated heat sink effects. Results show an overall average Dice score of 0.89 ± 0.04 (SEM < 0.01). The optimization of the parameters takes 1.05 ± 0.26 s for each acquired image. Future steps should consider the local optimization of the simulation parameters instead of a global one to better detect heat sinks without a priori knowledge. In addition, the use of a proper Kalman filter might increase robustness and accuracy if combined with our method.
157.
Schwenderling, L; Heinrich, F; Hansen, C
Augmented reality visualization of automated path planning for percutaneous interventions: a phantom study Journal Article
In: International Journal of Computer Assisted Radiology and Surgery, vol. 17, no. 11, pp. 2071–2079, 2022, ISSN: 1861-6429.
@article{schwenderling_augmented_2022,
title = {Augmented reality visualization of automated path planning for percutaneous interventions: a phantom study},
author = {L Schwenderling and F Heinrich and C Hansen},
url = {https://doi.org/10.1007/s11548-022-02690-4},
doi = {10.1007/s11548-022-02690-4},
issn = {1861-6429},
year = {2022},
date = {2022-11-01},
urldate = {2022-11-01},
journal = {International Journal of Computer Assisted Radiology and Surgery},
volume = {17},
number = {11},
pages = {2071–2079},
abstract = {Insertion point identification is a major challenge for percutaneous interventions. Planning in 2D slice image data is time-consuming and inefficient. Automated path planning can help to overcome these challenges. However, the setup of the intervention room is difficult to consider. In addition, transferring the insertion point to the skin is often prone to error. Therefore, a visualization for an automated path planning was implemented.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Insertion point identification is a major challenge for percutaneous interventions. Planning in 2D slice image data is time-consuming and inefficient. Automated path planning can help to overcome these challenges. However, the setup of the intervention room is difficult to consider. In addition, transferring the insertion point to the skin is often prone to error. Therefore, a visualization for an automated path planning was implemented.
156.
Schreiter, J; Schott, D; Schwenderling, L; Hansen, C; Heinrich, F; Joeres, F
AR-Supported Supervision of Conditional Autonomous Robots: Considerations for Pedicle Screw Placement in the Future Journal Article
In: Journal of Imaging, vol. 8, no. 10, pp. 255, 2022, ISSN: 2313-433X, (Publisher: Multidisciplinary Digital Publishing Institute).
@article{schreiter_ar-supported_2022,
title = {AR-Supported Supervision of Conditional Autonomous Robots: Considerations for Pedicle Screw Placement in the Future},
author = {J Schreiter and D Schott and L Schwenderling and C Hansen and F Heinrich and F Joeres},
url = {https://www.mdpi.com/2313-433X/8/10/255},
doi = {10.3390/jimaging8100255},
issn = {2313-433X},
year = {2022},
date = {2022-10-01},
urldate = {2022-10-01},
journal = {Journal of Imaging},
volume = {8},
number = {10},
pages = {255},
abstract = {Robotic assistance is applied in orthopedic interventions for pedicle screw placement (PSP). While current robots do not act autonomously, they are expected to have higher autonomy under surgeon supervision in the mid-term. Augmented reality (AR) is promising to support this supervision and to enable human–robot interaction (HRI). To outline a futuristic scenario for robotic PSP, the current workflow was analyzed through literature review and expert discussion. Based on this, a hypothetical workflow of the intervention was developed, which additionally contains the analysis of the necessary information exchange between human and robot. A video see-through AR prototype was designed and implemented. A robotic arm with an orthopedic drill mock-up simulated the robotic assistance. The AR prototype included a user interface to enable HRI. The interface provides data to facilitate understanding of the robot’s ”intentions”, e.g., patient-specific CT images, the current workflow phase, or the next planned robot motion. Two-dimensional and three-dimensional visualization illustrated patient-specific medical data and the drilling process. The findings of this work contribute a valuable approach in terms of addressing future clinical needs and highlighting the importance of AR support for HRI.},
note = {Publisher: Multidisciplinary Digital Publishing Institute},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Robotic assistance is applied in orthopedic interventions for pedicle screw placement (PSP). While current robots do not act autonomously, they are expected to have higher autonomy under surgeon supervision in the mid-term. Augmented reality (AR) is promising to support this supervision and to enable human–robot interaction (HRI). To outline a futuristic scenario for robotic PSP, the current workflow was analyzed through literature review and expert discussion. Based on this, a hypothetical workflow of the intervention was developed, which additionally contains the analysis of the necessary information exchange between human and robot. A video see-through AR prototype was designed and implemented. A robotic arm with an orthopedic drill mock-up simulated the robotic assistance. The AR prototype included a user interface to enable HRI. The interface provides data to facilitate understanding of the robot’s ”intentions”, e.g., patient-specific CT images, the current workflow phase, or the next planned robot motion. Two-dimensional and three-dimensional visualization illustrated patient-specific medical data and the drilling process. The findings of this work contribute a valuable approach in terms of addressing future clinical needs and highlighting the importance of AR support for HRI.
155.
Gulamhussene, G; Meyer, A; Rak, M; Bashkanov, O; Omari, J; Pech, M; Hansen, C
Predicting 4D liver MRI for MR-guided interventions Journal Article
In: Computerized Medical Imaging and Graphics, vol. 101, pp. 102122, 2022, ISSN: 0895-6111.
@article{gulamhussene_predicting_2022,
title = {Predicting 4D liver MRI for MR-guided interventions},
author = {G Gulamhussene and A Meyer and M Rak and O Bashkanov and J Omari and M Pech and C Hansen},
url = {https://www.sciencedirect.com/science/article/pii/S0895611122000921},
doi = {10.1016/j.compmedimag.2022.102122},
issn = {0895-6111},
year = {2022},
date = {2022-10-01},
urldate = {2022-10-01},
journal = {Computerized Medical Imaging and Graphics},
volume = {101},
pages = {102122},
abstract = {Organ motion poses an unresolved challenge in image-guided interventions like radiation therapy, biopsies or tumor ablation. In the pursuit of solving this problem, the research field of time-resolved volumetric magnetic resonance imaging (4D MRI) has evolved. However, current techniques are unsuitable for most interventional settings because they lack sufficient temporal and/or spatial resolution or have long acquisition times. In this work, we propose a novel approach for real-time, high-resolution 4D MRI with large fields of view for MR-guided interventions. To this end, we propose a network-agnostic, end-to-end trainable, deep learning formulation that enables the prediction of a 4D liver MRI with respiratory states from a live 2D navigator MRI. Our method can be used in two ways: First, it can reconstruct high quality fast (near real-time) 4D MRI with high resolution (209×128×128 matrix size with isotropic 1.8mm voxel size and 0.6s/volume) given a dynamic interventional 2D navigator slice for guidance during an intervention. Second, it can be used for retrospective 4D reconstruction with a temporal resolution of below 0.2s/volume for motion analysis and use in radiation therapy. We report a mean target registration error (TRE) of 1.19±0.74mm, which is below voxel size. We compare our results with a state-of-the-art retrospective 4D MRI reconstruction. Visual evaluation shows comparable quality. We compare different network architectures within our formulation. We show that small training sizes with short acquisition times down to 2 min can already achieve promising results and 24 min are sufficient for high quality results. Because our method can be readily combined with earlier time reducing methods, acquisition time can be further decreased while also limiting quality loss. We show that an end-to-end, deep learning formulation is highly promising for 4D MRI reconstruction.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Organ motion poses an unresolved challenge in image-guided interventions like radiation therapy, biopsies or tumor ablation. In the pursuit of solving this problem, the research field of time-resolved volumetric magnetic resonance imaging (4D MRI) has evolved. However, current techniques are unsuitable for most interventional settings because they lack sufficient temporal and/or spatial resolution or have long acquisition times. In this work, we propose a novel approach for real-time, high-resolution 4D MRI with large fields of view for MR-guided interventions. To this end, we propose a network-agnostic, end-to-end trainable, deep learning formulation that enables the prediction of a 4D liver MRI with respiratory states from a live 2D navigator MRI. Our method can be used in two ways: First, it can reconstruct high quality fast (near real-time) 4D MRI with high resolution (209×128×128 matrix size with isotropic 1.8mm voxel size and 0.6s/volume) given a dynamic interventional 2D navigator slice for guidance during an intervention. Second, it can be used for retrospective 4D reconstruction with a temporal resolution of below 0.2s/volume for motion analysis and use in radiation therapy. We report a mean target registration error (TRE) of 1.19±0.74mm, which is below voxel size. We compare our results with a state-of-the-art retrospective 4D MRI reconstruction. Visual evaluation shows comparable quality. We compare different network architectures within our formulation. We show that small training sizes with short acquisition times down to 2 min can already achieve promising results and 24 min are sufficient for high quality results. Because our method can be readily combined with earlier time reducing methods, acquisition time can be further decreased while also limiting quality loss. We show that an end-to-end, deep learning formulation is highly promising for 4D MRI reconstruction.
154.
Bublak, T; Bofferding, M; Olson, C; Jonas, H; Rademeier, J; Hansen, C
A Virtual Environment for Emergency Ultrasound Training during Cardiopulmonary Resuscitation Proceedings Article
In: Proceedings of Mensch und Computer 2022, pp. 608–610, Association for Computing Machinery, New York, NY, USA, 2022, ISBN: 978-1-4503-9690-5.
@inproceedings{bublak_virtual_2022,
title = {A Virtual Environment for Emergency Ultrasound Training during Cardiopulmonary Resuscitation},
author = {T Bublak and M Bofferding and C Olson and H Jonas and J Rademeier and C Hansen},
url = {https://doi.org/10.1145/3543758.3547516},
doi = {10.1145/3543758.3547516},
isbn = {978-1-4503-9690-5},
year = {2022},
date = {2022-09-01},
urldate = {2022-09-01},
booktitle = {Proceedings of Mensch und Computer 2022},
pages = {608–610},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
series = {MuC '22},
abstract = {We propose a virtual reality (VR) communication training environment to train cardiopulmonary resuscitation (CPR) and ultrasound (US) imaging simultaneously. Existing simulators only allow separate training of the two procedures. Our method separates the users spatially, as they each work on their own manikin. One manikin is part of a US simulator commonly used in medical training, and the other manikin is a CPR manikin. Our VR application simulates the users working on the same virtual patient. This allows for an immersive training experience in which communication skills, which are especially important in such critical situations, can be practiced by medical professionals. The prototype has the potential to provide an important foundation for advanced VR-based emergency medical training.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
We propose a virtual reality (VR) communication training environment to train cardiopulmonary resuscitation (CPR) and ultrasound (US) imaging simultaneously. Existing simulators only allow separate training of the two procedures. Our method separates the users spatially, as they each work on their own manikin. One manikin is part of a US simulator commonly used in medical training, and the other manikin is a CPR manikin. Our VR application simulates the users working on the same virtual patient. This allows for an immersive training experience in which communication skills, which are especially important in such critical situations, can be practiced by medical professionals. The prototype has the potential to provide an important foundation for advanced VR-based emergency medical training.
153.
Huber, T; Huettl, F; Hanke, L; Vradelis, L; Heinrich, S; Hansen, C; Boedecker, C; Lang, H
Liver Surgery 4.0 – Planning, Volumetry, Navigation and Virtual Reality Journal Article
In: Zentralblatt für Chirurgie – Zeitschrift für Allgemeine, Viszeral-, Thorax- und Gefäßchirurgie, vol. 147, no. 04, pp. 361–368, 2022, ISSN: 0044-409X, 1438-9592.
@article{huber_liver_2022,
title = {Liver Surgery 4.0 - Planning, Volumetry, Navigation and Virtual Reality},
author = {T Huber and F Huettl and L Hanke and L Vradelis and S Heinrich and C Hansen and C Boedecker and H Lang},
url = {http://www.thieme-connect.de/DOI/DOI?10.1055/a-1844-0549},
doi = {10.1055/a-1844-0549},
issn = {0044-409X, 1438-9592},
year = {2022},
date = {2022-08-01},
urldate = {2022-08-01},
journal = {Zentralblatt für Chirurgie - Zeitschrift für Allgemeine, Viszeral-, Thorax- und Gefäßchirurgie},
volume = {147},
number = {04},
pages = {361–368},
abstract = {Due to the optimisation of conservative treatment, the improvement of imaging methods and the continuous development of surgical techniques, the borders of resectability in liver surgery have changed significantly in recent decades.
Thanks to numerous technical developments, in particular three-dimensional segmentation, preoperative planning and orientation during the operation itself, can now be facilitated, especially in complex procedures.
New technologies such as 3D printing as well as virtual and augmented reality offer additional display options for the patients’ individual anatomy. Various intraoperative navigation options are intended to make preoperative planning available in the operating room in order to increase patient safety.
This review article is intended to provide an overview of the current state of available technologies and an outlook into the operating theatre of the future.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Due to the optimisation of conservative treatment, the improvement of imaging methods and the continuous development of surgical techniques, the borders of resectability in liver surgery have changed significantly in recent decades.
Thanks to numerous technical developments, in particular three-dimensional segmentation, preoperative planning and orientation during the operation itself, can now be facilitated, especially in complex procedures.
New technologies such as 3D printing as well as virtual and augmented reality offer additional display options for the patients’ individual anatomy. Various intraoperative navigation options are intended to make preoperative planning available in the operating room in order to increase patient safety.
This review article is intended to provide an overview of the current state of available technologies and an outlook into the operating theatre of the future.
Thanks to numerous technical developments, in particular three-dimensional segmentation, preoperative planning and orientation during the operation itself, can now be facilitated, especially in complex procedures.
New technologies such as 3D printing as well as virtual and augmented reality offer additional display options for the patients’ individual anatomy. Various intraoperative navigation options are intended to make preoperative planning available in the operating room in order to increase patient safety.
This review article is intended to provide an overview of the current state of available technologies and an outlook into the operating theatre of the future.
152.
Schreiter, J; Semshchikov, V; Hanses, M; Elkmann, N; Hansen, C
Towards a real-time control of robotic ultrasound using haptic force feedback Journal Article
In: Current Directions in Biomedical Engineering, vol. 8, no. 1, pp. 81–84, 2022, ISSN: 2364-5504, (Publisher: De Gruyter).
@article{schreiter_towards_2022,
title = {Towards a real-time control of robotic ultrasound using haptic force feedback},
author = {J Schreiter and V Semshchikov and M Hanses and N Elkmann and C Hansen},
url = {https://www.degruyterbrill.com/document/doi/10.1515/cdbme-2022-0021/html},
doi = {10.1515/cdbme-2022-0021},
issn = {2364-5504},
year = {2022},
date = {2022-07-01},
urldate = {2022-07-01},
journal = {Current Directions in Biomedical Engineering},
volume = {8},
number = {1},
pages = {81–84},
abstract = {Ultrasound is a widely used imaging technique and is appreciated for its non-invasiveness, absence of radiation, widespread availability, and compact equipment. Ergonomic difficulties in manual handling of the probe could be enhanced by a robotic controlled ultrasound. The paper addresses the development of such a system which enables remote operation of a ultrasound probe and includes haptic force feedback as well as video conferencing components for visual feedback. The development process followed a user-centered approach by investigating needs of potential end-users. Preliminary results demonstrated the functionality of the developed system for generating medical image data under laboratory conditions.},
note = {Publisher: De Gruyter},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Ultrasound is a widely used imaging technique and is appreciated for its non-invasiveness, absence of radiation, widespread availability, and compact equipment. Ergonomic difficulties in manual handling of the probe could be enhanced by a robotic controlled ultrasound. The paper addresses the development of such a system which enables remote operation of a ultrasound probe and includes haptic force feedback as well as video conferencing components for visual feedback. The development process followed a user-centered approach by investigating needs of potential end-users. Preliminary results demonstrated the functionality of the developed system for generating medical image data under laboratory conditions.
151.
Schott, D; Heinrich, F; Stallmeister, L; Hansen, C
Exploring object and multi-target instrument tracking for AR-guided interventions Journal Article
In: Current Directions in Biomedical Engineering, vol. 8, no. 1, pp. 74–77, 2022, ISSN: 2364-5504, (Publisher: De Gruyter).
@article{schott_exploring_2022,
title = {Exploring object and multi-target instrument tracking for AR-guided interventions},
author = {D Schott and F Heinrich and L Stallmeister and C Hansen},
url = {https://www.degruyterbrill.com/document/doi/10.1515/cdbme-2022-0019/html},
doi = {10.1515/cdbme-2022-0019},
issn = {2364-5504},
year = {2022},
date = {2022-07-01},
urldate = {2022-07-01},
journal = {Current Directions in Biomedical Engineering},
volume = {8},
number = {1},
pages = {74–77},
abstract = {The rapid development of available hard- and software for computer-assisted or augmented reality (AR) guided interventions creates a need for fast and inexpensive prototyping environments. However, intraoperative tracking systems in particular represent a high cost threshold. Therefore, this work presents a low-cost tracking method based on a conventional RGB camera. Here, a combined approach of multiple image targets and 3D object target recognition is implemented. The system is evaluated with a systematic accuracy assessment analyzing a total of 385 3D positions. On average, a deviation of 15,69+-9,95 mm was measured. In addition, a prototypical AR-based needle navigation visualization was developed using Microsoft HoloLens 2. This system’s feasibility and usability was evaluated positively in a pilot study (n=3).},
note = {Publisher: De Gruyter},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
The rapid development of available hard- and software for computer-assisted or augmented reality (AR) guided interventions creates a need for fast and inexpensive prototyping environments. However, intraoperative tracking systems in particular represent a high cost threshold. Therefore, this work presents a low-cost tracking method based on a conventional RGB camera. Here, a combined approach of multiple image targets and 3D object target recognition is implemented. The system is evaluated with a systematic accuracy assessment analyzing a total of 385 3D positions. On average, a deviation of 15,69+-9,95 mm was measured. In addition, a prototypical AR-based needle navigation visualization was developed using Microsoft HoloLens 2. This system’s feasibility and usability was evaluated positively in a pilot study (n=3).
150.
Schott, D; Heinrich, F; Labsch, D; Hensen, B; Hansen, C
Towards multimodal interaction for needlebased procedures in a virtual radiology suite Journal Article
In: Current Directions in Biomedical Engineering, vol. 8, no. 1, pp. 70–73, 2022, ISSN: 2364-5504, (Publisher: De Gruyter).
@article{schott_towards_2022,
title = {Towards multimodal interaction for needlebased procedures in a virtual radiology suite},
author = {D Schott and F Heinrich and D Labsch and B Hensen and C Hansen},
url = {https://www.degruyterbrill.com/document/doi/10.1515/cdbme-2022-0018/html},
doi = {10.1515/cdbme-2022-0018},
issn = {2364-5504},
year = {2022},
date = {2022-07-01},
urldate = {2022-07-01},
journal = {Current Directions in Biomedical Engineering},
volume = {8},
number = {1},
pages = {70–73},
abstract = {Touchless interaction is popular in the medical domain because it maintains sterility and ensures physicians’ autonomy. Evaluating these technologies, however, proves difficult due to technical and human hurdles. Virtual reality leaves these limitations behind and allows for the exploration of promising concepts by simulating an environment and the interactions that takes place within it.We present a virtual radiology suite in the context of needle-based MR-interventions to evaluate touchless interactions. Hand and foot inputs were implemented on a custom interface and evaluated in a user study (n= 16). Results show that activating the system and manipulating values was faster with foot input. However, multimodal interaction is preferable because it is less demanding.},
note = {Publisher: De Gruyter},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Touchless interaction is popular in the medical domain because it maintains sterility and ensures physicians’ autonomy. Evaluating these technologies, however, proves difficult due to technical and human hurdles. Virtual reality leaves these limitations behind and allows for the exploration of promising concepts by simulating an environment and the interactions that takes place within it.We present a virtual radiology suite in the context of needle-based MR-interventions to evaluate touchless interactions. Hand and foot inputs were implemented on a custom interface and evaluated in a user study (n= 16). Results show that activating the system and manipulating values was faster with foot input. However, multimodal interaction is preferable because it is less demanding.
149.
Alpers, J; Hensen, B; Rötzer, M; Reimert, D; Gerlach, T; Vick, R; Gutberlet, M; Wacker, F; Hansen, C
Comparison study of reconstruction algorithms for volumetric necrosis maps from 2D multi-slice GRE thermometry images Journal Article
In: Scientific Reports, vol. 12, no. 1, pp. 11509, 2022, ISSN: 2045-2322.
@article{alpers_comparison_2022,
title = {Comparison study of reconstruction algorithms for volumetric necrosis maps from 2D multi-slice GRE thermometry images},
author = {J Alpers and B Hensen and M Rötzer and D Reimert and T Gerlach and R Vick and M Gutberlet and F Wacker and C Hansen},
url = {https://www.nature.com/articles/s41598-022-15712-7},
doi = {10.1038/s41598-022-15712-7},
issn = {2045-2322},
year = {2022},
date = {2022-07-01},
urldate = {2022-07-01},
journal = {Scientific Reports},
volume = {12},
number = {1},
pages = {11509},
abstract = {Cancer is a disease which requires a significant amount of careful medical attention. For minimally‑
invasive thermal ablation procedures, the monitoring of heat distribution is one of the biggest
challenges. In this work, three approaches for volumetric heat map reconstruction (Delauney
triangulation, minimum volume enclosing ellipsoids (MVEE) and splines) are presented based on
uniformly distributed 2D MRI phase images rotated around the applicator’s main axis. We compare
them with our previous temperature interpolation method with respect to accuracy, robustness and
adaptability. All approaches are evaluated during MWA treatment on the same data sets consisting of
13 ex vivo bio protein phantoms, including six phantoms with simulated heat sink effects. Regarding
accuracy, the DSC similarity results show a strong trend towards the MVEE ( 0.80 ± 0.03) and the
splines (0.77 ± 0.04) method compared to the Delauney triangulation ( 0.75 ± 0.02) or the temperature
interpolation (0.73 ± 0.07). Robustness is increased for all three approaches and the adaptability
shows a significant trend towards the initial interpolation method and the splines. To overcome local
inhomogeneities in the acquired data, the use of adaptive simulations should be considered in the
future. In addition, the transfer to in vivo animal experiments should be considered to test for clinical
applicability.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Cancer is a disease which requires a significant amount of careful medical attention. For minimally‑
invasive thermal ablation procedures, the monitoring of heat distribution is one of the biggest
challenges. In this work, three approaches for volumetric heat map reconstruction (Delauney
triangulation, minimum volume enclosing ellipsoids (MVEE) and splines) are presented based on
uniformly distributed 2D MRI phase images rotated around the applicator’s main axis. We compare
them with our previous temperature interpolation method with respect to accuracy, robustness and
adaptability. All approaches are evaluated during MWA treatment on the same data sets consisting of
13 ex vivo bio protein phantoms, including six phantoms with simulated heat sink effects. Regarding
accuracy, the DSC similarity results show a strong trend towards the MVEE ( 0.80 ± 0.03) and the
splines (0.77 ± 0.04) method compared to the Delauney triangulation ( 0.75 ± 0.02) or the temperature
interpolation (0.73 ± 0.07). Robustness is increased for all three approaches and the adaptability
shows a significant trend towards the initial interpolation method and the splines. To overcome local
inhomogeneities in the acquired data, the use of adaptive simulations should be considered in the
future. In addition, the transfer to in vivo animal experiments should be considered to test for clinical
applicability.
invasive thermal ablation procedures, the monitoring of heat distribution is one of the biggest
challenges. In this work, three approaches for volumetric heat map reconstruction (Delauney
triangulation, minimum volume enclosing ellipsoids (MVEE) and splines) are presented based on
uniformly distributed 2D MRI phase images rotated around the applicator’s main axis. We compare
them with our previous temperature interpolation method with respect to accuracy, robustness and
adaptability. All approaches are evaluated during MWA treatment on the same data sets consisting of
13 ex vivo bio protein phantoms, including six phantoms with simulated heat sink effects. Regarding
accuracy, the DSC similarity results show a strong trend towards the MVEE ( 0.80 ± 0.03) and the
splines (0.77 ± 0.04) method compared to the Delauney triangulation ( 0.75 ± 0.02) or the temperature
interpolation (0.73 ± 0.07). Robustness is increased for all three approaches and the adaptability
shows a significant trend towards the initial interpolation method and the splines. To overcome local
inhomogeneities in the acquired data, the use of adaptive simulations should be considered in the
future. In addition, the transfer to in vivo animal experiments should be considered to test for clinical
applicability.
148.
Allgaier, M; Chheang, V; Saalfeld, P; Apilla, V; Huber, T; Huettl, F; Neyazi, B; Sandalcioglu, E; Hansen, C; Preim, B; Saalfeld, P
A comparison of input devices for precise interaction tasks in VR-based surgical planning and training Journal Article
In: Computers in Biology and Medicine, vol. 145, pp. 105429, 2022, ISSN: 0010-4825.
@article{allgaier_comparison_2022,
title = {A comparison of input devices for precise interaction tasks in VR-based surgical planning and training},
author = {M Allgaier and V Chheang and P Saalfeld and V Apilla and T Huber and F Huettl and B Neyazi and E Sandalcioglu and C Hansen and B Preim and P Saalfeld},
url = {https://www.sciencedirect.com/science/article/pii/S0010482522002219},
doi = {10.1016/j.compbiomed.2022.105429},
issn = {0010-4825},
year = {2022},
date = {2022-06-01},
urldate = {2022-06-01},
journal = {Computers in Biology and Medicine},
volume = {145},
pages = {105429},
abstract = {To exploit the potential of virtual reality (VR) in medicine, the input devices must be selected carefully due to their different benefits. In this work, input devices for common interaction tasks in medical VR planning and training are compared. Depending on the specific purpose, different requirements exist. Therefore, an appropriate trade-off between meeting task-specific requirements and having a widely applicable device has to be found. We focus on two medical use cases, liver surgery planning and craniotomy training, to cover a broad medical domain. Based on these, relevant input devices are compared with respect to their suitability for performing precise VR interaction tasks. The devices are standard VR controllers, a pen-like VR Ink, data gloves and a real craniotome, the medical instrument used for craniotomy. The input devices were quantitatively compared with respect to their performance based on different measurements. The controllers and VR Ink performed significantly better than the remaining two devices regarding precision. Qualitative data concerning task load, cybersickness, and usability and appropriateness of the devices were assessed. Although no device stands out for both applications, most participants preferred using the VR Ink, followed by the controller and finally the data gloves and craniotome. These results can guide the selection of an appropriate device for future medical VR applications.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
To exploit the potential of virtual reality (VR) in medicine, the input devices must be selected carefully due to their different benefits. In this work, input devices for common interaction tasks in medical VR planning and training are compared. Depending on the specific purpose, different requirements exist. Therefore, an appropriate trade-off between meeting task-specific requirements and having a widely applicable device has to be found. We focus on two medical use cases, liver surgery planning and craniotomy training, to cover a broad medical domain. Based on these, relevant input devices are compared with respect to their suitability for performing precise VR interaction tasks. The devices are standard VR controllers, a pen-like VR Ink, data gloves and a real craniotome, the medical instrument used for craniotomy. The input devices were quantitatively compared with respect to their performance based on different measurements. The controllers and VR Ink performed significantly better than the remaining two devices regarding precision. Qualitative data concerning task load, cybersickness, and usability and appropriateness of the devices were assessed. Although no device stands out for both applications, most participants preferred using the VR Ink, followed by the controller and finally the data gloves and craniotome. These results can guide the selection of an appropriate device for future medical VR applications.
147.
Chheang, V; Schott, D; Saalfeld, P; Vradelis, L; Huber, T; Huettl, F; Lang, H; Preim, B; Hansen, C
Towards Virtual Teaching Hospitals for Advanced Surgical Training Proceedings Article
In: 2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), pp. 410–414, 2022.
@inproceedings{chheang_towards_2022,
title = {Towards Virtual Teaching Hospitals for Advanced Surgical Training},
author = {V Chheang and D Schott and P Saalfeld and L Vradelis and T Huber and F Huettl and H Lang and B Preim and C Hansen},
url = {https://ieeexplore.ieee.org/document/9757460},
doi = {10.1109/VRW55335.2022.00089},
year = {2022},
date = {2022-03-01},
urldate = {2022-03-01},
booktitle = {2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)},
pages = {410–414},
abstract = {Existing virtual training environments in medicine usually focus only on the training of a specific medical skill and are conducted in a single virtual room. However, medical challenges are often in the context of the planning procedure and operating room intervention. Therefore, it is crucial to provide a training environment that tackles these issues. In this paper, we present a prototype of surgical department within a virtual teaching hospital. It supports multiple users, photo-realistic avatars, and training scenarios ranging from planning procedures to laparoscopic surgery in the virtual operating room. Medical data for each virtual patient is synchronized across the different rooms, allowing users to view, analyze and train on key decision points from diagnosis to surgical incision for each patient. The environment was evaluated in a pilot study with five surgical residents and one medical student. The experts assessed the environment as an essential tool to evaluate and improve surgical planning as well as the training during critical situations of the surgical procedures. We conclude that the development of the virtual teaching hospital components described here has the potential to be a basis for future generations of medical training simulators.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Existing virtual training environments in medicine usually focus only on the training of a specific medical skill and are conducted in a single virtual room. However, medical challenges are often in the context of the planning procedure and operating room intervention. Therefore, it is crucial to provide a training environment that tackles these issues. In this paper, we present a prototype of surgical department within a virtual teaching hospital. It supports multiple users, photo-realistic avatars, and training scenarios ranging from planning procedures to laparoscopic surgery in the virtual operating room. Medical data for each virtual patient is synchronized across the different rooms, allowing users to view, analyze and train on key decision points from diagnosis to surgical incision for each patient. The environment was evaluated in a pilot study with five surgical residents and one medical student. The experts assessed the environment as an essential tool to evaluate and improve surgical planning as well as the training during critical situations of the surgical procedures. We conclude that the development of the virtual teaching hospital components described here has the potential to be a basis for future generations of medical training simulators.
146.
Heinrich, F; Schwenderling, L; Joeres, F; Hansen, C
2D versus 3D: A Comparison of Needle Navigation Concepts between Augmented Reality Display Devices Proceedings Article
In: 2022 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pp. 260–269, 2022, (ISSN: 2642-5254).
@inproceedings{heinrich_2d_2022,
title = {2D versus 3D: A Comparison of Needle Navigation Concepts between Augmented Reality Display Devices},
author = {F Heinrich and L Schwenderling and F Joeres and C Hansen},
url = {https://ieeexplore.ieee.org/document/9756753},
doi = {10.1109/VR51125.2022.00045},
year = {2022},
date = {2022-03-01},
urldate = {2022-03-01},
booktitle = {2022 IEEE Conference on Virtual Reality and 3D User Interfaces (VR)},
pages = {260–269},
abstract = {Surgical procedures requiring needle navigation assistance suffer from complicated hand-eye coordination and are mentally demanding. Augmented reality (AR) can help overcome these issues. How-ever, only an insufficient amount of fundamental research has focused on the design and hardware selection of such AR needle navigation systems. This work contributes to this research area by presenting a user study (n=24) comparing three state-of-the-art navigation concepts displayed by an optical see-through head-mounted display and a stereoscopic projection system. A two-dimensional glyph visualization resulted in higher targeting accuracy but required more needle insertion time. In contrast, punctures guided by a three-dimensional see-through vision concept were less accurate but faster and were favored in a qualitative interview. The third concept, a static representation of the correctly positioned needle, showed too high target errors for clinical accuracy needs. This concept per-formed worse when displayed by the projection system. Besides that, no meaningful differences between the evaluated AR display devices were detected. User preferences and use case restrictions, e.g., sterility requirements, seem to be more crucial selection criteria. Future work should focus on improving the accuracy of the see-through vision concept. Until then, the glyph visualization is recommended.},
note = {ISSN: 2642-5254},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Surgical procedures requiring needle navigation assistance suffer from complicated hand-eye coordination and are mentally demanding. Augmented reality (AR) can help overcome these issues. How-ever, only an insufficient amount of fundamental research has focused on the design and hardware selection of such AR needle navigation systems. This work contributes to this research area by presenting a user study (n=24) comparing three state-of-the-art navigation concepts displayed by an optical see-through head-mounted display and a stereoscopic projection system. A two-dimensional glyph visualization resulted in higher targeting accuracy but required more needle insertion time. In contrast, punctures guided by a three-dimensional see-through vision concept were less accurate but faster and were favored in a qualitative interview. The third concept, a static representation of the correctly positioned needle, showed too high target errors for clinical accuracy needs. This concept per-formed worse when displayed by the projection system. Besides that, no meaningful differences between the evaluated AR display devices were detected. User preferences and use case restrictions, e.g., sterility requirements, seem to be more crucial selection criteria. Future work should focus on improving the accuracy of the see-through vision concept. Until then, the glyph visualization is recommended.
145.
Chheang, V; Heinrich, F; Joeres, F; Saalfeld, P; Preim, B; Hansen, C
Group WiM: A Group Navigation Technique for Collaborative Virtual Reality Environments Proceedings Article
In: 2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), pp. 556–557, 2022.
@inproceedings{chheang_group_2022,
title = {Group WiM: A Group Navigation Technique for Collaborative Virtual Reality Environments},
author = {V Chheang and F Heinrich and F Joeres and P Saalfeld and B Preim and C Hansen},
url = {https://ieeexplore.ieee.org/document/9757426},
doi = {10.1109/VRW55335.2022.00129},
year = {2022},
date = {2022-03-01},
urldate = {2022-03-01},
booktitle = {2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)},
pages = {556–557},
abstract = {In this work, we present a group World-in-Miniature (WiM) navigation technique that allows a guide to navigate a team in collaborative virtual reality (VR) environments. We evaluated the usability, discomfort, and user performance of the proposed technique compared to state-of-the-art group teleportation in a user study łeft(ntextbackslash,=textbackslash,21textbackslashright). The results show that the proposed technique induces less discomfort for the guide and has slight usability advantages. Additionally, the group WiM technique seems superior in regards to task completion time for obstructed target destination. However, it performs similarly to the group teleportation technique in direct line of sight cases. The group WiM technique provides potential benefits for effective group navigation in complex virtual environments and harder-to-reach target locations.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
In this work, we present a group World-in-Miniature (WiM) navigation technique that allows a guide to navigate a team in collaborative virtual reality (VR) environments. We evaluated the usability, discomfort, and user performance of the proposed technique compared to state-of-the-art group teleportation in a user study łeft(ntextbackslash,=textbackslash,21textbackslashright). The results show that the proposed technique induces less discomfort for the guide and has slight usability advantages. Additionally, the group WiM technique seems superior in regards to task completion time for obstructed target destination. However, it performs similarly to the group teleportation technique in direct line of sight cases. The group WiM technique provides potential benefits for effective group navigation in complex virtual environments and harder-to-reach target locations.
144.
Hombeck, J; Meuschke, M; Zyla, L; Heuser, A; Toader, J; Popp, F; Bruns, C; Hansen, C; Datta, R; Lawonn, K
Evaluating Perceptional Tasks for Medicine: A Comparative User Study Between a Virtual Reality and a Desktop Application Proceedings Article
In: 2022 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pp. 514–523, 2022, (ISSN: 2642-5254).
@inproceedings{hombeck_evaluating_2022,
title = {Evaluating Perceptional Tasks for Medicine: A Comparative User Study Between a Virtual Reality and a Desktop Application},
author = {J Hombeck and M Meuschke and L Zyla and A Heuser and J Toader and F Popp and C Bruns and C Hansen and R Datta and K Lawonn},
url = {https://ieeexplore.ieee.org/abstract/document/9756759},
doi = {10.1109/VR51125.2022.00071},
year = {2022},
date = {2022-03-01},
urldate = {2022-03-01},
booktitle = {2022 IEEE Conference on Virtual Reality and 3D User Interfaces (VR)},
pages = {514–523},
abstract = {Since for most consumers the Virtual Reality (VR) experience exceeds that of desktop applications, an increasing number of applications is being transferred from desktop to VR. Industrial and entertainment applications primarily expect for a richer consumer experience, while others, such as surgical applications, seek for improved precision over their desktop counterparts. One way to improve the performance of precision-based VR applications is to provide suitable visualizations. Today, these "suitable" visualizations are mostly transferred from desktop to VR without considering their spatial and temporal performance might change in VR. This may not lead to an optimal solution, which can be crucial for precision-based tasks. Misinterpretation of shape or distance in a surgical or pre-operative simulation can affect the chosen treatment and thus directly impact the outcome. Therefore, we evaluate the performance differences of multiple visualizations for 3D surfaces based on their shape and distance estimation for desktop and VR applications. We conducted a quantitative user study with 56 participants evaluating seven visualizations (Phong, Toon, Fresnel, Pseudo-Chromadepth, Heatmap, Isolines, and Arrow Glyphs). Our results show that the performance of each visualization varies depending on the task, system, and surface type, with VR generally providing improved results. While Isolines are able to improve distance estimation, Phong and Heatmaps are beneficial for shape estimation.},
note = {ISSN: 2642-5254},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Since for most consumers the Virtual Reality (VR) experience exceeds that of desktop applications, an increasing number of applications is being transferred from desktop to VR. Industrial and entertainment applications primarily expect for a richer consumer experience, while others, such as surgical applications, seek for improved precision over their desktop counterparts. One way to improve the performance of precision-based VR applications is to provide suitable visualizations. Today, these "suitable" visualizations are mostly transferred from desktop to VR without considering their spatial and temporal performance might change in VR. This may not lead to an optimal solution, which can be crucial for precision-based tasks. Misinterpretation of shape or distance in a surgical or pre-operative simulation can affect the chosen treatment and thus directly impact the outcome. Therefore, we evaluate the performance differences of multiple visualizations for 3D surfaces based on their shape and distance estimation for desktop and VR applications. We conducted a quantitative user study with 56 participants evaluating seven visualizations (Phong, Toon, Fresnel, Pseudo-Chromadepth, Heatmap, Isolines, and Arrow Glyphs). Our results show that the performance of each visualization varies depending on the task, system, and surface type, with VR generally providing improved results. While Isolines are able to improve distance estimation, Phong and Heatmaps are beneficial for shape estimation.
143.
Hombeck, J; Meuschke, M; Lieb, S; Lichtenberg, N; Datta, R; Krone, M; Hansen, C; Preim, B; Lawonn, K
Distance Visualizations for Vascular Structures Journal Article
In: 2022.
@article{hombeck_distance_2022,
title = {Distance Visualizations for Vascular Structures},
author = {J Hombeck and M Meuschke and S Lieb and N Lichtenberg and R Datta and M Krone and C Hansen and B Preim and K Lawonn},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
abstract = {The role of expressive surface visualizations in rendering vascular structures has seen an increased impact over the last years. Surface visualizations provide an overview of complex anatomical structures and support treatment planning as well as medical education. To support decision-making, physicians need visualizations that depict anatomical structures and their spatial relations to each other, i.e., well perceivable visual encodings of egocentric and endocentric distances.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
The role of expressive surface visualizations in rendering vascular structures has seen an increased impact over the last years. Surface visualizations provide an overview of complex anatomical structures and support treatment planning as well as medical education. To support decision-making, physicians need visualizations that depict anatomical structures and their spatial relations to each other, i.e., well perceivable visual encodings of egocentric and endocentric distances.
142.
Mielke, T; Joeres, F; Hansen, C
Natural 3D Object Manipulation for Interactive Laparoscopic Augmented Reality Registration Proceedings Article
In: Chen, Jessie Y. C.; Fragomeni, Gino (Ed.): Virtual, Augmented and Mixed Reality: Design and Development, pp. 317–328, Springer International Publishing, Cham, 2022, ISBN: 978-3-031-05939-1.
@inproceedings{mielke_natural_2022,
title = {Natural 3D Object Manipulation for Interactive Laparoscopic Augmented Reality Registration},
author = {T Mielke and F Joeres and C Hansen},
editor = {Jessie Y. C. Chen and Gino Fragomeni},
doi = {10.1007/978-3-031-05939-1_21},
isbn = {978-3-031-05939-1},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Virtual, Augmented and Mixed Reality: Design and Development},
pages = {317–328},
publisher = {Springer International Publishing},
address = {Cham},
abstract = {Due to the growing focus on minimally invasive surgery, there is increasing interest in intraoperative software support. For example, augmented reality can be used to provide additional information. Accurate registration is required for effective support. In this work, we present a manual registration method that aims at mimicking natural manipulation of 3D objects using tracked surgical instruments. This method is compared to a point-based registration method in a simulated laparoscopic environment. Both registration methods serve as an initial alignment step prior to surface-based registration refinement. For the evaluation, we conducted a user study with 12 participants. The registration methods were compared in terms of registration accuracy, registration duration, and subjective usability feedback. No significant differences could be found with respect to the previously mentioned criteria between the manual and the point-based registration methods. Thus, the manual registration did not outperform the reference method. However, we found that our method offers qualitative advantages, which may make it more suitable for some application scenarios. Furthermore we identified possible approaches for improvement, which should be investigated in the future to strengthen possible advantages of our registration method.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Due to the growing focus on minimally invasive surgery, there is increasing interest in intraoperative software support. For example, augmented reality can be used to provide additional information. Accurate registration is required for effective support. In this work, we present a manual registration method that aims at mimicking natural manipulation of 3D objects using tracked surgical instruments. This method is compared to a point-based registration method in a simulated laparoscopic environment. Both registration methods serve as an initial alignment step prior to surface-based registration refinement. For the evaluation, we conducted a user study with 12 participants. The registration methods were compared in terms of registration accuracy, registration duration, and subjective usability feedback. No significant differences could be found with respect to the previously mentioned criteria between the manual and the point-based registration methods. Thus, the manual registration did not outperform the reference method. However, we found that our method offers qualitative advantages, which may make it more suitable for some application scenarios. Furthermore we identified possible approaches for improvement, which should be investigated in the future to strengthen possible advantages of our registration method.
141.
Gabele, M; Thoms, A; Schröer, S; Hußlein, S; Hansen, C
Effects and Combination of Tailored Browser-Based and Mobile Cognitive Software Training Proceedings Article
In: Jónsson, B Þór; Gurrin, C; Tran, M; Dang-Nguyen, D; Hu, A; Thanh, B Huynh Thi; Huet, B (Ed.): MultiMedia Modeling, pp. 279–291, Springer International Publishing, Cham, 2022, ISBN: 978-3-030-98355-0.
@inproceedings{gabele_effects_2022,
title = {Effects and Combination of Tailored Browser-Based and Mobile Cognitive Software Training},
author = {M Gabele and A Thoms and S Schröer and S Hußlein and C Hansen},
editor = {B Þór Jónsson and C Gurrin and M Tran and D Dang-Nguyen and A Hu and B Huynh Thi Thanh and B Huet},
doi = {10.1007/978-3-030-98355-0_24},
isbn = {978-3-030-98355-0},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {MultiMedia Modeling},
pages = {279–291},
publisher = {Springer International Publishing},
address = {Cham},
abstract = {Software-based training supports maintenance or recovery of cognitive abilities. However, regular use needs a high self-motivation. For this, the individual needs of users should be taken into account during the development process. Thus, in this work, two possibilities for tailoring in combination of application, and media devices in practice are investigated in an explorative between-subject design and hypothesis-generating study (N = 68). First, the effect of a browser-based training without gamification or with assignment of gamification appropriate to mean user characteristic on training duration is investigated. Second, the effect of this training and a subsequent serious game app on perception and possible combinations of different media is investigated. The results of behavior lead to the hypothesis that with tailored gamification in browsers, users may train longer in the middle range of training duration. The results of perception show a significant difference and higher perceived value/usefulness and overall rating, as well as the assumption of a higher effect in computer and mobile training in the group with tailored gamification. This could therefore provide a basis for combinations. A combination of both usage scenarios is perceived by users as most reasonable for a positive training effect. Nevertheless, there is a high variance for self-assessed usage. Thus, the results support developing multiple combinable scenarios, tailored to the user, in media devices and game integration to address intended effects, development and user needs to support the effect of cognitive software training in practice.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Software-based training supports maintenance or recovery of cognitive abilities. However, regular use needs a high self-motivation. For this, the individual needs of users should be taken into account during the development process. Thus, in this work, two possibilities for tailoring in combination of application, and media devices in practice are investigated in an explorative between-subject design and hypothesis-generating study (N = 68). First, the effect of a browser-based training without gamification or with assignment of gamification appropriate to mean user characteristic on training duration is investigated. Second, the effect of this training and a subsequent serious game app on perception and possible combinations of different media is investigated. The results of behavior lead to the hypothesis that with tailored gamification in browsers, users may train longer in the middle range of training duration. The results of perception show a significant difference and higher perceived value/usefulness and overall rating, as well as the assumption of a higher effect in computer and mobile training in the group with tailored gamification. This could therefore provide a basis for combinations. A combination of both usage scenarios is perceived by users as most reasonable for a positive training effect. Nevertheless, there is a high variance for self-assessed usage. Thus, the results support developing multiple combinable scenarios, tailored to the user, in media devices and game integration to address intended effects, development and user needs to support the effect of cognitive software training in practice.
140.
Batz, V; Lipowski, I; Morfeld, M; Hansen, C; Herzog, M
Accessible Design of Serious Games for People with Intellectual Disabilities in Inclusive Vocational Education Proceedings Article
In: Proceedings of DiGRA 2022 Conference: Bringing Worlds Together, 2022, (ISSN: 2342-9666).
@inproceedings{batz_accessible_2022,
title = {Accessible Design of Serious Games for People with Intellectual Disabilities in Inclusive Vocational Education},
author = {V Batz and I Lipowski and M Morfeld and C Hansen and M Herzog},
url = {https://dl.digra.org/index.php/dl/article/view/1372},
doi = {10.26503/dl.v2022i1.1372},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Proceedings of DiGRA 2022 Conference: Bringing Worlds Together},
abstract = {People with disabilities are often denied the opportunity to obtain a vocational qualification. There is a lack of an appropriate infrastructure for an inclusive education system and recognized degrees. Therefore, didactic methods for inclusive learning situations are needed. This paper describes the user-centered development and evaluation of a serious game for inclusive vocational training in kitchen professions. The theoretical teaching content in the module “hygiene” is taught to people with and without cognitive impairments using a gamified learning application. The prototype was evaluated with 22 participants when used in an inclusive teaching setting with a mixed-methods design. The learning application caused an increase in motivation and the knowledge quiz achieved good results across all target groups (15 trainee cooks, 4 trainees as kitchen assistants, 3 employees of the sheltered workshop). The results from questioning, learning quiz, observation, and video analysis form design guidelines for future digital, interactive teaching methods for inclusive vocational education.},
note = {ISSN: 2342-9666},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
People with disabilities are often denied the opportunity to obtain a vocational qualification. There is a lack of an appropriate infrastructure for an inclusive education system and recognized degrees. Therefore, didactic methods for inclusive learning situations are needed. This paper describes the user-centered development and evaluation of a serious game for inclusive vocational training in kitchen professions. The theoretical teaching content in the module “hygiene” is taught to people with and without cognitive impairments using a gamified learning application. The prototype was evaluated with 22 participants when used in an inclusive teaching setting with a mixed-methods design. The learning application caused an increase in motivation and the knowledge quiz achieved good results across all target groups (15 trainee cooks, 4 trainees as kitchen assistants, 3 employees of the sheltered workshop). The results from questioning, learning quiz, observation, and video analysis form design guidelines for future digital, interactive teaching methods for inclusive vocational education.
2021
139.
Sabieleish, M; Heryan, K; Boese, A; Hansen, C; Friebe, M; Illanes, A
Study of needle punctures into soft tissue through audio and force sensing: can audio be a simple alternative for needle guidance? Journal Article
In: International Journal of Computer Assisted Radiology and Surgery, vol. 16, no. 10, pp. 1683–1697, 2021, ISSN: 1861-6410, 1861-6429.
@article{sabieleish_study_2021,
title = {Study of needle punctures into soft tissue through audio and force sensing: can audio be a simple alternative for needle guidance?},
author = {M Sabieleish and K Heryan and A Boese and C Hansen and M Friebe and A Illanes},
url = {https://link.springer.com/10.1007/s11548-021-02479-x},
doi = {10.1007/s11548-021-02479-x},
issn = {1861-6410, 1861-6429},
year = {2021},
date = {2021-10-01},
urldate = {2021-10-01},
journal = {International Journal of Computer Assisted Radiology and Surgery},
volume = {16},
number = {10},
pages = {1683–1697},
abstract = {Purpose Percutaneous needle insertion is one of the most common minimally invasive procedures. The clinician’s experience and medical imaging support are essential to the procedure’s safety. However, imaging comes with inaccuracies due to artifacts, and therefore sensor-based solutions were proposed to improve accuracy. However, sensors are usually embedded in the needle tip, leading to design limitations. A novel concept was proposed for capturing tip–tissue interaction information through audio sensing, showing promising results for needle guidance. This work demonstrates that this audio approach can provide important puncture information by comparing audio and force signal dynamics during insertion.
Methods An experimental setup for inserting a needle into soft tissue was prepared. Audio and force signals were synchronously recorded at four different insertion velocities, and a dataset of 200 recordings was acquired. Indicators related to different aspects of the force and audio were compared through signal-to-signal and event-to-event correlation analysis.
Results High signal-to-signal correlations between force and audio indicators regardless of the insertion velocity were obtained. The force curvature indicator obtained the best correlation performances to audio with more than 70% of the correlations higher than 0.6. The event-to-event correlation analysis shows that a puncture event in the force is generally identifiable in audio and that their intensities firmly related.
Conclusions Audio contains valuable information for monitoring needle tip/tissue interaction. Significant dynamics obtained from a well-known sensor as force can also be extracted from audio, regardless of insertion velocities.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Purpose Percutaneous needle insertion is one of the most common minimally invasive procedures. The clinician’s experience and medical imaging support are essential to the procedure’s safety. However, imaging comes with inaccuracies due to artifacts, and therefore sensor-based solutions were proposed to improve accuracy. However, sensors are usually embedded in the needle tip, leading to design limitations. A novel concept was proposed for capturing tip–tissue interaction information through audio sensing, showing promising results for needle guidance. This work demonstrates that this audio approach can provide important puncture information by comparing audio and force signal dynamics during insertion.
Methods An experimental setup for inserting a needle into soft tissue was prepared. Audio and force signals were synchronously recorded at four different insertion velocities, and a dataset of 200 recordings was acquired. Indicators related to different aspects of the force and audio were compared through signal-to-signal and event-to-event correlation analysis.
Results High signal-to-signal correlations between force and audio indicators regardless of the insertion velocity were obtained. The force curvature indicator obtained the best correlation performances to audio with more than 70% of the correlations higher than 0.6. The event-to-event correlation analysis shows that a puncture event in the force is generally identifiable in audio and that their intensities firmly related.
Conclusions Audio contains valuable information for monitoring needle tip/tissue interaction. Significant dynamics obtained from a well-known sensor as force can also be extracted from audio, regardless of insertion velocities.
Methods An experimental setup for inserting a needle into soft tissue was prepared. Audio and force signals were synchronously recorded at four different insertion velocities, and a dataset of 200 recordings was acquired. Indicators related to different aspects of the force and audio were compared through signal-to-signal and event-to-event correlation analysis.
Results High signal-to-signal correlations between force and audio indicators regardless of the insertion velocity were obtained. The force curvature indicator obtained the best correlation performances to audio with more than 70% of the correlations higher than 0.6. The event-to-event correlation analysis shows that a puncture event in the force is generally identifiable in audio and that their intensities firmly related.
Conclusions Audio contains valuable information for monitoring needle tip/tissue interaction. Significant dynamics obtained from a well-known sensor as force can also be extracted from audio, regardless of insertion velocities.
138.
Wagner, S; Belger, J; Joeres, F; Thöne-Otto, A; Hansen, C; Preim, B; Saalfeld, P
iVRoad: Immersive virtual road crossing as an assessment tool for unilateral spatial neglect Journal Article
In: Computers & Graphics, vol. 99, pp. 70–82, 2021, ISSN: 0097-8493.
@article{wagner_ivroad_2021,
title = {iVRoad: Immersive virtual road crossing as an assessment tool for unilateral spatial neglect},
author = {S Wagner and J Belger and F Joeres and A Thöne-Otto and C Hansen and B Preim and P Saalfeld},
url = {https://www.sciencedirect.com/science/article/pii/S009784932100128X},
doi = {10.1016/j.cag.2021.06.013},
issn = {0097-8493},
year = {2021},
date = {2021-10-01},
urldate = {2021-10-01},
journal = {Computers & Graphics},
volume = {99},
pages = {70–82},
abstract = {We developed a virtual road crossing assessment tool called iVRoad - immersive Virtual Road, which allows to put the patient into realistic road crossing situations and to record various parameters that can be used to quantify unilateral spatial neglect. We present a study with 18 stroke patients in which we evaluate our system with respect to usability, satisfaction, sense of presence and possible occurring cybersickness symptoms. Unilateral spatial neglect is a cognitive disturbance, often occurring after right hemispheric stroke. Conventional neuropsychological tests, such as paper-and-pencil tests, for assessing unilateral spatial neglect, often lack sensitivity. Especially in mild forms, symptoms can be seen in everyday life, but are hard to detect in formal testing. We examined patients with and without unilateral spatial neglect in order to identify parameters that could be feasible to separate these patient groups. Using everyday life tasks as a diagnostic instrument, however, is challenging because it is time-consuming, hard to control and to quantify. Computer-aided diagnostic systems are promising for analysing the behaviour of patients in detail. Modern virtual reality technology allows to place the patient in realistic situations. Especially situations in which patients often have difficulties or that are too dangerous in reality can be assessed with VR.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
We developed a virtual road crossing assessment tool called iVRoad – immersive Virtual Road, which allows to put the patient into realistic road crossing situations and to record various parameters that can be used to quantify unilateral spatial neglect. We present a study with 18 stroke patients in which we evaluate our system with respect to usability, satisfaction, sense of presence and possible occurring cybersickness symptoms. Unilateral spatial neglect is a cognitive disturbance, often occurring after right hemispheric stroke. Conventional neuropsychological tests, such as paper-and-pencil tests, for assessing unilateral spatial neglect, often lack sensitivity. Especially in mild forms, symptoms can be seen in everyday life, but are hard to detect in formal testing. We examined patients with and without unilateral spatial neglect in order to identify parameters that could be feasible to separate these patient groups. Using everyday life tasks as a diagnostic instrument, however, is challenging because it is time-consuming, hard to control and to quantify. Computer-aided diagnostic systems are promising for analysing the behaviour of patients in detail. Modern virtual reality technology allows to place the patient in realistic situations. Especially situations in which patients often have difficulties or that are too dangerous in reality can be assessed with VR.
137.
Chheang, V; Saalfeld, P; Joeres, F; Boedecker, C; Huber, T; Huettl, F; Lang, H; Preim, B; Hansen, C
A collaborative virtual reality environment for liver surgery planning Journal Article
In: Computers & Graphics, vol. 99, pp. 234–246, 2021, ISSN: 0097-8493.
@article{chheang_collaborative_2021,
title = {A collaborative virtual reality environment for liver surgery planning},
author = {V Chheang and P Saalfeld and F Joeres and C Boedecker and T Huber and F Huettl and H Lang and B Preim and C Hansen},
url = {https://www.sciencedirect.com/science/article/pii/S0097849321001400},
doi = {10.1016/j.cag.2021.07.009},
issn = {0097-8493},
year = {2021},
date = {2021-10-01},
urldate = {2021-10-01},
journal = {Computers & Graphics},
volume = {99},
pages = {234–246},
abstract = {Surgical planning software is a key component in the treatment of tumor diseases. However, desktop-based systems provide only limited visualization and interaction opportunities. Moreover, collaborative planning among members of a surgical team is only possible to a limited extent. In this work, a collaborative virtual reality (VR) environment to assist liver surgeons in tumor surgery planning is presented. Our aim is to improve virtual resection planning between surgeons in a remote or co-located environment. The system allows surgeons to define and adjust virtual resections on patient-specific organ 3D surfaces and 2D image slices. Changes on both modalities are synchronized, which will enable surgeons to iterate and refine the resection surfaces quickly. In addition, a real-time risk map visualization is presented that displays safety margins around tumors. An evaluation performed by liver surgeons provides information on potential benefits, such as the possibility to visualize complex cases and assessing the safety-critical areas, applicability, and limitations for further improvement.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Surgical planning software is a key component in the treatment of tumor diseases. However, desktop-based systems provide only limited visualization and interaction opportunities. Moreover, collaborative planning among members of a surgical team is only possible to a limited extent. In this work, a collaborative virtual reality (VR) environment to assist liver surgeons in tumor surgery planning is presented. Our aim is to improve virtual resection planning between surgeons in a remote or co-located environment. The system allows surgeons to define and adjust virtual resections on patient-specific organ 3D surfaces and 2D image slices. Changes on both modalities are synchronized, which will enable surgeons to iterate and refine the resection surfaces quickly. In addition, a real-time risk map visualization is presented that displays safety margins around tumors. An evaluation performed by liver surgeons provides information on potential benefits, such as the possibility to visualize complex cases and assessing the safety-critical areas, applicability, and limitations for further improvement.
136.
Joeres, F; Mielke, T; Hansen, C
Laparoscopic augmented reality registration for oncological resection site repair Journal Article
In: International Journal of Computer Assisted Radiology and Surgery, vol. 16, no. 9, pp. 1577–1586, 2021, ISSN: 1861-6429.
@article{joeres_laparoscopic_2021,
title = {Laparoscopic augmented reality registration for oncological resection site repair},
author = {F Joeres and T Mielke and C Hansen},
url = {https://doi.org/10.1007/s11548-021-02336-x},
doi = {10.1007/s11548-021-02336-x},
issn = {1861-6429},
year = {2021},
date = {2021-09-01},
urldate = {2021-09-01},
journal = {International Journal of Computer Assisted Radiology and Surgery},
volume = {16},
number = {9},
pages = {1577–1586},
abstract = {Resection site repair during laparoscopic oncological surgery (e.g. laparoscopic partial nephrectomy) poses some unique challenges and opportunities for augmented reality (AR) navigation support. This work introduces an AR registration workflow that addresses the time pressure that is present during resection site repair.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Resection site repair during laparoscopic oncological surgery (e.g. laparoscopic partial nephrectomy) poses some unique challenges and opportunities for augmented reality (AR) navigation support. This work introduces an AR registration workflow that addresses the time pressure that is present during resection site repair.
135.
Heinrich, F; Schwenderling, L; Streuber, M; Bornemann, K; Lawonn, K; Hansen, C
Effects of Surface Visualizations on Depth Perception in Projective Augmented Reality Proceedings Article
In: 2021 IEEE 2nd International Conference on Human-Machine Systems (ICHMS), pp. 1–6, 2021.
@inproceedings{heinrich_effects_2021,
title = {Effects of Surface Visualizations on Depth Perception in Projective Augmented Reality},
author = {F Heinrich and L Schwenderling and M Streuber and K Bornemann and K Lawonn and C Hansen},
url = {https://ieeexplore.ieee.org/document/9582452},
doi = {10.1109/ICHMS53169.2021.9582452},
year = {2021},
date = {2021-09-01},
urldate = {2021-09-01},
booktitle = {2021 IEEE 2nd International Conference on Human-Machine Systems (ICHMS)},
pages = {1–6},
abstract = {Depth perception is a common issue in augmented reality (AR). Projective AR, where the spatial relations between the projection surface and displayed virtual contents need to be represented properly, is particularly affected. This is crucial in the medical domain, e.g., for the distances between the patient’s skin and projected inner anatomical structures, but not much research was conducted in this context before. To this end, this work investigates the applicability of surface visualization techniques to support the perception of spatial relations in projective AR. Four methods previously explored in different domains were combined with the projection of inner anatomical structures on a human torso phantom. They were evaluated in a comparative user study (n=21) with respect to a distance estimation and a sorting task. Measures included Task completion time, accuracy, total Head movement and Confidence of the participants. Consistent results across variables show advantages of more occluding surface visualizations for the distance estimation task. Opposite results were obtained for the sorting task. This suggests that the amount of needed surface preservation depends on the use case and individual occlusion compromises need to be explored in future work.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Depth perception is a common issue in augmented reality (AR). Projective AR, where the spatial relations between the projection surface and displayed virtual contents need to be represented properly, is particularly affected. This is crucial in the medical domain, e.g., for the distances between the patient’s skin and projected inner anatomical structures, but not much research was conducted in this context before. To this end, this work investigates the applicability of surface visualization techniques to support the perception of spatial relations in projective AR. Four methods previously explored in different domains were combined with the projection of inner anatomical structures on a human torso phantom. They were evaluated in a comparative user study (n=21) with respect to a distance estimation and a sorting task. Measures included Task completion time, accuracy, total Head movement and Confidence of the participants. Consistent results across variables show advantages of more occluding surface visualizations for the distance estimation task. Opposite results were obtained for the sorting task. This suggests that the amount of needed surface preservation depends on the use case and individual occlusion compromises need to be explored in future work.
134.
Schwenderling, L; Hansen, C; Heinrich, F
AR visualization of automated access path planning for percutaneous interventions Journal Article
In: Current Directions in Biomedical Engineering, vol. 7, no. 1, pp. 48–52, 2021, ISSN: 2364-5504, (Publisher: De Gruyter).
@article{schwenderling_ar_2021,
title = {AR visualization of automated access path planning for percutaneous interventions},
author = {L Schwenderling and C Hansen and F Heinrich},
url = {https://www.degruyterbrill.com/document/doi/10.1515/cdbme-2021-1011/html},
doi = {10.1515/cdbme-2021-1011},
issn = {2364-5504},
year = {2021},
date = {2021-08-01},
urldate = {2021-08-01},
journal = {Current Directions in Biomedical Engineering},
volume = {7},
number = {1},
pages = {48–52},
abstract = {Minimally invasive interventions, e.g., percutaneous needle interventions, have many advantages compared to traditional surgery. However, they may require complex and time-consuming planning with experience-dependent success. Automated access path planning is faster and more consistent but individual preferences and situational circumstances are not considered. To this end, displaying the path planning results directly on the patient’s skin, using projector-based augmented reality (AR), was investigated. A constraint-based path planning was implemented to evaluate the quality of every path, taking into account risk structures and path length. A visualization was developed to display the results on the skin and to allow for path selection. The choice of the path followed by a navigated insertion was evaluated in a pilot study (n=5), considering four levels of the visualization with different amounts of displayed insertion points. Participants stated that they preferred to have multiple potential puncture points displayed. However, the results for the considered variables show only small differences. Overall, it has been shown that projectorbased AR visualization of automated access path planning is possible and enables individual, situation-adapted insertion point selection. More research is required to further explore optimal display of paths.},
note = {Publisher: De Gruyter},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Minimally invasive interventions, e.g., percutaneous needle interventions, have many advantages compared to traditional surgery. However, they may require complex and time-consuming planning with experience-dependent success. Automated access path planning is faster and more consistent but individual preferences and situational circumstances are not considered. To this end, displaying the path planning results directly on the patient’s skin, using projector-based augmented reality (AR), was investigated. A constraint-based path planning was implemented to evaluate the quality of every path, taking into account risk structures and path length. A visualization was developed to display the results on the skin and to allow for path selection. The choice of the path followed by a navigated insertion was evaluated in a pilot study (n=5), considering four levels of the visualization with different amounts of displayed insertion points. Participants stated that they preferred to have multiple potential puncture points displayed. However, the results for the considered variables show only small differences. Overall, it has been shown that projectorbased AR visualization of automated access path planning is possible and enables individual, situation-adapted insertion point selection. More research is required to further explore optimal display of paths.
133.
Heinrich, F; Apilla, V; Lawonn, K; Hansen, C; Preim, B; Meuschke, M
Estimating depth information of vascular models: A comparative user study between a virtual reality and a desktop application Journal Article
In: Computers & Graphics, vol. 98, pp. 210–217, 2021, ISSN: 0097-8493.
@article{heinrich_estimating_2021,
title = {Estimating depth information of vascular models: A comparative user study between a virtual reality and a desktop application},
author = {F Heinrich and V Apilla and K Lawonn and C Hansen and B Preim and M Meuschke},
url = {https://www.sciencedirect.com/science/article/pii/S0097849321001138},
doi = {10.1016/j.cag.2021.05.014},
issn = {0097-8493},
year = {2021},
date = {2021-08-01},
urldate = {2021-08-01},
journal = {Computers & Graphics},
volume = {98},
pages = {210–217},
abstract = {Vascular structures are assessed, e.g., in tumor surgery to understand the influence of a planned resection on the vascular supply and venous drainage. The understanding of complex branching vascular structures may benefit from immersive virtual reality (VR) visualization. Therefore, the estimation of distance, depth and shape information is a crucial task to support diagnosis and therapy decisions. Depending on the visualization techniques used, perceptual issues can influence this process and may thus lead to false conclusions. Many studies were carried out to study depth perception for different variants of vessel visualization. However, these studies are restricted to desktop applications. Since VR exhibits specific perceptual problems, we aim at an understanding of the appropriateness of vessel visualization techniques in VR. Therefore, this paper presents a user study that investigates the effects of three commonly used visualization techniques on depth perception. The set of visualization techniques comprises Phong shading, pseudo-chromadepth and fog shading. An immersive VR setup of the study using a head-mounted display (HMD) was compared to a traditional desktop setup. Results suggest that depth judgments are less error-prone and more certain in VR than in desktop environments. Moreover, depth-enhancing visualization techniques had greater effects in the desktop study compared to the VR study.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Vascular structures are assessed, e.g., in tumor surgery to understand the influence of a planned resection on the vascular supply and venous drainage. The understanding of complex branching vascular structures may benefit from immersive virtual reality (VR) visualization. Therefore, the estimation of distance, depth and shape information is a crucial task to support diagnosis and therapy decisions. Depending on the visualization techniques used, perceptual issues can influence this process and may thus lead to false conclusions. Many studies were carried out to study depth perception for different variants of vessel visualization. However, these studies are restricted to desktop applications. Since VR exhibits specific perceptual problems, we aim at an understanding of the appropriateness of vessel visualization techniques in VR. Therefore, this paper presents a user study that investigates the effects of three commonly used visualization techniques on depth perception. The set of visualization techniques comprises Phong shading, pseudo-chromadepth and fog shading. An immersive VR setup of the study using a head-mounted display (HMD) was compared to a traditional desktop setup. Results suggest that depth judgments are less error-prone and more certain in VR than in desktop environments. Moreover, depth-enhancing visualization techniques had greater effects in the desktop study compared to the VR study.
132.
Huettl, F; Saalfeld, P; Hansen, C; Preim, B; Poplawski, A; Kneist, W; Lang, H; Huber, T
In: Annals of Translational Medicine, vol. 9, no. 13, pp. 1074, 2021, ISSN: 2305-5839.
@article{huettl_virtual_2021,
title = {Virtual reality and 3D printing improve preoperative visualization of 3D liver reconstructions—results from a preclinical comparison of presentation modalities and user’s preference},
author = {F Huettl and P Saalfeld and C Hansen and B Preim and A Poplawski and W Kneist and H Lang and T Huber},
url = {https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8339861/},
doi = {10.21037/atm-21-512},
issn = {2305-5839},
year = {2021},
date = {2021-07-01},
urldate = {2021-07-01},
journal = {Annals of Translational Medicine},
volume = {9},
number = {13},
pages = {1074},
abstract = {Background
Preoperative three-dimensional (3D) reconstructions for liver surgery planning have been shown to be effective in reduction of blood loss and operation time. However, the role of the ‘presentation modality’ is not well investigated. We present the first study to compare 3D PDFs, 3D printed models (PR) and virtual reality (VR) 3D models with regard to anatomical orientation and personal preferences in a high volume liver surgery center.
Methods
Thirty participants, 10 medical students, 10 residents, 5 fellows and 5 hepatopancreatobiliary (HPB) experts, assigned the tumor-bearing segments of 20 different patient’s individual liver reconstructions. Liver models were presented in a random order in all modalities. Time needed to specify the tumor location was recorded. In addition, a score was calculated factoring in correct, wrong and missing segment assignments. Furthermore, standardized test/questionnaires for spatial thinking and seeing, vegetative side effects and usability were completed.
Results Participants named significantly more correct segments in VR (P=0.040) or PR (P=0.036) compared to PDF. Tumor assignment was significantly shorter with 3D PR models compared to 3D PDF (P<0.001) or VR application (P<0.001). Regardless of the modality, HPB experts were significantly faster (24±8 vs. 35±11 sec; P=0.014) and more often correct (0.87±0.12 vs. 0.83±0.15; P<0.001) than medical students. Test results for spatial thinking and seeing had no influence on time but on correctness of tumor assignment. Regarding usability and user experience the VR application achieved the highest scores without causing significant vegetative symptoms and was also the most preferred method (n=22, 73.3%) because of the multiple functions like scaling and change of transparency. Ninety percent (n=27) stated that this application can positively influence the operation planning.
Conclusions
3D PR models and 3D VR models enable a better and partially faster anatomical orientation than reconstructions presented as 3D PDFs. User’s preferred the VR application over the PR models and PDF. A prospective trial is needed to evaluate the different presentation modalities regarding intra- and postoperative outcomes.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Background
Preoperative three-dimensional (3D) reconstructions for liver surgery planning have been shown to be effective in reduction of blood loss and operation time. However, the role of the ‘presentation modality’ is not well investigated. We present the first study to compare 3D PDFs, 3D printed models (PR) and virtual reality (VR) 3D models with regard to anatomical orientation and personal preferences in a high volume liver surgery center.
Methods
Thirty participants, 10 medical students, 10 residents, 5 fellows and 5 hepatopancreatobiliary (HPB) experts, assigned the tumor-bearing segments of 20 different patient’s individual liver reconstructions. Liver models were presented in a random order in all modalities. Time needed to specify the tumor location was recorded. In addition, a score was calculated factoring in correct, wrong and missing segment assignments. Furthermore, standardized test/questionnaires for spatial thinking and seeing, vegetative side effects and usability were completed.
Results Participants named significantly more correct segments in VR (P=0.040) or PR (P=0.036) compared to PDF. Tumor assignment was significantly shorter with 3D PR models compared to 3D PDF (P<0.001) or VR application (P<0.001). Regardless of the modality, HPB experts were significantly faster (24±8 vs. 35±11 sec; P=0.014) and more often correct (0.87±0.12 vs. 0.83±0.15; P<0.001) than medical students. Test results for spatial thinking and seeing had no influence on time but on correctness of tumor assignment. Regarding usability and user experience the VR application achieved the highest scores without causing significant vegetative symptoms and was also the most preferred method (n=22, 73.3%) because of the multiple functions like scaling and change of transparency. Ninety percent (n=27) stated that this application can positively influence the operation planning.
Conclusions
3D PR models and 3D VR models enable a better and partially faster anatomical orientation than reconstructions presented as 3D PDFs. User’s preferred the VR application over the PR models and PDF. A prospective trial is needed to evaluate the different presentation modalities regarding intra- and postoperative outcomes.
Preoperative three-dimensional (3D) reconstructions for liver surgery planning have been shown to be effective in reduction of blood loss and operation time. However, the role of the ‘presentation modality’ is not well investigated. We present the first study to compare 3D PDFs, 3D printed models (PR) and virtual reality (VR) 3D models with regard to anatomical orientation and personal preferences in a high volume liver surgery center.
Methods
Thirty participants, 10 medical students, 10 residents, 5 fellows and 5 hepatopancreatobiliary (HPB) experts, assigned the tumor-bearing segments of 20 different patient’s individual liver reconstructions. Liver models were presented in a random order in all modalities. Time needed to specify the tumor location was recorded. In addition, a score was calculated factoring in correct, wrong and missing segment assignments. Furthermore, standardized test/questionnaires for spatial thinking and seeing, vegetative side effects and usability were completed.
Results Participants named significantly more correct segments in VR (P=0.040) or PR (P=0.036) compared to PDF. Tumor assignment was significantly shorter with 3D PR models compared to 3D PDF (P<0.001) or VR application (P<0.001). Regardless of the modality, HPB experts were significantly faster (24±8 vs. 35±11 sec; P=0.014) and more often correct (0.87±0.12 vs. 0.83±0.15; P<0.001) than medical students. Test results for spatial thinking and seeing had no influence on time but on correctness of tumor assignment. Regarding usability and user experience the VR application achieved the highest scores without causing significant vegetative symptoms and was also the most preferred method (n=22, 73.3%) because of the multiple functions like scaling and change of transparency. Ninety percent (n=27) stated that this application can positively influence the operation planning.
Conclusions
3D PR models and 3D VR models enable a better and partially faster anatomical orientation than reconstructions presented as 3D PDFs. User’s preferred the VR application over the PR models and PDF. A prospective trial is needed to evaluate the different presentation modalities regarding intra- and postoperative outcomes.
131.
Wei, W; Haishan, X; Alpers, J; Rak, M; Hansen, C
A deep learning approach for 2D ultrasound and 3D CT/MR image registration in liver tumor ablation Journal Article
In: Computer Methods and Programs in Biomedicine, vol. 206, pp. 106117, 2021, ISSN: 0169-2607.
@article{wei_deep_2021,
title = {A deep learning approach for 2D ultrasound and 3D CT/MR image registration in liver tumor ablation},
author = {W Wei and X Haishan and J Alpers and M Rak and C Hansen},
url = {https://www.sciencedirect.com/science/article/pii/S0169260721001929},
doi = {10.1016/j.cmpb.2021.106117},
issn = {0169-2607},
year = {2021},
date = {2021-07-01},
urldate = {2021-07-01},
journal = {Computer Methods and Programs in Biomedicine},
volume = {206},
pages = {106117},
abstract = {Background and Objective
Liver tumor ablation is often guided by ultrasound (US). Due to poor image quality, intraoperative US is fused with preoperative computed tomography or magnetic tomography (CT/MR) images to provide visual guidance. As of today, the underlying 2D US to 3D CT/MR registration problem remains a very challenging task.
Methods
We propose a novel pipeline to address this registration problem. Contrary to previous work, we do not formulate the problem as a regression task, which - for the given registration problem - achieves a low performance regarding accuracy and robustness due to the limited US soft-tissue contrast and the inter-patient variability on liver vessels. Instead, we first estimate the US probe angle roughly by using a classification network. Given this coarse initialization, we then improve the registration by formulation of the problem as a segmentation task, estimating the US plane in the 3D CT/MR through segmentation.
Results
We benchmark our approach on 1035 clinical images from 52 patients, yielding average registration errors of 11.6° and 4.7 mm, which outperforms the state of the art SVR method[1].
Conclusion
Our results show the efficiency of the proposed registration pipeline, which has potential to improve the robustness and accuracy of intraoperative patient registration.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Background and Objective
Liver tumor ablation is often guided by ultrasound (US). Due to poor image quality, intraoperative US is fused with preoperative computed tomography or magnetic tomography (CT/MR) images to provide visual guidance. As of today, the underlying 2D US to 3D CT/MR registration problem remains a very challenging task.
Methods
We propose a novel pipeline to address this registration problem. Contrary to previous work, we do not formulate the problem as a regression task, which – for the given registration problem – achieves a low performance regarding accuracy and robustness due to the limited US soft-tissue contrast and the inter-patient variability on liver vessels. Instead, we first estimate the US probe angle roughly by using a classification network. Given this coarse initialization, we then improve the registration by formulation of the problem as a segmentation task, estimating the US plane in the 3D CT/MR through segmentation.
Results
We benchmark our approach on 1035 clinical images from 52 patients, yielding average registration errors of 11.6° and 4.7 mm, which outperforms the state of the art SVR method[1].
Conclusion
Our results show the efficiency of the proposed registration pipeline, which has potential to improve the robustness and accuracy of intraoperative patient registration.
Liver tumor ablation is often guided by ultrasound (US). Due to poor image quality, intraoperative US is fused with preoperative computed tomography or magnetic tomography (CT/MR) images to provide visual guidance. As of today, the underlying 2D US to 3D CT/MR registration problem remains a very challenging task.
Methods
We propose a novel pipeline to address this registration problem. Contrary to previous work, we do not formulate the problem as a regression task, which – for the given registration problem – achieves a low performance regarding accuracy and robustness due to the limited US soft-tissue contrast and the inter-patient variability on liver vessels. Instead, we first estimate the US probe angle roughly by using a classification network. Given this coarse initialization, we then improve the registration by formulation of the problem as a segmentation task, estimating the US plane in the 3D CT/MR through segmentation.
Results
We benchmark our approach on 1035 clinical images from 52 patients, yielding average registration errors of 11.6° and 4.7 mm, which outperforms the state of the art SVR method[1].
Conclusion
Our results show the efficiency of the proposed registration pipeline, which has potential to improve the robustness and accuracy of intraoperative patient registration.
130.
Joeres, F; Heinrich, F; Schott, D; Hansen, C
Towards natural 3D interaction for laparoscopic augmented reality registration Journal Article
In: Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization, vol. 9, no. 4, pp. 384–391, 2021, ISSN: 2168-1163, (Publisher: Taylor & Francis _eprint: https://doi.org/10.1080/21681163.2020.1834877).
@article{joeres_towards_2021,
title = {Towards natural 3D interaction for laparoscopic augmented reality registration},
author = {F Joeres and F Heinrich and D Schott and C Hansen},
url = {https://doi.org/10.1080/21681163.2020.1834877},
doi = {10.1080/21681163.2020.1834877},
issn = {2168-1163},
year = {2021},
date = {2021-07-01},
urldate = {2021-07-01},
journal = {Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization},
volume = {9},
number = {4},
pages = {384–391},
abstract = {Augmented reality (AR) is a widely researched route for navigation support in laparoscopic surgery. Accurate registration is a crucial component for such AR systems. We introduce two methods for interactive registration that aim to be minimally invasive to the workflow and to mimic natural manipulation of 3D objects. The methods utilise spatially tracked laparoscopic tools to manipulate the virtual 3D content. We comparatively evaluated the methods against a reference, landmark-based registration method in a user study with 12 participants. We tested the methods for registration accuracy, time, and subjective usability perception. Our methods did not outperform the reference method on these parameters but showed promising results. The results indicate that our methods present no finalised solutions but that one of them is a promising approach for which we identified concrete improvement measures to be implemented in future research.},
note = {Publisher: Taylor & Francis
_eprint: https://doi.org/10.1080/21681163.2020.1834877},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Augmented reality (AR) is a widely researched route for navigation support in laparoscopic surgery. Accurate registration is a crucial component for such AR systems. We introduce two methods for interactive registration that aim to be minimally invasive to the workflow and to mimic natural manipulation of 3D objects. The methods utilise spatially tracked laparoscopic tools to manipulate the virtual 3D content. We comparatively evaluated the methods against a reference, landmark-based registration method in a user study with 12 participants. We tested the methods for registration accuracy, time, and subjective usability perception. Our methods did not outperform the reference method on these parameters but showed promising results. The results indicate that our methods present no finalised solutions but that one of them is a promising approach for which we identified concrete improvement measures to be implemented in future research.
129.
Gabele, M; Weicker, J; Wagner, S; Thoms, A; Hußlein, S; Hansen, C
Effects and Ways of Tailored Gamification in Software-Based Training in Cognitive Rehabilitation Proceedings Article
In: Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization, pp. 158–168, Association for Computing Machinery, New York, NY, USA, 2021, ISBN: 978-1-4503-8366-0.
@inproceedings{gabele_effects_2021,
title = {Effects and Ways of Tailored Gamification in Software-Based Training in Cognitive Rehabilitation},
author = {M Gabele and J Weicker and S Wagner and A Thoms and S Hußlein and C Hansen},
url = {https://dl.acm.org/doi/10.1145/3450613.3456828},
doi = {10.1145/3450613.3456828},
isbn = {978-1-4503-8366-0},
year = {2021},
date = {2021-06-01},
urldate = {2021-06-01},
booktitle = {Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization},
pages = {158–168},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
series = {UMAP '21},
abstract = {A high level of motivation and frequent training are relevant in software-based rehabilitation to improve cognitive functioning after acquired brain injury. We evaluated the benefit of tailored user-centered gamification elements in a clinical study with N=83 outpatients undergoing three weeks of cognitive training in their home environment. The use of gamification in relation to the patient’s player type was explored in three steps. First, we determined the individual player types and related requests for specific game elements by means of questionnaires. Afterwards, we examined the effect of gamified training based on a non-player character and training progress within a metaphor. We considered secondly the individual perception and emotional effect and thirdly the performance based on training duration. 37 elements were requested by patients of all types, 18 elements were partially requested, and 4 elements were rejected. A comparison shows that the requested game elements partly differ between healthy persons and patients. Overall, gamification was perceived positively and gamified training leads to an increase in enjoyment compared to non-gamified training. In detail, however, there were different effects on the individual player types: socialisers experienced more enjoyment while achievers perceived higher competence throughout gamified cognitive training. Also, differences in performance in training duration were found. Within gamified training, socialisers trained significantly more than patients not primarily assigned to this type. In contrast, no significant difference was found for achievers. By showing modulating requests and effects in player types, our results support user-centered tailoring of game elements in the development of software-based cognitive training in rehabilitation.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
A high level of motivation and frequent training are relevant in software-based rehabilitation to improve cognitive functioning after acquired brain injury. We evaluated the benefit of tailored user-centered gamification elements in a clinical study with N=83 outpatients undergoing three weeks of cognitive training in their home environment. The use of gamification in relation to the patient’s player type was explored in three steps. First, we determined the individual player types and related requests for specific game elements by means of questionnaires. Afterwards, we examined the effect of gamified training based on a non-player character and training progress within a metaphor. We considered secondly the individual perception and emotional effect and thirdly the performance based on training duration. 37 elements were requested by patients of all types, 18 elements were partially requested, and 4 elements were rejected. A comparison shows that the requested game elements partly differ between healthy persons and patients. Overall, gamification was perceived positively and gamified training leads to an increase in enjoyment compared to non-gamified training. In detail, however, there were different effects on the individual player types: socialisers experienced more enjoyment while achievers perceived higher competence throughout gamified cognitive training. Also, differences in performance in training duration were found. Within gamified training, socialisers trained significantly more than patients not primarily assigned to this type. In contrast, no significant difference was found for achievers. By showing modulating requests and effects in player types, our results support user-centered tailoring of game elements in the development of software-based cognitive training in rehabilitation.
128.
Meyer, A; Mehrtash, A; Rak, M; Bashkanov, O; Langbein, B; Ziaei, A; Kibel, A; Tempany, C; Hansen, C; Tokuda, J
Domain adaptation for segmentation of critical structures for prostate cancer therapy Journal Article
In: Scientific Reports, vol. 11, no. 1, pp. 11480, 2021, ISSN: 2045-2322.
@article{meyer_domain_2021,
title = {Domain adaptation for segmentation of critical structures for prostate cancer therapy},
author = {A Meyer and A Mehrtash and M Rak and O Bashkanov and B Langbein and A Ziaei and A Kibel and C Tempany and C Hansen and J Tokuda},
url = {https://www.nature.com/articles/s41598-021-90294-4},
doi = {10.1038/s41598-021-90294-4},
issn = {2045-2322},
year = {2021},
date = {2021-06-01},
urldate = {2021-06-01},
journal = {Scientific Reports},
volume = {11},
number = {1},
pages = {11480},
abstract = {Preoperative assessment of the proximity of critical structures to the tumors is crucial in avoiding
unnecessary damage during prostate cancer treatment. A patient-specific 3D anatomical model
of those structures, namely the neurovascular bundles (NVB) and the external urethral sphincters
(EUS), can enable physicians to perform such assessments intuitively. As a crucial step to generate a
patient-specific anatomical model from preoperative MRI in a clinical routine, we propose a multi-class
automatic segmentation based on an anisotropic convolutional network. Our specific challenge is to
train the network model on a unique source dataset only available at a single clinical site and deploy it
to another target site without sharing the original images or labels. As network models trained on data
from a single source suffer from quality loss due to the domain shift, we propose a semi-supervised
domain adaptation (DA) method to refine the model’s performance in the target domain. Our DA
method combines transfer learning and uncertainty guided self-learning based on deep ensembles.
Experiments on the segmentation of the prostate, NVB, and EUS, show significant performance gain
with the combination of those techniques compared to pure TL and the combination of TL with simple
self-learning ( p < 0.005 for all structures using a Wilcoxon’s signed-rank test). Results on a different
task and data (Pancreas CT segmentation) demonstrate our method’s generic application capabilities.
Our method has the advantage that it does not require any further data from the source domain,
unlike the majority of recent domain adaptation strategies. This makes our method suitable for clinical
applications, where the sharing of patient data is restricted.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Preoperative assessment of the proximity of critical structures to the tumors is crucial in avoiding
unnecessary damage during prostate cancer treatment. A patient-specific 3D anatomical model
of those structures, namely the neurovascular bundles (NVB) and the external urethral sphincters
(EUS), can enable physicians to perform such assessments intuitively. As a crucial step to generate a
patient-specific anatomical model from preoperative MRI in a clinical routine, we propose a multi-class
automatic segmentation based on an anisotropic convolutional network. Our specific challenge is to
train the network model on a unique source dataset only available at a single clinical site and deploy it
to another target site without sharing the original images or labels. As network models trained on data
from a single source suffer from quality loss due to the domain shift, we propose a semi-supervised
domain adaptation (DA) method to refine the model’s performance in the target domain. Our DA
method combines transfer learning and uncertainty guided self-learning based on deep ensembles.
Experiments on the segmentation of the prostate, NVB, and EUS, show significant performance gain
with the combination of those techniques compared to pure TL and the combination of TL with simple
self-learning ( p < 0.005 for all structures using a Wilcoxon’s signed-rank test). Results on a different
task and data (Pancreas CT segmentation) demonstrate our method’s generic application capabilities.
Our method has the advantage that it does not require any further data from the source domain,
unlike the majority of recent domain adaptation strategies. This makes our method suitable for clinical
applications, where the sharing of patient data is restricted.
unnecessary damage during prostate cancer treatment. A patient-specific 3D anatomical model
of those structures, namely the neurovascular bundles (NVB) and the external urethral sphincters
(EUS), can enable physicians to perform such assessments intuitively. As a crucial step to generate a
patient-specific anatomical model from preoperative MRI in a clinical routine, we propose a multi-class
automatic segmentation based on an anisotropic convolutional network. Our specific challenge is to
train the network model on a unique source dataset only available at a single clinical site and deploy it
to another target site without sharing the original images or labels. As network models trained on data
from a single source suffer from quality loss due to the domain shift, we propose a semi-supervised
domain adaptation (DA) method to refine the model’s performance in the target domain. Our DA
method combines transfer learning and uncertainty guided self-learning based on deep ensembles.
Experiments on the segmentation of the prostate, NVB, and EUS, show significant performance gain
with the combination of those techniques compared to pure TL and the combination of TL with simple
self-learning ( p < 0.005 for all structures using a Wilcoxon’s signed-rank test). Results on a different
task and data (Pancreas CT segmentation) demonstrate our method’s generic application capabilities.
Our method has the advantage that it does not require any further data from the source domain,
unlike the majority of recent domain adaptation strategies. This makes our method suitable for clinical
applications, where the sharing of patient data is restricted.
127.
Boedecker, C; Huettl, F; Saalfeld, P; Paschold, M; Kneist, W; Baumgart, J; Preim, B; Hansen, C; Lang, H; Huber, T
Using virtual 3D-models in surgical planning: workflow of an immersive virtual reality application in liver surgery Journal Article
In: Langenbeck's Archives of Surgery, vol. 406, no. 3, pp. 911–915, 2021, ISSN: 1435-2451.
@article{boedecker_using_2021,
title = {Using virtual 3D-models in surgical planning: workflow of an immersive virtual reality application in liver surgery},
author = {C Boedecker and F Huettl and P Saalfeld and M Paschold and W Kneist and J Baumgart and B Preim and C Hansen and H Lang and T Huber},
url = {https://doi.org/10.1007/s00423-021-02127-7},
doi = {10.1007/s00423-021-02127-7},
issn = {1435-2451},
year = {2021},
date = {2021-05-01},
urldate = {2021-05-01},
journal = {Langenbeck's Archives of Surgery},
volume = {406},
number = {3},
pages = {911–915},
abstract = {Three-dimensional (3D) surgical planning is widely accepted in liver surgery. Currently, the 3D reconstructions are usually presented as 3D PDF data on regular monitors. 3D-printed liver models are sometimes used for education and planning.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Three-dimensional (3D) surgical planning is widely accepted in liver surgery. Currently, the 3D reconstructions are usually presented as 3D PDF data on regular monitors. 3D-printed liver models are sometimes used for education and planning.