A Semi-Automated Usability Evaluation Framework for Interactive Image Segmentation Systems
For complex segmentation tasks, the achievable accuracy of fully automated systems is inherently limited. Specifically, when a precise segmentation result is desired for a small amount of given data sets, semi-automatic methods exhibit a clear benefit for the user. The optimization of human computer...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2019-01-01
|
| Series: | International Journal of Biomedical Imaging |
| Online Access: | http://dx.doi.org/10.1155/2019/1464592 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832548767379750912 |
|---|---|
| author | Mario Amrehn Stefan Steidl Reinier Kortekaas Maddalena Strumia Markus Weingarten Markus Kowarschik Andreas Maier |
| author_facet | Mario Amrehn Stefan Steidl Reinier Kortekaas Maddalena Strumia Markus Weingarten Markus Kowarschik Andreas Maier |
| author_sort | Mario Amrehn |
| collection | DOAJ |
| description | For complex segmentation tasks, the achievable accuracy of fully automated systems is inherently limited. Specifically, when a precise segmentation result is desired for a small amount of given data sets, semi-automatic methods exhibit a clear benefit for the user. The optimization of human computer interaction (HCI) is an essential part of interactive image segmentation. Nevertheless, publications introducing novel interactive segmentation systems (ISS) often lack an objective comparison of HCI aspects. It is demonstrated that even when the underlying segmentation algorithm is the same throughout interactive prototypes, their user experience may vary substantially. As a result, users prefer simple interfaces as well as a considerable degree of freedom to control each iterative step of the segmentation. In this article, an objective method for the comparison of ISS is proposed, based on extensive user studies. A summative qualitative content analysis is conducted via abstraction of visual and verbal feedback given by the participants. A direct assessment of the segmentation system is executed by the users via the system usability scale (SUS) and AttrakDiff-2 questionnaires. Furthermore, an approximation of the findings regarding usability aspects in those studies is introduced, conducted solely from the system-measurable user actions during their usage of interactive segmentation prototypes. The prediction of all questionnaire results has an average relative error of 8.9%, which is close to the expected precision of the questionnaire results themselves. This automated evaluation scheme may significantly reduce the resources necessary to investigate each variation of a prototype’s user interface (UI) features and segmentation methodologies. |
| format | Article |
| id | doaj-art-6ba8d87923b949508686aa55cb7c5ac0 |
| institution | Kabale University |
| issn | 1687-4188 1687-4196 |
| language | English |
| publishDate | 2019-01-01 |
| publisher | Wiley |
| record_format | Article |
| series | International Journal of Biomedical Imaging |
| spelling | doaj-art-6ba8d87923b949508686aa55cb7c5ac02025-02-03T06:13:07ZengWileyInternational Journal of Biomedical Imaging1687-41881687-41962019-01-01201910.1155/2019/14645921464592A Semi-Automated Usability Evaluation Framework for Interactive Image Segmentation SystemsMario Amrehn0Stefan Steidl1Reinier Kortekaas2Maddalena Strumia3Markus Weingarten4Markus Kowarschik5Andreas Maier6The Pattern Recognition Lab, Computer Science Department, Friedrich-Alexander University Erlangen-Nuremberg, GermanyThe Pattern Recognition Lab, Computer Science Department, Friedrich-Alexander University Erlangen-Nuremberg, GermanySiemens Healthineers AG, Forchheim, GermanySiemens Healthineers AG, Forchheim, GermanySiemens Healthineers AG, Forchheim, GermanySiemens Healthineers AG, Forchheim, GermanyThe Pattern Recognition Lab, Computer Science Department, Friedrich-Alexander University Erlangen-Nuremberg, GermanyFor complex segmentation tasks, the achievable accuracy of fully automated systems is inherently limited. Specifically, when a precise segmentation result is desired for a small amount of given data sets, semi-automatic methods exhibit a clear benefit for the user. The optimization of human computer interaction (HCI) is an essential part of interactive image segmentation. Nevertheless, publications introducing novel interactive segmentation systems (ISS) often lack an objective comparison of HCI aspects. It is demonstrated that even when the underlying segmentation algorithm is the same throughout interactive prototypes, their user experience may vary substantially. As a result, users prefer simple interfaces as well as a considerable degree of freedom to control each iterative step of the segmentation. In this article, an objective method for the comparison of ISS is proposed, based on extensive user studies. A summative qualitative content analysis is conducted via abstraction of visual and verbal feedback given by the participants. A direct assessment of the segmentation system is executed by the users via the system usability scale (SUS) and AttrakDiff-2 questionnaires. Furthermore, an approximation of the findings regarding usability aspects in those studies is introduced, conducted solely from the system-measurable user actions during their usage of interactive segmentation prototypes. The prediction of all questionnaire results has an average relative error of 8.9%, which is close to the expected precision of the questionnaire results themselves. This automated evaluation scheme may significantly reduce the resources necessary to investigate each variation of a prototype’s user interface (UI) features and segmentation methodologies.http://dx.doi.org/10.1155/2019/1464592 |
| spellingShingle | Mario Amrehn Stefan Steidl Reinier Kortekaas Maddalena Strumia Markus Weingarten Markus Kowarschik Andreas Maier A Semi-Automated Usability Evaluation Framework for Interactive Image Segmentation Systems International Journal of Biomedical Imaging |
| title | A Semi-Automated Usability Evaluation Framework for Interactive Image Segmentation Systems |
| title_full | A Semi-Automated Usability Evaluation Framework for Interactive Image Segmentation Systems |
| title_fullStr | A Semi-Automated Usability Evaluation Framework for Interactive Image Segmentation Systems |
| title_full_unstemmed | A Semi-Automated Usability Evaluation Framework for Interactive Image Segmentation Systems |
| title_short | A Semi-Automated Usability Evaluation Framework for Interactive Image Segmentation Systems |
| title_sort | semi automated usability evaluation framework for interactive image segmentation systems |
| url | http://dx.doi.org/10.1155/2019/1464592 |
| work_keys_str_mv | AT marioamrehn asemiautomatedusabilityevaluationframeworkforinteractiveimagesegmentationsystems AT stefansteidl asemiautomatedusabilityevaluationframeworkforinteractiveimagesegmentationsystems AT reinierkortekaas asemiautomatedusabilityevaluationframeworkforinteractiveimagesegmentationsystems AT maddalenastrumia asemiautomatedusabilityevaluationframeworkforinteractiveimagesegmentationsystems AT markusweingarten asemiautomatedusabilityevaluationframeworkforinteractiveimagesegmentationsystems AT markuskowarschik asemiautomatedusabilityevaluationframeworkforinteractiveimagesegmentationsystems AT andreasmaier asemiautomatedusabilityevaluationframeworkforinteractiveimagesegmentationsystems AT marioamrehn semiautomatedusabilityevaluationframeworkforinteractiveimagesegmentationsystems AT stefansteidl semiautomatedusabilityevaluationframeworkforinteractiveimagesegmentationsystems AT reinierkortekaas semiautomatedusabilityevaluationframeworkforinteractiveimagesegmentationsystems AT maddalenastrumia semiautomatedusabilityevaluationframeworkforinteractiveimagesegmentationsystems AT markusweingarten semiautomatedusabilityevaluationframeworkforinteractiveimagesegmentationsystems AT markuskowarschik semiautomatedusabilityevaluationframeworkforinteractiveimagesegmentationsystems AT andreasmaier semiautomatedusabilityevaluationframeworkforinteractiveimagesegmentationsystems |