Image Understanding of GUI Widgets for Test Reuse

Testing of UI after its development is an essential activity. Due to the complexity of the task of developing new tests for UI, it is an expensive one. Different approaches for reusing tests in similar scenarios have been created. These different methods employ different information available in the GUI to decide the appropriate transfer of tests from one UI to another. Similar GUI events are related together by these systems to decide the transferability of tests. The usage of semantic knowledge during the matching of GUI events has proven to be an important asset in understanding their similarities. The usage of textual information available in GUI and their semantic meanings in this task have promising results. However, the usage of semantic information that can be obtained from images and icons has remained unexplored. This study presents the integration of semantic annotation of icons, widgets and images for semantic matching of GUI events. Proposed findings demonstrate the importance of image understanding of GUI widgets in test reuse and offer practical insights for software developers and researchers. In addition to improving semantic matching, the integration of semantic annotation of icons, widgets, and images can also enhance the interpretability and maintainability of UI tests.