Exploring semantic roles of Web interface components

The adaptability of Web interfaces in response to changes in the interaction context, display environments (e.g., mobile screens) and user's personal preferences is becoming increasingly desirable due to the pervasive use of Web information. One of the major challenges in Web interface adaptation is to discover the semantic structure underlying a Web interface. This paper presents a robust and formal approach to recovering interface semantics using a graph grammar approach. Due to its distinct characteristics of spatial specification in the abstract syntax, the Spatial Graph Grammar (SGG) is used to perform semantic grouping and interpretation of segmented screen objects. We use the well-established image processing technology to recognize atomic interface objects in an interface image. The output is a spatial graph, which records significant spatial relations among recognized objects. Based on the spatial graph, the SGG parser recovers the hierarchical relations among interface objects and thus provides semantic interpretation suitable for adaptation.