Modeling interface signs for the systematic analysis of text, image and sound in interactive systems

This paper seeks to present standard descriptive structures for the systematic analysis of the visual and audio tracks in interactive systems. Semiotic theory provides an array of analytical frameworks concerned with the study of complex symbolic communications and the social and cultural aspects of their use and interpretation. The model structures introduced here employ structuralist formal descriptive methods for describing and quantifying the visual and audio tracks of interactive systems for semiotic investigations. Quantitative data may be extracted by analysts for research projects that explore viewer interaction from semiotic, ethnographic, and psychological, perspectives. The proposed model structures may prove invaluable for populating various behavioral, cognitive, or social models used in advanced social science investigations, applied semiotic research, or dynamic computational applications including AI, neural network, and other predictive modeling techniques.