An Ensemble Model for the Analysis of Parent-Child Interactions from Text and Audio

The interaction between parents and their children is foundational for their social and emotional development and well-being. Among measures that assess parent-child interaction quality, the Parent-Child Interaction Teaching Scale (PCI-TS) is an effective and well-established assessment tool that measures the quality of parent-child interactions. One of the significant hurdles in PCI-TS is identifying parent-child behaviors early on to aid parents in addressing initial behavioral issues. However, manual evaluations are resource-intensive and time-consuming, and restrict the accessibility of such assessments. In this research, we present an ensemble model to categorize various behavior types in parent-child interaction scales such as PCI-TS. We have collected over 2,000 minutes of video of parents and their children emerging in specific tasks such as drawing or object manipulation. These videos have been labeled based on PCI-TS by trained healthcare professionals. Our ensemble model has been trained to recognize emotions and psychological elements within parent-child interactions. This model is adjusted by multiple modalities and aspects of these psychological elements. We posit that our study introduces the first model that employs a new framework to integrate numerous behavioral factors to recognize these psychological elements within parent-child interactions using audio and text modalities. Our suggested model can discern and detect the semantic and syntactic features of audio and language in parent-child interactions according to the PCI-TS scale. The results of our evaluation showed an enhancement in performance when implementing this strategy, compared to similar approaches by 16% in F1-score.