AI-driven Family Interaction Over Melded Space and Time

Computer-mediated interaction services connect people over a distance. However, we address that those people are often “locked in a frame”—which includes an interaction mode, a point in time, or a context of either person. We observe that such lock-ins make it difficult to shape the interaction to be mutually symmetric. In this article, we propose a semantic-equivalent melding of space and time to provide a new form of empathetic interaction. We present HomeMeld and MomentMeld, which aim to meld space and time by applying AI, respectively. HomeMeld provides a sense of living together to a family living apart with AI-driven autonomous robotic avatars. MomentMeld utilizes an ensemble of visual AI to create interaction topics by matching semantic-equivalent photos. In-the-wild experiments reveal that HomeMeld and MomentMeld open new possibilities for empathetic interaction. Finally, we introduce a new interaction service leveraging technical synergy of HomeMeld and MomentMeld.