Media Orchestration Between Streams and Devices via New MPEG Timed Metadata

The proliferation of new capabilities in affordable smart devices capable of capturing, processing, and rendering audiovisual media content triggers a need for coordination and orchestration between these devices and their capabilities and of the content flowing from and to such devices. The upcoming Moving Picture Experts Group (MPEG) Media Orchestration standard (MORE, ISO/IEC 23001–13) enables the temporal and spatial orchestration of multiple media and metadata streams. The temporal orchestration is about time synchronization of media and sensor captures, processing, and renderings, for which the MORE standard uses and extends a Digital Video Broadcasting standard. The spatial orchestration is about the alignment of (global) position, altitude, and orientation for which the MORE standard provides dedicated timed metadata. Other types of orchestration involve timed metadata for the region of interest, perceptual quality of media, audio-feature extraction, and media timeline correlation. This paper presents the status of the MORE standard as well as the associated technical and experimental support materials. We also link MORE to the recently initiated MPEG immersive project.