Video Calling with Augmented Reality Using WebRTC API

The world is advancing with Augmented Reality (AR) and Virtual Reality (VR), along with this connecting people all over the world has taken a hike growth using various mobile applications for text messaging, voice and video calling. Augmented Reality, Virtual Reality and Mixed Reality has become a piece of everyday life with the development of technologies like Computer Vision, Machine Learning, Artificial Intelligence, mobile computing, sensing technologies, graphics, etc. This paper proposes an approach of incorporating these XR technology in communication for people by augmenting the person talking to other person in his real environment through machine learning algorithm by detecting and scanning the person removing the background using semantic segmentation which will then be augmented in another person’s living room using their mobile devices and the communication will take place using open source WebRTC APIs. Using the Semantic Segmentation, a Deep Learning technique, which will be used to separate the foreground from the background and perform the additional framing of window for smooth and beautiful experience while augmenting it to another person’s environment. This user experience of Augmented Reality in Video Calling can become a new standard for communication and will make another level of digital interaction with others across the world.