Temporal Consistent Semantic Video Color Transfer from Multiple References

Transferring the color from aesthetically high-quality reference color content to captured unpleasant color content is required for the media and entertainment industry. The expert color artists manually change and edit the color tone of unpleasant color content so that it becomes aesthetically pleasing and matches with the other scenes of the main content. Inspired by the style transfer works, the photorealistic color transfer approaches aim to transfer color and brightness from the reference or style video to the main content video. However, those approaches face significant challenges due to induced color artifacts in the final output, computationally expensive, and lacking semantic correspondence. In this work, we propose a temporally consistent semantic video color transfer approach that not only overcomes existing limitations of the color transfer approaches but provides flexibility to the colorist while performing color grading in studios. The temporal inconsistency due to temporally inconsistent semantic information incorporation is handled by an online training approach to make the output temporally consistent. A quantitative comparison shows the effectiveness of our approach as compared to existing solutions. We also perform extensive subjective analysis to showcase the shortcomings of existing solutions and how our solution addresses this.