Cooking Video Summarization Guided By Matching with Step-By-Step Recipe Photos

Cooking videos shared on social media enable users to easily understand the cooking process. However, it is difficult to create such summarization videos, as it requires video editing expertise to create high-quality content. In this paper we propose a semi-automatic system for summarizing cooking videos from a full video of the cooking process and a number of step-by-step recipe images. The system is designed to allow anyone to create shareable cooking videos. We focus on online recipe websites because their recipe images provide good descriptions of the food preparation procedures. Our system first searches for scenes similar to each recipe image using features of an Inception Net, fine-tuned to recipe images. These matches are used as key frame candidates, from which users can then select their favorite scenes. Finally, our system automatically creates a summarized video using editing techniques such as speeding up. User studies showed high satisfaction rates of both editors and viewers, demonstrating the benefits of the proposed system.