Automated Reverse Storyboarding

Storyboarding is a standard method for visual summarization of shots in lm and video preproduction. Reverse storyboarding is the generation of similar visualizations from existing footage. We identify the key attributes of preproduction storyboards then develop computational techniques that extract corresponding features from video, render them appropriately, then composite them into a single storyboard image. The result succinctly represents background composition, foreground object appearance and motion, and camera motion. For tracking shots, we show that the visual representation conveys all the essential elements of shot composition. Visual summaries play an important role in the production and analysis of media. Practitioners, researchers and archivists all demand that the information presented is accurate and described in a consistent form using common metaphors derived from industry nomenclature. The goal is to enable quick access to details of specic shots or sequences without having to view the footage itself. In the media production industry, visual summarization is typically achieved through storyboards. Storyboards are drawn during preproduction then used throughout production and postproduction in tasks like set design, location lighting and image compositing. They provide for all participants a common reference to the ivisioni of the piece. Shorthand descriptions of all important visual components of each shot provide clear and accurate depictions of motion sequences in static form. These include specic methods of describing camera or subject movement through the use of various drawing techniques. While the term storyboard has been applied in the context of automated media analysis to a sequence of consecutive still images extracted from a lm or video programme, the representations traditionally used in the production industry are much richer. Our usage in this paper corresponds to lm production storyboarding: i.e., we seek to describe the temporal evolution of a shot through a single picture using rich visual cues. Storyboards incorporate the following types of information: 1. Composition of the shot, including start, end and notable intermediate camera positions