光影魔幻工业

In professional video compositing workflows, artists must manually create environmental interactions-such as shadows, reflections, dust, and splashes-between foreground subjects and background layers. Existing video generative models struggle to preserve the input video while adding such effects, and current video inpainting methods either require costly per-frame masks or yield implausible results. We introduce augmented compositing, a new task that synthesizes realistic, semi-transparent environmental effects conditioned on text prompts and input video layers, while preserving the original scene. To address this task, we present Over++, a video effect generation framework that makes no assumptions about camera pose, scene stationarity, or depth supervision. We construct a paired effect dataset tailored for this task and introduce an unpaired augmentation strategy that preserves text-driven editability. Our method also supports optional mask control and keyframe guidance without requiring dense annotations. Despite training on limited data, Over++ produces diverse and realistic environmental effects and outperforms existing baselines in both effect generation and scene preservation.

We begin with an exploration of the evolution from the original 1996 Twister to 2024’s Twisters, highlighting technological advancements and their impact on visual storytelling. The seamless integration of special effects (SFX) and visual effects (VFX) is crucial for achieving realism, and we will discuss the collaboration between these teams to create cohesive, immersive experiences. A key focus is on the elemental analysis of storms, breaking down the visual components that form tornadoes. This includes understanding the recipe and rules of storms, their lifecycle, and their environmental impact. The tools and setups developed to recreate these phenomena will be showcased, illustrating the blend of science and creativity. Photography plays a vital role in capturing the essence of real storms. We will examine the extensive reference photography process, from hi-res storm chaser footage driving into storms to everyday iPhone and GoPro clips, and how these references shape our imagination and design of tornadoes. Finally, the talk will cover the process of transforming tornadoes into characters within the narrative. Each tornado in Twisters is unique in size, shape, and form, following the simple physics of these natural phenomena while pushing creative boundaries. This section will reveal how Industrial Light & Magic (ILM) rigged and animated different tornadoes, making them central figures in the story. This talk offers an in-depth look at the magic that brings nature’s fury to life, inspiring and educating the audience on the potential of visual effects in film to replicate nature.

LED volume virtual production brings its own unique set of challenges that differ from typical applications of real-time rendering. ILM StageCraft is a complete suite of virtual production tools that incorporates powerful off-the-shelf systems, such as Unreal, with proprietary solutions that allow us to overcome these challenges—always with the goal of using the very best tool for the job. To meet the most demanding requirements for The Mandalorian season 2 and beyond, Industrial Light & Magic (ILM) developed and augmented proprietary technology solutions in StageCraft 2.0, including a new real-time rendering engine, Helios, and a collection of interactive tools that give filmmakers the freedom, creative feedback, and real-time control to make immediate adjustments on set. Powered by ILM’s award-winning set of tools focused on rendering, lighting, and color, creatives can achieve their desired looks and goals without having to wait for traditional post-production or DI to take place. Since season 2 of The Mandalorian, ILM has continued to evolve the Helios renderer, expanding what’s possible not only in terms of visual fidelity but also visual adjustability, through a real-time toolset that allows on-set operators to rapidly respond (in a matter of seconds!) to late-breaking changes in creative direction, in ways that make intuitive sense for filmmakers rather than technologists and gamers. This groundbreaking approach saves significant time on set while also providing greater creative control and insights over the final result, allowing for an unprecedented degree of freedom and flexibility to make the right choices on the day. This talk will address why, in today’s landscape, you would even consider writing a set of proprietary tools and renderer. It will cover the various aspects of ILM StageCraft rendering and operations, from the streaming systems that feed the Helios renderer, to the core rendering passes (geometry submission, direct lighting, ray-traced reflections, volumetrics, etc.) that compose a frame, as well as the various controls we provide for on-the-day tweaks to virtual content. Along the way, we will discuss our integrated and seamless color pipeline that provides end-to-end confidence in the quality of the captured results. We will also point out some gotchas with using established real-time rendering techniques in the context of the LED volume and how we overcame them. Finally, we will touch on how ILM StageCraft fits within the larger picture of visual effects production and where we plan to take it in the future.

Atomically precise gold nanoclusters are a fascinating class of nanomaterials that exhibit molecule-like properties and have outstanding photoluminescence (PL). Their ultrasmall size, molecular chemistry, and biocompatibility make them extremely appealing for selective biomolecule labeling in investigations of biological mechanisms at the cellular and anatomical levels. In this work, we report a simple route to incorporate a preformed Au25 nanocluster into a model bovine serum albumin (BSA) protein. A new approach combining small-angle X-ray scattering and molecular modeling provides a clear localization of a single Au25 within the protein to a cysteine residue on the gold nanocluster surface. Attaching Au25 to BSA strikingly modifies the PL properties with enhancement and a redshift in the second near-infrared (NIR-II) window. This study paves the way to conrol the design of selective sensitive probes in biomolecules through a ligand-based strategy to enable the optical detection of biomolecules in a cellular environment by live imaging.

ILM and its partner visual effects companies take you behind the scenes of the second season of Lucasfilm's hit Disney+ series "The Mandalorian." The team will discuss an array of the 5,000 visual effects shots created by the global team from sand dragons to ice spiders and miniatures as well. advancements made in the virtual production arena. The panelists will discuss ILM StageCraft 2.0 as well as Helios, ILM's groundbreaking new real-time render engine implemented for the first time this season.