Pixar’s RenderMan renderer is used to render all of Pixar’s films and by many film studios to render visual effects for live-action movies. RenderMan started as a scanline renderer based on the Reyes algorithm, and it was extended over the years with ray tracing and several global illumination algorithms. This article describes the modern version of RenderMan, a new architecture for an extensible and programmable path tracer with many features that are essential to handle the fiercely complex scenes in movie production. Users can write their own materials using a bxdf interface and their own light transport algorithms using an integrator interface—or they can use the materials and light transport algorithms provided with RenderMan. Complex geometry and textures are handled with efficient multi-resolution representations, with resolution chosen using path differentials. We trace rays and shade ray hit points in medium-sized groups, which provides the benefits of SIMD execution without excessive memory overhead or data streaming. The path-tracing architecture handles surface, subsurface, and volume scattering. We show examples of the use of path tracing, bidirectional path tracing, VCM, and UPBP light transport algorithms. We also describe our progressive rendering for interactive use and our adaptation of denoising techniques.

We present a multiresolution radiosity caching method that allows global illumination to be computed efficiently in a single pass in complex CG movie production scenes. For distribution ray tracing in production scenes, the bottleneck is the time spent evaluating complex shaders at the ray hit points. We speed up this shader evaluation time for global illumination by separating out the view-independent component and caching its result — the radiosity. Our cache contains three resolutions of the radiosity; the resolution used for a given ray is selected using the ray’s differential. The resulting single-pass global illumination method is fast and flexible enough to be used in movie production, both for interactive lighting design and final rendering. It is currently being used in production at several studios. The multiresolution cache is also used to store shader opacity results for faster ray-traced shadows, ambient occlusion and volume extinction, and to store irradiance for efficient ray-traced subsurfac e scattering.

We describe a multiresolution radiosity caching method that enables efficient computation of global illumination (GI) in a single pass in complex CG movie production scenes.

One of the primary goals of computer graphics has been the creation of interactive photorealistic imagery. Unfortunately, the dual objectives of interactivity and photorealism are at odds with each other. Image-based rendering provides a method whereby an offline, computationally intensive image synthesis or computer vision algorithm can be paired with an online, interactive image synthesis algorithm to produce real-time renderings of complex scenes. This dissertation presents three novel view-dependent image-based representations that can be used to accelerate rendering of complex, naturalistic scenes. Hierarchical Image Caching automatically and dynamically caches parts of a scene into images that are rendered in place of geometry, at a much lower computational cost. Layered Depth Images efficiently render complex scenes in software using an image-order warp of pixels with associated depth. Storing multiple depth pixels along each ray of an image, a scene is sampled into a sparse, compact data structure in a view-dependent fashion. Tiling Layered Depth Images uses Layered Depth Images as a basic modeling primitive to create expansive renderings of richly textured terrains. A novel stochastic tiling algorithm is given that guarantees non-periodic tilings of the plane while using a small set of square tiles and a simple construction procedure. Lastly, all image-based representations can be cast as approximations to the plenoptic function, a seven dimensional function describing all of the light in a scene. A novel classification is presented that relates image-based representations, such as the three outlined above, through four common techniques used to reduce the complexity of the plenoptic function.

The full plenoptic function describes all of the light passing through some volume of space. This information has at least seven dimensions: a ray of light requires five numbers to locate it (a point in space and a direction), plus wavelength and time. Given a digital representation of the plenoptic function, we can reconstruct arbitrary views of a scene. Unfortunately, adequately sampling the seven dimensions of the full plenoptic function requires an unmanageable amount of storage. Even if we assume we are concerned with a snapshot in time and that light is monochromatic, we are still faced with the five spatial dimensions of a ray. At the other end of the spectrum, if we consider only a single point in space then we can reconstruct just one view (a 2D photograph). In between the 2D photograph and 5D plenoptic function are many possibilities that provide more freedom than a single image, but less than the full freedom and consequent complexity of capturing and manipulating a 5D object. Recently, there has been considerable research on this topic, typically under the name image-based modeling and rendering (IBMR). We review and classify this work based on the structure of the representation used to approximate the 5D plenoptic function.

The full plenoptic function describes all of the light passing through some volume of space. This information has at least seven dimensions: a ray of light requires five numbers to locate it (a point in space and a direction), plus wavelength and time. Given a digital representation of the plenoptic function, we can reconstruct arbitrary views of a scene. Unfortunately, adequately sampling the seven dimensions of the full plenoptic function requires an unmanageable amount of storage. Even if we assume we are concerned with a snapshot in time and that light is monochromatic, we are still faced with the five spatial dimensions of a ray. At the other end of the spectrum, if we consider only a single point in space then we can reconstruct just one view (a 2D photograph). In between the 2D photograph and 5D plenoptic function are many possibilities that provide more freedom than a single image, but less than the full freedom and consequent complexity of capturing and manipulating a 5D object. Recently, there has been considerable research on this topic, typically under the name image-based modeling and rendering (IBMR). We review and classify this work based on the structure of the representation used to approximate the 5D plenoptic function.

We present a system for modeling and real-time rendering of solid terrains. Using results from the field of tiling, we show how a global 3D texture can be computed for a 2D terrain using a small set of Layered Depth Images. We propose a new set of Wang tiles that are shown empirically to tile the plane without apparent periodic structure. Furthmore, we introduce a new imagebased data struture: multiresolution view-dependent Layered Depth Images. As our results show, this method produces natural looking 3D textures, with full parallax, in real time.

We describe a hardware and software system for digitizing the shape and color of large fragile objects under non-laboratory conditions. Our system employs laser triangulation rangefinders, laser time-of-flight rangefinders, digital still cameras, and a suite of software for acquiring, aligning, merging, and viewing scanned data. As a demonstration of this system, we digitized 10 statues by Michelangelo, including the well-known figure of David, two building interiors, and all 1,163 extant fragments of the Forma Urbis Romae, a giant marble map of ancient Rome. Our largest single dataset is of the David - 2 billion polygons and 7,000 color images. In this paper, we discuss the challenges we faced in building this system, the solutions we employed, and the lessons we learned. We focus in particular on the unusual design of our laser triangulation scanner and on the algorithms and software we developed for handling very large scanned models.

In this paper we present a set of efficient image based rendering methods capable of rendering multiple frames per second on a PC. The first method warps Sprites with Depth representing smooth surfaces without the gaps found in other techniques. A second method for more general scenes performs warping from an intermediate representation called a Layered Depth Image (LDI). An LDI is a view of the scene from a single input camera view, but with multiple pixels along each line of sight. The size of the representation grows only linearly with the observed depth complexity in the scene. Moreover, because the LDI data are represented in a single image coordinate system, McMillan’s warp ordering algorithm can be successfully adapted. As a result, pixels are drawn in the output image in back-to-front order. No z-buffer is required, so alphacompositing can be done efficiently without depth sorting. This makes splatting an efficient solution to the resampling problem.

We present a new method that utilizes path coherence to accelerate walkthroughs of geometrically complex static scenes. As a preprocessing step, our method constructs a BSP-tree that hierarchically partitions the geometric primitives in the scene. In the course of a walkthrough, images of nodes at various levels of the hierarchy are cached for reuse in subsequent frames. A cached image is reused by texture-mapping it onto a single quadrilateral that is drawn instead of the geometry contained in the corresponding node. Visual artifacts are kept under control by using an error metric that quantifies the discrepancy between the appearance of the geometry contained in a node and the cached image. The new method is shown to achieve speedups of an order of magnitude for walkthroughs of a complex outdoor scene, with little or no loss in rendering quality. CR

A prototype visualization management system is described which merges the capabilities of a database management system with any number of existing visualization packages such as AVS or IDL. The prototype uses the Postgres database management system to store and access Earth science data through a simple graphical browser. Data located in the database is visualized by automatically invoking a desired visualization package and downloading an appropriate script or program. The central idea underlying the system is that information on how to visualize a dataset is stored in the database with the dataset itself. As a result, scientists can concentrate more on their science rather than on the process of doing it since visualization programs do not have to be created or searched for each time a dataset is to be viewed.