The full plenoptic function describes all of the light passing through some volume of space. This information has at least seven dimensions: a ray of light requires five numbers to locate it (a point in space and a direction), plus wavelength and time. Given a digital representation of the plenoptic function, we can reconstruct arbitrary views of a scene. Unfortunately, adequately sampling the seven dimensions of the full plenoptic function requires an unmanageable amount of storage. Even if we assume we are concerned with a snapshot in time and that light is monochromatic, we are still faced with the five spatial dimensions of a ray. At the other end of the spectrum, if we consider only a single point in space then we can reconstruct just one view (a 2D photograph). In between the 2D photograph and 5D plenoptic function are many possibilities that provide more freedom than a single image, but less than the full freedom and consequent complexity of capturing and manipulating a 5D object. Recently, there has been considerable research on this topic, typically under the name image-based modeling and rendering (IBMR). We review and classify this work based on the structure of the representation used to approximate the 5D plenoptic function.