Image-based rendering describes the way camera motions in photographic scenes. First there are the 6 degrees of freedom. Three degrees result from the three axis for camera rotation, while image and viewpoint are stationary. This degrees can be simply described by an environment map which consists of an image of the whole panorama. While rotating the camera the visual part of the environment changes, but the viewpoint within (distance and direction) stays the same. Second, orbiting a camera around an object, is like changing the viewpoint but keeping the view direction centered. This type of movement is impossible to represent woth an environment map. Third results of the first two parts, called camera movement, requires changes of the viewpoint and the viewing direction and has all 6 degrees of freedom. Last but not least there is also Camera Zooming which can be represented through multiple resolution images. Another dimension is time-varying environments. While in a static environment one map represents the virtual spaces behaviour, in a dynamical scenario multiple environment maps are needed for description. Each of these topics are described next.
Camera Rotation consists of 3 degrees of freedom, the rotation about a horizontal axis (called pitch), the vertical axis (yaw) and the axis of the viewing direction (roll). Rolling can be simply described as image rotation. Pitch and Yaw can be accomplished by the repojection of an environment map. An environment map is a reprojection of an environment onto a simple shape (a cube or sphere). A cube consists of six texture mapped squares, while reprojecting a sphere requires non-linear warping of an image. Other projections are fish-eye, cylindrical or wide-planar map.
An environment map cannot represent the 360° view on an object. This is just possible with a method called the navigable movie approach. For an Object with full 360° rotation and 140° rotation in another direction at 10° distance between images, the object consists of 504 frames. At a resolution of 256x256 pixels using high color, the representation gets to a size of 5MB. Reducing through compression makes no sense because of the real-time constraint and the data-loss and it´s effect on complexity and detaility. Reduction can be reached by interpolation between images produced with a distance of about 20°.
This is the most challenging type of movement in a three-dimensional scene. Changing the viewpoint and the viewing direction simultaniously. Variable viewing direction can be satisfied with an environment map, described earlier. Changing the viewpoint must be described as a digital movement, instead of a free motion. An approximated path results of approximated positions along a three-dimensional grid, so that the viewpoint is represented by the nearest grid-point. Another possibility is to use a three-dimensional grid of environment maps. The environment map for the current viewpoint is interpolated from the nearest environment maps on the grid. This method smoothes the movement.
First of all, getting more data from an image through zooming is impossible, because of the Nyquist limit described in the sampling rate theorem. Each object or panorama has to support a quadtree-like structure to store multiple resolutions. To avoid loading big images like whole environment maps with a high resolution into memory, the image can be segmentated and each segment can be loaded seperately. For higher resolutions the segments are smaller but also the visible part of the whole environment map is smaller. The two inverse factors image resolution and visible part of the environment map grant a constant number of pixels necessary. Higher resolutions can also result from a different image to perform effects like the "infinite zoom".
4. How's Quicktime implemented