Compositing two-dimensional and 3-dimensional images

Details Compositing two-dimensional and 3-dimensional images Compositing two-dimensional and 3-dimensional images

2001-10-01

2004-10-26

Brier, Jeffery (Department: 2672)

Computer graphics processing and selective visual display system

Computer graphics processing

Graphic manipulation

Reexamination Certificate

active

06809745

ABSTRACT:

BACKGROUND
This invention relates to compositing two-dimensional (2D) and three-dimensional (3D) images in graphical image manipulation systems.
A graphical image manipulation application program, such as Photoshop® or After Effects®, from Adobe Systems Incorporated, of San Jose, Calif., can store a graphical image as a set of image layers. A final image can be generated by compositing the image layers together. The image layers can be thought of as stacked transparency sheets. The density of ink on a transparency sheet controls the extent to which the sheet obscures underlying sheets. In the computer program, the color and density of the ink on the transparency sheet are represented by a color value and an opacity (or “alpha”) value, respectively.
A conventional graphical image document includes a set of image layers, commonly denoted as layers
1
,
2
, . . . , n, that are organized in a layer stack. The bottom layer, that is, layer
1
, acts as the background or bottom sheet, whereas the other layers, i.e., layers
2
,
3
, . . . , n, act as the transparency sheets that are overlaid on the background. Examples of different types of layers that may be included in a composite image include image layers, track mattes and adjustment layers.
An image layer typically includes an image that is represented by an array of pixels, each pixel having a color and, optionally, an opacity. The layers can either be two-dimensional (2D) or three-dimensional (3D). A 2D image layer does not have any depth, and can only be moved along the horizontal (X) and vertical (Y) directions and rotated around the (Z) axis. A 3D image layer, on the other hand, has an associated depth coordinate (Z), defining its position along a depth axis. A 3D image layer can typically be manipulated in 3D space through rotation, translation and scaling operations, which allows the 3D layers to display more realistic depictions of how they reflect light, cast shadows, and move through space. 2D and 3D layers are typically differentiated by a tag that identifies the type of layer. Based on the type of layer, the application typically allows a user to perform a different set of operations on the layers.
A track matte is a layer whose alpha channel, that is, opacity, at each pixel is used to specify the transparency of another layer, or of the track matte itself. Track mattes can be animated over time. Like image layers, track mattes can be either 2D or 3D layers and can be treated in corresponding ways including rotation, translation, and scaling.
An adjustment layer is a layer in which effects can be stored and applied to multiple layers. The adjustment layer itself is not visible, but its effects are visible. Any effects applied to an adjustment layer affect all layers below the adjustment layer in the composition stacking order. Thus, placing an adjustment layer at the bottom of the stacking order does not have any effect.
The process of stacking layers to form a final image is known as compositing. A list of layers that are to be composited together is known as a composition. In a 2D environment, the layers are composited in order from bottom to top. Since different layers may have different transparency values, the stacking order of the layers may be important. A composite image with layers ordered from 1 to n, will often look different from a composite image in which the layers are ordered from n to 1. In a 3D environment, the order is determined by the geometry in that the 3D layer that is the “farthest away” from the viewpoint gets rendered first. In other words, what is important in both situations is the order in which the layers are processed during the compositing process.
Conventionally, the compositing process begins by calculating any dynamic data in the image layer, such as the color of the pixels in the image. Then the opacity of each pixel is determined. Finally, the color of each pixel in the image layer is combined with the composited color of the corresponding pixel in an output buffer where the final image is stored to generate a new composited color. This process is iterated for each layer until all the layers have been composited, thus generating the final image.
In many situations a user wishes to combine 2D and 3D objects in a scene. Conventionally, there are three ways this can be done. First, the 2D element (that is, the 2D layer) can be used as a background image. The background image is applied to the output buffer first and any 3D objects are composited in front of the background image. Second, the 2D image can be treated as a texture map, that is, a surface with a particular structure. The texture map is “glued” onto a portion of a plane in a 3D layer and the plane can then be manipulated in 3D space. Third, the 3D layers can be composited together into a temporary image buffer, which is then composited with one image buffer that contains 2D elements.
In all these methods, the 3D images are processed separately from the 2D images, in that the 2D images either are pre-processed, post-processed, or treated as if they were 3D geometry. From a workflow point of view, this is cumbersome for a user. A typical task for a user is to place titles or other types of text (that is, a 2D element) in a 3D scene. The user may wish to place the title in front of a 3D image, behind a 3D image, or between two 3D images. The pre-processing method (i.e. the first method) presented above does not allow 2D images to be placed in front of a 3D images. The post-processing method (that is, the third method) requires that the 2D image always be placed in front of the 3D image. The second method, in which the 2D images are treated as texture maps on 3D images, may work but presents difficulties in terms of aligning the plane on which the 2D image is placed to the 3D camera (that is, the projection of the composite image that is displayed to a user) and correctly scaling the geometry in order to achieve the desired image size. Furthermore, none of the methods presented above allows mixing track mattes or adjustment layers with image layers of a different dimensionality, such as mixing a 3D adjustment layer with a 2D image layer.
SUMMARY
In general, in one aspect, this invention provides methods and apparatus, including computer program products, implementing and using techniques for generating a composite image. Electronic data representing at least three image layers to be combined in a composite image is provided. At least one of the image layers has two-dimensional content and at least one of the image layers has three-dimensional content. The layers are arranged in a sequence represented by a composition sequence list. The at least three image layers are grouped into at least three metalayers, each of the metalayers including one or more image layers having content having a common dimensionality such that each image layer that is adjacent to an image layer having a common dimensionality is grouped in the same metalayer as that image layer. The metalayers are arranged in a metalayer sequence. A metalayer is selected according to the metalayer sequence and the image layers in the selected metalayer are processed to generate a raster representation of the image layers in the selected metalayer. The raster representation is composited into an image buffer and the process is repeated until the raster representation of each of the metalayers has been composited into the image buffer to form a composite image.
Advantageous implementations can include one or more of the following features.
Grouping the image layers can include sorting adjacent image layers with two-dimensional content into a two-dimensional bin and adjacent image layers with three-dimensional content into a three-dimensional bin. Visual parameters for the three-dimensional image layers can be stored in a global visual parameter list. Processing the image layers can include selectively applying the stored visual parameters to the image layers in the metalayer. The visual parameters can apply to all image layers that have already been processed according

Affiliated with

Also associated with

Rating

Compositing two-dimensional and 3-dimensional images does not yet have a rating. At this time, there are no reviews or comments for this patent.

If you have personal experience with Compositing two-dimensional and 3-dimensional images, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Compositing two-dimensional and 3-dimensional images will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFUS-PAI-O-3297928