Patent Application
20070035553
Not Applicable.
Not Applicable.
Digitally represented still images are commonly used in the computer environment as graphics for applications software, games, and as digitally stored photographs that can be easily manipulated, printed and transmitted for commercial and entertainment purposes. As such, it is often necessary to be able to manipulate and easily modify such images, or portions thereof, through various editing capabilities, including the ability to rotate an image, crop it, and correct its brightness, contrast, and tint. Additional editing capabilities include the ability to cut portions out of an image, incorporate images into a collage, and many other special effects known to those of ordinary skill in the art.
It is through an imaging engine that the editing and reformatting of digital images is processed. The images are broken down to the individual pixel level for purposes such as those described above. In a demand driven imaging engine, image processing effects request inputs to render a region of pixels for proper alignment with the desired output. Each of these regions is called a requested rectangle. Typically input and output regions are identical in size and the rendering from input to output is 1:1. On occasion, however, image processing effects may modify these regions by, for instance, enlarging the regions, trimming the regions, or merging multiple input regions, in order to render the image in accordance with the desired output. The effects may also return a buffer of different size than requested. This can occur when the return buffer has a lot of data in areas beyond the image bounds, that is, data in the ‘abyss’. In this situation, an effect can trim the region to reduce the number of abyss pixels.
Image processing effects are charged with the proper handling of input and output alignment. However, depending on the desired form of the output, the input regions do not always line up with the desired output region coordinates. In these situations it is necessary to be able to properly align the input and output regions such that the desired image is obtained from the imaging engine.
Embodiments of the present invention relate to texture alignment between input and output regions in image processing. Each input texture includes a bounding rectangle that defines the coordinates of the texture. These coordinates define where each texture is to be placed during image processing. “Texture alignment”, as the term is utilized herein, is a process for ensuring that all of the input textures are correctly placed according to their coordinates before image processing. When textures are properly aligned, they will appear visually similar in both the input and output regions. Texture alignment in accordance with embodiments of the present invention involves the use of a buffer between the input region and the output region. Texture coordinates derived from the input region are mapped to vertices of the buffer and such mappings are provided to a processing unit for processing. Once processed, the output region will include a texture, that is a result of the image processing effects applied on the textures of the input region that are correctly aligned with one another. Various embodiments of the buffer are described herein, selection of which may depend, for example, on the desired resolution level of the output and the quantity of inputs to the buffer.
This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.
The present invention is described in detail below with reference to the attached drawing figures, wherein:
The subject matter of the present invention is described with specificity herein to meet statutory requirements. However, the description itself is not intended to limit the scope of this patent. Rather, the inventors have contemplated that the claimed subject matter might also be embodied in other ways, to include different steps or combinations of steps similar to the ones described in this document, in conjunction with other present or future technologies. Moreover, although the terms “step” and/or “block” may be used herein to connote different elements of methods employed, the terms should not be interpreted as implying any particular order among or between various steps herein disclosed unless and except when the order of individual steps is explicitly described.
Embodiments of the present invention provide a method in image processing for aligning a texture from at least one input region to an output region. In one embodiment, the method includes receiving information pertaining to the at least one input region, the information including a texture and corresponding texture coordinates, receiving information pertaining to the output region, utilizing the information pertaining to the input region and the information pertaining to the output region to create a buffer, the buffer having a plurality of vertices, and mapping each of the texture coordinates of the input region to a vertex of the buffer.
Embodiments of the present invention further provide methods in image processing for translating texture coordinates from at least one input region to a buffer associated with an output region for the purpose of aligning textures. In one embodiment, the method includes providing at least one set of u and v coordinates, each coordinate ranging between 0 and +1, and providing at least one set of x and y buffer coordinates, each coordinate typically ranging between −1 and +1, wherein the at least one set of buffer coordinates is associated with a vertex (Vi) of the buffer. A method in accordance with this embodiment further includes mapping the at least one set of texture coordinates to the at least one set of buffer coordinates for the vertex (Vi) as xi, yi, ui, vi.
Computer readable media having computer-useable instructions for performing the methods described herein, as well as computers programmed to perform the described methods, are also provided.
The present invention further provides systems in image processing for aligning a texture from an input region to an output region. In one embodiment, the system includes an input receiving component for receiving at least one input region, the at least one input region having a texture and corresponding texture coordinates, an output receiving component for receiving an output region, and a buffer creating component for creating a buffer having a plurality of vertices each of which is capable of having the texture coordinates of the at least one input region mapped thereto.
Having briefly described an overview of the present invention, an exemplary operating environment for the present invention is described below.
Exemplary Operating Environment
Referring to the drawings in general, and initially to
The invention may be described in the general context of computer code or machine-useable instructions, including computer-executable instructions such as program modules, being executed by a computer or other machine, e.g., a personal data assistant or other handheld device. Generally, program modules including routines, programs, objects, components, data structures, and the like, refer to code that performs particular tasks or implements particular abstract data types. The invention may be practiced in a variety of system configurations, including hand-held devices, consumer electronics, general-purpose computers, more specialty computing devices, etc. The invention may also be practiced in distributed computing environments where tasks are performed by remote-processing devices that are linked through a communications network.
Computing device 100 includes a bus 110 that directly or indirectly couples the following devices: memory 112, one or more processors 114, one or more presentation components 116, input/output (I/O) port(s) 118, I/O components 120, and an illustrative power supply 122. Bus 110 represents what may be one or more busses (such as an address bus, data bus, or combination thereof). Although the various blocks of
Computing device 100 typically includes a variety of computer-readable media. By way of example, and not limitation, computer-readable media may comprise Random Access Memory (RAM); Read Only Memory (ROM); Electronically Erasable Programmable Read Only Memory (EEPROM); flash memory or other memory technologies; CDROM, digital versatile disks (DVD) or other optical or holographic media; magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, carrier wave, or any other medium that can be used to encode desired information and be accessed by computing device 100.
Memory 112 includes computer-storage media in the form of volatile and/or nonvolatile memory. The memory may be removable, nonremovable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard drives, optical-disc drives, etc. Computing device 100 includes one or more processors that read data from various entities such as memory 112 or I/O components 120. Presentation component(s) 116 present data indications to a user or other device. Exemplary presentation components include a display device, speaker, printing component, vibrating component, and the like.
I/O port(s) 118 allow computing device 100 to be logically coupled to other devices including I/O components 120, some of which may be built in. Illustrative components include a microphone, joystick, game pad, satellite dish, scanner, printer, wireless device, and the like.
If desired, computing device 100 may further include a dedicated Graphics Processing Unit (GPU) (not shown) to accelerate computer graphic processing.
General Framework for Aligning Textures
As previously mentioned, embodiments of the present invention relate to a method in image processing for aligning a texture from an input region to an output region. For clarity purposes, it is best to identify some of the common terminology that will be discussed in greater detail with respect to embodiments of the present invention. An “input region”, as the term is utilized herein, is a section of image data having a texture and corresponding texture coordinates. A “requested rectangle”, as the term is utilized herein, is a region of image data that a user wants to render. The requested rectangle may be smaller in size than the input region if the user is interested in only rendering a particular portion of interest. A “buffer”, as the term is utilized herein, contains a plurality of vertices and is the location to which input texture coordinates are mapped before being directed to a processing unit. The “output region”, as the term is utilized herein, is the region of image data that will be rendered/generated by a processing unit, such as a graphical processing unit (GPU), for rendering. The output region is sized to include the input region(s), the requested rectangle, and the buffer. The processing unit is preferably a GPU. However, other processing units such as a Central Processing Unit (CPU) may be used as well. All such variations, and any combinations thereof, are contemplated to be within the scope of embodiments of the present invention.
An exemplary method 200 for aligning a texture from an input region to an output region in accordance with an embodiment of the present invention is illustrated in the flow diagram of
Subsequently, utilizing the information pertaining to the input region and the output region, a buffer is created, as indicated at block 203. As more fully described below, the buffer includes a plurality of vertices. Next, each of the texture coordinates from the at least one input region are mapped to a vertex of the buffer via a mapping process that complies with the requirements of the output region. This is indicated at block 204. Note that in certain circumstances, for instance when there is no input to render at a particular location, there will be no input texture coordinates which correspond to certain coordinates of the buffer. This situation occurs where there is no input image to render at that location.
An exemplary method 300 in image processing for aligning the textures of input and output regions having more detail than the flow diagram of
However, if it is determined that there is more than one input region to consider, then information pertaining to each of the plurality of input regions is received, as indicated at block 303. Each input region for which information is received includes a texture and corresponding texture coordinates associated therewith. Subsequently, as indicated at block 304, information pertaining to the output region is received. Utilizing the received information, a buffer is subsequently created, as indicated at block 305, the buffer including a plurality of vertices and sized so as to set the limit as to what input regions (or portions thereof) will be considered for image processing.
Once the buffer has been created, the texture coordinates associated with each of the textures are mapped to a vertex of the buffer. This is indicated at block 306. Subsequently, it is determined whether or not all input texture coordinates have been mapped to the buffer vertices. This is indicated at block 307. If all input coordinates have not been mapped, then the mapping process returns to the step indicated at block 306 and this loop continues until all texture coordinates have been mapped to a buffer vertex.
As indicated at block 308, once all texture coordinates have been mapped to a buffer vertex, it is determined whether or not any of the buffer vertices have multiple texture coordinates mapped thereto. If it is determined that there is only a single set of texture coordinates for a vertex in question, then this data is directed to a processing unit, such as a GPU, which will configure the buffer data into the proper format for the desired output. This is indicated at block 309. If, however, it is determined that a particular vertex has multiple texture coordinates mapped thereto (from plurality of input regions), then these texture coordinates are directed to the processing unit where the plurality of texture coordinates are blended through a pixel shader function, that is, a function that processes input texture pixels so as to render the output pixels. Pixel shader functionality is known to those of ordinary skill in the art and, accordingly, is not further described herein.
Subsequently, a blended texture, or single set of data, is directed to the output, as indicated at block 311.
Various embodiments of the present invention depend upon the quantity of input regions and the configuration of the buffer corresponding thereto. For instance, exemplary buffer configurations are shown in the schematic diagrams of
Note that input region 401 is shown twice in
As previously stated, the input region 401 comprises a texture and a plurality of texture coordinates. The texture coordinates for an input region, including single input region 401, comprise a u coordinate and a v coordinate, with each of the u and v coordinates ranging between 0 and +1. For a single input region 401, only the texture coordinates contained within buffer 402 are mapped to coordinates of the buffer. However, the buffer coordinates have a different coordinate system than the input region coordinates. An x coordinate and a y coordinate correspond to each vertex 403 of the buffer 402, with each of the x and y coordinates typically ranging between −1 and +1. It should be noted that on occasion the −1 to +1 range may be extended to consider input pixels slightly beyond these outer limits in order to properly render the inputs at these outer limits. For example, if it is desired to blur an image, it is generally necessary to compute the average value of neighboring pixels. For pixels at the edge of the output, it is generally necessary to include input pixels slightly beyond the outer limits so as to accurately blur the edge pixels. Therefore, in order to map the u and v texture coordinates of the single input region 401 to the buffer 402, the u and v coordinates are transformed to x and y coordinates for the buffer such that Vertex i=xi, yi, zi, ui1, vi1 where i equals the vertex number, and xi, yi, zi are the vertex coordinates of Vertex i in the buffer. For a single input region 401, ui1, and vi1 correspond to the texture coordinates for the single input image. Where only u and v coordinates are present, i.e., in a two-dimensional image, ui1, and vi1 map only to x and y, respectively, such that zi is not used.
The general mapping of the input region 401 to the vertices 403 of the buffer 402 is shown in
Another exemplary buffer configuration 500 which is shown in
Furthermore, depending on the size of the input regions 501, 502 and the size of the buffer 503, it is possible for input regions 501, 502 to overlap with one another, as shown in
As with the buffer configuration illustrated in
While the buffer configurations 400 and 500 of
For the exemplary buffer configuration 600, the quadrilaterals of the buffer 603 are positioned such that each of the quadrilaterals is either entirely inside or outside of the plurality of input regions 601, 602. As a result of the plurality of quadrilaterals, the buffer 603 has a plurality of vertices 604, with the quantity of vertices 604 being a function of the quantity of quadrilaterals.
As stated previously, each input region 601, 602 comprises a texture and a plurality of texture coordinates. The texture coordinates for each input region 601, 602 comprise a u coordinate and a v coordinate, with each of the u and v coordinates ranging between 0 and +1. For a plurality of input regions 601, 602, only the texture coordinates contained within the buffer 603 are mapped thereto. However, an x coordinate and a y coordinate correspond to each vertex 604 of the buffer 603, with each of the x and y coordinates typically ranging between −1 and +1. Again, as previously mentioned, on occasion the −1 to +1 range may be extended to consider input pixels slightly beyond these outer limits in order to properly render the inputs at these outer limits.
Furthermore, depending on the size of the input regions 601, 602 and the size of buffer 663, it is possible for the input regions 601, 602 to overlap, as shown in
Take, for example, the vertex labeled Point A in
A general mapping of the vertices 604 is shown in
Note that in each of the exemplary buffer configurations illustrated in
Although two input regions are used as an example in
In one embodiment, a scenario can occur in which the input region is larger or smaller than the desired output region and the input must be scaled appropriately. In this situation, a buffer having a plurality of vertices is established to represent the desired output. In this embodiment, a vertex shader in the GPU calculates the required texture coordinate offsets and scaling such that the input is sized appropriately in the buffer for the desired output. Stated differently, the texture mapping computations mentioned above can be done in the GPU if it supports vertex shader. In that case, the rectangular area of each input region/output region, or alternatively the coefficients of coordinate transformation (scale and offset=translation), may be provided to the GPU. Subsequently, the actual calculations may be done in the vertex shader.
The present invention has been described in relation to particular embodiments, which are intended in all respects to be illustrative rather than restrictive. Alternative embodiments will become apparent to those of ordinary skill in the art to which the present invention pertains without departing from its scope.
From the foregoing, it will be seen that this invention is one well adapted to attain all the ends and objects set forth above, together with other advantages which are obvious and inherent to the system and method. It will be understood that certain features and subcombinations are of utility and may be employed without reference to other features and subcombinations. This is contemplated by and is within the scope of the claims.
