Patent Application
20120256948
As consumers have become increasingly comfortable with online shopping, many retailers of products offer a retail presence to take advantage of the ecommerce marketplace. Some online retailers offer products that can be customized or personalized based on user-selected choices or inputs, and/or customer-specific information. For example, the www.vistaprint.com web site offers printed, engraved, and embroidered products that can be customized by the customer to include text and images selected and/or uploaded by the customer. For such online retailers, many of the images on the web site and on marketing materials are devoted to showing content on products, and products in context.
For example, a preview of a customer's selected design personalized with information entered by the customer may be presented to a customer selecting customizations and/or personalizing it with user-entered text and/or uploaded images. Besides merely showing the design imprinted, engraved, or embroidered on the product, a good preview might also show the product in context, for example within a larger scene. Previews of the customized products assist the customer in determining where the content is going to be placed, how large the product is, and/or how the product might fit their needs.
Contextual scenes can be created as composite images, for example using Adobe® Photoshop. Photoshop can be used to layer images on top of one another, rotate, warp, and blend images. However, when the composite image is saved using Photoshop, it is saved as a static image and cannot accept dynamically generated content. Online retailers who wish to show images with dynamically generated content, for example for showing images of products personalized with customer information, need to be able to generate customized images and place them within a larger scene on the fly without significant delay in order to prevent or reduce customer drop-off during the browsing process.
In the past, in order to generate previews in context, each context image was implemented as a separate class and had its own unique and static way of drawing itself. Each context image is independently coded in a browser-renderable language (such as HTML, DHTML, etc.), and then dynamically-generated content is rendered by the browser together with the context image. Generating browser-renderable context images in this way requires significant coding time.
Accordingly, it would be desirable to have a better technique for quickly generating dynamically-generated content within contextual scenes.
Embodiments of the present invention includes systems and methods for generating and using a flexible scene framework to render dynamically-generated content within contextual scenes.
In an embodiment, a method for generating scenes with dynamically-generated content for display includes providing to the scene framework engine one or more injectables to be rendered in a composite scene, providing to a scene framework engine one or more scene description files according to the scene-rendering language, the scene description file identifying one or more resources and describing the layering, blending, and specific image manipulations that should be applied to one or more of the injectables or resources when injecting the injectables into the resources, wherein the scene framework engine is configured to layer and manipulate the one or more resources and/or the one or more injectables as described in the one or more scene description files.
In another embodiment, a system for generating and using a flexible scene framework to render dynamically-generated content within contextual scenes is provided.
Embodiments of the present invention utilize a novel scene framework to render dynamically-generated content within contextual scenes.
The scene framework 220 receives or obtains scene rendering code 222, one or more scene image(s) 224, and one or more image(s)/text/document(s) (hereinafter called “injectable(s)”) 226 to place within a generated scene. The scene framework 220 generates an image 228 containing the injectable(s) 224 composited into the received scene(s) 224 according to the scene rendering code 222. The scene rendering code 222 is implemented using an intuitive language (for example, in an XML format), and specifies the warping and compositing functionality to be performed on the injectable(s) 226 (and possibly the scene(s) 224) when generating the composite image 228. A rendering engine 230 receives the composite image 228 and renders it in a user's browser.
The scene framework 220 is a graphical composition framework that allows injection of documents, images, text, logos, uploads, etc., into a scene (which may be generated by layering one or more images). All layers of the composite image may be independently warped, and additional layering, coloring, transparency, and other inter-layer functions are provided. The scene framework 220 includes an engine which executes, interprets, consumes, or otherwise processes the scene rendering code 222 using the specified scene(s) 222 and injectable(s) 224.
At a high level, the Framework 220 is a scene rendering technology for showing customized products in context. A generated preview of the customized product itself may be transformed in various ways, and placed inside a larger scene. Examples of such generated previews implemented in contextual scenes are illustrated in
Scenes can be chained or cascaded, so that one scene can be part of another scene and so forth. A scene may incorporate more than one placeholder location for an injectable scene element such as the business card in each of the composite scenes in
In embodiment of the present invention, this is achieved by decorating rendered Previews with additional image assets. Previously, generating scenes incorporating Previews involved substantial development effort. This process has been vastly simplified thanks to the two key components of the scene framework:
Turning first to the Image Warping and Compositing Engine 210, this component performs the image transformations and compositing. Image warping and compositing are two ways to assemble new images from existing ones. Historically, they have been achieved using a variety of techniques which yield inconsistent results. Furthermore, the ad hoc nature of these techniques added unnecessary complexity to the code. The novel warping and compositing framework provides image warping and compositing functionality to render scenes with dynamically injected content.
Image warping is the act of taking a source image and moving its pixels onto a target image. A number of typical image operations can be described in terms of image warping. For instance, a simple scaling operation (e.g., reducing a large photo to a thumbnail) is an image warp. More sophisticated warps may involve nonlinear effects such as wrapping an image around a cylinder or sphere.
The Image Warping And Compositing Engine 210 performs image warping and transformations. In an embodiment, the Image Warping And Compositing Engine 210 provides a class to perform warping, herein referred to as the “Warper” class. The Warper class includes a static method Apply(Bitmap target, Bitmap source, IWarp warp). This method takes two bitmaps and an “IWarp” object which specifies the warp itself.
In one embodiment, the Warper class implements inverse warping with bilinear sampling. The Warper iterates over each pixel in the target image, figures out the location in the source image it should come from, and copies the pixel color over. If the location happens to be between pixels in the source image (as is often the case) it will linearly interpolate the colors of the neighboring pixels to get the result.
There are various types of warps. The simplest warp is known as the perspective warp (implemented as PerspectiveWarp). The PerspectiveWarp allows the user to move the corners of an image and warp the image accordingly.
Another type of warp is the “smooth” warp. The smooth warp is the most general type of warp. It is meant for cases which defy simple mathematical definition. For example, suppose we want to warp the logo 402 onto a scene 403 of a slightly curved sticky note, as shown in
Notice that the texFeatures are specified in normalized texture coordinates: [0,0] corresponds to the upper left and [1,1] corresponds to the lower right. The imgFeatures are given as standard pixel coordinates. The warp is defined as:
var warp=new Smooth Warp(imgFeatures, texFeatures);
It is possible to simulate other types of warps using a smooth warp given enough point correspondences. However, using the appropriate type of warp when available (e.g., perspective or cylinder) will typically yield better results with less user input.
All of the aforementioned warps implement the IWarp interface. The singular goal of an IWarp is to provide, for any rectangle in the target image, a corresponding set of texture coordinates in the source image to sample color information using bilinear interpolation. To implement a new warp, see the source code for examples (PerspectiveWarp is the simplest).
The Image Warping and Compositing Engine 210 also performs image compositing. Image compositing is the act of combining multiple images into a single image. The Image Warping and Compositing Engine 210 provides similar compositing functionality to common image manipulation switch, such as Adobe® Photoshop. For example, the following layering functionality is supported. Compositor duplicates these layer blending modes: Add, Darken, Difference, Exclusion, Lighten, Multiply, Normal, Overlay, Screen, Subtract.
Turning now to the Scene Framework 220, the scene rendering code adheres to a predefined format using a predefined scene-rendering language. In an embodiment, the scene rendering language utilizes an intuitive HTML- or XML-like language format that allows a user to specify image warping and compositing functions to describe how the image(s) are to be composited. In an embodiment, the Framework 220 utilizes an easy-to-understand XML notation for expressing how image elements should be composited to create the visually convincing renderings. The notation is simple enough that a creative designer can put together a sandwich that layers together imagery, documents, and transformation.
In an embodiment, scenes 224 are XML documents that reside in a web tree along with their corresponding image resources. A basic scene might consist of three scene files.
The scene-rendering code 222 is preferably an XML file implemented using the scene-rendering language and describes how these image resources are combined with a document (i.e., an injectable) to create the composite scene image 228. In an embodiment, configurable scenes have two sections: a <Warps> section that defines geometric transformations (as described in more detail below), and a <Composite> section that defines how to assemble the document itself and other images.
Below is an example scene file:
The simplest scene 224 is an image (i.e., “image.jpg”) itself.
All elements have width and heights defined.
Scenes allow users to composite them as follows:
This scene combines a scene image “image.jpg” with an injectable “Document”. In this example, a depth attribute has been added to the primitives to define layer ordering. Smaller depths indicate “closer” layers, so in this example the image “image.jpg” is “behind” the document “Document”.
Composites can also be nested. An internal composite is assembled and then treated exactly like it is an image. This means that any internal depth parameters are ignored when assembling the parent composite.
In the above example, the nested composite is treated as any other 100-by-100 image and is assembled with depth 50.
Warping is defined as any operation that changes the geometry of the image. It can range from a simple resizing operation to a highly complex and nonlinear deformation. Each warp is identified by a name and specifies an output width and height.
As shown above, the rectangle warp requires the user to specify the desired placement of the lower-left (0,0) and upper-right and upper-right (1,1) corners of the source image. It simply places the source image, whatever size it may be, as a 10-by-10 icon in the lower-left corner of the 100-by-100 target canvas (leaving all other pixels transparent). The exact same effect can be achieved using a perspective warp.
In contrast to the rectangle warp, the perspective warp requires the specification of all four corners of the source image. The above example is identical to a rectangle warp. More generally, a perspective warp allows users to “tilt the image away from the camera”.
In the above example, the document in the Composite now references the perspective warp by name “PerspectiveWarp”. The reference makes it unnecessary to define the width and height of the document. Instead, the width and height comes from the warp. As before, the sizes must be consistent (e.g., the warp can't have a different size as the composite) or it will result in a failure. In general, warps can be applied to both the document and image primitives as well as on nested composites.
The smooth warp follows the same template as the perspective warp but allows for more general deformations.
Notice that this looks exactly the same as the perspective warp, except it also specifies the desired location of the source image center (0.5,0.5). This smooth warp allows an arbitrary number of mappings and, unlike the perspective warp, does not require the corners to be specified.
In general, the warp=attribute may be applied wherever width=and height=are used, except for the top level <Scene>, and so long as all sizes are consistent.
To extend the capabilities of composites, scenes also allow several blending modes: Add, Darken, Difference, Exclusion, Lighten, Multiply, Normal, Overlay, Screen, Subtract. These are applied from background to foreground: the bottom/deepest layer/primitive is composited with the layer/primitive immediately above it, and the process is repeated until the image is flat. Blending modes in nested composites are not visible from the parent composite.
The Scene Framework 220 also supports a Mask mode, as in the following example:
The Mask mode applies the alpha channel of the image to the layers below it (while ignoring the color channels). Notice that the above example applies the mask in a nested composite. This is to avoid also masking the background image (again, since blending modes are not passed through).
The composition tree is successively flattened at the composite elements (in one embodiment, in a depth first manner) (step 506). Each element is ordered and merged with the other elements, as illustrated in
A set of injectables (e.g., document, upload, logo, etc.) is received by the Scene Framework 220 (step 508). The injectable(s) are placed in corresponding “IReplaceableImageContainer” (step 510).
In an embodiment, the scene rendering code 222 is styled within a predefined scene-rendering code template, such as the following:
Computer 810 typically includes a variety of computer readable media. Computer readable media can be any available media that can be accessed by computer 810 and includes both volatile and nonvolatile media, removable and non-removable media. By way of example, and not limitation, computer readable media may comprise computer storage media and communication media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CDROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by computer 810. Computer storage media typically embodies computer readable instructions, data structures, program modules or other data.
The system memory 830 includes computer storage media in the form of volatile and/or nonvolatile memory such as read only memory (ROM) 831 and random access memory (RAM) 832. A basic input/output system 833 (BIOS), containing the basic routines that help to transfer information between elements within computer 810, such as during start-up, is typically stored in ROM 831. RAM 832 typically contains data and/or program modules that are immediately accessible to and/or presently being operated on by processing unit 820. By way of example, and not limitation,
The computer 810 may also include other removable/non-removable, volatile/nonvolatile computer storage media. By way of example only,
The drives and their associated computer storage media discussed above and illustrated in
The computer 810 may operate in a networked environment using logical connections to one or more remote computers, such as a remote computer 880. The remote computer 880 may be a personal computer, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above relative to the computer 810, although only a memory storage device 881 has been illustrated in
When used in a LAN networking environment, the computer 810 is connected to the LAN 871 through a network interface or adapter 870. When used in a WAN networking environment, the computer 810 typically includes a modem 872 or other means for establishing communications over the WAN 873, such as the Internet. The modem 872, which may be internal or external, may be connected to the system bus 821 via the user input interface 860, or other appropriate mechanism. In a networked environment, program modules depicted relative to the computer 810, or portions thereof, may be stored in the remote memory storage device. By way of example, and not limitation,
