Patent Grant
8125482
The present application is related to a patent application being filed concurrently entitled Graphical User Interface Text-based Interpolation Searching, by Robert Coyne and Steven Greenspan.
The present invention relates generally to devices for computer animation performance and methods and computer-readable media for implementing the same. More particularly, the present invention relates to using an interactive graphical interface for animation performance.
Computer systems often utilize displays with icons, pictures, text, pop-up, drop-down menus and other graphical items. In these systems, individual graphical items or combinations of graphical items are typically shown on the display monitor. The graphical item is generally activated by a point and click method in which a cursor controlled by a mouse, touch device, pen or trackball is pointed at the item, and a button, keyboard command or activation unit is clicked.
Generally, three dimensional character animations are incrementally specified using highly technical interfaces wherein the user adjusts one parameter at a time and then previews the result. The final animation is an immutable sequence of rendered frames. At the other end of the spectrum of animation are video games wherein the user, in real-time, triggers predefined movements, but is given very little control over the nuances of expression and movement.
Generally, in many graphics implementations, animation may be performed in three dimensions, but typically utilizes a complex process called “rendering” to provide a representation of an image. For example, physical objects such as tubes, spheres, rectangles, and the like may be used to “build” other, more visually complex objects. Such objects may be rotated, stretched, twisted, or the like. However, such modeling generally requires utilizing modifiers, which may include a stack of parametric definitions for each desired geometric subclass, thus requiring a large amount of storage and a complex retrieval system.
Various three dimensional animation systems provide additive displacements as a method of combining partially defined gestures, facial expressions, or morphing targets. Such targets are generally controlled by bands of sliders.
Other related interfaces are the joystick music synthesizers, wherein the joystick is used to control various aspects of the timbre and dynamics of the sound. The joystick is moved from a first position to a second position, and the parameters are dynamically mixed based on the second position of the joystick.
In addition to automation, more generally, searches in databases are achieved by using key words or expressions so that an ordered list may be obtained. However, often the list is quite long, and thus is difficult to sort through to obtain the desired information.
Thus, there is a need for a device and method for a simplified graphical user interface that the user may utilize in an interactive fashion to provide relevance weightings for searches to facilitate re-ordering of search data and to add nuances of expression and movement in animation.
The present invention provides an electronic display system that facilitates the use of a graphical interface for interactive animation on a display device. The invention includes: a central processing unit; a memory loaded with an operating system, application programs and computer-executable instructions for using a graphical interface for achieving the animation; a display unit coupled to the system bus; a cursor control unit arranged to provide signals to control movement of a cursor on the display unit; and the system bus. The computer-executable instructions for achieving use of a graphical interface for animation include: (1) inserting a desired image onto a first window; (2) inserting anchors onto a second window by, for each anchor, selecting a desired pose from a plurality of predetermined poses; and (2) upon a cursor being dragged over the second window to a desired anchor, additively applying, for the desired anchor to the desired image based on a proximity of the cursor to the desired anchor. The characteristics for the anchors are, for animation, typically facial expressions, character poses or camera positions, and the like. The electronic display system is generally implemented as a computer display system. Inserting anchors may include combining a plurality of desired anchors to form a compound anchor, e.g., by dragging one or more anchors on top of another anchor. A palette may be shown in a third window on the display unit and may be used for selecting a desired image to be inserted onto the first window.
The present invention includes a method for facilitating use of an interactive graphical interface for animation on an electronic display system by a user. The steps of the method may include inserting a desired image onto a first window; inserting anchors onto a second window by, for each anchor, selecting a desired pose, from a plurality of preselected poses; and dragging a cursor over the second window to a desired anchor wherein characteristics for the desired anchor are additively applied to the desired image, based on a proximity of the cursor to the desired anchor. Characteristics for the anchors in the method are typically, for animation: facial expressions, character poses, orientation of an image or camera positions and the like. The electronic display system may be implemented by a computer display system. A plurality of desired anchors may be combined to form a compound anchor by dragging an anchor onto another anchor or pile of anchors. Where desired, a palette in a third window displayed on the display unit may be used for selecting a desired image to be inserted onto the first window.
Where desired, the steps of the method of the invention may be implemented by computer-executable instructions in a computer-readable medium.
The accompanying drawings, which are incorporated into and form a part of the specification, illustrate preferred embodiments of the present invention and, together with the description, serve to explain the principles of the invention. The drawings are only for the purpose of illustrating a preferred embodiment of the invention and are not to be construed as limiting the invention. In the drawings, like numbers refer to like parts throughout and mouse clicking for a cursor is shown by a circle:
The present invention provides a graphical user interface which allows the user to interactively change the state of an object or objects. The interface is controlled by a pointing device (mouse or tablet) or any other input device whose output may be translated to an x,y position on a second window.
In automation: The visual changes of the object that occur as the pointer is moved create an animation sequence. Since the user may see the resulting change of the object in real-time as the pointer moves, the process of creating the animation becomes a truly interactive performance. This is much different than the more step-by-step procedures used in traditional animation interfaces.
In a search: The interface of the present invention may also be used interactively to search to find a desired state of the object. Since the user may see the resulting change of the object as he moves the pointer, the search process is guided by a tight feedback loop. In the current art, database queries are delimited through a set of key words or expressions, and the result is displayed as an ordered list of documents that contain the set of keywords or expressions. The documents are ordered by relevance, where relevance is defined algorithmically as a match between the query terms and the frequency and distribution of those terms, or semantic associations of those terms, in the documents. Modifying the query will result in a reordered list. Examples of modifications are specifying different degrees of relevance of the terms, deleting some terms, introducing new terms, and indicating which terms should be close to one. The current art for describing these modifications includes Boolean strings, binary relevance operators (e.g., + or −), checklists, and graphical techniques for inserting or deleting terms. Often a query will return a large set of documents and it is difficult to determine algorithmically the appropriate relevance weightings that fit the needs of the user. In contrast, the present invention provides a novel method for rearranging the query result. Using techniques developed for real-time graphical manipulation of animated figures, a graphical user interface may be used in accordance with the present invention to rearrange the query result. Each node in a graph represents a term in the query. One advantage provided by the present invention is that the nodes may be arbitrarily rearranged in a two dimensional, or simulated three dimensional, display. Thus, terms with no semantic association may be arranged near one another. Positioning a cursor in the display results in a unique ordering of the query result. Notably, the cursor need not be on a node or on the direct path between nodes. If the cursor is on a node, then the term associated with that node receives the strongest relevance weighting. As the cursor is repositioned between nodes, the distance between the cursor and each node (i.e., term) is used to calculate a new set of relevance weightings. Rearranging the graph creates new possible combinations of weights. Positioning the cursor within the graph defines a unique combination of weights. The position of the cursor may be moved within the graph, and each repositioning causes a new list to be displayed.
In its basic configuration, the user interface of the present invention consists of two windows, a display window and a target (controller) window. The target window displays a two dimensional arrangement of target points where each target represents a DISPLACEMENT of the object from some BASE STATE. As the user drags the pointer over the target window, the object is updated and redrawn in the display window. The state change in the object is determined by the proximity of the pointer to the various targets.
The user is allowed to arrange the targets as desired, thus defining the potential search or animation space. Each target has an area of influence which is used to determine (based on the mouse position) how much of that target's displacement is applied to the object. The WEIGHTING CALCULATION may use any arbitrary function, but is typically based on the distance between the pointer and the target. How the target displacements are defined or selected is defined by a preselected external interface or procedure.
The object's state is changed as follows: every time the pointer position changes, the object is first put into its default base state. Then, all the targets are applied to the object based on their weighting, which is calculated from the mouse position. Often, only a small number of targets will be close enough to the mouse to have any influence. The object (“PARAMETER OBJECT”) may then either be directly displayed or its set of parameters and values may be applied to another “DISPLAY OBJECT” which the user sees. The following Lisp pseudo-code shows what happens every time the user moves the pointer:
The OBJECT (or “PARAMETER OBJECT”) is anything that may be represented or encoded by a sequence of properties and their associated numeric values. It is this object which is directly changed by the interface in the manner described in the algorithm.
A STATE is a given set of parameters and values associated with a particular object. So, for example, the state of a television set might be defined by the values for the current channel, contrast, brightness, color and tint. Or the state of a human body, for animation purposes, might be defined by the state of bone rotations around their respective joints. One set of angles may define a “throw” pose and another may define a “crouch” pose.
The DISPLAY OBJECT is the object that the user sees in the display window. The DISPLAY OBJECT potentially may be different from the parameter object. The translation from the parameter object to the display object may be arbitrarily complex. For example, if the parameter object represents various parameters used in a physical simulation (gravity, wind velocity, friction, etc.), those parameters will be applied in the simulated environment, and the simulated environment will be redisplayed. On the other hand, often the parameter object may be displayed directly itself. For example, a human animation skeleton may serve both as a displayable structure and as a container for the various parameters and values that define the different body pose states.
The BASE STATE of an object is a default state that is used as a reference to save displacements (see below).
A DISPLACEMENT is the difference between two states of an object. Typically, a DISPLACEMENT is the difference between a desired target state and the base state.
The WEIGHTING CALCULATION is the function used to compute how much influence a given target has. One simple weighting calculation uses a radial falloff where each target represents the center of a circle. When the mouse is directly over the target (in the center of the circle), the full weight of that target's displacement is applied to the object. When the mouse is outside of the circle, the target has no effect on the object's state, and the weighting inside the circle is a number from 0.0 to 1.0, reflecting the pointer's distance from the center.
It is clear that not all possible states may be reached by this interface given a particular set and arrangement of targets. Thus, only those states that result from the linear combination of the defined target displacements, wherein the weightings are derived from the mouse position, may be achieved. The present invention provides a new user interface technique, system or method that may be applied to various domains.
The present invention provides a real-time interface for performer animation that enables a user to create various classes of targets with different behaviors, internal structure, and graphical representations wherein the animation includes some of the expressive power of scripted systems with natural input gestures and real-time control of interactive systems. In addition, target poses for performer animation may be combined into compound target poses and treated as one unit. Thus, the present invention provides the advantages of real-time animation with less complexity in providing development f expression and movement for target poses.
The present invention may be used to control body movement, dancing and facial expressions of a three dimensional character in an on-line shared music environment, chat room, interactive game or the like. The invention may also be applied to dynamically control sound synthesis.
After selecting the basic unit to be animated, at least one target screen is selected for the upper right portion. In the example shown in
In addition, other layers may be added, as, for example, when the facial expression is to be combined with motion. For example, where a wire-frame figure or skin-covered figure is used for a body to be animated, the geometry may be selected by mouse-clicking in the geometry selection area at the lower left of
In addition, customized poses may be made, for example, by placing the cursor on each desired limb and dragging the limb to the desired position. The customized pose is then saved.
Sequential changes may be made to an arrangement of a plurality of boxes in accordance with the present invention. The boxes may, for example, be arranged in a circle, a line, a spiral, every third one located inward, every fifth one located inward, randomly, or as a vertical circle inside a horizontal circle. By dragging the cursor to the different dot targets or anchors, the arrangements are transformed from a first configuration to a second configuration. By adding a camera pose layer, the user may zoom in and out for the configurations.
The user may insert facial and skeletal target poses in a performer window by dragging and dropping a selected pose from a palette of pre-defined target poses in the target pose window. To animate the character, the user drags the mouse over the target pose window. The target poses are additively applied to the character based on proximity to the cursor. Thus, for example, a smile may dynamically morph into a frown as the cursor moves between a smile target and a frown target, or one dance step may transition to another dance step. The exact layout of targets defines a graphical lexicon of input gestures with corresponding character motions. Various classes of targets may be created with different behaviors, internal structure, and graphical representations. In addition to the target poses on the palette, there may be other choices such as “Begin Recording”, “End Recording”, and “Play Recording”. By selecting “Begin Recording”, the cursor path is recorded, and is shown as a marked path on the computer screen. This recording may be viewed as a video clip by selecting “Play Recording” on the palette and clicking on the marked path.
The present invention may be implemented as an electronic display system, shown in one embodiment in block diagram form in
Clearly, a computer-readable medium having computer-readable instructions 2408 may be used for implementing the present invention. The instructions provide for executing the steps of the method of the invention and are implemented in the fashion described above.
As shown in
For example, the user may use the pointer to position the plurality of target points, anchors, or node terms, either individually or as a group. Typically, each target has a predetermined area of influence that is used to determine, based on a position of the pointer, the displacement to be applied to the object. The state of the object is generally redrawn or updated by putting an object into a default base state when a position of the pointer changes, then applying targets, anchors, or node terms to the object based on a weighting of each target, anchor, or node term, wherein the weighting is calculated based on the displacement of the pointer from the target, anchor, or node term. Each redrawing or updating of the base state of the object may be recorded to provide an animation path or search path which may be used to reproduce the series of redrawings or updatings. The animation path or search path is typically editable.
For example, pointing and clicking may be used to select and change or delete any portion of the animation or search path. Where desired, multiple targets, anchors or node terms with individual weightings may be applied simultaneously.
Current search engines use a variety of measures to algorithmically determine whether or not a document (or more generally an object) is relevant to a query. Common examples include statistical measures such as term frequency and the inverse document frequency of the term and its distribution across all documents in the database. Documents that are longer, more recent, more popular, more frequently linked to by other documents, and whose title or meta-tags contain some of the query terms are typically judged more relevant than documents lacking these characteristics. Various algorithms (often proprietary) are used to expand the query terms and weight and combine the relevance ranking factors. The present invention is independent of the particular factors. What is important is that a set of factors and normative weights for each factor are defined.
Current search engines allow limited control over the weights and factors used to determine relevance rankings. Search engines such as ‘http://www.altavista.com’ and many other popular engines allow terms to be prefixed with a “+” indicating that relevant documents must contain the term, and “−” indicating that relevant documents must not contain the query term. A few search engines such as ‘http://www.infoseek.com’ weigh repeated query terms more heavily than term used only once in a query. For example, as shown in Table 1 above, where listings were generated by infoseek by varying the occurrences of “trust”, “psychology”, and “web” in a query, it is clear that the query terms “trust”, “psychology” and “web” retrieve different sets of document references depending upon how often each term is repeated. The query was intended to retrieve documents on the psychology underlying attributed trustworthiness of websites. As may be seen from examining Table 1, many of the retrieved references were not relevant to the search goal. Notably, in the examples shown in Table 1, the same number of document references was retrieved for each query. What differs is the relative rank assigned to each.
Table 2, shown above, is an illustration of a database generated in client memory to permit fast regeneration of a list of document references and summaries. Double-boxed cells are table column headers defining the content of each column. The second row of the table is an example of a row entry taken from the second entry of the web search results shown in
The examples shown in Table 1 could be generated by moving a cursor appropriately between the anchors, “trust”, “psychology” and “web”. The other factors, e.g., document, recency, and popularity, would continue to affect the relevancy rankings as before. The cursor's movements would only affect the weights associated with the chosen anchors. In the preferred arrangement, the list of relevant document references and summaries is re-generated whenever the mouse button is released in a drag and drop manipulation.
By including or excluding various anchors and moving the anchors and the position of the cursor relative to these anchors, a user may explore the list of retrieved document references far more efficiently than moving to each ‘next’ screen of 10 references, or resubmitting the query with repetitions of some of the query terms.
Although the present invention has been described in relation to particular preferred embodiments thereof, many variations, equivalents, modifications and other uses will become apparent to those skilled in the art. It is preferred, therefore, that the present invention be limited not by the specific disclosure herein, but only by the appended claims.
The present application is a continuation of U.S. patent application Ser. No. 09/672,475, filed Sep. 28, 2000, the content of which is incorporated herein by reference in its entirety.
| Number | Name | Date | Kind |
|---|---|---|---|
| 5847708 | Wolff | Dec 1998 | A |
| 5854634 | Kroitor | Dec 1998 | A |
| 5933150 | Ngo et al. | Aug 1999 | A |
| 6011562 | Gagne et al. | Jan 2000 | A |
| 6720962 | Alter | Apr 2004 | B1 |
| 7233327 | Isner | Jun 2007 | B2 |
| 7391420 | Coyne | Jun 2008 | B1 |
| Number | Date | Country | |
|---|---|---|---|
| 20080222555 A1 | Sep 2008 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 09672475 | Sep 2000 | US |
| Child | 12122147 | US |
