Patent Grant
8508549
The invention relates generally to graphical user interface design and more particularly to a means for specifying a graphical user interface object in a procedural and largely display resolution independent manner.
Designing an efficient, ergonomic and aesthetically pleasing user interface is an integral stage of most application development projects. The graphical user interface (“GUI”) is what the user sees and interacts with. Accordingly, the GUI must present information and choices to a user in a way that is not only pleasing and natural to the eye but conducive to efficient use of the underlying application. One major concern in the development of modern GUIs is the resolution of the various objects that comprise the GUI. Typically, a designer designs a graphical user interface object (e.g., a pushbutton, scrollbar, or slider) for a specified resolution. As the resolution of the user's display changes, however, display of the originally designed object may become distorted. This is particularly a problem when a graphical object is designed at a first resolution (e.g., 75 or 100 pixels per inch) and the user's display is at a second, higher resolution (e.g., 120 or 150 pixels per inch).
In the past, two general techniques have been used to address the problem associated with displaying objects designed for a first resolution but which are displayed at a second resolution. In the first, an original (low resolution) object is up-sampled to generate a larger image (e.g., through linear or bicubic interpolation). This technique results in blurry edges such that the user interface no longer looks crisp. In the second, an original object is designed for display at a high resolution and is then down-sampled to an unknown target resolution. While useful in some circumstances, it is not possible a priori to know what width to give a line (e.g., an object's edge) at the higher resolution such that when down-sampled it remains crisp. This is particularly true when there are multiple target resolutions. Thus, both up-sampling and down-sampling techniques tend to disturb the designer's specified line width. One of ordinary skill in the art will recognize that line width is a critical factor in GUI design as the width of lines define the edge of graphical objects. If edges appear blurry or ill-defined, the entire GUI design may be compromised.
Thus, it would be beneficial to provide a means to specify the design of a graphical user interface object independent of its display resolution. Such a description may advantageously be used by a rendering module to display the designed object at substantially any resolution.
In one embodiment, the invention provides a method to represent a graphical user interface object in a procedural, and largely, resolution independent manner. The method includes receiving values for each of a plurality of attributes associated with a graphical user interface object (the attributes specifying display characteristics of the graphical user interface object), associating at least one value for each of the plurality of attributes, and storing the plurality of attributes and their associated values in a file. Each of a first portion of the object's attributes may be associated with a single value—a value that is independent of the object's display resolution. Each of a second portion of the object's attributes may be associated with two or more values—each value (for a given attribute) associated with a distinct display resolution. The collection of attribute-value pairs comprise a complete description of the graphical user interface object and may be used by a rendering module to create a visual representation of the object at any number of resolutions.
In another embodiment, the invention provides a method to render or display a graphical user interface object specified in a procedural manner as described above. The method includes receiving a graphical user interface object recipe file (a file containing attribute-value pairs), identifying the graphical user interface object described by the recipe file, extracting a plurality of values from the file (each value being associated with an attribute for the identified graphical user interface object), and rendering the identified graphical user interface object based on the extracted plurality of values.
Those of ordinary skill in the art will recognize that methods in accordance with the described invention may be embodied in programs, program modules or applications that may be stored in any media that is readable and executable by a computer system.
Methods, devices and systems to describe or capture the design of graphical user interface objects in a procedural and, largely resolution independent, manner are described. The following embodiments of the invention, described in terms of graphical user interface object design conforming to the Apple Human Interface Guidelines, are illustrative only and are not to be considered limiting in any respect. (The Apple Human Interface Guidelines are available from Apple Computer, Inc. of Cupertino, Calif.)
It has been determined that a graphical user interface object may be completely described by a collection of resolution-independent attributes. The collection of all attributes for a given object type define that type of object (e.g., pushbuttons). While the attributes used to define an object may vary, in whole or in part, from object-type to object-type, one of ordinary skill in the art will recognize those attributes needed to completely specify a given object. For example, while some attributes may be associated with a number of different graphical interface objects (e.g., those associated with an object's location in a display window), many attributes may change from object to object (e.g., buttons have attributes associated with the “button” metaphor while check-boxes and slider tracks have attributes particular to their visual metaphor). Accordingly, the attributes identified herein are illustrative only and should not be used to limit the claimed methods, devices and systems.
Just as a specific collection of attributes define a specific type of graphical interface object (e.g., pushbuttons), the values associated with each of the specific attributes define a particular implementation or embodiment of the object (e.g., a regular size, deactivated pushbutton). In accordance with the invention, some attribute-values are specified independent of the resolution at which the object is to be displayed while other attribute-values are specified for two or more resolutions. In general, those attributes associated with the physical location or geometry of an object may be associated with a plurality of values—one value for each specified resolution. Other attributes are associated with fixed, resolution independent, values (e.g., visual characteristics such as opacity, color and curvature). By way of example, consider a pushbutton object whose radius (attribute “buttonRadius”) is specified for each of five predetermined resolutions (e.g., 100, 120, 125, 133.3 and 150 pixels per inch), but whose outline color (attributes “outlineRed”, “outlineGreen”, and “outlineBlue”) and opacity (attribute “buttonOpacity”) are fixed and resolution independent.
Thus, in accordance with the invention the collection of all attribute-values for a specified object completely define its visual characteristics and, as such, may be used to drive the graphical generation of the object (for example, by a rendering engine or operating system module responsible for rendering images). One benefit of object definitions in accordance with the invention is that graphical objects are defined in terms of a collection of resolution independent attributes. Another benefit of object definitions in accordance with the invention is that each attribute may be associated with a plurality of values, thereby permitting the designer to optimize the object's design for each of a specified number of resolutions. Still another benefit of object definitions in accordance with the invention is that if the actual displayed resolution of the graphical object is between two of the resolutions specified by the designer, the rendering engine may interpolate between the two values—a technique that generally provides a significantly improved display over prior art up-sampling or down-sampling techniques.
In accordance with the invention, a graphical user interface object's design is specified by a collection of attribute-value pairs that are retained or stored in a file, hereinafter referred to as a “recipe” file. In one embodiment, the recipe file may be a “flat” file consisting of sequential listing of attribute-value pairs. In another embodiment, the recipe file may be a hierarchically ordered file representing an inverted tree, where the root of the tree identifies the type of graphical object (e.g., a pushbutton, a check-box or a slider track) and the first level below the root identifies categories associated with the object (e.g., size, color and state). In one particular embodiment, hierarchically ordered recipe files are stored as eXtensible Markup Language (“XML”) files. Attributes and their values are then associated with each node. In this way, every aspect of a graphical user interface object may be systematically identified and recorded in the recipe file in a manner that is wholly independent from the method used to physically draw (render) the image on a computer display device.
Methods, devices and systems in accordance with the invention may be described in terms of two phases. In a first phase, recipe files are generated. In a second phase the recipe files are used to generate visual representations of the graphical user interface object for one or more applications at substantially any resolution.
Referring to
Referring to
Referring to
Browser region 405 permits the selected display of various aspects of an object's design. In particular, region 405 provides a graphical representation of a recipe file's hierarchical structure in an Apple standard “Finder” format: the left-most pane identifies the recipe file's root (which, in turn, identifies the type of object—a pushbutton); the middle pane identifies categories of pushbuttons (e.g., inactive, mini, pressed, pulsed, regular and small). Subsequent panes display subcategories associated with a selected category. For example, disclosure triangles 430 indicate that pushbutton categories mini, pressed, pulsed, regular and small have additional aspects—the details of which are displayed in the right-most (and possibly subsequent) panes when one of these categories is selected.
Resolution display region 410 identifies one or more resolutions for which the object is being designed. As shown, the designer has specified that at least some attribute values for a pushbutton are specified for resolutions of 100, 120, 125, 133.3 and 150 pixels per inch. As noted above, not all attribute values are specified for each of these resolutions, only those that the designer determines are significant to the object's display. Illustrative attribute-value pairs for a pushbutton object and a scrollbar object, including those attributes having multiple values, are shown in Table 1 below. (It will be recognized that the objects shown in region 410 are not actually displayed at the indicated resolution, but are instead “simulations” of how the object would appear at those resolutions.)
Expanded bit display region 415 shows an expanded representation of the selected resolution image. In the illustrated example of
Object shape region 420 permits the designer to select, view and specify attribute values associated with a particular shape of the object being designed. For example, in the illustrated embodiment a pushbutton's shape may be any one of the shapes identified by shape buttons 435: Round, Lozenge (“Lozen . . . ”), Round Rectangle (“Round . . . ”), odd (“Scroll bar cap odd end”) or custom. Immediately beneath shape buttons 435, area 440 shows specific attributes associated with the selected shape and, through controls such as slider 445, text box 450, radio button 455 or color well 460 permits the designer to change the value associated with those attributes.
User interface construction region 425 serves as the primary interface for viewing and specifying attribute values associated with an object's various visual characteristics or layers. In the embodiment of
Thus, in accordance with the invention a graphical user interface object may be completely defined by a collection of attribute-value pairs that may be used by a rendering engine (or similar module) to display the object. Further, one or more attributes may have two or more values, wherein each value is associated with a specific display resolution. This latter feature permits a designer to uniquely and specifically optimize a single design for multiple resolutions with the added benefit of providing sufficient information for interpolation (generally performed by the rendering engine) should the actual resolution be different from any of the specified resolutions. By way of example only, Table 1 comprises a listing of attributes and their associated values for a pushbutton object and a scrollbar object.
In a current embodiment, graphical user interface objects identified in Table 2 may be defined/specified using the attributes (left-hand column) identified in Table 1. It will also be recognized that while many of the attributes above are specified by values in units of pixels, in other embodiments attribute values may be expressed in terms of a relative factor to a predetermined size factor.
To create a graphical user interface object, the body color of the object (for each point on the object) and the anti-aliased visibility mask of the object are needed. The body color of an object may be obtained by using a three-dimensional representation of the object, or by creating a virtual representation of the object that defines the surface normal for each pixel on the object. Once a unit-length surface normal vector is computed at a point p, the x and y coordinate values of this vector may be used to compute the apparent color of the object at point p by looking up a color from the object's relevant material map. If the map is n pixels high and n pixels wide (this is done because a shaded sphere is inscribed in the map), one can address the material map at the two-dimensional location given by:
The color of the material map at this location may be used as the color for the object at point p. To get an accurate result, it is typical for a material map to be much larger (for example, 256×256 pixels or larger) than the graphical user interface object being rendered.
This same technique may be used in conjunction with a material map that possesses alpha (transparency) information. Once a color (with alpha) is looked up from a transparency material map, a highlight may be overlaid onto the object by using the alpha as a coverage fraction for the color from the map. Standard compositing methods may be used to accomplish this overlay operation.
An object's anti-aliased visibility mask may be obtained by computing a field that provides distance from the edge of the object. This field can be evaluated using a procedural description of the object. For example, a lozenge may be defined as the set of points at distance r or less from a finite non-zero-length line segment from point (p1x, p1y) to point (p2x, p2y). The distance d from the aforementioned line segment may be calculated at point (px, py) by a function such as that provided in Table 3 below.
Given the distance function d defined above (see Table 1), an anti-aliased transparency value (mask) for the graphical user interface object may be computed as shown in Table 4. The same distance field may be used to construct the outline of the user interface object.
Various changes or modifications in the foregoing description may be made without departing from the concept of the invention. For example, attributes other than, or in addition to, those identified in Table 1 may be used to specify an object. In addition, objects other than, or in addition to, those identified in Table 2 may be specified in accordance with the invention. Further, hierarchical storage means other than an XML file may be used to store an object's procedural specification.
It will be recognized that methods to represent and render a graphical user interface object in accordance with this description may be performed by a programmable control device executing instructions organized into one or more program modules. A programmable control device may be a single computer processor, a special purpose processor (e.g., a digital signal processor, a graphics processing unit or a programmable graphics processing unit), a plurality of processors coupled by a communications link or a custom designed state machine. Custom designed state machines may be embodied in a hardware device such as an integrated circuit including, but not limited to, application specific integrated circuits (“ASICs”) or field programmable gate array (“FPGAs”). Storage devices suitable for tangibly embodying program instructions include, but are not limited to: magnetic disks (fixed, floppy, and removable) and tape; optical media such as CD-ROMs and digital video disks (“DVDs”); and semiconductor memory devices such as Electrically Programmable Read-Only Memory (“EPROM”), Electrically Erasable Programmable Read-Only Memory (“EEPROM”), Programmable Gate Arrays and flash devices.
Accordingly, the preceding descriptions were presented to enable any person skilled in the art to make and use the invention as claimed and were provided in the context of the particular examples discussed above, variations of which will be readily apparent to those skilled in the art. Accordingly, the claims appended hereto are not intended to be limited by the disclosed embodiments, but are to be accorded their widest scope consistent with the principles and features disclosed herein.
This is a continuation application which claims priority to application Ser. No. 11/696,619, filed Apr. 4, 2007, which is a continuation of application Ser. No. 10/876,298, filed Jun. 24, 2004, now U.S. Pat. No. 8,068,103, all of which are hereby included in their entirety.
| Number | Name | Date | Kind |
|---|---|---|---|
| 5297250 | Leroy et al. | Mar 1994 | A |
| 5357603 | Parker | Oct 1994 | A |
| 5388201 | Hourvitz et al. | Feb 1995 | A |
| 5471572 | Buchner et al. | Nov 1995 | A |
| 5481665 | Okada et al. | Jan 1996 | A |
| 5490246 | Brotsky et al. | Feb 1996 | A |
| 5513342 | Leong et al. | Apr 1996 | A |
| 5537630 | Berry et al. | Jul 1996 | A |
| 5632033 | Guillen et al. | May 1997 | A |
| 5651107 | Frank et al. | Jul 1997 | A |
| 5721848 | Joseph | Feb 1998 | A |
| 5764229 | Bennett | Jun 1998 | A |
| 5793376 | Tanaka et al. | Aug 1998 | A |
| 5841435 | Dauerer et al. | Nov 1998 | A |
| 5877741 | Chee et al. | Mar 1999 | A |
| 5877762 | Young | Mar 1999 | A |
| 5933148 | Oka et al. | Aug 1999 | A |
| 5949409 | Tanaka et al. | Sep 1999 | A |
| 6006231 | Popa | Dec 1999 | A |
| 6031937 | Graffagnino | Feb 2000 | A |
| 6075543 | Akeley | Jun 2000 | A |
| 6166748 | Van Hook et al. | Dec 2000 | A |
| 6211890 | Ohba | Apr 2001 | B1 |
| 6246418 | Oka | Jun 2001 | B1 |
| 6272558 | Hui et al. | Aug 2001 | B1 |
| 6286038 | Reichmeyer et al. | Sep 2001 | B1 |
| 6307574 | Ashe et al. | Oct 2001 | B1 |
| 6313851 | Matthews, III et al. | Nov 2001 | B1 |
| 6369823 | Ohba | Apr 2002 | B2 |
| 6369830 | Brunner | Apr 2002 | B1 |
| RE37722 | Burnard et al. | May 2002 | E |
| 6411301 | Parikh et al. | Jun 2002 | B1 |
| 6421058 | Parikh et al. | Jul 2002 | B2 |
| 6424348 | Parikh | Jul 2002 | B2 |
| 6452600 | Parikh et al. | Sep 2002 | B1 |
| 6456290 | Parikh et al. | Sep 2002 | B2 |
| 6466218 | Parikh et al. | Oct 2002 | B2 |
| 6483524 | Petenkine et al. | Nov 2002 | B1 |
| 6489963 | Parikh et al. | Dec 2002 | B2 |
| 6509913 | Martin et al. | Jan 2003 | B2 |
| 6526174 | Graffagnino | Feb 2003 | B1 |
| 6542160 | Abgrall | Apr 2003 | B1 |
| 6571328 | Liao et al. | May 2003 | B2 |
| 6577317 | Duluk, Jr. et al. | Jun 2003 | B1 |
| 6580430 | Hollis et al. | Jun 2003 | B1 |
| 6606103 | Hamlet et al. | Aug 2003 | B1 |
| 6609977 | Shimizu et al. | Aug 2003 | B1 |
| 6614444 | Duluk, Jr. et al. | Sep 2003 | B1 |
| 6618048 | Leather | Sep 2003 | B1 |
| 6636214 | Leather et al. | Oct 2003 | B1 |
| 6639595 | Drebin et al. | Oct 2003 | B1 |
| 6664958 | Leather et al. | Dec 2003 | B1 |
| 6664962 | Komsthoeft et al. | Dec 2003 | B1 |
| 6697074 | Parikh et al. | Feb 2004 | B2 |
| 6707462 | Peercy et al. | Mar 2004 | B1 |
| 6717599 | Olano | Apr 2004 | B1 |
| 6734864 | Abgrall | May 2004 | B2 |
| 6785685 | Soetarman et al. | Aug 2004 | B2 |
| 6850255 | Muschetto | Feb 2005 | B2 |
| 6906720 | Emberling et al. | Jun 2005 | B2 |
| 6911984 | Sabella | Jun 2005 | B2 |
| 6982695 | Canova et al. | Jan 2006 | B1 |
| 7136790 | Hobbs | Nov 2006 | B1 |
| 7167173 | Balmelli et al. | Jan 2007 | B2 |
| 7234111 | Chu et al. | Jun 2007 | B2 |
| 7340716 | Chansler | Mar 2008 | B1 |
| 7590947 | Gay et al. | Sep 2009 | B1 |
| 7675529 | Brunner et al. | Mar 2010 | B1 |
| 8004535 | Blaukopf et al. | Aug 2011 | B2 |
| 20010024195 | Hayakawa | Sep 2001 | A1 |
| 20010033295 | Phillips | Oct 2001 | A1 |
| 20010050679 | Shigeta | Dec 2001 | A1 |
| 20020067418 | I | Jun 2002 | A1 |
| 20020091758 | Singh et al. | Jul 2002 | A1 |
| 20020093516 | Brunner et al. | Jul 2002 | A1 |
| 20020093538 | Carlin | Jul 2002 | A1 |
| 20020118217 | Fujiki | Aug 2002 | A1 |
| 20020140727 | Lindholm | Oct 2002 | A1 |
| 20020147729 | Balfour | Oct 2002 | A1 |
| 20020171682 | Frank et al. | Nov 2002 | A1 |
| 20020174181 | Wei | Nov 2002 | A1 |
| 20030058487 | Lay | Mar 2003 | A1 |
| 20030067489 | Wong et al. | Apr 2003 | A1 |
| 20030123739 | Graffagnino | Jul 2003 | A1 |
| 20030126293 | Bushey | Jul 2003 | A1 |
| 20030146934 | Bailey et al. | Aug 2003 | A1 |
| 20030174136 | Emberling et al. | Sep 2003 | A1 |
| 20030210268 | Kataoka et al. | Nov 2003 | A1 |
| 20030234799 | Lee | Dec 2003 | A1 |
| 20040032409 | Girard | Feb 2004 | A1 |
| 20040054610 | Amstutz et al. | Mar 2004 | A1 |
| 20040189670 | Govindan et al. | Sep 2004 | A1 |
| 20040212610 | Hamlin | Oct 2004 | A1 |
| 20040223003 | Heirich et al. | Nov 2004 | A1 |
| 20040243940 | Lee et al. | Dec 2004 | A1 |
| 20050010872 | Lee et al. | Jan 2005 | A1 |
| 20050021970 | Reese et al. | Jan 2005 | A1 |
| 20050041045 | Plut | Feb 2005 | A1 |
| 20050081161 | MacInnes et al. | Apr 2005 | A1 |
| 20050088447 | Hanggie et al. | Apr 2005 | A1 |
| 20050094207 | Lo et al. | May 2005 | A1 |
| 20050108656 | Wu et al. | May 2005 | A1 |
| 20050140694 | Subramanian et al. | Jun 2005 | A1 |
| 20050168471 | Paquette | Aug 2005 | A1 |
| 20050187805 | English | Aug 2005 | A1 |
| 20050190203 | Gery et al. | Sep 2005 | A1 |
| 20050219255 | Lin et al. | Oct 2005 | A1 |
| 20050229110 | Gegner et al. | Oct 2005 | A1 |
| 20050231512 | Niles et al. | Oct 2005 | A1 |
| 20050248545 | Nishimura et al. | Nov 2005 | A1 |
| 20060055700 | Niles et al. | Mar 2006 | A1 |
| 20060224627 | Manikutty et al. | Oct 2006 | A1 |
| 20070263007 | Robotham et al. | Nov 2007 | A1 |
| Number | Date | Country |
|---|---|---|
| 548586 | Jun 1993 | EP |
| 0694879 | Jan 1996 | EP |
| 1383080 | Jan 2004 | EP |
| 0972273 | Mar 2004 | EP |
| 9845815 | Oct 1998 | WO |
| 0209039 | Jan 2002 | WO |
| 2004027707 | Apr 2004 | WO |
| Entry |
|---|
| Akeley, et al. “Real-Time Graphics Architecture” http://www.graphics.stanford.edu/courses/cs448a-01-fall, The OpenGL ® Graphics System—CS448 Lecture 15, Fall 2001, pp. 1-20. |
| “Apple Human Interface Guidelines,”Apple Computer, Inc. Cupertino, CA Mar. 29, 2004. 317 pages. |
| Elliott, “Programming Graphics Processors Functionally.” 2004. |
| Gelder, et al. “Direct Volume Rendering with Shading via Three Dimensional Textures” Computer Science Department, Univ. of California, Santa Cruz, CA 95064, 1996. |
| Haeberli, P. et al. “The Accumulation Buffer: Hardware Support for High-Quality Rendering” Computer Graphics, New York, NY, vol. 24, No. 4, Aug. 1, 1990, pp. 309-318. |
| nVIDIA, “Cg—Teaching Cg” Power Point Presentation, 2003, http://developer.nvidia.com/object/cg—tutorial—teaching. html. |
| Segal et al. “The OpenGL ® Graphics System: A Specification (Version 1.5)” Copyright © 1992-2003 Silicon Graphics, Inc. Oct. 30, 2003. |
| Shantzis, “A Model for Efficient and Flexible Image Computing” Computer Graphics Proceedings, Annual Conference Series, 1994, pp. 147-154. |
| Erikson, Carl “Polygonal Simplification: An Overview.” TR96-016:Department of Computer Science CB # 3175, Sitterson Hall, UNC—Chapel Hill, Chapel Hill, NC. 1996m 32 pages. |
| International Search Report dated Jul. 27, 2005 (PCT/US05/008804). |
| International Search Report dated Aug. 8, 2005 (PCT/US05/008805). |
| Number | Date | Country | |
|---|---|---|---|
| 20120120076 A1 | May 2012 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 10876298 | Jun 2004 | US |
| Child | 13355461 | US | |
| Parent | 11696619 | Apr 2007 | US |
| Child | 10876298 | US |
