Patent Grant
7870501
The present invention relates generally to graphical user interfaces. Specifically, aspects of the present invention relate to providing an indication to a user that certain graphical objects have been selected and/or highlighted. Further aspects of the present invention allow users to manipulate the selected graphical objects.
A picture is worth a thousand words. The sentiment echoes throughout various aspects of our lives, and the world of computing is no exception. Since its inception, graphical user interfaces (GUIs) have become the standard, and preferred, way for millions of computer users to interact with their computers. Accordingly, more and more importance is being placed on the way in which computers visually show the user's actions and the environment in which the user may take these actions.
The typical GUI uses a number of onscreen graphical objects to visually represent functions, applications, files, menus, and a host of other features for the computing environment. The user typically uses a mouse input device to move an onscreen pointer to select a particular graphical object for action.
This GUI has proven effective, but a new step in the evolution of computing has revealed several drawbacks to existing GUIs. Specifically, the introduction of pen-based computing devices, such as the hand-held Personal Data Assistant (PDA) or the Tablet PC being introduced by Microsoft Corp., has changed the way we view the GUI, and the manner in which users can interact with their computers. This new approach to user interfaces has revealed problems and deficiencies in the traditional GUI described above. Examples of these problems will be discussed below.
One common use of computers and GUIs is to generate and edit electronic documents. These electronic documents can contain text (e.g., electronic word processors) and/or images (e.g., pictures), which are displayed on the user's screen for editing. To interact with these onscreen objects, the user typically uses a mouse input device to move an onscreen pointer to the desired object, and presses a button on the mouse to select the object.
The selection of the particular object may be reflected in a change in its appearance. For example, electronic word processing programs, such as the MICROSOFT WORD (R) program, may display text as shown in
Although this prior art approach to highlighting text works well in the uniform, line-by-line environment of traditional word processors, this approach is less desirable in other environments that allow a greater degree of freedom, such as pen-based computing devices. For example, in systems where the text is handwritten (e.g., on a personal data assistant using a touch-sensitive screen), the user may be permitted to write text above and below any such regimented lines.
A further drawback to this type of selection highlighting lies in the colors used for the selection block, and for the selected text appearing in the block. Although users of word processing programs are often provided with the ability to select font colors and/or background colors, they are not provided with the ability to select the color to be used for the selection block highlight, nor are they offered the ability to select the color of the selected text. Instead, these word processing programs automatically use predetermined colors for the selection block and selected text, which are often entirely different colors from the font color and background color. By using these new colors, these word processing programs may inadvertently obliterate any meaning that had been assigned to the original font color. For example, if a user were accustomed to using a red font color to identify text added by a certain person, and this color were changed upon selection of the text, the user might then be unable to tell whether the selected text was originally red or not.
The prior art approach to highlighting images, as opposed to the regimented text of traditional word processors, does not offer much of a better solution.
Accordingly, there is now a need for an improved approach to identifying selected graphical objects in a GUI environment that can overcome one or more of the deficiencies identified above.
Embodiments of the present invention may solve one or more of the problems and/or deficiencies identified above. In one aspect, a graphical object may appear in one color (or pattern) against a background of another color (or pattern). To highlight the graphical object, its body may be changed to a predetermined background color (such as white), or changed in color to appear transparent, allowing the color or pattern of the underlying background to show. A “halo” is displayed around the periphery of the graphical object, where the halo is displayed in a color (or pattern) matching the graphical object's original color or pattern. In further aspects, the halo may have a change in transparency or darkness of the color or pattern, allowing for a “glowing” appearance.
In some embodiments, the halo follows the contours of the outer periphery of the original graphical object. In further embodiments, the halo follows the exact contour of the outer periphery of the original graphical object. In yet further embodiments, the halo may be of a predetermined, constant, thickness.
In further embodiments, the highlighted graphical object may also be surrounded with a bounding shape, such as a rectangle. In yet further embodiments, the bounding shape may include selection handles.
In some embodiments, selection of the halo by the user may result in the selection of the graphical object.
In some embodiments, the highlighted graphical object may be moved to a different location on the graphical user interface. In further embodiments, a separate, moving version of the graphical object may be displayed to follow the movement of the graphical object. In yet further embodiments, the moving version of the graphical object may be drawn in a fainter color and/or pattern than the original graphical object. In further embodiments, the moving version of the graphical object may include a bounding box. In still further embodiments, the bounding box of the moving version of the graphical object may follow the outer contour of the graphical object.
In some embodiments, the halo retains its color and/or pattern as the graphical object is moved from a first area of the GUI, having a first pair of foreground/background colors, to a second area of the GUI having a different pair of foreground/background colors.
In further embodiments, the graphical object may have a varying thickness that, upon highlighting, is changed to a constant thickness, which simplifies the processing required to render the highlighted object.
In yet further embodiments, the halo may be rendered using a plurality of graphical layers. In some embodiments, the halo may be rendered by overlaying a duplicate of the halo over an enlarged duplicate of the halo, where the first duplicate is of a different color. In further embodiments, the halo may be rendered by overlaying a plurality of enlarged duplicates of the object, where each of the enlarged duplicates is of a slightly different size and color.
The above and other objects, features and advantages of the present invention will be readily apparent and fully understood from the following detailed description of preferred embodiments, taken in connection with the appended drawings.
a-9b show alternate embodiments of moving graphical objects according to further embodiments of the present invention.
a-b show an example of a highlighted line in the prior art.
a-b are photographic examples of one embodiment of the present invention.
a-b are photographic examples of another embodiment of the present invention.
The present invention may be more readily described with reference to
A basic input/output system 160 (BIOS), containing the basic routines that help to transfer information between elements within the computer 100, such as during start-up, is stored in the ROM 140. The computer 100 also includes a hard disk drive 170 for reading from and writing to a hard disk (not shown), a magnetic disk drive 180 for reading from or writing to a removable magnetic disk 190, and an optical disk drive 191 for reading from or writing to a removable optical disk 192 such as a CD ROM or other optical media. The hard disk drive 170, magnetic disk drive 180, and optical disk drive 191 are connected to the system bus 130 by a hard disk drive interface 192, a magnetic disk drive interface 193, and an optical disk drive interface 194, respectively. The drives and their associated computer-readable media provide nonvolatile storage of computer readable instructions, data structures, program modules and other data for the personal computer 100. It will be appreciated by those skilled in the art that other types of computer readable media that can store data that is accessible by a computer, such as magnetic cassettes, flash memory cards, digital video disks, Bernoulli cartridges, random access memories (RAMs), read only memories (ROMs), and the like, may also be used in the example operating environment.
A number of program modules can be stored on the hard disk drive 170, magnetic disk 190, optical disk 192, ROM 140 or RAM 150, including an operating system 195, one or more application programs 196, other program modules 197, and program data 198. A user can enter commands and information into the computer 100 through input devices such as a keyboard 101 and pointing device 102. Other input devices (not shown) may include a microphone, joystick, game pad, satellite dish, scanner or the like. These and other input devices are often connected to the processing unit 110 through a serial port interface 106 that is coupled to the system bus, but may be connected by other interfaces, such as a parallel port, game port or a universal serial bus (USB). Further still, these devices may be coupled directly to the system bus 130 via an appropriate interface (not shown). A monitor 107 or other type of display device is also connected to the system bus 130 via an interface, such as a video adapter 108. In addition to the monitor, personal computers typically include other peripheral output devices (not shown), such as speakers and printers. In a preferred embodiment, a pen digitizer 165 and accompanying pen or stylus 166 are provided in order to digitally capture freehand input. Although a direct connection between the pen digitizer 165 and the processing unit 110 is shown, in practice, the pen digitizer 165 may be coupled to the processing unit 110 via a serial port, parallel port or other interface and the system bus 130 as known in the art. Furthermore, although the digitizer 165 is shown apart from the monitor 107, it is preferred that the usable input area of the digitizer 165 be co-extensive with the display area of the monitor 107. Further still, the digitizer 165 may be integrated in the monitor 107, or may exist as a separate device overlaying or otherwise appended to the monitor 107.
The computer 100 can operate in a networked environment using logical connections to one or more remote computers, such as a remote computer 109. The remote computer 109 can be a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above relative to the computer 100, although only a memory storage device 111 has been illustrated in
When used in a LAN networking environment, the computer 100 is connected to the local network 112 through a network interface or adapter 114. When used in a WAN networking environment, the personal computer 100 typically includes a modem 115 or other means for establishing a communications over the wide area network 113, such as the Internet. The modem 115, which may be internal or external, is connected to the system bus 130 via the serial port interface 106. In a networked environment, program modules depicted relative to the personal computer 100, or portions thereof, may be stored in the remote memory storage device.
It will be appreciated that the network connections shown are examples and other techniques for establishing a communications link between the computers can be used. The existence of any of various well-known protocols such as TCP/IP, Ethernet, FTP, HTTP and the like is presumed, and the system can be operated in a client-server configuration to permit a user to retrieve web pages from a web-based server. Any of various conventional web browsers can be used to display and manipulate data on web pages.
Although the
A stylus could be equipped with buttons or other features to augment its selection capabilities. In one embodiment, a stylus could be implemented as a “pencil” or “pen”, in which one end constitutes a writing portion and the other end constitutes an “eraser” end, and which, when moved across the display, indicates portions of the display are to be erased. Other types of input devices, such as a mouse, trackball, or the like could be used. Additionally, a user's own finger could be used for selecting or indicating portions of the displayed image on a touch-sensitive or proximity-sensitive display. Consequently, the term “user input device”, as used herein, is intended to have a broad definition and encompasses many variations on well-known input devices.
Anything appearing on a graphical user interface may be considered to be a graphical object, and
The body 301 of graphical object 300 may be drawn, or rendered, using any color or pattern value (e.g., hatched, shaded, a series of colors, etc.), and may be drawn on a background of a different color or pattern. In the
For example, the background may include a series of parallel horizontal (or vertical) lines to assist the user in keeping handwritten text in line. The object need not be constrained to such lines, however, and may simply overlap 1, 2, 3 or more of them.
The background may also include various other objects, such as text, images, drawings, handwriting, etc.
The graphical object 300 may become highlighted to represent that it has been selected, or to otherwise draw attention to it. This may occur in any number of ways. For example, in a pen-based computing device, the user may position the stylus over the displayed graphical object to select the object. The user may move a pointer over the graphical object to select it, and may also press a button to select it. Any other known method for selecting a graphical object in a GUI will also suffice.
When the graphical object is to be highlighted, its appearance changes. The body 301 may have its color value changed to a transparent value, such that the background 302 becomes visible in the area previously occupied by the body 301. The highlighted graphical object may also have an outline, or “halo” 401, of additional color and/or pattern drawn around its periphery, surrounding the original body 301 of the graphical object. Thus, as an example, a graphical object in a first color/pattern (e.g., red) on a background of a second color/pattern (e.g., black striped) will, upon selection, be changed in appearance to have a halo in the first color/pattern (i.e., red), and a body in the second color/pattern (i.e., black striped) due to its transparency. In alternative embodiments, the body 301 of the graphical object may have its color changed to a predetermined color, such as white, rather than a transparent color. The use of a predetermined color may advantageously simplify the processing required to render the highlighted graphical object.
In alternative embodiments, the halo may have a varying thickness to give the selected object a more distinguished appearance. For example, as shown in
As shown in
As another feature, some embodiments of the present invention allow the user to select the graphical object by selecting its halo. For example, a pen-based computing device may allow users to select graphical objects by placing the stylus on or over the highlighted graphical object. Similar selections may be made using a mouse that moves an onscreen pointer, pressing a sequence of keys on a keyboard, or any other form of GUI interaction. With such a selectable halo, graphical objects having a thin appearance (e.g., a single hand-drawn stroke or line) may be easily selected after they have been highlighted.
One advantage offered by some embodiments of the halos described above is apparent in situations where multiple graphical objects appear on the display screen, and overlap one another. If one of these graphical objects is selected, while the other is not, the transparent, or “hollow,” body of the selected graphical object, together with its halo, clearly set the object apart from the other unselected graphical object(s).
In a further aspect, the highlighted graphical object may also be surrounded by a bounding box.
In alternate embodiments, the bounding box 801 need not be rectangular, and need not be a box, at all. For example, the bounding box may actually be implemented as any other shape, such as a circle, oval, diamond, etc. to surround the graphical object being highlighted.
In further embodiments, the bounding box 801 may include one or more selection handles 802a,b that may be used to interact with the object for further handling, such as moving and/or resizing the object. Selection handles 802a may be located at the corners of the bounding box 801, and/or at other locations 802b along the borders of the box. By selecting one of these handles (e.g., using a mouse-controlled pointer, stylus, or any other GUI selection method), the user may resize the graphical object, and/or select the graphical object for further processing/editing. In some embodiments, the bounding box border distance (d1) is configured to be of sufficient size such that the selection handles do not overlap or obscure any portion of the highlighted graphical object.
In one aspect, the user may move a highlighted graphical object through various portions of the GUI. This movement may be made, for example, by the user selecting the graphical object in a pen-based computing system by placing a stylus on the graphical object, pressing a button on the stylus to select the graphical object, and then moving the stylus across a display screen to the desired new location. During this movement, some embodiments of the present invention may display a moving version of the graphical object at a location corresponding to the user's device. In this manner, the user is given feedback regarding the movement he/she is carrying out, as well as the system's acknowledgement that the move operation has begun.
The moving version of the graphical object may appear as a simple outline of the object, as shown in
The moving version may alternatively appear as a bounding box with a pointer or cursor, as shown in
As a further alternative, the moving version of the graphical object may simply be a copy of the graphical object rendered in a different color and/or pattern, such as using a lighter shade of the color, or a fainter version of the color that allows more transparency. Using a lighter or fainter color for the moving version of the graphical object, the user is provided with a clear indication of what will be obscured upon repositioning of the object.
The moving versions described above may be rendered with its own bounding box and/or selection handles, as described above, to further indicate that a movement process has begun.
In some embodiments, the GUI may include several distinct areas in which different color contrasts are used. For example, a GUI may have a first area in which black graphical objects appear against a white background, and a second area in which red graphical objects appear against a green background. Upon selection in the first area, a graphical object originally appearing in black ink may be replaced with a transparent body and a black halo. The user may move this object from the first area to the second area. In some embodiments, upon movement to this second area, the color of the highlighted graphical object's body and halo remain the same. In this manner, any meaning associated with the color of the underlying graphical object is preserved in the new GULI area. Alternatively, the highlighted graphical object's halo may change in color to that of the second area (in this example, red).
Increasing the dimensions by a predetermined amount advantageously allows consistency in the visual appearance of highlighted graphical objects. On the other hand, increasing the dimensions by a percentage may allow for a more dynamic visual experience. In some embodiments, a graphical object comprised of lines may be enlarged by having its lines widened by the predetermined amount. The enlarged version 1101 in
The computer display system may generate the enlarged version 1101 in an internal memory or display buffer, and need not actually display or render the enlarged version on screen immediately.
The computer display system may also generate another copy 1201 of the original graphical object 1000, where the copy 1201 has the same dimensions as the original object. The copy 1201 may be assigned a color value that is the background color value of the area on which the original object was displayed. The copy 1201 may then be overlaid upon the enlarged version 1101, resulting in the highlighted graphical object shown in
In some embodiments, the color value of the copy 1201 will be such that background images, text, and/or objects may be visible. For example, if the display system supports the use of transparency as a color value, then the copy 1201 may be assigned this transparency color value, and when this copy 1201 is overlaid upon the enlarged version, the interior of the enlarged version becomes transparent in color, allowing background images to appear. This effect may be achieved even in systems that do not directly support the use of such transparency values. For example, rather than simply assign the copy 1201 to a single background color, the copy 1201 may have the appearance of the graphical image, text, etc. that would have appeared directly behind the original object 1001. Alternatively, the copy 1201 may be assigned a predetermined color, such as white, to simplify the processing required to generate the halo. Using the background minimizes the amount of visual real estate that is obscured by the halo, while using a predetermined color simplifies the processing involved. One additional advantage to allowing the background pattern to show lies in the readability of text that already has another highlighting (such as a yellow highlight) associated with it. If the text had already been highlighted, then allowing this highlighting to show through the hollow body of the halo helps preserve the meaning associated with that highlighting. The user is still able to see that the selected text had the highlight associated with it.
A similar process of “overlaying” versions may be used to create the “glowing” effect of the halo shown in
In further embodiments, the original graphical object 1000 may have its width changed before the various copies and enlarged versions are created. For example, the object 1000 may originally have a varying width, but for purposes of generating the halo, it may have its width changed to that of a constant value. This constant value may be predefined, or may simply be the average width calculated along various points of the center line 1001. Using such a constant width may simplify the processing required to generate the various copies and enlarged versions, and is particularly useful when the original graphical object was nearly of constant width to begin with (e.g., if the original graphical object was constructed using a linear stroke of a stylus).
In some embodiments, the various steps of “overlaying” described above may be achieved using a version of the object(s) stored in a memory or display buffer. The overlaying may be performed on a pixel-by-pixel basis by replacing the individual pixels of a portion of one image with those of another image.
As an alternative, the graphical halo may be generated using the outer contour of the graphical object to define the contour that is to be followed by the halo. To generate the halo, the computer system may calculate the path to be taken by the halo, and render just that path on the screen to create the halo effect. The halo may be rendered by tracing a single line around the contour. In such an embodiment, no processing is necessary for the “hollow” portion of the halo, but greater processing is needed to trace the path. In some embodiments, the halo outline may be anti-aliased for improved resolution.
a-b are photographic images of one example embodiment. In
a-b are photographic images of another example embodiment. In
Various embodiments and aspects of the present invention have been described above, and it will be understood by those of ordinary skill that the present invention includes within its scope all combinations and subcombinations of these embodiments and aspects. The full scope of the present invention should only be limited by the claims, which read as follows:
The present application is a divisional application of U.S. application Ser. No. 10/144,256, filed May 14, 2002, entitled “Method and Apparatus for Hollow Selection Feedback,” the specification of which is hereby incorporated by reference.
| Number | Name | Date | Kind |
|---|---|---|---|
| 5339387 | Frankel | Aug 1994 | A |
| 5410331 | Schuneman | Apr 1995 | A |
| 5442742 | Greyson et al. | Aug 1995 | A |
| 5459831 | Brewer et al. | Oct 1995 | A |
| 5577170 | Karow | Nov 1996 | A |
| 5631975 | Riglet et al. | May 1997 | A |
| 5977981 | Brown | Nov 1999 | A |
| 5982381 | Joshi et al. | Nov 1999 | A |
| 6057858 | Desrosiers | May 2000 | A |
| 6091505 | Beaman et al. | Jul 2000 | A |
| 6243093 | Czerwinski et al. | Jun 2001 | B1 |
| 6337698 | Keely, Jr. | Jan 2002 | B1 |
| 20010020956 | Moir | Sep 2001 | A1 |
| 20020046386 | Skoll et al. | Apr 2002 | A1 |
| 20020064308 | Altman et al. | May 2002 | A1 |
| 20080072168 | Conway et al. | Mar 2008 | A1 |
| 20080155439 | Stern et al. | Jun 2008 | A1 |
| Number | Date | Country |
|---|---|---|
| 0454477 | Oct 1991 | EP |
| 0225211 | Mar 2002 | WO |
| Number | Date | Country | |
|---|---|---|---|
| 20050149882 A1 | Jul 2005 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 10144256 | May 2002 | US |
| Child | 11067949 | US |
