Patent Application
20110231793
1. Field of the Invention
The invention relates to an improved user interface, and particularly but not exclusively to a user interface presented in combination with an interactive display surface.
2. Description of the Related Art
Interactive display systems are well-known. In an interactive display system, a user (or users) interact with a display surface on which an image is projected. In one known environment, the interactive display surface may be a display surface of an electronic whiteboard, which is used in a classroom environment for educational purposes.
In such systems, the user stands at or close to the display surface, and interacts with the display surface. Different types of interactive display surface are possible, and the user may interact with the surface by using a finger in a touch-sensitive system, or by using a pointer. Where a pointer is used, the interaction between the pointer and the display surface may be by means other than touch-sensitive means.
In such systems, the use of the pointer (or finger) at the interactive display surface may be for the same purpose as a mouse in a desktop computer system. The user uses the pointer to control a cursor displayed on the display screen, and to select icons and tools displayed on the display screen. In this way the user can manipulate the information displayed in the same manner as they may manipulate information using a desktop computer, but the manipulation takes place at the display on which information is displayed to a classroom. In this way the display is an electronic whiteboard.
It is known in the art to provide pointers for use with such interactive display systems with buttons, which buttons can be used to simulate “mouse clicks”. It is also known in the art to use pressure-sensitive pointers, which can be used to simulate “mouse clicks”.
Whilst there is provided in the art pointers which are adapted to allow the functionality of a mouse to be replicated, when a user is using a desktop computer they may also use one or more keyboard keys in combination with using a mouse or mouse buttons to select certain functionality. In an interactive display system, the use of the keyboard is generally not possible, and is generally undesirable as the purpose of the interactive display is for the user to be able to stand at or close to the display surface and not use a keyboard.
Furthermore there is a need in the art to provide a greater degree of control over certain functionality. In the art certain default actions may be implemented in a given circumstance, with no possibility to control or modify such default action.
It is an aim of the invention to provide a technique which addresses one or more of the above-stated problems.
In one aspect the invention provides a method of controlling a user interface comprising the steps of: positioning a cursor over a displayed object and dragging the cursor; and determining a state of a mode of operation, wherein: in a first state, the object under the cursor is dragged with the cursor; and in a second state, a selection area is dragged with the cursor.
The selection area may be a marquee selection box. In said second state any item which intersects the selection area may be selected. In said second state any item which is contained within the selection area may be selected.
The mode of operation may be toggled between the first and second states.
The state of said mode may be toggled by selection of any one of: an icon of a graphical user interface; a keypad of a keyboard; or a button of a pointing device.
The method may comprise determining the state of a further mode of operation when a selection area is formed, comprising: in a first state of said further mode, any object selected by the selection area replaces one or more previously selected objects; in a second state of said further mode, any object selected by the selection area is added to or removed from a selection set.
In the second state of the further mode if an object selected by the selection area is already in said selection set, it may be removed from said selection set. In the second state of the further mode if an object selected by the selection area is not in said selection set, it may be added to said selection set.
The cursor may be controlled by a pointer on an interactive display surface.
In another aspect the invention may provide a computer system comprising: means for positioning a cursor over a displayed object and dragging the cursor; and means for determining a state of a mode of operation, wherein: in a first state, the object under the cursor is dragged with the cursor; and in a second state, a selection area is dragged with the cursor.
In another aspect the invention may provide method of controlling a user interface comprising the step of: defining a selection set, being a set of one or more selected objects; selecting an object; and determining a state of a mode of operation, wherein: in a first state any selected object forms the selection set; and in a second state any selected object is added to or moved from the selection set.
The step of selecting an object may comprise: dragging a cursor on a display over one or more displayed objects to define a selection area; and wherein the steps of determining the state of the mode of operation includes: in the first state any object determined to be in the selection area forms the selection set; in the second state any object determined to be in the selection area is added to or removed from the selection set.
In the second state any selected object determined to be in the selection set may be removed from the selection set. In the second state any selected object determined to not be in the selection set may be added to the selection set.
In the first state any selected object may replace any object in the selection set.
Any object which is encompassed by the selection area may be defined to be in the selection area.
Any object which intersects the selection area may be defined to be in the selection area.
The method may further comprise the step of determining a state of a preliminary mode of operation, wherein in a first state of said preliminary mode of operation an object under the cursor is dragged as the cursor is dragged, and in a second state of said preliminary mode of operation the selection area is defined.
In another aspect the invention may provide a computer system comprising: means for defining a selection set, being a set of one or more selected objects; means for selecting an object; and means for determining a state of a mode of operation, wherein: in a first state any selected object forms the selection set; and in a second state any selected object is added to or moved from the selection set.
The invention will now be described by way of example with reference to the accompanying figures in which:
a and 2b illustrate a first mode of operation in a first state at a user interface in accordance with an exemplary embodiment of the invention;
c and 2d illustrate the first mode of operation in a second state at a user interface in accordance with an exemplary embodiment of the invention;
The invention is described herein by way of reference to specific preferred embodiments and implementations. One skilled in the art will appreciate that the invention is not limited to the specifics of any arrangement described herein. In particular the invention is described herein in the context of an exemplary interactive display system, and one skilled in the art will appreciate that the invention is not limited to the specifics of the described interactive display system. The invention is in general advantageously applicable to any arrangement in which a pointing device (which may be a physical device, a user's finger) interacts with a display surface, but is not limited to such arrangements.
With reference to
The operation of interactive display systems such as that illustrated in
The invention is now described by way of reference to exemplary embodiments.
The principles of embodiments of the present invention will now be illustrated by way of example with reference to
Embodiments of the invention provide for first, second and third modes of operation. Each mode toggles between one of two states. In one state the mode of operation is selected (or enabled), and in another state the mode of operation is deselected (or disabled).
Preferably, the toggling between the two states of a mode of operation is achieved by displaying on the display screen an icon associated with the graphical user interface. By selecting the icon a user can switch or toggle between the two states of the mode of operation. Thus the mode of operation is always either enabled or disabled.
This is particularly advantageous in an interactive display system, where a user interacts with the display surface. The user may use a pointer, for example, to select the displayed icon and toggle between the two states of the mode of operation.
The toggling between states may be achieved in other ways. For example in an interactive display system a button provided on a pointing device may be selected to toggle between states. In certain embodiments, a key or a keyboard may be depressed to toggle between states.
With respect to
With respect to
The operation of the second state of the first mode of operation is described with reference to
In the second state of the first mode of operation the dragging of the cursor from one position to another position creates a selection box, as denoted by dash-line rectangular box 304 in
Thus it can be understood, from the description of
It should be understood that while embodiments of the invention are particularly advantageous for use in interactive display systems where the user interacts with an interactive surface using a pointing means, it is not limited to such. The invention may still be implemented in an arrangement where a mouse or other input device is used in association with a computer to control a cursor on a display screen, the mouse or other input device being used to control the cursor to toggle between states.
This first mode of operation, switchable between two states, as described above with reference to
It should be noted that whilst reference is made to a marquee or selection ‘box’, in embodiments different shapes may be created by dragging the cursor, such as a lasso shape. In general, there is created a marquee or selection area or shape.
The described first mode of operation, switchable between two states, is a new mode of operation. In the prior art, moving the cursor as shown in
In a further embodiment a second mode of operation also having two states is provided, in addition to the mode described above. As with the mode described above, in one state this mode is enabled, and in the other state it is disabled. This further mode of operation may be referred to as a “crossing mode” of operation. As with the first mode, an icon may be displayed on the display screen which can be selected by the mouse, selecting the icon toggling the mode between a first and second state, being enabled and disabled states.
When toggled on, in this second mode of operation any items which cross or intersect the marquee selection area, when marquee mode is enabled, will be selected. When toggled off, only those items which are completely enclosed by the marquee selection area will be included in the selection.
The second (crossing) mode of operation can be further understood with reference to
As can be seen in
With the second mode of operation enabled in the first state, the object 208 is selected by the selection area. With the second mode of operation disabled in the second state, the object 208 is not selected by the selection area.
In one arrangement, for the object to be wholly encompassed within a selection area, its entire area and edges must be within the selection area.
Thus this additional second mode of operation is preferably provided as an augmentation to the marquee mode of operation described above. However this second mode of operation, the crossing mode, may also be provided independently of the first (marquee) mode as a stand-alone mode.
A third mode of operation, referred to as “collect mode”, may preferably be implemented as a further augmentation, either for the marquee mode or the crossing mode (or both). However the third (collect) mode may also be provided independently of the first (marquee) or second (crossing) modes.
For this third mode of operation, again two states are provided which can be toggled between, in one state the mode being enabled and in the other state the mode being disabled. As with the modes described above, an icon may be displayed on the display screen associated with this mode, selection of the icon toggling between the two states.
The third (collect) mode of operation can be further understood with reference to
As can be seen in
With the third mode of operation enabled, in the first state, the selected objects 206 and 208 are added to a previous selection of objects or, if already in that previous selection, removed from a previous selection of objects. With the third mode of operation disabled, in the second state, the selected objects 206 and 208 replace any previously selected objects.
When this third mode is enabled, the behaviour is to add items to the selection in the first mode, or remove items if they are currently selected, rather than replacing the current selection with a new item as in the second mode.
Whilst this third mode is described as requiring a selection area to select objects, this is an exemplary arrangement. An object may be selected, in addition or as an alternative, by ‘clicking on’ the object: i.e. moving the cursor over the object and using the selection mechanism provided by the system, such as depressing a pointer on an interactive surface.
This third mode of operation, when enabled, is as if the control key on a Windows system (or the command key on an Apple system) is being held down whilst the cursor is being moved.
This allows a functionality to be selected for a user operating at the display surface of an interactive display, which functionality can advantageously be selected without having to additionally use a key(s) of a keyboard. In the prior art, it is known to implement a function for providing a collect mode on dragging of a mouse by holding down a control key on a keyboard while dragging the mouse. Thus in the prior art the collect mode of operation is known, the selection between states of the collect mode of operation being achieved by selecting a key of a keyboard in addition to operating the mouse.
Thus in this mode of operation, the behaviour matches that which can be achieved when a keyboard is available, by holding down the control key on Windows and the command key on Apple Macintosh operating systems. Where a keyboard is not available, such as when stood at an interactive display surface, this option is particularly valuable.
In the third mode of operation, there is thus presented an example where toggling between states for a particular function enables simulation of the ‘control key’ on a keyboard being depressed.
Preferably this does not emulate to the operating system that the ‘control key’ is depressed. Rather the software application is adapted to respond to the toggling into the given state to enable the same functionality as would be enabled by the ‘control key’ being depressed. The toggling between states is achieved using the user interface rather than the ‘control key’.
With reference to
In step 502, a cursor is dragged on the display from a first position to a second position. This corresponds to the dragging of the cursor as shown in
In step 504, it is then determined whether the first mode, marquee mode, is active. This is determined by the state of the mode. If the state is such that the marquee mode is not active, then in a step 506 it is determined whether there is an object under the cursor. If there is an object under the cursor, then in a step 522 that object is dragged. The final position of the object corresponds to the point at which the cursor dragging is stopped.
If in step 506 it is determined that there is no object under the cursor, then in a step 508 a marquee shape, or selection area, is formed by the dragging of the cursor.
If in step 504 it is determined that marquee mode is active, determined by the appropriate state, then in step 508 the marquee shape, or selection area, is formed by the dragging of the cursor.
Upon release of the cursor following step 508, a selection area is defined. In a step 510 it is determined whether the second mode, being the crossing mode, is active. This is determined in accordance with the state of the crossing mode.
If it is determined that the crossing mode is not active, then in a step 512 only those objects which are determined to be contained inside the marquee shape or selection area are identified. As noted above, this determination may be made in a number of ways.
On the other hand, if it is determined in step 510 that the crossing mode is active, then in a step 514 all objects which are completely contained inside the selection area, and those which intersect with the selection area, are identified.
The identified objects then form a selection set, being the objects which have been selected as a result of the marquee mode of operation. The crossing mode of operation determines whether the marquee mode of operation selects only those objects which are determined to be contained within the selection area, or those which are determined to be not contained within the selection area (i.e. those which cross with the selection area).
After either of steps 512 or 514, in a step 516 it is determined whether the third mode of operation, being the collect mode, is active. Whether this mode is active is determined by the state of the mode of operation.
If it is determined that the collect mode is not active, then in a step 518 the object(s) selected by the selection area become the selection set. Any previously selected items are unselected, and the only selected items are those which have now been selected in the current selection area.
If it is determined that in step 516 the collection mode is active, then in a step 520 the objects selected in the selection area are added to or removed from the selection set. If an object in the current selection area is already part of the selection set, i.e. is already a selected object, then it is removed from the selection set. If an object identified in the current selection area is not part of the current selection set, then it is added to the selection area. In a step 520, therefore, the selection of the objects for the selection set are toggled in dependence upon whether they are selected in the current selection area. Thus an existing set of selected objects is modified in dependence on the objects identified in the current selection area.
With respect to the third mode of operation in the exemplary arrangement where it is combined with the first and second modes, the object selection is preferably determined by the selection area. In other arrangements the object selection, in the third mode, may simply be by selection of an object without a selection area.
With reference to
The display 702 may be integrated with the computer system or be external to the computer system. The display 702 may be, for example, a display of an interactive display system. The input device 714 may be integrated with the computer system or external thereto. The input device 714 may be a pointing device associated with an interactive display surface.
In other exemplary arrangements, the display 702 may be an integrated display of a personal data system (PDA) device or other form of portable computer system. The input device 714 may be an integrated keypad of a PDA, a keyboard associated with a computer system, or a touch surface. One skilled in the art will appreciate the possible options for providing inputs to different types of computer system, and for displaying data from different types of computer system.
The methods described hereinabove may be implemented on computer software running on a computer system. The invention may therefore be embodied as computer program code being executed under the control of a processor of a computer system. The computer program code may be stored on a computer program product. A computer program product may include a computer memory, a portable disk or portable storage memory, or hard disk memory.
The invention is described herein in the context of its application to a computer system forming part of an interactive display system. It will be understood by one skilled in the art that the principles of the invention, and the embodiments described herein, are not however limited to an interactive display system. The principles of the invention and its embodiments may be implemented in any computer system including a display and a user interface. The invention and its embodiments is also not limited to the use of a pointer or touch surface type arrangement in order to move a cursor on a display. The invention encompasses any technique for the movement of a cursor, including the movement of a cursor using a conventional computer mouse.
The invention has been described herein by way of reference to particular examples and exemplary embodiments. One skilled in the art will appreciate that the invention is not limited to the details of the specific examples and exemplary embodiments set forth. Numerous other embodiments may be envisaged without departing from the scope of the invention, which is defined by the appended claims.
