Patent Application
20150301697
This invention relates to computing devices, in particular to a portable computing device with a touch screen for providing a selection method for menu structures. The invention further relates to a method, an apparatus and a computing device program product for providing a selection method for menu structures of a computing device.
Portable computing devices, such as mobile phones or smart phones, with physical key(s) or a physical keyboard can be used without looking at a display of the device since the keys could be felt with fingers. In addition, for easy handling some of the keys are designed differently or the keys are indicated by bossages or the like to indicate the location of certain keys. However, portable computing devices with touch screen keys cannot be used without looking at a display of the device, because touch screen keys are arranged to be used by presence and location of a touch within a certain area of the display.
Conventionally, menu structures in portable computing devices with physical keys or touch screen keys have been implemented as pop-up menus which appear when, for example, a specified key or a touch screen is touched in a specified place such as at the bottom of the screen. From the pop-up menus, users can make a selection by mouse, by physical keys or by touching the wanted menu item or items. The type of selection depends on the type of the portable computing device.
The present invention relates to a method for a computing device, for example, for a portable computing device with a touch screen. The user of the computing device may handle menu structures and selection of menu options of a graphical user interface of the computing device by touch. The user may open menu structures by touching to an arbitrary point on the touch screen and select menu options from the open menu structure by releasing the touch when the menu option within or near a selection area of the menu structure. The computing device may be a computer, such as a smart phone, having a touch screen and running an operating system, such as Linux, iOS or Android, or some other computing device or portable computing device with a touch screen, such as a game console, an all-in-one computer, a tablet computer, a personal digital assistant (PDA), a satellite navigation device, a mobile phone, or a video game.
Various aspects of the invention include a method for a computing device having a graphical user interface comprising a touch screen, an apparatus that is configured to perform the method and a computer program product comprising instructions to perform the method.
According to a first aspect of the invention, there is provided a method for a computing device having a graphical user interface comprising a display for displaying a user interface view and a touch screen for receiving input from a user. The method comprises displaying a user interface element on the display, opening a menu structure on a side of the user interface element by a touch on the touch screen, wherein the menu structure comprises at least one menu option and a selection area and wherein said at least one menu option is movable in comparison to the selection area, and activating a menu option that is closest to the selection area.
According to an embodiment, the menu structure that is closest to the selection area is activated when the touch is detected to be released from the touch screen. According to an embodiment, the method further comprises indicating an openable menu structure by a menu indicator on a side of the user interface element. According to an embodiment, the menu structure indicated by the menu indicator is when the detected touch is detected to move away from the menu indicator. According to an embodiment, the method further comprises indicating movement of at least one menu option over the selection area by visual, audio and/or haptic feedback. According to an embodiment, indicating the activation of the menu option from the menu structure by visual, audio and/or haptic feedback. According to an embodiment, the method further comprises indicating a second menu indicator on a second side of the user interface element if a menu structure is available. According to an embodiment, the user interface element is a photo, a lock screen view or any other view of a terminal device, a web page, a calendar application or a list of phone numbers, phone history, bookmarks, dates, or photos of photo gallery. According to an embodiment, the menu option that is closest to the selection area is displayed, other menu options are invisible. According to an embodiment, the method further comprises providing a feedback for the user when the menu option is activated. According to an embodiment, the method further comprises locking the menu structure open and providing a feedback for the user when the end of the menu structure is reached. According to an embodiment, the feedback provided for the user when the end of the menu structure is reached is different than feedback provided for the user when the menu option is activated.
According to a second aspect of the invention, there is provided an apparatus comprising a processing unit, a memory coupled to said processing unit and a graphical user interface comprising of a display for displaying a user interface element and a touch screen for receiving input from a user. The memory is configured to store computer program code and a user interface data. The graphical user interface is coupled to said memory and data processing unit. The processing unit is configured to execute the program code stored in the memory, whereupon the apparatus is configured to display a user interface element on the display, open a menu structure on a side of the user interface element by a touch on the touch screen, wherein the menu structure comprises at least one menu option and a selection area and wherein said at least one menu option is movable in comparison to the selection area and activate a menu option that is closest to the selection area. The apparatus may be an example of a computing device.
According to an embodiment, the menu structure that is closest to the selection area is activated when the touch is detected to be released from the touch screen. According to an embodiment, the apparatus is further arranged to indicate an openable menu structure by a menu indicator on a side of the user interface element. According to an embodiment, the menu structure indicated by the menu indicator is opened when the detected touch is detected to move away from the menu indicator. According to an embodiment, the apparatus is further arranged to indicate movement of at least one menu option over the selection area by visual, audio and/or haptic feedback. According to an embodiment, the apparatus is further arranged to indicate the activation of the menu option from the menu structure by visual, audio and/or haptic feedback. According to an embodiment, the apparatus is further arranged to indicate a second menu indicator on a second side of the user interface element if a menu structure is available. According to an embodiment, the user interface element is a photo, a lock screen view or any other view of a terminal device, a web page, a calendar application or a list of phone numbers, phone history, bookmarks, dates, or photos of photo gallery. According to an embodiment, the menu option that is closest to the selection area is displayed, other menu options are invisible. According to an embodiment, the apparatus is further arranged to provide a feedback for the user when the menu option is activated. According to an embodiment, the apparatus is further arranged to lock the menu structure open and provide a feedback for the user when the end of the menu structure is reached. According to an embodiment, the feedback provided for the user when the end of the menu structure is reached is different than feedback provided for the user when the menu option is activated. According to an embodiment, the apparatus that is a computing device is a smart phone.
According to a third aspect of the invention, there is provided a computer program product, stored on a computer readable medium and executable in a data processing device comprising a graphical user interface comprising a display for displaying a user interface element and a touch screen for receiving input from a user. The computer program product comprises instructions to display a user interface element on the display, open a menu structure on a side of the user interface element by a touch on the touch screen, wherein the menu structure comprises at least one menu option and a selection area and wherein said at least one menu option is movable in comparison to the selection area and activate a menu option that is closest to the selection area and activate a menu option that is closest to the selection area.
According to an embodiment, the menu structure that is closest to the selection area is activated when the touch is detected to be released from the touch screen. According to an embodiment, the computer program product further comprises instructions to indicate an openable menu structure by a menu indicator on a side of the user interface element. According to an embodiment, the menu structure indicated by the menu indicator is opened when the detected touch is detected to move away from the menu indicator. According to an embodiment, the computer program product further comprises instructions to indicate movement of at least one menu option over the selection area by visual, audio and/or haptic feedback. According to an embodiment, the computer program product further comprises instructions to indicate the activation of the menu option from the menu structure by visual, audio and/or haptic feedback. According to an embodiment, the computer program product further comprises instructions to indicate a second menu indicator on a second side of the user interface element if a menu structure is available. According to an embodiment, the user interface element is a photo, a lock screen view or any other view of a terminal device, a web page, a calendar application or a list of phone numbers, phone history, bookmarks, dates, or photos of photo gallery. According to an embodiment, the menu option that is closest to the selection area is displayed, other menu options are invisible. According to an embodiment, the computer program product further comprises instructions to provide a feedback for the user when the menu option is activated. According to an embodiment, the computer program product further comprises instructions to lock the menu structure open and provide a feedback for the user when the end of the menu structure is reached. According to an embodiment, the feedback provided for the user when the end of the menu structure is reached is different than feedback provided for the user when the menu option is activated. According to an embodiment, the data processing device is a smart phone.
For a more complete understanding of example embodiments of the present invention, reference is now made to the following descriptions taken in connection with the accompanying drawings in which
a-c show views of a display of a touch screen apparatus according to an embodiment of the invention;
a-b show views of a display of a touch screen apparatus according to an embodiment of the invention;
a-f show views of a display of a smart phone having a touch screen according to an embodiment of the invention;
a-b show views of a display of a smart phone having a touch screen according to an embodiment of the invention
a-b show steps of selecting method by views of a display of a smart phone having a touch screen according to an embodiment of the invention;
a-b show views of a display of a smart phone having a touch screen according to an embodiment of the invention; and
A touch screen is an electronic visual display of a terminal device that can detect the presence and location of a touch within the area of the display. Usually, users touch the touch screen of the device with a finger/fingers or a hand, or in some cases with a stylus. Nowadays, touch screens are more common in portable computing devices such as game consoles, all-in-one computers, tablet computers, smart phones, personal digital assistants (PDA), satellite navigation devices, mobile phones, and video games. There is a trend especially in smart phones to implement terminals with only few or no real hardware buttons. All or most of the functionalities are controlled with so called on-screen buttons, on-screen toolbars or software menu structures.
The touch screen enables the user to interact with the computing device of the device directly with what is displayed i.e. with on-screen buttons. The user selects, for example, by a finger, on-screen buttons, for example, displayed icons or keys. When the touch screen is used, no additional device is needed, i.e. there is no need for external hardware such as a mouse or a touchpad through which a pointer can be indirectly controlled, or an external keyboard. Touch screen displays are popular, for example, in personal electronics, in the medical field, in heavy industry, and in kiosks where keyboard and mouse systems do not allow a suitably intuitive, rapid, or accurate interaction by the user with the content on the display.
However, on-screen functionalities of touch screen devices, i.e. on-screen buttons, toolbars or software menu structures use some of the area of the display, thus limiting the amount of information which can be presented on the display for the user. Traditionally, this problem has been solved by increasing the screen size or by decreasing the size of information displayed on the screen. However, the larger screen size leads to larger manufacturing costs of the terminal, and the decreased size of presented information reduces the usability characteristics of the device.
Additionally, the use of touch screens of conventional portable computing devices with on-screen buttons, toolbars or software menu structures requires the user to look at the screen when using the touch screen and to tap exactly on the on-screen control places i.e. a defined area on the touch screen. This leads to another kind of usability problem. The device cannot be used in a safe manner while moving, driving or cycling etc. or in a comfortable manner while watching TV, discussing with other people or doing something that requires the use of eyes but leaves the hands free. For example, tapping and then moving out of the defined area of the on-screen control normally discards the started action. In addition, existing touch screen devices give little or no possibilities to be used by visually impaired people, because it might be impossible or at least hard to find on-screen control locations on the touch screen. In addition, the screen size in some of the devices might be dimensioned in a way that it is not possible to reach on-screen buttons with one hand or without moving the device within the user's hand.
These problems can be solved by a graphical user interface of a computing device, wherein menu structures comprising menu options such as actions and toolbar items can be opened and scrolled via a touch interface such as a touch screen by an upward and/or downward and/or leftward and/or rightward movement of a finger of a user, and wherein menu options can be selected by releasing the finger when a menu option is within a selection area of the menu structure. Releasing of the finger activates the menu option that is within the selection area, and the menu structure may collapse using spring animation. In the other words, an upward, downward, leftward or rightward movement of the finger opens the menu structure indicated by a menu indicator on the edge of a user interface element; i.e., the movement of the finger away from the menu indicator opens the menu structure. For example, the menu opens when the user scrolls up (moves the finger down) when a top menu indicator is shown on the top of the screen i.e. above the user interface element, or scrolls down when a bottom menu indicator is shown on the bottom of the screen i.e. under the user interface element, or scrolls left when a left menu indicator is shown on the left side of the screen i.e. on the left side of the user interface element or scrolls right when a right menu indicator is shown on the right side of the screen i.e. on the right side of the user interface element. The computing device may be a computer, such as a smart phone, having a touch screen and running an operating system, such as Linux, iOS or Android, or some other computing device or portable computing device with a touch screen, such as a game console, an all-in-one computer, a tablet computer, a personal digital assistant (PDA), a satellite navigation device, a mobile phone, or a video game. The user interface element may be, for example, a photo, a lock screen view or any other view of a terminal device, a web page, a calendar application or any other application of a terminal device, a list of, for example, phone numbers, phone history, bookmarks, dates, photos of photo gallery, etc. Here the term “on the side of a user interface element” refers to a top border area, a bottom border area, a left border area and/or a right border area of the user interface element.
According to another embodiment, it is also possible to open a menu structure in the middle of the user interface element, for example, in the middle of the list of lit items by dragging list items from both sides of the intermediate menu indicator by two fingers when a menu structure is available in the middle of the list and indicated by the intermediate menu indicator. Here the term middle of the user interface element refers to a place that is not in the border area of the user interface element, for example, in the beginning or in the end of the list.
According to another embodiment, indicators can be shown on all sides of the user interface element at the same time or on two or three or four sides at the same time, for example, in case of moving a web page on the screen. When the top, the bottom, a side; left end or right end, is reached, a menu corresponding menu indicator, top menu indicator, bottom menu indicator, left menu indicator or right menu indicator, can be open and shown.
According to another embodiment, there may be one or more openable menu structures on the different sides of a user interface element that are not indicated by an indicator. According to another embodiment, one or more menu structures are indicated by the indicator and one or more are not. For example, a menu structure on the top side of a user interface element is not indicated by an indicator, but another menu structure on the left side of a user interface element is indicated by a left menu indicator.
Further, according to another embodiment, a menu indicator can be provided at any arbitrary point of any user interface element if there is scrollable content (menu structure) in context with the menu indicator.
Menu structures according to the invention include a selection area for selecting menu options of the menu structure. The desired menu option can be arranged to the selection area by a scrolling action. Scrolling of the menu options over the selection area and selection of a menu option from the menu structure may cause visual, audio and/or haptic feedback for the user. It is also possible, that the apparatus reads the menu options and/or the selected menu option.
Menu options of a menu structure may be associated with actions such as making a phone call, launching a keypad, sending a message, modifying an item, playing music, going to previous page (when using the menu in browser environment), going to next page, entering Uniform Resource Locator, previous pages in web browsing, starting another application, closing current application, saving status, toggling to other application, etc. There can also be additional info relating to the displayed content, like week days on Calendar view, metadata of photos etc. The menu options may relate to an application that is run by the apparatus, and those options may be default options that are prearranged into menu structures. However it is also possible that the user may select his/her own menu options for each application or views of the apparatus.
However, the common feature for all openings of menu structures indicated by menu indicators and selecting menu options from the menu structure is that there is no need to touch a certain place on the touch screen. It is sufficient to touch the screen and to move the touch (finger) in the desired direction.
An example embodiment of the present invention and its potential advantages are understood by referring to
The apparatus according to
The apparatus 100 further comprises a memory (
a shows a view of a display of a touch screen apparatus according to an embodiment of the invention. The apparatus 300 has a graphical user interface 302 comprising the display for displaying a user interface view and information, and a touch screen for receiving input from a user. The touch screen may be controlled, for example, with a finger 306. In this embodiment there is disclosed a general list 304 as the user interface element. The list 304 comprises 9 list items; List items 1 to 9. The list items may be for example phone numbers from the phone directory of the phone, links from a web site, photos of a photo gallery, emails of an email service, calendar marks of calendar application, or any other items which relate to each other in some way and which are suitable to be shown as a list. As shown by a bidirectional arrow 307 in
b shows a view of a display of a touch screen apparatus according to an embodiment of the invention, where the user has scrolled up the list 304 of
c shows a view of a display of a touch screen apparatus according to an embodiment of the invention, where the user has scrolled down the list 304 of
The menu indicators 308, 310 may be, for example, graphical elements, such as an illuminated line, a special coloured line, a glow etc. which is shown, for example, on the top or bottom of the screen.
a-b show views of a screen of a touch screen apparatus 300 according to an embodiment of the invention, where the user has further scrolled up the list 304 with the finger 306, i.e. moved the finger down after the list 304 has ended, and the menu indicator has displayed above the list items of the list 304 on the user interface 302 (menu indicator is shown in
The menu 402 may have a defined selection area, i.e. a zone 408. The selection area 408 may be visually indicated, for example, by a graphical element, such as an illuminated line, a special coloured line, a special coloured menu option text, a glow or by a dashed line as in
In addition, it is also possible that when the user scrolls the list 404 down with the finger 306, i.e. moves the finger 306 up after the list 404 has ended, a pull-down menu appears under the list items of the list 404; i.e. the list 404 moves upwards so that the menu 402 may be shown under the last list item (List Item 9). The menu 402 may comprise the same or different menu options as the above mentioned pull-down list above the menu 402. In the pull-down menu, under the list items of the list 404 there is also a selection area. The number of menu items in the pull-down menu is not restricted.
Thus, the above mentioned ways of displaying menu options give a possibility to select menu items in fast manner and without touching a certain place on the touch screen. In a preferred embodiment, the apparatus indicates, for example, by audio or by vibrating, that a menu item is within a zone 408, thus enabling the user to activate menu options without looking at the display. In addition, the apparatus may also indicate the selection of this menu option.
a-f show views of a display of a smart phone having a touch screen according to an embodiment of the invention. The smart phone 500 has a graphical user interface comprising the display for displaying a user interface view and information and a touch screen for receiving input from a user.
a shows an embodiment where a photo is displayed as a user interface element on the display of a smart phone 500. In the display, menu indicators, top menu indicator 502 and down menu 514 and left side indicator 516 are also displayed, indicating that there are menu structures to be used. The user has to move down the finger at some point on the touch screen and opened a pull-down menu 504 indicated by the top menu indicator 502. The photo is shrunk so that the menu 504 can be shown. Alternatively, the photo can be shifted so that the menu can be shown, for example, the photo may be shifted downward, upward, leftward or rightward depending on the opening direction of the menu. The menu 504 displays two menu options, Send option and Set to Wallpaper option. There is also indicated a selection area 505 by dashed lines and also by an illuminated line 506. The Set to Wallpaper option is within the selection area 505. There is also an empty space in the menu 504. In this embodiment, this empty space is a so called Close Menu option 508. If the Close Menu option 508 is within the selection area 505 when the user releases his/her finger from the touch screen, the menu 502 will be closed. It is also possible to show the name of this Close Menu option 508.
b shows an embodiment where a photo is displayed on the display of the smart phone 500. On the display, a menu indicator 502 is also displayed, indicating that there is a pull-down menu structure on top of the screen to be used. The user has to move down the finger on the touch screen and opened the pull-down menu 504 indicated by the top menu indicator 502. The menu 504 is displayed translucently on top of the displayed photo. The menu 504 comprises three menu options, Send option, Set to Wallpaper option and Close Menu Option. A selection area 505 is also indicated by dashed lines. The Close Menu option is within the selection area 505. There is also an empty space 510 above menu options in the menu 504. In this embodiment, this empty space locks the menu 504 if the empty space is within the selection area 505 so the user may tap by the finger 306 to activate a menu option. However, it is also possible that menu 504 is displayed non-translucently on top of the displayed photo or on top of displayed on-going action. The on-going action may be, for example, a phone call, Internet browser application, web application, messaging application such as email, photo sharing application, multimedia messaging service application, short message service application etc.
c shows an embodiment of the invention where a photo is displayed on the display of the smart phone 500. In this embodiment only the menu option which is within the selection area 505 is shown. Thus only Set to Wallpaper option is shown, but Close Menu 508 and Send 512 options are not shown. However, when the finger is moved further down, the Set to Wallpaper option will disappear and the Close Menu Option 508 will be shown. However, it is also possible that all menu options are shown but the one which is within the selection area 505 is highlighted with a certain colour or some other indicator.
d shows an embodiment where a photo is displayed on the display of the smart phone 500. In this embodiment the selection area 505 is only displayed by an illuminated line 506. In this embodiment, when the user releases the finger when the “Send” option is within the selection area 505 a new menu options might be presented or messaging application such as email, photo sharing application, or multimedia messaging service application might be launched depending on settings.
e shows an embodiment where a photo is displayed on the display of the smart phone 500. On the display, a menu indicator 514 is also displayed which indicates that there is a menu structure to be used. The user has moved up the finger on the touch screen and opened a pull-down menu 509 indicated by the down menu indicator 514. The photo is shrunk so that the menu 504 can be shown under the photo. Additionally the photo can be moved upward so that the photo is shown only partially. The menu 509 displays three menu options, Delete option, Information option and Close option. A selection area 505 is also indicated by dashed lines and also by an illuminated line 506. The Information option is currently within the selection area 505.
f shows an embodiment where a photo is displayed on the display of the smart phone 500. On the display, two menu indicators, top menu indicator 502 and down menu indicator 514 are also displayed, indicating that there are menu structures to be used. The user has moved up the finger on the touch screen and opened a pull-down menu 509 indicated by the down menu indicator 514. The menu 509 is displayed translucently on top of the displayed photo. There are three menu options displayed on the menu 509, Delete option, Information option and Close option. A selection area 505 is indicated by an illuminated line 506. The Delete option is currently within the selection area 505.
a and 6b show views of a display of a smart phone having a touch screen according to an embodiment of the invention. The smart phone 600 has a graphical user interface comprising the display for displaying a user interface view and information, and a touch screen for receiving input from a user.
a shows an embodiment where a lock screen view is displayed on the display of the smart phone 600. On the display, a menu indicator and a top menu indicator 602 are also displayed, indicating that there are menu structures to be used.
b shows an embodiment where a lock screen view is displayed as a user interface element on the display of the smart phone 600. On the display, a pull down menu 604 that has been indicated by the top menu indicator 602 is also displayed. The user has opened the pull-down menu 604 by moving down the finger at some point on the touch screen. The menu 604 displays three menu options 608 in this view: Silent profile, Phone and Camera options. There is also indicated an additional information area 610 comprising information about the date. This additional information may also be a menu option that starts a calendar application. The selection area 605 is indicated by an illuminated line 606, and the Phone option is currently within the selection area 605.
a, 7b and 7c show steps of a method for selecting a menu option by views of a display of a smart phone 700 having a touch screen according to an embodiment of the invention. The apparatus 700 has a graphical user interface comprising the display for displaying a user interface view and information, and a touch screen for receiving input from a user.
Call to menu option was within the selecting area 705. After selection of menu option, the menu 704 is closed and the selected menu option is activated. In this embodiment, activation of Call to menu option launches a phone number dial pad 708 and opens an area for a phone number to be entered 710. Different menu options activate different applications.
a and 8b show a generic overview of the user interface views of a display of a computing apparatus having a touch screen according to an embodiment of the invention. The apparatus 800 has a graphical user interface comprising the display for displaying a user interface view and information, and a touch screen for receiving input from a user.
In an embodiment shown in
In an embodiment shown in
Alternatively, in
The options and features of one embodiment may also be used in context with other embodiments, for example, options and features shown in embodiment of
According to a further embodiment, top menu and bottom menu indicators and menu structures can also be used in Internet browser applications. For example, when user is in the top part of a web page a top menu indicator is shown or when the user scrolls the HTML (Hyper Text Markup Language) document down menu items relevant for web browsing can be shown. Shown items can be, for example, “Enter URL” (Universal Resource Locator), “Go back”, “Bookmark”, “Share the page” etc.
According to further embodiments, information content of a web page (as defined in HTML or similar file) can include instructions on what menu options should be in menu structures according to the invention and downloadable applications may contain menu structures and indicators according to the invention.
The term “on the side of a user interface element” refers to a top border area, a bottom border area, a—left border area and/or a right border area of the user interface element. According to a further embodiment when the menu indicators are on left or right edge/side of the display the menus might be scrollable from left to right or from right to left.
The various embodiments of the invention can be implemented with the help of computer program code that resides in a memory and causes an apparatus to carry out the invention. For example, the apparatus that is a computing device may comprise circuitry and electronics for handling, receiving and transmitting data, a computer program code in a memory, and a processor which, when running the computer program code, causes the apparatus to carry out the features of an embodiment.
It is obvious that the present invention is not limited solely to the above-presented embodiments, but it can be modified within the scope of the appended claims.
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/FI2012/051143 | 11/20/2012 | WO | 00 |
