The present invention relates to a method for displaying graphical information on a screen of a device. The present invention also relates to a computer program product, a module and a device for displaying graphical information.
Software applications are usually presented on a display connected to a device controlled by a computer. The graphical user interface (GUI), which allows a user to interact with the device, typically comprises both information about the processing status of the software applications, as well as surrounding graphics that helps the user to understand and interpret the software applications.
Traditionally, GUI systems on personal computers or mobile phones present an image on the display using a software that directly controls the position and look of the applications. Each application is normally created by a programmer who has the ultimate control of where a certain graphical element should appear, what it should contain, and how it should behave. When an application is initialized, the GUI system presents all elements directly to the display on top of each other, and there are generally no efficient processes to implement movements or animations of the elements.
Also, programmers typically populate the elements with data that may actually never be displayed, just because the GUI system requires it.
To some extent, there are other methods for addressing the problem of creating graphically rich user interfaces. There are e.g. techniques allowing the graphical designer, instead of the programmer, to completely define the GUI. Even if such techniques may reduce the required processing power in comparison with how software applications are displayed traditionally, they still require a lot of memory due to the fact that the programmer needs to push all application data into the design level in order for it to be readily available for the designer.
Thus, it is necessary to optimize how software applications are presented on a device display.
US 2004/0021659 describes a system for generating an image. The image comprises subject graphics data, corresponding to the processing status of an application, and a GUI. The subject graphics data and the GUI data originate from a single graphics application program, and they are decoupled for purposes of rendering. The system includes a first graphics pipeline for rendering the subject graphics image, which can be thought of as the contents of a window in the graphics application. This yields rendered subject graphics data. The invention also includes a second graphics pipeline for rendering the GUI graphics. This yields rendered GUI graphics data. Third, the invention includes a compositor for compositing the rendered subject graphics data produced by the first graphics pipeline, and the rendered GUI graphics data produced by the second graphics pipeline.
Efficient methods for displaying software applications are particularly imperative within portable electronic devices. Such devices are restricted in comparison to desktop computers, e.g. with respect to the central processing unit, available memory, memory architecture, real-time operating system, and display resolution. Moreover, in most cases there are no hardware graphics accelerators in portable electronic devices. Thus, there is still a need for an efficient method for displaying software applications.
The word “application service” is herein referred to computer software configured to communicate with a graphical user interface presented on a screen. Thus, application services involve computer software that is capable of providing information to be presented to the user, e.g. a media player service, a contact manager service, a messenger service, a web browser service etc.
The word “item” is herein referred to an object that may be presented graphically to a user. In case of a media player, an item may e.g. be a navigation button, a window or a menu. Further, an item may also refer to a part of a navigation button, a part of a window or a part of a menu.
The word “module” is herein referred to a software module, a hardware module such as an ASIC, or a combination thereof such as a FPGA.
An “element” should be interpreted as being equal to an item.
In view of the foregoing, it is an object of the present invention to provide an improvement of the above techniques and prior art. More particularly, it is an object of the invention to provide an advanced graphical user interface while reducing the necessary processing power and memory resource. Another object of the invention is to provide a method which enables efficient displaying of animations, movements and effects.
At least some of the above objects are achieved by means of a method, a computer program product, a module and a device according to the independent claims. Specific embodiments of the invention are set forth in the dependent claims.
According to a first aspect of the invention, a method for displaying graphical information on a screen of a device is provided. The method comprises the steps of composing resulting image data from at least one application service, and transmitting the resulting image data to the screen of the device. The step of composing resulting image data further comprises identifying items associated with each of the at least one application service, determining at least one item that is in a visible state, fetching information associated with the at least one item that is in a visible state, and calculating the resulting image data from the fetched information. The method is advantageous in that several application services can be presented in an efficient manner with reduced processing power.
The step of identifying items may be performed prior to the step of determining at least one item that is in a visible state, and the step of determining at least one item that is in a visible state may be performed prior to the step of fetching information.
The step of fetching information may only fetch information associated with the at least one item that is in a visible state, which is advantageous in that minimum information is used to compose the resulting image data. Further, it is advantageous in that the items which are not in a visible state are excluded from being transmitted to the screen, whereby the quantity of information that is transmitted to the screen is reduced.
The step of fetching information may further comprise the steps of fetching data associated with each of the at least one item in the visible state from a first memory, and fetching graphical declarations associated with each of the at least one item in the visible state from a second memory. This enables programmers to define application data, and graphical designers to define the graphical appearance of the application services.
The step of fetching graphical declarations may further comprises the step of connecting to a remote server, wherein said server comprises the second memory. The appearance of application services can thus correspond to the appearance of e.g. user accounts on the internet.
The method may further comprise the step of calculating at least one attribute of each identified item, wherein the attribute is selected from a group consisting of position, size and rotation. This provides efficient handling of items when the appearance of the items are changed.
The step of determining at least one item that is in a visible state may further comprise the steps of determining items which are in a non-visible state, and determining items which are in a partially visible state. This is advantageous in that items are categorized in a feasible manner.
The method may further comprise the step of receiving command data corresponding to an event triggered by input data, wherein the step of composing resulting image data is repeated for every received command data. Thus, resulting image data can be transmitted to the screen when triggered by e.g. user input, moving animations etc., thus providing smooth animations and movements of graphical items.
According to a second aspect of the invention, a computer program product comprising program code means stored in a computer readable medium is provided. The program code means are adapted to perform any of the steps of the method according to the first aspect of the invention when the program is run on a computer. The advantages of the first aspect of the invention are also applicable to the second aspect of the invention.
According to a third aspect of the invention, a module for displaying graphical information on a screen of a device is provided. The module comprises a compositor configured to compose resulting image data from at least one application service, and a transmitter configured to transmit the resulting image data to the screen of the device. The compositor further comprises an identifier configured to identify items associated with each of the at least one application service, a determinator configured to determine at least one item that is in a visible state, a fetching means configured to fetch information associated with the at least one item that is in the visible state, and a calculator configured to calculate the resulting image data from the fetched information. The advantages of the first aspect of the invention are also applicable to the third aspect of the invention.
The fetching means may only fetch information associated with the at least one item that is in a visible state.
The fetching means may comprise a first fetching means configured to fetch data associated with each of the at least one item that is in the visible mode from a first memory, and a second fetching means configured to fetch graphical declarations associated with each of the at least one item that is in the visible mode from a second memory.
The second memory may be arranged on a remote server.
The module may further comprise a second calculating means configured to calculate at least one attribute of each identified item, wherein the attribute is selected from a group consisting of position, size and rotation.
The second determinator may be configured to determine items which are in a non-visible state, and to determine items which are in a partially visible state.
The module may further comprise a receiver configured to receive command data corresponding to an event triggered by input data, wherein the command data is configured to control the compositor.
According to a fourth aspect of the invention, a device is provided. The device comprises means for initializing at least one application service, a module according to the third aspect of the invention, and a screen configured to display the image data. The advantages of the first aspect of the invention are also applicable to the fourth aspect of the invention.
The device may be a mobile terminal.
According to a fifth aspect of the invention, a system comprising a device according to the fourth aspect of the invention is provided. The system further comprises a remote server which is connected to the device, wherein the remote server is storing information associated with the at least one application service. This is advantageous in that the appearance of application services can correspond to the appearance of e.g. user accounts on the internet.
Embodiments of the present invention will now be described, by way of example, with reference to the accompanying schematic drawings.
a-c shows different devices according to the fourth embodiment of the present invention.
In
The step of identifying items 22 may also comprise calculating parameters of the items. If one item change its size, this will probably affect the layout of surrounding items as well. Therefore, the position, size, rotation, color, opaqueness, etc are calculated for all items in the graphical scene.
After the step of identifying items 22, in step 24 the method determines which of the identified items are in a visible state. In this step, items which are positioned behind solid items or arranged outside the visible screen of the device are determined to be in a non-visible state. Consequently, in step 24 the method determines which of the identified items are in a visible state, a non-visible state, as well as which of the identified items are in a partially visible state. This can be applied to items in a two-dimensional (2D) space as well as to items in a three-dimensional (3D) space. In 2D space, two or more items may overlap but only the item in front will be determined to be in a visible state. However, in 3D space items may be overlapping or not overlapping depending on view angles, shadowing etc. Also in this case the item in front will be determined and set as being in a visible state.
In step 26 information associated with the items in the visible state and the partially visible state is fetched. More particularly, application service data is fetched from a hard coded memory. If one item in the visible state or the partially visible state is associated with a data reference pointer, the appointed data is fetched from a XML-file containing graphical declarations. In specific embodiments, the data reference pointer can also address an image, a video file, a sound file etc. Only information associated with the items determined to be in a visible state or a partially visible state is fetched.
The information is fetched from two different resources. Data associated with the application service 12a is fetched from a first memory, and graphical declarations associated with the application service 12a is fetched from a second memory. The second memory can e.g. be located on a remote server, and the graphical declarations can thus correspond to a personal profile on an internet account. The application data and the graphical declarations are used to compose the resulting image data.
In one embodiment, application data are hard coded instructions which are programmed in e.g. C or C++ by an application service programmer. The graphical declarations control the layout, appearance and animations of the software applications. The information associated with graphical declarations is represented by the XML-file which is defined by a graphical designer.
Next, the resulting image data is calculated in step 28. The calculating step 28 uses the fetched information as input, i.e. information associated with several items is composed to a single resulting image data. Thereafter, the resulting image data is transmitted to the screen of the device in step 29, wherein the method is repeated in order to provide a continuous user interface.
When providing a dynamic user interface, the method is repeated in order to provide smooth transitions and movements of items. A command data is first received, corresponding to an event triggered by input data. The input data may be a user input, such as the user pushing a button, or an initiated animation that is used to create a special effect. Thus, the step of composing resulting image data 20 is repeated for every received command data or during the length of an animation sequence.
The fetched information is stored temporarily in a cache. When the user interface is changed during a dynamic sequence, the cached information may be removed from the cache when the state of an item is changed from visible to non-visible. However, the composing step may further comprise a step of evaluating if a previously visible item will be visible again. Thus, if an item in the visible state is hidden by e.g. an animation, the composing step will evaluate the animation and the information will be retained in the cache so the information associated with the temporary non-visible item will be easily accessed when the animation no longer hides the item.
The user interface is represented by a tree structure.
With reference to
As shown in
Returning to the method as shown in
By fetching only the relevant information, i.e. information associated with application items determined to be in a visible state or a partially visible state, the performance of the method is optimized. This means that information associated with the text array 134b corresponding to the phone number of the non-highlighted contact and the picture 134c of the non-highlighted contact are not fetched.
In a subsequent step, the fetched information is composed to resulting image data and transmitted to the screen of the device.
With reference to
As shown in
Returning to the method as shown in
In a subsequent step, the fetched information is composed to resulting image data and transmitted to the screen of the device.
The XML-file containing graphical declarations of the application services comprises a sequence of XML tags. As an example, the comprehensive graphical appearance of the application services is also defined by XML tags. A page control may e.g. be represented by the following command.
The page control contains a number of attributes, wherein the value of the id attribute is the name of the page control, the value of the title attribute is the title of the page as retrieved from a language database, the value of the visuals attribute is a reference to the visual representation of the page, and the value of the menuBarItemSource attribute is the reference to a model defining the softkey behavior of the page. Further, a list control is declared within the page control. The list control contains a number of attributes, wherein the value of the id attribute is the name of the list control, the value of the itemSource attribute is the reference to a model defining the contact items, the value of the type attribute defines that the list is a check list, the value of the checkProperty attribute defines which property should be used for checking or unchecking an item, the value of the visuals attribute is a reference to the visual representation of the list, and the value of the itemVisuals attribute defines which visuals should be used for the list items.
The method for displaying graphical information on a screen of a device as previously described may be implemented by a module. The module comprises a compositor configured to compose resulting image data from a number of running application services. Further, the module has a transmitter configured to transmit the resulting image data to the screen of a device. In more detail, the compositor has an identifier, a determinator, a fetching means and a calculator. During operation, the module is arranged to display a number of application services on a screen of a device. At least one of the application services occupies specific pixels of the screen, i.e. contributes to the graphical scene of the device. The identifier is configured to identify items associated with each application service, and the determinator determines which of the items are in a visible state, a non-visible state and a partially visible state. In a more particular embodiment, parameters such as position, size, rotation etc of the identified items may be calculated. The determinator will transmit information to the fetching means which is configured to fetch information associated with the items in the visible state or the partially visible state. The fetching means will fetch application data from a first memory and graphical declarations from a second memory.
The fetched information will be transmitted from the fetching means to the calculator. The calculator is configured to create a resulting image data from the information associated with the items in the visible state. The resulting image data is transmitted to a screen of a device, thus displaying a user interface providing information about the application services.
Now referring to
The devices of
Now referring to
Although embodiments of the present invention have been described above with reference to various examples, it should be appreciated that modifications to the examples given can be made without departing from the scope of the invention as claimed.
Number | Date | Country | Kind |
---|---|---|---|
0702912-7 | Dec 2007 | SE | national |
This is a National Phase of PCT Patent Application No. PCT/EP2008/010830, filed on Dec. 18, 2008, which claims priority under 35 U.S.C. §119 to Swedish Patent Application No. 0702912-7, filed on Dec. 21, 2007, and U.S. Provisional Application No. 61/015,923, filed on Dec. 21, 2007, the contents of each of which are hereby incorporated by reference in their entirety.
Filing Document | Filing Date | Country | Kind | 371c Date |
---|---|---|---|---|
PCT/EP2008/010830 | 12/18/2008 | WO | 00 | 9/14/2010 |
Number | Date | Country | |
---|---|---|---|
Parent | 61015923 | Dec 2007 | US |
Child | 12735173 | US |