This invention relates to displaying linked information objects in a virtual space and, more particularly, to displaying a plurality of linked information contents disposed in a virtual space.
A typical example of prior art information content display means is Microsoft Internet Explorer®, a browser of Microsoft Corporation. This browser is arranged such that a plurality of successively linked hypertext contents such as HTML documents can be displayed on one window on a computer display screen. When a mouse is clicked on a displayed content at a location where linking to other content is defined, the other content to be linked to is displayed on the same window as or on a different window from the window on which the first content is displayed. With this browser, when one content changes to another linked content, change of the displayed images is not continuous. Therefore, it is difficult for a user to understand the relations between successively linked contents, and, sometimes the user tends to become at a loss where in the linkage he or she is.
The inventors have proposed means for solving the above-described problems. Specifically, they disclosed “APPLICATION PROGRAM STARTING METHOD, RECORDING MEDIUM ON WHICH SAID APPLICATION PROGRAM IS RECORDED, AND COMPUTER SYSTEM” in Japanese Unexamined Patent Publication No HEI 11-24878 A laid open for public inspection on Jan. 29, 1999, which corresponds to U.S. patent application Ser. No. 08/974,246 filed on Nov. 19, 1997; “ELECTRONIC INFORMATION DISPLAYING METHOD, ELECTRONIC INFORMATION BROWSING APPARATUS AND ELECTRONIC INFORMATION BROWSING PROGRAM RECORDING MEDIUM” in Japanese Unexamined Patent Publication No. 2000-172248 A laid open for public inspection on Jun. 23, 2000, which corresponds to U.S. Ser. No. 09/407,376 filed on Sep. 28, 1999; “DOCUMENT DISPLAY APPARATUS AND METHOD FOR DISPLAYING DOCUMENTS” in Japanese Unexamined Patent Publication No. 2000-207269 A laid open for public inspection on Jul. 28, 2000, which corresponds to U.S. Ser. No. 09/473,049 filed on Dec. 28. 1999; and “DOCUMENT BROWSING SYSTEM, AND DATA READ-IN APPARATUS AND DOCUMENT DISPLAY APPARATUS FOR USE IN DOCUMENT BROWSiNG SYSTEM” in Japanese Unexamined Patent Publication No. 2001-60166 A laid open for public inspection on Mar. 6, 2001. These publications disclose a technique enabling browsing successively linked information contents by traversing the linkage by smoothly moving a viewpoint, whereby a user can instantly understand the linkage between the contents during browsing.
However, with the techniques disclosed in the Japanese publications cited above, depending on system conditions or content conditions, it is frequently impossible to smoothly change the images by shifting a visual field, in such a case that the wait state occurs due to reading in content data or an amount of processing increases to thereby lessen the processing capacity for the purpose when the content data is read in. Furthermore, since content data reading and display preparation procedures of the systems are not optimized, it frequently takes a long time until a desired content is displayed. Also, there remain undeveloped areas in methods for displaying, on the same display screen, different types of contents including three-dimensional images and moving images or for enabling contents of additional types to be displayed.
An object of the present invention is to provide means which enables easier attainment of a desired one of information objects, in displaying the linked information objects in an information processing apparatus, by traversing the links between successively linked information objects.
Another object of the invention is to provide smoother image change with visual field shifting in displaying linked information objects in an information processing apparatus.
Still another object of the invention is to reduce time required for a desired content to be displayed.
Further objects of the invention are to enable different types of contents to be displayed, to provide integrating means for displaying different types of contents, and to facilitate an additional type of content to be displayed.
According to one aspect of the present invention, an information processing apparatus displays a plurality of linked information objects in a virtual space in accordance with visual field data. One of the information objects represents a content of one content type. The visual field data defines a visual field in the virtual space. The information processing apparatus includes hold means and first and second means. The hold means holds, in an executable condition, intermediate data generating means specific to the content type and display image generating means specific to the content type. The first means causes the intermediate data generating means to generate intermediate data for displaying the one information object, when it is determined, from the geometric relation between the visual field and the one information object, that the intermediate data of the one information object should be generated. The second means causes the display image generating means to generate a display image of the one information object based on the generated intermediate data, when it is determined, from the geometric relation between the visual field and the one information object, that the display image of the one information object should be generated.
According to another feature of the invention, the hold means of the information processing apparatus holds at least one content type specific data processing means in an executable state. The data processing means includes the intermediate data generating means, the display image generating means, capturing means specific to the content type for capturing content data of the one information object, and deleting means specific to the content type for deleting the generated intermediate data in an intermediate data memory.
The information processing apparatus may include third means for selecting one of the information objects as a representative object and defining the visual field by defining the geometric relation between the visual field and the representative object. The third means changes the representative object in response to the movement of the visual field in the virtual space, without changing the geometric relation of the plural information objects to the visual field. The first means traverses links between the information objects, starting from the representative object, and determines whether or not the intermediate data of the one information object should be generated.
The information processing apparatus may include further a memory region and memory managing means. Display data, including intermediate data for displaying the plurality of information objects, is stored in the memory region. The memory managing means determines the amount of the memory region occupied by the display data and deletes sequentially at least part of the intermediate data in the memory region which has not been used for the longest time for display image generation each time it determines that the amount occupied by the display data is larger than a predetermined threshold.
According to another aspect of the invention, a program for use in an information processing apparatus is provided for displaying a plurality of linked information objects in a virtual space in accordance with a visual field data. One of the plural information objects represents a content of a certain content type. The visual field data defines a visual field in the virtual space. The information processing apparatus includes a hold region which holds, in an executable condition, a program specific to the content type for causing an intermediate data generating step and a display image generating step to be executed. The intermediate data generating step is for generating intermediate data specific to the content type, and the display image generating step is for generating, from the generated intermediate data, a display image specific to the content type. The first program includes a step of generating an intermediate data for displaying the one information object by means of the intermediate data generating step, when it is determined, from the geometric relation between the visual field and the one information object, that the intermediate data for the one information object should be generated. The first program further includes a step of generating, from the generated intermediate data, a display image of the one information object by means of the display image generating step, when it is determined, from the geometric relation between the visual field and the one information object, that the one information object should be displayed.
According to the invention, in displaying linked information objects in an information processing apparatus, it is easier to reach a desired one of the linked information objects by traversing the links between the information objects.
According to the invention, smoother image change with shift of a visual field can be realized in displaying linked information object in an information processing apparatus.
According to the invention, time required for a desired content to be displayed can be reduced.
According to the invention, the different types of contents can be displayed, displaying different types of contents can be realized by integrating means, and displaying additional types of contents is also facilitated.
According to the invention, in displaying information contents existing in a memory device of a computer and in a computer and the like which may be connected to networks, such as the Internet, a desired one of a vast amount of linked information contents, such as HTML texts in WWW, can be readily reached by traversing the links. Specifically, a visual field is smoothly moved over a plurality of linked contents displayed on the same one display, to thereby traverse the linkage, and the contents are displayed in accordance with the visual field, whereby a large amount of information can be browsed comfortably by continuously moving the visual field.
The present invention is described by an example of displaying a plurality of linked information objects disposed in a three dimensional virtual space. An information object visually and/or audibly represents any information content treated as a unit in a virtual space, e.g. one text document, one document including text and graphics, one image, a stream of moving images, one three-dimensional image formed of plural parts, or an audio stream. Such an information object may be dynamically generated by a program.
The visual field data managing means 102, the overall processing control means 103, the display picture generation managing means 105 and the memory managing means 110 may be implemented in the form of a processor or processors having programs for the respective means, or may be a processor or processors including the functions of the respective means in the form of integrated circuits.
The content type specific data processing program lop includes a content data capturing or taking-in program lip, an object displaying intermediate data generating program 12p, an object displaying intermediate data deleting program 13p, and an object display image generating program 14p. The other content type specific data processing programs 20p, . . . , 21p have the same configuration as the program 10p.
Alternatively, the data processing programs 10p, 20p, . . . , 21p may be held in a memory in the object data processor 111 in an executable manner, as in the program memory unit 112.
The object data memory unit 113 stores therein a plurality of object display managing data (i.e. managing data for displaying objects) 30, 24, 25, . . . , and a plurality of object display intermediate data (i.e. intermediate data for displaying objects) 50, 27, 28, . . . . An object display managing data and an object display intermediate data are prepared for each of the information objects to be displayed.
The input device 101 generates operation input data relating to display picture visual field movement and the like in response to the operation of the device 101 by a user, and supplies the input data to the visual field data managing means 102. The visual field data managing means 102 holds visual field data 60 and renews or updates it in accordance with the data inputted by the user.
The overall processing control means 103 controls overall processing done in the information processing apparatus 100. including operations of the visual field data managing means 102, the object data managing means 104 and the display picture generation managing means 105. The overall processing control means 103 controls operations of various sections of the information processing apparatus 100 so that processing for display can be done in real time.
The object data managing means 104 generates, renews or deletes the object display managing data 30, 34, 25, . . . , and the object display intermediate data 50, 27, 28, . . . , using an object display intermediate data generator function 12 and an object display intermediate data deletion function 13 within the object type specific data processor function 10, in accordance with the visual field data 60.
The object data managing means 104 performs traversing or search of links between the information objects, starting from a representative or base object, and then performs processing for preparation for information object display, whereby the link traverse and display preparation for information objects involved in display image generation can be processed efficiently.
The object data managing means 104 determines for each information object, whether the object display intermediate data generation should be executed or not, based on the geometric relation between a visual field 71 and that information object. In accordance with the determination, the object data managing means 104 causes the object display intermediate data generator function 12 to execute the object display intermediate data generation, to thereby generate the object displaying intermediate data at an appropriate timing with change of the visual field 71.
The display picture generation managing means 105 manages a display list 90 and generates a display picture by rendering, on a frame memory 91, two-dimensional image data generated by the object display image generator function 14 in accordance with the visual field data 60 and the object display intermediate data 50.
The display device 106 displays a picture generated by the display picture generation managing means 105. The display picture generation managing means 105 determines, from the geometric relation between the visual field 71 and information objects, whether or not a particular information object should be displayed on the display screen, and causes the object display image generator function 14 to execute real-time rendering of that information object on the display frame screen if that information object is determined to be displayed. In this manner an information object to be displayed is properly selected for displaying in accordance with the shift of the visual field 71. The object display image generator function 14 can display information objects in different forms In accordance with display priority of the information objects and/or geometric relations between the visual field 71 and the information objects. This enables proper displaying of the information objects for the geometric relation between the visual field 71 and the information objects, whereby the amount of processing for low display priority information objects can be kept low, and the information objects can be displayed in variable form depending on the geometric relations between the visual field 71 and the information objects.
The information storage device 107 stores source content data. The device 107 includes, e.g. a hard disc, a CD, an MO, a DVD, a flash memory and the like.
The network interface 108 includes, e.g. a modem and a dialer and is used for connecting the information processing apparatus 100 for communication with an external network (not shown), e.g. the Internet. The network interface 108 is connected to the external network for communication therewith when, for example, a content data capturer or taking-in device function 11 captures or takes in source content data from the external network.
The cache data storage device 109 temporarily stores the content data captured by the content data capturer function 11 and other data to be used in display, for subsequent use in the information processing apparatus 100.
The memory managing means 110 monitors the amount of memory used to store the object display intermediate data 50 and the like and to reduce or delete data in the memory when, for example, the amount of used memory increases.
The object display managing data 30 for each information object includes a plurality of fields for content identification 31, content type specific data processing function identification 32, data 33 representing the geometric relation to the visual field, display priority 34, access record 35, object display intermediate data identification 36, and a link data list 40.
The content identification 31 is information, such as URL, file name, or directory, for identifying the content to be displayed.
The content type specific data processing function identification 32 is information for identifying the content type specific data processor function 10 for the data processing of the information object being managed by the object information managing data 30.
The geometric relation representative data 33 defines the geometric relation of an information object 73 to a virtual visual field coordinate system 72 established for displaying information objects disposed in a two-dimensional or three-dimensional virtual space. The geometric relation is a relative relation in position, orientation, scale ratio and the like.
Referring back to
The access record 35 is a record of the time of the processing last made for displaying an information object. The access record 35 is mainly used as the basis on which it is determined whether the object display intermediate data 50 or the object display managing data 30 of information objects which have not been displayed for a long time should be deleted or reduced or not.
The object display intermediate data identification 36 identifies the object information display intermediate data 50 that is directly used in generating a display image of the information object being managed by the object display managing data 30. The identification 36 may be, for example, a pointer to the object display intermediate data 50 in the memory.
The link data list 40 is a list of information for links from one information object to other information objects, and includes a plurality of link data sets 37.
Each link data set 37 includes a field for a pointer to another object to be linked to (hereinafter referred to as object linkage pointer) 38 representing the linking of one information object to another, and a field for data 39 representing a geometric relation of another information object to be linked to (hereinafter referred to as other linked information object), which is an information object the current object is linked to, relative to a current object information. The object linkage pointer 38 can be a pointer to the object information managing data of the other linked object information, for example. The data 39 representing the geometric relation of the other linked information object relative to a current information object can be expressed by for example a coordinate transform matrix for transforming the local coordinate system of the other linked information object into the local coordinate system 74 of the current information object.
When a link is defined between two information objects, the link data list 40 in the object display managing data 30 of one information object contains the link data set 37 for the link to the other object and vice versa in principle. For example, in source data of HTML documents, in general, only a one-way link from a current document to another subsequent document (hereinafter also referred to as other linked document) is described. Even in such a case, when the link data set 37 for the link to the other linked document is prepared in the link data list 40 of the object display managing data 30 of the current document, another link data set 37 for the link with the current document is also prepared in the link data list 40 of the corresponding object display managing data 30 of the other linked document. This makes bidirectional link traverse possible. The link data set 37 may be arranged to distinguish between a previous object linked to the current object and a subsequent object linked from the current object, and may contain a description representing a link with the previous or parent object and a description representing a link to the other linked or child object.
The content type specific data processor functions 10, 20, . . . , 21 provide different data processing procedures for respective content types. Each content specific data processor function, e.g. 10, includes a content data capturer function 11, an object displaying intermediate data generator function 12, an object displaying intermediate data deleting device function 13, and an object display image generator function 14.
The content data capturer function 11 captures content data for displaying an information object, through the information storage device 107, the network interface 108 or the cache data storage device 109 of the information processing apparatus 100 when such content data is required for displaying the object, and supplies it to the object displaying intermediate data generator function 12. The content data capturer function 11, when it has captured content data, may store the content data in the cache data storage device 109 for later re-use.
The object displaying intermediate data generator function 12 generates and renews object displaying intermediate data 50 in accordance with content data captured through the content data capturer function 11. The object display intermediate data generation and renewal are executed under the control of the object data managing means 104, when the display priority 34 has a value higher than the predetermined threshold value as the data 33 representing the geometric relation of that information object to the visual field changes.
The object displaying intermediate data deleting device function 13 deletes all or part of object display intermediate data. The deletion is performed under the control of the object data managing means 104 and/or the memory managing means 110.
Under the control of the display picture generation managing means 105, the object display image generator function 14 generates in real time, in the display picture generating process, part of an object within a display picture that is smoothly changing as the visual field shifts smoothly, using the object display managing data 30 and the object displaying intermediate data 50.
The object displaying intermediate data generator function 12, the object displaying intermediate data deleting device function 13 and the object display image generator function 14 independently operate asynchronously with respect to each other, which may be done executing them in separate threads or processes.
The object display intermediate data 50 which Is being generated or renewed by the object displaying intermediate data generator function 12, and also the intermediate data 50, part or all of which has been deleted by the object displaying intermediate data deleting device function 13, can be looked into asynchronously at any desired time by the object display image generator function 14 and can be used in real time display Image generation. For that purpose, the object displaying intermediate data generator function 12 and the object displaying intermediate data deleting device function 13 generate and renew and delete the object display intermediate data 50, in such a manner and sequence as not to cause any contradiction when the object display image generator function 14 looks into the object display intermediate data 50 currently being processed and generates the Subject display image.
Asynchronous operation of the object displaying intermediate data generator function 12, the object displaying intermediate data deleting device function 13 and the object display image generator function 14 makes it possible to generate a display picture smoothly changing as the visual field smoothly shifts, using the object display intermediate data 50 currently available. Even when it takes a long time to capture content data and to generate and renew the object displaying intermediate data 50, these functions 12, 13 and 14 are not required to wait until these capture, generation and renewal are completed.
The content type specific data processor function 10 may be pre-held in the object data processor 111 or in the program memory unit 112 of the information processing apparatus 100. Alternatively, the program for the content type specific data processing function 10 may be captured through the information storage means 107 or the network interface 108 and then held in the object data processor 111 or in the program memory unit 112.
The object displaying intermediate data 50 is generated by the object displaying intermediate data generator function 12 from the content data captured by the content data capturer function 11 in the content type specific data processor function 10 for an information object being handled, and is used by the object display image generator function 14 to generate a display image.
The object displaying intermediate data 50 is processed in the corresponding one of the content type specific data processor functions 10 provided for respective content types. Accordingly, the data format of the object displaying intermediate data 50 can be defined for each content type. Since optimum formats can be defined individually for the respective content types, display can be provided efficiently, and, in addition, a new content type specific data processor function 10 can be added easily and corresponding information contents can be processed easily.
The object data processor 111 holds therein or in the program memory unit 112, one or more content type specific data processor functions 10, 20, . . . , 21, which have functions to capture in different types of content data and to generate, renew, delete and look into specific types of object displaying intermediate data. Accordingly, different types of contents can be displayed, and it becomes easy to add a function for displaying another type of contents.
The representative-object geometric relation data 62 defines the geometric relation of the representative object to a virtual visual field which is defined in order to display the information object 73 disposed in the two-dimensional or three-dimensional virtual space shown in
The visual field parameter 63 defines a visual field angle or the like for generating an image, and is looked into, when a display picture is generated, by the object display image generator function 14 and the like controlled by the display picture generation managing means 105.
In the embodiment of the invention, a smooth change of the visual field 71 is realized by smoothly changing the representative-object geometric relation 62 relative to the visual field. The visual field data managing means 102 changes the representative-object geometric relation 62 relative to the visual field smoothly and continuously in accordance with the direction given through input data received from the input device 101 or a program.
The visual field data managing means 102 selects one of the information objects as a representative or base object, stores the selected representative object, defines the visual field by defining the geometric relation between the representative object and the visual field, and renews the visual field data when the representative object is changed without changing geometric relations of the respective information objects to the visual field. In this manner, the visual field can be defined simply by means of the relative geometric relation to a desired one of the information objects. Accordingly, there is no need for defining the absolute coordinate system, and, therefore, the reachable range is not limited by the scale for a particular coordinate system.
The display picture generation managing means 105 controls the object display image generator function 14 to generate in real time a display image in accordance with the visual field data 60, to thereby obtain a display picture smoothly changing with a smooth visual field change.
As the representative-object geometric relation 62 to the visual field changes, the geometric relations 33 of the respective information objects to the visual field also change, and the display priorities 34 change, which may cause the display priority 34 of some other information object to be higher than that of the representative object. If this happens, the information object having the higher display priority is selected as the representative object, and the representative object identification 61 in the visual field data 60 is rewritten. At the same time, the geometric relation to the visual field in the object display managing data 30 of the information object newly selected as the representative object is copied on the representative-object geometric relation data 62 in the visual field data 60, and stored. In this way, the representative object can be changed, while keeping the continuity of the visual field change. Thus, the linkage can be traversed infinitely in principle by shifting the visual field to display information objects without being limited by the scale ratio, by successively changing the representative object so that an information object whose geometric relation to the visual field can be defined with a standard scale is selected to be the representative abject.
The object display image generator function 14 contained in the content type specific data processor function 10 for the content type of the information object uses such object display intermediate data 50 to generate in real time the object display image.
The object display intermediate data 50 may be of any form specific to the content type. The content type specific data processor function 10 and the data format of the object display intermediate data 50 are preset for each content type so that the respective processing procedures in the content type specific data processor function 10 can process the object display intermediate data 50 of the corresponding content types.
At Step 203, the control means 103 causes the visual field data managing means 102 to renew the visual field data. The visual field data managing means 102 computes the visual field coordinate system 72 so as to smoothly vary the visual field 71 in accordance with the input data from, for example, the input device 101, and renews the visual field data 60. When it is necessary to change the representative object, the control means 103 performs processing involved in the change of the representative object through the object data managing means 104 as described above.
At Step 204, the control means 103 causes the object data managing means 104 to perform the traverse of the links between information objects. In the linkage traverse, the object display intermediate data generator functions 12 determine the data 33 representing the geometric relations of all of the object display managing data 30, 24, 25, . . . within the object data memory unit 113, relative to the visual field, and determine the display priorities 34 based on the data 33. For that purpose, the object display intermediate data generator functions 12 call the linkage traverse processing. In this case, in the linkage traverse processing TRAVERSE, the representative object pointer 61 is set to the first argument, and the representative-object geometric relation data 62 is set to the second argument.
At Step 205, the control means 103 causes the object data managing means 104 to prepare display data. In accordance with the display priorities 34 of the respective information objects, the object data managing means 104 causes the object display intermediate data generator functions 12 and the object display intermediate data deleting device functions 13 of the type specific data processor functions 10 to generate, renew or delete the object display intermediate data 50, and also renews the link data list 40 in the object display managing data 30.
At Step 206, the control means 103 causes the object data managing means 104 to control the object display image generator functions 14 to generate a display picture. The object image display generator function 14 generates a display image in accordance with the data 33 representing the geometric relation of each information object to the visual field, the display priority 34 and the object display intermediate data 50.
Under the control of the control means 103, the display picture generation control means 105 renders the display images generated by the object display image generator function 14 on to the frame memory 91 for displaying on the display device 10.
At Step 207, the information processing apparatus 100 causes, through the object data managing means 104, the object display Image deleting device function 13 to delete data when required. The object data managing means 104 reduces the data stored in the object data memory unit 113 when the amount of memory of the unit 113 occupied exceeds the predetermined threshold value.
At Step 208, the control means 103 determines whether the information display should be ended or not. If it determines that the information display should be continued, the processing returns to the Step 203. To end the information display, the procedure advances to Step 209.
In
At Step 303, the object data managing means 104 stores the geometric relation data which is the second argument, as the data 33 representing the geometric relation to the visual field in the object display managing data 30 to be processed.
At Step 304, the object data managing means 104 computes the display priority from the geometric relation representative data 33 and stores it in the display priority field 34 in the object display managing data 30.
Steps between Steps 305 and 308 inclusive form a loop. The object data managing means 104 executes the respective steps in the loop for all of the link data sets 37 in the link data list 40 of the object display managing data 30 to be currently processed.
At Step 306 in the loop, the object data managing means 104 determines the geometric relation of another linked object, to which the current object is to be linked, to the visual field, on the basis of the data 33 representing the geometric relation of the current object to be processed to the visual field and the data representing the geometric relation 39 of the other linked object to the current object to be processed.
At Step 307, the object data managing means 104 sets the object linkage pointer 38 to the first argument and sets, to the second argument, the data representing the geometric relation of the other linked object relative to the visual field as determined at Step 306, and iteratively calls and executes the processing TRAVERSE. In the TRAVERSE processing, processing is done as at Steps 302-304.
At Step 308, the processing in the loop is continued until the processing of all of the link data sets 37 is completed. When the processing loop from Step 305 to Step 308 is completed, the procedure advances to Step 309, where the linkage traverse is ended.
Steps between Step 410 and Step 440 inclusive form a loop, and the object data managing means 104 executes the respective steps in the loop for all of the object display managing data 30, 24, 25, . . . , stored in the object data memory unit 113.
At Step 420 in the loop shown in
At Step 421, the object data managing means 104 determines, with respect to the object display managing data 30, whether the generation and renewal of the object display intermediate data 50 and the link data list 40 have been completed. If they are determined to have been completed, the procedure advances to Step 440. If they are determined not to have been finished, the procedure advances to Step 422.
At Step 422, the object data managing means 104 determines whether the number NG of other information objects (or object display managing data) of which a display priority PD is higher than the display priority PD 34 and of which object display intermediate data is being generated or being renewed, is larger than a predetermined threshold value NTH. If the number NG is determined to be larger than the threshold NTH, the procedure advances to Step 424. If the number NG is determined not to be larger than the threshold NTH, the procedure advances to Step 423.
At Step 423, the object data managing means 104 calls a processing PREPARE for generating object display intermediate data shown in
At Step 424, the object data managing means 104 determines, with respect to the link data list 40, whether or not the addition, renewal or deletion of the link data set 37 has been done. If such addition, renewal or deletion is determined to have been done, the procedure advances to Step 425. If such addition, renewal or deletion is determined not to have been done, the procedure advances to Step 440.
At Step 425, in relation to the added, renewed or deleted link data set 37, the object data managing means 104 generates, renews or deletes object managing data for the other linked object. Thereafter, the procedure advances to Step 440.
At Step 431 branching from Step 420, if the object display intermediate data is being generated or renewed, the object data managing means 104 interrupts the generating or renewing processing.
At Step 432, the object data managing means 104 determines whether the display priority PD 34 is lower than the threshold value PTHDI, which is the condition for deleting the object display intermediate data 50. If the priority PD 34 is determined to be lower than the threshold PTHDI, the procedure advances to Step 433. If the priority PD 34 is determined not to be lower than the threshold PTHDI, the procedure advances to Step 440. At Step 433, the object data managing means 104 causes the object display intermediate data deleting device function 13 to delete the object display intermediate data 50. If it is estimated that the deletion will not be completed in a short time, the deletion is done in a different thread.
At Step 434, the object data managing means 104 determines whether or not the display priority PD 34 is lower than a threshold value PTHDM, which is defined as the condition for deleting the object display managing data 30. If the display priority PD 34 is determined to be lower than the threshold PTHDM, the procedure advances to Step 435. If the display priority PD 34 is determined not to be lower than the threshold PTHDM, the procedure advances to Step 440. At Step 435, the object data managing means 104 deletes the object display managing data. After that, the procedure advances to Step 440.
At Step 440, the processing in the loop is continued until the display data preparation is completed for all of the object display managing data. When the processing in the loop between Step 410 and Step 440 inclusive is completed, the procedure advances to Step 450, and the display data preparation is completed.
Step 503, Step 507 and the steps between them form a loop. The display picture generation managing means 105 executes successively the steps in the loop for all the object display managing data 30, 24, 25, . . . , stored in the object data memory unit 113.
At Step 504 in the loop, the display picture generation managing means 105 determines, for one set of object display managing data, e.g. data 30, whether the display priority PD 34 is higher than the threshold value PTHD, which is set as the condition of displaying an object. If the display priority PD is determined to be higher than the threshold PTHD, the procedure advances to Step 505. If the display priority PD is determined not to be higher than the threshold PTHD, the procedure advances to Step 507.
At Step 505, information identifying the object or, in fact, the display managing data 30 of that object, e.g. a pointer to the object display managing data 30, is added to the display list 90. It should be noted, however, that for display devices 106 of some types, in order to obtain proper display images, a display image of the information object located deepest from the visual field 71 must be first generated, and, then, a display image of the second deepest one and so on toward the nearest one. In this case, the display list 90 for the deepest information object is first prepared and then the lists 90 for successively nearer ones.
At Step 506, the display picture generation managing means 105 or the object data managing means 104 records the current time and other information in a field of the access record 35 in the display managing data 30 of the subject information object.
At Step 507, the processing in the loop is continued until the preparation of the display lists for all the object managing data is completed. If the processing in the loop formed by the steps between Steps 503 and 507 inclusive is completed, the procedure advances to Step 508.
Steps between Step 508 and Step 510 inclusive form a loop. For all information objects in the display list 90, the display picture generation managing means 105 executes the steps in the loop. If the display picture generation is performed first for the information object located deepest from the visual field 71 toward the nearest one, the respective steps in the loop are executed for the information object farthest (deepest) from the visual field 70 successively to the nearest information object, using the display list 90 prepared by the execution of the steps in the loop formed by the steps between Steps 503 and 507 inclusive.
At Step S509 in the loop, the display picture generation managing means 105 causes the object display image generator function 14 corresponding to the content type of the object being handled to generate an object display image in accordance with the geometric relation representative data 33, the display priority 34 and the object display intermediate data 50, and render the generated object display image on the display picture frame.
For one or more information objects, simultaneously (in parallel) with real time display picture generation under the control of the display picture generation managing means 105, the generations, renewals or deletions of a plurality of object display intermediate data are simultaneously executed under the control of the control means 103 or the object data managing means 104. This allows the object display intermediate data to be generated, renewed or deleted without interrupting real-time display image generation, even when there are information objects which require a long time for generation, renewal or deletion of their object display intermediate data. Simultaneous generation and renewal of the object display intermediate data for a plurality of objects can reduce the time required before the respective information objects are displayed.
The object display Image generator function 14 may generate a display image in varying forms in accordance with varying geometric relation representative data 33 or with varying values of the display priority PD 34. For example, the transparency of the information object may vary depending on the display priority PD 34. For example, an information object of lower display priority PD 34 may be displayed with a higher transparency. In this case, as the visual field 71 becomes nearer to a remote information object, the display priority 34 of that object increases, so that the transparent information object gradually becomes more visible. In the object display intermediate data 50 having the format shown in
At Step 510, the execution of the steps in the loop formed by Steps 508-510 is repeated until the rendering for all of the information objects in the display list 90 is completed at Step S510. When the execution of the steps in the loop is completed, the procedure advances to Step 511, and the display picture generation is ended.
At Step 602, the object data managing means 104 determines whether the amount of memory occupied by the object display intermediate data 50 exceeds a predetermined threshold value. If the amount of memory occupied is determined to have exceeded the predetermined threshold, the procedure advances to Step 603. If the amount occupied is determined not to have exceeded the threshold, the procedure advances to Step 605, and the data reduction is finished.
At Step 603, the object data managing means 104 selects one of information objects whose object display intermediate data 50 is to be deleted, based on the display priority 34 of that information object and the access record 35. For example, an information object whose last use in generating a display image recorded in the access record 35 is oldest maybe selected. Alternatively, the object having the lowest display priority 34 may be selected. As a further alternative, the selection of object may be determined based on both of the display priority 34 and the access record 35. By releasing part of the memory region used in this manner, increase of the amount of memory occupied during a time period of the display processing can be avoided so as to keep the amount of memory occupied less than the predetermined value.
At Step 604, the object data managing means 104 causes the object display intermediate data deleting device function 13 of the content type specific data processor function 10 corresponding to the information object selected at Step 603, to delete the object display intermediate data 50 of the information object selected at Step 603. In this case, the object data managing means 104 may delete all or part of the object display intermediate data 50 of the selected information object. Alternatively, items to be deleted may be determined differently depending on the value of the display priority 34 of information objects.
If part of the object display intermediate data 50 has been deleted, the object data managing means 104 may renew or delete the object display intermediate data identification 36 in the object display managing data 30, as occasion demands. If the object display intermediate data 50 has been completely deleted, the object data managing means 104 empties the object display intermediate data identification 36.
The generation of object display intermediate data for each information object is performed by means of the object display intermediate data generator function 12 corresponding to that information object. The content type specific data processor function 10 including that object display intermediate data generator function 12 is identified by the content type specific data processing function identification 32 of the object display managing data 30. In this manner, the object display intermediate data 50 is generated by means of processing procedures specific to the content type of each information object.
At Step 702 in
At Step 703, the object display intermediate data generator function 12 starts another thread to start the intermediate data generating thread PREPARATION-THREAD as exemplified in
At Step 703, the procedure advances to Step 704 to finish the intermediate data generating processing PREPARE without awaiting the end of the intermediate data generating thread PREPARATION-THREAD.
At Step 753, the object display intermediate data generator function 12 processes the content data which has been already captured, to generate or renew the object display intermediate data 50. If necessary, the object display intermediate data generator function 12 renews the object display intermediate data identification 36 of the object display managing data 30.
At Step 754, the object display intermediate data generator function 12 processes the content data which has been already captured and the object display intermediate data 50, and generate or renew the link data set 37 in the link data list 40 of the object display managing data 30 for that information object.
At Step 755, the object data managing means 104 determines whether the display priority PD 34 of a particular object display managing data 30 is higher than the threshold value PTHG, the condition for starting the generation of the object display intermediate data 50. If the display priority PD 34 is determined to be higher than the threshold PTHG, the procedure advances to Step 756. If the display priority PD 34 is determined not to be higher than the threshold PTHG, the procedure advances to Step 757 to interrupt the generation of the object display intermediate data 50 in the intermediate data generating thread PREPARATION-THREAD, and this thread is finished.
At Step 756, the object data managing means 104 determines whether the generation and renewal of the object display intermediate data 50 and link data list 40 for that object display managing data 30 have been completed or not. If the generation and renewal are determined to have been completed, the procedure advances to Step 757, to interrupt the generation of the object display intermediate data 50 in the intermediate data generating thread PREPARATION-THREAD, and this thread is finished. If the generation and renewal of the object display intermediate data 50 and link data list 40 are determined not to have been completed, the procedure returns to Step 752, and the processing is continued.
The number of links from one information object is zero (0) or one (1) in
Actually, for a group of information objects having linkage like the one as shown in
For the picture on the screen 901, the information object 81 is selected as the representative object. On the screen 902, the information object 82 is selected as the representative object. For the pictures of the screens 903-906, the information object 83 is selected as the representative object. For the picture on the screen 908, the information object 81 is selected again as the representative object. In the illustrated example, it is somewhat difficult for the user to judge which one of the information objects is the representative object, but the picture may indicate, in a perceptible manner, which one of the information objects is the representative object.
At Steps 420, 422, 432, 504 and 755 of the above-described embodiment, whether or not the object display intermediate data should be generated, renewed or deleted, and whether or not the object display managing data should be deleted, and whether or not the object should be displayed are determined by comparing the display priority PD 34 with the reference thresholds for the respective conditions. However, they may be determined in other suitable ways as occasion demands.
The invention has been described by means of an embodiment which mainly deals with displaying visual information objects on a display screen. However, an audio signal generating function may be implemented as one of the functions of the object display image generator 14, with audio output means added to the apparatus, to thereby enable audio information objects to be produced.
The above-described embodiment is only a typical example, and its modifications and variations are apparent to people skilled in the art. It should be noted that people skilled in the art can make various modifications to the above-described embodiment without departing from the principle of the invention and the accompanying claims.
Advantages of the Invention
According to the invention, in displaying linked information objects in an information processing apparatus, advantageously, means is provided for enabling easier attainment of a desired one of the linked information objects by traversing the links between the information objects, smoother image change with shift of a visual field can be realized, and time required for a desired content to be displayed can be reduced. According to the invention, advantageously, the different types of contents can be displayed, displaying different types of contents can be realized by integrating means, and displaying additional types of contents is also facilitated.
Number | Date | Country | Kind |
---|---|---|---|
2001-162322 | May 2001 | JP | national |
Number | Name | Date | Kind |
---|---|---|---|
5295243 | Robertson et al. | Mar 1994 | A |
5684969 | Ishida | Nov 1997 | A |
5778372 | Cordell et al. | Jul 1998 | A |
6049795 | Kamiwada et al. | Apr 2000 | A |
6088032 | Mackinlay | Jul 2000 | A |
6147703 | Miller et al. | Nov 2000 | A |
6486895 | Robertson et al. | Nov 2002 | B1 |
6549944 | Weinberg et al. | Apr 2003 | B1 |
6608640 | Nagahara et al. | Aug 2003 | B1 |
6650343 | Fujita et al. | Nov 2003 | B1 |
6681370 | Gounares et al. | Jan 2004 | B2 |
6683990 | Yagishita et al. | Jan 2004 | B2 |
6724407 | Cheng | Apr 2004 | B1 |
6877135 | Kamiwada et al. | Apr 2005 | B1 |
Number | Date | Country | |
---|---|---|---|
20020180808 A1 | Dec 2002 | US |