Patent Application
20250208809
The present invention relates to a communication terminal, a control method, and a storage medium.
Communication apparatuses have become widespread that are equipped with a web browser (hereinbelow, referred to as a browser) and allows a user to browse web pages on the browser. Such communication apparatuses display a web page of the external service via the browser to be able to link with an external service.
One form of browser is referred to as a cloud browser, in which a browser engine for generating a rendering result of a web page is arranged on a cloud server. The cloud browser is a method for browsing a web page by transmitting a rendering result rendered by the browser engine on the cloud server to a communication apparatus and displaying the rendering result thereon.
The browser engine performs high loaded processing, such as web page analysis processing or execution processing, and thus it is possible to reduce the load on the communication apparatus (Japanese Patent Application Laid-Open No. 2014-182793) by using the cloud browser.
Various device resources required for a personal computer (PC) are virtually allocated to a virtual machine in the cloud. Thus, the browser engine running on the virtual machine can operate in the same way as it is running on a PC.
However, if an application that displays a rendering result on the communication apparatus abnormally terminates due to insufficient memory or the like in the cloud browser, there is an issue that the previously displayed rendering result cannot be resumed at the time of resuming the application in the communication apparatus.
On the other hand, in a form in which a communication apparatus equipped with a browser engine communicates with a server to perform web access, a method is discussed for resuming an operation in a case where abnormal termination occurs on the server side (Japanese Patent Application Laid-Open No. 2010-219792).
According to Japanese Patent Application Laid-Open No. 2010-219792, it is discussed that user interface (UI) information is presented to the communication apparatus to enable it to resume processing when abnormal termination occurs on the server side, but a method for resuming the operation if abnormal termination occurs on the communication apparatus is not discussed. Further, a form in which the communication apparatus does not include a browser engine, such as a cloud browser, is also not discussed.
The present invention is realized by the following configurations in order to achieve the above-described issue.
According to an aspect of the present invention, a communication terminal includes a reception unit configured to receive image information acquired by rendering a web page, a display unit configured to display an image based on the image information, and a control unit configured to differentiate the image to be displayed between a case where the display unit displays the image after display of the image is abnormally terminated and a case where the display unit displays the image in normal startup.
Further features of the present invention will become apparent from the following description of embodiments with reference to the attached drawings.
Embodiments for implementing the present invention (hereinbelow, referred to as “embodiments”) will be described in detail below with reference to the attached drawings. However, the following embodiments do not limit the present invention according to the scope of claims. Further, not all combinations of features described in the embodiments are necessarily essential to the solutions of the present invention. The same components are denoted by the same reference numerals throughout the descriptions of the embodiments. Each of the embodiments of the present invention described below can be implemented solely or as a combination of a plurality of the embodiments or features thereof where necessary or where the combination of elements or features from individual embodiments in a single embodiment is beneficial.
The communication apparatus 101 is an apparatus on which a user can view a screen and perform an operation. The communication apparatus 101 is an example of a communication terminal according to the present invention. For example, the communication apparatus 101 is a thin client personal computer (PC) or the like that does not have a printer or a sophisticated browsing function.
The image generation system 103 includes a virtual machine 104 equipped with a browser engine 105 and a storage 106.
The browser engine 105 runs on the virtual machine 104, analyzes and renders a web content, and performs imaging of a rendering result.
The virtual machine 104 stores the image generated by imaging by the browser engine 105 in the storage 106.
The virtual machine 104 is connected to the communication apparatus 101, a web server 102, the storage 106, and the like via a network. The web server 102 stores the web content.
Examples of web contents include Hyper Text Markup Language (HTML), Cascading Style Sheets (CSS), JavaScript®, an image file in a format such as Joint Photographic Experts Group (JPEG), Portable Network Graphic (PNG), or the like, and a moving image file in a format such as Moving Picture Experts Group (MPEG)-4 Part 14 (MP4), WebM, or the like.
The communication apparatus 101 is also connected to the virtual machine 104, the storage 106, and the like via the network. The network may be, for example, the Internet, a wide area network (WAN), a local area network (LAN), or a combination of these. A plurality of communication apparatuses 101 may also exist.
A flow up to generating a screen of the communication apparatus 101 will now be briefly described, and then a configuration and a processing flow will be described in more detail. The communication apparatus 101 transmits uniform resource locator (URL) information of a web page to the virtual machine 104 to request the virtual machine 104 to read the URL of the specified web page.
When the virtual machine 104 receives the URL information of the web page transmitted from the communication apparatus 101, the virtual machine 104 requests the browser engine 105 to read the web page with the corresponding URL from the web server 102.
In response to the request, the browser engine 105 reads the web page with the corresponding URL from the web server 102. In other words, the browser engine 105 acquires the web content from the web server 102.
Subsequently, the browser engine 105 generates a rendering result acquired by rendering the web content and performs imaging the rendering result. The image after the imaging is then transmitted by the virtual machine 104 to the storage 106 and stored therein. The communication apparatus 101 acquires the stored image from the storage 106 and displays the acquired image, so that a screen of the communication apparatus 101 is generated.
Next, hardware and software configurations of the communication apparatus 101, a hardware configuration of the virtual machine 104, and software configurations of the virtual machine 104 and the browser engine 105 will specifically be described.
The communication apparatus 101 includes a central processing unit (CPU) 201, a read-only memory (ROM) 202, a random access memory (RAM) 203, a graphical user interface 204, and a network interface 205 as the hardware configuration.
The CPU 201 controls the communication apparatus 101.
The ROM 202 stores a program and a parameter that do not require modification.
The RAM 203 temporarily stores a program and data supplied from an external apparatus or the like.
The graphical user interface 204 is a display operation unit that can be used by a screen for displaying stored data and supplied data, a pointing device such as a mouse and a touch panel for inputting data in response to a user operation, and an input device such as a keyboard.
The graphical user interface 204 is an example of a display unit according to the present invention.
The network interface 205 is used to connect to the network.
The communication apparatus 101 may also include a hard disk or a memory card, which are fixedly installed therein. The communication apparatus 101 may also include a detachable external storage device including an optical disk, such as a flexible disk (FD) and a compact disk (CD), a magnetic card, an optical card, an integrated circuit (IC) card, and a memory card.
The communication apparatus 101 includes a communication unit 301, a storage unit 302, an operation detection unit 303, a screen generation unit 304, a response analysis unit 305, an application unit 306, and a status determination unit 307 as the software configuration.
The communication unit 301 is an example of a communication unit according to the present invention that controls communication performed by the communication apparatus 101 via the network interface 205. The communication unit 301 functions as a transmission unit and a reception unit according to the present invention. The communication unit 301 switches an operation information transmission flag in the storage unit 302, which will be described below.
The storage unit 302 stores information such as image information, an abnormal termination flag, and the operation information transmission flag, which will be described below. The storage unit 302 is an example of a storage unit according to the present invention.
The operation detection unit 303 detects an operation content when the graphical user interface 204 is operated. The operation detection unit 303 also determines whether an operated screen is an operation on an apparatus specific screen (described below) that is specific to the communication apparatus 101. The operation detection unit 303 is an example of an operation detection unit according to the present invention.
The screen generation unit 304 generates a screen on the graphical user interface 204. Specifically, for example, the screen generation unit 304 displays an image based on the image information, which will be described below, on the graphical user interface 204, and thus the screen on the graphical user interface 204 is generated. The screen generation unit 304 is an example of a screen generation unit according to the present invention.
(Response Analysis unit 305)
The response analysis unit 305 analyzes a content of information that the communication unit 301 receives from the virtual machine 104. The response analysis unit 305 is an example of a response analysis unit according to the present invention.
The application unit 306 controls an application, such as starting and ending the application to perform processing as a cloud browser. The application unit 306 also switches the abnormal termination flag, which will be described below, in the storage unit 302. The application unit 306 is an example of an application control unit according to the present invention.
The status determination unit 307 determines the abnormal termination flag and the operation information transmission flag, both of which will be described below, in the storage unit 302. The status determination unit 307 is an example of a status determination unit according to the present invention.
The CPU 401 controls the virtual machine 104, the browser engine 105, and the like.
The ROM 402 stores a program and a parameter that do not require modification.
The RAM 403 temporarily stores a program and data supplied from an external apparatus or the like.
The network interface 404 is used to connect to the network.
The virtual machine 104 includes a communication unit 501, a storage unit 502, a request determination unit 503, a browser application unit 504, and the browser engine 105 as the software configuration.
The communication unit 501 controls communication performed by the virtual machine 104 via the network interface 404. The communication unit 501 also performs communication between the virtual machine 104 and the browser engine 105. The communication unit 501 is an example of a communication unit according to the present invention.
The storage unit 502 stores operation information transmitted from the communication apparatus 101. An example of the operation information transmitted from the communication apparatus 101 is URL information of a web page.
Based on the URL information of the web page, the browser engine 105 acquires and renders the web content, and performs imaging of the rendering result. The storage unit 502 also stores these results, which will be described in detail below.
In other words, the storage unit 502 also stores the web content, the rendering result acquired by rendering the web content, and the image obtained by imaging the rendering result. The storage unit 502 is an example of a storage unit according to the present invention.
The request determination unit 503 determines a content that the communication unit 501 receives from the communication apparatus 101 or the browser engine 105. The request determination unit 503 is an example of a request determination unit according to the present invention.
The browser application unit 504 controls a browser application, such as starting and ending the browser application to perform processing as the cloud browser. The browser application unit 504 is an example of a browser application control unit according to the present invention.
The browser engine 105 includes a web content analysis unit 505, a web content rendering unit 506, and an image generation unit 507.
The web content analysis unit 505 performs analysis of the web content, execution of operation information reflection processing in response to an operation information reflection processing request from the virtual machine 104, and the like. The web content analysis unit 505 is an example of a web content analysis unit according to the present invention. For example, it is assumed that the operation information reflection processing request is a request to read the web page with the corresponding URL from the web server 102. The web content analysis unit 505 reads the web page with the corresponding URL from the web server 102 and acquires the web content as the operation information reflection processing corresponding to the operation information reflection processing request. Subsequently, when the operation information reflection processing is completed, the web content analysis unit 505 transmits an operation information result to notify the virtual machine 104 of completion of the processing. The web content analysis unit 505 may determine whether the operation information reflection processing request is received from the virtual machine 104, determine a timing for web content time lapse rendering reflection processing, and perform a web operation on the analyzed web content.
The web content rendering unit 506 renders the analyzed web content. For example, the web content rendering unit 506 renders the web content read by the web content analysis unit 505 and generates a rendering result. In a case where the web content analysis unit 505 determines the timing for the web content time lapse rendering reflection processing, the web content rendering unit 506 may perform rendering reflection processing with a lapse of time based on the determination. After completing the rendering processing, the web content rendering unit 506 may transmit a rendering completion notification to notify the virtual machine 104 of the completion of rendering processing.
The image generation unit 507 performs imaging of the rendering result rendered by the web content rendering unit 506. The image generation unit 507 is an example of an image generation unit according to the present invention.
Processing flows of the communication apparatus 101, the virtual machine 104, and the browser engine 105 according to the first embodiment of the present invention will be described in order below with reference to
A relationship between the screens in
As will be described below, in a case where the application is abnormally terminated in previous processing, the abnormal termination flag in the storage unit 302 is ON.
In step S801, the status determination unit 307 determines whether the application has been abnormally terminated in a previous time based on the abnormal termination flag in the storage unit 302. As a result of the determination, if the abnormal termination flag is ON (YES in step S801), the processing proceeds to step S802. If the abnormal termination flag is OFF (NO in step S801), the processing proceeds to step S804. Here, it is assumed that it is started normally, the abnormal termination flag is OFF, and the processing proceeds to step S804.
In step S804, since the application has been normally terminated in the previous time, the screen generation unit 304 displays the image for the home screen illustrated in
In step S805, the communication unit 301 functions as the transmission unit and transmits the operation information to the virtual machine 104. The operation information includes, for example, a termination request to terminate a browser application, and transmission of the URL information of the web page. Here, it is assumed that the URL information of the web page is transmitted as the operation information. The URL information of the web page as the operation information that is transmitted by the communication unit 301 as the transmission unit is also referred to as the operation information of the web page.
In step S806, the communication unit 301 functions as the reception unit and receives the operation information result from the virtual machine 104. The response analysis unit 305 functions as the response analysis unit and analyzes a content of the operation information result. The operation information result is information about a result corresponding to the operation information transmitted in step S805.
As described above, the URL information of the web page is transmitted as the operation information, so that web page read information is received from the virtual machine 104 as the operation information result corresponding thereto. The web page read information, which will be described below, is information regarding an image (hereinbelow, also referred to as image information) for notifying that the image acquired by rendering the web page is stored in the storage 106. On the other hand, in a case where the operation information transmitted in step S805 is transmission of the termination request to terminate the browser application, the operation information result corresponding thereto is information about the termination of the browser application from the virtual machine 104.
In step S807, the application unit 306 determines whether the received operation information result is the termination of the browser application. Here, the web page read information is received from the virtual machine 104, so that the determination in step S807 is NO, and the processing proceeds to step S808. In a case where the received operation information result is the information about the termination of the browser application (YES in step S807), the application unit 306 functions as the application control unit and performs termination processing on the application running on the communication apparatus 101. The processing then proceeds to step S811. In step S811, the application unit 306 determines whether the application is terminated. In a case where the determination in step S807 is YES and the processing proceeds to step S811, the determination in step S811 is also YES, and the present processing is terminated.
In step S808, the communication unit 301 acquires an image from the storage 106 based on the web page read information. As described above, the web page read information is the image information, and thus the communication unit 301 functions as the reception unit that acquires the image based on the image information from the storage 106.
In step S809, the screen generation unit 304 functions as the screen generation unit and generates a screen based on the image based on the image information received by the communication unit 301 as the reception unit in step S808. For example, it is assumed that the image illustrated in
In step S810, the storage unit 302 stores the image displayed on the graphical user interface 204 in step S808, that is, the image illustrated in
In step S811, the application unit 306 determines whether the application is terminated as described above. Here, since the application is not yet terminated, the application unit 306 determines as NO in step S811, and the processing proceeds to step S805.
There may be a case where the application goes down or abnormally terminates due to insufficient memory or the like until the determination in step S811 becomes YES. For example, to make it easy to understand, it is assumed that the application is abnormally terminated after the image illustrated in
In step S802, the screen generation unit 304 generates a screen by displaying the image based on the image information stored in the storage unit 302 on the graphical user interface 204. As described above, the image illustrated in
In step S803, the application unit 306 turns OFF the abnormal termination flag in the storage unit 302. Then, the processing proceeds to step S805 again, and the processing in step S805 and subsequent steps are repeated until it is determined as YES in step S811.
In this way, the screen before the abnormal termination illustrated in
Next, the processing flow of the virtual machine 104 according to the first embodiment of the present invention will be described.
In step S901, the request determination unit 503 determines whether the communication unit 501 has received the operation information from the communication apparatus 101. As a result of the determination, if the communication unit 501 has received the operation information (YES in step S901), the processing proceeds to step S902. If the communication unit 501 has not received the operation information (NO in step S901), the processing proceeds to step S907. As described above, the operation information includes the termination request to terminate the browser application and the URL information of the web page. However, the URL information of the web page has been transmitted as the operation information, so here, it is assumed that the URL of the web page has been received.
In step S902, the communication unit 501 transmits the operation information reflection processing request to the browser engine 105. A content of the transmission is determined from the operation information received in step S901, but it is assumed that the URL of the web page is received, so that a read request for the URL of the web page received in step S901 is transmitted. In a case where the operation information is the termination request to terminate the browser application, the browser application unit 504 performs termination processing on the browser application as the operation information reflection processing request, and the processing proceeds to step S903.
In step S903, the browser application unit 504 determines whether the operation information reflection processing request transmitted in step S902 is the termination request to terminate the browser application. If the operation information reflection processing request is the termination request (YES in step S903), the processing proceeds to step S906. In step S906, the communication unit 501 transmits the information about the termination of the browser application to the communication apparatus 101 as the operation information result. At this time, the termination processing is performed on the browser application as described above, and thus the determination in step S907 is also YES, and the processing is terminated. In response to this, the determination result in step S807 is YES in the processing of the communication apparatus 101 as described above. Accordingly, termination processing is performed on the application running on the communication apparatus 101. On the other hand, if it is not the termination of the browser application (NO in step S903), the processing proceeds to step S904.
In step S904, the communication unit 501 receives the operation information result from the browser engine 105. As described above, the operation information result is based on the transmission of the read request for the URL of the received web page, and therefore the web page read information is received. As will be described below, the web page read information is information about storing the image illustrated in
In step S905, the communication unit 501 transmits the image illustrated in
In step S906, the communication unit 501 transmits the operation information result to the communication apparatus 101. Specifically, the web page read information that is the operation information result received in step S904 is transmitted, but at this time, the image illustrated in
In step S907, the browser application unit 504 determines whether the browser application is terminated. In a case where the browser application is terminated (YES in step S907), the present processing is terminated. In a case where the browser application is not terminated (NO in step S907), the processing proceeds to step S901. In the processing in the present flowchart, the processing in step S901 and subsequent steps are repeated until it is determined as YES in step S907.
Next, the processing flow of the browser engine 105 according to the first embodiment of the present invention will be described.
In step S1001, the web content analysis unit 505 receives the operation information reflection processing request from the virtual machine 104. As described above, the operation information reflection processing request transmitted from the virtual machine 104 is the read request for the URL of the web page, so that the read request for the URL of the web page is received here.
In step S1002, it is determined whether the received operation information reflection processing request is the termination request instructing the termination of the browser application. The received operation information reflection processing request is not the termination request instructing the termination of the browser application, so that the determination is NO (NO in step S1002), and the processing proceeds to step S1003. In contrast, in a case where the received operation information reflection processing request is the termination request (YES in step S1002), the browser application unit 504 performs the termination processing on the browser application, and the processing proceeds to step S1006. In step S1006, the browser application unit 504 determines whether the browser engine is terminated. In this case, it is determined as YES in step S1006, and the present processing is terminated.
In step S1003, the web content analysis unit 505 performs the operation information reflection processing based on the operation information reflection processing request received in step S1001. In other words, the web content analysis unit 505 reads the web page with the corresponding URL (i.e., the web content) from the web server 102 via the communication unit 501 based on a web page read request for the URL of the received web page.
In step S1004, the web content rendering unit 506 generates a rendering result of the web content, and the image generation unit 507 generates an image by performing imaging the rendering result. Specifically, the image generation unit 507 generates the image illustrated in
In step S1005, the web content analysis unit 505 transmits the operation information result to the virtual machine 104. In other words, the web content analysis unit 505 transmits information about reading the web page with the corresponding URL (i.e., the web content) from the web server 102 performed in step S1003. At this time, the storage unit 502 already stores the image illustrated in
In step S1006, it is determined whether the browser engine 105 is terminated. As a result of the determination, in a case where the browser engine 105 is terminated (YES in step S1006), the present processing is terminated. In a case where the browser engine 105 is not terminated (NO in step S1006), the processing proceeds to step S1001.
In the processing in the present flowchart, the processing in step S1001 and subsequent steps are repeated until it is determined as YES in step S1006.
Performing the above-described processing has a following advantage because the application can be restarted from the screen that has been displayed at the time of the previous abnormal termination in a case where the application is abnormally terminated in the previous time.
Even in a case where the application of the communication apparatus 101 is abnormally terminated due to insufficient memory size or the like, the processing performed on the virtual machine 104 remains in a standby state.
However, the processing is restarted from reading the URL page after the application of the communication apparatus 101 is restarted and the browser application is accessed again using the URL.
In contrast, the application can be restarted from the screen that was displayed at the time of the previous abnormal termination in a case where the application is abnormally terminated in the previous time, according to the present embodiment as described above. Thus, the processing can be restarted from the state before the abnormal termination rather than from reading the URL page.
Thus, the processing can be restarted from continuation of input information or scroll information that has been performed before the previous abnormal termination.
The communication apparatus 101 and the virtual machine 104 according to a second embodiment of the present invention will now be described with reference to FIGS. 11 and 12. According to the second embodiment, the configuration of the information processing system 100 and the hardware and software configurations of the communication apparatus 101 are respectively the same as those in
The second embodiment is an embodiment that takes into account a case where an application is abnormally terminated after the operation information is transmitted in step S805 in
In this case, the image stored in the storage unit 302 is stored prior to the image illustrated in
According to the present embodiment, the processing in the present flowchart is also started in a state where the communication apparatus 101 starts up an application in the application unit 306 and is connected to the virtual machine 104 and the storage 106 via the network, and the operation detection unit 303 receives the operation information.
According to the present embodiment, the abnormal termination flag is also turned ON in a case where the application unit 306 detects the abnormal termination of the application.
According to the present embodiment, in step S801, in a case where the status determination unit 307 determines whether the application has been abnormally terminated in the previous time based on the abnormal termination flag in the storage unit 302, and the abnormal termination flag is ON (YES in step S801), the processing proceeds to step S1101.
In step S1101, the status determination unit 307 determines the operation information transmission flag in the storage unit 302. Firstly, the operation information transmission flag will be described. The operation information transmission flag is turned ON in step S1106. Step S1106 proceeds after the transmission of the operation information (e.g., transmission of the URL information of the web page in the example described above) is performed in step S805 in
In step S1102, the communication unit 301 transmits previous operation result confirmation information to the virtual machine 104. The previous operation result confirmation information is a request to the virtual machine 104 to transmit the previous operation information result and is information determined in advance between the communication apparatus 101 and the virtual machine 104. In other words, the previous operation result confirmation information is a request for the operation information result (web page read information) corresponding to transmission of the previous operation information transmission (e.g., transmission of the URL information of the web page). In this way, in step S1101, in a case where the status determination unit 307 serving as the determination unit determines that the image is not stored in the storage unit 302 in step S810, the communication unit 301 functions as a confirmation information request transmission unit that transmits a confirmation information request as the previous operation result confirmation information.
In step S1103, the communication unit 301 receives previous operation result information (e.g., previous web page read information) from the virtual machine 104, and the response analysis unit 305 analyzes a content of the previous operation result information.
In step S1104, the communication unit 301 acquires an image from the storage 106. At this time, the image stored in the storage 106 is the image illustrated in
In step S1105, the screen generation unit 304 serving as the screen generation unit causes the graphical user interface 204 to display the image illustrated in
In step S1106, the communication unit 301 turns ON the operation information transmission flag in the storage unit 302 as described above.
In step S1107, the communication unit 301 turns OFF the operation information transmission flag in the storage unit 302 as described above.
As in the first embodiment, the present processing repeats the processing in step S805 and subsequent steps until it is determined as YES in step S811.
According to the present embodiment, the processing in the present flowchart is also started in a state where the virtual machine 104 starts up the browser application.
As described in the first embodiment, the request determination unit 503 determines, in step S901, whether the communication unit 501 has received the operation information from the communication apparatus 101. As in the first embodiment, in a case where the communication unit 501 receives the operation information from the communication apparatus 101, the determination in step S901 is YES, and then the processing proceeds to steps S902, S903, S904, and S1201.
In contrast, according to the second embodiment, the previous operation result confirmation information is received in some cases as described above with reference to the flowchart in
In step S1201, the browser application unit 504 stores the operation information result received in step S904 in the storage unit 502, and then the processing proceeds to steps S905, S906, and S1202. According to the example described in the first embodiment, the web page read information received in step S904 is stored in the storage unit 502. Specifically, the web page read information to be received is information about storing the image illustrated in
In step S1202, the request determination unit 503 determines whether the communication unit 501 has received the previous operation result confirmation information. A case where the processing proceeds to step S1202 via step S1201 is a case where the determination result in step S901 is not the confirmation information but the operation information, so that the determination in step S1202 is NO, and the processing proceeds to step S907. In contrast, in the case of the above-described example, the confirmation information is received and the determination result in step S901 is NO, whereby the processing proceeds to step S1202. Thus, in this case, the determination result in step S1202 is YES, and the processing proceeds to step S1203.
In step S1203, the browser application unit 504 reads the operation information result stored in the storage unit 502. Following the above-described example, it is assumed that the web page read information is read from the storage unit 502.
In step S1204, the communication unit 501 transmits the previous operation result information (previous web page read information) to the communication apparatus 101. In response to this, the communication apparatus 101 receives the previous operation result information in step S1103 described above with reference to
According to the second embodiment as described above,, startup can be reliably performed in a screen state generated by the image illustrated in
In other words, the screen illustrated in
As can be seen from the above-described processing flow, the operation information transmitted before the previous abnormal termination is not transmitted again. Thus, processing for acquiring the web content again does not occur to acquire the image illustrated in
A third embodiment according to the present invention will now be described. In the third embodiment, it is also assumed that a screen specific to a communication apparatus (hereinbelow, also referred to as an apparatus specific screen) is further displayed on the graphical user interface 204. The apparatus specific screen is generated by the communication apparatus 101 itself (e.g., a software keyboard). The software keyboard is an example of a user interface according to the present invention, and such an apparatus specific screen may be referred to as a “UI”.
The screen on the graphical user interface 204 illustrated in
According to the present embodiment, the screens of the software keyboard illustrated in
When the user touches the screen on the software keyboard and performs an operation, the operation detection unit 303 determines that an operation is performed on the apparatus specific screen.
In contrast, when the user performs an operation by touching, for example, a screen illustrated in
However, in a case where the user performs an operation of touching the text input field 700 on the screen illustrated in
Next, a processing flow of the communication apparatus 101 according to the third embodiment will be described with reference to
According to the third embodiment, the configuration of the information processing system 100 and the hardware and software configurations of the communication apparatus 101 are respectively the same as those in
Further, the hardware configuration of the virtual machine 104 and the software configurations of the virtual machine 104 and the browser engine 105 are respectively the same as those in
Furthermore, a processing flow of the virtual machine 104 is the same as that in
The processing in steps S801, S803 to S808, and S811 in
In the following descriptions, it is assumed that, in a case where the processing in the present flowchart is started, the communication apparatus 101 has already started up an application in the application unit 306 and is connected to the virtual machine 104 and the storage 106 via the network as in the above-described cases.
However, to make it easier to understand the following description, it is assumed that the startup of the application is a restart after the abnormal termination, and the abnormal termination flag is ON.
It is also assumed that the restart is performed in a state where the image illustrated in
In step S801, determination is performed based on the abnormal termination flag as in the above-described cases. However, the abnormal termination flag is ON in this case as described above, and thus the processing proceeds to step S1401.
In step S1401, as described in step S802, the screen generation unit 304 displays the image illustrated in
In step S1402, the operation detection unit 303 determines whether an operation is performed on the apparatus specific screen. As described above, the current screen on the graphical user interface 204 is the screen illustrated in
In step S1404, the screen generation unit 304 determines whether the apparatus specific screen is to be generated. In step S806, the result that the text input field 700 is touched is received as the operation information result, and as described above, the screen generation unit 304 determines that the apparatus specific screen is to be generated in a case where the operation of touching the text input field 700 is performed. Thus, the screen generation unit 304 determines as YES in step S1404 based on the operation information result received in step S806, and the processing proceeds to step S1405. (Step S1405)
In step S1405, the screen generation unit 304 generates the apparatus specific screen. As described above, in a case where the result is that the text input field 700 is touched, the screen generation unit 304 generates the software keyboard as the apparatus specific screen. Specifically, the screen generation unit 304 generates the software keyboard based on the ID of the screen to be used to generate the software keyboard illustrated in
In step S1406, the screen generation unit 304 performs screen display to display the image acquired by adding the software keyboard (UI) generated in step S1405 to the image received in step S808 on the graphical user interface 204. The display on the graphical user interface 204 is not limited to the image acquired by adding the software keyboard (UI) to the image received in step S808 and may be either one of the software keyboard or the image received in step S808. According to the present embodiment, as described above, the software keyboard (UI) is located at the top, and thus the screen on the graphical user interface 204 is the screen illustrated in
In step S1407, the storage unit 302 stores the image used to generate the screen in step S1406 and the apparatus specific screen information (e.g., the ID of the screen required to generate the software keyboard). If the determination result in step S811 is NO, the processing proceeds to step S1402 again. As described below, in a case where the determination result in step S1402 is YES and the processing proceeds to step S1407 via step S1403, the operation is performed on the apparatus specific screen information (e.g., software keyboard). In this case, the storage unit 302 may store the operation information that is the content of the operation.
A case where the processing proceeds to step S1403 will be specifically described. It is assumed that a user performs a touch operation for inputting characters by touching the software keyboard currently displayed on the graphical user interface 204. For example, if the user performs a touch operation of inputting a letter “A” as illustrated in
In step S1403, the screen generation unit 304 updates the apparatus specific screen. Thereafter, the screen generation unit 304 displays the generated screen on the graphical user interface 204. In this case, the screen on the graphical user interface 204 is similar to the screen illustrated in
In a case where the application is abnormally terminated and restarted, the processing in step S1401 may be performed as follows.
In the processing in step S1401 at the time of restart, the screen generation unit 304 automatically executes processing based on the operation information of the touch operation of inputting “A” stored in the storage unit 302 so that the screen in the state after the operation appears.
In a case where the screen generation unit 304 performs display after the abnormal termination in this way, the screen generation unit 304 also functions as a UI operation execution unit that executes the operation on the software keyboard serving as a UI stored in the UI operation storage unit.
As described above, according to the third embodiment, it is possible to resume processing from the screen display before the abnormal termination when the application is abnormally terminated, in a case where the communication apparatus 101 generates the apparatus specific screen, combines the apparatus specific screen with the image of the rendering result of the web content, and displays the result on the communication apparatus 101.
The present invention is described above based on the specific embodiments. However, the present invention is not limited to the above-described embodiments.
Any modification and improvement of the above-described embodiments are also included within the technical scope of the present invention, and it will be apparent to those skilled in the art from the descriptions of the claims.
Embodiment(s) of the present invention can also be realized by a computer of a system or apparatus that reads out and executes computer executable instructions (e.g., one or more programs) recorded on a storage medium (which may also be referred to more fully as a ‘non-transitory computer-readable storage medium’) to perform the functions of one or more of the above-described embodiment(s) and/or that includes one or more circuits (e.g., application specific integrated circuit (ASIC)) for performing the functions of one or more of the above-described embodiment(s), and by a method performed by the computer of the system or apparatus by, for example, reading out and executing the computer executable instructions from the storage medium to perform the functions of one or more of the above-described embodiment(s) and/or controlling the one or more circuits to perform the functions of one or more of the above-described embodiment(s). The computer may comprise one or more processors (e.g., central processing unit (CPU), micro processing unit (MPU)) and may include a network of separate computers or separate processors to read out and execute the computer executable instructions. The computer executable instructions may be provided to the computer, for example, from a network or the storage medium. The storage medium may include, for example, one or more of a hard disk, a random-access memory (RAM), a read only memory (ROM), a storage of distributed computing systems, an optical disk (such as a compact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD) TM), a flash memory device, a memory card, and the like.
While the present invention has been described with reference to embodiments, it is to be understood that the invention is not limited to the disclosed embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.
This application claims the benefit of Japanese Patent Applications No. 2023-219907, filed Dec. 26, 2023, and No. 2024-130202, filed Aug. 6, 2024, which are hereby incorporated by reference herein in their entirety.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2023-219907 | Dec 2023 | JP | national |
| 2024-130202 | Aug 2024 | JP | national |
