Patent Application
20140267440
The present invention relates to a display control apparatus, a display control system, a display control method, and a computer program product including a computer program for executing the display control method.
Various types of display apparatuses for use at a meeting are known in the prior art (see e.g., Japanese Laid-Open Patent Publication No. 2011-141835). The conventional display apparatus may be connected to a personal computer (PC) that outputs an output image to the display apparatus so that the display apparatus may display the output image. Further, a technique is known for enabling a user to draw characters and/or images on a display region of the display apparatus. In this way, the display apparatus may display the drawn image superposed on the output image output by the PC.
It is a general object of at least one embodiment of the present invention to provide a display control apparatus, a display control system, a display control method, and a computer program that can substantially obviate one or more problems caused by the limitations and disadvantages of the related art.
One particular object of at least one embodiment of the present invention is to provide a display control apparatus, a display control system, a display control method, and a computer program that can accurately display an output image output by an information processing apparatus even when the output image is switched or altered.
According to one embodiment of the present invention, a display control apparatus that prompts a display unit to display an output image output by an information processing apparatus includes a storage unit that stores a stored output image that is already displayed by the display unit in association with a drawn image that is drawn on the display unit by a user when the stored output image is displayed by the display unit; a determination unit that determines whether the output image output by the information processing apparatus corresponds to the stored output image stored in the storage unit; a storage control unit that prompts the storage unit to store the output image output by the information processing apparatus when the determination unit determines that the output image output by the information processing apparatus does not correspond to the stored output image stored in the storage unit; and a display control unit that prompts the display unit to display the output image that is output by the information processing apparatus and stored in the storage unit.
According to an aspect of the present invention, an output image output by an information processing apparatus may be accurately displayed on a display unit even when the output image is switched or altered.
In the following, embodiments of the present invention are described with reference to the accompanying drawings. It is noted that identical features and components used in different embodiments are given the same reference numerals and their descriptions are omitted.
The display control apparatus 200 and the display system 1000 are preferably adapted for use in a meeting. Accordingly, in the following descriptions, it is assumed that the display control apparatus 200 and the display system 1000 are used in a meeting.
The display unit 202 may include a projector or a flat panel made of liquid crystal or plasma with a screen size of about 40-60 inches, for example. The display unit 202 also has a touch panel function. A user (e.g., presenter of a meeting) may use drawing means to draw an image on the display unit 202. In a case where an output image is displayed by the display unit 202, the user may superpose the image drawn on the display unit 202 onto the output image displayed by the display unit 202 to present various items of information. It is noted that the drawing means may be the user's fingertips, a stylus pen, or an operations unit 262 (e.g., a mouse or keyboard) shown in
The operations unit 262 (e.g., mouse or keyboard) is mounted on the mounting table 204 so that the operations unit 262 may be used to input information to the display control apparatus 200. The front side of the printer accommodating unit 206 is arranged into a door and the door may be opened to accommodate a printer (not shown) for printing the output image displayed by the display unit 202.
The front side of the video accommodating unit 208 is arranged into a door and the door may be opened to enable the user to load or eject an external medium such as a video tape or a laser disk. The front side of the computer accommodating unit 210 is arranged into a door and the door may be opened to accommodate a computer (corresponding to a control unit 250, a main storage unit 252, and an auxiliary storage unit 254 shown in
The control unit 250 controls the operations of various units and performs data computation and processing within the computer (i.e., the computer accommodated inside the computer accommodating unit 210 shown in
The main storage unit 252 may be a ROM (Read-Only Memory) and a RAM (Random Access Memory), for example, that stores data and programs of basic software such as operating software and application software to be executed by the control unit 250.
The auxiliary storage unit 254 may be a HDD (Hard Disk Drive) that stores data associated with application software, for example. The external storage device interface 256 is as an interface between the display control apparatus 200 and a storage medium 258 (e.g., USB memory or CD-ROM) that is connected to the display control apparatus 200 via a data transmission line such as a USB (Universal Serial Bus).
The storage medium 258 stores a predetermined program that may be installed in the display control apparatus 200 via the external storage device interface 256 to be executed by the display control apparatus 200.
The external apparatus connection interface 259 may be a physical interface such as a VGA (Video Graphics Array) input connector or a USB port for establishing connection between the display control apparatus 200 and an external apparatus such as the information processing apparatus 100 (e.g., PC) shown in
The network interface 260 acts as an interface between the display control apparatus 200 and the information processing apparatus 100 that is connected to the display control apparatus 200 via a network such as a local area network (LAN) or a wide area network (WAN) made up of wired and/or wireless data transmission lines.
The operations unit 262 may be a mouse or a keyboard, for example, that enables a user to input various commands to the display control apparatus 200. For example, the operations unit 262 may be operated by the user to start or end an application installed in the display control apparatus 200.
It is noted that in
The touch panel 266 realizes the touch panel function of the display unit 202. The touch panel 266 includes a receiving unit 268 that receives a wireless signal from a drawing unit 270. The drawing unit 270 may be an electronic pen, for example, that may be operated on the touch panel 266 to generate a drawn image or to select an image being displayed. For example, the receiving unit 268 may receive a signal from the drawing unit 270 indicating that the touch panel 266 has been touched.
The touch panel 266 is arranged to overlap with the display unit 202. When an image is drawn by operating the drawing unit 270 on the touch panel 266, the drawn image is displayed together with the output image displayed by the display unit 202.
It is noted that in the example shown in
The display unit 202 of
To start the display control process of
For example, the display control apparatus 200 may be arranged to recognize the start of a meeting when an application downloaded in the display control apparatus 200 for executing a display control function according to an embodiment of the present invention is started. In another example, the display control apparatus 200 may be arranged to recognize the start of a meeting when the main switch of the display control apparatus 200 is turned on.
<First-Time Process>
In the following, exemplary process steps of a first-time process performed for the first time right after the start of a meeting (i.e., process performed after the predetermined time period t has elapsed from the time the display control apparatus 200 recognizes the start of the meeting) are described.
When the display control apparatus 200 recognizes the start of a meeting, the determination unit 300 determines whether an output image is stored in the storage unit 252 (S71).
In the present embodiment, the storage unit 252 is configured to store one or more output images output from the information processing apparatus 100 via the external apparatus connection interface 259 or the network interface 260. In the following, the output image stored in the storage unit 252 is referred to as “stored output image” and the image output by the information processing apparatus 100 is referred to as “output image.”
When the display control process of
In step S73, the storage control unit 304 captures a new output image output by the information processing apparatus 100 and stores the captured output image in the storage unit 252 as a stored output image.
In the present example, an image has not yet been drawn by the user so that no drawn image is stored in the display image list and only a stored output image X is stored as “display image 1”. The display image list of
In step S74, the display control unit 302 generates a composite image using the stored output image X (see
As is described above, in one preferred embodiment, the display control process of
<Subsequent Process>
In the following, exemplary process steps of a subsequent process performed after the first-time process (i.e., process performed after the predetermined time period t elapses from the time the first-time process is completed) are described. In the example described below, it is assumed that the information processing apparatus 100 is outputting the output image X to the display control apparatus 200 as is shown in
In step S71, the determination unit 300 determines whether a stored output image is stored in the storage unit 252. Since the storage unit 252 stores the stored output image X in the present example, the determination unit 300 determines that a stored output image is stored (YES in S71).
In step S72, the determination unit 300 determines whether the output image corresponds to the stored output image, and whether the output image position corresponds to the stored output image position. It is noted that the determination as to whether the output image corresponds to the stored output image may be referred to as “first determination” and the determination as to whether the output image position corresponds to the stored output image position may be referred to as “second determination.” Also, it is noted that “output image position” refers to the position of the output image within the display region 202a of the display unit 202.
It is noted that in the second determination, the determination unit 300 may determine that the output image position does not correspond to the stored output image position when the output image output by the information processing apparatus 100 is scrolled, for example. The scrolling of the output image may be performed at the information processing apparatus 100 (e.g., by drag-and-drop operations or window scrolling operations using a mouse) or at the display unit 202. For example, the user may perform scrolling operations on the display unit 202 using the touch panel function of the display unit 202 (e.g., sliding a finger against the display unit 202).
In the present example where the output image X (pie chart image) is output by the information processing apparatus 100 and the same output image is displayed on the display region 202a as is shown in
In step S76, if a drawn image is drawn by a user, the storage control unit 304 stores the drawn image in the storage unit 252 in association with the stored output image that has been determined t correspond to the output image output by the information processing apparatus 100. It is noted that the drawn image corresponds to an image drawn on the display unit 202 by the user (presenter) using drawing means (e.g., finger tips or a stylus pen) as is described above.
It is noted that in the present embodiment, the detection unit 306 determines whether an image has been drawn by the user. The detection unit 306 detects when the drawing means has come into contact with the display unit 202 and determines that an image has been drawn by the drawing means. In the exemplary case shown in
In step S77, the display control unit 302 combines the stored output image determined to correspond to the output image (the output image X shown in
Then, in step S75, the display control unit 302 prompts the display unit 202 to display the composite image (corresponding to the stored output image X in the present example). It is noted that at the time step S75 is completed, the output image output by the information processing apparatus 100 and the display image displayed by the display unit 202 may remain as is shown in
<Process for Displaying Drawn Image>
In the following, exemplary process steps of the display control process of
In step S71, since the stored output image X is stored in the storage unit 252 in the present example, the determination unit 300 makes a positive determination (YES). Then, the determination unit 300 determines that the output image X (i.e., pie chart image) corresponds to the stored output image X in the first determination of step S72, and determines that the output image position corresponds to the stored output image position in the second determination of step S72. Thus, the process proceeds to step S76.
In step S76, if a drawn image is detected, the storage control unit 304 stores the drawn image in association with the stored output image that has been determined to correspond to the output image. Since the drawn image x is drawn by the user in
In
In
In step S77, the display control unit 302 combines the stored output image X that has been determined to correspond to the output image X with the drawn images x1-xn stored in association with the stored output image X to generate a composite image. Then, in step S75, the display control unit 302 prompts the display unit 202 to display the composite image. In this way, the display unit 202 may display a composite image of the output image X and the drawn image x as is shown in
<Process when Output Image is Switched>
In the following, process steps of the display control process of
In this case, since the stored output image X is already stored in the storage unit 252, the determination unit 300 makes a positive determination (YES) in step S71. In the first determination of step S72, the determination unit 300 determines whether the output image corresponds to the stored output image. In the present example, since the output image Y is not yet stored in the storage unit 252, the determination unit 300 determines that the output image does not correspond to the stored output image. Thus, the process proceeds to step S73.
In step S73, the storage control unit 304 stores the new output image Y that is output by the information processing apparatus 100 in the storage unit 252.
In this way, the storage unit 252 may store one or more output images and/or drawn images that have already been displayed during a meeting. Also, the storage unit 252 may store one or more drawn images in association with a stored output image.
In the following, process steps of the display control process of
In step S77, the display control unit 302 combines the stored output image X determined to correspond to the output image output by the information processing apparatus 100 (stored output image X shown in
Also, in a case where the user draws a drawn image y (not shown) on the display unit 202 while the display image Y corresponding to the output image Y is displayed as is shown in
It is noted that in a case where the user switches an output image output by the information processing apparatus 100 without drawing any images on the display unit 202 while the output image output before the switch is displayed on the display unit 202, the storage unit 252 does not store any drawn images in association with the stored output image corresponding to the output image output before the switch.
According to one aspect of the present embodiment, the determination unit 300 determines whether an output image output by the information processing apparatus 100 corresponds to a stored output image stored in the storage unit 252. For example, in the case where the user draws an image x on the output image X as is shown in
In this way, when the output image X is switched to the output image Y, a display image corresponding to the output image Y may be accurately displayed without the drawn image x drawn on the output image X remaining displayed on the display unit 202.
According to another aspect of the present embodiment, in the case where the output image X and the drawn image x are displayed as is shown in
In the following, a second embodiment of the present invention for accommodating a case in which an output image is scrolled is described. In the example described below, it is assumed that a display image including the output image X and the drawn image x is displayed on the display region 202a of the display unit 202 as is shown in
In
In the following, process steps that are performed right after the output image X is scrolled to display the scrolled output image X′ (i.e., after the output image and display image change from
According to a first example, in step S71, the determination unit 300 makes a positive determination (YES). Then, in step S72, the determination unit 300 determines that the output image X′ corresponds to the stored output image X, but the position of the output image X′ does not correspond to the position of the stored output image X. Thus, the process proceeds to step S78.
In step S78, the calculation unit 308 calculates the offset distance d between the position of the output image X′ and the position of the stored output image X. It is noted that the manner in which the calculation unit 308 calculates the offset distance d is described later. The offset distance d includes a distance component in the X-axis direction and a distance component in the Y-axis direction.
Then, in step S79, the storage control unit 304 updates the stored output image X stored in the storage unit 252 to stored output image X′. Also, the storage control unit 304 updates the stored drawn image x stored in the storage unit 252 to drawn image x′. It is noted that the updating of the stored drawn image x to the drawn image x′ may be realized by moving the drawn image x by the offset distance d.
In the present example, it is assumed that the offset distance d calculated by the calculation unit 308 is +10 in both the X-axis direction and the Y-axis direction.
In step S80, the updated stored output image X and the updated drawn image x′ are combined to generate a composite image. Then, in step S75, the display control unit 302 prompts the display unit 202 to display the composite image.
By performing the above process steps S78, S79, and S80, the scrolled output image and the drawn image that is moved by the offset distance d may be displayed by the display unit 202. In this way, even when the user scrolls an output image (e.g., output image X scrolled to output image X′) the drawn image associated with the output image may be adjusted according to the scrolling movement of the output image (e.g., drawn image x updated to drawn image x′).
It is noted that in the above first example, when the output image is scrolled, the storage control unit 304 updates the stored output image X to stored output image X′ and updates the stored drawn image x to stored drawn image x′ in step S79. Then, the display control unit 302 uses the updated stored output image X′ and the updated drawn image x′ to generate a composite image.
In a second example, instead of having the storage control unit 302 perform step S79, the detection unit 306 may be arranged to detect when the output image is scrolled, and the calculation unit 308 may be arranged to calculated the offset distance d. Then, the display control unit 302 may be arranged to prompt the display unit 202 to display the scrolled output image and the associated drawn image that is moved by the offset distance d.
In this example, in order to enable the detection unit 306 to detect when the output image is scrolled, the information processing 100 has to have a mechanism for sending a scrolling signal to the display control apparatus 200 indicating that the output image has been scrolled.
It is noted that in the first example, the information processing apparatus 100 does not need such a mechanism for sending a scrolling signal to the display control apparatus 200, and the display of the drawn image may be adjusted according to the scrolling movement of the output image by simply having the determination unit 300 determine whether the output image corresponds to the stored output image.
On the other hand, in the second example, the drawn image may be adjusted according to the scrolling movement of the output image without requiring the storage control 302 unit to perform step S79.
According to an aspect of, the present embodiment, even when an output image is scrolled, a drawn image associated with the output image may be adjusted according to the scrolling movement of the output image.
In the following, the display image list that is generated during a meeting is described as a third embodiment of the present invention. The display image list such as that shown in
It is noted that the display list according to the present embodiment may be generated each time a meeting is held, for example. One meeting may be from the time the start of the meeting is recognized until the end of the meeting is recognized, for example. The start of a meeting may be recognized when an application downloaded in the display control apparatus 200 for executing a display control function according to an embodiment of the present invention is started or when the main switch of the display control apparatus 200 is turned on, for example. The end of a meeting may be recognized when the application is ended or when the main switch of the display control apparatus 200 is turned off, for example.
In one embodiment, a display image list may be generated each time a meeting is held, and the generated display image list may be deleted when the meeting ends. In this way, the storage load of the storage unit 252 may be reduced.
In another embodiment, rather than deleting the display image list generated in a meeting, the display image list may be stored as an archive in association with predetermined identification information. The predetermined identification information may be a user ID or the date of the meeting, for example.
Also, by storing a display image list in association with a user ID, the display image list may be protected from being accessed by other users so that security of the display image list may be ensured, for example.
In the following, a determination timing of the determination unit 300 is described as a fourth embodiment of the present invention.
The determination unit 300 may be arranged to make the determinations of step S72 of
In the latter case, to enable the display control apparatus 200 to recognize the switching of the output image, the information processing apparatus 100 has a mechanism for sending an image switch signal to the display control apparatus 200 when an output image is switched.
It is noted that when the determination unit 300 is arranged to make the determinations of step S72 periodically at intervals of a predetermined time period t, the information processing apparatus 100 does not need the mechanism for sending the image switch signal described above. On the other hand, when the determination unit 300 is arranged to make the determinations of step S72 when the output image is switched, the frequency of performing the determination process may be reduced.
[Determination Unit 300]
In the following, an exemplary manner in which the determination unit 300 performs a determination process is described. In the example described below, the determination unit 300 is arranged to make a determination using normalized cross-correlation.
In
By repeating the process steps of determining whether the image characteristic values of pixels covered by the determination blocks correspond and moving the determinations blocks, the determination process may be performed on the entire regions of the output image and the stored output image. When the determination process is completed for the entire regions of the output image and the stored output image, the number of determination block pairs that have been determined to correspond are counted. In the present example, the number of determination block pairs having the same image characteristic values represents the degree of correspondence between the output image and the stored output image.
Next, the determination unit 300 determines whether the degree of correspondence (number of determination block pairs that have the same image characteristic values) is equal to or above a predetermined threshold value. The threshold value may be set equal to the total number S of determination blocks that make up the output image or the stored output image, or the threshold value may be set equal to a number slightly less than the total number S (e.g., S−1). That is, assuming the output image and the stored output image are each made up of Nall×Mall pixels, S=(Nall×Mall)÷(N×M).
In the case where the threshold value is set equal to S, the output image and the stored output image are determined to correspond when the image characteristic values of all the pixels of the output image and all the pixels of the stored output image are exactly the same. In the case where the threshold value is set equal to S−1, the output image and the stored output image may be determined to correspond even if there are slight differences in their image characteristic values.
Slight differences between the output image and the stored output image may occur when a pointer of a mouse is included in the output image being captured, for example. In such a case, even if the position of the mouse pointer in the output image and the position of the mouse pointer in the stored output image are different, the output image and the stored image actually represent the same image. Thus, the output image and the stored output image may be determined to correspond if the threshold value is set equal to a value slightly less than S.
It is noted that the determination process of the determination unit 300 using the determination blocks is described above as one illustrative example, and the determination unit 300 may use other means to determine the correspondence between the output image and the stored output image.
Also, it is noted that although the determination process using the determination blocks is described above in connection with determining whether the output image and the stored output image correspond, the determination blocks may be used in a similar manner to determine whether the output image position and the stored output image position correspond, for example.
[Calculation Unit 308]
In the following, an exemplary manner in which the calculation unit 308 calculates the offset distance d is described.
As is shown in
Then, as is shown in
[Background Image]
It is noted that various applications may be downloaded into the display apparatus 200 including an application for displaying an operations image (menu image) of the display control apparatus 200, for example. An image displayed by such an application may be referred to as “background image” or “application image”.
In
In another embodiment, a display image may be switched from that shown in
Further, the present invention is not limited to these embodiments, and numerous variations and modifications may be made without departing from the scope of the present invention.
The present application is based on and claims the benefit of the priority dates of Japanese Patent Application Nos. 2011-262970 and 2012-151474, filed on Nov. 30, 2011, and Jul. 5, 2012, respectively, the entire contents of which are hereby incorporated by reference.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2011-262970 | Nov 2011 | JP | national |
| 2012-151474 | Jul 2012 | JP | national |
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/JP2012/078413 | 10/26/2012 | WO | 00 | 4/15/2014 |
