Patent Application
20170307970
The present invention relates to a display device, a projector and a display control method for displaying an image corresponding to a position indicated by a pointer.
A technique in which a display device such as a projector detects the position of a digital pen (pointer) on a screen and projects an image (hand-drawn image) corresponding to a trace of the digital pen is known. This function is called an electronic blackboard function or electronic whiteboard function. A projector with the electronic blackboard function divides the screen into a plurality of areas, taking advantage of the large size of the screen, and draws hand-drawn images separately in the individual areas, using a plurality of digital pens (PTL 1 and PTL 2, for example).
PTL 1: JP-A-2013-168142
PTL 2: JP-A-2013-235416
In an electronic blackboard system, cut-and-paste and copy-and-paste functions, commonly used in personal computers, can be used. However, the techniques disclosed in PTL 1 and PTL 2 have a problem that if, after cutting and before pasting a virtual object with a first pointer, a different virtual object is cut with a second pointer, data of the different virtual object is stored in a storage area called a clipboard buffer and subsequently the different virtual object is pasted if an instruction to paste is given with the first pointer.
Meanwhile, the invention provides a technique which improves the convenience of the cut-and-paste or copy-and-paste function in a display device which displays images corresponding to the positions of a plurality of pointers.
The invention provides a display device including: a display unit which displays an image; a detection unit which detects a position of a first pointer and a position of a second pointer, on the image; a storage unit having a storage area corresponding to the first pointer and the second pointer; a storage control unit which causes a first virtual object associated with the first pointer to be stored in the storage area, with the first virtual object corresponding to a first identifier identifying the first pointer, and which causes a second virtual object associated with the second pointer to be stored in the storage area, with the second virtual object corresponding to a second identifier identifying the second pointer; and a pasting unit which pastes the first virtual object stored corresponding to the first identifier in the storage area to the image in response to an operation of the first pointer, and which pastes the second virtual object stored corresponding to the second identifier in the storage area to the image in response to an operation of the second pointer. According to this display device, cut-and-paste or copy-and-paste can be carried out separately with the first pointer and the second pointer.
This display device may include a switching unit which switches an operation mode of the pasting unit to one of a plurality of operation modes including a first mode and a second mode. In the first mode, the pasting unit may paste the first virtual object stored corresponding to the first identifier in the storage area to the image in response to an operation of the first pointer, and paste the second virtual object stored corresponding to the second identifier in the storage area, to the image in response to an operation of the second pointer. In the second mode, the pasting unit may paste a virtual object that is stored last in the storage area, of the first virtual object and the second virtual object, to the image in response to an operation of the first pointer or the second pointer. According to this display device, the mode can be switched between the mode in which cut-and-paste or copy-and-paste is carried out separately with the first pointer and the second pointer and the mode in which cut-and-paste or copy-and-paste is carried out with the first pointer and the second pointer in common.
The detection unit may detect a position of a third pointer on the image. The pasting unit may paste a virtual object that is stored last in the storage area, of the first virtual object and the second virtual object, to the image in response to an operation of the third pointer. According to this display device, cut-and-paste or copy-and-paste can be carried out with the third pointer in common with the first pointer and the second pointer.
The third pointer may be a finger of a user. According to this display device, cut-and-paste or copy-and-paste can be carried out with the user's finger in common with the first pointer and the second pointer.
The first virtual object may be a virtual object on which an instruction to copy is given using the first pointer. According to this display device, copy-and-paste can be carried out on a virtual object on which an instruction to copy is given with the first pointer.
The first virtual object may be a virtual object drawn using the first pointer. According to this display device, cut-and-paste or copy-and-paste can be carried out on a virtual object drawn with the first pointer.
In the display device, the first virtual object may be a virtual object selected using the first pointer. According to this display device, cut-and-paste or copy-and-paste can be carried out on a virtual object selected with the first pointer.
The invention also provides a projector including: a projection unit which projects an image; a detection unit which detects a position of a first pointer and a position of a second pointer, on the image; a storage unit having a storage area corresponding to the first pointer and the second pointer; a storage control unit which causes a first virtual object associated with the first pointer to be stored in the storage area, with the first virtual object corresponding to a first identifier identifying the first pointer, and which causes a second virtual object associated with the second pointer to be stored in the storage area, with the second virtual object corresponding to a second identifier identifying the second pointer; and a pasting unit which pastes the first virtual object stored corresponding to the first identifier in the storage area to the image in response to an operation of the first pointer, and which pastes the second virtual object stored corresponding to the second identifier in the storage area to the image in response to an operation of the second pointer. According to this projector, cut-and-paste or copy-and-paste can be carried out separately with the first pointer and the second pointer.
Moreover, the invention provides a display control method in a display device including a display unit which displays an image and a storage unit having a storage area corresponding to a first pointer and a second pointer. The method includes: detecting a position of the first pointer and a position of the second pointer, on the image; causing a first virtual object associated with the first pointer to be stored in the storage area, with the first virtual object corresponding to a first identifier identifying the first pointer, and causing a second virtual object associated with the second pointer to be stored in the storage area, with the second virtual object corresponding to a second identifier identifying the second pointer; and pasting the first virtual object stored corresponding to the first identifier in the storage area to the image in response to an operation of the first pointer, and pasting the second virtual object stored corresponding to the second identifier in the storage area to the image in response to an operation of the second pointer. According to this display control method, cut-and-paste or copy-and-paste can be carried out separately with the first pointer and the second pointer.
The display device 1 includes a display unit 10, a detection unit 11, a storage unit 12, a storage control unit 13, a pasting unit 14, and a switching unit 15. The display unit 10 displays an image corresponding to image data supplied thereto. The detection unit 11 detects the position of the first pointer and the position of the second pointer. The storage unit 12 has a storage area corresponding to the first pointer and the second pointer and stores a first virtual object and a second virtual object. The first virtual object is a virtual object associated with the first pointer. The second virtual object is a virtual object associated with the second pointer. A virtual object is an object that is virtual and expressed by data, for example, an image, line drawing, graphic pattern, icon or character string. The storage control unit 13 causes the first virtual object to be stored in the storage area of the storage unit 12, corresponding to a first identifier, and causes the second virtual object to be stored in the storage area of the storage unit 12, corresponding to a second identifier. The first identifier is an identifier which identifies the first pointer. The second identifier is an identifier which identifies the second pointer. The pasting unit 14 pastes the virtual objects stored in the storage area of the storage unit 12 to the image displayed by the display unit 10 in response to an operation of the pointers. The switching unit 15 switches the operation mode of the pasting unit 14 to one of a plurality of operation modes including a first mode and a second mode. In the first mode, the pasting unit 14 pastes the first virtual object stored corresponding to the first identifier in the storage area to the image in response to an operation of the first pointer, and pastes the second virtual object stored corresponding to the second identifier in the storage area to the image in response to an operation of the second pointer. Meanwhile, in the second mode, the pasting unit 14 pastes the virtual object that is stored last in the storage area of the storage unit 12, of the first virtual object and the second virtual object, to the image in response to an operation of the first pointer or the second pointer.
The display device 1 includes a CPU (central processing unit) 100, a ROM (read only memory) 101, a RAM (random access memory) 102, a storage unit 103, an IF (interface) unit 104, an image processing circuit 105, a projection unit 106, an image pickup unit 107, and an operation panel 108.
The CPU 100 is a control device which controls each component of the display device 1. The ROM 101 is a non-volatile storage device in which various programs and data are stored. The RAM 102 is a volatile storage device which stores data, and functions as a work area when the CPU 100 executes processing. The storage unit 103 is a non-volatile storage device which stores data and programs, such as an HDD (hard disk drive) or SSD (solid state drive), for example. In this example, the storage unit 103 stores a program for causing the display device 1 to implement the whiteboard function (hereinafter referred to as a “whiteboard program”).
The IF unit 104 is an interface which mediates the communication of signals or data to and from an external device which serves as a video source. The IF unit 104 has a terminal for communicating signals or data to and from the external device (for example, a VGA terminal, USB terminal, wired LAN interface, S terminal, RCA terminal, HDMI (High-Definition Multimedia Interface: trademark registered) terminal, or the like) and a wireless LAN interface.
The image processing circuit 105 performs predetermined image processing on a video signal inputted thereto (hereinafter referred to as an “input video signal”). The projection unit 106 projects an image on a screen according to the video signal on which image processing is performed. The projection unit 106 has a light source, light modulator, an optical system, and a drive circuit for these (none of which is illustrated). The light source is a lamp such as a high-pressure mercury lamp, halogen lamp or metal halide lamp, or a solid-state light source such as an LED (light emitting diode) or laser diode. The light modulator is a device which modulates the light cast from the light source, according to the video signal, and includes a liquid crystal panel, for example. The optical system is an element which projects the light modulated by the light modulator, onto the screen, and has a lens and prism, for example. In this example, the light source and the light modulator are provided for each color component. Specifically, separate light sources and light modulators are provided for the primary colors of red, green and blue, respectively. The light modulated by the respective liquid crystal panels is combined by the optical system and projected on the screen.
The image pickup unit 107 picks up an image of an area including the screen where an image is projected, in order to specify the positions of the digital pen 2 and the digital pen 3. In this example, each of the digital pen 2 and the digital pen 3 has a light emitting element which outputs light with a unique wavelength, and a pressure sensor at the tip of the pen. As the pressure sensor detects pressure, the light emitting element outputs light. The CPU 100 detects the positions of the digital pen 2 and the digital pen 3, based on the positions of the points corresponding to the light with their wavelengths on the image picked up by the image pickup unit 107. The image pickup unit 107 picks up an image of the area including the screen on a predetermined cycle. Arranging the detected positions in time series provides the traces of the digital pens. The CPU 100 controls the image processing circuit 105 and the projection unit 106 to draw an image corresponding to the detected positions (trace) of the digital pen 2 and an image corresponding to the positions (trace) of the digital pen 3.
The operation panel 108 is an input device for the user to input instructions to the display device 1, and includes a keyboard, button or touch panel, for example.
In this example, the projection unit 106 is an example of the display unit 10 and an example of the projection unit. The image pickup unit 107 and the CPU 100 are an example of the detection unit 11. The RAM 102 is an example of the storage unit 12. The CPU 100 executing the whiteboard program is an example of the storage control unit 13, the pasting unit 14 and the switching unit 15.
Before starting the explanation of this embodiment, problems of the related-art technique will be described.
In this example, a user A using the digital pen 2 and a user B using the digital pen 3, that is, a total of two users, are drawing images using the whiteboard function of the display device 1. In
In Step S1, the user A draws a picture of a penguin. Meanwhile, the user B draws a picture of a bear.
In Step S2, the user A selects the picture of the penguin drawn by the user A him/herself. The user B selects the picture of the bear. The digital pen 2 and the digital pen 3 have a button to give an instruction to select a virtual object. If the user touches a virtual object or draws a graphic pattern surrounding virtual object while pressing this button, the virtual object is selected. As the virtual object is selected, the whiteboard program displays a menu including processing items that can be executed on the selected virtual object.
In Step S4, the user B copies the selected virtual object (in this example, the picture of the bear) to the clipboard buffer. That is, the user B selects the item of “copy” from the menu in
In Step S5, the user A gives an instruction to paste the virtual object onto the screen. For example, if the user A touches the screen with the digital pen 2 while pressing the button of the digital pen 2, the whiteboard program displays a menu including the item of “paste”. If the user A selects the item of “paste” from this menu, an instruction to paste is given.
In Step S6, the whiteboard program pastes the virtual object stored in the clipboard buffer, that is, in this example, the picture of the bear, to a designated position on the screen.
At this time, which virtual object the user A intends to paste depends on circumstances. For example, if the user A is aware that the user B has copied the picture of the bear, it is conceivable that the user A intends to paste the picture of the bear. However, if the user A is unaware that the user B has copied the picture of the bear, it is conceivable that the user A intends to paste the picture of the penguin. In this case, the picture of the bear is pasted against the intention of the user A. This embodiment deals with such problems.
In Step S11, the user A draws a picture of a penguin. Meanwhile, the user B draws a picture of a bear. In Step S12, the user A selects the picture of the bear drawn by the user A him/herself. The user B selects the picture of the bear.
In Step S13, the user A copies the selected virtual object (in this example, the picture of the penguin) to the clipboard buffer. That is, the user A selects the item of “copy” from the menu in
In Step S14, the user B copies the selected virtual object (in this example, the picture of the bear) to the clipboard buffer. That is, the user B selects the item “copy” from the menu in
In Step S15, the user A gives an instruction to paste the virtual object onto the screen. The user B, too, gives an instruction to paste the virtual object onto the screen.
In Step S16, the whiteboard program pastes the virtual objects. That is, the whiteboard program pastes the virtual object stored corresponding to the identifier C1 in the clipboard buffer, that is, in this example, the picture of the penguin, to the position indicated by the digital pen 2, and pastes the virtual object stored corresponding to the identifier C2 in the clipboard buffer, that is, in this example the picture of the bear, to the position indicated by the digital pen 3.
In the operation example 1, an example where the two digital pens are used by different users is described. However, the two digital pens can be used by one user. In this case, on assumption that the processing is underway as shown in
For example, an icon to access a screen for changing various settings is displayed on the screen displayed by the display device 1. The user can access this menu by touching this icon with the digital pen. This menu includes an item for switching between the operation modes.
In this case, in the shared mode, only the virtual object that is copied last may be stored in the clipboard buffer, or the order in which virtual objects are copied may be stored along with the virtual objects and the virtual object in the last place in the order may be pasted.
In the operation example 2, an example where the user changes a setting on the menu screen, thereby switching between the operation modes, is described. However, the whiteboard program may automatically switch between the operation modes. In this example, the whiteboard program switches between the operation modes according to whether the pointer used is a digital pen or a user's finger. Specifically, if a user's finger is used as the pointer, the operation mode is switched to the shared mode. If a digital pen is used as the pointer, the operation mode is switched to the independent mode. According to this operation example, the user can intuitively switch between the operation modes without any menu operations.
In this case, the whiteboard program has the function of recognizing an input made with the user's finger from an image picked up by the image pickup unit 107. Light parallel to the screen and with a uniform and unique wavelength is cast on the surface of the screen, and if the user's finger interrupts the light, the image pickup unit 107 captures and recognizes its reflection as the corresponding position. In this case, in order to discriminate this light from the light from the digital pen 2 or the digital pen 3, the light emission from the digital pens is carried out in a specific pattern, and the operations with the digital pens and with the finger are discriminated from each other by recognizing this pattern. Also, the function of recognizing the shape of a user's hand from an image picked up by the image pickup unit 107 may be provided. If the user's hand is in a specific shape (for example, with the forefinger up), the whiteboard program recognizes the hand as the pointer and recognizes the position of the tip of the forefinger as the position indicated by the pointer.
The invention is not limited to the above embodiment and can be carried out with various modifications. Several modifications will be described below. Two or more of the modifications below may be used in combination.
Which pointer identified by an identifier the virtual object should correspond to when stored in the clipboard buffer, that is, the relation between the position of the pointer and the virtual object, is not limited to the example described in the embodiment. In the embodiment, the whiteboard program stores a virtual object in the clipboard buffer in the state where the virtual object corresponds to the identifier identifying the pointer with which an instruction to copy the virtual object is given. However, the whiteboard program may store a virtual object in the clipboard buffer in the state where the virtual object corresponds to the identifier identifying the pointer with which the virtual object is drawn. For example, if the picture of the penguin drawn with the digital pen 2 is selected with the digital pen 3, the picture of the penguin may be stored in the clipboard buffer in the state where the picture of the penguin corresponds to identifier C1. As another example, the whiteboard program may store a virtual object in the clipboard buffer in the state where the virtual object corresponds to the identifier identifying the pointer with which the virtual object is selected.
If a finger is used as the pointer, the whiteboard program may provide a clipboard buffer for the finger. In this case, the clipboard buffer for the digital pen and the clipboard buffer for the finger are properly used.
The capacity of the clipboard buffer is not limited to two virtual objects. The clipboard buffer may store three or more virtual objects.
The hardware to implement the functions shown in
A part of the functions shown in
1 . . . display device, 2 . . . digital pen, 3 . . . digital pen, 10 . . . display unit, 11 . . . detection unit, 12 . . . storage unit, 13 . . . storage control unit, 14 . . . pasting unit, 15 . . . switching unit, 100 . . . CPU, 101 . . . ROM, 102 . . . RAM, 103 . . . storage unit, 104 . . . IF unit, 105 . . . image processing circuit, 106 . . . projection unit, 107 . . . image pickup unit, 108 . . . operation panel.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2014-226044 | Nov 2014 | JP | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/JP2015/005446 | 10/29/2015 | WO | 00 |
