Patent Application
20160063763
This application is based upon and claims the benefit of priority from the prior Japanese Patent Application No. 2014-171877, filed on Aug. 26, 2014, the entire contents of which are incorporated herein by reference.
Embodiments relate to an image processor and an information processor for processing a captured image.
Electronic dictionary terminals and electronic dictionary software have been more and more used to look up the meaning of a certain word or translate a certain word into another language. The user of the electronic dictionary terminal can automatically obtain search results by only inputting a word, instead of manually turning over the pages of a paper dictionary to look up the word. Further, when using electronic dictionary software, a word to be searched can be selected through copy & paste or mouse click, which leads to more effective dictionary search.
However, in the existing electronic dictionary terminals and electronic dictionary software, search results are displayed on the display screen of the electronic dictionary terminal, or on the screen of a computer running the electronic dictionary software, which inevitably requires the user to take his/her eyes from the paper he/she is reading to check the search results of a word. Since this may possibly reduce the user's concentration, ideas for further improving convenience are required.
An image processor according to the present embodiment is an image processor for processing an image of an object visible through a transparent display. The image processor includes an acquisition unit and a controller. The acquisition unit acquires display information corresponding to the object and obtained by performing recognition processing on the image. The controller displays, on the transparent display, the display information.
Embodiment will now be explained with reference to the accompanying drawings.
Each of
In the information processor 100, the image capture unit 210 captures an image of an object, which is at least a part of the image visible through the transparent display 300, and the housing 200 performs recognition processing on the captured image to acquire display information corresponding to the object so that image determined by this display information is displayed on the transparent display 300.
The image capture unit 210, which is, e.g., a CMOS sensor or a CCD sensor, is incorporated in the housing 200. The transparent display 300 displays an image of a sheet of paper etc. which is arranged directly beneath and visible through the transparent display 300. The image of an object is included in the image visible through the transparent display 300. The image capture unit 210 captures the image of the object through the transparent display 300. The transparent display 300 may display a range 400 within which the image capture unit 210 can capture the object, using a rectangular frame for example. Within the possible capturing range, the image capture unit 210 comes into focus, and the image of the object included within this range is treated as the target of image processing.
In order that the image capture unit 210 vividly captures the image visible through the transparent display 300, the image capture unit 210 must come into focus on the surface of the transparent display 300. However, the distance between the image capture unit 210 and the transparent display 300 changes depending on the rotational angle of the housing 200. Thus, a click mechanism may be applied to the rotating shaft 201 and its bearing so that the housing 200 can be temporarily fixed when the image capture unit 210 comes into focus on the surface of the transparent display 300 with an appropriate rotational angle.
On the other hand, the housing 200 of
The housing 200 of
Note that each support part 228 has protrusions at both ends thereof. The protrusion at one end is engaged with the housing 200, and the protrusion at the other end is engaged with the transparent display 300. Thus, each of the housing 200 and the transparent display 300 must have holes to engage these protrusions with the holes. After the protrusions at the other ends are engaged with the holes provided on the side faces of the transparent display 300, the housing 200 is rotatable with respect to the transparent display 300 through the support parts 228.
Note that the support parts 228 may be integrated into a cover which protects the outer surface of the housing 200. In this case, there is no need to provide the protrusions at one ends of the support parts 228, and to provide the holes on the housing 200. When the support parts 228 are integrally attached to the cover storing the housing 200, the protrusions at the other ends of the support parts are engaged with the transparent display 300, which makes it possible to rotate the housing with respect to the transparent display 300 similarly to
In the case of
As stated above, even when the image capture unit 210 comes into focus on the surface of the transparent display 300, the range within which the image capture unit 210 can vividly capture an image is limited. Thus, a frame showing the range 400 within which the object can be extracted may be displayed on the surface of the transparent display 300. This frame may be displayed on the transparent display 300 based on an image signal from the housing 200, or may be previously printed on the surface of the transparent display 300.
The image signal from the housing 200 is wirelessly transmitted to the transparent display 300. In this case, e.g.,
Bluetooth (registered trademark) is used as a wireless method, but another wireless method may be employed instead.
On the other hand, in
Next, each component shown in
The image capture unit 210 captures an image of an object visible through the transparent display 300, and converts it into image data. This image capture unit 210 may have functions for changing the capture range and focus using a lens and electronic zoom. Instead, the image capture unit 210 may have a single focus lens.
In
Each component of the acquisition unit 220 shown in
The image recognition unit 221 corrects distortion of the data of a captured image. For example, the image recognition unit 221 generates correction data by performing matching processing between a captured image of a calibration pattern visible through the transparent display 300 and an image of the pattern before being captured, and uses this correction data to correct the captured image. Such correction data is, e.g., an inverse projective transformation matrix showing the relationship between an image of a calibration pattern visible through the transparent display 300 and an image of the pattern before being captured. The image recognition unit 221 converts image data using this inverse projective transformation matrix to remove distortion caused through capturing.
When capturing images while variously changing the rotational angle of the housing 200 with respect to the transparent display 300, correction data corresponding to each rotational angle is previously acquired and stored.
Further, the image recognition unit 221 removes noise from the image data removed of distortion. At this time, it is possible to use any one of or both of a spatial denoising filter and a temporal denoising filter. Then, the image recognition unit 221 extracts object data using the image data removed of noise, and performs recognition processing to obtain identification information of the object. Here, the identification information means information related to the object. For example, if the object is a character string, the character string obtained through the image recognition is treated as the identification information.
Further, the image recognition unit 221 may generate supplementary information for controlling the display state and display position of the object on the transparent display 300.
The information acquisition unit 222 obtains, from the storage 223, display information corresponding to the identification information of the object and obtained by the image recognition unit 221.
The storage 223 stores plural pieces of identification information and display information corresponding thereto. For example, the storage 223 stores display information of an English word corresponding to the identification information of an English character string. The display information in this case is a literal translation of the English word. That is, the storage 223 in this case is a relational database relating the literal translation to the display information corresponding to the identification information of the English word set as a primary key.
Note that the storage 223 can be formed as a nonvolatile memory such as a ROM, a flash memory, and a NAND-type memory. Further, for example, the storage 223 may be provided in an external device such as a server so that the information acquisition unit 222 accesses the storage 223 through a communication network such as Wi-Fi (registered trademark) and Bluetooth.
In the example shown in
Note that communication with the processing device may be performed using any one of or combination of Wi-Fi, Bluetooth, and mobile network communication.
The transparent display 300 can display an image determined by the image signal from the housing 200. That is, the transparent display 300 can display the image determined by the image signal over a sheet of paper arranged directly beneath the transparent display 300. The transparent display 300 is formed as, e.g., an organic EL display, which is a self-emitting flat display device requiring no backlight device.
The controller 230 controls the operation of each component in the information processor 100. The controller 230 may include a memory which stores application software for image processing, and a CPU which executes this application software. In this case, the CPU executes the application software to control the image capture unit 210, the acquisition unit 220, and the transparent display 300.
he controller 230 instructs the image capture unit 210 to capture an object. Further, the controller 230 instructs the acquisition unit 220 to acquire display information corresponding to the object, and performs control to display, on the transparent display 300, image determined by the acquired display information. In this way, the image determined by the display information is displayed on the transparent display 300 together with the image of the object visible through the transparent display 300. Accordingly, the user can see the display information corresponding to the object without taking his/her eyes from the transparent display 300, which improves convenience.
In the configuration shown in
The sensor 229 is an acceleration sensor, for example.
Hereinafter, an image processing method according to an embodiment will be explained referring to
The controller 230 judges whether a change in the image of an object visible through the transparent display 300 per unit time has a predetermined value Th1 or smaller, based on the output signal from the sensor 229 capable of detecting the movement of the transparent display 300 (S302). If the change has the predetermined value Th1 or smaller (in the case of YES), there is a strong possibility that the image capture unit 210 can capture a clear image, and thus the controller 230 instructs the image capture unit 210 to capture the image of the object. Upon receiving this instruction, the image capture unit 210 captures the image of the object, and transfers data of the captured image to the acquisition unit 220 (S303). Note that the image capture unit 210 may start capturing a moving image in synchronization with the timing of turning on the power. In this case, the controller 230 may judge whether a change in the image of the object per unit time has the predetermined value Th1 or smaller, based on the results obtained by detecting the movement of the moving image of image data captured in chronological order.
Next, the image recognition unit 221 obtains color information of at least one of hue, lightness, and chroma of the object and image surrounding the object, based on the image data (S304). Step S304 is provided to prevent the color of display information from being similar to the colors of the object and its background when displaying the display information on the transparent display 300.
Further, the image recognition unit 221 acquires image data removed of distortion (S305). In this step, distortion is removed from the image data using an inverse projective transformation matrix for example. The image recognition unit 221 removes noise from the image data removed of distortion (S306). Next, the image recognition unit 221 recognizes characters using the image data removed of noise, to generate text data (S307).
The image recognition unit 221 performs binarization to separate the image data into character regions and the other regions (S401). For example, in this binarization, the value of 0 is given to each pixel having a predetermined pixel value or smaller, and the value of 1 is given to each of the other pixels.
In
Next, the image recognition unit 221 extracts binarized data of pixel rows sandwiched between the line spaces, using the position information of the line spaces (S403).
Next, the image recognition unit 221 detects each space between words from the binarized data extracted at Step S403, and recognizes the binarized data sandwiched between interword spaces as a word to clip the binarized data of each word (S404).
Next, the image recognition unit 221 performs recognition processing on the binarized data of each word to convert it into text data (S405).
Next, the image recognition unit 221 judges, e.g., whether every word in the range 400 has been converted into text data (S406). If there is a line which has not been converted yet, Step
S403 and subsequent Steps should be repeated. The image recognition unit 221 ends Step S307 when all lines are completely converted.
By performing the steps of
The controller 230 instructs the transparent display 300 to display image determined by the display information using the auxiliary information (S309). For example, when the line space is larger than the character size, the controller 230 instructs to display the image of the literal translation in the line space under (in the Y-direction) the word. Here, the character size of the image may be the same as the character size of its corresponding word. Based on color information, the color of the image is set so that the display information can be distinguished from the image of the object and its background image.
Further, the character size may be changed depending on the line space. For example, it is desirable to display the image with a smaller character size depending on the size of the line space. In this case, the character may be displayed in a color (e.g., a complementary color of the object) which becomes more different from the color of the object as the character size is set smaller. This makes it easy to distinguish the object from the image even when the character of the image becomes smaller.
Further, when the line space has a predetermined value or smaller, the image may be displayed in a blank space other than the line space
Further, the word clarifying a character string by recognition processing may be displayed with underline image. Alternatively, the word may be enclosed, or the word or its background may be decorated. This makes it possible for the user to easily recognize the target of translation, which improves convenience.
Note that the controller 230 may display, on an external display (e.g., smartphone), detailed information on the usage of an English word corresponding to the object.
Next, the controller 230 judges whether a change in the image of an object visible through the transparent display 300 per unit time has a predetermined value Th2 or greater, based on the output signal from the sensor which detects the movement of the transparent display 300 (S310). If the change has the predetermined value Th2 or greater (in the case of YES), there is a strong possibility that a position gap is formed between the object and image, and thus the controller 230 stops displaying the image on the transparent display 300 (S311). This makes it possible to prevent the image which is not corresponding to the object from being displayed, and to prevent unnecessary image from being displayed in the recaptured image of the object.
The image capture unit 210, which continuously captures the image of the object when power is turned on in the example shown in the flow chart of
In the example explained in the above embodiments, an object including character strings is treated as a target. However, the present embodiment can be applied when recognizing the image of an object including information other than character strings.
For example, the object may be animal, plant, human face, car, etc. In this case, the image recognition unit 221 may change the algorithm for recognizing the captured image of the object, depending on the type of the object. For example, when the object includes a human face, the recognition algorithm for human faces should be used. Further, plural pieces of identification information stored in the storage 223 should be also changed corresponding to the identification information obtained through the recognition algorithm. For example, when a human face is included in the object, it is desirable to store, in the storage 223, a plurality of typical face patterns as identification information.
Instead, when a human face is included in the object, the storage 223 may store a plurality of portraits corresponding to plural pieces of identification information, as display information. As stated above, the display information should not be necessarily limited to character information.
How to display the image on the transparent display 300 of
While certain embodiments have been described, these embodiments have been presented by way of example only, and are not intended to limit the scope of the inventions. Indeed, the novel methods and systems described herein may be embodied in a variety of other forms; furthermore, various omissions, substitutions and changes in the form of the methods and systems described herein may be made without departing from the spirit of the inventions. The accompanying claims and their equivalents are intended to cover such forms or modifications as would fall within the scope and spirit of the inventions.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2014-171877 | Aug 2014 | JP | national |
