Patent Application
20120140284
1. Field of the Invention
The present invention relates to an image forming apparatus having an operation panel removable from an apparatus main body, a method of controlling the image forming apparatus, and an image display apparatus.
2. BACKGROUND ART
Conventionally, in an image forming apparatus such as a multi-function peripheral, an image forming apparatus main body (hereinbelow, “main body”) is provided with an operation panel having a display unit. Upon error recovery or adjustment, a procedure for such purpose or the like is displayed on the display unit. Further, in a large sized machine having a finisher and various paper discharge processing devices in its main body, the operation panel is removable from the main body. In this arrangement, upon error recovery or adjustment, a user moves with the operation panel to a position to conduct an operation, and the user can perform the operation while watching the operation procedure displayed on the display unit of the operation panel.
Further, in recent years, a so-called augmented reality technique, capable of animation-like expression of augmented actual video image, by superposing an animation previously generated with CG or the like on a video image obtained with a camera, has been proposed (Japanese Patent Laid-Open No. 2001-92995).
When the scale of the system is increased by attaching various paper discharge and post processing devices to the image forming apparatus, the number of operations depending on the operator is increased. On the other hand, the operation procedure by the user becomes complicated, and it is impossible to perform sufficient maintenance operation merely with a manual or an operation guide displayed on the operation panel for an apparatus without a specialized operator.
An aspect of the present invention is to eliminate the above-mentioned problems with the conventional technology.
One of the features of the present invention is that an operation panel, removable from an apparatus main body and capable of wireless communication with the apparatus main body, is provided with an image capturing unit, and an operation guide image is combined with a video image, captured and displayed by the image capturing unit. With this arrangement, it is possible to improve operability of an operator.
According to one aspect of the present invention, there is provided an image forming apparatus for forming an image on a recording medium based on image data, comprising: mounting means for removably mounting an operation panel having a display unit, an imaging unit, and a wireless communication unit communicable with the image forming apparatus; communication means for wireless communication with the wireless communication unit of the operation panel; and control means for, in a status where the operation panel is not mounted on the mounting means, performing control to combine an image indicating an operation method with a video image obtained by the imaging unit and displayed on the display unit and display the combined image.
According to another aspect of the present invention, there is provided a control method for an image forming apparatus having a mounting unit to removably mount an operation panel having a display unit, an imaging unit, and a wireless communication unit communicable with an image forming apparatus, comprising: a communication step of performing wireless communication with the wireless communication unit of the operation panel; and a control step of, in a status where the operation panel is not mounted on the mounting unit, performing control to combine an image indicating an operation method with a video image obtained by the imaging unit and displayed on the display unit and display the combined image.
According to still another aspect of the present invention, there is provided an image display apparatus having a display unit and an imaging unit, removable from an image forming apparatus, comprising: wireless communication means for wireless communication with the image forming apparatus; display control means for, in a status where the operation panel is not mounted on the image forming apparatus, displaying a video image obtained by the imaging unit on the display unit; acquisition means for acquiring information on an error occurred in the image forming apparatus; recognition means for recognizing a part as an operation subject from the video image displayed by the display control means based on the information related to the error acquired by the acquisition means; and operation guide display means for combining an image indicating an operation guide corresponding to the part recognized by the recognition means, based on the information related to the error, with the video image, and displaying the combined image.
Further features and aspects of the present invention will become apparent from the following description of exemplary embodiment, with reference to the attached drawings.
The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate an embodiment of the invention and, together with the description, serve to explain the principles of the invention.
An embodiment of the present invention will now be described hereinafter in detail, with reference to the accompanying drawings. It is to be understood that the following embodiment is not intended to limit the claims of the present invention, and that not all of the combinations of the aspects that are described according to the following embodiment is necessarily required with respect to the means to solve the problems according to the present invention.
Further, in the present embodiment, an image forming apparatus (image processing apparatus) will be described, however, the present invention is not limited to the image forming apparatus.
The image forming apparatus (image processing apparatus) 1000 is a so-called print-on-demand (POD) machine which responds to various types of printing and bookbinding requirements by combining various options for saddle stitching, cutting, folding process and the like. The present embodiment shows an example of a system as a combination of the image forming apparatus main body (hereinbelow, “main body”) 1000, a paper deck (feeding unit) 5000, a binder (bookbinding unit) 6000 and a finisher (cutting unit or the like) 7000.
The main body 1000 is connected to a personal computer 9000 via a LAN 8000. The personal computer 9000 generates a print job such as print page generation/editing, setting of bookbinding, cutting, folding process, or the like. The generated print job is sent to the main body 1000 via the LAN 8000.
Further, in
Note that the optional devices including the paper deck 5000, the binder 6000 and the finisher 7000 are not directly related to the present invention, therefore, the detailed descriptions of these devices will be omitted.
As shown in
The controller board 1100 has a CPU 1101, a flash ROM 1102, a RAM 1103, a network interface card (NIC) 1104, a main channel controller 1105, and a sub channel controller 1106. Further, the controller board 1100 has a disk controller (DKC) 1107, a scanner interface (SIF) 1108 and a printer interface(PIF) 1109. These devices 1101 to 1109 are respectively connected via a bus 1110 to the CPU 1101.
The CPU 1101 controls the respective devices connected to the bus 1110, and executes a control program stored in the flash ROM 1102 and the HDD 1400. The RAM 1103 is used as a main memory and as a work area for the CPU 1101. The NIC 1104 performs bidirectional data transmission via the LAN 8000 with the personal computer 9000 and another image forming apparatus. The HDD 1400, accessed via the DKC 1107, is used for temporary storage of image data in addition to storage of the control program.
The scanner 1300 has a reading sensor, a document feeding mechanism and the like (all not shown). The reading sensor, the document feeding mechanism and the like are controlled in accordance with a software program executed by the CPU 1101 via the SIF 1108 packaged on the controller board 1100 and an SIF 1301 packaged on the scanner 1300. As a result, the scanner 1300 reads an original with the reading sensor, and transfers the obtained data via the SIF 1301 and the SIF 1108 to the controller board 1100.
Further, the print engine 1200 has an electrophotographic type print unit, a paper cassette, a paper feeding unit and the like (all not shown). A print request based on a print job is sent from the controller board 1100 via the PIF 1109 and the PIF 1201 packaged on the print engine 1200. The print unit, the paper feeding unit and the like are controlled in accordance with a program executed by the CPU 1101 via the PIF 1109 and the PIF 1201 as in the case of the print request. As a result, the print engine 1200 forms an image corresponding to a print request on paper.
The main channel controller 1105 and the sub channel controller 1106 are used upon communication between the main body 1000 and the operation panel 3000 that is removable from the main body 1000. The details of the channel controllers will be described later.
Next, the home position 2000 will be described.
As shown in
Next, the operation panel 3000 will be described.
The removable operation panel 3000 mainly has a main board 3100, the display unit (LCD) 3200, a touch panel 3300, a button device (buttons) 3400, and the connector 3500. The main board 3100 has a CPU 3101, an IEEE 802.11b module 3102, an irDA module 3103 and a power source controller 3104. Further, the main board 3100 has a display controller (DISPC) 3105, a panel controller (PANELC) 3106, a flash ROM 3107 and a RAM 3108. The respective modules 3101 to 3108 are connected with a bus (not shown) as in the case of the controller board 1100.
The CPU 3101 is a processor which controls the respective devices connected to the bus, and executes a control program stored in the flash ROM 3107. The RAM 3108 functions as a main memory and as a work area for the CPU 3101, and a storage area for video data to be displayed on the display unit 3200. The imaging module (camera) 3110 has its optical lens in the rear of the display unit 3200. When an operator holds the screen of the display unit 3200 in front of a subject, video image information in the rear surface direction of the display unit 3200 is stored into a frame buffer 3109. In the present embodiment, the video image information stored in the frame buffer 3109 is subjected to recognition processing using the CPU 3101. Then, the processed actual video image information and animation CG information previously stored in the flash ROM 3107 are superimposed in the RAM 3108, and the result is displayed on the display unit 3200.
The display controller (DISPC) 3105 transfers the video image loaded on the RAM 3108 to the display unit 3200 in correspondence with a request from the CPU 3101 and controls the display unit 3200. As a result, the image is displayed on the display unit 3200. The panel controller (PANELC) 3106 controls the touch panel 3300 and the button device 3400 in correspondence with a request from the CPU 3101. In accordance with the control, a depression position on the touch panel 3300, a key code of the depressed button on the button device 3400 and the like are returned to the CPU 3101. The power source controller 3104, connected to the connector 3500, is supplied with electric power from the power source module 1500 of the main body 1000 when it is in contact with the connector 2200 of the home position 2000. With this arrangement, electric power is supplied to the entire operation panel 3000 while a rechargeable battery 3111 connected to the power source controller 3104 is charged. When electric power is not supplied from the power source module 1500, electric power from the rechargeable battery 3111 is supplied to the entire operation panel 3000.
The IEEE 802.11b module 3102 establishes wireless communication with the IEEE 802.11b module 2101 on the home position 2000 based on the control of the CPU 3101, and serves as an interface between the operation panel 3000 and the main body 1000. The irDA module 3103 establishes infrared communication with the irDA module 2102 on the home position 2000 based on the control of the CPU 3101, and serves as an interface between the operation panel 3000 and the main body 1000.
Next, the wireless communication as a main channel according to the present embodiment will be described.
As described in
When plural main bodies exists within a range of a radio wave from the operation panel 3000, as in the case of an existing personal computer, the ESSIDs of plural communicable main bodies are displayed on the display unit of the operation panel 3000 so that one of the ESSIDs can be selected.
When communication with a destination has been established by association, the operation panel 3000 according to the present embodiment operates as a screen transfer type thin client. That is, most of the actual processings and video image generation are performed by the CPU 1101 of the main body 1000. Then the generated video data is sent by wireless communication in accordance with a predetermined protocol from the main body 1000 to the operation panel 3000. The CPU 3101 of the operation panel 3000 receives the video data, then controls the DISPC 3105 while loading the received video data on the RAM 3108, to display an image on the display unit 3200.
On the other hand, information related to the user's operation with respect to the touch panel 3300 and the button device 3400 of the operation panel 3000 is also sent by wireless communication in accordance with a predetermined protocol from the operation panel 3000 to the main body 1000. The information related to the operation includes a depression position on the touch panel 3300 and a key code corresponding to the depressed button of the button device 3400. The CPU 1101 of the main body 1000 receives the information related to the operation, controls the respective operations based on the sent information, updates video data in accordance with necessity, and sends the video data to the operation panel 3000 as described above.
As described above, in the system according to the present embodiment, wireless communication between the main body 1000 and the operation panel 3000 can be performed.
Next, the initial screen display of the image forming apparatus according to the embodiment of the present invention will be described. In the image forming apparatus according to the present embodiment, the removable operation panel 3000 is set so as to display an information display screen to indicate a job status, a job history, counter information, error information, device information and the like, as an initial screen. When the removed operation panel 3000 is moved closer to the main body 1000 and communicable with the main body 1000, the initial screen is displayed in correspondence with the above setting.
(1) Setting Item (1) “set ‘counter check screen’ as initial screen” is to set whether or not the counter check screen in
(2) Setting Item (2) “set ‘error status screen’ as priority screen” is to set whether or not the error screen in
(3) Setting Item (3) “select initial screen of ‘system status screen’ is to set whether or not the device information screen in
These initial screen settings can be set by the user by operating the operation panel 3000. These initial screen settings set with the operation panel 3000 are stored in the flash ROM 3107 of the operation panel 3000.
Hereinbelow, initial screen display processing in the image forming apparatus according to the present embodiment will be described with reference to
First, the initial screen display processing by the operation panel 3000 will be described with reference to
First, in step S1, the CPU 3101 of the operation panel 3000 determines the wireless communication status of the main channel (communication status determination processing), to determine whether or not wireless communication is being performed with the main body 1000. When it is determined that the wireless communication as a main channel is not performed, the process proceeds to step S2, in which the CPU 3101 transmits a request for establishment of main channel communication with the main body 1000 (SQ1 in
In step S5, the CPU 3101 determines whether or not the operation panel 3000 is placed in the home position 2000 on the main body 1000 and the operation panel 3000 and the main body 1000 are connected. Then, when it is determined that the operation panel 3000 and the main body 1000 are connected, the CPU 3101 terminates the processing of the present flowchart. On the other hand, when it is determined that the operation panel 3000 and the main body 1000 are not connected, the process proceeds to step S6. Note that in step S6 and the subsequent steps, the CPU 3101 reads the initial screen settings (the settings (1) to (3) in
In step S6, the CPU 3101 determines whether or not the setting (1) in
On the other hand, in step S6, when it is determined that the setting (1) is not setting to set the “counter check screen” as an initial screen, the process proceeds to step S9. In step S9, the CPU 3101 determines whether or not the setting (2) in
On the other hand, when it is determined in step S9 that the setting (2) in
On the other hand, when it is determined in step S14 that the setting (3) is not setting to select the “device information screen” as an initial screen of the “system status screen”, the process proceeds to step S17. In step S17, the CPU 3101 determines whether or not the setting (3) is setting to select the “job history screen” as an initial screen of the “system status screen”. When it is determined that the setting (3) is setting to select the “job history screen” as an initial screen of the “system status screen”, the process proceeds to step S18. In step S18, the CPU 3101 issues a job history acquisition request to the main body 1000 (SQ6 in
On the other hand, when it is determined in step S17 that the setting (3) is not setting to select the “job history screen” as an initial screen of the “system status screen”, the process proceeds to step S20. In step S20, the CPU 3101 determines whether or not the setting (3) is setting to select the “job status screen” as an initial screen of the “system status screen”. When it is determined that the setting (3) is setting to select the “job status screen” as an initial screen of the “system status screen”, the process proceeds to step S21. In step S21, a job status acquisition request is issued to the main body 1000 (SQ7 in
On the other hand, when it is determined in step S23 that any currently-performed job does not exist, the process proceeds to step S26. In step S26, the CPU 3101 displays a copy stand-by screen (e.g., the screen in
Next, the details of the processing in step S27 in
First, in step S201, when the error information from the main body 1000 indicates depletion of paper, the process proceeds to step S202, in which it is determined whether or not the current mode is changed to an imaging mode by depression of an imaging mode key (not shown) displayed on the display unit 3200 by the operator. The imaging mode is a mode to display a video image obtained with the image module 3110 on the display unit 3200. Accordingly, when the operator performs imaging with the image module 3110 directed toward the front surface of the main body 1000 while watching the display unit 3200, video image information including a cassette to which recording paper is to be replenished is obtained. Accordingly, when it is determined that the current mode is the imaging mode, the process proceeds to step S203, in which frame image data from the frame buffer 3109 is temporarily stored in the RAM 3108.
The video image information stored as above is, e.g., as shown in
When it is determined in step S206 that the subject is displayed in the predetermined size, the process proceeds to step S207, in which the size of prepared operation guide animation is adjusted. This adjustment is performed for the purpose of changing the size of the cassette in the animation to a size approximately corresponding to the size of the cassette recognized in
The above operation guide display in step S27 using the augmented reality technique is basically maintained until the error status is released.
Next, an example of the error upon overflow of waste toner container will be described with reference to
The flow of this processing is realized similarly to the aforementioned processing described with reference to the flowchart of
In the recognition processing in step S204, first, a release lever of the waster toner container is recognized. When the release lever is recognized, it is displayed with a broken line 1444 as shown in
Note that in this case, the animation expresses processes of turning the release lever of the waste toner container at a predetermined angle to release the lock of the waste toner container, and taking the waste toner container to a front position. In this manner, the animation where the release lever and the waste toner container in approximately the same size are superimposed in the actual video image and the images are watermark-combined. Accordingly, a display expressing as if actual members move can be produced.
Note that although already explained in
The augmented reality technique according to the present embodiment can be adopted upon various error recovery and adjustment in the image forming apparatus. Accordingly, all the error recovery and adjustment operations detected by the main body 1000, the animation CG as operation guide and the recognition subject are associated with each other, and respectively used selectively in accordance with determination similar to that in step S201 in
Next, data processing in the main body 1000 will be described with reference to
First, in step S101, the CPU 1101 of the main body 1000 determines the wireless communication status of the main channel (communication status determination processing), and determines whether or not wireless communication is being performed with the operation panel 3000. Then, when it is determined that wireless communication as a main channel is not performed, the process proceeds to step S102. In step S102, the CPU 1101 broadcasts information including the ESSID as processing for main channel communication, to notify the operation panel 3000 of its own ESSID, then the process proceeds to step S103. In step S103, the CPU 1101 determines whether or not the operation panel 3000 has been detected based on whether or not the above request (SQ1 in
In step S105, the CPU 1101 determines the presence/absence of the counter information acquisition request (SQ3 in
On the other hand, when it is determined in step S105 that the counter information acquisition request has not been received, the process proceeds to step S107. In step S107, the CPU 1101 determines the presence/absence of the error information acquisition request (SQ4 in
On the other hand, when it is determined in step S107 that the error information acquisition request has not been received, the process proceeds to step S109. In step S109, the CPU 1101 determines the presence/absence of the device information acquisition request (SQ5 in
In step S111, the CPU 1101 determines the presence/absence of the job history acquisition request (SQ6 in
Next, a particular example of the operation using the operation panel 3000 according to the embodiment will be described using
In this example, the operation panel 3000 is previously set to select the “job status screen” as an initial screen of the “system status screen” (
In
Note that in the above embodiment, the animation information is stored in the flash ROM 3107 of the operation panel 3000, however, the present invention is not limited to this arrangement. For example, control may be performed such that the animation information is stored in the HDD 1400 of the main body 1000, then in step S207 of
Further, the recognition unit to recognize a part as the operation subject from a video image captured and displayed on the display unit may be included not only in the operation panel 3000 but also in the main body 1000. In such case, it is necessary to transmit video image information obtained with the operation panel 3000 to the main body 1000 by main channel communication.
As described above, according to the present embodiment, when the operator has to perform maintenance or adjustment work, the operator uses the operation panel 3000, which is capable of wireless communication with the main body 1000 and which has a camera function, to display a video image of a peripheral area of a particular work position in the image forming apparatus. Then, the operation subject is automatically recognized from the video image. Then previously-stored operation guide animation CG is watermark-combined with the recognized operation subject and the combined image is displayed for explaining a necessary operation. With this arrangement, it is possible to guide the operator to perform a proper and quick operation.
Further, as the video image capture is performed with the camera provided in the rear of the operation panel 3000, the operator can work while observing the operation subject and the display on the operation panel 3000 simultaneously.
Aspects of the present invention can also be realized by a computer of a system or apparatus (or devices such as a CPU or MPU) that reads out and executes a program recorded on a memory device to perform the functions of the above-described embodiment, and by a method, the steps of which are performed by a computer of a system or apparatus by, for example, reading out and executing a program recorded on a memory device to perform the functions of the above-described embodiment. For this purpose, the program is provided to the computer for example via a network or from a recording medium of various types serving as the memory device (for example, computer-readable medium).
While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiment. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.
This application claims the benefit of Japanese Patent Application No. 2010-245705, filed Nov. 1, 2010, which is hereby incorporated by reference herein in its entirety.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2010-245705 | Nov 2010 | JP | national |
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/JP2011/073754 | 10/7/2011 | WO | 00 | 11/29/2011 |
