This patent application is based on and claims priority pursuant to 35 U.S.C. § 119(a) to Japanese Patent Application No. 2019-024912, filed on Feb. 14, 2019, in the Japan Patent Office, the entire disclosure of which is hereby incorporated by reference herein.
The present invention relates to an image forming apparatus, a display control method, and a recording medium.
There are conventionally known image forming apparatuses such as a multifunction peripheral (MFP) in which operation instructions are given to a control panel.
For example, a printing setting for an image forming apparatus may be instructed by voice, and the screen for a setting item displayed on a control panel changes in response to the user's voice. For example, it is possible to display at least one intermediate layer in order to check or change a setting item in an upper layer than the layer on the screen for the setting item specified by voice.
According to the above-described voice operation technique, the layers are tracked so that the setting details for a setting item may be checked. However, there is a disadvantage that it is difficult to simultaneously set a plurality of setting items due to the restriction of the existing screen flow.
Example embodiments include an image forming apparatus including circuitry to acquire a voice instructing to set a plurality of setting items to be used for executing image forming, control a display to display a setting screen indicating a plurality of setting items having been set using one or more keywords recognized from the voice instructing to set the plurality of setting items, and perform image forming based on the plurality of setting items having been set based on the voice.
Example embodiments include a method for controlling display of the above-described image forming apparatus, and a control program for controlling display of the above-described image forming apparatus.
A more complete appreciation of the disclosure and many of the attendant advantages and features thereof can be readily obtained and understood from the following detailed description with reference to the accompanying drawings, wherein:
The accompanying drawings are intended to depict embodiments of the present invention and should not be interpreted to limit the scope thereof. The accompanying drawings are not to be considered as drawn to scale unless explicitly noted.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present invention. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise.
In describing embodiments illustrated in the drawings, specific terminology is employed for the sake of clarity. However, the disclosure of this specification is not intended to be limited to the specific terminology so selected and it is to be understood that each specific element includes all technical equivalents that have a similar function, operate in a similar manner, and achieve a similar result.
Referring to the accompanying drawings, an embodiment of an image forming apparatus, an image forming method, and a control program is described below in detail.
Schematically, in the printing system 1, the MFP 9 captures the voice produced by the user through a microphone 971 in a microphone/speaker 970 provided in the MFP 9 (1). The MFP 9 converts the voice data into digital data and transmits the digital voice data to the voice recognition server 5 (2).
The voice recognition server 5 performs a voice recognition process (3) and transmits a voice recognition result to the MFP 9 (4).
The MFP 9 changes the setting details based on the voice recognition result transmitted from the voice recognition server 5 and applies the setting details to display the result on a screen (5). The MFP 9 outputs the changed setting details, etc. via a speaker 972 in the microphone/speaker 970 by voice (6).
First, an example of the hardware configuration of the MFP 9 is described.
The controller 910 includes, as the primary part of the computer, a central processing unit (CPU) 901, a system memory (MEM-P) 902, a north bridge (NB) 903, a south bridge (SB) 904, an application specific integrated circuit (ASIC) 906, a local memory (MEM-C) 907 as a storage area, a hard disk drive (HDD) controller 908, and a hard disk (HD) 909 as a storage area. The NB 903 and the ASIC 906 are coupled to each other via an accelerated graphics port (AGP) bus 921.
The CPU 901 includes a processor that executes an operating system (OS) and an application to perform the overall control on the MFP 9. The NB 903 is a bridge to couple the CPU 901, the MEM-P 902, the SB 904, and the AGP bus 921. The NB 903 includes a memory controller that controls, for example, the reading and writing from and to the MEM-P 902, a peripheral component interconnect (PCI) master, and an AGP target.
The MEM-P 902 includes a read only memory (ROM) 902a that includes a memory that stores programs and data for performing each function of the controller 910, and a random access memory (RAM) 902b that is used as, for example, a memory to load programs or data and execute drawing for printing from the memory. The ROM 902a and the RAM 902b store programs such as the OS and applications of the MFP 9. The ROM 902a and the RAM 902b also store a setting value for a device and the IP address of a target device to be synchronized. A program stored in the RAM 902b may be provided by being recorded in a recording medium readable by a computer, such as a compact disc read only memory (CD-ROM), a compact disk-recordable (CD-R), or a digital versatile disk (DVD), in the form of file installable or executable.
A program executed by the MFP 9 may be provided by being stored on a computer connected via a network, such as the Internet, and being downloaded via the network. A program executed by the MFP 9 may be provided or distributed via a network such as the Internet.
The SB 904 is a bridge to couple the NB 903 to a PCI device or a peripheral device. The ASIC 906 is an integrated circuit (IC) used for image processing including a hardware element used for image processing. The ASIC 906 serves as a bridge to couple the control panel 940, the USB I/F 960, the AGP bus 921, a PCI bus 922, the HDD controller 908, and the MEM-C 907. The ASIC 906 includes a PCI target, an AGP master, an arbiter (ARB) that is the core of the ASIC 906, a memory controller that controls the MEM-C 907, a plurality of direct memory access controllers (DMACs) that, for example, rotates image data by using a hardware logic, etc., and a PCI unit that transfers data to a scanner 931 and a plotter 932 via the PCI bus 922. The ASIC 906 may be coupled to a USB interface or an Institute of Electrical and Electronics Engineers 1394 (IEEE 1394) interface.
The MEM-C 907 is a local memory used as a copy image buffer and a code buffer. The HD 909 is a storage to store image data, store font data used during printing, and store a form. The HDD controller 908 controls the reading or writing data from or to the HD 909 under the control of the CPU 901. The AGP bus 921 is a bus interface for a graphics accelerator card developed to increase the speed of graphics processing. The AGP bus 921 directly accesses the MEM-P 902 at a high throughput to increase the speed of the graphics accelerator card.
The short-range communication circuit 920 includes a short-range communication circuit antenna 920a. The short-range communication circuit 920 includes a communication circuit for near field communication (NFC), Bluetooth (registered trademark), etc.
The engine control unit 930 includes the scanner 931 and the plotter 932. The scanner 931 includes a device that reads a sheet document. The plotter 932 includes a device that prints an image on the sheet document read by the scanner 931 or an electronic image transmitted from an external device, such as a personal computer (PC), onto a transfer sheet. The scanner 931 or the plotter 932 includes an image processor that performs error diffusion, gamma conversion, etc.
The control panel 940 is used when the OS or an application displays some information to the user or when some instruction is received from the user. The control panel 940 includes: a panel display section 940a, such as a touch panel, which displays, for example, the current set value or the selection screen to receive an input from the user; and a control panel 940b including, for example, a numeric keypad, which receives the set value for a condition regarding image formation, such as the set condition for the density, and a start key for receiving a copy start instruction. The controller 910 controls the entire MFP 9 to control for example drawing, communication, and input from the control panel 940.
The MFP 9 allows the document box function, the copy function, the print function, and the facsimile function to be sequentially selected by using an application selection key on the control panel 940. The document box mode is set when the document box function is selected. The copy mode is set when the copy function is selected. The print mode is set when the print function is selected. The facsimile mode is set when the facsimile function is selected.
The network I/F 950 is an interface for the data communication with other devices (the voice recognition server 5 according to the present embodiment) via the communication network 7. The network I/F 950 is a network communication unit for Ethernet (registered trademark), Wi-Fi (registered trademark), etc.
The USB I/F 960 is an interface to couple, for example, the microphone/speaker 970. The microphone/speaker 970 includes: the microphone 971 that captures sound; and the speaker 972 that outputs sound. The microphone/speaker 970 further includes an analog-to-digital (A/D) converter that converts an analog sound to a digital sound. The microphone/speaker 970 and the MFP 9 may be indirectly coupled to each other.
Next, an example of the hardware configuration of the voice recognition server 5 is described. For example, the voice recognition server 5 is a server (PC). Instead of the server (PC), the voice recognition server 5 may be integrated with, for example, the MFP 9 to execute voice recognition inside the MFP 9.
The CPU 501 controls the overall operation of the voice recognition server 5. The ROM 502 stores programs, such as initial program loader (IPL), used to drive the CPU 501. The RAM 503 is used as a work area for the CPU 501. The HD 504 stores various types of data for programs. The HDD controller 505 controls reading and writing of various types of data from and to the HD 504 under the control of the CPU 501. The display 506 presents various types of information such as cursor, menu, window, characters, or images. The external device connection I/F 508 is an interface to couple various external devices. In this case, the external device is, for example, a USB memory or a printer. The network I/F 509 is an interface for the data communication via the communication network 7. The data bus 510 is an address bus, a data bus, etc. for electrically coupling components, such as the CPU 501 illustrated in
The keyboard 511 is one type of input device including a plurality of keys to input characters, numerical values, various instructions, etc. The pointing device 512 is one type of input device to, for example, select or execute various instructions, select the processing target, or move the cursor. The DVD-RW drive 514 controls the reading or writing of various types of data from or to a DVD-RW 513 that is an example of a removable recording medium. DVD-R, etc. as well as DVD-RW may be used. The medium I/F 516 controls the reading or writing (storing) of data from or to a recording medium 515 such as a flash memory.
A program executed by the voice recognition server 5 may be provided by being recorded in a recording medium readable by a computer, such as a CD-ROM, a flexible disk (FD), a CD-R, a DVD, or a flash memory, in the form of file installable or executable.
A program executed by the voice recognition server 5 may be stored in a computer connected via a network such as the Internet and provided by being downloaded via the network. A program executed by the voice recognition server 5 may be provided or distributed via a network such as the Internet.
Next, the functions provided by the printing system 1 are described. Each function of the printing system 1 is provided when the CPU of each device operates in accordance with a program stored in a storage device of the device. This is not a limitation, and some or all of the functions of the printing system 1 may be provided as hardware functions such as a field programmable gate array (FPGA) and an ASIC.
The MFP 9 includes a voice acquiring unit 91, a display control unit 92, an operation control unit 93, and a voice output unit 94. These functional units of the MFP 9 is implemented by the instructions of the CPU 901 according to a control program stored in any desired memory illustrated in
The voice acquiring unit 91 acquires, via the microphone 971, a voice regarding a plurality of setting items to be used for the execution of an image forming process. Such voice regarding the plurality of setting items instructs to set the plurality of setting items, and include, for example, a request for setting values of the plurality of setting items.
The display control unit 92 generates a setting screen or a copy top screen in which a plurality of setting items corresponding to a keyword included in the voice are designated. The display control unit 92 simultaneously displays a plurality of setting items on the setting screen. Alternatively, the display control unit 92 sequentially displays a plurality of setting items on the setting screen, for example, one page by one page.
The voice acquiring unit 91 acquires, via the microphone 971, the voice regarding the display mode of the setting screen displayed on the display control unit 92. In this case, the display control unit 92 updates the setting screen corresponding to the keyword included in the voice regarding the display mode. For example, the voice regarding the display mode of the setting screen instructs to display information regarding the plurality of setting items having been set in the setting screen, such as a request for displaying setting of a particular one (or particular ones) of the plurality of setting items having been set in the setting screen.
The display control unit 92 gives an instruction to the voice output unit 94 to output the utterance content regarding the setting items by voice. The voice output unit 94 controls the speaker 972 to output the utterance content instructed by the display control unit 92 by voice.
The operation control unit 93 causes the control panel 940 to display the setting screen generated and updated by the display control unit 92. The operation control unit 93 causes the control panel 940 to display the copy top screen generated by the display control unit 92.
The operation control unit 93 receives an operation regarding the display mode of the setting screen via the control panel 940. The voice acquiring unit 91 acquires the operation regarding the display mode of the setting screen displayed on the control panel 940, and the display control unit 92 updates the setting screen in response to the operation.
The voice recognition server 5 includes a voice recognizing unit 51. The voice recognizing unit 51 is implemented by the CPU 501 according to a control program stored in any desired memory illustrated in
The voice recognizing unit 51 recognizes a plurality of setting items corresponding to a keyword included in the voice and a request. The voice recognizing unit 51 recognizes the display mode regarding to a plurality of setting items corresponding to a keyword included in the voice.
Next, a process performed by the printing system 1 is described in detail.
The voice acquiring unit 91 notifies the display control unit 92 of the voice data on the user's utterance (Step S2).
The display control unit 92 transmits the voice data to the voice recognizing unit 51 of the voice recognition server 5 (Step S3), and the voice recognizing unit 51 of the voice recognition server 5 executes a voice recognition process (Step S4).
The voice recognizing unit 51 of the voice recognition server 5 notifies the display control unit 92 what kind of request the user's utterance content is and the result recognized as a setting item regarding the request (Step S5).
In the example illustrated in
When the user requests “double-sided→double-sided copy” or “stapling at two points on the left”, the user may confirm or change the setting on the screen as described later; therefore, a voice inquiry such as “Is the document side one-sided or double-sided?” is not given when the user says “double-sided”. Operating and checking every setting by voice is annoying and time-consuming. With the configuration according to the embodiment of the present invention, such a disadvantage may be prevented.
As illustrated in
According to the example illustrated in
The display control unit 92 generates the screen in which the setting items of “color”, “double-sided”, and “stapling” are continuously arranged and delivers the screen to the operation control unit 93 (Step S6).
The operation control unit 93 causes the control panel 940 to display the screen in which the setting items “color”, “double-sided”, and “stapling” are continuously arranged (Step S7).
The display control unit 92 gives an instruction to the voice output unit 94 about the utterance content indicating that the settings for “color”, “double-sided”, and “stapling” have been made (Step S8).
The voice output unit 94 controls the speaker 972 to give utterance “The settings for the color mode, the double-sided copy, and the stapling have been made. Check the details on the screen” (Step S9).
On the screen illustrated in
Therefore, according to the present embodiment, when the user gives utterance to display items that are partially or entirely hidden, the screen is updated and scrolled to the position to display the entire items.
Returning back to
The voice acquiring unit 91 notifies the display control unit 92 of the voice data on the user's utterance (Step S11).
The display control unit 92 transmits the voice data to the voice recognizing unit 51 of the voice recognition server 5 (Step S12), and the voice recognizing unit 51 of the voice recognition server 5 executes a voice recognition process (Step S13).
The voice recognizing unit 51 of the voice recognition server 5 notifies the display control unit 92 what kind of request the user's utterance content is and the result recognized as the (multiple) targets (Step S14). According to the present embodiment, as the user says “Show me the settings of stapling”, the voice recognizing unit 51 of the voice recognition server 5 recognizes that the user's utterance is a “display request” and the target is “stapling” and notifies the information to the display control unit 92.
The display control unit 92 updates the screen so that the setting items for “stapling” are visible and delivers the screen to the operation control unit 93 (Step S15).
The operation control unit 93 causes the control panel 940 to display the screen that has been updated to be visible the setting items for “stapling” (Step S16).
The display control unit 92 gives the voice output unit 94 an instruction about the utterance content indicating that an update has been made so that the setting items for “stapling” are visible (Step S17).
The voice output unit 94 controls the speaker 972 to speak the current setting value for stapling and the details as to what the user is to do subsequently (Step S18). For example, the voice output unit 94 causes the speaker 972 to speak “Stapling on the diagonally upper left. Please say “Start copying” when you start copying. For other settings, please say the setting details or press the OK button”.
As illustrated in
The display control unit 92 updates the screen so that the setting items for “stapling” are visible in accordance with the content of the flick operation on the control panel 940 and delivers the screen to the operation control unit 93 (Step S21).
The operation control unit 93 causes the control panel 940 to display the screen that has been updated so that the setting items for “stapling” are visible (Step S22).
The user operates the OK button on the screen displayed on the control panel 940 in a state where the screen is displayed after being updated so that the setting items for “stapling” are visible as described above (Step S23).
The operation control unit 93 notifies the display control unit 92 of the content of the button operation performed by the user via the control panel 940 (Step S24).
The display control unit 92 generates a copy top screen in accordance with the content of the button operation and delivers the copy top screen to the operation control unit 93 (Step S25).
Then, the operation control unit 93 causes the control panel 940 to display the copy top screen (Step S26).
After the setting items are set via the setting screen as described above, the CPU 901 instructs the engine control unit 930 to perform image forming according to the setting items set via the setting screen.
As described above, according to the present embodiment, it is possible to simultaneously designate a plurality of setting items by voice so as to improve the operability. Furthermore, the setting screen for a plurality of items for which the user has given utterance is displayed simultaneously or sequentially so that the screen is displayed in accordance with the user's request. Thus, the user may confirm or change the settings as appropriate without feeling unsure.
According to the present embodiment, the control panel 940 displays the screen in which the setting items for “color”, “double-sided”, and “stapling” are continuously arranged; however, this is not a limitation, and the control panel 940 may display the screen of the setting items for “color”, “double-sided”, and “stapling” on a per-page basis.
When the user gives utterance to display the setting items for “stapling” that is not displayed, the page transitions to the page where the setting items for “stapling” are visible so that the screen is updated.
In the description above, for example, the image forming apparatus according to the embodiment of the present invention is applied to an MFP having at least two functions among a copier function, a printer function, a scanner function, and a facsimile function. However, the image forming apparatus according to the embodiment of the present invention is applicable to any image forming apparatus such as a copier, a printer, a scanner, or a facsimile machine.
Each of the functions of the described embodiments may be implemented by one or more processing circuits or circuitry. Processing circuitry includes a programmed processor, as a processor includes circuitry. A processing circuit also includes devices such as an application specific integrated circuit (ASIC), digital signal processor (DSP), field programmable gate array (FPGA), and conventional circuit components arranged to perform the recited functions.
The above-described embodiments are illustrative and do not limit the present invention. Thus, numerous additional modifications and variations are possible in light of the above teachings. For example, elements and/or features of different illustrative embodiments may be combined with each other and/or substituted for each other within the scope of the present invention.
For example, any screen such as the setting screen does not have to be generated at the display control unit 92. For example, any server communicable with the MFP 9 may generate the screen to be displayed at the MFP 9.
Any one of the above-described operations may be performed in various other ways, for example, in an order different from the one described above.
Number | Date | Country | Kind |
---|---|---|---|
2019-024912 | Feb 2019 | JP | national |