This application claims priority to and the benefit of Japanese Patent Application No. 2016-147890 filed Jul. 27, 2016, the entire contents of which are incorporated herein by reference.
This disclosure relates to an electronic device and a control method.
For example, a mobile terminal that uses an infrared sensor to recognize a gesture is known.
Such a mobile terminal then processes an input operation corresponding to the gesture on a window being displayed.
Only one window is displayed on the aforementioned mobile terminal.
Therefore, it would be helpful to provide an electronic device that can improve operability for a plurality of windows.
An electronic device according to one aspect includes:
a non-contact detection sensor;
a display configured to display a first window and a second window; and
a controller configured to determine a first gesture and a second gesture by referring to output of the non-contact detection sensor;
such that the controller
An electronic device according to another aspect includes:
a non-contact detection sensor;
a display configured to display a first window and a second window; and
a controller configured to determine a first gesture and a second gesture by referring to output of the non-contact detection sensor;
such that the controller
A control method according to another aspect is for an electronic device including a non-contact detection sensor, the control method including:
displaying a first window and a second window on a display;
determining a first gesture and a second gesture by referring to output of the non-contact detection sensor;
allocating operation of the first window to the first gesture, and
allocating operation of the second window to the second gesture.
A control method according to another aspect is for an electronic device including a non-contact detection sensor, the control method including:
displaying a first window and a second window on a display;
determining a first gesture and a second gesture by referring to output of the non-contact detection sensor;
allocating the first gesture to an operation to select between the first window and the second window, and
allocating the second gesture to an operation of whichever of the first window and the second window is selected.
The electronic device according to one of the embodiments of this disclosure can improve the operability of a plurality of windows.
In the accompanying drawings:
As illustrated in
The controller 11 is a processor such as a Central Processing Unit (CPU). The controller 11 may be a System-on-a-Chip (SoC) or other type of integrated circuit in which other components are integrated. The controller 11 may be configured by combining a plurality of integrated circuits. The controller 11 implements a variety of functions by comprehensively controlling operations of the electronic device 1.
In greater detail, the controller 11 refers as necessary to data stored in the storage 14. The controller 11 implements a variety of functions by executing instructions included in programs stored in the storage 14 to control other functional components, such as the display 13.
The camera 12 captures images of subjects around the electronic device 1. As an example, the camera 12 in one embodiment is a front camera provided on the face of the electronic device 1 where the display 13 is provided. The camera 12 is one example of a non-contact detection sensor and can detect a gesture by the user by referring to the result of image analysis performed on a captured image.
The display 13 is provided with a display device such as a liquid crystal display or an organic electro-luminescence panel. The display 13 may instead be provided with an inorganic electro-luminescence panel. The display 13 can display a plurality of windows. The display 13 can display information such as letters, images, symbols, or graphics. The display 13 may include a touchscreen and detect contact on the touchscreen by a finger, stylus pen, or other object. In this case, the display 13 can detect the position on the touchscreen contacted by a plurality of fingers, stylus pens, or other objects. The display 13 is provided in the electronic device 1 in
The storage 14 functions as a memory storing programs and data. The storage 14 temporarily stores the processing result of the controller 11. For example, the storage 14 stores the type of window operation allocated to a gesture. The storage 14 may include any storage device, such as a semiconductor storage device or a magnetic storage device. The storage 14 may also include a plurality of types of storage devices. The storage 14 may include a combination of a portable storage medium, such as a memory card, and an apparatus for reading the storage medium.
The programs stored in the storage 14 include applications that run in the foreground or the background and a control program that supports operations of the applications. The applications for example cause the controller 11 to execute processing corresponding to a gesture. The control program may, for example, be an Operating System (OS). The applications and the control program may be installed on the storage 14 through wireless communication by the communication interface 15 or from a storage medium.
The communication interface 15 is for communicating over a wired or wireless connection. The communication method of the communication interface 15 in one embodiment is prescribed by a wireless communication standard. For example, a cellular phone communication standard such as 2G, 3G, or 4G may be used as the wireless communication standard. Examples of cellular phone communication standards include Long Term Evolution (LTE), Wideband Code Division Multiple Access (W-CDMA), CDMA2000, Personal Digital Cellular (PDC), Global System for Mobile communications (GSM® (GSM is a registered trademark in Japan, other countries, or both)), and Personal Handy-phone System (PHS). Examples of wireless communication standards include Worldwide Interoperability for Microwave Access (WiMAX), Institute of Electrical and Electronics Engineers (IEEE) 802.11, Bluetooth® (Bluetooth is a registered trademark in Japan, other countries, or both), Infrared Data Association (IrDA), and Near Field Communication (NFC). The communication interface 15 may support one or more of the aforementioned communication standards.
The proximity sensor 16 is one example of a non-contact detection sensor. Without contact, the proximity sensor 16 detects information such as the relative distance from an object near the electronic device 1 and the movement direction (gesture) of the object. The proximity sensor 16 in one embodiment includes four visible light photodiodes that can detect white, red, green, and blue. The proximity sensor 16 can measure the relative distance from the object. The proximity sensor 16 also includes one infrared Light Emitting Diode (LED) that acts as a light source and four infrared photodiodes that detect the up, down, left, and right directions. An object is irradiated by infrared light from the infrared LED acting as a light source, and the proximity sensor 16 detects the movement direction of the object by the difference in time at which reflected light from the object is incident on each of the infrared photodiodes. Accordingly, the proximity sensor 16 can detect an air gesture (gesture) that the user of the electronic device 1 performs without touching the electronic device 1.
The illumination sensor 17 is one example of a non-contact detection sensor and detects the degree of illumination of surrounding light that is incident on the illumination sensor 17.
By referring to output from the proximity sensor 16, illumination sensor 17, or camera 12, the controller 11 can determine which gesture the user performed. As an example in one embodiment, the proximity sensor 16 is described as being used as the non-contact detection sensor. In one embodiment, either the first window D1 or the second window D2 is an active window that can accept an input operation. Therefore, upon receiving an operation corresponding to a gesture, the controller 11 executes the operation corresponding to the gesture on whichever of the first window D1 and the second window D2 is the active window. The operation corresponding to the gesture may, for example, be a page forward, a page back, vertical scrolling, horizontal scrolling, a pinch-in, a pinch-out, or other such operation.
The electronic device 1 in
The electronic device 1 provided with the display 13 may also be adopted in an in-vehicle control panel mounted in the steering wheel, car navigation, or dashboard of an automobile, or may be adopted in any other type of transportation.
The processing executed by the controller 11 of the electronic device 1 in one embodiment is described below. The controller 11 determines whether the electronic device 1 has entered a predetermined mode in response to user operation. The predetermined mode in one embodiment is, for example, a kitchen mode in which an application that displays a cooking recipe on the display 13 has been launched. The predetermined mode is not limited to this example, however, and may be any mode on the electronic device 1. With the electronic device 1 in the predetermined mode, the controller 11 for example executes the following processing upon detecting that a plurality of windows displaying a plurality of cooking recipes are displayed on the display 13. In other words, while a plurality of windows are being displayed, the controller 11 can allocate operation of the first window D1 to a first gesture and operation of the second window D2 to a second gesture (first control method). Instead of the first control method, the controller 11 can also allocate the first gesture to an operation to select between the first window D1 and the second window D2 and can allocate the second gesture to operation of the first window D1 or the second window D2 (second control method). Details on the first and second control methods are provided below.
The first gesture and the second gesture may be any combination of a horizontal gesture, vertical gesture, diagonal gesture, a gesture to move closer to or away from the electronic device 1, a gesture to draw a circle clockwise, a gesture to draw a circle counterclockwise, or other such gesture. For example, the horizontal gesture is performed in a direction roughly parallel to the transverse direction of the electronic device 1. The vertical gesture is performed in a direction roughly parallel to the longitudinal direction of the electronic device 1. The diagonal gesture is performed in a plane roughly parallel to the electronic device 1 in a direction not parallel to either the longitudinal direction or the transverse direction of the electronic device 1.
Next, the first and second control methods executed by the controller 11 are described in detail. For example, in accordance with a user setting, the controller 11 determines which of the first control method and the second control method to execute.
First Control Method
Upon determining that the display 13 is displaying the first window D1 and the second window D2, the controller 11 allocates operation of the first window D1 to the first gesture and operation of the second window D2 to the second gesture. For example, the controller 11 allocates a vertical scrolling operation on the first window to a horizontal gesture and allocates a vertical scrolling operation on the second window to a vertical gesture.
When the controller 11 detects a horizontal gesture by the user in the state illustrated in
When the controller 11 detects a vertical gesture by the user in the state illustrated in
Depending on the position passed through by the object of detection, such as the user's hand, the controller 11 can switch the active window.
The controller 11 allocates operation of the first window to a gesture performed in the upper area R1 and allocates operation of the second window to a gesture performed in the lower area R2.
In other words, when determining that a gesture has been performed on the upper area R1, the controller 11 executes an operation on the first window in accordance with the detected gesture. For example, when determining that the first gesture (for example, a vertical gesture) has been performed on the upper area R1, the controller 11 may execute a first operation (for example, vertical scrolling) on the first window. When determining that the second gesture (for example, a horizontal gesture) has been performed on the upper area R1, the controller 11 may execute a second operation (for example, horizontal scrolling) on the first window.
On the other hand, when determining that a gesture has been performed on the lower area R2, the controller 11 executes an operation on the second window in accordance with the detected gesture. For example, when determining that the first gesture (for example, a vertical gesture) has been performed on the lower area R2, the controller 11 may execute a third operation (for example, page forward) on the second window. When determining that the second gesture (for example, a horizontal gesture) has been performed on the lower area R2, the controller 11 may execute a fourth operation (for example, page back) on the second window.
The following illustrates a method for more accurately determining the area in which the user performed a gesture.
Therefore, by acquiring and determining the degree of change in the output from the illumination sensor 17 while acquiring the output from the proximity sensor 16, the controller 11 may determine whether the user performed a gesture in the upper area R3 or in the lower area R4. With this determination, the controller 11 for example can allocate operation of the first window to a gesture performed in the upper area R3 and allocate operation of the second window to a gesture performed in the lower area R4.
As described above, in the first control method, the controller 11 allocates operation of the first window to the first gesture and operation of the second window to the second gesture. As a result, for example when cooking, the user can display different windows for a plurality of recipes and perform operations on the windows without contacting the electronic device 1. The operability of a plurality of windows is thereby improved.
Second Control Method
Next, the second control method is described. The controller 11 allocates the first gesture to an operation to select between the first window and the second window. The controller 11 can also allocate the second gesture to an operation of whichever of the first window and the second window is selected. For example, when only vertical scrolling is possible on both of the first window and the second window, and horizontal scrolling is not possible on either window, the controller 11 allocates a horizontal gesture to an operation to select which of the first window and the second window to make active. The controller 11 allocates a vertical gesture to a vertical scrolling operation on whichever of the first window and the second window is the selected, active window.
In the second control method, the controller 11 allocates the first gesture to an operation to select between the first window and the second window and allocates the second gesture to an operation on whichever of the first window and the second window is selected. As a result, the user can select the window he wants to operate without contacting the electronic device 1 and can execute an operation on the window. The operability of a plurality of windows is thereby improved.
The electronic device 1 allocates operation of the first window to the first gesture and operation of the second window to the second gesture (step S1). For example, the first gesture may be a horizontal gesture, a gesture performed in the upper area of the electronic device 1, or other such gesture. The second gesture may be a vertical gesture, a gesture performed in the lower area of the electronic device 1, or other such gesture. The electronic device 1 acquires the output of the proximity sensor 16 (step S2). By referring to the output, the electronic device 1 determines whether the first gesture is detected (step S3). Upon detecting the first gesture (step S3: Yes), the electronic device 1 executes the operation allocated to the first gesture on the first window (step S4).
Conversely, when determining that the first gesture is not detected (step S3: No), the electronic device 1 determines whether the second gesture is detected (step S5). Upon detecting the second gesture (step S5: Yes), the electronic device 1 executes the operation allocated to the second gesture on the second window (step S6).
The electronic device 1 allocates the first gesture to an operation to select between the first window and the second window and allocates the second gesture to an operation of whichever of the first window and the second window is selected (step S11). The first gesture may, for example, be a horizontal gesture on a vertically scrolled window. The second gesture may, for example, be a vertical gesture on a vertically scrolled window. The electronic device 1 acquires the output of the proximity sensor 16 (step S12). By referring to the output, the electronic device 1 determines whether the first gesture is detected (step S13). Upon detecting the first gesture (step S13: Yes), the electronic device 1 switches the selected, active window to another window (step S14).
Conversely, when determining that the first gesture is not detected (step S13: No), the electronic device 1 determines whether the second gesture is detected (step S15). Upon detecting the second gesture (step S15: Yes), the electronic device 1 executes the operation allocated to the second gesture on whichever of the first window and the second window is the selected, active window (step S16).
In the above embodiment, only one proximity sensor 16 is provided at the lower portion of the electronic device 1. In another embodiment, as illustrated in
In the first control method, the controller 11 allocates operation of the first window to the first gesture and operation of the second window to the second gesture. As one example in another embodiment, three windows are displayed in partial overlap on the display 13. In this case, the controller 11 may allocate operation of the first window to the first gesture, operation of the second window to the second gesture, and operation of the third window to the third gesture. Alternatively, the controller 11 may designate the three overlapping windows as the first window, the second window, and the third window in order from the front. The controller 11 may then allocate operation of the first window to the first gesture and operation of the second window to the second gesture. In other words, the controller 11 may assign priorities to the plurality of windows and refrain from allocating a gesture to a window with a low assigned priority.
In the second control method, for example when only vertical scrolling is possible on both of the first window and the second window, and horizontal scrolling is not possible on either window, the controller 11 allocates a horizontal gesture to an operation to select the first window or the second window. As one example in another embodiment, the first gesture is allocated to an operation to select among the first window, the second window, and the third window, since three windows are displayed on the display 13. The same approach is taken when four or more windows are displayed on the display 13.
In the second control method, for example when only vertical scrolling is possible on both of the first window and the second window, and horizontal scrolling is not possible on either window, the controller 11 allocates a horizontal gesture to an operation to select the first window or the second window. In another embodiment, only horizontal scrolling is possible on both of the first window and the second window, and vertical scrolling is not possible on either window. In this case, the controller 11 allocates a vertical gesture to an operation to select which of the first window and the second window to make active and allocates a horizontal gesture to a horizontal scrolling operation on whichever of the first window and the second window is the active window.
In the above embodiment, the proximity sensor 16 is disposed at the lower portion of the electronic device 1, and the illumination sensor 17 is disposed at the upper portion of the electronic device 1. In another embodiment, the proximity sensor 16 and the illumination sensor 17 may be disposed at the lower portion of the electronic device 1.
Much of the subject matter of the present disclosure is described as a series of operations executed by a computer system and other hardware that can execute program instructions. Examples of the computer system and other hardware include a general-purpose computer, a Personal Computer (PC), a dedicated computer, a workstation, a Personal Communications System (PCS), a mobile (cellular) phone, a mobile phone with a data processing function, a Radio Frequency IDentification (RFID) receiver, a game machine, an electronic notepad, a laptop computer, a GPS receiver, and other programmable data processing apparatuses. It should be noted that in each embodiment, various operations are executed by a dedicated circuit (for example, individual logical gates interconnected in order to execute a particular function) implemented by program instructions (software), or by a logical block, program module, or the like executed by one or more processors. The one or more processors that execute a logical block, program module, or the like include, for example, one or more of a microprocessor, CPU, Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), processor, controller, microcontroller, microprocessor, electronic device, other apparatus designed to be capable of executing the functions disclosed here, and/or a combination of any of the above. The embodiments disclosed here are, for example, implemented by hardware, software, firmware, middleware, microcode, or a combination of any of these. The instructions may be program code or a code segment for executing the necessary tasks. The instructions may be stored on a machine-readable, non-transitory storage medium or other medium. The code segment may indicate a combination of any of the following: procedures, functions, subprograms, programs, routines, subroutines, modules, software packages, classes, instructions, data structures, or program statements. The code segment may transmit and/or receive information, data arguments, variables, or memory content to or from another code segment or hardware circuit in order for the code segment to connect to another code segment or hardware circuit.
The storage used here may also be configured by a computer-readable, tangible carrier (medium) in the categories of solid-state memory, magnetic disks, and optical discs. Data structures and an appropriate set of computer instructions, such as program modules, for causing a processor to execute the techniques disclosed herein are stored on these media. Examples of computer-readable media include an electrical connection with one or more wires, a magnetic disk storage medium, a magnetic cassette, a magnetic tape, or other magnetic or optical storage medium, such as a Compact Disc (CD), Laser Disc®, Digital Versatile Disc (DVD®), Floppy® disk, and Blu-ray Disc® (laser disc, DVD, floppy, and Blu-ray Disc are registered trademarks in Japan, other countries, or both). Further examples include a portable computer disk, Random Access Memory (RAM), Read-Only Memory (ROM), rewritable programmable ROM such as Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), or flash memory, another tangible storage medium that can store information, or a combination of any of these. The memory may be provided internal and/or external to a processor or processing unit. As used in this disclosure, the term “memory” refers to all types of long-term storage, short-term storage, volatile, non-volatile, or other memory. No limitation is placed on the particular type or number of memories, or on the type of medium for memory storage.
Number | Date | Country | Kind |
---|---|---|---|
2016-147890 | Jul 2016 | JP | national |
Number | Name | Date | Kind |
---|---|---|---|
9501150 | Rodgers | Nov 2016 | B2 |
9684378 | Lazaridis | Jun 2017 | B2 |
20110310005 | Chen et al. | Dec 2011 | A1 |
20120254790 | Colombino | Oct 2012 | A1 |
20130097556 | Louch | Apr 2013 | A1 |
20140164990 | Kim | Jun 2014 | A1 |
20140282273 | Anderson | Sep 2014 | A1 |
20150116206 | Irie | Apr 2015 | A1 |
20150346831 | Nii | Dec 2015 | A1 |
20160117513 | Shariq | Apr 2016 | A1 |
20160139804 | Hsu et al. | May 2016 | A1 |
20170131894 | de Moraes | May 2017 | A1 |
Number | Date | Country |
---|---|---|
2011192081 | Sep 2011 | JP |
2013534009 | Aug 2013 | JP |
2015087824 | May 2015 | JP |
2015127976 | Jul 2015 | JP |
2015225493 | Dec 2015 | JP |
2016512632 | Apr 2016 | JP |
Entry |
---|
An Office Action issued by the Japanese Patent Office dated Oct. 3, 2017, of the prior Japanese Patent Application No. 2016-147890 and is related to U.S. Appl. No. 15/654,872; with English Concise Explanation. |
An Office Action issued by the Japanese Patent Office dated Oct. 18, 2016, of the prior Japanese Patent Application No. 2016-147890, with English language Concise Explanation. |
An Office Action issued by the Japanese Patent Office dated Nov. 29, 2016, of the prior Japanese Patent Application No. 2016-147890, with English language Concise Explanation. |
An Office Action issued by the Japanese Patent Office dated Mar. 21, 2017, of the corresponding Japanese Patent Application No. 2017-027305, with English language Concise Explanation. |
Number | Date | Country | |
---|---|---|---|
20180034950 A1 | Feb 2018 | US |