Apparatus and method for electronic labeling of electronic equipment

Abstract
An electronic device is coupled to electronic paper display. The electronic device may be utilized as part of an industrial control system. The central processing unit programmatically and automatically updates the electronic paper display with information to be displayed.
Description
BACKGROUND OF THE INVENTION

Field of the Invention


The subject matter disclosed herein generally relates to the identification of electronic equipment and the labeling of this equipment.


Brief Description of the Related Art


Today, electronically programmable devices having configurable parameters have paper labels that are used to display configuration or registration information to a user. The labels may identify various types of other information related to these devices as well.


The labels must either be printed by machine or by hand, and then attached to the product or device. This process is labor-intensive and is prone to errors or falling out of date if the device is updated. Alternately, an active display such as a liquid crystal display (LCD) display or a light emitting diode (LED) array can be provided to display this information. However, these methods require additional energy to keep the information visible and are not available if the product is powered off.


These shortcomings have resulted in some user dissatisfaction with previous approaches.


BRIEF DESCRIPTION OF THE INVENTION

Programmable labeling is provided that eliminates or substantially reduces the problems associated with paper labels or LCD/LED displays. In one aspect, a display, screen, or label is provided (utilizing electronic paper that employs e-ink to create images). In some examples, the screen, display, or label has an appearance similar to that of a paper label (with color or black and white printing). Through an electronic interface, the screen, display, or label is updated programmatically and/or automatically. It will be appreciated that the terms “screen,” “display,” “electronic paper display,” and “label” are used interchangeably herein. This screen could hold a static image or text without any required electricity to maintain the image. The screens provided herein have the benefit of being configurable, but not requiring additional power and being available even if the electronic module to which the screen is coupled is powered down.


In another advantage, programmable labeling for electronic devices could replace both static labels and LCD/LED displays for conveying configuration, registration, user interface, user interaction, or status information for programmable or pre-programmed electronic devices. Dynamic updates could be provided and as mentioned, once made, would require no power or maintenance to continue to display values. The present approaches are particularly useful and advantageous for devices where components of the device might be configured for multiple purposes that are not obvious by their physical interfaces.


One example of a device that might utilize programmable labels is an industrial automation input/output module that has one or more data channels that can be configured to receive discrete or analog inputs or send discrete or analog outputs of a variety of types. The labels for each channel could be updated to indicate the configured for the purpose for each channel along with its status and user-defined value.


Further applications of the present approaches include speeding mass-production of fixed-function input/output (IO) and/or communications modules. The present approaches could additionally be used to display configuration and identification (ID) settings for universal IO and/or communications modules, and facilitate and provide universal replacement spare parts. The present approaches provide customized maintenance or replacement instructions, even when the module is powered down or damaged. The present approaches also provide language localization advantages. For example, labels can be created on-the-fly using any desired language (e.g., English, Chinese, Japanese, German, or French to mention a few examples).


The interface between an electronic device and the programmable labeling could either be through a physical connection or through a wireless connection. In one aspect, programmable labeling would have to have an identification mechanism or unique ID to allow pairing to occur between the electronic device and the programmable label to avoid accidental programming by other neighboring or nearby electronic devices. That is, a label (having a unique ID) could only be re-programmed when a change request invoked the label's unique ID so that the label knew that it should be re-programmed with incoming information. Also, the label could detect a mis-pairing and notify the user that it was installed with the wrong electronic device, mitigating potential damage due to IO improperly connected to the wrong electronic device.


Additional connectivity could be avoided by incorporating an energy-harvesting technology to power the display. To avoid delays at power-up or during configuration cycles, the programmable label could maintain a check-sum or other unique present-configuration ID that could be compared before updating the programmable label. In the event that a programmable label update failed, the programmable label could be configured to either return to its last-good image and text display, or could revert to a default display that would indicate the failure and necessary steps to recover.


By allowing label data to be programmed, much faster mass-production of similar products with configurable capabilities is achieved by allowing a final programming or configuration operation to also label the product without human intervention and without inventory of labels or high-speed printing equipment.





BRIEF DESCRIPTION OF THE DRAWINGS

For a more complete understanding of the disclosure, reference should be made to e following detailed description and accompanying drawings wherein:



FIG. 1 comprises a block diagram of a system with an electronic paper device having an electronic display using e-ink according to various embodiments of the present invention;



FIG. 2 comprises a flow chart of programming an electronic paper display using e-ink according to various embodiments of the present invention;



FIG. 3 comprises a flow chart of programming an electronic paper display using e-ink according to various embodiments of the present invention;



FIG. 4 comprises a diagram of an electronic device with an electronic paper display using e-ink according to various embodiments of the present invention;



FIG. 5 comprises a diagram of an electronic device with an electronic paper display using e-ink according to various embodiments of the present invention;



FIG. 6 comprises a diagram of an electronic device with an electronic paper display using e-ink according to various embodiments of the present invention;



FIG. 7 comprises a diagram of an electronic paper display using e-ink according to various embodiments of the present invention.





Skilled artisans will appreciate that elements in the figures are illustrated for simplicity and clarity. It will further be appreciated that certain actions and/or steps may be described or depicted in a particular order of occurrence while those skilled in the art will understand that such specificity with respect to sequence is not actually required. It will also be understood that the terms and expressions used herein have the ordinary meaning as is accorded to such terms and expressions with respect to their corresponding respective areas of inquiry and study except where specific meanings have otherwise been set forth herein.


DETAILED DESCRIPTION OF THE INVENTION

The present approaches allow for seamless localization of products into any language based solely on configuration. Additionally, the present approaches allow universal IO and universal communications modules to have clear, accurate identification and interface labeling without any extra user intervention. Unlike LCD or active displays, the present approaches require no power to maintain the display. The display is visible when the electronic device is unpowered or even damaged. The present approaches advantageously allow universal replacement parts that assume the identities and display configurations of other universal modules or even fixed modules for which a universal module is a valid substitute. As mentioned, the present approaches also allow for user-defined labels and names to appear without any extra user intervention, but unlike LCD or active displays, require no power to maintain the display and is visible when the electronic device is unpowered or even damaged. The present approaches provide customized maintenance or replacement instructions, even when the module is powered down or damaged.


Advantageously, the present approaches provide up-to-date links to user documentation and/or product technical support on the product itself. The present approaches also greatly simplify equipment labeling or updating for system Integrators or end users. The present approaches additionally allow for common components to be produced at different price points with different configured capabilities and provide streamlined and efficient user interface and documentation of universal IO or universal communications modules. The present approaches allow for specific maintenance or replacement data to be displayed on the device without the device being powered up.


Referring now to FIG. 1, one example of a system 100 that uses electronic labeling is described. The system includes an electronic back plane 102. Coupled to the back plane 102 are a network module 104 (coupled to a network 106), a central processing unit (CPU) 108, a first electronic module 110 (coupled to a first electronic paper display 112) and a second electronic module 114 (coupled to a second electronic paper display 116).


The electronic back plane 102 is an electrical communication path between different system elements. The electronic back plane 102 provides an electrical connection between these elements.


The network module 104 provides connection functionality to one or more communication networks. For example, the network module 104 may be an Ethernet module or provide connections to the Internet.


The network 106 may be any network or any combination of networks. For example, it may be the Internet, Ethernet, a cellular network, a local area network or any combination of these networks.


The central processing unit (CPU) 108 provides supervisory and processing functions for the system. The CPU 108 may determine when and how to download information to the first electronic paper display 112 and the second electronic paper display 116.


The first electronic module 110 and the second electronic module 114 are electronic modules that provide various control system functionalities. They may include switches or implement various processing functions of any type. They may include a local process that receives information from the CPU 108. The modules 110 and 114 may be universal IO or communication modules.


The first electronic paper display 112 and the second electronic paper display 116 are coupled to the first electronic module 110 and the second electronic module 114. The first electronic paper display 112 and the second electronic paper display 116 are displays that use e-ink to render displays of information to users. For example, configuration and ID information for particular ones of the modules 110 and 114 may be displayed. The first electronic paper display 112 and the second electronic paper display 116 may be organized as matrices and receive commands that are de-coded to render the information to the user in e-ink.


In one example of the operation of the system of FIG. 1, the CPU 108 is programmed with information concerning labels. This information may be organized according to customers or equipment type or configuration, and will be presented on the first electronic paper display 112 and the second electronic paper display 116. The information is downloaded to the first electronic module 110 or the second electronic module 114. The information is transmitted from the first electronic module 110 to the first electronic paper display 112 and from the second electronic module 114 to the second electronic paper display 116. The information is rendered on the displays.


In one example, input/output (IO) modules typically contain 1-32 or more channels of IO that perform the same function with limited individual functional configuration (e.g., high-low ranges, set points, scaling, and so forth). Universal IO modules are capable of configuring individual IO channels to be any type of IO from an analog input, analog output, discrete input, or discrete output. Individual channels could also be configured to accept a range of connections such as 5 VDC discrete input or output, 24 VDC discrete input or output, 4-20 mA current analog input or output, RTD analog input, −10VDC-+10VDC analog output or input, and so forth. It will be appreciated that these are examples only and that other examples are possible.


The first electronic paper display 112 and the second electronic paper display 116 allow the customer to automatically and dynamically display accurately the exact configuration for each IO channel, along with its user-defined logical ID. For example, a wiring diagram for a universal IO module (e.g., one of the modules 110 or 114) would also be automatically updated at a label to reflect the user's specifications of each IO Channel. In another example, a two-dimensional (2D) matrix barcode at the label could be updated to point to appropriately tailored user documentation or program information. If maintenance required a change in the definition of either the function of a particular channel or its user-defined, logical ID, these changes would be reflected automatically with no additional user intervention.


Communications modules (e.g., one of the modules 110 or 114) sometimes have a variety of user-defined logical device names and be configured for a variety of protocols over their physical media. These universal communications modules could be programmed for any variety of protocol support, which would be reflected on appropriate displays of the first electronic paper display 112 and the second electronic paper display 116.


Another use of the universal IO module with the first electronic paper display 112 and the second electronic paper display 116 would be to provide a universal spare IO module. Users could purchase IO modules with fixed IO channel values of a variety of discrete and analog types, but then purchase a single type of universal IO as spare modules. When a fixed IO channel module failed, it could be replaced with a universal IO module of suitable configurability which would automatically assume the configuration of the fixed IO channel module it replaced and update its programmable labeling to even identify itself as the module it replaced and indicate it was configured and operating as the module it replaced. This approach would avoid confusion about replacement modules and drastically simplify maintenance. If the module was not a suitable substitute for the module the user attempted to replace, the first electronic paper display 112 and the second electronic paper display 116 could clearly and immediately indicate the incompatibility and either explicitly recommend a suitable substitute or update its display (e.g., via a 2D matrix barcode) to provide a link to documentation on suitable substitutes or even a contact number or a URL for customer support.


In the event of a failure of a fixed communications or universal communications module, the universal communications module assumes the identity and configuration of a failed fixed-communications module and updates its labels to identify itself as the failed module it replaces, and shows the configuration information of the fixed communications module it replaced, including additional information like supported protocols. Again, if the module was not a suitable substitute for the module the user attempted to replace, the first electronic paper display 112 and the second electronic paper display 116 could clearly and immediately indicate the incompatibility and either explicitly recommend a suitable substitute or update (e.g., via 2D matrix barcode) to provide a link to documentation on suitable substitutes or even a contact number or URL for customer support.


In both the cases of IO and Communications modules, the first electronic paper display 112 and the second electronic paper display 116 could be used to display user-maintenance steps or instructions in the event of a failure or an event requiring user intervention or maintenance.


Another use for the programmable labeling is language localization for modules located in different geographic regions. For example, the first electronic paper display 112 and the second electronic paper display 116 could be programmed to render its displays in Chinese if in China, or render its displays in English if the module were located in an English-speaking country or region.


As mentioned, the interface between the electronic modules 110 and 114 and the electronic paper displays 112 and 116 could either be through a physical connection or through a wireless connection. Possible physical connections could be an I2C or SPI interface. Possible wireless connections could be through a low-power, directed transmitter and receiver configurations using protocols like Bluetooth, Zigbee, Wi-Fi or other wireless communications protocols. Other examples are possible.


The first electronic paper display 112 and the second electronic paper display 116 may have an identification mechanism or unique ID to allow for pairing between electronic device and programmable label to avoid accidental programming by other neighboring or nearby electronic devices. That is, the first electronic paper display 112 and the second electronic paper display 116 (each having unique IDs) could only be re-programmed when a change request invoked the unique ID so that the label knew that it should be re-programmed with incoming information. This pairing mechanism could also be used to indicate on the electronic paper display information regarding incorrect mating of a terminal block (containing the electronic paper label) with an incorrect electronic device.


In the event that the first electronic paper display 112 and the second electronic paper display 116 were paired through a wireless medium, additional connectivity could be avoided by incorporating an energy-harvesting technology to power the display for the short intervals required for updates. Examples of energy-harvesting technologies could be thermocouple power harvesting, piezoelectric power harvesting, magnetic induction harvesting, or some other form of power harvesting technology. Also, capacitive coupling power harvesting could be used in another example.


To avoid delays at power-up or during configuration cycles, the first electronic paper display 112 and the second electronic paper display 116 could maintain a check-sum or other unique present-configuration ID that could be compared before updating the programmable label. When the module power up or an configuration update is applied, if it does not change the information conveyed by the first electronic paper display 112 and the second electronic paper display 116, then the update could be skipped as it would result in no difference in display at the first electronic paper display 112 and the second electronic paper display 116.


In the event that a programmable label update failed, the first electronic paper display 112 and the second electronic paper display 116 could be configured to either return to its last-good image and text display, or could revert to a default display that would indicate the failure and necessary steps to recover.


Referring now to FIG. 2, one example of transmitting information to the label is described. At step 202, label information is received at the CPU. At this step, the CPU may be programmed with certain label information for various electronic devices. For instance, certain customers may have certain information to be displayed and arranged in certain ways.


At step 204, this label information is sent to the electronic module. At step 206, the information is sent from the electronic module to the electronic paper display. This may be accomplished wirelessly or across a wired connection. At step 208, the electronic display renders or displays the information on the label.


Referring now to FIG. 3, another example of transmitting information to the label is described. At step 302, label information is received at the network module. The label information may originate from the Internet or some other network. Consequently, an outside user may program or enter the label information on-the-fly from any remote terminal at any location that is connected to a network.


At step 304, this label information is transmitted to the CPU. At step 306, this label information is sent to the electronic module. At step 308, it is sent to the electronic paper display. This may be accomplished wirelessly or across a wired connection. At step 310, the electronic display renders or displays the information on the label.



FIG. 4, FIG. 5, and FIG. 6 are examples of IO and communications modules with programmable labeling are utilized. It will be appreciated that these are examples only and that the specific function of the electronic device using programmable labeling is limitless. For example programmable logic controllers (PLCs), a distributed control system (DCS), computers or industrial computers, networking equipment, sensors or other interface devices, can utilize these approaches.



FIG. 4 represents the front cover of an IO module 400 with, in this example, 32 channels with module identification and status information, a channel configured function and channel number indicator, as well as a section for User-defined Channel Logical IDs. The module 400 includes a light emitting diode (LED) area 402 (with LEDs indicating health status indicators and channel status indicators) and an electronic paper display (or label) 404. The electronic paper display 404 may be a single screen or display area, or multiple screens or display areas. The label 404 includes a 2D matrix barcode 406, a module model number 408, a channel configured function and number 410, a user-defined channel logic ID 412, and a module color code function indicator 414.


All or some of these items might change and be an electronic paper display rendered in e-ink. For example, The 2D Matrix or International (or European) Article Number (EAN) barcode 406 is an electronic paper display that in one aspect provides a link (e.g., via smart phone) to the latest user manuals and detailed configuration settings for the IO module 400, which could be updated by the system controller connected to the Internet when module configuration or user manual updates are made. Also, user-defined information like user-defined channel logical ID 412 is an electronic paper display that is automatically populated or updated when the user-defined configuration for the module was updated programmatically.



FIG. 5 is a wiring diagram 502 that ships with the IO module to show the customer proper wiring for the IO module channels and is located on the inside of the front cover. This might be changed over time. The entire wiring diagram or portions of it may be an electronic paper display.



FIG. 6 is a communications module 600 with module identification and status information, as well as configurable network addressing information. The module 600 includes a module/port status area 602, a module model number 604, module installation information 606, user defined logical device name 608, a network logic address 610, a network physical address 612, a port configuration 614, and a 2D matrix/barcode 616.


Some of these fields may change and may be electronic paper displays. The 2D Matrix or EAN barcode 616 is an electronic paper display that provides a link (e.g., via smart phone or tablet) to the latest user manuals and detailed configuration settings for the communications module 600, which could be updated by the system controller connected to the Internet when module configuration or user manual updates are made. Also, the user-defined logical device name 608 is an electronic paper display that is automatically populated or updated when the user-defined configuration for the module was updated programmatically.


As shown in FIG. 4, FIG. 5, and FIG. 6, programmable labeling could allow for user-defined items to be updated upon configuration of the fixed-function module. In addition, module firmware revisions or other upgradeable revision information could be updated. Further, a base module could receive a fixed program during manufacturing that would turn it into a fixed-function IO or communications module that could be displayed on the programmable label. This eliminates the need for carrying multiple labels in a manufacturing setting thus speeding and allowing for more flexibility in the module manufacturing process.


Referring now to FIG. 7, one example of an electronic paper display 700 is described. The display 700 includes an e-ink display area 702, an interface to an electronic device (or module) 704, a rendering device 706, and a receiver 708. The receiver 708 receives label information. This information is passed to the rendering device 704, which translates this into a display on e-ink on the e-ink display area 702. The e-ink display area 702 may be a single screen/display area or divided into separate, smaller screens/display areas of any shape or size. In one example, the displays may be made in black-and-white, but color displays may also be made.


Alternatively, a wired connection to an electronic device (or module) may couple to the interface 704. This information is passed to the rendering device 704, which translates this into a display on e-ink on the e-ink display 702. Consequently, the electronic paper display 700 may receive information over a wire, wirelessly, or in both manners.


It will be appreciated by those skilled in the art that modifications to the foregoing embodiments may be made in various aspects. Other variations clearly would also work, and are within the scope and spirit of the invention. It is deemed that the spirit and scope of that invention encompasses such modifications and alterations to the embodiments herein as would be apparent to one of ordinary skill in the art and familiar with the teachings of the present application.

Claims
  • 1. A method, comprising: receiving, by an industrial automation input/output (I/O) module having a plurality of data channels each having a respective type of analog input (AI), analog output (AO), discrete input (DI), or discrete output (DO), a request that includes electronic label information for reprogramming a label paired to the industrial automation I/O module to identify one or more of the plurality of data channels of the industrial automation I/O module, wherein the label is identified by a first ID, wherein the request has a second ID, wherein the electronic label information includes configuration information representative of a configuration of the industrial automation I/O module, and wherein the label is configurable and capable of displaying an image or text without any power required to maintain the image or text;transmitting, by the industrial automation I/O module, the electronic label information to the label wirelessly or across a wired connection;when the second ID matches the first ID, reprogramming the label with the electronic label information, andwhen the first ID of the label does not match an ID of the electronic module, issuing, by the label, an alert for display on the label.
  • 2. The method of claim 1, wherein the step of receiving the electronic label information is made via at least one of a wired connection and a wireless connection.
  • 3. The method of claim 1, wherein the electronic label information further includes a wiring diagram of the industrial automation I/O module.
  • 4. The method of claim 1, wherein the label comprises a display that renders images using e-ink.
  • 5. The method of claim 4, wherein the rendered image comprises product technical support.
  • 6. The method of claim 4, wherein the display is configured to render images in color.
  • 7. An apparatus, comprising: an industrial automation input/output (I/O) module operating in an industrial control system, the industrial automation I/O module having a plurality of data channels each having a respective type selected from analog input (AI), analog output (AO), discrete input (DI), or discrete output (DO); anda label paired to the industrial automation I/O module to identify one or more of the plurality of data channels of the industrial automation I/O module, wherein the label is identified by a first ID;wherein the industrial automation I/O module is configured to (a) receive a request that includes electronic label information for reprogramming the label, the request having a second ID, (b) transmit the electronic label information to the label wirelessly or across a wired connection, and (c) cause the electronic label information to be displayed at the label when the second ID matches the first ID, the label being configurable and capable of (i) displaying an image or text without any power required to maintain the image or text and (ii) issuing an alert to a user when the first ID of the label does not match an ID of the industrial automation I/O module to which the label is paired,wherein the electronic label information includes configuration information representative of a configuration of the industrial automation I/O module.
  • 8. The apparatus of claim 7, wherein the industrial automation I/O module receives the electronic label information via a wired connection.
  • 9. The apparatus of claim 7 where the industrial automation I/O module receives the electronic label information via a wireless connection.
  • 10. The apparatus of claim 7, wherein the label comprises a display that renders images using e-ink.
  • 11. The method of claim 1, further comprising: receiving updated electronic label information;displaying the updated electronic label information at the label automatically without user intervention.
  • 12. A method comprising: providing in an industrial control system a universal input/output (I/O) module able to be configured to have any of the I/O types (i) analog input (AI), (ii) analog output (AO), (iii) discrete input (DI), (iv) discrete output (DO), wherein the universal I/O module includes a label identified by a first ID, and wherein the universal I/O module is an I/O module operating within the industrial control system;replacing a failed I/O module with the universal I/O module within the industrial control system, the failed I/O module configured one of (i) an analog input (AI), (ii) an analog output (AO), (iii) a discrete input (DI), or (iv) a discrete output (DO), the universal I/O module assuming a configuration of the failed I/O module;receiving, by the universal I/O module, electronic label information in a request, wherein the request has a second ID;transmitting, by the universal I/O module, the electronic label information to the label wirelessly or across a wired connection;automatically updating the label with the electronic label information to identify the universal I/O module when the second ID matches the first ID, the label being configurable and capable of (i) displaying an image or text without any power required to maintain the image or text and (ii) issuing an alert to a user when the first ID of the label does not match an ID of the universal I/O module to which the label is paired, wherein the electronic label information includes configuration information representative of the configuration of the failed module.
CROSS REFERENCE TO RELATED APPLICATIONS

This application claims the benefit under 35 U.S.C. § 119 (e) to U.S. Provisional Application No. 62/022,981 entitled APPARATUS AND METHOD FOR ELECTRONIC LABELING OF ELECTRONIC EQUIPMENT, filed Jul. 10, 2014, the content of which is incorporated herein by reference in its entirety.

PCT Information
Filing Document Filing Date Country Kind
PCT/US2014/069247 12/9/2014 WO
Publishing Document Publishing Date Country Kind
WO2016/007192 1/14/2016 WO A
US Referenced Citations (128)
Number Name Date Kind
4099247 Mikada et al. Jul 1978 A
5900877 Weiss et al. May 1999 A
7062716 Washington Jun 2006 B2
7593000 Chin Sep 2009 B1
8059101 Westerman et al. Nov 2011 B2
8125312 Orr Feb 2012 B2
8255867 Chaplin et al. Aug 2012 B1
8286102 Wilensky Oct 2012 B1
8405616 Jung et al. Mar 2013 B2
8445793 Westerman et al. May 2013 B2
8458485 Bandyopadhyay et al. Jun 2013 B2
8525799 Grivna et al. Sep 2013 B1
8536978 Coggill Sep 2013 B2
8619052 Benko et al. Dec 2013 B2
8638939 Casey et al. Jan 2014 B1
8686958 Rutledge et al. Apr 2014 B2
8823642 Valik et al. Sep 2014 B2
8824040 Buchheit et al. Sep 2014 B1
8830072 Batra et al. Sep 2014 B2
9001061 Locker et al. Apr 2015 B2
9030418 Ku et al. May 2015 B2
9165159 McDonnell Oct 2015 B1
9189614 DeLuca Nov 2015 B2
9262603 Dow et al. Feb 2016 B2
9357391 Alsvig et al. May 2016 B1
9460575 Park et al. Oct 2016 B2
9600103 Eischeid et al. Mar 2017 B1
9703392 Wakabayashi Jul 2017 B2
9983664 Kim et al. May 2018 B2
20020054120 Kawano et al. May 2002 A1
20020109677 Taylor Aug 2002 A1
20020140688 Steinberg et al. Oct 2002 A1
20020167500 Gelbman Nov 2002 A1
20040003036 Eagle et al. Jan 2004 A1
20040156170 Mager et al. Aug 2004 A1
20060284852 Hofmeister et al. Dec 2006 A1
20070024551 Gelbman Feb 2007 A1
20070177803 Elias et al. Aug 2007 A1
20070285385 Albert et al. Dec 2007 A1
20080136587 Orr Jun 2008 A1
20080303637 Gelbman et al. Dec 2008 A1
20090051648 Shamaie et al. Feb 2009 A1
20090089701 Baier et al. Apr 2009 A1
20090122018 Vymenets et al. May 2009 A1
20090135147 Hsu et al. May 2009 A1
20090195496 Koyama et al. Aug 2009 A1
20090225023 Szolyga et al. Sep 2009 A1
20090262379 Miyake et al. Oct 2009 A1
20090278807 Hu Nov 2009 A1
20090322700 D'Souza et al. Dec 2009 A1
20100020025 Lemort et al. Jan 2010 A1
20100031200 Chen Feb 2010 A1
20100031344 Zhao et al. Feb 2010 A1
20100073303 Wu et al. Mar 2010 A1
20100115473 Reeves et al. May 2010 A1
20100138764 Hatambeiki et al. Jun 2010 A1
20100162182 Oh et al. Jun 2010 A1
20100177660 Essinger Jul 2010 A1
20100194702 Chen Aug 2010 A1
20100245102 Yokoi Sep 2010 A1
20100245341 Tanaka Sep 2010 A1
20100322485 Riddiford Dec 2010 A1
20110041102 Kim Feb 2011 A1
20110069018 Atkins et al. Mar 2011 A1
20110078568 Park et al. Mar 2011 A1
20110118694 Yodfat May 2011 A1
20110156867 Carrizo et al. Jun 2011 A1
20110157375 Kusumoto Jun 2011 A1
20110175839 Prabhu Jul 2011 A1
20110242022 Wen Oct 2011 A1
20110260829 Lee Oct 2011 A1
20110273388 Joo et al. Nov 2011 A1
20110285645 Cho et al. Nov 2011 A1
20110320978 Horodezky et al. Dec 2011 A1
20120023574 Osborn et al. Jan 2012 A1
20120066650 Tirpak et al. Mar 2012 A1
20120184368 Yamaoka Jul 2012 A1
20120206474 Holland et al. Aug 2012 A1
20120256863 Zhang et al. Oct 2012 A1
20120291120 Griffin Nov 2012 A1
20120306793 Liu et al. Dec 2012 A1
20130033436 Brinda et al. Feb 2013 A1
20130057070 Onishi et al. Mar 2013 A1
20130104065 Stecher Apr 2013 A1
20130135178 Miyahara May 2013 A1
20130227496 Maekawa Aug 2013 A1
20130241844 Chang Sep 2013 A1
20130268900 Ferren et al. Oct 2013 A1
20130298071 Wine Nov 2013 A1
20140026055 Cohn et al. Jan 2014 A1
20140035853 Ok et al. Feb 2014 A1
20140092031 Schwartz et al. Apr 2014 A1
20140109018 Casey et al. Apr 2014 A1
20140123080 Gan May 2014 A1
20140143859 Linge et al. May 2014 A1
20140173529 Hicks Jun 2014 A1
20140189855 Moradi et al. Jul 2014 A1
20140223381 Huang et al. Aug 2014 A1
20140223549 Quintanilla Aug 2014 A1
20140245203 Lee et al. Aug 2014 A1
20140267015 Saatchi et al. Sep 2014 A1
20140277753 Eiynk et al. Sep 2014 A1
20140298237 Galu, Jr. Oct 2014 A1
20140372896 Raman Dec 2014 A1
20150007308 Mankowski Jan 2015 A1
20150029095 Gomez et al. Jan 2015 A1
20150038072 Cordier et al. Feb 2015 A1
20150046885 Zhang et al. Feb 2015 A1
20150067578 Ryu et al. Mar 2015 A1
20150072784 Lee Mar 2015 A1
20150121314 Bombolowsky Apr 2015 A1
20150135129 Kwon et al. May 2015 A1
20150138142 Liao et al. May 2015 A1
20150153932 Jiang et al. Jun 2015 A1
20150169141 Kim et al. Jun 2015 A1
20150169216 Cho Jun 2015 A1
20150169502 Koenig et al. Jun 2015 A1
20150188970 Kowshik et al. Jul 2015 A1
20150220182 Avrahami Aug 2015 A1
20150294096 Grigg et al. Oct 2015 A1
20150355805 Chandler et al. Dec 2015 A1
20150365492 Kalan et al. Dec 2015 A1
20160054851 Kim et al. Feb 2016 A1
20170039691 Sugioka Feb 2017 A1
20170230378 Bliss Aug 2017 A1
20180267690 Kemp et al. Sep 2018 A1
20190095075 Yang Mar 2019 A1
20190174069 Poindexter, Jr. Jun 2019 A1
Foreign Referenced Citations (8)
Number Date Country
101251884 Aug 2008 CN
102592524 Jul 2012 CN
203204640 Sep 2013 CN
2 416 308 Feb 2012 EP
2416308 Feb 2012 EP
2042955 Apr 2016 EP
9721204 Jun 1997 WO
WO-2015012789 Jan 2015 WO
Non-Patent Literature Citations (19)
Entry
European Search Report and Opinion issued in connection with related EP Application No. 16168865.0 dated Jul. 12, 2016.
U.S. Non-Final Office Action issued in connection with related U.S. Appl. No. 14/713,467 dated Oct. 4, 2016.
European Search Report and Opinion issued in connection with related EP Application No. 16168865.0 dated Oct. 17, 2016.
U.S. Final Office Action issued in connection with related U.S. Appl. No. 14/713,467 dated Apr. 19, 2017.
Machine Translation and First Office Action and Search issued in connection with corresponding CN Application No. 201480080514.0 dated Jun. 29, 2018.
Third Office Action for Chinese Application No. 201480080514.0, dated Apr. 12, 2019.
Copending U.S. Appl. No. 15/145,073, filed May 3, 2016, and the prosecution history thereof.
Copending U.S. Appl. No. 15/145,087, filed May 3, 2016, and the prosecution history thereof.
Copending U.S. Appl. No. 15/145,095, filed May 3, 2016, and the prosecution history thereof.
International Preliminary Report on Patentability for Application No. PCT/US2014/069247, dated Jan. 10, 2017.
International Search Report and Written Opinion for Application No. PCT/US2014/069247, dated Jun. 23, 2015.
Jiao, et al., “An Investigation of Two-Handled Manipulation and Related Techniques in Multi-touch Interaction”, Machine Vision and Human-Machine Interface (MVHI), 2010, 565-568.
Lee, et al., “Access to an Automated Security System Using Gesture-Based Passwords”, Network-Based Information Systems (NBiS), 2012 15 International Conference, 2012, 760-765.
Niu, Yuan et al., “Gesture Authentication with Touch Input for Mobile Devices,” Third International ICST Conference, MobiSec 2011, Aalborg, Denmark, May 17-19, 2011, pp. 13-24.
Office Action, European patent application No. 14824987.3, dated Jul. 9, 2019.
Sae-Bae, et al., “Multitouch Gesture-Based Authentication”, Information Forensics and Security, IEEE Transactions, 2014, 568-582.
Tsagaris, et al., “Methodology for finger gesture control of mechatronic systems”, Mechatronika, 2012, 1-6.
Wang, et al., “VirtualTouch: A finger glove to simulate touch screen commands”, Sensors, 2012 IEEE, 2012, 1-4.
Decision on Rejection for Chinese Application No. 201480080514.0, dated Sep. 30, 2019.
Related Publications (1)
Number Date Country
20170206047 A1 Jul 2017 US
Provisional Applications (1)
Number Date Country
62022981 Jul 2014 US