Patent Application
20220374096
Certain computing devices, such as smart watches, have displays with smaller areas than other devices (e.g. smart phones, laptop computers, and the like). The displays of devices such as smart watches may therefore be poorly suited to presenting keyboards or other input interfaces with which an operator of the device can interact, e.g. via touch. Other input mechanisms, such as scripting or voice input, may be less accurate than touch-based keypad input, and/or may be unsuitable for use in certain operating environments.
In an embodiment, the present invention is a wearable computing device, comprising: a display; a motion sensor; a controller configured to: define, in a local coordinate system, a pose of a simulated input object having a plurality of selectable input elements; using the motion sensor, determine a sequence of current poses of the display in the local coordinate system, and for each current pose in the sequence: (i) based on the current pose and the pose of the simulated input object, select a portion of the simulated input object, including a subset of the selectable input elements, and (ii) render the portion of the simulated input object on the display; for at least one of the sequence of current poses, detect a simulated key press associated with one of the subset of selectable input elements, and generate input data corresponding to the one of the subset of selectable input elements; a housing containing the display, the motion sensor, and the controller; and a mounting component, coupled to the housing and configured to removably affix the housing to a forearm of an operator.
In another embodiment, the present invention is a method in a wearable computing device, the method comprising: defining, in a local coordinate system, a pose of a simulated input object having a plurality of selectable input elements; using a motion sensor of the wearable computing device, determining a sequence of current poses of a display of the wearable computing device in the local coordinate system; for each current pose in the sequence: (i) based on the current pose and the pose of the simulated input object, selecting a portion of the simulated input object, including a subset of the selectable input elements, and (ii) rendering the portion of the simulated input object on the display; for at least one of the sequence of current poses, detecting a simulated key press associated with one of the subset of selectable input elements; and generating input data corresponding to the one of the subset of selectable input elements.
The accompanying figures, where like reference numerals refer to identical or functionally similar elements throughout the separate views, together with the detailed description below, are incorporated in and form part of the specification, and serve to further illustrate embodiments of concepts that include the claimed invention, and explain various principles and advantages of those embodiments.
Skilled artisans will appreciate that elements in the figures are illustrated for simplicity and clarity and have not necessarily been drawn to scale. For example, the dimensions of some of the elements in the figures may be exaggerated relative to other elements to help to improve understanding of embodiments of the present invention.
The apparatus and method components have been represented where appropriate by conventional symbols in the drawings, showing only those specific details that are pertinent to understanding the embodiments of the present invention so as not to obscure the disclosure with details that will be readily apparent to those of ordinary skill in the art having the benefit of the description herein.
The device 100 includes a housing 104 containing or otherwise supporting various other components of the device 100, described in greater detail below. The device 100 also includes a mounting component 108, such as a band (e.g. a wristband) configured to removably affix the housing 104 to a forearm of an operator.
The components contained or supported within or by the housing 104 include a display 112, which is supported by the housing so as to be connected to internal components of the device 100, as well as accessible from the exterior of the device 100. The display 112 has a smaller form factor than various other forms of mobile computing devices, such as tablet computers and laptop computers. For example, while a smart phone may have a display with a diagonal dimension of about five inches or greater, the display 112 has a diagonal dimension 116 of about two inches or less. In some examples, such as those in which the device 100 is implemented as a ring worn on one or more fingers of an operator, the diagonal 116 may be smaller than about one inch. The display 112 may therefore be poorly suited to displaying a keypad or keyboard for receiving operator input. In particular, rendering a keyboard (e.g. including sixty or more keys) on the display 112 results in each key being significantly smaller than the fingertip of the operator, with the result that visibility of the keys may be reduced, and increasing the likelihood of mistakenly selected keys.
The device 100 therefore implements functionality to enable keypad-based input on small displays such as the display 112, by presenting a subset of keys or other selectable elements on the display 112 rather than an entire keypad, keyboard or the like. For example, the subset of keys may be presented on the display 112 without presenting the remaining keys, thus allowing those presented to be rendered with a size that facilitates visibility and selection. The device 100 further implements functionality to update which subset of keys is visible over time, enabling access to a full keypad, keyboard, or the like, without showing the full keypad at one time.
To that end, certain internal components of the device 100 are also shown in
The memory 124 stores computer readable instructions for execution by the processor 120. In particular, the memory 124 stores an input application 128 (also referred to simply as the application 128) which, when executed by the processor 120, configures the processor 120 to perform various functions discussed below in greater detail and related to the above-mentioned enablement of keypad-based input on the device 100. The application 128 may also be implemented as a suite of distinct applications in other examples. Those skilled in the art will appreciate that the functionality implemented by the processor 120 via the execution of the application 128 may also be implemented by one or more specially designed hardware and firmware components, such as field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs) and the like in other embodiments.
The device 100 further includes a motion sensor 132, such as an inertial measurement unit (IMU) including one or more accelerometers and/or one or more gyroscopes. The motion sensor 132 is configured provide sensor data to the processor 120, from which the processor 120 can detect movement of the housing 104. More specifically, from the sensor data generated by the motion sensor 132, the processor 120 can track a pose of the device 100, including a location of the device 100 in three dimensions, and an orientation of the device 100 in three dimensions.
The device 100 can further include an input device 136 configured to receive input from an operator of the device 100. The input device 136 can include a touch screen integrated with the display 112, for example. In other examples, the input device 136 can include, in addition to or instead of the touch screen, one or more physical buttons, e.g. disposed on a perimeter of the housing 104.
As will be described below, the device 100 is configured to simulate an input object, such as a keypad, keyboard, or the like. Based on the current pose of the device 100 as measured via the motion sensor 132, the device 100 is configured to render only a portion of the input object on the display 112, increasing visibility and selectability of that portion. The device 100 is further configured to detect selections associated with the simulated input object via the input device 136, and to generate input data in response to such selections (e.g. for use by another application in the memory 124, or the like).
Turning to
At block 205, the processor 120 is configured to receive an input mode command. The input mode command is an instruction to enter an input mode, in which the device 100 enables the operator to provide input data to the processor 120. The input data can be used for a variety of purposes. For example, the input data can simply be stored in the memory 124, or can be used by an application other than the application 128. For example, the processor 120 can also execute a calendar application, which implements functionality enabling the operator to create and store a calendar event. Execution of the calendar application can cause the processor 120 to present an event creation interface on the display 112, including a field into which the operator can enter a title for the event. Selection of that field can constitute an input mode command. For example, in response to detecting a selection of the field on the display 112 (e.g. via the touch screen), the calendar application can activate an input mode (i.e. generate an input mode command). The processor 120 can execute the application 128 simultaneously with the calendar application, in order to detect and act on the input mode command. As will be apparent, a wide variety of other scenarios can also lead to the generation of an input mode command.
At block 210, in response to the input mode command, the processor 120 is configured to initiate pose tracking of the device 100, and to define a pose of a simulated input object with a plurality of selectable input elements.
To initiate pose tracking of the device 100, the processor 120 is configured to generate a local coordinate system. The local coordinate system is referred to as “local” because it need not have any relation to a global coordinate system or facility-wide coordinate system. The location and orientation of the local coordinate system relative to the exterior of the device 100, in other words, need not be considered. In the present example, the local coordinate system is initialized at block 210 such that the origin of the local coordinate system is at a center of the display 112, and such that a plane of the local coordinate system is parallel to the display 112.
Turning to
As noted above, the processor 120 is also configured to define a pose of the simulated input object at block 310. The memory 124 contains a definition of the simulated input object, e.g. as a portion of the instructions defining the application 128. The simulated input object includes, for example, a virtual keypad, keyboard, control panel, or the like, and includes a plurality of selectable input elements each occupying distinct spatial portions of the input object. The selectable elements are, in other words, virtual keys, buttons, or the like. The simulated input object definition stored in the memory 124 can therefore define the shape and/or dimensions of a perimeter of the input object, as well as the shape and/or dimensions of each selectable element, and the positions of the selectable input elements relative to a coordinate system specific to the input object (i.e. independent of the local coordinate system 300).
Turning to
The definition 400 includes a perimeter or base 404, enclosing a plurality of simulated keys 408 (i.e. selectable input elements). Each key 408 includes an indicium indicating the input data generated via selection of the key 408. The keys 408 in this example include numerical values, as well as punctuation marks, a backspace key (the indicium for which is a left-facing arrow) and an enter or accept key (“OK”). A wide variety of other indicia are also contemplated. Other simulated input objects can include greater or smaller numbers of keys 408, in grid layouts as shown in
Defining a pose of the simulated input object includes determining a transformation between the local coordinate system 300 and the coordinate system 412 of the definition 400 itself. In some examples, the device 100 can store a predetermined indication of a portion of the definition 400 to appear centered on the display 112 upon initialization of the input mode. For example, the predetermined indication can set the “8” key of the definition to be centered. As a result, as shown in
Returning to
At block 220, the processor 120 is configured to select a portion of the simulated input object based on the current pose of the display 112 (i.e. the most recent pose in the sequence mentioned above), and the previously defined pose of the simulated input object. In other words, while the simulated input object is placed at a fixed pose in the coordinate system 300, the display 112 can be moved, e.g. as a result of the operator of the device 100 moving their arm, hand, or the like. The position of the display 112 relative to the simulated input object therefore changes over time.
To select a portion of the simulated input object, the processor 120 determines a current boundary of the display 112 in the coordinate system 300, using the tracked pose of the display and predetermined dimensions of the display 112. The boundary of the display is then overlaid on the simulated input object in the coordinate system 300. Turning to
At block 225, the processor 120 is configured to determine whether a key press (i.e. a selection of one of the selectable input elements presented on the display 112) has been detected. The selection at block 225 can be detected, for example, using the touch screen mentioned above. For instance, if a tap is detected on the display 112/touch screen over the area containing the “8” key, a selection of that key 408 is detected. In other examples, the motion sensor 132 can be employed by the processor 120 to detect key presses. For example, motion of the device 100 substantially perpendicular to the plane of the display 112 (i.e. in the Z axis of the coordinate system 300 as illustrated) can be interpreted as a key press. For example, a downward motion of the device, followed by a substantially equal upward motion, can be interpreted as a key press. Such an implementation facilitates single-handed selection of keys, whereby the operator of the device can, with the same forearm and/or hand that the device is mounted to, both control the visible portion of the input object and select keys from the input object. Various other gestures can also be used as key presses, as will be apparent to those skilled in the art.
When the determination at block 225 is negative, the processor 120 returns to block 215, and continues updating the pose of the display 112 and the corresponding portion of the simulated input object that is visible on the display 112 via blocks 215 and 220. When the determination at block 225 is affirmative, the processor 120 is configured to advance to block 230.
At block 230, the processor 120 is configured to generate input data corresponding to the selected key from block 225. In the example shown in
At block 235, the processor 120 is configured to determine whether the input mode is terminated. Termination can result, for example, from a detection of a particular selectable element, such as the “OK” key mentioned earlier. In some examples, termination can also result from activating a separate input of the device 100, such as a mechanical button on the housing 104, or the like. When the determination at block 235 is negative, the processor 120 returns to block 215 to repeat the above process. For example,
Following an affirmative determination at block 235, the processor 120 proceeds to block 240. At block 240, the input sequence is returned, e.g. to the application that generated the input mode command from block 205. The input sequence can then be discarded, and the coordinate system 300 may also be discarded.
As will be apparent from the discussion above, performance of the method 200 enables an input mode that can enhance visibility and selectability of selectable input elements, particularly for devices with small displays. The above functionality may therefore reduce the likelihood of errors in input data collected by the device 100.
Further variations to the above input mechanisms are contemplated. For example, referring to
The above description refers to a block diagram of the accompanying drawings. Alternative implementations of the example represented by the block diagram includes one or more additional or alternative elements, processes and/or devices. Additionally or alternatively, one or more of the example blocks of the diagram may be combined, divided, re-arranged or omitted. Components represented by the blocks of the diagram are implemented by hardware, software, firmware, and/or any combination of hardware, software and/or firmware. In some examples, at least one of the components represented by the blocks is implemented by a logic circuit. As used herein, the term “logic circuit” is expressly defined as a physical device including at least one hardware component configured (e.g., via operation in accordance with a predetermined configuration and/or via execution of stored machine-readable instructions) to control one or more machines and/or perform operations of one or more machines. Examples of a logic circuit include one or more processors, one or more coprocessors, one or more microprocessors, one or more controllers, one or more digital signal processors (DSPs), one or more application specific integrated circuits (ASICs), one or more field programmable gate arrays (FPGAs), one or more microcontroller units (MCUs), one or more hardware accelerators, one or more special-purpose computer chips, and one or more system-on-a-chip (SoC) devices. Some example logic circuits, such as ASICs or FPGAs, are specifically configured hardware for performing operations (e.g., one or more of the operations described herein and represented by the flowcharts of this disclosure, if such are present). Some example logic circuits are hardware that executes machine-readable instructions to perform operations (e.g., one or more of the operations described herein and represented by the flowcharts of this disclosure, if such are present). Some example logic circuits include a combination of specifically configured hardware and hardware that executes machine-readable instructions. The above description refers to various operations described herein and flowcharts that may be appended hereto to illustrate the flow of those operations. Any such flowcharts are representative of example methods disclosed herein. In some examples, the methods represented by the flowcharts implement the apparatus represented by the block diagrams. Alternative implementations of example methods disclosed herein may include additional or alternative operations. Further, operations of alternative implementations of the methods disclosed herein may combined, divided, re-arranged or omitted. In some examples, the operations described herein are implemented by machine-readable instructions (e.g., software and/or firmware) stored on a medium (e.g., a tangible machine-readable medium) for execution by one or more logic circuits (e.g., processor(s)). In some examples, the operations described herein are implemented by one or more configurations of one or more specifically designed logic circuits (e.g., ASIC(s)). In some examples the operations described herein are implemented by a combination of specifically designed logic circuit(s) and machine-readable instructions stored on a medium (e.g., a tangible machine-readable medium) for execution by logic circuit(s).
As used herein, each of the terms “tangible machine-readable medium,” “non-transitory machine-readable medium” and “machine-readable storage device” is expressly defined as a storage medium (e.g., a platter of a hard disk drive, a digital versatile disc, a compact disc, flash memory, read-only memory, random-access memory, etc.) on which machine-readable instructions (e.g., program code in the form of, for example, software and/or firmware) are stored for any suitable duration of time (e.g., permanently, for an extended period of time (e.g., while a program associated with the machine-readable instructions is executing), and/or a short period of time (e.g., while the machine-readable instructions are cached and/or during a buffering process)). Further, as used herein, each of the terms “tangible machine-readable medium,” “non-transitory machine-readable medium” and “machine-readable storage device” is expressly defined to exclude propagating signals. That is, as used in any claim of this patent, none of the terms “tangible machine-readable medium,” “non-transitory machine-readable medium,” and “machine-readable storage device” can be read to be implemented by a propagating signal.
In the foregoing specification, specific embodiments have been described. However, one of ordinary skill in the art appreciates that various modifications and changes can be made without departing from the scope of the invention as set forth in the claims below. Accordingly, the specification and figures are to be regarded in an illustrative rather than a restrictive sense, and all such modifications are intended to be included within the scope of present teachings. Additionally, the described embodiments/examples/implementations should not be interpreted as mutually exclusive, and should instead be understood as potentially combinable if such combinations are permissive in any way. In other words, any feature disclosed in any of the aforementioned embodiments/examples/implementations may be included in any of the other aforementioned embodiments/examples/implementations.
The benefits, advantages, solutions to problems, and any element(s) that may cause any benefit, advantage, or solution to occur or become more pronounced are not to be construed as a critical, required, or essential features or elements of any or all the claims. The claimed invention is defined solely by the appended claims including any amendments made during the pendency of this application and all equivalents of those claims as issued.
Moreover in this document, relational terms such as first and second, top and bottom, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. The terms “comprises,” “comprising,” “has”, “having,” “includes”, “including,” “contains”, “containing” or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises, has, includes, contains a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by “comprises . . . a”, “has . . . a”, “includes . . . a”, “contains ...a” does not, without more constraints, preclude the existence of additional identical elements in the process, method, article, or apparatus that comprises, has, includes, contains the element. The terms “a” and “an” are defined as one or more unless explicitly stated otherwise herein. The terms “substantially”, “essentially”, “approximately”, “about” or any other version thereof, are defined as being close to as understood by one of ordinary skill in the art, and in one non-limiting embodiment the term is defined to be within 10%, in another embodiment within 5%, in another embodiment within 1% and in another embodiment within 0.5%. The term “coupled” as used herein is defined as connected, although not necessarily directly and not necessarily mechanically. A device or structure that is “configured” in a certain way is configured in at least that way, but may also be configured in ways that are not listed.
The Abstract of the Disclosure is provided to allow the reader to quickly ascertain the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In addition, in the foregoing Detailed Description, it can be seen that various features are grouped together in various embodiments for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter may lie in less than all features of a single disclosed embodiment. Thus, the following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as a separately claimed subject matter.
