Patent Application
20150346814
Many users perform tasks using computing devices. In an example, a user may map directions from a current location to an amusement park using a mobile device. In another example, a user may read a book using a tablet device. Various types of input may be used to perform tasks, such as touch gestures, mouse input, keyboard input, voice commands, motion control, etc. An object detection component may, for example, be configured to visually detect body movement of a user as input for performing tasks and/or commands.
This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key factors or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
Among other things, one or more systems and/or techniques for gaze tracking are provided herein. A gaze tracking component may be associated with a user tracking component and an eye capture camera configuration. A user may take affirmative action to provide opt-in consent to allow the gaze tracking component to perform gaze tracking of the user and/or other users. The user tracking component may comprise a depth camera, a passive sensor, an active sensor, an infrared device, a time of flight device, a camera, or any other type of tracking device. The eye capture camera configuration may comprise a plurality of eye capture cameras (e.g., relatively high resolution cameras comprising narrow field of view lenses). In an example, the eye capture cameras are configured according to a fixed view frustum configuration, as opposed to a pan/tilt or other movement configuration that may otherwise reduce durability and/or increase cost of the eye capture camera configuration due to, among other things, moving parts and/or associated controls. In an example, the gaze tracking component maintains the eye capture cameras in a powered down state (e.g., a low power state or an off state) when not in active use for gaze tracking (e.g., an eye capture camera may be powered on while a user is detectable by the eye capture camera, and may be turned off when the user is not detectable by the eye capture camera such as due to the user moving away from the eye capture camera), which may reduce power consumption and/or bandwidth consumption.
In an example of gaze tracking, the gaze tracking component may utilize the user tracking component to obtain user tracking data for a user. The gaze tracking component may evaluate the user tracking data to identify a spatial location of the user. An eye capture camera may be selected from the eye capture camera configuration based upon the eye capture camera having a view frustum corresponding to the spatial location. The eye capture camera may be invoked to obtain eye region imagery of the user. In an example, eye capture cameras having view frustums that do not correspond to the spatial location may be powered down or maintained in the powered down state. The gaze tracking component may generate gaze tracking information for the user based upon the eye region imagery. Various tasks may be performed based upon the gaze tracking information (e.g., a videogame command may be performed, interaction with a user interface may be facilitated, a file may be opened, an application may be executed, a song may be played, a movie may be played, and/or a wide variety of other computing commands may be performed). In an example, the gaze tracking component may be configured to concurrently track gaze tracking information for multiple users that are detected by the user tracking component.
To the accomplishment of the foregoing and related ends, the following description and annexed drawings set forth certain illustrative aspects and implementations. These are indicative of but a few of the various ways in which one or more aspects may be employed. Other aspects, advantages, and novel features of the disclosure will become apparent from the following detailed description when considered in conjunction with the annexed drawings.
The claimed subject matter is now described with reference to the drawings, wherein like reference numerals are generally used to refer to like elements throughout. In the following description, for purposes of explanation, numerous specific details are set forth to provide an understanding of the claimed subject matter. It may be evident, however, that the claimed subject matter may be practiced without these specific details. In other instances, structures and devices are illustrated in block diagram form in order to facilitate describing the claimed subject matter.
One or more techniques and/or systems for gaze tracking are provided herein. A user tracking component may be utilized to obtain user tracking data for one or more users (e.g., users providing opt-in consent for gaze tracking). When a user is identified as being in a spatial location corresponding to a view frustum of an eye capture camera, then the eye capture camera may be invoked to obtain eye region imagery of the user. The eye region imagery may be used to generate gaze tracking information, which may be used to perform various tasks such as opening files, executing applications, controlling videogames, and/or interacting with user interfaces. Eye capture cameras may be maintained in a powered down state (e.g., turned off) when not actively tracking a user, which may reduce power and/or bandwidth consumption. In an example, gaze tracking information may be concurrently generated and/or tracked for multiple users (e.g., a first user may control a first avatar of a videogame using eye commands, and a second user may concurrently control a second avatar of the videogame using eye commands).
An embodiment of gaze tracking is illustrated by an exemplary method 100 of
Eye capture cameras may be selectively utilized to concurrently track gaze tracking information of one or more users. In an example, eye capture cameras may be maintained in a powered down state when not being utilized for gaze tracking (e.g., an eye capture camera may be powered on for gaze tracking when a user is within a view frustum of the eye capture camera, and the eye capture camera may be powered down when the user leaves the view frustum), which may mitigate power and/or bandwidth consumption.
At 104, the user tracking component may be utilized to obtain first user tracking data for a first user at a first time T1 (e.g., the user may sit on a couch in a living room within which the user tracking component is located). At 106, the first user tracking data may be evaluated to identify a first spatial location of the first user at the first time T1 (e.g., a spatial location of the couch within the living room). At 108, a first eye capture camera within the eye capture camera configuration may be selected based upon the first eye capture camera having a first view frustum corresponding to the first spatial location (e.g., the first eye capture camera may be positioned towards the couch where the user is sitting, and thus may be capable of capturing imagery of the first user's eyes). At 110, the first eye capture camera may be invoked to obtain first eye region imagery of the first user at or around the first time T1 (e.g., about 150 pixels or more across at least one eye of the first user). In an example, if the first eye capture camera was in a powered down state, then the first eye capture camera may be powered on and instructed to capture imagery that may depict the first user's eyes.
At 112, first gaze tracking information for the first user (e.g., corresponding to the first time T1) may be generated based upon the first eye region imagery (e.g., the first eye region imagery may comprise a plurality of images indicating pupil/eye movement by the first user). A task may be performed based upon the first gaze tracking information. For example, the first gaze tracking information may indicate that the user looked left, which may be mapped to a command that may be executed to perform a task (e.g., a look left gaze input may be mapped to a steer car left input for a driving videogame; the look left gaze input may be mapped to a play previous song input for a music player app; the look left gaze input may be mapped to a backspace input for a typing interface; etc.). It may be appreciated that a wide variety of tasks may be performed (e.g., controlling a videogame based upon analog and/or digital commands derived from gaze tracking information), and that merely a few examples are provided.
In an example of gaze tracking where the user is located within overlapping view frustums, a second eye capture camera may be selected from the eye capture camera configuration based upon the second eye capture camera having a second view frustum corresponding to the first spatial location (e.g., the user may be sitting on a portion of the couch that corresponds to an overlap between the first view frustum of the first eye capture camera and the second view frustum of the second eye capture camera). The second eye capture camera may be invoked to obtain second eye region imagery of the first user at or around the first time T1. The first eye region imagery and the second eye region imagery may be combined (e.g., using image stitching functionality; measurement combination functionality, and/or any other technique(s)) to generate the first gaze tracking information.
Gaze tracking may be performed for the first user as the first user moves around the living room, such as within detectable range of the user tracking component. In an example, the user tracking component may obtain first user tracking data indicating that the first user is at second spatial location at a second time T2 (e.g., the first user may have walked from the couch to a table in the living room). A third eye capture camera may be selected from the eye capture camera configuration based upon the third eye capture camera having a third view frustum corresponding to the second spatial location (e.g., the user may walk into a third view frustum 220 associated a third camera 206 illustrated in
In an example, spatial location data of the first user may be evaluated to predict a potential new spatial location for the first user. For example, previous spatial location data may indicate that the first user is within the first view frustum but is walking towards the second view frustum (e.g., and thus will presumably enter the second view frustum within a particular time/duration). Accordingly, the second eye capture camera may be awakened into a capture ready state for obtaining eye region imagery (e.g., slightly) prior to when the first user is expected/predicted to enter the second view frustum, based upon the spatial location data. In this way, lag associated with obtaining gaze tracking information between multiple eye capture cameras may be mitigated.
Gaze tracking information may be concurrently tracked for multiple users. In an example, the user tracking component may be utilized to obtain second user tracking data for a second user at the first time T1. The second user tracking data may be evaluated to identify a spatial location of the second user at the first time T1. An eye capture camera may be selected from the eye capture camera configuration based upon the eye capture camera having a view frustum corresponding to the spatial location of the second user at the first time T1. The eye capture camera may be invoked to obtain eye region imagery of the second user at or around the first time T1. Gaze tracking information may be generated for the second user based upon the eye region imagery of the second user at the first time T1. In this way, gaze tracking may be concurrently performed for multiple users, which may allow multiple users to perform tasks (e.g., the first user may control a first avatar of a videogame, and the second user may control a second avatar of the videogame). At 114, the method ends.
Still another embodiment involves a computer-readable medium comprising processor-executable instructions configured to implement one or more of the techniques presented herein. An example embodiment of a computer-readable medium or a computer-readable device is illustrated in
Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing at least some of the claims.
As used in this application, the terms “component,” “module,” “system”, “interface”, and/or the like are generally intended to refer to a computer-related entity, either hardware, a combination of hardware and software, software, or software in execution. For example, a component may be, but is not limited to being, a process running on a processor, a processor, an object, an executable, a thread of execution, a program, and/or a computer. By way of illustration, both an application running on a controller and the controller can be a component. One or more components may reside within a process and/or thread of execution and a component may be localized on one computer and/or distributed between two or more computers.
Furthermore, the claimed subject matter may be implemented as a method, apparatus, or article of manufacture using standard programming and/or engineering techniques to produce software, firmware, hardware, or any combination thereof to control a computer to implement the disclosed subject matter. The term “article of manufacture” as used herein is intended to encompass a computer program accessible from any computer-readable device, carrier, or media. Of course, many modifications may be made to this configuration without departing from the scope or spirit of the claimed subject matter.
Although not required, embodiments are described in the general context of “computer readable instructions” being executed by one or more computing devices. Computer readable instructions may be distributed via computer readable media (discussed below). Computer readable instructions may be implemented as program modules, such as functions, objects, Application Programming Interfaces (APIs), data structures, and the like, that perform particular tasks or implement particular abstract data types. Typically, the functionality of the computer readable instructions may be combined or distributed as desired in various environments.
In other embodiments, device 512 may include additional features and/or functionality. For example, device 512 may also include additional storage (e.g., removable and/or non-removable) including, but not limited to, magnetic storage, optical storage, and the like. Such additional storage is illustrated in
The term “computer readable media” as used herein includes computer storage media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions or other data. Memory 518 and storage 520 are examples of computer storage media. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, Digital Versatile Disks (DVDs) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by device 512. Computer storage media does not, however, include propagated signals. Rather, computer storage media excludes propagated signals. Any such computer storage media may be part of device 512.
Device 512 may also include communication connection(s) 526 that allows device 512 to communicate with other devices. Communication connection(s) 526 may include, but is not limited to, a modem, a Network Interface Card (NIC), an integrated network interface, a radio frequency transmitter/receiver, an infrared port, a USB connection, or other interfaces for connecting computing device 512 to other computing devices. Communication connection(s) 526 may include a wired connection or a wireless connection. Communication connection(s) 526 may transmit and/or receive communication media.
The term “computer readable media” may include communication media. Communication media typically embodies computer readable instructions or other data in a “modulated data signal” such as a carrier wave or other transport mechanism and includes any information delivery media. The term “modulated data signal” may include a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal.
Device 512 may include input device(s) 524 such as keyboard, mouse, pen, voice input device, touch input device, infrared cameras, video input devices, and/or any other input device. Output device(s) 522 such as one or more displays, speakers, printers, and/or any other output device may also be included in device 512. Input device(s) 524 and output device(s) 522 may be connected to device 512 via a wired connection, wireless connection, or any combination thereof. In one embodiment, an input device or an output device from another computing device may be used as input device(s) 524 or output device(s) 522 for computing device 512.
Components of computing device 512 may be connected by various interconnects, such as a bus. Such interconnects may include a Peripheral Component Interconnect (PCI), such as PCI Express, a Universal Serial Bus (USB), firewire (IEEE 1394), an optical bus structure, and the like. In another embodiment, components of computing device 512 may be interconnected by a network. For example, memory 518 may be comprised of multiple physical memory units located in different physical locations interconnected by a network.
Those skilled in the art will realize that storage devices utilized to store computer readable instructions may be distributed across a network. For example, a computing device 530 accessible via a network 528 may store computer readable instructions to implement one or more embodiments provided herein. Computing device 512 may access computing device 530 and download a part or all of the computer readable instructions for execution. Alternatively, computing device 512 may download pieces of the computer readable instructions, as needed, or some instructions may be executed at computing device 512 and some at computing device 530.
Various operations of embodiments are provided herein. In one embodiment, one or more of the operations described may constitute computer readable instructions stored on one or more computer readable media, which if executed by a computing device, will cause the computing device to perform the operations described. The order in which some or all of the operations are described should not be construed as to imply that these operations are necessarily order dependent. Alternative ordering will be appreciated by one skilled in the art having the benefit of this description. Further, it will be understood that not all operations are necessarily present in each embodiment provided herein. Also, it will be understood that not all operations are necessary in some embodiments.
Further, unless specified otherwise, “first,” “second,” and/or the like are not intended to imply a temporal aspect, a spatial aspect, an ordering, etc. Rather, such terms are merely used as identifiers, names, etc. for features, elements, items, etc. For example, a first object and a second object generally correspond to object A and object B or two different or two identical objects or the same object.
Moreover, “exemplary” is used herein to mean serving as an example, instance, illustration, etc., and not necessarily as advantageous. As used herein, “or” is intended to mean an inclusive “or” rather than an exclusive “or”. In addition, “a” and “an” as used in this application are generally be construed to mean “one or more” unless specified otherwise or clear from context to be directed to a singular form. Also, at least one of A and B and/or the like generally means A or B and/or both A and B. Furthermore, to the extent that “includes”, “having”, “has”, “with”, and/or variants thereof are used in either the detailed description or the claims, such terms are intended to be inclusive in a manner similar to the term “comprising”.
Also, although the disclosure has been shown and described with respect to one or more implementations, equivalent alterations and modifications will occur to others skilled in the art based upon a reading and understanding of this specification and the annexed drawings. The disclosure includes all such modifications and alterations and is limited only by the scope of the following claims. In particular regard to the various functions performed by the above described components (e.g., elements, resources, etc.), the terms used to describe such components are intended to correspond, unless otherwise indicated, to any component which performs the specified function of the described component (e.g., that is functionally equivalent), even though not structurally equivalent to the disclosed structure. In addition, while a particular feature of the disclosure may have been disclosed with respect to only one of several implementations, such feature may be combined with one or more other features of the other implementations as may be desired and advantageous for any given or particular application.
