Patent Application
20090279736
The present disclosure is generally related to imaging systems, and, more particularly, is related to magnetic resonance imaging systems and methods.
Eye tracking data is commonly recorded using specialized equipment during cognitive studies or clinical tests. The most common eye tracking approach in a magnetic resonance imaging (MRI) environment is to use reflected infrared light from the cornea to track eye movement and determine fixation. Installation of such a system can pose a significant challenge since the optics and path of the transmitted and reflected infrared light usually must avoid interference with the visual paradigm display and are confined within the limited access to the subject's eye within the scanner. During an experiment, setup of the optics extends the time of the experiment. In addition, the quality of the infrared image from standard eye tracking data may not be sufficient for accurate determination of the position of the pupil. Another drawback is that magnetic resonance compatible eye-tracking systems are generally expensive.
Embodiments of the present disclosure provide magnetic resonance eye tracking systems and methods.
Briefly described, in architecture, one embodiment of the system, among others, comprises memory with software stored therein, and a processor configured with the software to receive first eye fixation coordinates and first image data corresponding to a calibration scan, generate one or more models based on the first eye fixation coordinates and the first image data, receive second image data corresponding to a non-calibration scan, and estimate a subject's eye fixation based on the second image data and the one or more models.
Embodiment of the present disclosure can also be viewed as magnetic resonance eye tracking methods. In this regard, one embodiment of such a method, among others, can be broadly summarized as receiving magnetic resonance based data and determining direction of a subject's gaze based on the data.
Other systems, methods, features, and advantages of the present disclosure will be or become apparent to one with skill in the art upon examination of the following drawings and detailed description. It is intended that all such additional systems, methods, features, and advantages be included within this description, be within the scope of the present disclosure, and be protected by the accompanying claims.
Many aspects of the disclosure can be better understood with reference to the following drawings. The components in the drawings are not necessarily to scale, emphasis instead being placed upon clearly illustrating the principles of the disclosed systems and methods. Moreover, in the drawings, like reference numerals designate corresponding parts throughout the several views.
Disclosed herein are various embodiments of magnetic resonance eye tracking systems and methods (herein, also referred to collectively as magnetic resonance eye tracking systems). At least one goal of such magnetic resonance eye tracking systems is to determine (e.g., estimate) a subject's direction of gaze from a subject's magnetic resonance (MR) signal such that the MR signal alone can lead to an estimate of the true direction of gaze. Thus, the magnetic resonance eye tracking systems disclosed herein utilize a nuclear magnetic resonance (NMR) signal or, broadly speaking, the MR signal itself, to determine a subject's gaze, and hence the movement of the eye. That is, magnetic resonance eye tracking systems as described herein include the use of NMR to determine physical properties of the eye such as direction of gaze, and amount of eye movement over a period of time.
Since the MR signal may be dependent on eye movement and/or position, certain embodiments of the magnetic resonance eye tracking systems mathematically/statistically establish this dependence through a calibration or training stage and exploit this dependence for eye tracking in the rest of the study. In other words, certain embodiments of magnetic resonance eye tracking systems are based on a mathematical/statistical relationship between the MR signal and the position of the eyes. Note that “study” is used herein to refer to a period of time in which a subject is inside or otherwise exposed to scan signals emanating from a scanner continuously (as opposed to interrupted, such as by undergoing a scan in the morning and returning in the evening for an additional scan). Additionally, within a study, the calibration may be implemented in the beginning, the end, or any time in between with no particular preference for any time slot as long as the subject's head position remains fixed. The magnetic resonance eye tracking systems have a nominal hardware investment, and are easy to use. Thus, the magnetic resonance eye tracking systems can potentially save thousands of dollars compared to many infrared-based systems, and save a significant amount of experimental set-up time.
Although described below in the context of a human subject and the gaze corresponding to a human subject's eyes, a “subject” as used herein can refer to any life form that comprises eyes or other movable, spatially directed sensory organs. Additionally, the MR signal can represent a reconstructed image volume, but in some embodiments, need not be limited as such. That is, in some cases, actual two-dimensional (2-D) or three-dimensional (3-D) images may not be required. For example, certain embodiments of the magnetic resonance eye tracking systems may detect changes in eye orientation (e.g. is the person fixating at the right location or not) using a few specially acquired MR signals.
Further, although described in the context of functional magnetic resonance imaging (fMRI), it should be appreciated by those having ordinary skill in the art in the context of this disclosure that other MR data and modeling approaches can be used in some embodiments. For instance, some embodiments may utilize other applications of MRI where simultaneous eye position/movement is desired.
Although described herein in conjunction with a visual display 106, a stimulus can embody any visual display or even any other sensory modulation that constitutes a natural or instructed relationship between eye fixation and that stimulus. In some embodiments, the subject 108 can generate the stimulus. The visual display 106 may be embodied as a visual, computer-generated display seen through goggles or projected onto a visual screen. Other stimuli that can be used in some embodiments include an auditory signal that can be spatially localized by the subject 108 (e.g. left/right emanating sounds), or instructions to move eyes based on tactile stimuli (e.g. “move eyes to the right when you feel a sensation (such as from a pulse of air) on your right hand”), among others.
By changing the location of a fixation symbol, images (e.g., image data) are transferred to the processing device 104a, the latter which comprises logic (e.g., learning module 360, explained below) to estimate a model or models that relates each image volume for a particular time to the fixation location (e.g., fixation coordinates for time t, or (h,v)t) at that time. The model may be a mathematical formula, or in some embodiments, may be a lookup table. In particular, the input to the processing device 104a is multivariate. The processing device 104a comprises mathematical tools that enable extraction of salient information (e.g. features) from this multivariate input data. The training or calibration stage establishes the most relevant features to extract and the relationship between the feature(s) and the eye positions. The model parameters fitted from the calibration data may be represented as a matrix, and in some embodiments, may be used to generate a lookup table. While calibration data is collected during a training session (e.g., approximately 1-2 minutes), such data may be collected before or after the actual data of interest.
Now that an exemplary description of how a model is determined has been provided above, reference is now made to the functional block diagram shown in
Note that although described using the same subject 108 in the same head orientation between calibration and standard MR scan sessions within a study, the magnetic resonance eye tracking system 100 is not limited to such implementations. Preferably, a model is generated as the result of a calibration session while a subject's head is in the same position and close in time (e.g., within the same study). However, a model may be generated by calibrating on the same subject in a separate scanning session (e.g., same study yet not close in time, or in a different study) and with a slightly different head position. In this latter circumstance, a 3-D image registration algorithm may be applied to register the calibration data and experimental data to the same 3-D space (e.g., using auxiliary data such as very high resolution images).
Additionally, though less desirable, a different subject may be used between calibration and normal sessions (and thus, different study). In this latter implementation, registration to another's (e.g., another individual or group) model, similar to that described above for differences in head position, is one approach to be used.
Having described one embodiment of a magnetic resonance eye tracking system 100, reference is now made to
Generally, in terms of hardware architecture, the processing device 104 includes a processor 312, memory 314, and one or more input and/or output (I/O) devices 316 (or peripherals) that are communicatively coupled via a local interface 318. The local interface 318 may be, for example, one or more buses or other wired or wireless connections. The local interface 318 may have additional elements such as controllers, buffers (caches), drivers, repeaters, and receivers, to enable communication. Further, the local interface 318 may include address, control, and/or data connections that enable appropriate communication among the aforementioned components.
The processor 312 is a hardware device for executing software, particularly that which is stored in memory 314. The processor 312 may be any custom made or commercially available processor, a central processing unit (CPU), an auxiliary processor among several processors associated with the magnetic resonance eye tracking software 300, a semiconductor-based microprocessor (in the form of a microchip or chip set), a macroprocessor, or generally any device for executing software instructions.
The I/O devices 316 may include input devices such as, for example, a keyboard, mouse, scanner, microphone, etc. Furthermore, the I/O devices 316 may also include output devices such as, for example, a printer, display, etc. Finally, the I/O devices 316 may further include devices that communicate both inputs and outputs such as, for instance, a modulator/demodulator (modem for accessing another device, system, or network), a radio frequency (RF) or other transceiver, a telephonic interface, a bridge, a router, etc.
The memory 314 may include any one or combination of volatile memory elements (e.g., random access memory (RAM)) and nonvolatile memory elements (e.g., ROM, hard drive, etc.). Moreover, the memory 314 may incorporate electronic, magnetic, optical, and/or other types of storage media. Note that the memory 314 may have a distributed architecture in which various components are situated remotely from one another but may be accessed by the processor 312.
The software in memory 314 may include one or more separate programs, each of which comprises an ordered listing of executable instructions for implementing logical functions. In the example of
The magnetic resonance eye tracking software 300 is a source program, executable program (object code), script, or any other entity comprising a set of instructions to be performed. As described previously, the magnetic resonance eye tracking software 300 can be implemented, in one embodiment, as a distributed network of modules, where one or more of the modules can be accessed by one or more applications or programs or components thereof.
The learning module 350 is configured to provide a stimulus that prompts the fixation of a subject's gaze, and is further configured to relate magnetic resonance data to fixation coordinates prompted by a stimulus or stimuli imposed on a subject and generate a model to be used in estimating eye fixation for a subject. In one embodiment, the learning module 350 implements a predictive eye estimation regression (PEER) algorithm 352 to determine fixation on an image-by-image basis. That is, the PEER algorithm 352 comprises a calibration or training session as described herein and the execution of a regression algorithm such as support vector regression (SVR), among others. In such an approach, eye-tracking calibration takes place during a preliminary or training imaging run whose sequence parameters (e.g., slice prescription, repetition time (TR), echo time (TE), flip angle, bandwidth, etc.) match those of the magnetic resonance imaging scans in the same study. Note that the image scans of the training session may be used to determine direction of gaze as well, although redundant to the implementation of the PEER algorithm. The learning module 350 further implements SVR to model each calibration image and its corresponding (known) fixation location. This model can then be used to predict eye fixation for other images in the study (e.g., using identical sequence parameters). In one embodiment, a separate regression model is used for horizontal and vertical fixations, although not necessarily limited to using separate regression models.
Other mathematical/statistical models besides SVR can be applied in some embodiments. For instance, SVR can be replaced with like-approaches used in different scientific disciplines (e.g., mathematics, statistics, pattern recognition, machine learning, etc.). Other current alternatives include, but are not limited to, neural networks, general linear model (GLM), multivariate adaptive regression splines (MARS), ridge regression, and Lasso regression. Such alternative regression approaches include empirically derived regression models, models based on first principles of MR physics and tissue material properties, among others.
The model application module 360 is configured to apply magnetic resonance data to the model generated by the learning module 350. That is, the aforementioned calibration model or models generated by the learning module 350 can be used by the model application module 360 to estimate the horizontal and vertical locations during other MR scan sessions. Note that more than a single model may be generated, such as individual horizontal and vertical models, or equivalent models can be combined, such as SVR and Lasso.
When the magnetic resonance eye tracking software 300 is a source program, then the program is translated via a compiler, assembler, interpreter, or the like, which may or may not be included within the memory 314, so as to operate properly in connection with the O/S 322. Furthermore, the magnetic resonance eye tracking software 300 can be written with (a) an object oriented programming language, which has classes of data and methods, or (b) a procedure programming language, which has routines, subroutines, and/or functions, for example but not limited to, C, C++, Pascal, Basic, Fortran, Cobol, Perl, Java, and Ada.
When the magnetic resonance eye tracking software 300 is in operation, the processor 312 is configured to execute software stored within the memory 314, to communicate data to and from the memory 314, and to generally control operations of the magnetic resonance eye tracking software 300 pursuant to the software. The magnetic resonance eye tracking software 300 and the O/S 322, in whole or in part, but typically the latter, are read by the processor 312, buffered within the processor 312, and then executed.
When the magnetic resonance eye tracking software 300 is implemented all or primarily in software, as is shown in
In an alternative embodiment, where functionality of the magnetic resonance eye tracking software 300 is implemented in whole or in part in hardware, such functionality can be implemented with any or a combination of the following technologies, which are each well known in the art: a discrete logic circuit(s) having logic gates for implementing logic functions upon data signals, an application specific integrated circuit (ASIC) having appropriate combinational logic gates, a programmable gate array(s) (PGA), a field programmable gate array (FPGA), etc; or can be implemented with other technologies now known or later developed.
In view of the above description, it should be appreciated that one embodiment of a magnetic resonance eye tracking method 300a, illustrated in
In view of the above description, it should be appreciated that one embodiment of a magnetic resonance eye tracking method 300b (and in particular, functionality corresponding to the learning module 350 and PEER algorithm 352), illustrated in
In view of the above description, it will be appreciated that one embodiment of a magnetic resonance eye tracking method 300c (and in particular, functionality corresponding to the model application module 360), illustrated in
Any process descriptions or blocks in flow charts should be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps in the process, and alternate implementations are included within the scope of the preferred embodiment of the present disclosure in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present disclosure. Further, it should be understood that the methods shown in, and described in association with,
Referring now to
Note that, although a specific implementation utilizing a standard pulse sequence frequently (but not exclusively) performed in fMRI experiments is described, numerous other known and future pulse sequences can be used in some embodiments. Further, very rapid eye movements, such as saccades, may require faster sampling frequencies. Additionally, the use of the PEER algorithm 352 does not alter fMRI results, and as a retrospective analysis tool, can be used at any fMRI site. As such, calibration runs can be acquired at any point in a scanning session.
Note that some embodiments may utilize conventional or future techniques in combination with the magnetic resonance eye tracking software 300 to improve the accuracy of the estimate. For instance, IR based eye tracking systems may be used with the magnetic eye tracking software 300 in some embodiments to provide an estimate (or improved estimate) of the true direction of a subject's gaze.
It should be emphasized that the above-described embodiments of the present disclosure, particularly, any “preferred” embodiments, are merely possible examples of implementations, merely set forth for a clear understanding of the principles of the disclosed systems and methods. Many variations and modifications may be made to the above-described embodiment(s) without departing substantially from the spirit and principles of the disclosure. All such modifications and variations are intended to be included herein within the scope of this disclosure.
This application claims priority to copending U.S. provisional application entitled, “MAGNETIC RESONANCE EYE TRACKING SYSTEMS AND METHODS,” having Ser. No. 60/793,887, filed Apr. 21, 2006, which is entirely incorporated herein by reference.
This invention was made with government support under grant number R01EB002009 and R21NS050183 awarded by the NIH. The government has certain rights in the invention.
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/US07/67192 | 4/23/2007 | WO | 00 | 9/30/2008 |
| Number | Date | Country | |
|---|---|---|---|
| 60793887 | Apr 2006 | US |
