Patent Application
20150077542
Imaging systems based on light waves are becoming more widely used for object detection as semiconductor processes have become faster to support such systems. Some imaging systems are capable of providing dozens of images per second, making such systems useful for object detection and/or tracking as well. Due to their potentially small form factor and potentially high signal fidelity, some imaging systems are well suited for application in many types of vehicles (cars, busses, trains, etc.). Additionally, some imaging systems are well suited for gesture control, or the like, in many types of consumer devices (e.g., television, computers, tablets, smartphones, etc.). While the resolution of such imaging systems may vary, applications using these systems are able to take advantage of the speed of their operation.
A moving vehicle such as an automobile, for example, may use an imaging system to detect an object (a pedestrian, for example) in the path of the vehicle, to avoid hitting the object. For example, an imaging system may be employed on the front, rear, and/or the side(s) of the vehicle to detect objects in the forward or reverse paths of the vehicle, to the side of the vehicle, or in the blind spots of the vehicle.
Time-of-flight cameras, for example, may use imaging devices to measure the distance of an object from the camera. Imaging devices using multiple pixels may also be used, where light signals associated with individual pixels may provide distance measurements for discrete points on the object, forming a three-dimensional “distance image.” This can be made possible with light signals reflected off of the discrete points, for example.
Whether deployed in a vehicle application, a consumer device application, or other type of application, it is desirable to increase the efficiency of imaging systems while maintaining or improving the performance. For example, the usefulness of an imaging system as applied to a vehicle, mobile device, or the like, is diminished when the accuracy or clarity of the system is low, but the power demand is great.
The detailed description is set forth with reference to the accompanying figures. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. The use of the same reference numbers in different figures indicates similar or identical items.
For this discussion, the devices and systems illustrated in the figures are shown as having a multiplicity of components. Various implementations of devices and/or systems, as described herein, may include fewer components and remain within the scope of the disclosure. Alternately, other implementations of devices and/or systems may include additional components, or various combinations of the described components, and remain within the scope of the disclosure.
This disclosure is related to imaging systems (imaging systems using emitted electromagnetic (EM) radiation, for example) that are arranged to detect, image, recognize, and/or track objects and/or gestures in a predetermined area relative to the imaging systems. For example, an imaging system associated with a vehicle may be used to detect an object in the path of the vehicle, or in an area near the vehicle. Additionally, the imaging system may track the object or provide an image (such as a three-dimensional image, for example) of the object. In other examples, an imaging system may be used to detect and recognize gestures of an object or a human hand, for instance, in an area near a computing device. The imaging system may recognize when the object or hand is making a gesture, and track the hand-gesture combination as a replacement for a mouse or other input to the computing device.
In various implementations, the imaging system uses time-of-flight principles, such as distance calculations of reflected EM emissions (i.e., electro-magnetic radiation), to detect, image, recognize, and/or track objects, for example. Time-of-flight distance calculations may be based on receiving reflections of emitted EM (“light”) radiation, as the light radiation is reflected off objects in a predetermined area. For example, in one case, the distance calculations may be based on the speed of light and the travel time of the reflected light radiation.
Representative implementations of devices and techniques provide adaptive distance estimation for imaging devices and systems. In various implementations, distance estimation calculations may be based on phase delays in the reflected light radiation. In one non-limiting example, a phase delay value may be defined as the sampled output of a pixel, where the output of a pixel is the correlation between the incoming light (reflected by the scene) and the reference signal (generated on chip) with a fixed reference phase shift with respect to the emitted light signal. For example, phase delay values captured in the reflected light during a frame event (i.e., image capture at a point in time) can be used to determine the distance of an object from an imaging device, where the phase shift from the incident light to the captured reflected light indicates a distance traveled by the light.
In some implementations, one or more reference phases may be used to index the corresponding one or more phase shifts in the reflected light. The reference phases used to determine the phase delay values may be changed (e.g., alternated, rotated, etc.) for subsequent frame events. For example, consecutive frame events (i.e., image captures) may use different sets of one or more reference phases to determine the phase delays in the reflected light.
In an implementation, phase information from multiple frame events may be selected for some distance calculations, rather than using a single image capture. For example, a present frame event may be jointly processed with one or more past frame events to calculate the distance of an object from the imaging system. In an additional implementation, a quantity of past frame events may be selected based on whether object movement is detected within a predetermined area (e.g., the imaging area). In one implementation, the quantity of frame events selected is based on the relative speed of the object movement.
Various implementations and arrangements for imaging systems, devices, and techniques are discussed in this disclosure. Techniques and devices are discussed with reference to example light-based imaging systems and devices illustrated in the figures. However, this is not intended to be limiting, and is for ease of discussion and illustrative convenience. The techniques and devices discussed may be applied to any of various imaging device designs, structures, and the like (e.g., radiation based, sonic emission based, particle emission based, etc.) and remain within the scope of the disclosure.
Implementations are explained in more detail below using a plurality of examples. Although various implementations and examples are discussed here and below, further implementations and examples may be possible by combining the features and elements of individual implementations and examples.
In further implementations, the imaging system 102 may be applied with mobile devices or other computing devices such as laptop or notebook computers, hand-held computing devices, tablet computing devices, netbook computing devices, personal digital assistant (PDA) devices, reader devices, smartphones, mobile telephones, media players, wearable computing devices, stationary computers, personal or desktop computers, televisions, set-top boxes, gaming consoles, audio/video systems, appliances, and the like.
The scope of the disclosure is not intended to be limited to the examples listed. For convenience, reference within this disclosure to a vehicle 104 also refers to all applications employing an imaging system 102 as mentioned above, and otherwise.
In various implementations, the imaging system 102 may be integrated with the vehicle 104, or may have some components separate or remote from the vehicle 104. For example, some processing for the imaging system 102 may be located remotely (e.g., cloud, network, etc.). In another example, some outputs from the imaging system may be transmitted, displayed, or presented on a remote device or at a remote location.
The imaging system 102 may be used to detect an object 106, such as a pedestrian, for example, in a preselected area 108. In various implementations, the imaging system 102 may be arranged to detect, recognize, and/or track the movement of the object 106. In one implementation, the imaging device is arranged to output an image (e.g., a two-dimensional or three-dimensional image) of the object 106. In an example implementation, an output of the imaging system 102 may be presented or displayed on a display device, for example (e.g., a mobile computing device, a smartphone, a vehicle information system, etc.).
An example object 106 may include any item that an imaging system 102 may be arranged to detect, recognize, track, and/or the like. Such items may include a person or an animal, for example. Other examples of an object 106 may include an obstruction, a target component, another vehicle, and so forth. In various implementations, the imaging system 102 may also be arranged to detect, recognize, and/or track a gesture or configuration of the object 106. A gesture or configuration may include any movement or position of the object 106 that is expressive of an idea. For example, a gesture or configuration may include positioning an object 106 in an orientation and/or a movement of the object 106 in a pattern (e.g., in an elliptical motion, in a substantially linear motion, etc.). In one example, the imaging system 102 may be used to detect movement (e.g., a gesture) of a human hand, for example, as a replacement for a mouse or other input device for a computing device, mobile device, or the like.
The imaging system 102 may be arranged to detect, recognize, and/or track an object 106 that is within a preselected area 108 relative to the imaging system 102. A preselected area 108 may be chosen to encompass an area that objects 106 may be within, for example. In one case, the preselected area 108 may encompass an area that represents an immediate or distant forward or reverse path for the vehicle 104. This area 108 may also be to the front, side, or around the imaging system 102, for example.
The illustration of
As discussed above, the techniques, components, and devices described herein with respect to an imaging system 102 are not limited to the illustration in
In an implementation, the illumination module 202 is arranged to emit electromagnetic (EM) radiation (e.g., light radiation) to illuminate the preselected area 108. For example, the illumination module 202 may be arranged to illuminate one or more objects 106 that may be present in the area 108, to detect the objects, or to detect movement of the objects 106. In various implementations, the illumination module 202 includes an illumination source such as a light emitter, for example. In one implementation, the illumination source comprises a light-emitting diode (LED). In another implementation, the illumination source comprises a laser emitter. In one implementation, the illumination module 202 illuminates the entire environment (e.g., the preselected area 108) with each light pulse emitted. In an alternate implementation, the illumination module 202 illuminates the environment in stages or scans.
In various implementations, different forms of EM radiation may be emitted from the illumination module 202. In some implementations, infrared light, or the like, is emitted. For example, the light radiation may comprise one or more modulated light pulses. The illumination module 202 may be switched on for a short interval, allowing the emitted light pulse(s) to illuminate the area 108, including any objects 106 within the area 108. Infrared light, for example, provides illumination to the area 108 that is not visible to the human eye, and so is not distracting. In other implementations, other types or frequencies of EM radiation may be emitted that provide visual feedback or the like. As mentioned above, in alternate implementations, other energy forms (e.g., radiation based, sonic emission based, particle emission based, etc.) may be emitted by the illumination module 202.
If included in an implementation, the modulation component 204 may be arranged to modulate the EM radiation emitted from the illumination module 202 and/or to modulate one or more components (e.g., photosensitive pixels 302) of the sensor module 206. In various implementations, the modulation component 204 may be arranged to correlate the modulation of the EM radiation with the modulation of the components of the sensor module 206 during time-of-flight operation (e.g., to calculate a distance of an object 106 from the imaging system 102 and/or to form a two-dimensional or three-dimensional image of the object 106, for example).
In various implementations, as shown in
In another example, the sensor module 206 is arranged to provide phase shift information between the incident light radiation and the reflected light radiation for distance measurement calculations. For instance, in an implementation, the sensor module 206 is arranged to capture one or more phase delay values 410 (shown in
In an implementation, as shown in
In an implementation, each of the plurality of photosensitive pixels 302 are arranged to capture one or more phase delay values 410 of the light radiation reflected off one or more objects 106 in the area 108. In various implementations, the phase information 410 captured by individual pixels 302 during a frame event 406 may be processed by one or more processing components 402 (e.g., within the control module 208) to determine individual distances from the object(s) 106 in the area 108 to the individual pixels 302. The distances associated with the individual pixels 302 may be combined to form the three-dimensional distance image of the object(s) 106 and/or of the area 108.
When included in an implementation, the control module 208 may be arranged to provide controls and/or processing to the imaging system 102. For example, the control module 208 may control the operation of the imaging system 102, control the operation of one or more of the other modules (202, 204, 206), and/or process the signals and information output by one or more of the other modules (202, 204, 206). In various implementations, the control module 208 is arranged to communicate with one or more of the illumination module 202, modulation component 204, and sensor module 206. In some implementations, the control module 208 may be integrated into one or more of the other modules (202, 204, 206), or be remote to the modules (202, 204, 206).
In one implementation, the control module 208 is arranged to detect, recognize, and/or track an object 106 within the area 108, based on information received from the sensor module 206. In various implementations, the control module 208 may be programmed to recognize some objects 106 and/or gestures and exclude others.
In various implementations, the control module 208 is arranged to calculate a distance from the one or more objects 106 to the sensor module 206, based on one or more phase delay values 410 (e.g., I0, I1, I2, I3) of the light radiation captured by the sensor module 206 during the frame event. The phase delay values 410 are captured at one or more reference phase shifts with respect to the transmitted EM waveform. In one example, the phase delay values 410 are samples of the correlation function, which is the analog output of a pixel 302.
For example, in an implementation, the distance from an object 106 to the sensor module 206 may be estimated based on a phase change of the incident light with respect to the reflected light. The captured phase delay values 410 indicate the distance the light traveled as it was emitted from the illumination module 202, reflected off an object 106, and captured by the sensor module 206 during a frame event.
In one implementation, where the sensor module 206 is comprised of multiple pixels 302, the control module 208 is arranged to calculate distances from the one or more objects 106 within the area 108 to individual pixels 302 of the plurality of pixels 302. In the implementation, the distance calculations are based on one or more phase delay values 410 captured by the individual pixels 302. Each set of phase delay values 410 captured may be used by the control module 208 to estimate a distance from the object(s) 106 to the individual pixel 302 that captured the set.
In some implementations, three, four, or another quantity of phase delay values 410 may be used for each pixel's distance estimation, to improve accuracy of the estimation. Accordingly, the number of distance calculations (per pixel 302) performed for a frame event may be based on the quantity of pixels 302 comprising the sensor module 208 (i.e., the resolution of the sensor module 208) and the quantity of phase delay values 410 captured per pixel 302.
In an implementation, as illustrated in
Referring to
In an implementation, the phase delay values 410, as well as other information, may be stored in the memory 404, and accessed by the processor 402. In the implementation, the memory 404 comprises any memory storage device (integrated or separate hardware from the processor 402) capable of storing digital information (electrically, optically, magnetically, etc.) accessible by a processing component, such as the processor 402.
For example, in an implementation, the processing component 402 is arranged to select a set of reference phases 408 from the plurality of sets of reference phases 408 for each frame event 406, according to an algorithm. As shown in
A first example scenario is described with reference to
Further, at the previous frame k−1, the set of three reference phases 408 E, F, and G were selected by the processor 402, according to the algorithm. For example, the reference phases 408 E, F, and G may comprise 180°, 300°, and 60°. The resulting phase delay values 410 captured by the sensor module 206 (or an individual pixel 302 of the sensor module 206) to be used in distance estimation include I180(k−1), I300(k−1), and I60(k−1).
Then, at the present frame k, the set of three reference phases 408 A, B, and C were selected again by the processor 402, according to the algorithm. The resulting phase delay values 410 captured by the sensor module 206 (or an individual pixel 302 of the sensor module 206) to be used in distance estimation for the present frame include I0(k), I120(k), and I240(k).
In this example, the algorithm includes an alternating pattern, where the processor 402 alternately selects between two sets of reference phases 408 (A, B, C and E, F, G). Thus, for a subsequent frame event k+1, the set of three reference phases 408 E, F, and G will be selected again by the processor 402, according to the algorithm. The resulting phase delay values 410 to be captured by the sensor module 206 (or an individual pixel 302 of the sensor module 206) and to be used in distance estimation for the subsequent frame include I180(k+1), I300(k+1), and I60(k+1). In other examples, the alternating sets of reference phases 408 may include fewer or more phase angles (e.g., four phase angles, such as: 45°, 135°, 225°, and 315°, and the like).
In the example just described, the second set of reference phases 408 (E, F, and G) having the values of 180°, 300°, and 60° can be obtained by a signal inversion of the first set of reference phases 408 (A, B, and C) having the values of 0°, 120°, and 240°. In other examples, the phase angles of the reference phases 408 may have other relationships, or no relationship. Further, in an alternate example, the phase angles of the sets of reference phases 408 may not be equally spaced, and may be selected based on desired design criteria.
A second example scenario is illustrated with reference to
In other examples, rotating sets of reference phases 408 may include fewer or more phase angles, and/or a greater quantity of sets of reference phases 408 may be rotated through, according to the algorithm used. In further examples, other algorithms may be employed for selection of set(s) of reference phase 408 by the processor 402, having other patterns and/or selection criteria. For example, in an implementation, the processor 402 is arranged to randomly or quasi-randomly select a set of reference phases 408 from a plurality of sets of reference phases 408 for distance estimation. In an implementation, the processor 402 is arranged to select sets of reference phases 408 per pixel 302, according to algorithms selected per pixel 302. In other words, in one implementation, adjacent pixels 302 of the sensor module 206 may use different reference phases, according to algorithms, for a given frame event 406.
In one implementation, the processing component 402 is arranged to calculate distance estimation for a frame event 406 using phase delay values 410 stored for that single frame event 406. Using the example above, the phase shift of a pixel 302 (such as a photon mixing device (PMD) pixel 302, for example) at frame k−1 can be expressed as:
x(k−1)=−1·(I300(k−1)−I60(k−1))·√{square root over (3)},
y(k−1)=−1·(2·I180(k−1)−I300(k−1)−I60(k−1))
θ(k−1)=a tan 2(y(k−1),x(k−1))
and for frame k, the phase shift can be expressed as:
x(k)=(I120(k)−I240(k))·√{square root over (3)}
y(k)=2·I0(k)−I120(k)−I240(k)
θ(k)=a tan 2(y(k),x(k))
In another implementation, the processing component 402 is arranged to calculate distance estimation for a frame event 406 using joint processing of phase delay values 410 stored for that frame event 406 as well as one or more past frame events 406 to improve accuracy of the distance estimation. In other words, the processing component 402 is arranged to calculate the distance from one or more objects 106 to the imaging sensor 206 (or to a pixel 302 of the imaging sensor 206) based on one or more phase delay values 410 of light radiation captured during a present frame event 406 combined with one or more phase delay values 410 of light radiation captured during one or more past frame events 406.
Again, using the above example, the phase shift of a pixel 302 (such as a PMD pixel 302, for example) at present frame event k, using phase delay values 410 from frame events 406 k and k−1 can be expressed as:
θ(k)=a tan 2(y(k)+y(k−1),x(k)+x(k−1)).
In alternate implementations, the processing component 402 may be arranged to calculate distance estimation for a frame event 406 using phase delay values 410 stored for several frame events 406, while the phase delay 410 information is accessible within the memory 404. For example, the processing component 402 may use the phase delay values 410 of one or more temporally sequential or non-sequential frame events 406, with or without the phase delay values 410 of the present frame event 406, to calculate distance estimation.
In various implementations, the joint processing of one or more previous frame 406 phase delay values 410 with the current frame 406 phase delay values 410 in distance calculations improves the accuracy of the distance estimation. The use of multiple frame events 406 can have a noise averaging effect. Further, better modeling accuracy may be achieved due to more reference phase values 408 used.
In an implementation, the processing component 402 is arranged to adaptively select a quantity of one or more past frame events 406 (and to calculate distances using the associated phase delay values 410) based on whether movement of an object 106 is detected within the area 108 within a preset time duration. For example, in one implementation, the quantity of past frame events 406 is adaptively selected based on the speed of the object 106 movement detected within the area 108 within the preset time duration. In various implementations, the speed of object 106 movement is relative, and may include the speed of the object 106 itself, the speed of the imaging system 102 with respect to the object 106, or a combination of the two.
In one implementation, the processing component 402 is arranged to select fewer past frame events 406 when faster object 106 movement is detected within the area 108 within the preset time duration, and to select more past frame events when slower object 106 movement is detected within the area 108 within the preset time duration. For example, the processing component 402 may be arranged to dynamically make new quantity selections for successive frame events 406 (e.g., for each new “present” frame) during runtime. In an alternate implementation, more past frame events 406 may be selected when faster object 106 movement is detected and fewer frame events 406 may be selected when slower object 106 movement is detected.
In an implementation, one or more relative speed thresholds may be preselected for determining a quantity of frame events 406 to be selected as the relative speed reaches the thresholds. For example, in one example where the relative speed of the object 106 with respect to the imaging system 102 reaches a maximum threshold, a single frame event 406 may be selected, and the associated phase delay values 410 may be used for distance calculations. In another example where the relative speed of the object 106 with respect to the imaging system 102 reaches a minimum threshold (i.e., little or no relative movement, for example), a predefined maximum quantity of frame events 406 may be selected, and the associated phase delay values 410 used for distance calculations. An additional quantity of thresholds may be preselected to trigger other progressive quantities of frame events 406 between the two extremes. Alternately, the preselected thresholds may trigger the quantity of frame events 406 to be selected in a reverse manner, as mentioned above.
In various implementations, the quantity of frame events 406 selected based on the detection and the speed of object 106 movement is performed on a pixel-by-pixel 302 basis. For example, an object 106 may be moving in one portion of the area 108 corresponding to one or more pixels 302, and the remaining portions of the area 108 may be free from object 106 movement. In such a case, selection of frame events 406 to be used to calculate distances may be performed per pixel 302 for improved efficiency.
Additionally, when no object 106 movement is detected in a portion of the area 108 over a predetermined time frame, the imaging system 102 may refrain from making new distance calculations with respect to the pixels 302 associated with the portion of the area 108, and may instead reuse previous distance calculations from earlier frame events 406 at the pixels 302. In this way, the imaging system 102 can improve processor 402 and pixel 302 signaling efficiency by reducing redundant calculations.
In an implementation, the control module 206 is arranged to adaptively select a quantity of one or more past frame events 406 during runtime, based on minimizing the occurrence of motion artifacts. For example, in some cases, motion artifacts can appear while joint processing over multiple frames, particularly when the scene is rapidly changing (e.g., when objects 106 within the area 108 are moving rapidly with respect to the imaging system 102). In the implementation, the adaptive selection of past frame events 406 by the processing component 402 can mitigate the occurrence of motion artifacts. As mentioned above, the adaptive selection may be performed based on a “speed of change” criterion (e.g., the speed of object 106 movement with respect to the imaging system 102).
In an implementation, the processing component 402 is arranged to detect object 106 movement within the area 108 based on an analysis of the temporal evolution (e.g., from frame 406 to frame 406) of one or more phase delay values 410 of light radiation captured during a present frame event 406 and during one or more past frame events 406. For example, the processing component 402 may analyze a sequence of phase shifts of the phase delay values 410 over a series of frame events (e.g., over a time duration) for one or more pixels 302 to detect object 106 movement within the area 108. The sequence of phase shifts of the phase delay values 410 (or functions thereof) can indicate object 106 movement by showing changing phase shifts relative to a pixel 302, for example.
In various implementations, one or more detection algorithms may be used to detect object 106 movement within the area 108. Determination of object 106 movement thresholds may be based on what is considered to be a significant change of distance for a particular application. For example, in one implementation, a significant distance change may be defined as on the order of several centimeters. In other examples, a significant distance change may be much greater or much smaller.
Referring back to the example expression for a phase shift of a pixel 302, using phase delay values 410 from frame events 406 k and k−1:
θ(k)=a tan 2(y(k)+y(k−1),x(k)+x(k−1))
the input values of the a tan 2 function may be used to detect changing distances between the imaging system 102 and an object 106 in the area 108. In an example, the analysis is performed before the main processing function of the expression above.
In a first example detection algorithm, less computational effort is used. The distance between an object 106 and a pixel 302 has changed if the absolute value of x(k)−x(k−1) or y(k)−y(k−1) is larger than one or more predefined movement thresholds. Example computations for the first example detection algorithm include
d
x(k)=|x(k)−x(k−1)|
d
y(k)=|y(k)−y(k−1)|
In an example, the first detection algorithm may be used if dx(k)>δx or dy(k)>δy, otherwise, the second example detection algorithm (shown below) is used. In one implementation, the thresholds δx and δy are dependent on an expected noise level in x and y.
In the second example detection algorithm, more computational effort may be used than with the first example detection algorithm, but better accuracy may be achieved. The second example detection algorithm is based on a first order Taylor Series approximation of the a tan 2( )) function in the vicinity of the input values at frame event 406 (e.g., time index) k−1 (i.e., x(k−1) and y(k−1)), according to:
In the example, the present phase shift at frame event 406 k is expressed as the previous phase shift at frame event 406 k−1, plus an update term. In other words, the update term is a measure of the phase change (and distance change) from time k−1 to time k. In an implementation, the second example detection algorithm can be accurate for very small phase changes.
In an example, the second example detection algorithm may be used if
otherwise, the first example detection algorithm (shown above) is used. In one implementation, the division by (x(k−1))+(y(k−1))2 of the second example detection algorithm may be omitted, to increase processing efficiency if desired.
In an alternate implementation, the first example detection algorithm and the second example detection algorithm may be applied jointly.
The captured phase delay values 410 (by the pixel 302) from the reflected optical signal s(t), based on the reference phase values 408, are stored in the memory 404 for each frame event 406. The processing component 402 retrieves the phase delay values 410 for the current frame event 406 k, and processes them to calculate phase shift indeces according to the following expressions:
{tilde over (x)}
k: =√{square root over (3)}(I1(k)−I2(k))
{tilde over (y)}
k:=2I0(k)−I1(k)−I2(k)
The phase shifts from the present frame event 406 may be jointly processed with the phase delay values 410/phase shifts from one or more previous frame events 406 to determine a distance estimation {circumflex over (φ)}, according to the a tan 2 expression discussed above. In an example, as shown in
In various implementations, the control module 208 may also be arranged to calculate a distance of the object 106 from the imaging system 102, based on the measured time of the reflected EM radiation. Accordingly, the control module 208 may be arranged to convert the current signal output from the sensor module 206 (or from the pixels of the sensor module 206) to a distance of the object 106 from the imaging system 102. Further, in an implementation, the control module 208 may be arranged to convert the current signal to a three-dimensional image of the object 106.
In one implementation, the control module 208 is arranged to output the calculated distance and/or the three-dimensional image of the object 106. For example, the imaging system 102 may be arranged to output a distance, a three-dimensional image of the detected object 106, tracking coordinates of the object 106, and so forth, to a display device, to another system arranged to process the information, or the like.
In various implementations, additional or alternative components may be used to accomplish the disclosed techniques and arrangements.
The order in which the process is described is not intended to be construed as a limitation, and any number of the described process blocks can be combined in any order to implement the process, or alternate processes. Additionally, individual blocks may be deleted from the process without departing from the spirit and scope of the subject matter described herein. Furthermore, the process can be implemented in any suitable materials, or combinations thereof, without departing from the scope of the subject matter described herein.
In one example, the EM radiation may be emitted by an emitter (such as illumination module 202, or the like) comprising an LED or laser emitter, for example. In some implementations, the EM radiation may be modulated light. For example, the light may be modulated using a modulation component (such as modulation component 204) and may be modulated in correlation to the modulation of one or more photosensitive pixels of the imaging device.
At block 602, the process includes capturing light radiation reflected off one or more objects (such as object 106, for example) in an area (such as area 108, for example) during a frame event (i.e., an image capture at a point in time), with an imaging sensor (such as sensor module 206 and/or pixel 302, for example). The imaging sensor may be comprised of one or more photosensitive pixels (such as pixels 302), for example. The EM reflection may be received by the imaging sensor via optics, a receiver, an antenna, or the like, for instance.
At block 604, the process includes selecting a set of reference phase values (such as reference phase values 408, for example) from a plurality of sets of reference phase values, according to an algorithm. The process further includes selecting a subsequent set of reference phase values from the plurality of sets of reference phase values for each subsequent frame event according to the algorithm. In an example, the reference phase values change for each subsequent frame event from the previous frame event. In some implementations, the reference phase values alternate, rotate, and the like, among a quantity of sets of reference phases.
At block 606, the process includes determining one or more phase delay values (such as phase delay values 410, for example) of light radiation captured during the current frame event, corresponding to the set of reference phase values selected.
At block 608, the process includes calculating a distance from the one or more objects to the imaging sensor, based on the one or more phase delay values of the light radiation captured during the current frame event. In an implementation, the process includes calculating the distance from the one or more objects to the imaging sensor based on one or more phase delay values of light radiation captured during a present frame event combined with one or more phase delay values of light radiation captured during one or more past frame events.
In an implementation, the process includes calculating a distance from the one or more objects to a pixel of the imaging sensor. The process further includes adaptively selecting a quantity of frame events for the distance calculation relative to the pixel during runtime (e.g., per pixel selection). For example, the selection may be based on detecting a threshold object movement within a portion of the area corresponding to the pixel.
In an implementation, the process includes adaptively selecting the quantity of frame events for the distance calculation based on a speed of object movement detected. For example, the process includes increasing a quantity of frame events selected when the speed of object movement decreases and decreasing a quantity of frame events selected when the speed of object movement increases. In an alternate implementation, the process may include decreasing a quantity of frame events selected when the speed of object movement decreases and increasing a quantity of frame events selected when the speed of object movement increases, based on the application of the imaging device.
In an implementation, the past frame events selected for the distance calculation are a sequential set of frame events (e.g., a sequence of time). In various implementations, the frame events selected for the distance calculation relative to the pixel are temporally non-sequential. For example, the frame events selected may conform to another pattern that is non-sequential or they may be random or quasi-random.
In an implementation, the process includes reusing a distance calculation from one or more objects to a pixel when no threshold object movement is detected relative to the pixel, within a preselected time duration. For example, instead of recalculating a distance from the pixel to an object when there is no relative movement of the object, previous distance calculations may be reused for the object. This may increase processor efficiency when only a portion of a scene is changing (e.g., an object at one portion of the area corresponding to one or more pixels is moving, but the remainder of the area is static).
In an implementation, the process includes mitigating an occurrence of motion artifacts and/or averaging imaging noise by adaptively selecting a quantity of past frame events for the distance calculation, based on a speed of object movement detected within the area.
In another implementation, the process includes analyzing a temporal sequence of phase shifts of the phase delay values over a series of frame events to detect object movement within the area. In the implementation, a threshold change in a phase delay value or a phase shift relative to a pixel of the imaging sensor indicates a change in a distance of the object within a portion of the area corresponding to the pixel.
In one implementation, the process includes defining a phase shift of a phase delay value at a present frame event as a former phase shift of a previous frame event plus an update term. For example, to calculate the distance from the one or more objects to the imaging sensor, the update term indicates a change in distance of the one or more objects from a time of the previous frame event to a time of the present frame event.
In an implementation, the process further includes time-of-flight techniques such as measuring a time from emitting the EM radiation to receiving the reflection of the EM radiation and calculating a distance of an object based on the measured time. In various implementations, the process includes outputting image or gesture information to a computing device for input control, or the like. Additionally, the process may include outputting imaging information, such as a distance, a three-dimensional image of the detected object, tracking coordinates of the object, and so forth, to a display device, to another system arranged to process the information, or the like.
In alternate implementations, other techniques may be included in the process in various combinations, and remain within the scope of the disclosure.
Although the implementations of the disclosure have been described in language specific to structural features and/or methodological acts, it is to be understood that the implementations are not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as representative forms of implementing example devices and techniques.
