Patent Application
20240200933
Consumer electronic devices such as smartphones may include Lidar modules and/or other sensor modules, which permit such consumer electronic devices to generate or sense a 3D scan of a spatial region. Consumer electronic devices may use such 3D scan data as part of a facial recognition process, a biometric authentication process, augmented reality process, autofocusing process, and/or another process. The effectiveness of such processes may be dependent upon the designed scanning or sensing range of the Lidar modules and/or other sensor modules. Moreover, one scanning range may not be sufficient for all processes for which the consumer electronic device may want to utilize captured 3D scan data. As such, the consumer electronic device may include multiple Lidar modules and/or other sensor modules in order to provide the scanning ranges required by the various processes.
Shown in and/or described in connection with at least one of the figures, and set forth more completely in the claims are sensors with addressable depth range and/or field of view and computing devices with such sensors.
These and other advantages, aspects and novel features of the present disclosure, as well as details of illustrated embodiments thereof, will be more fully understood from the following description and drawings.
The various features and advantages of the present disclosure may be more readily understood with reference to the following detailed description taken in conjunction with the accompanying drawings, wherein like reference numerals designate like structural elements.
The following discussion provides various examples of sensors and computing devices. Such examples are non-limiting, and the scope of the appended claims should not be limited to the particular examples disclosed. In the following discussion, the terms “example” and “e.g.” are non-limiting.
The figures illustrate the general manner of construction, and descriptions and details of well-known features and techniques may be omitted to avoid unnecessarily obscuring the present disclosure. In addition, elements in the drawing figures are not necessarily drawn to scale. For example, the dimensions of some of the elements in the figures may be exaggerated relative to other elements to help improve understanding of the examples discussed in the present disclosure. The same reference numerals in different figures denote the same elements.
The term “or” means any one or more of the items in the list joined by “or”. As an example, “x or y” means any element of the three-element set {(x), (y), (x, y)}. As another example, “x, y, or z” means any element of the seven-element set {(x), (y), (z), (x, y), (x, z), (y, z), (x, y, z)}.
The terms “comprises,” “comprising,” “includes,” and/or “including,” are “open ended” terms and specify the presence of stated features, but do not preclude the presence or addition of one or more other features.
The terms “first,” “second,” etc. may be used herein to describe various elements, and these elements should not be limited by these terms. These terms are only used to distinguish one element from another. Thus, for example, a first element discussed in this disclosure could be termed a second element without departing from the teachings of the present disclosure.
Unless specified otherwise, the term “coupled” may be used to describe two elements directly contacting each other or describe two elements indirectly connected by one or more other elements. For example, if element A is coupled to element B, then element A can be directly contacting element B or indirectly connected to element B by an intervening element C. Similarly, the terms “over” or “on” may be used to describe two elements directly contacting each other or describe two elements indirectly connected by one or more other elements.
Generally, aspects of the present disclosure are directed to sensors with an addressable depth range and/or field of view. Further aspects of the present disclosure are directed to computing devices with such sensors. To this end, a sensor may include multiple output couplers that are positioned at different distances from a receiver. Through selective enabling of the output couplers, light or other radiation generated by a transmitter may be directed toward the output couplers and selectively emitted from the sensor at different distances from the transmitter. Such selective emitting may adjust a depth range and/or field of view of the sensor. In this manner, a computing device may include a single sensor which the computing device may utilize across applications and/or processes having different needs with regard to depth range and/or field of view.
Referring to
The storage devices 120 may include one or more volatile storage devices and/or one or more non-volatile storage devices. In general, a storage device 120 may store software and/or firmware instructions, which may be executed by a processor 110. The storage devices 120 may further store various types of data which the processor 110 may access, modify, and/otherwise manipulate in response to executing instructions. To this end, the storage device 120 may include random access memory (RAM) device(s), read only memory (ROM) device(s), sold state device (SSD) drive(s), flash memory device(s), etc. In some embodiments, one or more devices of the storage devices 120 may be integrated with one or more processors 110.
The display device 130 may present visual output resulting from execution of instructions of the processor 110. To this end, the display device 130 may include one or more liquid-crystal display (LCD) devices, liquid-crystal on silicon (LCoS) devices, light-emitting diode (LED) devices, organic light-emitting diode (OLED) device, quantum dot devices, interferometric modulator device, or some other display device.
As explained in greater detail below, the sensor 140 may include a transmitter and a receiver that cooperate to generate a 3D scan of a spatial region within its addressable depth range and/or field of view. The computing device 100 may use the generated 3D scan data as part of a facial recognition process, a biometric authentication process, an augmented reality process, an autofocusing process, and/or another process. In particular, the processor 110 may execute instructions of an operating system, device driver, application, and/or some other software and/or firmware module resulting in the generation of control signals that adjust a depth range and/or field of view of the sensor 140.
The other I/O devices 150 may provide devices which enable a user or another device (e.g., another computing device, networking device, etc.) to interact with the computing device 100. For example, the I/O devices 150 may include buttons, touch screens, keyboards, microphones, audio speakers, etc. via which a person may interact with the computing device 100. The I/O devices 150 may also include network interfaces that permit the computing device 100 to communicate with other computing devices and/or networking devices. To this end, the networking interfaces may include a wired networking interface such as an Ethernet (IEEE 802.3) interface; a wireless networking interface such as a WiFi (IEEE 802.11) interface, BlueTooth (IEEE 802.15.1) interface; a radio or mobile interface such as a cellular interface (GSM, CDMA, LTE, etc.), and/or some other type of networking interface capable of providing a communications link between the computing device 100 and another computing device and/or networking device.
The above describes aspects of the computing device 100. However, there may be significant variation in actual implementations of the computing device 100. For example, a smart phone implementation of the computing device 100 may use vastly different components and may have a vastly different architecture than a laptop implementation of the computing device 100. Despite such differences, computing devices still generally include processors that execute software and/or firmware instructions in order to implement various functionality. As such, the above described aspects of the computing device 100 are not presented from a limiting standpoint but from a generally illustrative standpoint.
Certain aspects of the present disclosure may be especially useful for computing devices implemented as mobile consumer electronic devices (e.g., smartphones, tablets, laptops, etc.). However, the present disclosure envisions that aspects will find utility across a vast array of different computing devices and/or computing platforms and the intention is not to limit the scope of the present disclosure to a specific computing device and/or computing platform beyond any such limits that may be found in the appended claims.
Referring now to
As shown, the sensor 200 may include a guide 210 (e.g., a light guide or a waveguide), an integrated circuit device 220, a transmitter 230, a receiver 240, an input coupler 250, a first output coupler 260, a second output coupler 270, and a third output coupler 280. In various embodiments, the sensor 200 may provide an infrared (IR) based 3D scanning hardware solution for consumer electronics, Lidar modules, and/or other 3D scanning environments. In response to control signals, sensor 200 may selectively enable or activate individual couplers of the couplers 260, 270, 280, which dynamically configure the sensor 200 to provide a desired addressable depth range and/or field of view.
In certain embodiments, the guide 210 may be placed over the display device 130 (e.g., OLED screen) of the computing device 100. As shown, the guide 210 may be positioned or formed over a top side of an integrated circuit device 220 such that a bottom side of the guide 210 is on the top side of the integrated circuit device 220. The guide 210 may include the input coupler 250 along a bottom side of the guide 210 and over the transmitter 230. The guide 210 may further include the output couplers 260, 270, 280 along a top side of the guide 210. The input coupler 250 may couple an output of the transmitter 230 to the guide 210 so that the guide 210 may guide or direct the output of the transmitter 230 toward the output couplers 260, 270, 280.
The integrated circuit device 220 may include the transmitter 230. In various embodiments, the integrated circuit device 220 may include additional circuitry and/or components directed to the operation of the sensor 200 and/or directed to other functionality. Furthermore, in various embodiments, the transmitter 230 may be implemented as a device separate from the integrated circuit device 220.
The transmitter 230 may generate a beam 231 of light and/or other radiation based on control signals received by the sensor 200. In various embodiments, the transmitter 230 may generate a beam 231 of infrared (IR) light, which is emitted by the sensor 200 via a selected output coupler 260, 270, 280 to perform an IR scan of a spatial region per the selected addressable depth range. To this end, the transmitter 230 may include a vertical-cavity surface-emitting laser (VCSEL) array, a light-emitting diode (LED), and/or another infrared source. While the sensor 200 in various embodiments may use a beam of infrared light to generate a 3D scan of a spatial region, the transmitter 230 may generate beam(s) of other forms of radiation and the guide 210 may be manufactured to guide such beam(s) to the output couplers 260, 270, 280 so as to generate 3D scan data using such directed beam(s).
The receiver 240 may be positioned to receive, capture, and/or detect the beam 231 of the transmitter 230 after such beam 231 is emitted from the sensor 200 and reflected by object(s) within a scanning or depth range of the sensor 200. To this end, the receiver 240 may include an image sensor such as a charge-coupled device (CCD) image sensor, complementary metal-oxide semiconductor (CMOS) image sensor, or some other image sensor or camera suitable for receiving the beam 231 emitted by the transmitter 230. As such, the receiver 240 may be separate from the integrated circuit device 220. However, in various embodiments, the receiver 240 may be integrated with the integrated circuit device 220 and transmitter 230.
The processor 110 may generate control signals that dynamically control material properties of the output couplers 260, 270, 280. In this manner, the processor 110 may selectively configure each output coupler 260, 270, 280 between a total internal reflection (TIR) state, in which the respective coupler 260, 270, 280 confines the beam 231 to the guide 210, and an out-coupling state, in which the respective coupler 260, 270, 280 permits the beam 231 to escape the guide 210 via the respective coupler 260, 270, 280. In particular, the processor 110 as a result of executing system level software, may generate voltage, current, differential, and/or other control signals that selectively alter properties of the output couplers 260, 270, 280 so as to place respective output couplers 260, 270, 280 in a TIR state or an out-coupling state. Through the generation of such control signals, the sensor 200 may be configured to illuminate a depth range and/or field of view of interest.
Confinement in the guide 210 may be achieved by a reflective coating along the top side and bottom side of the guide 210. The reflective coating may have a high reflectivity at glazing angles and low reflectivity at near normal angles. The reflective coating may be selectively removed from areas where the beam is to be out coupled via the output couplers 260, 270, 280.
The output couplers 260, 270, 280 may be implemented based on prismatic, diffractive, or metasurface properties. In particular, the output couplers 260, 270, 280 may be placed in the guide 210 by embedding or etching in the guide 210 or by mounting on the top side of the guide 210. In certain embodiments, the output couplers 260, 270, 280 may comprise an active material and a metasurface optical element (MOE) designed for a specific material refractive index in grooves of the output couplers 260, 270, 280. In other embodiments, the output couplers 260, 270, 280 may comprise an active layer on the top side of the guide 210. In such embodiments, the guide 210 may comprise a silicon layer and the active layer of the output couplers 260, 270, 280 may comprise a liquid crystal on silicon (LCOS) layer. In yet other embodiments, the guide 210 itself may comprise an active layer and the output couplers 260, 270, 280 may be formed in the bulk of the active layer. In such embodiments, the output couplers 260, 270, 280 may use any region within the active layer of the guide 210 that is not required to maintain an exact refractive index.
As noted, the sensor 200 may be configured to provide a desired scanning or depth range. As shown in
A depth range or scanning range z of the sensor 200 is proportional to the baseline distance b between emitted beam 231 and the receiver 240 and may be represented by:
where z corresponds to the depth or scanning range, f corresponds to the focal length of the emitted beam 231, b corresponds to the baseline distance, d corresponds to the disparity between the focal length f and the location of the scanned object. By varying the baseline distance (i.e., distance between the receiver 240 and the emitted beam 231), the processor 110 may configure the sensor for different depth ranges and thus may generate 3D scans of objects at differing distances z from the sensor 200.
Referring now to
To this end, the processor 110 at 410 may generate control signals that configure the sensor 200 for a first depth range. For example, the processor 110 may enable the output coupler 260 and disable the output couplers 270, 280 so that beam 231 is emitted from output coupler 260 at a first baseline distance b1. At time T1, the receiver 240 may receive the beam 231 emitted at the first baseline distance b1 as the beam 231 is reflected by the scanned object. The processor 110 at 420 may receive, from the receiver 240, the scan data generated from the beam 231 emitted at the first baseline distance b1.
After receiving the data for the first baseline distance b1, the processor 110 at 430 may generate control signals that configure the sensor 200 for a second depth range. For example, the processor 110 may enable the output coupler 270 and disable the output couplers 260, 280 so that beam 231 is emitted from output coupler 270 at a second baseline distance b2. At time T2, the receiver 240 may receive the beam 231 emitted at the second baseline distance b2 as the beam 231 is reflected by the scanned object. The processor 110 at 440 may receive, from the receiver 240, the scan data generated from the beam 231 emitted at the second baseline distance b2.
After receiving the data for the second baseline distance b2, the processor 110 at 450 may generate control signals that configure the sensor 200 for a third depth range. For example, the processor 110 may enable the output coupler 280 and disable the output couplers 260, 270 so that beam 231 is emitted from output coupler 280 at a third baseline distance b3. At time T3, the receiver 240 may receive the beam 231 emitted at the third baseline distance b3 as the beam 231 is reflected by the scanned object. The processor 110 at 460 may receive, from the receiver 240, the scan data generated from the beam 231 emitted at the third baseline distance b3.
The processor 110 at 470 may generate 3D scan data based on the data received for the each of the baseline distances. In particular, the processor 110 may utilize various interpolation techniques in an attempt to improve accuracy of the 3D scan data, especially for zones of the scan that are overlapped by the depth ranges. In general, depth error is the lowest at the center of a scanned depth range. Overlapping zones results in the processor 110 receiving multiple data points for portions at the boundaries of the depth ranges. Through interpolation techniques (e.g., pointwise averaging, window averaging, etc.), the processor 110 may utilize these multiple or redundant data points to improve accuracy of the depth sensing and decrease depth error in the generated 3D scan data.
At 480, the processor 110 may utilized the generated 3D scan data. In particular, the processor 110 may use the generated 3D scan data as part of a facial recognition process, a biometric authentication process, an augmented reality process, an autofocusing, and/or another process.
Referring now to
Referring now to
The guide 210 may also provide optical structures on a top side or over each output coupler 260, 270, 280. For example, the guide 210 may provide the output coupler 260 with a diffractive optical element (DOE), the output coupler 270 with a diffuser, and the output coupler 280 with a focusing lens. Other embodiments may pair such optical structures with the output couplers 260, 270, 280 in a different manner and/or may combine optical structures for a single output coupler of the output couplers 260, 270, 280. Such flexibility in mixing-and-matching the optical structures for each output coupler 260, 270, 280 and such selectivity with regard to which of the output couplers 260, 270, 280 are placed in an out-coupling state may enable using a single transmitter 230 (e.g., a single IR source, a single VCSEL array, etc.) to support multiple IR based applications such as structured light, illumination, proximity sensing. Optical structures may be also formed within the guide 210.
For various applications, it may be desirable for the processor 110 to simultaneously enable multiple couplers 260, 270, 280. As such, the processor 110 and/or the sensors 200, 200′, 200″ may not be limited to out-coupling only via a single output coupler 260, 270, 280. Instead, the processor 110 and/or sensors 200, 200′, 200″ may permit out-coupling via various combinations of the output couplers 260, 270, 280. Furthermore, while sensors 200, 200′, 200″ are depicted with three output couplers, other embodiments of the sensors 200, 200′, 200″ may include a fewer number of output couplers and some embodiments of the sensors 200, 200′, 200″ may include a greater number of output couplers.
The present disclosure includes reference to certain examples, however, it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted without departing from the scope of the disclosure. In addition, modifications may be made to the disclosed examples without departing from the scope of the present disclosure. Therefore, it is intended that the present disclosure not be limited to the examples disclosed, but that the disclosure will include all examples falling within the scope of the appended claims.
