The present application claims priority to Indian Application No. 202141054242 filed on Nov. 24, 2021 in the Indian Patent Office, the contents of which are incorporated herein by reference in its entirety.
The present disclosure relates, in general, to a spot Time of Flight (ToF) sensor, and more particularly but not exclusively to a method and a system for optimizing sampling in a spot ToF sensor using Complementary Metal-Oxide-Semiconductor CMOS Image Sensor (CIS) data.
A Time-of-Flight (ToF) sensor that generates high-resolution depth inferences at a group of sample locations is known as a Spot-ToF sensor. Spot ToF employs high-power LASER beams for active depth sensing, depth is computed by measuring a difference in time or phase between emitted and received light. Spot ToF includes a full resolution indirect flood ToF depth sensor in front of a specific module along with a spot ToF module.
However, Spot ToF has some disadvantages. For example, low-density sampling may result in the loss of 3D structures. On the other hand, uniform high-density sampling may necessitate substantial power, which is not available on portable devices. Also, a sample pattern is not based on near regional depth discontinuities, which is required for successful 3D reconstruction.
One approach to addressing these disadvantages uses Intelligent Detection and Ranging (iDAR), wherein a dynamic or adaptive Light Detection and Ranging (LiDAR) with a variable illumination pattern is used. Here, information from a High Definition (HD) camera is utilized for adaptive 3D sensing using LiDAR signals. Also, Deep Neural Network (DNN) is used to map scene structure to an illumination pattern. However, such an approach requires a higher power-consuming DNN, running on compute-intensive Graphics Processing Units (GPU) to infer a rough scene structure and an illuminating pattern suitable to that scene. Also, training the DNN requires a large volume of carefully annotated data. Moreover, a DNN trained for one scenario may not scale well to other scenarios. For example, autonomous driving scenarios are very different from Augmented Reality (AR)\Virtual Reality (VR) scenarios. Thus, it is advantageous to optimize the Spot ToF power utilization, since such optimization allows for higher frame rates and longer usage time before recharging a portable device. Also, it is advantageous to have a Spot ToF that scales well to unseen scenarios and does not require retraining.
According to an aspect of one or more embodiments, there is provided a method comprising receiving, by a sampling system, one or more images of a scene captured using a Complementary Metal Oxide Semiconductor (CMOS) Image Sensor (CIS) camera; dividing, by the sampling system, each of the one or more images into a plurality of rectangular regions, based on an edge feature identified in the one or more images; computing, by the sampling system, an edge region alignment for each of the plurality of rectangular regions by analyzing a Histogram of oriented Gradients (HoG) distribution corresponding to each of the plurality of rectangular regions; re-projecting, by the sampling system, Time of Flight (ToF) data on a CIS image plane according to the edge region alignment and a directional sampling filter for computing a regional depth variance; sampling, by the sampling system, one or more rectangular regions from among the plurality of rectangular regions by comparing the regional depth variance with a threshold depth variance; and dynamically reconfiguring, by the sampling system, an illumination pattern for a spot ToF sensor image frame using the one or more rectangular regions that are sampled, for reconstructing a three dimensional (3D) model of the scene.
According to another aspect of one or more embodiment, there is provided a sampling system comprising a processor, and a memory communicatively coupled to the processor, the memory storing processor-executable instructions which when accessed and executed by the processor causes the processor to receive one or more images of a scene captured using a Complementary Metal Oxide Semiconductor (CMOS) Image Sensor (CIS) camera; divide each of the one or more images into a plurality of rectangular regions, based on an edge feature identified in the one or more images; compute an edge region alignment for each of the plurality of rectangular regions by analyzing a Histogram of oriented Gradients (HoG) distribution corresponding to each of the plurality of rectangular regions; re-project Time of Flight (ToF) data on a CIS image plane according to the edge region alignment and a directional sampling filter for computing a regional depth variance; sample one or more rectangular regions from among the plurality of rectangular regions by comparing the regional depth variance with a threshold depth variance; and dynamically reconfigure an illumination pattern for a spot ToF sensor image frame using the one or more rectangular regions that are sampled, for reconstructing a 3D model of the scene.
According to another aspect of one or more embodiment, there is provided a method comprising receiving, by a processor, an image of a scene; dividing, by the processor, the image into a plurality of rectangular regions, based on an edge feature in the image; computing, by the processor, an edge region alignment for each rectangular region by analyzing a Histogram of oriented Gradients (HoG) distribution corresponding to the rectangular region; re-projecting, by the processor, Time of Flight (ToF) data on a Complementary Metal Oxide Semiconductor (CMOS) Image Sensor (CIS) image plane according to the edge region alignment; sampling, by the processor, one or more rectangular regions from among the plurality of rectangular regions by comparing a regional depth variance of each rectangular region with a threshold depth variance; and reconfiguring, by the processor, an illumination pattern for a spot ToF sensor image frame using the one or more rectangular regions that are sampled.
The above and/or other aspects will become apparent and more readily appreciated from the following description of various embodiments, taken in conjunction with the accompanying drawings, in which:
The accompanying drawings, which are incorporated in and constitute a part of the present disclosure, illustrate exemplary embodiments and, together with the description, explain principles consistent with the present disclosure. The same reference numbers are used throughout the figures to reference like features and components.
It should be appreciated by those skilled in the art that any block diagrams herein represent conceptual views of illustrative systems embodying the principles of the present subject matter. Similarly, it will be appreciated that any flow charts, flow diagrams, state transition diagrams, pseudo code, and the like represent various processes which may be substantially represented in a computer readable medium which, when accessed and executed by a computer or processor, cause the computer or processor to execute the various processes represented in the flow chart, flow diagrams, state transition diagrams, pseudo code and the like.
In the present document, the word “exemplary” is used herein to mean “serving as an example, instance, or illustration.” Any embodiment or implementation of the present subject matter described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.
While the disclosure is susceptible to various modifications and alternative forms, specific embodiments thereof has been shown by way of example in the drawings and will be described in detail below. It should be understood, however that these specific embodiments are not intended to limit the disclosure to the specific forms disclosed, but on the contrary, the disclosure is to cover all modifications, equivalents, and alternative falling within the scope of the disclosure and claims.
The terms “comprise”, “comprising”, or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a setup, device, or method that comprises a list of components or steps does not include only those components or steps but may include other components or steps not expressly listed or inherent to such setup or device or method. In other words, one or more elements in a system or apparatus proceeded by “comprises . . . a” does not, without more constraints, preclude the existence of other elements or additional elements in the system or method.
The terms “include”, “including”, or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a setup, device, or method that includes a list of components or steps does not include only those components or steps but may include other components or steps not expressly listed or inherent to such setup or device or method. In other words, one or more elements in a system or apparatus proceeded by “includes . . . a” does not, without more constraints, preclude the existence of other elements or additional elements in the system or method.
In the following detailed description of the embodiments of the disclosure, reference is made to the accompanying drawings that form a part hereof, and in which are shown by way of illustration specific embodiments in which the disclosure may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the disclosure, and it is to be understood that other embodiments may be utilized and that changes may be made without departing from the scope of the present disclosure and claims. The following description is, therefore, not to be taken in a limiting sense.
The present disclosure relates to a method and a sampling system for optimizing sampling in a spot Time-of-Flight (ToF) sensor using CMOS Image Sensor (CIS) data. The method is configured to receive one or more images of a scene from an image capturing device using a CIS camera. After receiving the scene (for example, a 3-Dimension (3D) scene) from an image capturing device, the method is configured to determine a scene structure by dividing each of the one or more images into a plurality of rectangular regions of a certain size. The certain size may be predetermined. Once the scene structure is determined, the method is configured to compute an edge region alignment for each of the plurality of rectangular regions by analyzing a Histogram of oriented Gradients (HoG) distribution for establishing a sampling kernel. Thereafter, the method is configured to re-project ToF data on a CIS image plane according to the edge region alignment and apply an appropriate directional sampling filter for computing a regional depth variance. Once the regional depth variance is computed, the method is configured to sample one or more rectangular regions from the scene structure by comparing the regional depth variance with a threshold depth variance. Based on a comparison between the regional depth variance and a threshold, rectangular regional samples are removed (on the identification of a 2-Dimension (2D) edge) or rectangular regional samples are accumulated (on the identification of a 3D edge). Further, the method is configured to dynamically reconfigure an illumination pattern for the spot ToF sensor image frame using a filtered sampled scene structure for reconstructing a 3D model of the scene.
In this manner, methods, devices, and systems consistent with the present disclosure optimizes sampling in a spot Time-of-Flight (ToF) sensor using CMOS Image Sensor (CIS) data, which in turn reduces power consumption on every active sampling. In addition, the method may be implemented to function with low-power mobile devices by utilizing straightforward image features such as edges of objects.
In an embodiment, an exemplary environment 100 may include, without limitation, an image capturing device 101, an edge detection device 103, a sampling system 105 and a three-Dimensional (3D) model 107. The sampling system 105 comprises a processor 109, an input/output (I/O) Interface 111 and a memory 113. The processor 109 may interface with the memory 113 and may be configured for performing one or more functions of the sampling system 105 while optimizing the sampling in the spot Time-of-Flight (ToF) sensor. The I/O Interface 111 may be configured for receiving one or more images of a scene from the image capturing device 101. The memory 113 may also store data 115 and modules 117 of the sampling system 105.
The Time-of-Flight (ToF) technique is an active imaging technique, which provides a distance between the camera and an object. There are two techniques in ToF, namely Direct ToF (dToF) and Indirect ToF (iToF). In the iToF method, sensors assess the depth from phase-delay. In the direct ToF method, the sensors measure the depth from the time-delay between emitted and received light pulses.
In an embodiment, the iToF may be used with Flood iToF (for example, continuous samples) or spot iToF (for example, discrete samples). The spot-ToF sensor comprises a full-resolution Flood ToF which is present at the front end of the system and a spot ToF module that emits high-power Light Amplification by Simulated Emission of Radiation (LASER) light for accurate depth at a set of sample points. For example, a 640×480 pixels Flood ToF canvas may comprise 700 sample points.
In an embodiment, the image capturing device 101 may be configured for capturing one or more images of a scene. As an example, the image capturing device 101 may be, without limitation, a Complementary Metal-Oxide-Semiconductor (CMOS) Image Sensor (CIS). In some embodiments, the image capturing device 101 may be used as an intermediate storage for receiving and storing the one or more images of the scene from other image capturing devices (not shown in
In an embodiment, the sampling system 105 may receive the one or more images from the image capturing device 101 and divide each of the one or more images into a plurality of rectangular regions of a certain size, based on an edge feature identified in the one or more images. For example, a 3D scene image obtained from a CIS sensor edge map may be divided into rectangular regions of dimension 32×32 pixels. Region-based edge features of the CIS camera may be used for identifying a scene structure. After dividing each of the one or more images, an edge region alignment may be computed based on the Histogram of Oriented Gradients (HOG) distribution for determining a sampling kernel. For example, the edge region alignment may be computed using HoG values segregated in bins corresponding to a 0° alignment, a 45° alignment, a 90° alignment, and a 135° alignment. Furthermore, the ToF data may be re-projected onto a CIS image plane based on edge region alignment and by using a suitable primitive/directional sampling filter to calculate regional depth variance. The directional sampling filters may include a horizontal direction filter, a vertical direction filter, a circular filter, a diagonal direction filter and an anti-diagonal direction.
In an embodiment, the iToF depth on the CIS edge map may be re-projected for computing regional depth variance (e.g., SD) around the edges. The 2D edges may be filtered based on a comparison between neighborhood depth variance and a Threshold (TH). Sampling may be applied on one or more rectangular regions from the scene structure by comparing the regional depth variance with a threshold depth variance. Thereafter, one or more rectangular regions with two-Dimensional (2D) edges and one or more rectangular regions with 3D edges may be identified after sampling the one or more rectangular regions. Once the regional depth variance is compared, if the depth variance is less than the threshold value (SD<TH), the regional samples on the identification of the 2D edge are removed. On the other hand, when the depth variance is equal to or exceeds the threshold value, the samples on 3D edge identification are accumulated. Further, the illumination pattern for spot ToF frame is re-configured based on re-projected filtered samples for reconstructing a 3D model 107 of the scene.
In some implementations, the sampling system 105 receives data 115 through the I/O interface 111. The received data 115 is stored within the memory 113. In an embodiment, the data 115 stored in the memory 113 may include, without limitation, data 115 related to one or more images 201 (or image data 201), region data 202, sensor data 203, depth variance data 204, edges data 205 and other data 206 associated with the sampling system 105.
In one embodiment, the data 115 may be stored in memory 113 in the form of various data structures. Additionally, the data 115 may be organized using data models, such as relational or hierarchical data models. The other data 206 may store data, including various temporary data and temporary files, generated by modules 117 for performing the various functions of the sampling system 105. As an example, the other data 206 may include, without limitation, temporarily stored previous input data or stored data collected from the various images.
In an embodiment, the image data 201 comprises one or more images of a scene. The images may be captured in an indoor environment or an outdoor environment. The captured images may include, without limitation, High-Definition (HD) images, Red Green Blue (RGB) color images, or hyperspectral images. The image data 201 may also contain historic images of the scene obtained under various environmental conditions.
In an embodiment, the region data 202 may be the raw data obtained by dividing the one or more images into a plurality of rectangular regions. In some embodiments, the rectangular regions may be of small discrete pixels. The rectangular regions may be of a size that is predetermined. As an example, in some embodiments, the size of each of the rectangular regions may be 32×32 pixels.
In an embodiment, the sensor data 203 may comprise data obtained from various sensors including the Spot ToF sensor, CMOS image sensor and the like. Data generated and captured with the spot ToF sensors may be used to estimate the depth of the 3D scene. ToF depth sensor devices enable the sampling system 105 to easily retrieve scene depth data with high frame rates.
In an embodiment, the depth variance data 204 may contain both regional depth variance and threshold depth variance data. Variance distribution in a depth image may be taken at an average distance from a scene. and the depth may contain a large amount of noise near the corners of the image. The threshold depth variance is determined by analyzing each of the one or more images of the scene.
In an embodiment, the edges data 205 comprises information related to the 2D edges and the 3D edges of the captured images. The edge of an object is a feature used to determine the shapes, positions, or size of the object. The 2D edges are identified when the regional depth variance is less than the threshold depth variance, wherein the threshold depth variance is determined by how aggressively the 2D edges are pruned, and the threshold depth variance may be tuned to achieve a desired output sampling density. The 3D edges are identified when the regional depth variation is equal to or more than the threshold depth variance.
In an embodiment, the data 115 stored in memory 113 may be processed by one or more modules 117 of the sampling system 105. The modules 117 may be stored within the memory 113 as shown in
In one implementation, the modules 117 may include, for example, a receiving module 208, a determining module 209, a computing module 210, a re-projection module 211, a sampling module 212, a reconfiguring module 213, and other modules 214. The other modules 214 may be used to perform various miscellaneous functionalities of the sampling system 105. It will be appreciated that such aforementioned modules 117 may be represented as a single module or a combination of different modules 117.
In an embodiment, the receiving module 208 may be configured to receive one or more images of a scene captured using a Complementary Metal Oxide Semiconductor (CMOS) Image Sensor (CIS) camera. The receiving module 208 may also be configured to select an appropriate edge detection technique used for detecting edges in the one or more images, from an edge detection device 103 associated with the sampling system 105 (see
In an embodiment, the determining module 209 may be configured to determine a scene structure by dividing each of the one or more images. For example, the one or more images may be divided into a plurality of rectangular regions, based on an edge feature. For example, the rectangular regions may be of a size that is predetermined. For example, in some embodiments, the size of each of the plurality of rectangular regions may be 32×32 pixels for an RGB image of a scene.
In an embodiment, the computing module 210 may be configured to compute an edge region alignment for each of the plurality of rectangular regions. By analyzing a Histogram of oriented Gradients (HoG) distribution corresponding to each of the plurality of rectangular regions, the computing module 210 may compute an edge region alignment. Further, the edge region alignment values may be segregated into 4 bins, each bin corresponding, respectively, to a 0° alignment, a 45° alignment, a 90° alignment, and a 135° alignment.
In an embodiment, the re-projection module 211 may be configured to re-project ToF data on a CIS image plane according to the edge region alignment. The re-projection module 211 may also be configured to compute a regional depth variance after applying a primitive directional sampling filter to a scene.
In an embodiment, the sampling module 212 may be configured to sample one or more rectangular regions from the scene structure by comparing the regional depth variance with a threshold depth variance. The sampling module 212 may also be configured to identify one or more rectangular regions with two-Dimensional (2D) edges and one or more rectangular regions with three-Dimensional (3D) edges. Further, the 2D edges are eliminated when the regional depth variance is less than the threshold depth variance.
In an embodiment, the reconfiguring module 213 may be configured to dynamically reconfigure an illumination pattern for the spot ToF sensor image frame. The reconfiguration may be performed based on re-projected filtered samples for reconstructing a 3D model 107 of the scene.
As illustrated in
The order in which the method is described is not intended to be construed as a limitation, and any number of the described method blocks may be combined in any order to implement the method. Additionally, individual blocks may be deleted from the methods without departing from the spirit and scope of the claims and the subject matter described herein. Furthermore, the method may be implemented in any suitable hardware, software, firmware, or combination thereof.
At block 301, the method comprises receiving, by a sampling system 105, one or more images of a scene. In some embodiments, the one or more images may be captured using a Complementary Metal Oxide Semiconductor (CMOS) Image Sensor (CIS) camera. As an example, one or more images of a scene (e.g., a 3D scene) may be captured by the image capturing device 101 and obtained at the initial stage of the sampling process. The one or more images of the scene may vary based on whether the image is captured in an indoor environment or an outdoor area.
At block 303, the method comprises dividing, by the sampling system 105, each of the one or more images into a plurality of rectangular regions. For example, each of the one or more images may be divided into the plurality of rectangular regions to determine a scene structure. The rectangular regions may be of a predetermined size. For example, in some embodiments, the size of each of the plurality of rectangular regions may be 32×32 pixels which are based on an edge feature identified in the one or more images.
At block 305, the method comprises computing, by the sampling system 105, an edge region alignment for each of the plurality of rectangular regions by analyzing a Histogram of oriented Gradients (HoG) distribution corresponding to each of the plurality of rectangular regions. The edge region alignment may be computed using HoG values segregated into a plurality of bins, wherein the bins correspond, respectively, to a 0° alignment, a 45° alignment, a 90° alignment, and a 135° alignment.
At block 307, the method comprises re-projecting, by the sampling system 105, ToF data on a CIS image plane according to the edge region alignment and a directional sampling filter for computing a regional depth variance.
At block 309, the method comprises sampling, by the sampling system 105, one or more rectangular regions of the plurality of rectangular regions by comparing the regional depth variance with a threshold depth variance. For example, in some embodiments, the one or more rectangular regions of the plurality of rectangular regions may be sampled by comparing the regional depth variance of each of the plurality of rectangular regions with a threshold depth variance. Filtering 2D edges from the 3D edges may be performed based on the comparison between neighborhood depth variance and the threshold depth variance.
At block 311, the method comprises dynamically reconfiguring, by the sampling system 105, an illumination pattern for the spot ToF sensor image frame using sampled one or more rectangular regions for reconstructing a 3D model 107 of the scene with low-power active sampling. For example, reconfiguring the illumination pattern for the Spot ToF frame may be performed based on explicit scene structure (i.e., depth-discontinuities).
At block 401, the one or more images of the scene are projected with flood iToF on a CIS edge map. At block 403, the obtained edge map from a CIS sensor may be divided into a plurality of rectangular regions of size 32×32 pixels to determine the scene structure. At block 405, the method computes an edge region alignment based on HoG distribution with 4 bins with an alignment of, for example, 0° alignment, a 45° alignment, a 90° alignment, and a 135° alignment, respectively. At block 407, a primitive sampling filter is applied on the computed edge region alignment. For example, for the resulting sampled locations, a primitive/directional sampling filter with regard to horizontal, vertical, circular, and diagonal directions may be applied as shown in
As shown in
In an embodiment, the 3D reconstruction from Point Cloud Data (PCD) may be processed using an Open3D reference algorithm applied on sub-sampled PCDs. For example, consider Samsung's Azure Kinect dataset, where a sequence of frames consists of 472 RGB-Depth (RGBD) frames each of size 540×720 pixels, in a room containing monitors, cabinets, chairs, and boards. The Kinect Azure device supports the highest RGB resolution among other comparison cameras. The accuracy may be measured by measuring the inaccuracy along the X, Y, and Z axes and the mean latency of proposed blocks is 39.17 ms per frame (i.e., ˜1 frame delay @ 30 fps). The results on the Samsung Azure Kinect dataset are tabulated in Table 1 below.
In an embodiment, a dataset from Imperial College London and National University of Ireland Maynooth (ICL-NUIM) office dataset may be considered for evaluation of 3D reconstruction, where an “office room” sequence is 0 and consists of 1508 RGB-D frames (RGB-Depth) each of size 640×480 pixels. The ICL-NUIM dataset consists of handheld RGB-D camera sequences captured in synthetically produced situations. To accurately assess the correctness of a specific image, these sequences were shot in a living room and an office area with precise ground-truth positions. The mean latency of the proposed blocks is 36.97 ms per frame (i.e., ˜1 frame delay @ 30 fps). The results on ICL-NUIM Office 0 dataset are tabulated in Table 2 below.
In an embodiment, the final latency may be much lower with optimization and running on a hardware accelerator or ASIC. Hence, the spot-ToF sensing with high accuracy may be useful in applications such as augmented reality, mixed reality 3D gaming, 3D reconstruction and multi-view rendering of objects, measuring the physical world through image sensors on phones, and autonomous driving.
The processor 702 may be disposed in communication with one or more input/output (I/O) devices (711 and 712) via I/O interface 701. The I/O interface 701 may employ communication protocols/methods such as, without limitation, audio, analog, digital, stereo, IEEE-1394, serial bus, Universal Serial Bus (USB), infrared, PS/2, BNC, coaxial, component, composite, Digital Visual Interface (DVI), high-definition multimedia interface (HDMI), Radio Frequency (RF) antennas, S-Video, Video Graphics Array (VGA), IEEE 802.n/b/g/n/x, Bluetooth, cellular (e.g., Code-Division Multiple Access (CDMA), High-Speed Packet Access (HSPA+), Global System For Mobile Communications (GSM), Long-Term Evolution (LTE) or the like), etc. Using the I/O interface 701, the computer system 700 may communicate with one or more I/O devices 711 and 712. The computer system 700 may receive data from image capturing device 101 and edge detection device 103.
In some embodiments, the processor 702 may be disposed in communication with a communication network 709 via a network interface 703. The network interface 703 may communicate with the communication network 709. The network interface 703 may employ connection protocols including, without limitation, direct connect, Ethernet (e.g., twisted pair 10/100/1000 Base T), Transmission Control Protocol/Internet Protocol (TCP/IP), token ring, IEEE 802.11a/b/g/n/x, etc.
The communication network 709 may be implemented as one of the several types of networks, such as intranet or Local Area Network (LAN) and such within the organization. The communication network 709 may either be a dedicated network or a shared network, which represents an association of several types of networks that use a variety of protocols, for example, Hypertext Transfer Protocol (HTTP), Transmission Control Protocol/Internet Protocol (TCP/IP), Wireless Application Protocol (WAP), etc., to communicate with each other. Further, the communication network 709 may include a variety of network devices, including routers, bridges, servers, computing devices, storage devices, etc.
In some embodiments, the processor 702 may be disposed in communication with a memory 705 (e.g., RAM 713, ROM 714, etc. as shown in
The memory 705 may store a collection of program or database components, including, without limitation, user/application 706, an operating system 707, a web browser 708, mail client 715, mail server 716, web server 717 and the like. In some embodiments, computer system 700 may store user/application data 706, such as the data, variables, records, etc. as described in this invention. Such databases may be implemented as fault-tolerant, relational, scalable, secure databases such as Oracle® or Sybase®.
The operating system 707 may facilitate resource management and operation of the computer system 700. Examples of operating systems include, without limitation, APPLE MACINTOSH® OS X, UNIX®, UNIX-like system distributions (E.G., BERKELEY SOFTWARE DISTRIBUTION™ (BSD), FREEBSD™, NETBSD™, OPENBSD™, etc.), LINUX DISTRIBUTIONS' (E.G., RED HAT™, UBUNTU™, KUBUNTU™, etc.), IBM™ OS/2, MICROSOFT™ WINDOWS™ (XP™ VISTA™/7/8, 10 etc.), APPLE® IOS™ GOOGLE® ANDROID™, BLACKBERRY® OS, or the like. A user interface may facilitate display, execution, interaction, manipulation, or operation of program components through textual or graphical facilities. For example, user interfaces may provide computer interaction interface elements on a display system operatively connected to the computer system 400, such as cursors, icons, check boxes, menus, windows, widgets, etc. Graphical User Interfaces (GUIs) may be employed, including, without limitation, APPLE MACINTOSH® operating systems, IBM™ OS/2, MICROSOFT™ WINDOWS™ (XP™, VISTA™/7/8, 10 etc.), Unix® X-Windows, web interface libraries (e.g., AJAX™, DHTML™, ADOBE® FLASH™ JAVASCRIPT™, JAVA™, etc.), or the like.
Furthermore, one or more computer-readable storage media may be utilized in implementing embodiments consistent with the present disclosure. A computer-readable storage medium refers to any type of physical memory on which information or data readable by a processor may be stored. Thus, a computer-readable storage medium may store instructions for execution by one or more processors, including instructions for causing the processor(s) to perform steps or stages consistent with the embodiments described herein. The term “computer-readable medium” should be understood to include tangible items and exclude carrier waves and transient signals, i.e., non-transitory. Examples include Random Access Memory (RAM), Read-Only Memory (ROM), volatile memory, nonvolatile memory, hard drives, Compact Disc (CD) ROMs, Digital Video Disc (DVDs), flash drives, disks, and any other known physical storage media.
In an embodiment, the present disclosure provides a method and system for optimizing sampling in spot Time-of-Flight (ToF) sensor using CIS data.
In an embodiment, the present disclosure may operate with a low-power mobile device using primitive image features.
In an embodiment, the present disclosure may distinguish 2D edges from 3D edges and reconfigure the illumination pattern of a Spot-ToF sensor to sample near-depth discontinuities.
In an embodiment, the present disclosure may reduce total power consumption with higher frame rates and may not require prior training because the features of the dataset are not learned from the sample dataset.
In an embodiment, the present disclosure may make augmented reality and virtual reality consumer experiences more realistic and user friendly, which increases accessibility.
The terms “an embodiment”, “embodiment”, “embodiments”, “the embodiment”, “the embodiments”, “one or more embodiments”, “some embodiments”, and “one embodiment” mean “one or more (but not all) embodiments of the invention(s)” unless expressly specified otherwise.
The terms “including”, “comprising”, “having” and variations thereof mean “including but not limited to”, unless expressly specified otherwise.
The enumerated listing of items does not imply that any or all the items are mutually exclusive, unless expressly specified otherwise.
The terms “a”, “an” and “the” mean “one or more”, unless expressly specified otherwise.
A description of an embodiment with several components in communication with each other does not imply that all such components are required. On the contrary, a variety of optional components are described to illustrate the wide variety of possible embodiments consistent with the present disclosure.
When a single device or article is described herein, it will be clear that more than one device/article (whether they cooperate) may be used in place of a single device/article. Similarly, where more than one device or article is described herein (whether they cooperate), it will be clear that a single device/article may be used in place of the more than one device or article or a different number of devices/articles may be used instead of the shown number of devices or programs. The functionality and/or the features of a device may be alternatively embodied by one or more other devices which are not explicitly described as having such functionality/features. Thus, other embodiments of the invention need not include the device itself.
Finally, the language used in the specification has been principally selected for readability and instructional purposes, and it may not have been selected to delineate or circumscribe the subject matter. It is therefore intended that the scope of the present disclosure be limited not by this detailed description, but rather by any claims that issue on an application based here on. Accordingly, the embodiments are intended to be illustrative, but not limiting, of the scope, which is set forth in the following claims.
While various aspects and embodiments have been disclosed herein, other aspects and embodiments will be apparent to those skilled in the art. The various aspects and embodiments disclosed herein are for purposes of illustration and are not intended to be limiting, with the true scope and spirit being indicated by the following claims.
Number | Date | Country | Kind |
---|---|---|---|
202141054242 | Nov 2021 | IN | national |