Patent Grant
11635486
The invention relates to a detector, a detector system and a method for determining a position of at least one object. The invention further relates to a human-machine interface for exchanging at least one item of information between a user and a machine, an entertainment device, a tracking system, a camera, a scanning system, a readout device for optical storage media and various uses of the detector device. The devices, systems, methods and uses according to the present invention specifically may be employed for example in various areas of daily life, gaming, traffic technology, production technology, security technology, photography such as digital photography or video photography for arts, documentation or technical purposes, medical technology or in the sciences. Further, the invention specifically may be used for scanning one or more objects and/or for scanning a scenery, such as for generating a depth profile of an object or of a scenery, e.g., in the field of architecture, metrology, archaeology, arts, medicine, engineering or manufacturing. However, other applications are also possible.
A large number of optical sensors and photovoltaic devices are known from the prior art. While photovoltaic devices are generally used to convert electromagnetic radiation, for example, ultraviolet, visible or infrared light, into electrical signals or electrical energy, optical detectors are generally used for picking up image information and/or for detecting at least one optical parameter, for example, a brightness.
A large number of optical sensors which can be based generally on the use of inorganic and/or organic sensor materials are known from the prior art. Examples of such sensors are disclosed in US 2007/0176165 A1, U.S. Pat. No. 6,995,445 B2, DE 2501124 A1, DE 3225372 A1 or else in numerous other prior art documents. To an increasing extent, in particular for cost reasons and for reasons of large-area processing, sensors comprising at least one organic sensor material are being used, as described for example in US 2007/0176165 A1. In particular, so-called dye solar cells are increasingly of importance here, which are described generally, for example in WO 2009/013282 A1.
A large number of detectors for detecting at least one object are known on the basis of such optical sensors. Such detectors can be embodied in diverse ways, depending on the respective purpose of use. Examples of such detectors are imaging devices, for example, cameras and/or microscopes. High-resolution confocal microscopes are known, for example, which can be used in particular in the field of medical technology and biology in order to examine biological samples with high optical resolution. Further examples of detectors for optically detecting at least one object are distance measuring devices based, for example, on propagation time methods of corresponding optical signals, for example laser pulses. Further examples of detectors for optically detecting objects are triangulation systems, by means of which distance measurements can likewise be carried out. Kurt Konolige et al., A Low-Cost Laser Distance Sensor, 2008 IEEE International Conference on Robotics and Automation, Pasadena, Calif., USA, May 19-23, 2008, discuss competing technologies using triangulation for planar laser distance sensor (LDS). Structured line devices use a light stripe laser and offset camera to determine range to a set of points. Because the laser energy is spread over a line, it is difficult to achieve accurate range, especially in the presence of ambient light, or with darker objects. Point scan devices for 3D scanning of small objects typically use a scanning mirror to direct a point laser beam and redirect the laser return to an optical receiver. Such devices cannot be miniaturized, and their cost and mechanical fragility will remain high. Centroid point modules typically use position-sensitive devices (PSD). These devices measure the centroid of all light impinging on their surface. Although modulation techniques can be used to offset some of the effects of ambient light, PSDs do not perform well unless the laser spot has a very strong reflection, limiting their use to ranges of a meter or less. Pixel-based point modules search the pixel with maximum signal intensity to determine the position of the light spot on the sensor. Typically CMOS line arrays are used for detection. Konolige et al. introduce a low-cost version of a Pixel-based point module.
U.S. Pat. No. 5,235,377 A describes a measuring apparatus for a camera, wherein light is projected to an object from a projection section. Light reflected on the object is received at a first light receiving section and a second light receiving section arranged to be adjacent to each other. In the first light receiving section, a first and second signals relating to a light receiving position and a quantity of received light are output, and in the second light receiving section, a third signal relating to a quantity of light of the reflected light is output. In a discriminating and calculating section, if it is discriminated that the reflected light is positioned at a place, which is nearer than a predetermined distance, a measuring distance can be calculated from an output ratio of the first signal to the second signal by a measuring calculation circuit. On the other hand, if it is discriminated that the reflected light is positioned at a place, which is farther than the predetermined distance, a measuring distance can be calculated from the output ratio of the sum of the first and second signals to the third signal by the measuring calculation circuit.
U.S. Pat. No. 5,512,997 A describes a distance measuring device having an infrared light-emitting diode by which an infrared light is emitted towards a subject, and first and second linear line sensors on which an image corresponding to light reflected from the subject is formed. The first and second linear line sensors are disposed adjacently and in parallel to each other, and offset by an amount corresponding to half of the predetermined A/D converting step. Analog distance data outputted from the linear line sensors are converted to digital distance data using the predetermined A/D converting step, and then, the digital distance data are added together to obtain a subject distance with a high accuracy.
DE 42 11 875 A1 describes an optical rangefinder with electronic correction for spot eccentricity. The optical rangefinder has an evaluation circuit, which determines if an output distance measurement value is valid or erroneous due to e.g., inclination of object surface. The distance signal is determined by evaluation of a position of a light spot on a first detector. A second control signal, which characterizes inclination of the object surface, is determined by evaluating eccentricity of a light spot on a second detector, arranged in a different plane.
WO 2016/051323 A1 describes a detector and a method for optically determining a position of at least one object. The detector comprises at least one optical sensor for determining a position of at least one light beam and at least one evaluation device for generating at least one item of information on a transversal position of the object and at least one item of information on a longitudinal position of the object. The sensor has at least a first electrode and a second electrode. At least one photovoltaic material is embedded in between the first electrode and the second electrode. The first electrode or the second electrode is a split electrode having at least three partial electrodes. The detector and the method can determine three-dimensional coordinates of an object in a fast and efficient way.
In WO 2012/110924 A1, the content of which is herewith included by reference, a detector for optically detecting at least one object is proposed. The detector comprises at least one optical sensor. The optical sensor has at least one sensor region. The optical sensor is designed to generate at least one sensor signal in a manner dependent on an illumination of the sensor region. The sensor signal, given the same total power of the illumination, is dependent on a geometry of the illumination, in particular on a beam cross section of the illumination on the sensor area. The detector furthermore has at least one evaluation device. The evaluation device is designed to generate at least one item of geometrical information from the sensor signal, in particular at least one item of geometrical information about the illumination and/or the object.
WO 2014/097181 A1, the full content of which is herewith included by reference, discloses a method and a detector for determining a position of at least one object, by using at least one transversal optical sensor and at least one optical sensor. Specifically, the use of sensor stacks is disclosed, in order to determine a longitudinal position of the object with a high degree of accuracy and without ambiguity.
WO 2015/024871 A1, the full content of which is herewith included by reference, discloses an optical detector, comprising:
U.S. Pat. No. 4,767,211 discloses an apparatus for and a method of measuring a boundary surface of a sample in which a ratio of the light quantity of a part of reflected light from a sample which travels in the vicinity of the optical axis of the reflected light, to the light quantity of another part of the reflected light which is directed to a position deviating from the optical axis by a predetermined distance is used to accurately measure a boundary surface of a sample. Since the accuracy of measurement is increased by using the above ratio, light capable of passing through the sample can be used as incident light. Thus, a deep hole in the surface of the sample and a void such as an air bubble in a living being sample, which cannot be measured by the prior art, can be measured very accurately.
WO 2014/198629 A1, the full content of which is herewith included by reference, discloses a detector for determining a position of at least one object, comprising:
Further, generally, for various other detector concepts, reference may be made to WO 2014/198626 A1, WO 2014/198629 A1 and WO 2014/198625 A1, the full content of which is herewith included by reference. Further, referring to potential materials and optical sensors which may also be employed in the context of the present invention, reference may be made to European patent applications EP 15 153 215.7, filed on Jan. 30, 2015, EP 15 157 363.1, filed on Mar. 3, 2015, EP 15 164 653.6, filed on Apr. 22, 2015, EP 15177275.3, filed on Jul. 17, 2015, EP 15180354.1 and EP 15180353.3, both filed on Aug. 10, 2015, and EP 15 185 005.4, filed on Sep. 14, 2015, EP 15 196 238.8 and EP 15 196 239.6, both filed on Nov. 25, 2015, EP 15 197 744.4, filed on Dec. 3, 2015, the full content of all of which is herewith also included by reference.
Further, reference may be made to detector concepts comparing signals of at least two different sources for determining a position of an object. Thus, as an example, reference may be made to EP 16155834.1, EP 16155835.8 or EP 16155845.7, all filed on Feb. 16, 2016, the full disclosure of which is herewith included by reference.
U.S. Pat. No. 5,235,377 A describes a measuring apparatus for a camera. Light is projected to an object from a projection section, and light reflected on the object is received at a first light receiving section and a second light receiving section arranged to be adjacent to each other. In the first light receiving section, a first and second signals relating to a light receiving position and a quantity of received light are output, and in the second light receiving section, a third signal relating to a quantity of light of the reflected light is output. In a discriminating and calculating section, if it is discriminated that the reflected light is positioned at a place, which is nearer than a predetermined distance, a measuring distance can be calculated from an output ratio of the first signal to the second signal by a measuring calculation circuit. On the other hand, if it is discriminated that the reflected light is positioned at a place, which is farther than the predetermined distance, a measuring distance can be calculated from the output ratio of the sum of the first and second signals to the third signal by the measuring calculation circuit.
U.S. Pat. No. 5,512,997 A describes a distance measuring device having an infrared light-emitting diode by which an infrared light is emitted towards a subject, and first and second linear line sensors on which an image corresponding to light reflected from the subject is formed. The first and second linear line sensors are disposed adjacently and in parallel to each other, and offset by an amount corresponding to half of the predetermined A/D converting step. Analog distance data outputted from the linear line sensors are converted to digital distance data using the predetermined A/D converting step, and then, the digital distance data are added together.
DE 42 11 875 A1 describes that a beam from e.g., a collimated laser diode passes through crossed cylinder lenses which focus the rays in both sectional planes and adapt the diameter of the beam to the numerical aperture of an objective lens. A mirror section cuts off half of the incident beam and deflects light scattered from the object to a differential photodiode detector. A beam splitter reflects part of the deflected beam on to a four-quadrant photodetector in an orthogonal plane. A first electronic circuit produces a signal corresponding to the position of the light spot on the detector. A second circuit produces a monitoring or correction signal derived from the eccentricity of the spot position.
U.S. Pat. No. 4,767,211 A describes an apparatus for and a method of measuring a boundary surface of a sample are disclosed in which a ratio of the light quantity of a part of reflected light from a sample which travels in the vicinity of the optical axis of the reflected light, to the light quantity of another part of the reflected light which is directed to a position deviating from the optical axis by a predetermined distance is used to measure a boundary surface of a sample.
Despite the advantages implied by the above-mentioned devices and detectors, several technical challenges remain. Thus, generally, a need exists for detectors for detecting a position of an object in space which is both reliable and may be manufactured at low cost. Specifically, a need exists for 3D-sensing concepts. Various known concepts are at least partially based on using so-called Focus-induced Photoresponse (FiP) sensors, such as several of the above-mentioned concepts. Therein, as an example, large area sensors may be used, in which the individual sensor pixels are significantly larger than the light spot and which are fixed to a specific size. Still, large area sensors in many cases are inherently limited in the use of the FiP measurement principle, specifically in case more than one light spot is to be investigated simultaneously.
A further challenge using FiP detectors is detector area or active area. Typically, for distance measurements, a large active area of the detector is used or even is required. This area, however, may cause noise problems, specifically when the tetralateral conductivity concept is employed to build a position-sensitive detector (PSD). This often results in poor signal-to-noise-ratios and slow detector response times due to the large capacitance in conjunction with the series resistance of the detector. A further challenge using FiP detectors is cost of manufacturing. Thus, in many cases, typical FiP sensors are expensive, as compared to e.g., conventional Si-based photodiodes. Further, the evaluation of the measurement results of measurements using FiP-sensors remains an issue, specifically in case the total power of a light beam is unknown. In case a plurality of FiP-sensors is used, located at different positions along an axis of propagation of the light beam, a range of measurement typically is limited to the range in between the two positions of the sensors. Further, many FiP-detectors show a luminance dependency which renders the evaluation of the measurement result more difficult, and, additionally, FiP-measurements in many cases are dependent on a target spot size.
Referring to concepts comparing sensor signals generated by at least two different sources such as at least two different optical sensors, one technical challenge remains a reduction of cost of the overall system and, specifically, of the optical sensors. Thus, in many concepts, special optical sensors have to be used, having a dedicated design and placement, which typically requires setting up an expensive optoelectronic semiconductor manufacturing process and a complex assembly scheme.
It is therefore an object of the present invention to provide devices and methods facing the above-mentioned technical challenges of known devices and methods. Specifically, it is an object of the present invention to provide cost-efficient devices and methods which reliably may determine a position of an object in space, preferably with a low technical effort and with low requirements in terms of technical resources and cost.
This problem is solved by the invention with the features of the independent patent claims. Advantageous developments of the invention, which can be realized individually or in combination, are presented in the dependent claims and/or in the following specification and detailed embodiments.
As used in the following, the terms “have”, “comprise” or “include” or any arbitrary grammatical variations thereof are used in a non-exclusive way. Thus, these terms may both refer to a situation in which, besides the feature introduced by these terms, no further features are present in the entity described in this context and to a situation in which one or more further features are present. As an example, the expressions “A has B”, “A comprises B” and “A includes B” may both refer to a situation in which, besides B, no other element is present in A (i.e., a situation in which A solely and exclusively consists of B) and to a situation in which, besides B, one or more further elements are present in entity A, such as element C, elements C and D or even further elements.
Further, it shall be noted that the terms “at least one”, “one or more” or similar expressions indicating that a feature or element may be present once or more than once typically will be used only once when introducing the respective feature or element. In the following, in most cases, when referring to the respective feature or element, the expressions “at least one” or “one or more” will not be repeated, non-withstanding the fact that the respective feature or element may be present once or more than once.
Further, as used in the following, the terms “preferably”, “more preferably”, “particularly”, “more particularly”, “specifically”, “more specifically” or similar terms are used in conjunction with optional features, without restricting alternative possibilities. Thus, features introduced by these terms are optional features and are not intended to restrict the scope of the claims in any way. The invention may, as the skilled person will recognize, be performed by using alternative features. Similarly, features introduced by “in an embodiment of the invention” or similar expressions are intended to be optional features, without any restriction regarding alternative embodiments of the invention, without any restrictions regarding the scope of the invention and without any restriction regarding the possibility of combining the features introduced in such a way with other optional or non-optional features of the invention.
In a first aspect of the present invention, a detector for determining a position of at least one object is disclosed. As used herein, the term “position” refers to at least one item of information regarding a location and/or orientation of the object and/or at least one part of the object in space. Thus, the at least one item of information may imply at least one distance between at least one point of the object and the at least one detector. As will be outlined in further detail below, the distance may be a longitudinal coordinate or may contribute to determining a longitudinal coordinate of the point of the object. Additionally or alternatively, one or more other items of information regarding the location and/or orientation of the object and/or at least one part of the object may be determined. As an example, additionally, at least one transversal coordinate of the object and/or at least one part of the object may be determined. Thus, the position of the object may imply at least one longitudinal coordinate of the object and/or at least one part of the object. Additionally or alternatively, the position of the object may imply at least one transversal coordinate of the object and/or at least one part of the object. Additionally or alternatively, the position of the object may imply at least one orientation information of the object, indicating an orientation of the object in space.
The detector comprises:
As used herein, an “optical sensor” generally refers to a light-sensitive device for detecting a light beam, such as for detecting an illumination and/or a light spot generated by at least one light beam. As further used herein, a “light-sensitive area” generally refers to an area of the optical sensor which may be illuminated externally, by the at least one light beam, in response to which illumination the at least one sensor signal is generated. The light-sensitive area may specifically be located on a surface of the respective optical sensor. Other embodiments, however, are feasible. As used herein, the term “at least two optical sensors, each having at least one light sensitive area” refers to configurations with two single optical sensors each having one light sensitive area and to configurations with one combined optical sensor having at least two light sensitive areas. Thus, the term “optical sensor” furthermore refers to a light-sensitive device configured to generate one output signal, whereas, herein, a light-sensitive device configured to generate two or more output signals, for example at least one bi-cell and/or quadrant diode device, is referred to as two or more optical sensors. As will further be outlined in detail below, each optical sensor, may be embodied such that precisely one light-sensitive area is present in the respective optical sensor, such as by providing precisely one light-sensitive area which may be illuminated, in response to which illumination precisely one uniform sensor signal is created for the whole optical sensor. Thus, each optical sensor may be a single area optical sensor. Other embodiments, however, are feasible. The use of the single area optical sensors, however, renders the setup of the detector specifically simple and efficient. Thus, as an example, commercially available photosensors, such as commercially available silicon photodiodes, each having precisely one sensitive area, may be used in the setup.
As further used herein, a “sensor signal” generally refers to a single signal or a plurality of sensor signals such as a series of sensor signals or a continuous sensor signal, generated by an optical sensor in response to the illumination by the light beam. Specifically, the sensor signal may be or may comprise at least one electrical signal, such as at least one analogue electrical signal and/or at least one digital electrical signal. More specifically, the sensor signal may be or may comprise at least one voltage signal and/or at least one current signal. More specifically, the sensor signal may comprise at least one photocurrent. Further, either raw sensor signals may be used, or the detector, the optical sensor or any other element may be adapted to process or preprocess the sensor signal, thereby generating secondary sensor signals, which may also be used as sensor signals, such as preprocessing by filtering or the like.
The light-sensitive areas specifically may be oriented towards the object. As used herein, the term “is oriented towards the object” generally refers to the situation that the respective surfaces of the light-sensitive areas are fully or partially visible from the object. Specifically, at least one interconnecting line between at least one point of the object and at least one point of the respective light-sensitive area may form an angle with a surface element of the light-sensitive area which is different from 0°, such as an angle in the range of 20° to 90°, preferably 80 to 90° such as 90°. Thus, when the object is located on the optical axis or close to the optical axis, the light beam propagating from the object towards the detector may be essentially parallel to the optical axis. As used herein, the term “essentially perpendicular” refers to the condition of a perpendicular orientation, with a tolerance of e.g., ±20° or less, preferably a tolerance of ±10° or less, more preferably a tolerance of ±5° or less. Similarly, the term “essentially parallel” refers to the condition of a parallel orientation, with a tolerance of e.g., ±20° or less, preferably a tolerance of ±10° or less, more preferably a tolerance of ±5° or less.
The light beam may propagate from the object towards the detector. As will be outlined in further detail below, the light beam may originate from the object, such as by the object and/or at least one illumination source integrated or attached to the object emitting the light beam, or may originate from a different illumination source, such as from an illumination source directly or indirectly illuminating the object, wherein the light beam is reflected or scattered by the object and, thereby, is at least partially directed towards the detector. The illumination source, as an example, may be or may comprise one or more of an external illumination source, an illumination source integrated into the detector or an illumination source integrated into a beacon device being one or more of attached to the object, integrated into the object or held by the object.
As further used herein, the term “light beam” generally refers to an amount of light, specifically an amount of light traveling essentially in the same direction, including the possibility of the light beam having a spreading angle or widening angle. The light beam specifically may be a Gaussian light beam, as will be outlined in further detail below. Other embodiments are feasible, however.
The optical sensors may be sensitive in one or more of the ultraviolet, the visible or the infrared spectral range. Specifically, the optical sensors each, independently, may be or may comprise at least one element selected from the group consisting of a photodiode, a photocell, a photoconductor, a phototransistor or any combination thereof. Any other type of photosensitive element may be used. As will be outlined in further detail below, the photosensitive element generally may fully or partially be made of inorganic materials and/or may fully or partially be made of organic materials. Most commonly, as will be outlined in further detail below, one or more photodiodes may be used, such as commercially available photodiodes, e.g., inorganic semiconductor photodiodes.
The light-sensitive areas of the optical sensors may overlap, as visible from the object, or may not overlap, i.e., may be placed next to each other without overlap. The light-sensitive areas may be spaced apart from each other or may directly be adjacent.
The detector has an optical axis. As used herein, the term “optical axis” generally refers to a line of symmetry of the optical setup of the detector. In case the detector has a lens or lens system, such as a lens or lens system being part of a transfer system which will be explained in further detail below, the optical axis, as an example, may be an axis of mirror symmetry or rotational symmetry of the lens or lens system. Thus, as will be outlined in further detail below, the detector may comprise at least one transfer system, preferably at least one transfer system having at least one lens. The transfer system, as an example, may comprise at least one beam path, with the elements of the transfer system in the beam path being located in a rotationally symmetrical fashion with respect to the optical axis. Still, as will also be outlined in further detail below, one or more optical elements located within the beam path may also be off-centered or tilted with respect to the optical axis. In this case, however, the optical axis may be defined sequentially, such as by interconnecting the centers of the optical elements in the beam path, e.g., by interconnecting the centers of the lenses, wherein, in this context, the optical sensors are not counted as optical elements. The optical axis generally may denote the beam path. Therein, the detector may have a single beam path along which a light beam may travel from the object to the optical sensors, or may have a plurality of beam paths. As an example, a single beam path may be given or the beam path may be split into two or more partial beam paths. In the latter case, each partial beam path may have its own optical axis, and the condition noted above generally may refer to each beam path independently. The optical sensors may be located in one and the same beam path or partial beam path. Alternatively, however, the optical sensors may also be located in different partial beam paths. In case the optical sensors are distributed over different partial beam paths, the above-mentioned condition may be described such that at least one first optical sensor is located in at least one first partial beam path, being offset from the optical axis of the first partial beam path by a first spatial offset, and at least one second optical sensor is located in at least one second partial beam path, being offset from the optical axis of the second partial beam path by at least one second spatial offset, wherein the first spatial offset and the second spatial offset are different.
The detector may comprise more than two optical sensors. In any case, i.e., in the case of the detector comprising precisely two optical sensors and in the case of the detector comprising more than two optical sensors, the optical sensors may comprise at least one first optical sensor being spaced apart from the optical axis by a first spatial offset and at least one second optical sensor being spaced apart from the optical axis by a second spatial offset, wherein the first spatial offset and the second spatial offset differ. In case further optical sensors are provided, besides the first and second optical sensors, these additional optical sensors may also fulfill the condition or, alternatively, may be spaced apart from the optical axis by the first spatial offset, by the second spatial offset or by a different spatial offset. The first and second spatial offsets, as an example, may differ by at least a factor of 1.2, more preferably by at least a factor of 1.5, more preferably by at least a factor of two. The spatial offsets may also be zero or may assume negative values, as long as the above-mentioned conditions are fulfilled.
As outlined above, each light-sensitive area has a geometrical center. As used herein, the term “geometrical center” of an area generally may refer to a center of gravity of the area. As an example, if an arbitrary point inside or outside the area is chosen, and if an integral is formed over the vectors interconnecting this arbitrary point with each and every point of the area, the integral is a function of the position of the arbitrary point. When the arbitrary point is located in the geometrical center of the area, the absolute value of the integral is minimized. Thus, in other words, the geometrical center may be a point inside or outside the area with a minimum overall or sum distance from all points of the area.
Each geometrical center of each light-sensitive area is spaced apart from the optical axis of the detector, such as the optical axis of the beam path or the respective beam path in which the respective optical sensor is located. The distance between the geometrical center and the optical axis is denoted by the term “spatial offset”.
As outlined above, the optical sensors specifically may be located in one and the same plane, which, preferably, is a plane perpendicular to the optical axis. Other configurations, however, are possible. Thus, two or more of the optical sensors may also be spaced apart in a direction parallel to the optical axis.
In a preferred embodiment, which is rather simple and cost-efficiently realizable, the optical sensors are partial diodes of a segmented diode, with a center of the segmented diode being off-centered from the optical axis of the detector. As used herein, the term “partial diode” may comprise several diodes that are connected in series or in parallel. This example is rather simple and cost-efficiently realizable. Thus, as an example, bi-cell diodes or quadrant diodes are widely commercially available at low cost, and driving schemes for these quadrant diodes are generally known. As used herein, the term “bi-cell diode” generally refers to a diode having two partial diodes in one packaging. Bi-cell and quadrant diodes may have two or four separate light sensitive areas, in particular two or four active areas. As an example, the bi-cell diodes may each form independent diodes having the full functionality of a diode. As an example, each of the bi-cell diodes may have a square or rectangular shape, and the two diodes may be placed in one plane such that the two partial diodes, in total, form a 1×2 or 2×1 matrix having a rectangular shape. In the present invention, however, a new scheme for evaluating the sensor signals of the bi-cell diodes or quadrant diode is proposed, as will be outlined in further detail below. Generally, however, the optical sensors specifically may be partial diodes of a quadrant diode, with a center of the quadrant diode being off-centered from the optical axis of the detector. As used herein, the term “quadrant diode” generally refers to a diode having four partial diodes in one packaging. As an example, the four partial diodes may each form independent diodes having the full functionality of a diode. As an example, the four partial diodes may each have a square or rectangular shape, and the four partial diodes may be placed in one plane such that the four partial diodes, in total, form a 2×2 matrix having a rectangular or square shape. In a further example, the four partial diodes, in total, may form a 2×2 matrix having a circular or elliptical shape. The partial diodes, as an example, may be adjacent, with a minimum separation from one another. However, the separation between adjacent diodes may also be varied to improve the dynamic range of the quotient signal. As an example, the separation between two adjacent diodes may be increased by using an opaque mask. This may reduce the light on one of the diodes relative and/or absolute as compared to the adjacent diode which may increase a quotient of the signal of the two diodes. The rows and columns of the matrix specifically may be arranged in a rectangular fashion. It shall be outlined, however, that other arrangements are feasible, such as nonrectangular arrangements. As an example, circular arrangements are also feasible, wherein the partial diodes are arranged in concentric circles or ellipses about a center point, hexagonal arrangements, triangular arrangements, polygonial arrangements or combinations thereof are also feasible, wherein the partial diodes are arranged as regular or irregular polygons such as in hexagons or the like.
Further, arrangements may be feasible, wherein at least one of the partial diodes of a polygonial pattern may be arranged as star shaped element in a triangular pattern and/or striped element in a rectangular pattern, or the like. Further, arrangements may be feasible, wherein the partial diodes of a polygonial pattern are summarized to a larger, unseparated partial diode, such as star shaped partial diodes in a triangular pattern, striped partial diodes in a rectangular pattern, or the like. Further, separated partial diodes may be summarized or connected to a single partial diode such as the corner partial diodes forming a first partial diode and the non-corner partial diodes forming a second partial diode. Further, for example, the matrix may be a single row of partial diodes. Further, arrangements, where the optical sensors and/or at least one of partial diodes may have a star shape and/or circular shape, and/or cross shape may be feasible. For example, at least two of the partial diodes may be arranged in star shaped manner or cross shaped manner, whereas the other partial diodes may be arranged as such that they complete a square or rectangular shape of the optical sensor, so that a square shaped or rectangular shaped substrate can be employed. Other arrangements are feasible.
In case a quadrant diode is used, having a 2×2 matrix of partial diodes, the center of the quadrant diode specifically may be off-centered or offset from the optical axis. Thus, as an example, the center of the quadrant diodes, which may be an intersection of the geometrical centers of the optical sensors of the quadrant diode, may be off-centered from the optical axis by at least 0.2 mm, more preferably by at least 0.5 mm, more preferably by at least 1.0 mm or even 2.0 mm. Similarly, when using other types of optical sensors setups having a plurality of optical sensors, an overall center of the optical sensors may be offset from the optical axis by the same distance.
As outlined above, bi-cell and quadrant diodes may have two or four separate light sensitive areas, in particular two or four active areas. The light sensitive areas may be separated by a dividing line. The dividing line, for example, of two Si-diodes, can be arranged parallel to the baseline of the detector, in particular parallel to a spot movement on the optical sensors. However, other arrangements are feasible. For example, the quadrant diode may comprise two dividing lines. The dividing lines may be arranged orthogonal to each other. The orthogonal arrangement of the dividing lines allows adjusting of the quotient signal for near field and far field applications independently from each other. In addition to determining the quotient signal of sensor signals of two optical sensors of the quadrant diode, the evaluation device may be adapted to determine a second quotient using at least three or all four sensor signals of the quadrant diode. The two quotients can be formed such that two distinct distance ranges are covered. The two quotient signals for the near field and far field may have an overlap region in which both quotient signals allow obtaining reasonable determination of the longitudinal distance z. For example, the quotient signal may be determined by dividing the sensor signals of two top quadrants, also called top segment, by the sensor signals of two bottom quadrants, also called bottom segment. Using the quotient signal of sensor signals determined by two sensor areas which have a dividing line parallel to the baseline of the detector may allow determining of the quotient without any distance dependent movement of the light spot. In particular, as an example, if the dividing line between the top and bottom segment is parallel to the baseline, the quotient signal determined from top segment divided by bottom segment may be used in the near field, wherein the light spot may illuminate only one of a left or right segment of the quadrant diode. In this case determining the quotient signal by dividing sensor signals of the left and right segments may not be possible. However, determining the quotient by dividing the sensor signals of top and bottom segments may provide a reasonable distance measurement. The quotient signal determined by dividing sensor signals of the left and right segments may be used for far field measurement, wherein the light spot illuminates both left and right segments. Furthermore, the evaluation device may be adapted to determine the quotient signal by dividing sensor signals of opposing segments or neighboring segments. The evaluation device may be adapted to combine the acquired sensor signals of the quadrants such that distance measurement is possible over a wide range with a large resolution.
The detector may comprise at least one quadrant diode comprising four quadrants. For example, an upper left quadrant of the quadrant diode may be denoted “ol”, a lower left quadrant “ul”, an upper right quadrant may be denoted “or” and a lower right quadrant may be denoted “ur”. The light spot may move from the left in a small distance range to the left in a far distance range. The detector may comprise at least one neutral density filter. The quadrants of the small distance range may be covered by the neutral density filter. In the small distance range the quotient ol/ul may be used to determine the longitudinal coordinate z. In the far distance range a quotient or/ur and/or a quotient from a combination of all quadrants may be used. The neutral density filter allows that objects in the small distance range are adapted to illuminate the optical sensors brighter by constant amplification without saturating the amplifier. At the same time it is possible to obtain in the far distance range more of the signal. Thus, the neutral density filter allows to enhance dynamic of the detector.
Generally, the light-sensitive areas of the optical sensors may have an arbitrary surface area or size. Preferably, however, specifically in view of a simplified evaluation of the sensor signals, the light-sensitive areas of the optical sensors are substantially equal, such as within a tolerance of less than 10%, preferably less than 5% or even less than 1%. This, specifically, is the case in typical commercially available quadrant diodes.
As further used herein, the term evaluation device generally refers to an arbitrary device adapted to perform the named operations, preferably by using at least one data processing device and, more preferably, by using at least one processor and/or at least one application-specific integrated circuit. Thus, as an example, the at least one evaluation device may comprise at least one data processing device having a software code stored thereon comprising a number of computer commands. The evaluation device may provide one or more hardware elements for performing one or more of the named operations and/or may provide one or more processors with software running thereon for performing one or more of the named operations.
Specifically, as will be outlined in further detail below, the evaluation device may be configured to determine the at least one longitudinal coordinate z of the object by using at least one known, determinable or predetermined relationship between sensor signals and/or any secondary signal derived thereof and the longitudinal coordinate. Thus, the evaluation device may be configured for determining at least one combined sensor signal out of the at least two sensor signals, i.e., of the at least one sensor signal of at least one first optical sensor and out of the at least one sensor signal of at least one second optical sensor. As generally used herein, the term “combine” generally may refer to an arbitrary operation in which two or more components such as signals are one or more of mathematically merged in order to form at least one merged combined signal and/or compared in order to form at least one comparison signal or comparison result. As will be outlined in further detail below, the combined sensor signal or secondary signal may be or may comprise at least one quotient signal.
The evaluation device specifically may be configured for deriving at least one quotient signal Q by one or more of: dividing the sensor signals, dividing multiples of the sensor signals or dividing linear combinations of the sensor signals. The evaluation device may further be configured for determining the longitudinal coordinate by evaluating the quotient signal Q. In case more than two sensor signals are provided, as an example, more than one quotient signal may be generated, such as by forming quotient signals of more than one pair of sensor signals.
As an example, Q may simply be determined as
Q=s1/s2
or
Q=s2/s1,
with s1 denoting a first one of the sensor signals and s2 denoting a second one of the sensor signals. Additionally or alternatively, Q may be determined as
Q=a·s1/b·s2
or
Q=b·s2/a·s1,
with a and b being real numbers which, as an example, may be predetermined or determinable. Additionally or alternatively, Q may be determined as
Q=(a·s1+b·s2)/(c·s1+d·s2),
with a, b, c and d being real numbers which, as an example, may be predetermined or determinable. As a simple example for the latter, Q may be determined as
Q=s1/(s1+s2),
or, as a further example, Q may be determined as
Q=(s1−s2)/(s1+s2).
Other quotient signals are feasible. Thus, as an example, in case more than two optical sensors are provided, the above-mentioned quotient formation may take place between two of the sensor signals generated by these optical sensors or may take place between more than two of the sensor signals. Thus, instead of using the first one of the sensor signals and the second one of the sensor signals in the formulae given above, combined sensor signals may be used for quotient formation.
The quotient signal Q generally is an example for an asymmetry parameter denoting an asymmetry or an asymmetric distribution of the light spot generated by the light beam on the light-sensitive areas. The quotient of the two or more optical sensors, such as the two or more photodiodes, may provide a combined signal which typically is monotonically dependent on the distance between the detector and the object from which the light beam travels towards the detector, as will be shown by experimental data below. In addition or as an alternative to the quotient signal, other types of combined functions implementing the sensor signals of two or more sensors in the setup of the present invention may be used, which also may show a dependency on the distance between the object and the detector. The asymmetry or the asymmetry parameter of the light spot, as an example, may be an indication of the width of a light beam. If this asymmetry parameter depends on the distance only, the measurement can be used to determine the distance.
The evaluation device may be configured for deriving the quotient signal Q by one or more of dividing the sensor signals, dividing multiples of the sensor signals, dividing linear combinations of the sensor signals. The evaluation device may be configured for using at least one predetermined relationship between the quotient signal Q and the longitudinal coordinate for determining the longitudinal coordinate. For example, the evaluation device is configured for deriving the quotient signal Q by
wherein x and y are transversal coordinates, A1 and A2 are areas of the beam profile at the sensor position, and E(x,y,zo) denotes the beam profile given at the object distance zo. Area A1 and area A2 may differ. In particular, A1 and A2 are not congruent. Thus, A1 and A2 may differ in one or more of the shape or content. As used herein, the term “beam profile” relates to a spatial distribution, in particular in at least one plane perpendicular to the propagation of the light beam, of an intensity of the light beam. The beam profile may be a cross section of the light beam. The beam profile may be selected from the group consisting of a trapezoid beam profile; a triangle beam profile; a conical beam profile and a linear combination of Gaussian beam profiles. Generally the beam profile is dependent on luminance L(zo) and beam shape S(x,y;zo), E(x,y;zo)=L·S.
Each of the sensor signals may comprise at least one information of at least one area of the beam profile of the light beam. As used herein, the term “area of the beam profile” generally refers to an arbitrary region of the beam profile at the sensor position used for determining the quotient signal Q. The light-sensitive regions may be arranged such that a first sensor signal comprises information of a first area of the beam profile and a second sensor signal comprises information of a second area of the beam profile. The first area of the beam profile and second area of the beam profile may be one or both of adjacent or overlapping regions. The first area of the beam profile and the second area of the beam profile may be not congruent in area.
The evaluation device may be configured to determine and/or to select the first area of the beam profile and the second area of the beam profile. The first area of the beam profile may comprise essentially edge information of the beam profile and the second area of the beam profile may comprise essentially center information of the beam profile. The beam profile may have a center, i.e., a maximum value of the beam profile and/or a center point of a plateau of the beam profile and/or a geometrical center of the light spot, and falling edges extending from the center. The second region may comprise inner regions of the cross section and the first region may comprise outer regions of the cross section. As used herein, the term “essentially center information” generally refers to a low proportion of edge information, i.e., proportion of the intensity distribution corresponding to edges, compared to a proportion of the center information, i.e., proportion of the intensity distribution corresponding to the center. Preferably the center information has a proportion of edge information of less than 10%, more preferably of less than 5%, most preferably the center information comprises no edge content. As used herein, the term “essentially edge information” generally refers to a low proportion of center information compared to a proportion of the edge information. The edge information may comprise information of the whole beam profile, in particular from center and edge regions. The edge information may have a proportion of center information of less than 10%, preferably of less than 5%, more preferably the edge information comprises no center content. At least one area of the beam profile may be determined and/or selected as second area of the beam profile if it is close or around the center and comprises essentially center information. At least one area of the beam profile may be determined and/or selected as first area of the beam profile if it comprises at least parts of the falling edges of the cross section. For example, the whole area of the cross section may be determined as first region. The first area of the beam profile may be area A2 and the second area of the beam profile may be area A1.
Other selections of the first area A1 and second area A2 may be feasible. For example, the first area may comprise essentially outer regions of the beam profile and the second area may comprise essentially inner regions of the beam profile. For example, in case of a two-dimensional beam profile, the beam profile may be divided in a left part and a right part, wherein the first area may comprise essentially areas of the left part of the beam profile and the second area may comprise essentially areas of the right part of the beam profile.
The edge information may comprise information relating to a number of photons in the first area of the beam profile and the center information may comprise information relating to a number of photons in the second area of the beam profile. The evaluation device may be adapted for determining an area integral of the beam profile. The evaluation device may be adapted to determine the edge information by integrating and/or summing of the first area. The evaluation device may be adapted to determine the center information by integrating and/or summing of the second area. For example, the beam profile may be a trapezoid beam profile and the evaluation device may be adapted to determine an integral of the trapezoid. Further, when trapezoid beam profiles may be assumed, the determination of edge and center signals may be replaced by equivalent evaluations making use of properties of the trapezoid beam profile such as determination of the slope and position of the edges and of the height of the central plateau and deriving edge and center signals by geometric considerations.
Additionally or alternatively, the evaluation device may be adapted to determine one or both of center information or edge information from at least one slice or cut of the light spot. This may be realized, for example, by replacing the area integrals in the quotient signal Q by a line integrals along the slice or cut. For improved accuracy, several slices or cuts through the light spot may be used and averaged. In case of an elliptical spot profile, averaging over several slices or cuts may result in an improved distance information.
The evaluation device may be configured to derive the quotient signal Q by one or more of dividing the edge information and the center information, dividing multiples of the edge information and the center information, dividing linear combinations of the edge information and the center information. Thus, essentially, photon ratios may be used as the physical basis of the method.
In one embodiment, the light beam propagating from the object to the detector may illuminate the optical sensors with at least one pattern comprising at least one feature point. As used herein, the term “feature point” refers to at least one at least partially extended feature of the pattern. The feature point may be selected from the group consisting of: at least one point, at least one line, at least one edge. The pattern may be generated by the object, for example, in response to an illumination by the at least one light source with an illumination pattern comprising the at least one pattern. A1 may correspond to a full or complete area of a feature point on the optical sensors. A2 may be a central area of the feature point on the optical sensors. The central area may be a constant value. The central area may be smaller compared to the full area of the feature point. For example, in case of a circular feature point, the central area may have a radius from 0.1 to 0.9 of a full radius of the feature point, preferably from 0.4 to 0.6 of the full radius.
For example, the light beam propagating from the object to the detector may illuminate the optical sensors with at least one line pattern. The line pattern may be generated by the object, for example in response to an illumination by the at least one illumination source with an illumination pattern comprising the at least one line pattern. A1 may correspond to an area with a full line width of the line pattern on the optical sensors, in particular on the light sensitive area of the optical sensors. The line pattern on the optical sensors may be widened and/or displaced compared to the line pattern of the illumination pattern such that the line width on the optical sensors is increased. In particular, in case of the optical sensors are arranged as a sensor element having at least two columns, the line width of the line pattern on the optical sensors may change from one column to another column. A2 may be a central area of the line pattern on the optical sensors. The line width of the central area may be a constant value, and may in particular correspond to the line width in the illumination pattern. The central area may have a smaller line width compared to the full line width. For example, the central area may have a line width from 0.1 to 0.9 of the full line width, preferably from 0.4 to 0.6 of the full line width. The line pattern may be segmented on the optical sensors. Each column of optical sensors may comprise center information of intensity in the central area of the line pattern and edge information of intensity from regions extending further outwards from the central area to edge regions of the line pattern. For example, the detector may comprise at least two rows of optical sensors, wherein the rows are arranged on top of each other. The line pattern may be imaged parallel to the rows such that the center of the line pattern is fixed or moves only little relative to a dividing line of the rows. The A1 may correspond to an area above the dividing line and A2 may be an area below the dividing line.
The light beam may be a monochromatic light beam. The light beam may comprise a narrow band of wavelengths, preferably the light beam may comprise a single wavelength. The at least one light source may be adapted to generate at least one monochromatic light beam and/or the detector may comprise at least one filter element adapted to filter a narrow band of wavelength, such as a monochromator.
The illumination source may illuminate the object with a convergent and/or divergent and/or collimated light beam.
The illumination source and the sensor element may be arranged in a common plane or in different planes. The illumination source and the optical sensors may have different spatial orientation. In particular, the illumination source and the optical sensors may be arranged in a twisted arrangement.
In typical setups, commercially available quadrant diodes such as quadrant photodiodes are used for positioning, i.e., for adjusting and/or measuring a transversal coordinate of a light spot in the plane of the quadrant photodiode. Thus, as an example, laser beam positioning by using quadrant photodiodes is well known. According to a typical prejudice, however, quadrant photodiodes are used for xy-positioning, only. According to this assumption, quadrant photodiodes are not suitable for measuring distances. The above-mentioned findings, however, using an off-centered quadrant photodiode with regard to an optical axis of the detector, show otherwise, as will be shown in further measurements below. Thus, as indicated above, in quadrant photodiodes, the asymmetry of the spot can be measured by shifting the quadrant diode slightly off-axis, such as by the above-mentioned offset. Thereby, a monotonically z-dependent function may be generated, such as by forming the quotient signal Q of two or more of the sensor signals of two or more partial photodiodes, i.e., quadrants, of the quadrant photodiode. Therein, in principle, only two photodiodes are necessary for the measurement. The other two diodes may be used for noise cancellation or to obtain a more precise measurement.
In addition or as an alternative to using a quadrant diode or quadrant photodiode, other types of optical sensors may be used. Thus, as will be shown in further detail below, staggered optical sensors may be used.
The use of quadrant diodes provides a large number of advantages over known optical detectors. Thus, quadrant diodes are used in a large number of applications in combination with LEDs or active targets and are widely commercially available at very low price, with various optical properties such as spectral sensitivities and in various sizes. No specific manufacturing process has to be established, since commercially available products may be implemented into the detector according to the present invention.
The detector according to the present invention specifically may be used in multilayer optical storage discs, such as disclosed by international patent application number PCT/IB2015/052233, filed on Mar. 26, 2015. Measurements performed by using the detector according to the present invention specifically may be used in order to optimize the focus position in optical storage discs.
As will be outlined in further detail below, the distance measurement by using the detector according to the present invention may be enhanced by implementing one or more additional distance measurement means into the detector and/or by combining the detector with other types of distance measurement means. Thus, as an example, the detector may comprise or may be combined with at least one triangulation distance measurement device. Thus, the distance measurement can be enhanced by making use of a combination of the measurement principle discussed above and a triangulation type distance measurement. Further, means for measuring one or more other coordinates, such as x- and/or y-coordinates, may be provided.
In case a quadrant diode is used, the quadrant diode may also be used for additional purposes. Thus, the quadrant diode may also be used for conventional x-y-measurements of a light spot, as generally known in the art of optoelectronics and laser physics. Thus, as an example, the lens or detector position can be adjusted using the conventional xy-position information of the quadrant diode to optimize the position of the spot for the distance measurement. As a practical example, the light spot, initially, may be located right in the center of the quadrant diode, which typically does not allow for the above-mentioned distance measurement using the quotient function Q. Thus, firstly, conventional quadrant photodiode techniques may be used for off-centering a position of the light spot on the quadrant photodiode, such that, e.g., the spot position on the quadrant diode is optimal for the measurement. Thus, as an example, the different off-centering of the optical sensors of the detector may simply be a starting point for movement of the optical sensors relative to the optical axis such that the light spot is off-centered with respect to the optical axis and with respect to a geometrical center of the array of the optical sensors.
Thus, generally, the optical sensors of the detector may form a sensor array or may be part of a sensor array, such as the above-mentioned quadrant diode. Thus, as an example, the detector may comprise an array of optical sensors, such as a rectangular array, having m rows and n columns, with m, n, independently, being positive integers. Preferably, more than one column and more than one row is given, i.e., n>1, m>1. Thus, as an example, n may be 2 to 16 or higher and m may be 2 to 16 or higher. Preferably, the ratio of the number of rows and the number of columns is close to 1. As an example, n and m may be selected such that 0.3 £ m/n £ 3, such as by choosing m/n=1:1, 4:3, 16:9 or similar. As an example, the array may be a square array, having an equal number of rows and columns, such as by choosing m=2, n=2 or m=3, n=3 or the like. The case m=2, n=2 is the case of the quadrant diode or quadrant optical sensor, which, for practical reasons, is one of the preferred cases, since quadrant photodiodes are widely available.
Thus, generally, the optical sensors of the detector may form a sensor array or may be part of a sensor array, such as the above-mentioned quadrant diode. Thus, as an example, the detector may comprise an array of optical sensors, such as a rectangular array, having m rows and n columns, with m, n, independently, being positive integers. Preferably, more than one column and more than one row is given, i.e., n>1, m>1. Thus, as an example, n may be 2 to 16 or higher and m may be 2 to 16 or higher. Preferably, the ratio of the number of rows and the number of columns is close to 1. As an example, n and m may be selected such that 0.3 £ m/n £ 3, such as by choosing m/n=1:1, 4:3, 16:9 or similar. As an example, the array may be a square array, having an equal number of rows and columns, such as by choosing m=2, n=2 or m=3, n=3 or the like. The case m=2, n=2 is the case of the quadrant diode or quadrant optical sensor, which, for practical reasons, is one of the preferred cases, since quadrant photodiodes are widely available.
As a starting point, a geometrical center of the optical sensors within the array may be off-centered from the optical axis, such as by the above-mentioned offset. The sensor array specifically may be movable relative to the optical axis, for example along a gradient, preferably automatically, such as by moving the sensor array, e.g., in a plane perpendicular to the optical axis, and/or by moving the optical axis itself, e.g shifting the optical axis in a parallel shift and/or tilting the optical axis. Thus, the sensor array may be shifted in order to adjust a position of a light spot generated by the light beam in the plane of the sensor array. Additionally or alternatively, the optical axis may be shifted and/or tilted by using appropriate elements, such as by using one or more deflection elements and/or one or more lenses. The movement, as an example, may take place by using one or more appropriate actuators, such as one or more piezo actuators and/or one or more electromagnetic actuators and/or one or more pneumatic or mechanical actuators, which, e.g., move and/or shift the array and/or move and/or shift and/or tillage one or more optical elements in the beam path in order to move the optical axis, such as parallel shifting the optical axis and/or tilting the optical axis. The evaluation device specifically may be adjusted to control a relative position of the sensor array to the optical axis, e.g., in the plane perpendicular to the optical axis. An adjustment procedure may take place in that the evaluation device is configured for, firstly, determining the at least one transversal position of a light spot generated by the light beam on the sensor array by using the sensor signals and for, secondly, moving the array relative to the optical axis, such as by moving the array and/or the optical axis, e.g., by moving the array in the plane to the optical axis until the light spot is off-centered and/or by tilting a lens until the light spot is off-centered. As used therein, a transversal position may be a position in a plane perpendicular to the optical axis, which may also be referred to as the x-y-plane. For the measurement of the transversal coordinate, as an example, the sensor signals of the optical sensors may be compared. As an example, in case the sensor signals are found to be equal and, thus, in case it is determined that the light spot is located symmetrically with respect to the optical sensors, such as in the center of the quadrant diodes, a shifting of the array and/or a tilting of a lens may take place, in order to off-center the light spot in the array. Thus, as outlined above, the off-centering of the array from the optical axis, such as by off-centering the center of the quadrant photodiode from the optical axis, may simply be a starting point in order to avoid the situation which is typical, in which the light spot is located on the optical axis and, thus, is centered. By off-centering the array relative to the optical axis, thus, the light spot should be off-centered. In case this is found not to be true, such that the light spot, incidentally, is located in the center of the array and equally illuminates all optical sensors, the above-mentioned shifting of the array relative to the optical axis may take place, preferably automatically, in order to off-center the light spot on the array. Thereby, a reliable distance measurement may take place.
Further, in a scanning system with a movable light source, the position of the light spot on the quadrant diode may not be fixed. This is still possible, but may necessitate that different calibrations are used, dependent on the xy-position of the spot in the diode.
Further, the use of the above-mentioned quotient signal Q is a very reliable method for distance measurements. Typically, Q is a monotonic function of the longitudinal coordinate of the object and/or of the size of the light spot such as the diameter or equivalent diameter of the light spot. Thus, as an example, specifically in case linear optical sensors are used, the quotient Q=s1/s2 is a monotonically decreasing function of the size of the light spot. Without wishing to be bound by this theory, it is believed that this is due to the fact that, in the preferred setup described above, the sensor signals, such as the above-mentioned first sensor signal s1 and the above-mentioned second sensor signal s2, decrease as a square function with increasing distance to the light source, since the amount of light reaching the detector decreases. Therein, however, due to the off-centering, the one of the sensor signals decreases more rapidly than the other, since, in the optical setup as used in the experiments, the light spot in the image plane grows and, thus, is spread over a larger area. By spreading the light spot, however, the portion of the light illuminating the one or more optical sensors outside the center of the light spot increases, as compared to a situation of a very small light spot. Thus, the quotient of the sensor signals continuously changes, i.e., increases or decreases, with increasing diameter of the light beam or diameter of the light spot. The quotient, further, may further be rendered mainly independent from the total power of the light beam, since the total power of the light beam forms a factor in all sensor signals. Consequently, the quotient Q may form a secondary signal which provides a unique and unambiguous relationship between the sensor signals and the size or diameter of the light beam.
Since, on the other hand, the size or diameter of the light beam is dependent on a distance between the object, from which the light beam propagates towards the detector, and the detector itself, i.e., dependent on the longitudinal coordinate of the object, a unique and unambiguous relationship between the first and second sensor signals and the longitudinal coordinate may exist. For the latter, reference e.g., may be made to one or more of the above-mentioned prior art documents, such as WO 2014/097181 A1. The predetermined relationship may be determined by analytical considerations, such as by assuming a linear combination of Gaussian light beams, by empirical measurements, such as measurements measuring the first and second sensor signals or a secondary signal derived thereof as a function of the longitudinal coordinate of the object, or both.
Thus, generally, the evaluation device may be configured for determining the longitudinal coordinate by evaluating the quotient signal Q. The evaluation device may be configured for using at least one predetermined relationship between the quotient signal Q and the longitudinal coordinate. The predetermined relationship may be one or more of an empiric relationship, a semi-empiric relationship and an analytically derived relationship. The evaluation device may comprise at least one data storage device for storing the predetermined relationship, such as a lookup list or a lookup table.
The quotient signal Q may be determined by using various means. As an example, a software means for deriving the quotient signal, a hardware means for deriving the quotient signal, or both, may be used and may be implemented in the evaluation device. Thus, the evaluation device, as an example, may comprise at least one divider, wherein the divider is configured for deriving the quotient signal. The divider may fully or partially be embodied as one or both of a software divider or a hardware divider.
In view of the technical challenges involved in the prior art documents discussed above, specifically in view of the technical effort which is required for generating the FiP effect, it has to be noted that the present invention specifically may be realized by using non-FiP optical sensors. In fact, since optical sensors having the FiP characteristic typically exhibit a strong peak in the respective sensor signals at a focal point, the range of measurement of a detector according to the present invention using FiP sensors as optical sensors may be limited to a range in between the two positions and which the optical sensors are in focus of the light beam. When using linear optical sensors, however, i.e., optical sensors not exhibiting the FiP effect, this problem, with the setup of the present invention, generally may be avoided. Consequently, the optical sensors may each have, at least within a range of measurement, a linear signal characteristic such that the respective first and second sensor signals may be dependent on the total power of illumination of the respective optical sensor and may be independent from a diameter of a light spot of the illumination. It shall be noted, however, that other embodiments are feasible, too.
The optical sensors each specifically may be semiconductor sensors, preferably inorganic semiconductor sensors, more preferably photodiodes and most preferably silicon photodiodes. Thus, as opposed to complex and expensive FiP sensors, the present invention simply may be realized by using commercially available inorganic photodiodes, i.e., one small photodiode and one large area photodiode. Thus, the setup of the present invention may be realized in a cheap and inexpensive fashion.
In one embodiment, the detector may comprise at least one FiP sensor adapted for generating the so called FiP effect as described in WO 2015/024871 or WO2016/120392. For example, the bi-cell or quadrant diode may be adapted to generate a so called FiP signal. As outlined e.g., in WO 2015/024871 or WO2016/120392, the FiP signal can be used to determine depth information over a wide distance range. The FiP sensor may be adapted to exhibit a positive and/or a negative FiP effect. According to the FiP-effect, the longitudinal sensor signal, given the same total power, may exhibit at least one pronounced maximum for one or a plurality of focusings and/or for one or a plurality of specific sizes of the light spot on the sensor region or within the sensor region. For purposes of comparison, an observation of a maximum of the longitudinal sensor signal in a condition in which the corresponding material is impinged by a light beam with the smallest possible cross-section, such as when the material may be located at or near a focal point as affected by an optical lens, may be denominated as a “positive FiP-effect”. The negative FiP effect describes an observation of a minimum of the longitudinal sensor signal under a condition in which the corresponding material is impinged by a light beam with the smallest available beam cross-section, in particular, when the material may be located at or near a focal point as effected by an optical lens. The negative FiP effect may be used to tune small image effects at high distances. Image changes such as position, size, shape, sharpness, etc. may vanish at high distances while the negative FiP effect increases. Furthermore, no luminance dependence may be introduced since both cells are at the same longitudinal position and thus receive identical photon density.
At least one of the optical sensors may be adapted to generate at least one sensor signal dependent on a time-of-flight (TOF) the illumination light beam has traveled from the illumination source to the object and the reflection light beam has traveled from the object to the light sensitive area of the optical sensor. The evaluation device may be configured for determining at least one TOF-longitudinal coordinate zTOF of the object by evaluating the TOF sensor signal. The optical sensor adapted to generate the TOF sensor signal may be designed as time-of-flight detector. The time-of-flight detector may be selected from the group consisting of: at least one pulsed time-of-flight detector; at least one phase modulated time-of-flight detector; at least one direct time-of-flight detector; at least one indirect time-of-flight detector. For example, the pulsed time-of-flight detector may be at least one range gated imager and/or at least one direct time-of-flight imager. For example the phase modulated time-of-flight detector may be at least one RF-modulated light source with at least one phase detector. The optical sensor may be adapted to determine a time delay between emission of the illumination light beam by the illumination source and receipt of the reflection light beam.
For example, the optical sensor adapted to generate the TOF sensor signal may be designed as pulsed time-of-flight detector. The detector may comprise the at least one interruption device, such as at least one shutter element, adapted to generate a pulsed light beam. The optical sensor may be adapted to store the TOF sensor signal dependent on the receiving time of the reflection light beam in a plurality of time windows, in particular subsequent time windows. The optical sensor may be adapted to store dependent on receiving time of the reflection light beam the generated TOF sensor signal in at least one first time window and/or in at least one second time window. The first and second time windows may be correlated with the opening and closure of the interruption device. Duration of first and second time windows may be pre-defined. For example, the TOF sensor signal may be stored in the first time window during opening of the interruption device, whereas during closure of the interruption device the TOF sensor signal may be stored in the second time window. Other durations of time windows are thinkable. The first and the second time window may comprise information about background, signal height and signal shift.
For example, the optical sensor adapted to generate the TOF sensor signal may be designed as direct time-of-flight imager. The direct time-of-flight imager may comprise the at least one illumination source adapted to generate at least one single laser pulse. The single laser pulse may be reflected back from the object onto the optical sensor. The optical sensor may comprise at least one photodiode, for example at least one Avalanche Photo Diode (APD), such as at least one Si APD, or such as at least one InGaAs APD, or at least one PIN photo detector array, or at least one Single-Photon Avalanche Photodiode (SPAD), adapted to image the reflection light beam. The direct time-of flight imager may be adapted to image at least one image comprising spatial and temporal data.
For example, the optical sensor adapted to generate the TOF sensor signal may be designed as phase modulated time-of-flight modulator. The phase modulated time-of-flight modulator may be adapted to measure a difference in phase, in particular a phase shift, by determining a correlated signal, for example by multiplying a received signal, i.e., of the reflection light beam, with the emitted signal, i.e., the illumination light beam. A DC component of the correlated signal may comprise an information about the difference in phase. The evaluation device may be adapted to determine the second longitudinal coordinate of the object from the phase difference. For example, the illumination source and the optical sensor adapted to generate the TOF sensor signal may be designed as RF-modulated light source with at least one phase detector. The illumination source may comprise, for example, at least one LED and/or at least one laser. The illumination source may comprise at least one modulation device adapted to modulate the light beam having a pre-defined phase shift. For example, the modulation device may comprise at least one radio frequency module. The radio frequency module may be adapted to modulate the illumination beam with an RF carrier. The optical sensor may be adapted to determine a phase shift of the reflective light beam impinging on the optical sensor.
The optical sensor may be designed as and/or may comprise at least one time-of-flight pixel. Preferably, the detector may comprise at least two optical sensors, wherein each optical sensor is designed as and/or comprises at least one TOF pixel. For example, the detector, in particular the optical sensor may comprise a quadrant diode adapted to generate the TOF sensor signal. For example, the detector, in particular the optical sensor, may comprise at least one pixelated TOF-imager.
The evaluation device may be configured for determining the at least one TOF longitudinal coordinate zTOF of the object by evaluating the TOF sensor signal. As used herein, the term “TOF longitudinal coordinate” refers to a longitudinal coordinate derived from the TOF sensor signal. As described above, the illumination light source may be adapted to periodically generate at least one light pulse. The detector may be adapted to generate the first longitudinal sensor signal for each period. The evaluation device may be adapted to determine in which pulse period the TOF sensor signal was generated using the second longitudinal coordinate. The detector may be adapted to uniquely assign to which period the ToF-signal refers to by using the combined sensor signal. Both, the TOF sensor signal and the combined sensor signal, may be non-monotonic functions of the longitudinal coordinate zreal. Thus, the longitudinal coordinate may not be uniquely determined from one of the TOF sensor signal or the combined sensor signal alone and a measurement range may not be restricted to a longitudinal range in which the signals are unique functions of zreal. The term “measurement range” generally refers to a range from the object to the detector in which determination of the longitudinal coordinate is possible. The term “longitudinal range” generally refers to a range from the object to the detector in which unique determination of the longitudinal coordinate is possible. Below and/or above certain distances from the object to the detector a determination of the longitudinal coordinate may not be possible. For example, time-of-flight measurements are not possible below a certain distance from the object to the detector, possibly due to the minimum measurement time of the internal clock. Furthermore, in time-of-flight measurements the sensor signal may be unique within a longitudinal period, but the sensor signal may be the same in case integer multiples of the longitudinal period are added such that the determined longitudinal coordinate may be non-unique. Thus, the same TOF sensor signal will be obtained for a distance z1 and a distance z1+n·z1p, where n is an integer denoting the longitudinal period and z1p is the longitudinal period of the TOF sensor signal, wherein the distances z1 and z1+n-z1p are within the measurement range. As used herein, the term “longitudinal period” refers to partitions of a period, in particular a distance range, in which the longitudinal coordinate can be unambiguously determined from the TOF sensor signal. Non-unique longitudinal coordinates may be denoted as relative longitudinal coordinates and the unique longitudinal coordinates may be denoted as absolute longitudinal coordinates.
If both the TOF sensor signal F1 and the combined sensor signals F2 are available, it may be possible to uniquely determine the longitudinal coordinate and to extend the longitudinal range, as long as each signal pair (F1,F2) corresponds to a unique distance and vice versa. In particular, if a unique signal pair (F1,F2) exists for each longitudinal coordinate and vice versa, the evaluation device may be adapted to determine the unique combined longitudinal coordinate by
(1) selecting at least one first selected signal such as the TOF sensor signal and/or the combined sensor signal and determining non-unique first longitudinal coordinates;
(2) selecting a second selected signal such as the combined signal Q and/or the TOF sensor signal, which was not selected in step (1), and determining non-unique second longitudinal coordinates;
(3) determining whether one of the non-unique first longitudinal coordinates and the non-unique second longitudinal coordinates match up to a predetermined tolerance threshold;
(4) setting a combined unique longitudinal coordinate to be a matching longitudinal coordinate.
In step (1) and step (2), signals may be selected in the given order or may be performed in a different order. For example, in step (1) the TOF sensor signal may be selected and in step (2) the combined signal Q may be selected. In another example, in step (1) the combined sensor signal may be selected, and the non-unique first longitudinal sensor signal may be determined therefrom. In step (2), the TOF sensor signal may be selected.
Additionally or alternatively to step (4), the evaluation device may be adapted to output an error signal in case no matching coordinates are found and/or to output an error signal in case more than one matching coordinates are found. Additionally or alternatively, the signal pairs and their corresponding longitudinal coordinates may be stored in a look-up table. Additionally or alternatively, the signal pairs and their corresponding longitudinal coordinates may be approximated or described by an analytical function which is evaluated to find the longitudinal coordinate corresponding to a given signal pair.
The evaluation device may comprise at least two memory elements. As used herein, the term “memory element” refers to a device adapted to store information. The evaluation device may be adapted to receive and to store information provided by the optical sensors, for example the at least one first sensor signal. Such information may comprise raw sensor data and/or processed sensor data. For example, the memory element may be adapted to store information for further evaluation by the evaluation device. The memory element may be a volatile or non-volatile memory element.
As described above, the optical sensor may be designed as and/or may comprise at least one ToF pixel. The detector may comprise at least two switches. Each of the switches may be connected to the optical sensor adapted to generate the first sensor signal, for example by at least one connector. In particular, each of the switches may be connected to the ToF pixel. The switches are adapted to provide the TOF sensor signal to one of the memory elements. In particular, the switches may be adapted to let, dependent on receiving time of the reflection light beam, the generated TOF sensor signal pass through one of the switches. For example, the TOF sensor signal may pass one of the switches during opening of the interruption device, whereas during closure of the interruption device the TOF sensor signal may pass the other switch. Each of the switches may be controlled by a control signal having a pulse length identical to a pulse length of a light pulse generated by the illumination source. The control signal of one of the switches may be delayed. For example, the delay may correspond to the pulse length of the light pulse. The evaluation device may be adapted to sample and/or store depending on the delay a first part, or fraction, of the TOF sensor signal through a first switch in a first memory element and the other, second part, or fraction, of the TOF sensor signal through a second switch in a second memory element. The evaluation device may be adapted to determine the first longitudinal coordinate by evaluating the first part and second part of the TOF sensor signal. The evaluation device may be adapted to determine the first longitudinal coordinate z, by
wherein c is the speed of light, t0 is the pulse length of the illumination light beam, z0 is a distance offset often determined by the resolution of the time measurement, and S11 and S12 are the first part and second part of the TOF sensor signal, respectively.
As outlined above, the detector may further comprise one or more additional elements such as one or more additional optical elements. Further, the detector may fully or partially be integrated into at least one housing.
As outlined above, specifically, quadrant photodiodes may be used. As an example, commercially available quadrant photodiodes may be integrated in order to provide four optical sensors, such as one or more quadrant photodiodes available from Hamamatsu Photonics Deutschland GmbH, D-82211 Herrsching am Ammersee, Germany, such as quadrant Si PIN photodiodes of the type S4349, which are sensitive in the UV spectral range to the near IR spectral range. In case an array of optical sensors is used, the array may be a naked chip or may be an encapsulated array, such as encapsulated in a TO-5 metal package. Additionally or alternatively, a surface mounted device may be used, such as TT Electronics OPR5911 available from TT Electronics plc, Fourth Floor, St Andrews House, West Street Woking Surrey, GU21 6EB, England. It shall be noted that other optical sensors may also be used.
Further, it shall be noted that, besides the option of using precisely one quadrant photodiode, two or more quadrant photodiodes may also be used. Thus, as an example, a first quadrant photodiode may be used for the distance measurement, as described above, providing the two or more optical sensors. Another quadrant photodiode may be used, e.g., in a second partial beam path split off from the beam path of the first quadrant photodiode, for a transversal position measurement, such as for using at least one transversal coordinate x and/or y. The second quadrant photodiode, as an example, may be located on-axis with respect to the optical axis.
Further, it shall be noted that, besides the option of using one or more quadrant photodiodes, one or more quadrant photodiodes or further photodiode arrays may also be replaced or mimicked by separated photodiodes that are arranged or assembled close to each other, preferably in a symmetric shape such as a rectangular matrix, such as a 2×2 matrix. However further arrangements are feasible. In such an arrangement or assembly, the photodiodes may be arranged or assembled in a housing or mount, such as all photodiodes in a single housing or mount or groups of photodiodes in one housing or mount, or each of the photodiodes in a separate housing or mount. Further, the photodiodes may also be assembled directly on a circuit board. In such arrangements or assemblies, photodiodes may be arranged as such that the separation between the active area of the photodiodes, has a distinct value less than one centimeter, preferably less than one millimeter, more preferably as small as possible. Further, to avoid optical reflexes, distortions, or the like that may deteriorate the measurement, the space between the active areas may be either empty or filled with a material, preferably with a light absorbing material such as a black polymer, such as black silicon, black polyoxymethylene, or the like, more preferably optically absorbing and electrically insulating material, such as black ceramics or insulating black polymers such as black silicon, or the like. Further, the distinct value of the photodiode separation may also be realized by adding a distinct building block between the photodiodes such as a plastic separator. Further embodiments are feasible. The replacement of quadrant photodiodes by single diodes arranged in a similar setup such as in a 2×2 rectangular matrix with minimal distance between the active areas may further minimize the costs for the optical detector. Further, two or more diodes from a quadrant diode may be connected in parallel or in series to form a single light sensitive area.
Additionally or alternatively, the optical sensors may comprise at least one optical sensor of an intrinsic photovoltaic type, more preferably at least one semiconductor photodiode selected from the group consisting of: a Ge photodiode, an InGaAs photodiode, an extended InGaAs photodiode, an InAs photodiode, an InSb photodiode, a HgCdTe photodiode. Additionally or alternatively, the optical sensors may comprise at least one optical sensor of an extrinsic photovoltaic type, more preferably at least one semiconductor photodiode selected from the group consisting of: a Ge:Au photodiode, a Ge:Hg photodiode, a Ge:Cu photodiode, a Ge:Zn photodiode, a Si:Ga photodiode, a Si:As photodiode. Additionally or alternatively, the optical sensors may comprise at least one bolometer, preferably a bolometer selected from the group consisting of a VO bolometer and an amorphous Si bolometer. The optical sensors may comprise differing materials or photovoltaic concepts. As an example, at least one first optical sensor may be a Ge photodiode, whereas at least one second optical sensor may be an InGaAs photodiode. As a further example, the first and second optical sensor may comprise the same material with a differing doping such as a differing doping concentration or doping material.
The optical sensors each, independently, may be opaque, transparent or semitransparent. For the sake of simplicity, however, opaque sensors which are not transparent for the light beam, may be used, since these opaque sensors generally are widely commercially available.
The optical sensors each specifically may be uniform sensors having a single light-sensitive area each. Thus, the optical sensors specifically may be non-pixelated optical sensors.
The detector may further comprise an illumination source for illuminating the object. As an example, the illumination source may be configured for generating an illuminating light beam for illuminating the object. The detector may be configured such that the illuminating light beam propagates from the detector towards the object along an optical axis of the detector. For this purpose, the detector may comprise at least one reflective element, preferably at least one prism, for deflecting the illuminating light beam onto the optical axis. Specifically, the illumination source may comprise at least one laser and/or laser source. Various types of lasers may be employed, such as semiconductor lasers. Additionally or alternatively, non-laser light sources may be used, such as LEDs and/or light bulbs. The illumination source may be adapted to generate and/or to project a cloud of points, for example the illumination source may comprise one or more of at least one digital light processing (DLP) projector, at least one LCoS projector, at least one spatial light modulator; at least one diffractive optical element; at least one array of light emitting diodes; at least one array of laser light sources.
The illuminating light beam generally may be parallel to the optical axis or tilted with respect to the optical axis, e.g., including an angle with the optical axis. As an example, the illuminating light beam, such as the laser light beam, and the optical axis may include an angle of less than 10°, preferably less than 5° or even less than 2°. Other embodiments, however, are feasible. Further, the illuminating light beam may be on the optical axis or off the optical axis. As an example, the illuminating light beam may be parallel to the optical axis having a distance of less than 10 mm to the optical axis, preferably less than 5 mm to the optical axis or even less than 1 mm to the optical axis or may even coincide with the optical axis.
The detector may comprise the at least one illumination source. The illumination source may be adapted to illuminate the object with at least one illumination light beam. The illumination light beam may be fully or partially reflected by the object and may travel back towards the detector. The illumination source may be arranged such that a direction of propagation of the illumination light beam is essentially parallel to the optical axis. As used herein, the term “essentially parallel to the optical axis” refers to the condition of a parallel orientation, with a tolerance of e.g., ±20° or less, preferably a tolerance of ±10° or less, more preferably a tolerance of ±5° or less. The illumination source may be arranged such that a line running in the direction of propagation of the illumination light beam and the optical axis do not intersect in space. In particular, the illumination may be on-axis. In order to provide such an on-axis illumination, as an example, one or more reflective elements may be used, such as one or more prisms and/or mirrors, such as dichroitic mirrors, such as movable mirrors or movable prisms. The reflected light beam may be essentially parallel to the optical axis. Thus, according to the present invention, the object distance is not determined from an angle of the light beam, i.e., a position of the light spot on a sensor, as done in triangulation methods. Instead, the object distance may be determined using light beams essential parallel to the optical axis. Determination of the object distance by a non-triangulation method but by combining the at least two sensor signals, allows arranging the illumination source such that the direction of propagation of the illumination light beam is essentially parallel to the optical axis.
The illumination source and the optical axis may be separated by a small baseline. As used herein, the term “baseline”, also denoted as basis line, refers to a distance, for example in a xy-plane, between the optical axis and the illumination source, in particular a distance between the optical axis and a z-component of the illumination light beam. The illumination source may be spaced apart from the optical axis by a minimum distance. The minimum distance from the optical axis may be defined by further detector elements such as size and position of the optical sensors and transfer device. The baseline may be less than 0.1 m, preferably less than 0.05 m, more preferably less than 0,025 m. For example, the baseline may be 21 mm. Preferably, the illumination source may be arranged right next to the transfer device. For example, the transfer device may be flattened such that the illumination source can be positioned even closer to the optical axis. The illumination source may be arranged behind the transfer device. Triangulation systems require a sufficient baseline, however due to the baseline in the near filed no detection may be possible. Near field detection may be possible if the light spot is tilted in direction of the transfer device. However, the tilting leads to that the light spot will move out of the field of view which limits detection in far field regions. Thus, in triangulation systems, the non-zero baseline will always lead to a substantial reduction in the measurement range, in the near field, and/or in the far field. Reducing the baseline as possible with the detector according to the present invention will thus always increase the measurement range. For example, the detector may be a compact device without further optical elements, wherein the illumination source may be placed as close as possible to an edge of the transfer device. Thus, the baseline may be close to half a diameter of the transfer device, in particular the lens diameter and housings of lens and light source. For example, the detector may be an even more compact device, wherein a mirror, in particular a small mirror, may be positioned in front of the transfer device, in particular in a center, for example a geometrical center, or close to the center of the transfer device, in order to couple the illumination beam into the beam path. Thus, the baseline may be less than half the diameter of the transfer device. The illumination source may be arranged such that the baseline is as small as possible. By arranging the illumination source such that the direction of propagation of the illumination light beam is essentially parallel to the optical axis and that the illumination source and the optical axis are separated by the small baseline, very compact devices are possible. For example, a distance from the center of the transfer device to the illumination source, in particular along a connecting line from the center of the transfer device to the illumination source, may be preferably less than 2.5 times a distance from the center of the transfer device to an edge of the transfer device, more preferably less than 1.5 times the distance center to edge of the transfer device, and most preferably less 1 times the distance center to edge of the transfer device. The transfer device may have an arbitrary shape, in particular a non-circular shapes are possible. At small distances an aperture of the illumination source may be small and the baseline may be small. At large distances an aperture of the illumination source may be large and the baseline may be small. This is contrarily as in triangulation methods, wherein at large distances a large baseline is necessary. Further, triangulation based systems have a minimum detection range significantly greater than zero, for example such as 20 cm from the detector system, due to necessary spatial extend of the baseline. Such a large baseline may result in that the illuminated light scattered from the object may not reach the light sensitive area of the optical sensor behind the transfer device. In addition, in triangulation based systems, using a small baseline would reduce the minimum detection range, however, at the same time would reduce a maximum detection range. Further, triangulation based systems require a plurality of light sensitive areas and sensor signals, for example sensor signals of at least one detector row. According to the invention, determination of the longitudinal coordinate z is possible with a reduced number of sensor signals, in particular with less than 20, preferably less than 10 and more preferably less than 5 sensor signals.
The illumination source may be adapted to generate at least one illumination pattern for illuminating the object. Additionally or alternatively, the illumination pattern may be generated by at least one ambient light source. The detector may be configured such that the illumination pattern propagates from the detector, in particular from at least one opening of the housing, towards the object along and/or parallel to an optical axis of the detector. For this purpose, the detector may comprise at least one reflective element, preferably at least one prism, for deflecting the illumination pattern such that it propagates along or parallel to the optical axis. Specifically, the illumination source may comprise at least one laser and/or laser source. Various types of lasers may be employed, such as semiconductor lasers. Additionally or alternatively, non-laser light sources may be used, such as LEDs and/or light bulbs. As used herein, the term “pattern” refers to an arbitrary known or predetermined arrangement comprising at least one arbitrarily shaped feature. The pattern may comprise at least one feature such as a point or symbol. The pattern may comprise a plurality of features. The pattern may comprise an arrangement of periodic or non-periodic features. As used herein, the term “illumination pattern” refers to a pattern which illuminates the object. The illumination pattern may be generated by ambient light, such as by at least one ambient light source, or by the at least one illumination source. The illumination pattern may comprise at least one pattern selected from the group consisting of: at least one point pattern, at least one pattern comprising at least one pre-known feature; at least one regular pattern; at least one line pattern comprising at least one line; at least one line pattern comprising at least two lines such as parallel or crossing lines.
The illumination source may comprise one or more of at least one light projector; at least one digital light processing (DLP) projector, at least one LCoS projector, at least one spatial light modulator; at least one diffractive optical element; at least one array of light emitting diodes; at least one array of laser light sources. The illumination source may comprise at least one light source adapted to generate the illumination pattern directly. For example, the illumination source may comprise at least one laser source. For example, the illumination source may comprise at least one line laser. The line laser may be adapted to send a laser line to the object, for example a horizontal or vertical laser line. The illumination source may comprise a plurality of line lasers. For example, the illumination source may comprise at least two line lasers which may be arranged such that the illumination pattern comprises at least two parallel or crossing lines. The illumination source may comprise the at least one light projector adapted to generate at least one point, e.g., a cloud of points such that the illumination pattern may comprise a plurality of point pattern. The illumination source may comprise at least one mask adapted to generate the illumination pattern from at least one light beam generated by the illumination source.
The illumination source may be one of attached to or integrated into a mobile device such as a smartphone. The illumination source may be used for further functions that may be used in determining an image such as for an autofocus function. The illumination source may be attached to a mobile device such as by using a connector such as a USB- or phone-connector such as the headphone jack.
The illumination source may be adapted to generate pulsed illumination. The illumination source may be adapted to generate at least one light pulse. As used herein, the term “light pulse” or “pulsed illumination” refers to a light beam limited in time. The light pulse may have a pre-defined length or time duration, for example in the nanoseconds range. For example, the illumination source may be adapted to generate pulses with a pulse length of less than a nanosecond, such as a tenth of a nanosecond, up to a tenth of a second. The illumination source may be adapted to periodically generate the light pulse. For example, the illumination source may be adapted to generate the light pulse with a frequency of 10 Hz to 10 GHz.
The illumination source may be adapted to generate a pulsed light beam. For example, the illumination source may be adapted to generate a continuous illumination light beam and the detector may comprise at least one interruption device adapted to interrupt the illumination, in particular periodically. The interruption device may comprise at least one shutter and/or a beam chopper or some other type of mechanical or electronical periodic beam interrupting device, for example comprising at least one interrupter blade or interrupter wheel, which preferably rotates at constant speed and which can thus periodically interrupt the illumination. By way of example, the at least one interruption device can also be wholly or partly integrated into the illumination source. Various possibilities are conceivable.
The detector may further comprise at least one transfer device, the transfer device being adapted to guide the light beam onto the optical sensors. The transfer device specifically may comprise one or more of: at least one lens, preferably at least one focus-tunable lens; at least one beam deflection element, preferably at least one mirror; at least one diffractive optical element; at least one Fresnel lens; at least one beam splitting element, preferably at least one of a beam splitting cube or a beam splitting mirror; at least one multi-lens system.
The optical sensors may be positioned off focus. As used herein, the term “focus” generally refers to one or both of a minimum extend of a circle of confusion of the light beam, in particular of at least one light beam emitted from one point of the object, caused by the transfer device or a focal length of the transfer device. As used herein, the term “circle of confusion” refers to a light spot caused by a cone of light rays of the light beam focused by the transfer device. The circle of confusion may depend on a focal length f of the transfer device, a longitudinal distance from the object to the transfer device, a diameter of an exit pupil of the transfer device, a longitudinal distance from the transfer device to the light sensitive area, a distance from the transfer device to an image of the object. For example, for Gaussian beams, a diameter of the circle of confusion may be a width of the Gaussian beam. In particular, for a point like object situated or placed at infinite distance from the detector the transfer device may be adapted to focus the light beam from the object into a focus point at the focal length of the transfer device. For non-point like objects situated or placed at infinite distance from the detector the transfer device may be adapted to focus the light beam from at least one point of the object into a focus plane at the focal length of the transfer device. For point like objects not situated or placed at infinite distance from the detector, the circle of confusion may have a minimum extend at least at one longitudinal coordinate. For non-point like objects not situated or placed at infinite distance from the detector, the circle of confusion of the light beam from at least one point of the object may have a minimum extend at least at one longitudinal coordinate. As used herein, the term “positioned off focus” generally refers to a position other than the minimum extend of a circle of confusion of the light beam caused by the transfer device or a focal length of the transfer device. In particular, the focal point or minimum extend of the circle of confusion may be at a longitudinal coordinate lfocus, whereas the position of each of the optical sensors may have a longitudinal coordinate lsensor different from lfocus For example, the longitudinal coordinate lsensor may be, in a longitudinal direction, arranged closer to the position of the transfer device than the longitudinal coordinate lfocus or may be arranged further away from the position of the transfer device than the longitudinal coordinate lfocus. Thus, the longitudinal coordinate lsensor and the longitudinal coordinate lfocus may be situated at different distances from the transfer device. For example, the optical sensors may be spaced apart from the minimum extend of the circle of confusion in longitudinal direction by ±2% focal length, preferably by ±10 focal length, most preferably ±20% focal length. For example, at a focal length of the transfer device may be 20 mm and the longitudinal coordinate lsensor may be 19.5 mm, i.e., the sensors may be positioned at 97.5% focal length, such that lsensor is spaced apart from the focus by 2.5% of focal length. The extent of the circle of confusion may be larger than the extent of the optical sensor. For example, the optical sensor may be positioned as such that the circle of confusion extends beyond the optical sensor. The optical sensor may thus only partially evaluate the beam profile of the light beam.
The above-mentioned operations, including determining the at least one longitudinal coordinate of the object, are performed by the at least one evaluation device. Thus, as an example, one or more of the above-mentioned relationships may be implemented in software and/or hardware, such as by implementing one or more lookup tables. Thus, as an example, the evaluation device may comprise one or more programmable devices such as one or more computers, application-specific integrated circuits (ASICs), Field Programmable Gate Arrays (FPGAs), or digital signal processors (DSPs) which are configured to perform the above-mentioned evaluation, in order to determine the at least one longitudinal coordinate of the object. Additionally or alternatively, however, the evaluation device may also fully or partially be embodied by hardware.
As outlined above, by evaluating the sensor signals, the detector may be enabled to determine the at least one longitudinal coordinate of the object, including the option of determining the longitudinal coordinate of the whole object or of one or more parts thereof. In addition, however, other coordinates of the object, including one or more transversal coordinates and/or rotational coordinates, may be determined by the detector, specifically by the evaluation device. Thus, as an example, one or more additional transversal sensors may be used for determining at least one transversal coordinate of the object. Various transversal sensors are generally known in the art, such as the transversal sensors disclosed in WO 2014/097181 A1 and/or other position-sensitive devices (PSDs), such as quadrant diodes, CCD or CMOS chips or the like. These devices may generally also be implemented into the detector according to the present invention. As an example, a part of the light beam may be split off within the detector, by at least one beam splitting element. The split-off portion, as an example, may be guided towards a transversal sensor, such as a CCD or CMOS chip or a camera sensor, and a transversal position of a light spot generated by the split-off portion on the transversal sensor may be determined, thereby determining at least one transversal coordinate of the object. Consequently, the detector according to the present invention may either be a one-dimensional detector, such as a simple distance measurement device, or may be embodied as a two-dimensional detector or even as a three-dimensional detector. Further, as outlined above or as outlined in further detail below, by scanning a scenery or an environment in a one-dimensional fashion, a three-dimensional image may also be created. Consequently, the detector according to the present invention specifically may be one of a one-dimensional detector, a two-dimensional detector or a three-dimensional detector. The evaluation device may further be configured to determine at least one transversal coordinate x, y of the object. The evaluation device may be adapted to combine the information of the longitudinal coordinate and the transversal coordinate and to determine a position of the object in space.
As outlined above, the detector may further comprise one or more additional elements such as one or more additional optical elements. Further, the detector may fully or partially be integrated into at least one housing. The detector specifically may comprise at least one transfer device, the transfer device being adapted to guide the light beam onto the optical sensor. The transfer device may comprise one or more of: at least one lens, preferably at least one focus-tunable lens; at least one beam deflection element, preferably at least one mirror; at least one beam splitting element, preferably at least one of a beam splitting cube or a beam splitting mirror; at least one multi-lens system. Further, if applicable, Fresnel lenses or diffractive optical elements may be used. Therein, generally, the term “transfer device” may generally refer to one or more optical elements which are adapted to modify the light beam, such as by modifying one or more of a beam parameter of the light beam, a width of the light beam or a direction of the light beam. The transfer device may be adapted to guide the light beam onto the optical sensors. The transfer device specifically may comprise one or more of: at least one lens, for example at least one lens selected from the group consisting of, preferably at least one focus-tunable lens, at least one aspheric lens, at least one spheric lens, at least one Fresnel lens; at least one diffractive optical element; at least one concave mirror; at least one beam deflection element, preferably at least one mirror; at least one beam splitting element, preferably at least one of a beam splitting cube or a beam splitting mirror; at least one multi-lens system.
In a further aspect of the present invention, a detector system for determining a position of at least one object is disclosed. The detector system comprises at least one detector according to the present invention, such as according to one or more of the embodiments disclosed above or according to one or more of the embodiments disclosed in further detail below. The detector system further comprises at least one beacon device adapted to direct at least one light beam towards the detector, wherein the beacon device is at least one of attachable to the object, holdable by the object and integratable into the object. Further details regarding the beacon device will be given below, including potential embodiments thereof. Thus, the at least one beacon device may be or may comprise at least one active beacon device, comprising one or more illumination sources such as one or more light sources like lasers, LEDs, light bulbs or the like. As an example, the light emitted by the illumination source may have a wavelength of 300-500 nm. Additionally or alternatively, the at least one beacon device may be adapted to reflect one or more light beams towards the detector, such as by comprising one or more reflective elements. Further, the at least one beacon device may be or may comprise one or more scattering elements adapted for scattering a light beam. Therein, elastic or inelastic scattering may be used. In case the at least one beacon device is adapted to reflect and/or scatter a primary light beam towards the detector, the beacon device may be adapted to leave the spectral properties of the light beam unaffected or, alternatively, may be adapted to change the spectral properties of the light beam, such as by modifying a wavelength of the light beam.
In a further aspect of the present invention, a human-machine interface for exchanging at least one item of information between a user and a machine is disclosed. The human-machine interface comprises at least one detector system according to the embodiments disclosed above and/or according to one or more of the embodiments disclosed in further detail below. Therein, the at least one beacon device is adapted to be at least one of directly or indirectly attached to the user or held by the user. The human-machine interface is designed to determine at least one position of the user by means of the detector system, wherein the human-machine interface is designed to assign to the position at least one item of information.
In a further aspect of the present invention, an entertainment device for carrying out at least one entertainment function is disclosed. The entertainment device comprises at least one human-machine interface according to the embodiment disclosed above and/or according to one or more of the embodiments disclosed in further detail below. The entertainment device is configured to enable at least one item of information to be input by a player by means of the human-machine interface. The entertainment device is further configured to vary the entertainment function in accordance with the information.
In a further aspect of the present invention, a tracking system for tracking a position of at least one movable object is disclosed. The tracking system comprises at least one detector system according to one or more of the embodiments referring to a detector system as disclosed above and/or as disclosed in further detail below. The tracking system further comprises at least one track controller. The track controller is adapted to track a series of positions of the object at specific points in time.
In a further aspect of the present invention, a camera for imaging at least one object is disclosed. The camera comprises at least one detector according to any one of the embodiments referring to a detector as disclosed above or as disclosed in further detail below.
In a further aspect of the present invention, a readout device for optical storage media is proposed. The readout device comprises at least one detector according to any one of the preceding embodiments referring to a detector. As used therein, a readout device for optical storage media generally refers to a device which is capable of optically retrieving information stored in optical storage media such as optical storage discs, e.g., CCD, DVD or Blu-ray discs. Thus, the above-described measurement principle of the detector according to the present invention may be used for detecting data modules within an optical storage medium such as in optical storage discs. As an example, in case a reflective data module is present and reflects the illuminating light beam, the detector will not only detect the reflected light beam according to the above-mentioned measurement principle but will also detect a distance between the detector and the reflective data module, i.e., a depth of the reflective data module within the optical storage medium. Thus, as an example, the detector may be used for detecting different layers of information modules or data modules within the optical storage medium. Thereby, as an example, two layer discs or three layer discs or even discs having more than three layers may be generated and read out.
In a further aspect of the present invention, a scanning system for determining a depth profile of a scenery, which may also imply determining at least one position of at least one object, is provided. The scanning system comprises at least one detector according to the present invention, such as at least one detector as disclosed in one or more of the embodiments listed above and/or as disclosed in one or more of the embodiments below. The scanning system further comprises at least one illumination source adapted to scan the scenery with at least one light beam, which may also be referred to as an illumination light beam or scanning light beam. As used herein, the term “scenery” generally refers to a two-dimensional or three-dimensional range which is visible by the detector, such that at least one geometric or spatial property of the two-dimensional or three-dimensional range may be evaluated with the detector. As further used herein, the term “scan” generally refers to a consecutive measurement in different regions. Thus, the scanning specifically may imply at least one first measurement with the illumination light beam being oriented or directed in a first fashion, and at least one second measurement with the illumination light beam being oriented or directed in a second fashion which is different from the first fashion. The scanning may be a continuous scanning or a stepwise scanning. Thus, in a continuous or stepwise fashion, the illumination light beam may be directed into different regions of the scenery, and the detector may be detected to generate at least one item of information, such as at least one longitudinal coordinate, for each region. As an example, for scanning an object, one or more illumination light beams may, continuously or in a stepwise fashion, create light spots on the surface of the object, wherein longitudinal coordinates are generated for the light spots. Alternatively, however, a light pattern may be used for scanning. The scanning may be a point scanning or a line scanning or even a scanning with more complex light patterns. The illumination source of the scanning system may be distinct from the optional illumination source of the detector. Alternatively, however, the illumination source of the scanning system may also be fully or partially identical with or integrated into the at least one optional illumination source of the detector.
Thus, the scanning system may comprise at least one illumination source which is adapted to emit the at least one light beam being configured for the illumination of the at least one dot located at the at least one surface of the at least one object. As used herein, the term “dot” refers to an area, specifically a small area, on a part of the surface of the object which may be selected, for example by a user of the scanning system, to be illuminated by the illumination source. Preferably, the dot may exhibit a size which may, on one hand, be as small as possible in order to allow the scanning system to determine a value for the distance between the illumination source comprised by the scanning system and the part of the surface of the object on which the dot may be located as exactly as possible and which, on the other hand, may be as large as possible in order to allow the user of the scanning system or the scanning system itself, in particular by an automatic procedure, to detect a presence of the dot on the related part of the surface of the object.
For this purpose, the illumination source may comprise an artificial illumination source, in particular at least one laser source and/or at least one incandescent lamp and/or at least one semiconductor light source, for example, at least one light-emitting diode, in particular an organic and/or inorganic light-emitting diode. As an example, the light emitted by the illumination source may have a wavelength of 300-500 nm. On account of their generally defined beam profiles and other properties of handleability, the use of at least one laser source as the illumination source is particularly preferred. Herein, the use of a single laser source may be preferred, in particular in a case in which it may be important to provide a compact scanning system that might be easily storable and transportable by the user. The illumination source may thus, preferably be a constituent part of the detector and may, therefore, in particular be integrated into the detector, such as into the housing of the detector. In a preferred embodiment, particularly the housing of the scanning system may comprise at least one display configured for providing distance-related information to the user, such as in an easy-to-read manner. In a further preferred embodiment, particularly the housing of the scanning system may, in addition, comprise at least one button which may be configured for operating at least one function related to the scanning system, such as for setting one or more operation modes. In a further preferred embodiment, particularly the housing of the scanning system may, in addition, comprise at least one fastening unit which may be configured for fastening the scanning system to a further surface, such as a rubber foot, a base plate or a wall holder, such as a base plate or holder comprising a magnetic material, in particular for increasing the accuracy of the distance measurement and/or the handleability of the scanning system by the user.
Particularly, the illumination source of the scanning system may, thus, emit a single laser beam which may be configured for the illumination of a single dot located at the surface of the object. By using at least one of the detectors according to the present invention, at least one item of information about the distance between the at least one dot and the scanning system may, thus, be generated. Hereby, preferably, the distance between the illumination system as comprised by the scanning system and the single dot as generated by the illumination source may be determined, such as by employing the evaluation device as comprised by the at least one detector. However, the scanning system may, further, comprise an additional evaluation system which may, particularly, be adapted for this purpose. Alternatively or in addition, a size of the scanning system, in particular of the housing of the scanning system, may be taken into account and, thus, the distance between a specific point on the housing of the scanning system, such as a front edge or a back edge of the housing, and the single dot may, alternatively, be determined.
Alternatively, the illumination source of the scanning system may emit two individual laser beams which may be configured for providing a respective angle, such as a right angle, between the directions of an emission of the beams, whereby two respective dots located at the surface of the same object or at two different surfaces at two separate objects may be illuminated. However, other values for the respective angle between the two individual laser beams may also be feasible. This feature may, in particular, be employed for indirect measuring functions, such as for deriving an indirect distance which may not be directly accessible, such as due to a presence of one or more obstacles between the scanning system and the dot or which may otherwise be hard to reach. By way of example, it may, thus, be feasible to determine a value for a height of an object by measuring two individual distances and deriving the height by using the Pythagoras formula. In particular for being able to keep a predefined level with respect to the object, the scanning system may, further, comprise at least one leveling unit, in particular an integrated bubble vial, which may be used for keeping the predefined level by the user.
As a further alternative, the illumination source of the scanning system may emit a plurality of individual laser beams, such as an array of laser beams which may exhibit a respective pitch, in particular a regular pitch, with respect to each other and which may be arranged in a manner in order to generate an array of dots located on the at least one surface of the at least one object. For this purpose, specially adapted optical elements, such as beam-splitting devices and mirrors, may be provided which may allow a generation of the described array of the laser beams. In particular, the illumination source may be directed to scan an area or a volume by using one or more movable mirrors to redirect the light beam in a periodic or non-periodic fashion.
Thus, the scanning system may provide a static arrangement of the one or more dots placed on the one or more surfaces of the one or more objects. Alternatively, the illumination source of the scanning system, in particular the one or more laser beams, such as the above described array of the laser beams, may be configured for providing one or more light beams which may exhibit a varying intensity over time and/or which may be subject to an alternating direction of emission in a passage of time, in particular by moving one or more mirrors, such as the micro-mirrors comprised within the mentioned array of micro-mirrors. As a result, the illumination source may be configured for scanning a part of the at least one surface of the at least one object as an image by using one or more light beams with alternating features as generated by the at least one illumination source of the scanning device. In particular, the scanning system may, thus, use at least one row scan and/or line scan, such as to scan the one or more surfaces of the one or more objects sequentially or simultaneously. As non-limiting examples, the scanning system may be used in safety laser scanners, e.g., in production environments, and/or in 3D-scanning devices as used for determining the shape of an object, such as in connection to 3D-printing, body scanning, quality control, in construction applications, e.g., as range meters, in logistics applications, e.g., for determining the size or volume of a parcel, in household applications, e.g., in robotic vacuum cleaners or lawn mowers, or in other kinds of applications which may include a scanning step. As non-limiting examples, the scanning system may be used in industrial safety curtain applications. As non-limiting examples, the scanning system may be used to perform sweeping, vacuuming, mopping, or waxing functions, or yard or garden care functions such as mowing or raking. As non-limiting examples, the scanning system may employ an LED illumination source with collimated optics and may be adapted to shift the frequency of the illumination source to a different frequency to obtain more accurate results and/or employ a filter to attenuate certain frequencies while transmitting others. As non-limiting examples, the scanning system and/or the illumination source may be rotated as a whole or rotating only a particular optics package such as a mirror, beam splitter or the like, using a dedicated motor as such that in operation, the scanning system may have a full 360 degree view or even be moved and or rotated out of plane to further increase the scanned area. Further, the illumination source may be actively aimed in a predetermined direction. Further, to allow the rotation of wired electrical systems, slip rings, optical data transmission, or inductive couplings may be employed.
As a non-limiting example, the scanning system may be attached to a tripod and point towards an object or region with a several corners and surfaces. One or more flexibly movable laser sources may be attached to the scanning system. The one or more laser sources may be moved as such that they illuminate points of interest. The position of the illuminated points with respect to the scanning system may be measured when pressing a designated button on the scanning system and the position information is transmitted via a wireless interface to a mobile phone. The position information may be stored in a mobile phone application. The laser sources may be moved to illuminate further points of interest the position of which are measured and transmitted to the mobile phone application. The mobile phone application may transform the set of points into a 3d model by connecting adjacent points with planar surfaces. The 3d model may be stored and processed further. The distances and/or angles between the measured points or surfaces may be displayed directly on a display attached to a scanning system or on the mobile phone to which the position information is transmitted.
As a non-limiting example, a scanning system may comprise two or more flexible movable laser sources to project points and further one movable laser source projecting a line. The line may be used to arrange the two or more laser spots along a line and the display of the scanning device may display the distance between the two or more laser spots that may be arranged along the line, such as at equal distance. In the case of two laser spots, a single laser source may be used whereas the distance of the projected points is modified using one or more beam-splitters or prisms, where a beam-splitter or prism can be moved as such that the projected laser spots move apart or closer together. Further, the scanning system may be adapted to project further patterns such as a right angle, a circle, a square, a triangle, or the like, along which a measurement can be done by projecting laser spots and measuring their position.
As a non-limiting example, the scanning system may be adapted to support the work with tools, such as wood or metal processing tools, such as a saw, a driller, or the like. Thus, the scanning system may be adapted to measure the distance in two opposite directions and display the two measured distances or the sum of the distances in a display. Further, the scanning system may be adapted to measure the distance to the edge of a surface as such that when the scanning system is placed on the surface, a laser point is moved automatically away from the scanning system along the surface, until the distance measurement shows a sudden change due to a corner or the edge of a surface. This makes it possible to measure the distance of the end of a wood plank while the scanning device is placed on the plank but remote from its end. Further, the scanning system may measure the distance of the end of a plank in one direction and project a line or circle or point in a designated distance in the opposite direction. The scanning system may be adapted to project the line or circle or point in a distance depending on the distance measured in the opposite direction such as depending on a predetermined sum distance. This allows working with a tool such as a saw or driller at the projected position while placing the scanning system in a safe distance from the tool and simultaneously perform a process using the tool in a predetermined distance to the edge of the plank. Further, the scanning system may be adapted to project points or lines or the like in two opposite directions in a predetermined distance. When the sum of the distances is changed, only one of the projected distances changes.
As a non-limiting example, the scanning system may be adapted to be placed onto a surface, such as a surface on which a task is performed, such as cutting, sawing, drilling, or the like, and to project a line onto the surface in a predetermined distance that can be adjusted such as with buttons on the scanning device.
As non-limiting examples, the scanning system may be used in safety laser scanners, e.g., in production environments, and/or in 3D-scanning devices as used for determining the shape of an object, such as in connection to 3D-printing, body scanning, quality control, in construction applications, e.g., as range meters, in logistics applications, e.g., for determining the size or volume of a parcel, in household applications, e.g., in robotic vacuum cleaners or lawn mowers, or in other kinds of applications which may include a scanning step.
The optional transfer device can, as explained above, be designed to feed light propagating from the object to the detector to the optical sensor, preferably successively. As explained above, this feeding can optionally be effected by means of imaging or else by means of non-imaging properties of the transfer device. In particular the transfer device can also be designed to collect the electromagnetic radiation before the latter is fed to the optical sensor. The optional transfer device can also be wholly or partly a constituent part of at least one optional illumination source, for example by the illumination source being designed to provide a light beam having defined optical properties, for example having a defined or precisely known beam profile, for example at least one Gaussian beam, in particular at least one laser beam having a known beam profile.
For potential embodiments of the optional illumination source, reference may be made to WO 2012/110924 A1. Still, other embodiments are feasible. Light emerging from the object can originate in the object itself, but can also optionally have a different origin and propagate from this origin to the object and subsequently toward the transversal and/or longitudinal optical sensor. The latter case can be effected for example by at least one illumination source being used. This illumination source can for example be or comprise an ambient illumination source and/or may be or may comprise an artificial illumination source. By way of example, the detector itself can comprise at least one illumination source, for example at least one laser and/or at least one incandescent lamp and/or at least one semiconductor illumination source, for example, at least one light-emitting diode, in particular an organic and/or inorganic light-emitting diode. On account of their generally defined beam profiles and other properties of handleability, the use of one or a plurality of lasers as illumination source or as part thereof, is particularly preferred. The illumination source itself can be a constituent part of the detector or else be formed independently of the detector. The illumination source can be integrated in particular into the detector, for example a housing of the detector. Alternatively or additionally, at least one illumination source can also be integrated into the at least one beacon device or into one or more of the beacon devices and/or into the object or connected or spatially coupled to the object.
The light emerging from the one or more optional beacon devices can accordingly, alternatively or additionally from the option that said light originates in the respective beacon device itself, emerge from the illumination source and/or be excited by the illumination source. By way of example, the electromagnetic light emerging from the beacon device can be emitted by the beacon device itself and/or be reflected by the beacon device and/or be scattered by the beacon device before it is fed to the detector. In this case, emission and/or scattering of the electromagnetic radiation can be effected without spectral influencing of the electromagnetic radiation or with such influencing. Thus, by way of example, a wavelength shift can also occur during scattering, for example according to Stokes or Raman. Furthermore, emission of light can be excited, for example, by a primary illumination source, for example by the object or a partial region of the object being excited to generate luminescence, in particular phosphorescence and/or fluorescence. Other emission processes are also possible, in principle. If a reflection occurs, then the object can have for example at least one reflective region, in particular at least one reflective surface. Said reflective surface can be a part of the object itself, but can also be for example a reflector which is connected or spatially coupled to the object, for example a reflector plaque connected to the object. If at least one reflector is used, then it can in turn also be regarded as part of the detector which is connected to the object, for example, independently of other constituent parts of the detector.
The beacon devices and/or the at least one optional illumination source generally may emit light in at least one of: the ultraviolet spectral range, preferably in the range of 200 nm to 380 nm; the visible spectral range (380 nm to 780 nm); the infrared spectral range, preferably in the range of 780 nm to 3.0 micrometers. For thermal imaging applications the target may emit light in the far infrared spectral range, preferably in the range of 3.0 micrometers to 20 micrometers. Most preferably, the at least one illumination source is adapted to emit light in the visible spectral range, preferably in the range of 500 nm to 780 nm, most preferably at 650 nm to 750 nm or at 690 nm to 700 nm.
The feeding of the light beam to the optical sensor can be effected in particular in such a way that a light spot, for example having a round, oval or differently configured cross section, is produced on the optional sensor area of the optical sensor. By way of example, the detector can have a visual range, in particular a solid angle range and/or spatial range, within which objects can be detected. Preferably, the optional transfer device is designed in such a way that the light spot, for example in the case of an object arranged within a visual range of the detector, is arranged completely on a sensor region and/or on a sensor area of the optical sensor. By way of example, a sensor area can be chosen to have a corresponding size in order to ensure this condition.
In a further aspect, the present invention discloses an inertial measurement unit for use in an electronic device. The electronic device may be a mobile electronic device. The electronic de-vice may be a camera. The electronic device may be a mobile phone. The inertial measurement unit is adapted to receive data determined by at least one inertial measurement unit comprising at least one detector according to the present invention, such as according to one or more of the embodiments referring to a detector as disclosed above or as disclosed in further detail below. As used herein, the term “data determined by the at least one detector” refers to at least one information on the at least one longitudinal coordinate z. The inertial measurement unit further is adapted to receive data determined by at least one further sensor selected from the group consisting of: a wheel speed sensor, a turn rate sensor, an inclination sensor, an orientation sensor, a motion sensor, a magneto hydro dynamic sensor, a force sensor, an angular sensor, an angular rate sensor, a magnetic field sensor, a magnetometer, an accelerometer; a gyroscope. As used herein, the term “data determined by the at least one further sensor” refers to at least one information selected from the group consisting of: angle information; speed information; information on turn rate; information on inclination. The inertial measurement unit is adapted to determine by evaluating the data from the detector and the at least one further sensor at least one property of the electronic device selected from the group consisting of: position in space, relative or absolute motion in space, rotation, acceleration, orientation, angle position, inclination, turn rate, speed. The inertial measurement unit may comprise at least one processor. The processor may be adapted to evaluate data recorded by the further sensor. In particular, the processor may be adapted to determine and/or calculate one or more of spatial position, spatial orientation, movement and velocity. The inertial measurement unit may comprise a plurality of further sensors. The inertial measurement unit may be adapted to fuse information determined from at least two of the further sensors. The inertial measurement unit may be adapted to fuse information of at least two further sensors by using at least one Kalman filter. As outlined above, the detector may be adapted to provide an absolute measurement of the longitudinal coordinate z. The processor, e.g., the evaluation device as described above, may be adapted to fuse the information of the at least two further sensors considering the longitudinal coordinate z. The various sensor signals may be used within a Kalman filter or a linear quadratic estimated to take into account that each sensor signal is subject to measurement errors and inaccuracies. Fusion of these sensor signals within a Kalman filter may yield an improved estimate such as for the measurement of the longitudinal coordinate.
In a further aspect, the present invention discloses a method for determining a position of at least one object by using a detector, such as a detector according to the present invention, such as according to one or more of the embodiments referring to a detector as disclosed above or as disclosed in further detail below. Still, other types of detectors may be used.
The method comprises the following method steps, wherein the method steps may be performed in the given order or may be performed in a different order. Further, one or more additional method steps may be present which are not listed. Further, one, more than one or even all of the method steps may be performed repeatedly.
The method steps are as follows:
Specifically, evaluating the first and second sensor signal may comprise deriving a quotient signal Q by one or more of dividing the sensor signals, dividing multiples of the sensor signals or dividing linear combinations of the sensor signals, and wherein the determining of the longitudinal coordinate comprises evaluating the quotient signal Q. Further, the determining of the longitudinal coordinate may comprise evaluating the quotient signal Q.
For details, options and definitions, reference may be made to the detector as discussed above. Thus, specifically, as outlined above, the method may comprise using the detector according to the present invention, such as according to one or more of the embodiments given above or given in further detail below.
In a further aspect of the present invention, use of the detector according to the present invention, such as according to one or more of the embodiments given above or given in further detail below, is proposed, for a purpose of use, selected from the group consisting of: a position measurement in traffic technology; an entertainment application; an optical data storage application; a security application; a surveillance application; a safety application; a human-machine interface application; a tracking application; a photography application; a machine vision application; a robotics application; a quality control application; a manufacturing application.
The object generally may be a living or non-living object. The detector or the detector system even may comprise the at least one object, the object thereby forming part of the detector system. Preferably, however, the object may move independently from the detector, in at least one spatial dimension. The object generally may be an arbitrary object. In one embodiment, the object may be a rigid object. Other embodiments are feasible, such as embodiments in which the object is a non-rigid object or an object which may change its shape.
As will be outlined in further detail below, the present invention may specifically be used for tracking positions and/or motions of a person, such as for the purpose of controlling machines, gaming or simulation of sports. In this or other embodiments, specifically, the object may be selected from the group consisting of: an article of sports equipment, preferably an article selected from the group consisting of a racket, a club, a bat; an article of clothing; a hat; a shoe.
Thus, generally, the devices according to the present invention, such as the detector, may be applied in various fields of uses. Specifically, the detector may be applied for a purpose of use, selected from the group consisting of: a position measurement in traffic technology; an entertainment application; a security application; a human-machine interface application; a tracking application; a photography application; a mapping application for generating maps of at least one space, such as at least one space selected from the group of a room, a building and a street; a mobile application; a webcam; an audio device; a dolby surround audio system; a computer peripheral device; a gaming application; a camera or video application; a security application; a surveillance application; an automotive application; a transport application; a medical application; a sports' application; a machine vision application; a vehicle application; an airplane application; a ship application; a spacecraft application; a building application; a construction application; a cartography application; a manufacturing application. Additionally or alternatively, applications in local and/or global positioning systems may be named, especially landmark-based positioning and/or navigation, specifically for use in cars or other vehicles (such as trains, motorcycles, bicycles, trucks for cargo transportation), robots or for use by pedestrians. Further, indoor positioning systems may be named as potential applications, such as for household applications and/or for robots used in manufacturing, logistics, surveillance, or maintenance technology.
The devices according to the present invention may be used in mobile phones, tablet computers, laptops, smart panels or other stationary or mobile or wearable computer or communication applications. Thus, the devices according to the present invention may be combined with at least one active light source, such as a light source emitting light in the visible range or infrared spectral range, in order to enhance performance. Thus, as an example, the devices according to the present invention may be used as cameras and/or sensors, such as in combination with mobile software for scanning and/or detecting environment, objects and living beings. The devices according to the present invention may even be combined with 2D cameras, such as conventional cameras, in order to increase imaging effects. The devices according to the present invention may further be used for surveillance and/or for recording purposes or as input devices to control mobile devices, especially in combination with voice and/or gesture recognition. Thus, specifically, the devices according to the present invention acting as human-machine interfaces, also referred to as input devices, may be used in mobile applications, such as for controlling other electronic devices or components via the mobile device, such as the mobile phone. As an example, the mobile application including at least one device according to the present invention may be used for controlling a television set, a game console, a music player or music device or other entertainment devices.
Further, the devices according to the present invention may be used in webcams or other peripheral devices for computing applications. Thus, as an example, the devices according to the present invention may be used in combination with software for imaging, recording, surveillance, scanning or motion detection. As outlined in the context of the human-machine interface and/or the entertainment device, the devices according to the present invention are particularly useful for giving commands by facial expressions and/or body expressions. The devices according to the present invention can be combined with other input generating devices like e.g., mouse, keyboard, touchpad, microphone etc. Further, the devices according to the present invention may be used in applications for gaming, such as by using a webcam. Further, the devices according to the present invention may be used in virtual training applications and/or video conferences. Further, devices according to the present invention may be used to recognize or track hands, arms, or objects used in a virtual or augmented reality application, especially when wearing head mounted displays.
Further, the devices according to the present invention may be used in mobile audio devices, television devices and gaming devices, as partially explained above. Specifically, the devices according to the present invention may be used as controls or control devices for electronic devices, entertainment devices or the like. Further, the devices according to the present invention may be used for eye detection or eye tracking, such as in 2D- and 3D-display techniques, especially with transparent displays for augmented reality applications and/or for recognizing whether a display is being looked at and/or from which perspective a display is being looked at. Further, devices according to the present invention may be used to explore a room, boundaries, obstacles, in connection with a virtual or augmented reality application, especially when wearing a head-mounted display.
Further, the devices according to the present invention may be used in or as digital cameras such as DSC cameras and/or in or as reflex cameras such as SLR cameras. For these applications, reference may be made to the use of the devices according to the present invention in mobile applications such as mobile phones, as disclosed above.
Further, the devices according to the present invention may be used for security or surveillance applications. Thus, as an example, at least one device according to the present invention can be combined with one or more digital and/or analogue electronics that will give a signal if an object is within or outside a predetermined area (e.g., for surveillance applications in banks or museums). Specifically, the devices according to the present invention may be used for optical encryption. Detection by using at least one device according to the present invention can be combined with other detection devices to complement wavelengths, such as with IR, x-ray, UV-VIS, radar or ultrasound detectors. The devices according to the present invention may further be combined with an active infrared light source to allow detection in low light surroundings. The devices according to the present invention are generally advantageous as compared to active detector systems, specifically since the devices according to the present invention avoid actively sending signals which may be detected by third parties, as is the case e.g., in radar applications, ultrasound applications, LIDAR or similar active detector devices. Thus, generally, the devices according to the present invention may be used for an unrecognized and undetectable tracking of moving objects.
Additionally, the devices according to the present invention generally are less prone to manipulations and irritations as compared to conventional devices.
Further, given the ease and accuracy of 3D detection by using the devices according to the present invention, the devices according to the present invention generally may be used for facial, body and person recognition and identification. Therein, the devices according to the present invention may be combined with other detection means for identification or personalization purposes such as passwords, finger prints, iris detection, voice recognition or other means. Thus, generally, the devices according to the present invention may be used in security devices and other personalized applications.
Further, the devices according to the present invention may be used as 3D barcode readers for product identification.
In addition to the security and surveillance applications mentioned above, the devices according to the present invention generally can be used for surveillance and monitoring of spaces and areas. Thus, the devices according to the present invention may be used for surveying and monitoring spaces and areas and, as an example, for triggering or executing alarms in case prohibited areas are violated. Thus, generally, the devices according to the present invention may be used for surveillance purposes in building surveillance or museums, optionally in combination with other types of sensors, such as in combination with motion or heat sensors, in combination with image intensifiers or image enhancement devices and/or photomultipliers. Further, the devices according to the present invention may be used in public spaces or crowded spaces to detect potentially hazardous activities such as commitment of crimes such as theft in a parking lot or unattended objects such as unattended baggage in an airport.
Further, the devices according to the present invention may advantageously be applied in camera applications such as video and camcorder applications. Thus, the devices according to the present invention may be used for motion capture and 3D-movie recording. Therein, the devices according to the present invention generally provide a large number of advantages over conventional optical devices. Thus, the devices according to the present invention generally require a lower complexity with regard to optical components. Thus, as an example, the number of lenses may be reduced as compared to conventional optical devices, such as by providing the devices according to the present invention having one lens only. Due to the reduced complexity, very compact devices are possible, such as for mobile use. Conventional optical systems having two or more lenses with high quality generally are voluminous, such as due to the general need for voluminous beam-splitters. Further, the devices according to the present invention generally may be used for focus/autofocus devices, such as autofocus cameras. Further, the devices according to the present invention may also be used in optical microscopy, especially in confocal microscopy.
Further, the devices according to the present invention generally are applicable in the technical field of automotive technology and transport technology. Thus, as an example, the devices according to the present invention may be used as distance and surveillance sensors, such as for adaptive cruise control, emergency brake assist, lane departure warning, surround view, blind spot detection, traffic sign detection, traffic sign recognition, lane recognition, rear cross traffic alert, light source recognition for adapting the head light intensity and range depending on approaching traffic or vehicles driving ahead, adaptive frontlighting systems, automatic control of high beam head lights, adaptive cut-off lights in front light systems, glare-free high beam front lighting systems, marking animals, obstacles, or the like by headlight illumination, rear cross traffic alert, and other driver assistance systems such as advanced driver assistance systems, or other automotive and traffic applications. Further, devices according to the present invention may be used in driver assistance systems anticipating maneuvers of the driver beforehand for collision avoidance or the like. Further, the devices according to the present invention can also be used for velocity and/or acceleration measurements, such as by analyzing a first and second time-derivative of position information gained by using the detector according to the present invention. This feature generally may be applicable in automotive technology, transportation technology or general traffic technology. Applications in other fields of technology are feasible. A specific application in an indoor positioning system may be the detection of positioning of passengers in transportation, more specifically to electronically control the use of safety systems such as airbags. The use of an airbag may be prevented in case the passenger is located as such, that the use of an airbag will cause a severe injury. Further, in vehicles such as cars, trains, planes or the like, especially in autonomous vehicles, devices according to the present invention may be used to determine whether a driver pays attention to the traffic or is distracted, or asleep, or tired, or incapable of driving such as due to the consumption of alcohol or the like.
In these or other applications, generally, the devices according to the present invention may be used as standalone devices or in combination with other sensor devices, such as in combination with radar and/or ultrasonic devices. Specifically, the devices according to the present invention may be used for autonomous driving and safety issues. Further, in these applications, the devices according to the present invention may be used in combination with infrared sensors, radar sensors, which are sonic sensors, two-dimensional cameras or other types of sensors. In these applications, the generally passive nature of the devices according to the present invention is advantageous. Thus, since the devices according to the present invention generally do not require emitting signals, the risk of interference of active sensor signals with other signal sources may be avoided. The devices according to the present invention specifically may be used in combination with recognition software, such as standard image recognition software. Thus, signals and data as provided by the devices according to the present invention typically are readily processable and, therefore, generally require lower calculation power than established 3D measurement systems. Given the low space demand, the devices according to the present invention such as cameras may be placed at virtually any place in a vehicle, such as on or behind a window screen, on a front hood, on bumpers, on lights, on mirrors or other places and the like. Various detectors according to the present invention such as one or more detectors based on the effect disclosed within the present invention can be combined, such as in order to allow autonomously driving vehicles or in order to increase the performance of active safety concepts. Thus, various devices according to the present invention may be combined with one or more other devices according to the present invention and/or conventional sensors, such as in the windows like rear window, side window or front window, on the bumpers or on the lights.
A combination of at least one device according to the present invention such as at least one detector according to the present invention with one or more rain detection sensors is also possible. This is due to the fact that the devices according to the present invention generally are advantageous over conventional sensor techniques such as radar, specifically during heavy rain. A combination of at least one device according to the present invention with at least one conventional sensing technique such as radar may allow for a software to pick the right combination of signals according to the weather conditions.
Further, the devices according to the present invention generally may be used as break assist and/or parking assist and/or for speed measurements. Speed measurements can be integrated in the vehicle or may be used outside the vehicle, such as in order to measure the speed of other cars in traffic control. Further, the devices according to the present invention may be used for detecting free parking spaces in parking lots.
Further, the devices according to the present invention may be used in the fields of medical systems and sports. Thus, in the field of medical technology, surgery robotics, e.g., for use in endoscopes, may be named, since, as outlined above, the devices according to the present invention may require a low volume only and may be integrated into other devices. Specifically, the devices according to the present invention having one lens, at most, may be used for capturing 3D information in medical devices such as in endoscopes. Further, the devices according to the present invention may be combined with an appropriate monitoring software, in order to enable tracking and analysis of movements. This may allow an instant overlay of the position of a medical device, such as an endoscope or a scalpel, with results from medical imaging, such as obtained from magnetic resonance imaging, x-ray imaging, or ultrasound imaging. These applications are specifically valuable e.g., in medical treatments where precise location information is important such as in brain surgery and long-distance diagnosis and tele-medicine. Further, the devices according to the present invention may be used in 3D-body scanning. Body scanning may be applied in a medical context, such as in dental surgery, plastic surgery, bariatric surgery, or cosmetic plastic surgery, or it may be applied in the context of medical diagnosis such as in the diagnosis of myofascial pain syndrome, cancer, body dysmorphic disorder, or further diseases. Body scanning may further be applied in the field of sports to assess ergonomic use or fit of sports equipment.
Body scanning may further be used in the context of clothing, such as to determine a suitable size and fitting of clothes. This technology may be used in the context of tailor-made clothes or in the context of ordering clothes or shoes from the internet or at a self-service shopping device such as a micro kiosk device or customer concierge device. Body scanning in the context of clothing is especially important for scanning fully dressed customers.
Further, the devices according to the present invention may be used in the context of people counting systems, such as to count the number of people in an elevator, a train, a bus, a car, or a plane, or to count the number of people passing a hallway, a door, an aisle, a retail store, a stadium, an entertainment venue, a museum, a library, a public location, a cinema, a theater, or the like. Further, the 3D-function in the people counting system may be used to obtain or estimate further information about the people that are counted such as height, weight, age, physical fitness, or the like. This information may be used for business intelligence metrics, and/or for further optimizing the locality where people may be counted to make it more attractive or safe. In a retail environment, the devices according to the present invention in the context of people counting may be used to recognize returning customers or cross shoppers, to assess shopping behavior, to assess the percentage of visitors that make purchases, to optimize staff shifts, or to monitor the costs of a shopping mall per visitor. Further, people counting systems may be used for anthropometric surveys. Further, the devices according to the present invention may be used in public transportation systems for automatically charging passengers depending on the length of transport. Further, the devices according to the present invention may be used in playgrounds for children, to recognize injured children or children engaged in dangerous activities, to allow additional interaction with playground toys, to ensure safe use of playground toys or the like.
Further the devices according to the present invention may be used in construction tools, such as a range meter that determines the distance to an object or to a wall, to assess whether a surface is planar, to align objects or place objects in an ordered manner, or in inspection cameras for use in construction environments or the like.
Further, the devices according to the present invention may be applied in the field of sports and exercising, such as for training, remote instructions or competition purposes. Specifically, the devices according to the present invention may be applied in the fields of dancing, aerobic, football, soccer, basketball, baseball, cricket, hockey, track and field, swimming, polo, handball, volleyball, rugby, sumo, judo, fencing, boxing, golf, car racing, laser tag, battlefield simulation etc. The devices according to the present invention can be used to detect the position of a ball, a bat, a sword, motions, etc., both in sports and in games, such as to monitor the game, support the referee or for judgment, specifically automatic judgment, of specific situations in sports, such as for judging whether a point or a goal actually was made.
Further, the devices according to the present invention may be used in the field of auto racing or car driver training or car safety training or the like to determine the position of a car or the track of a car, or the deviation from a previous track or an ideal track or the like.
The devices according to the present invention may further be used to support a practice of musical instruments, in particular remote lessons, for example lessons of string instruments, such as fiddles, violins, violas, celli, basses, harps, guitars, banjos, or ukuleles, keyboard instruments, such as pianos, organs, keyboards, harpsichords, harmoniums, or accordions, and/or percussion instruments, such as drums, timpani, marimbas, xylophones, vibraphones, bongos, congas, timbales, djembes or tablas.
The devices according to the present invention further may be used in rehabilitation and physiotherapy, in order to encourage training and/or in order to survey and correct movements. Therein, the devices according to the present invention may also be applied for distance diagnostics.
Further, the devices according to the present invention may be applied in the field of machine vision. Thus, one or more of the devices according to the present invention may be used e.g., as a passive controlling unit for autonomous driving and/or working of robots. In combination with moving robots, the devices according to the present invention may allow for autonomous movement and/or autonomous detection of failures in parts. The devices according to the present invention may also be used for manufacturing and safety surveillance, such as in order to avoid accidents including but not limited to collisions between robots, production parts and living beings. In robotics, the safe and direct interaction of humans and robots is often an issue, as robots may severely injure humans when they are not recognized. Devices according to the present invention may help robots to position objects and humans better and faster and allow a safe interaction. Given the passive nature of the devices according to the present invention, the devices according to the present invention may be advantageous over active devices and/or may be used complementary to existing solutions like radar, ultrasound, 2D cameras, IR detection etc. One particular advantage of the devices according to the present invention is the low likelihood of signal interference. Therefore multiple sensors can work at the same time in the same environment, without the risk of signal interference. Thus, the devices according to the present invention generally may be useful in highly automated production environments like e.g., but not limited to automotive, mining, steel, etc. The devices according to the present invention can also be used for quality control in production, e.g., in combination with other sensors like 2-D imaging, radar, ultrasound, IR etc., such as for quality control or other purposes. Further, the devices according to the present invention may be used for assessment of surface quality, such as for surveying the surface evenness of a product or the adherence to specified dimensions, from the range of micrometers to the range of meters. Other quality control applications are feasible. In a manufacturing environment, the devices according to the present invention are especially useful for processing natural products such as food or wood, with a complex 3-dimensional structure to avoid large amounts of waste material. Further, devices according to the present invention may be used to monitor the filling level of tanks, silos etc. Further, devices according to the present invention may be used to inspect complex products for missing parts, incomplete parts, loose parts, low quality parts, or the like, such as in automatic optical inspection, such as of printed circuit boards, inspection of assemblies or sub-assemblies, verification of engineered components, engine part inspections, wood quality inspection, label inspections, inspection of medical devices, inspection of product orientations, packaging inspections, food pack inspections, or the like.
Further, the devices according to the present invention may be used in vehicles, trains, airplanes, ships, spacecraft and other traffic applications. Thus, besides the applications mentioned above in the context of traffic applications, passive tracking systems for aircraft, vehicles and the like may be named. The use of at least one device according to the present invention, such as at least one detector according to the present invention, for monitoring the speed and/or the direction of moving objects is feasible. Specifically, the tracking of fast moving objects on land, sea and in the air including space may be named. The at least one device according to the present invention, such as the at least one detector according to the present invention, specifically may be mounted on a still-standing and/or on a moving device. An output signal of the at least one device according to the present invention can be combined e.g., with a guiding mechanism for autonomous or guided movement of another object. Thus, applications for avoiding collisions or for enabling collisions between the tracked and the steered object are feasible. The devices according to the present invention generally are useful and advantageous due to the low calculation power required, the instant response and due to the passive nature of the detection system which generally is more difficult to detect and to disturb as compared to active systems, like e.g., radar. The devices according to the present invention are particularly useful but not limited to e.g., speed control and air traffic control devices. Further, the devices according to the present invention may be used in automated tolling systems for road charges.
The devices according to the present invention generally may be used in passive applications. Passive applications include guidance for ships in harbors or in dangerous areas, and for aircraft when landing or starting. Therein, fixed, known active targets may be used for precise guidance. The same can be used for vehicles driving on dangerous but well defined routes, such as mining vehicles. Further, the devices according to the present invention may be used to detect rapidly approaching objects, such as cars, trains, flying objects, animals, or the like. Further, the devices according to the present invention can be used for detecting velocities or accelerations of objects, or to predict the movement of an object by tracking one or more of its position, speed, and/or acceleration depending on time.
Further, as outlined above, the devices according to the present invention may be used in the field of gaming. Thus, the devices according to the present invention can be passive for use with multiple objects of the same or of different size, color, shape, etc., such as for movement detection in combination with software that incorporates the movement into its content. In particular, applications are feasible in implementing movements into graphical output. Further, applications of the devices according to the present invention for giving commands are feasible, such as by using one or more of the devices according to the present invention for gesture or facial recognition. The devices according to the present invention may be combined with an active system in order to work under e.g., low light conditions or in other situations in which enhancement of the surrounding conditions is required. Additionally or alternatively, a combination of one or more devices according to the present invention with one or more IR or VIS light sources is possible. A combination of a detector according to the present invention with special devices is also possible, which can be distinguished easily by the system and its software, e.g., and not limited to, a special color, shape, relative position to other devices, speed of movement, light, frequency used to modulate light sources on the device, surface properties, material used, reflection properties, transparency degree, absorption characteristics, etc. The device can, amongst other possibilities, resemble a stick, a racket, a club, a gun, a knife, a wheel, a ring, a steering wheel, a bottle, a ball, a glass, a vase, a spoon, a fork, a cube, a dice, a figure, a puppet, a teddy, a beaker, a pedal, a switch, a glove, jewelry, a musical instrument or an auxiliary device for playing a musical instrument, such as a plectrum, a drumstick or the like. Other options are feasible.
Further, the devices according to the present invention may be used to detect and or track objects that emit light by themselves, such as due to high temperature or further light emission processes. The light emitting part may be an exhaust stream or the like. Further, the devices according to the present invention may be used to track reflecting objects and analyze the rotation or orientation of these objects.
Further, the devices according to the present invention generally may be used in the field of building, construction and cartography. Thus, generally, one or more devices according to the present invention may be used in order to measure and/or monitor environmental areas, e.g., countryside or buildings. Therein, one or more devices according to the present invention may be combined with other methods and devices or can be used solely in order to monitor progress and accuracy of building projects, changing objects, houses, etc. The devices according to the present invention can be used for generating three-dimensional models of scanned environments, in order to construct maps of rooms, streets, houses, communities or landscapes, both from ground or from air. Potential fields of application may be construction, cartography, real estate management, land surveying or the like. As an example, the devices according to the present invention may be used in drones or multicopters or the like to monitor buildings, production sites, chimneys, agricultural production environments such as fields, production plants, or landscapes, to support rescue operations, to support work in dangerous environments, to support fire brigades in a burning location indoors or outdoors, or to find or monitor one or more persons or animals, or the like, or for entertainment purposes, such as a drone following and recording one or more persons doing sports such as skiing or cycling or the like, which could be realized by following a helmet, a mark, a beacon device, or the like. Devices according to the present invention could be used to recognize obstacles, follow a predefined route, follow an edge, a pipe, a building, or the like, or to record a global or local map of the environment. Further, devices according to the present invention could be used for indoor or outdoor localization and positioning of drones, for stabilizing the height of a drone indoors where barometric pressure sensors are not accurate enough, or for the interaction of multiple drones such as concertized movements of several drones or recharging or refueling in the air or the like.
Further, the devices according to the present invention may be used within an interconnecting network of home appliances such as CHAIN (Cedec Home Appliances Interoperating Network) to interconnect, automate, and control basic appliance-related services in a home, e.g., energy or load management, remote diagnostics, pet-related appliances, child-related appliances, child surveillance, appliance related surveillance, support or service to elderly or ill persons, home security and/or surveillance, remote control of appliance operation, and automatic maintenance support. Further, the devices according to the present invention may be used in heating or cooling systems such as an air-conditioning system, to locate which part of the room should be brought to a certain temperature or humidity, especially depending on the location of one or more persons. Further, the devices according to the present invention may be used in domestic robots, such as service or autonomous robots which may be used for household chores. The devices according to the present invention may be used for a number of different purposes, such as to avoid collisions or to map the environment, but also to identify a user, to personalize the robot's performance for a given user, for security purposes, or for gesture or facial recognition. As an example, the devices according to the present invention may be used in robotic vacuum cleaners, floor-washing robots, dry-sweeping robots, ironing robots for ironing clothes, animal litter robots, such as cat litter robots, security robots that detect intruders, robotic lawn mowers, automated pool cleaners, rain gutter cleaning robots, window washing robots, toy robots, telepresence robots, social robots providing company to less mobile people, or robots translating and speech to sign language or sign language to speech. In the context of less mobile people, such as elderly persons, household robots with the devices according to the present invention may be used for picking up objects, transporting objects, and interacting with the objects and the user in a safe way. Further, the devices according to the present invention may be used in robots operating with hazardous materials or objects or in dangerous environments. As a non-limiting example, the devices according to the present invention may be used in robots or unmanned remote-controlled vehicles to operate with hazardous materials such as chemicals or radioactive materials especially after disasters, or with other hazardous or potentially hazardous objects such as mines, unexploded arms, or the like, or to operate in or to investigate insecure environments such as near burning objects or post disaster areas or for manned or unmanned rescue operations in the air, in the sea, underground, or the like.
Further, devices according to the present invention may be used for navigation purposes, where Global Positioning Systems (GPS) are not sufficiently reliable. GPS signals commonly use radio waves that can be blocked or difficult to receive indoors or outdoors in valleys or in forests below the treeline. Further, especially in unmanned autonomous vehicles, the weight of the system may be critical. Especially unmanned autonomous vehicles need high-speed position data for reliable feedback and stability of their control systems. Using devices according to the present invention may allow short time response and positioning without adding weight due to a heavy device.
Further, the devices according to the present invention may be used in household, mobile or entertainment devices, such as a refrigerator, a microwave, a washing machine, a window blind or shutter, a household alarm, an air condition device, a heating device, a television, an audio device, a smart watch, a mobile phone, a phone, a dishwasher, a stove or the like, to detect the presence of a person, to monitor the contents or function of the device, or to interact with the person and/or share information about the person with further household, mobile or entertainment devices.
Further, the devices according to the present invention may be used to support elderly or disabled persons or persons with limited or no vision, such as in household chores or at work such as in devices for holding, carrying, or picking objects, or in a safety system with optical or acoustical signals signaling obstacles in the environment.
The devices according to the present invention may further be used in agriculture, for example to detect and sort out vermin, weeds, and/or infected crop plants, fully or in parts, wherein crop plants may be infected by fungus or insects. Further, for harvesting crops, the devices according to the present invention may be used to detect animals, such as deer, which may otherwise be harmed by harvesting devices. Further, the devices according to the present invention may be used to monitor the growth of plants in a field or greenhouse, in particular to adjust the amount of water or fertilizer or crop protection products for a given region in the field or greenhouse or even for a given plant. Further, in agricultural biotechnology, the devices according to the present invention may be used to monitor the size and shape of plants.
Further, devices according to the present invention may be used to automatically remove weeds such as with mechanical means, such as to avoid the use of herbicides. Further, devices according to the present invention may be used in the field of agriculture, in particular to detect and/or locate specific insects such as to decide whether or not to apply a crop protection or fertilization substance, such as to reduce the amount of applied substance or to protect specific groups of animals such as bees.
Further, the devices according to the present invention may be combined with sensors to detect chemicals or pollutants, electronic nose chips, microbe sensor chips to detect bacteria or viruses or the like, Geiger counters, tactile sensors, heat sensors, or the like. This may for example be used in constructing smart robots which are configured for handling dangerous or difficult tasks, such as in treating highly infectious patients, handling or removing highly dangerous substances, cleaning highly polluted areas, such as highly radioactive areas or chemical spills, or for pest control in agriculture.
One or more devices according to the present invention can further be used for scanning of objects, such as in combination with CAD or similar software, such as for additive manufacturing and/or 3D printing. Therein, use may be made of the high dimensional accuracy of the devices according to the present invention, e.g., in x-, y- or z-direction or in any arbitrary combination of these directions, such as simultaneously. Further, the devices according to the present invention may be used in inspections and maintenance, such as pipeline inspection gauges. Further, in a production environment, the devices according to the present invention may be used to work with objects of a badly defined shape such as naturally grown objects, such as sorting vegetables or other natural products by shape or size or cutting products such as meat or objects that are manufactured with a precision that is lower than the precision needed for a processing step.
Further, the devices according to the present invention may be used in local navigation systems to allow autonomously or partially autonomously moving vehicles or multicopters or the like through an indoor or outdoor space. A non-limiting example may comprise vehicles moving through an automated storage for picking up objects and placing them at a different location. Indoor navigation may further be used in shopping malls, retail stores, museums, airports, or train stations, to track the location of mobile goods, mobile devices, baggage, customers or employees, or to supply users with a location-specific information, such as the current position on a map, or information on goods sold, or the like.
Further, the devices according to the present invention may be used to ensure safe driving of motorcycles such as driving assistance for motorcycles by monitoring speed, inclination, upcoming obstacles, unevenness of the road, or curves or the like. Further, the devices according to the present invention may be used in trains or trams to avoid collisions.
Further, the devices according to the present invention may be used in handheld devices, such as for scanning packaging or parcels to optimize a logistics process. Further, the devices according to the present invention may be used in further handheld devices such as personal shopping devices, RFID-readers, handheld devices for use in hospitals or health environments such as for medical use or to obtain, exchange or record patient or patient health related information, smart badges for retail or health environments, or the like.
As outlined above, the devices according to the present invention may further be used in manufacturing, quality control or identification applications, such as in product identification or size identification (such as for finding an optimal place or package, for reducing waste etc.). Further, the devices according to the present invention may be used in logistics applications. Thus, the devices according to the present invention may be used for optimized loading or packing containers or vehicles. Further, the devices according to the present invention may be used for monitoring or controlling of surface damages in the field of manufacturing, for monitoring or controlling rental objects such as rental vehicles, and/or for insurance applications, such as for assessment of damages. Further, the devices according to the present invention may be used for identifying a size of material, object or tools, such as for optimal material handling, especially in combination with robots. Further, the devices according to the present invention may be used for process control in production, e.g., for observing filling level of tanks. Further, the devices according to the present invention may be used for maintenance of production assets like, but not limited to, tanks, pipes, reactors, tools etc. Further, the devices according to the present invention may be used for analyzing 3D-quality marks. Further, the devices according to the present invention may be used in manufacturing tailor-made goods such as tooth inlays, dental braces, prosthesis, clothes or the like. The devices according to the present invention may also be combined with one or more 3D-printers for rapid prototyping, 3D-copying or the like. Further, the devices according to the present invention may be used for detecting the shape of one or more articles, such as for anti-product piracy and for anti-counterfeiting purposes.
Thus, specifically, the present application may be applied in the field of photography. Thus, the detector may be part of a photographic device, specifically of a digital camera. Specifically, the detector may be used for 3D photography, specifically for digital 3D photography. Thus, the detector may form a digital 3D camera or may be part of a digital 3D camera. As used herein, the term photography generally refers to the technology of acquiring image information of at least one object. As further used herein, a camera generally is a device adapted for performing photography. As further used herein, the term digital photography generally refers to the technology of acquiring image information of at least one object by using a plurality of light-sensitive elements adapted to generate electrical signals indicating an intensity and/or color of illumination, preferably digital electrical signals. As further used herein, the term 3D photography generally refers to the technology of acquiring image information of at least one object in three spatial dimensions. Accordingly, a 3D camera is a device adapted for performing 3D photography. The camera generally may be adapted for acquiring a single image, such as a single 3D image, or may be adapted for acquiring a plurality of images, such as a sequence of images. Thus, the camera may also be a video camera adapted for video applications, such as for acquiring digital video sequences.
Thus, generally, the present invention further refers to a camera, specifically a digital camera, more specifically a 3D camera or digital 3D camera, for imaging at least one object. As outlined above, the term imaging, as used herein, generally refers to acquiring image information of at least one object. The camera comprises at least one detector according to the present invention. The camera, as outlined above, may be adapted for acquiring a single image or for acquiring a plurality of images, such as image sequence, preferably for acquiring digital video sequences. Thus, as an example, the camera may be or may comprise a video camera. In the latter case, the camera preferably comprises a data memory for storing the image sequence.
As used within the present invention, the expression “position” generally refers to at least one item of information regarding one or more of an absolute position and an orientation of one or more points of the object. Thus, specifically, the position may be determined in a coordinate system of the detector, such as in a Cartesian coordinate system. Additionally or alternatively, however, other types of coordinate systems may be used, such as polar coordinate systems and/or spherical coordinate systems.
As outlined above and as will be outlined in further detail below, the present invention preferably may be applied in the field of human-machine interfaces, in the field of sports and/or in the field of computer games. Thus, preferably, the object may be selected from the group consisting of: an article of sports equipment, preferably an article selected from the group consisting of a racket, a club, a bat, an article of clothing, a hat, a shoe. Other embodiments are feasible.
As used herein, the object generally may be an arbitrary object, chosen from a living object and a non-living object. Thus, as an example, the at least one object may comprise one or more articles and/or one or more parts of an article. Additionally or alternatively, the object may be or may comprise one or more living beings and/or one or more parts thereof, such as one or more body parts of a human being, e.g., a user, and/or an animal.
With regard to the coordinate system for determining the position of the object, which may be a coordinate system of the detector, the detector may constitute a coordinate system in which an optical axis of the detector forms the z-axis and in which, additionally, an x-axis and a y-axis may be provided which are perpendicular to the z-axis and which are perpendicular to each other. As an example, the detector and/or a part of the detector may rest at a specific point in this coordinate system, such as at the origin of this coordinate system. In this coordinate system, a direction parallel or antiparallel to the z-axis may be regarded as a longitudinal direction, and a coordinate along the z-axis may be considered a longitudinal coordinate. An arbitrary direction perpendicular to the longitudinal direction may be considered a transversal direction, and an x- and/or y-coordinate may be considered a transversal coordinate.
Alternatively, other types of coordinate systems may be used. Thus, as an example, a polar coordinate system may be used in which the optical axis forms a z-axis and in which a distance from the z-axis and a polar angle may be used as additional coordinates. Again, a direction parallel or antiparallel to the z-axis may be considered a longitudinal direction, and a coordinate along the z-axis may be considered a longitudinal coordinate. Any direction perpendicular to the z-axis may be considered a transversal direction, and the polar coordinate and/or the polar angle may be considered a transversal coordinate.
The detector may be a device configured for providing at least one item of information on the position of the at least one object and/or a part thereof. Thus, the position may refer to an item of information fully describing the position of the object or a part thereof, preferably in the coordinate system of the detector, or may refer to a partial information, which only partially describes the position. The detector generally may be a device adapted for detecting light beams, such as the light beams propagating from the beacon devices towards the detector.
The evaluation device and the detector may fully or partially be integrated into a single device. Thus, generally, the evaluation device also may form part of the detector. Alternatively, the evaluation device and the detector may fully or partially be embodied as separate devices. The detector may comprise further components.
The detector may be a stationary device or a mobile device. Further, the detector may be a stand-alone device or may form part of another device, such as a computer, a vehicle or any other device. Further, the detector may be a hand-held device. Other embodiments of the detector are feasible.
The detector specifically may be used to record a light-field behind a lens or lens system of the detector, comparable to a plenoptic or light-field camera. Thus, specifically, the detector may be embodied as a light-field camera adapted for acquiring images in multiple focal planes, such as simultaneously. The term light-field, as used herein, generally refers to the spatial light propagation of light inside the detector such as inside the camera. The detector according to the present invention, specifically having a stack of optical sensors, may have the capability of directly recording a light-field within the detector or camera, such as behind a lens. The plurality of sensors may record images at different distances from the lens. Using, e.g., convolution-based algorithms such as “depth from focus” or “depth from defocus”, the propagation direction, focus points, and spread of the light behind the lens can be modeled. From the modeled propagation of light behind the lens, images at various distances to the lens can be extracted, the depth of field can be optimized, pictures that are in focus at various distances can be extracted, or distances of objects can be calculated. Further information may be extracted.
The use of several optical sensors further allows for correcting lens errors in an image processing step after recording the images. Optical instruments often become expensive and challenging in construction, when lens errors need to be corrected. These are especially problematic in microscopes and telescopes. In microscopes, a typical lens error is that rays of varying distance to the optical axis are distorted differently (spherical aberration). In telescopes, varying the focus may occur from differing temperatures in the atmosphere. Static errors such as spherical aberration or further errors from production may be corrected by determining the errors in a calibration step and then using a fixed image processing such as fixed set of pixels and sensor, or more involved processing techniques using light propagation information. In cases in which lens errors are strongly time-dependent, i.e., dependent on weather conditions in telescopes, the lens errors may be corrected by using the light propagation behind the lens, calculating extended depth of field images, using depth from focus techniques, and others.
The detector according to the present invention may further allow for color detection. For color detection, a plurality of optical sensors having different spectral properties may be used, and sensor signals of these optical sensors may be compared.
The evaluation device may be or may comprise one or more integrated circuits, such as one or more application-specific integrated circuits (ASICs), and/or one or more data processing devices, such as one or more computers, preferably one or more microcomputers and/or microcontrollers. Additional components may be comprised, such as one or more preprocessing devices and/or data acquisition devices, such as one or more devices for receiving and/or preprocessing of the sensor signals, such as one or more AD-converters and/or one or more filters. Further, the evaluation device may comprise one or more measurement devices, such as one or more measurement devices for measuring electrical currents and/or electrical voltages. Further, the evaluation device may comprise one or more data storage devices. Further, the evaluation device may comprise one or more interfaces, such as one or more wireless interfaces and/or one or more wire-bound interfaces.
The at least one evaluation device may be adapted to perform at least one computer program, such as at least one computer program adapted for performing or supporting one or more or even all of the method steps of the method according to the present invention. As an example, one or more algorithms may be implemented which, by using the sensor signals as input variables, may determine the position of the object.
The evaluation device can be connected to or may comprise at least one further data processing device that may be used for one or more of displaying, visualizing, analyzing, distributing, communicating or further processing of information, such as information obtained by the optical sensor and/or by the evaluation device. The data processing device, as an example, may be connected or incorporate at least one of a display, a projector, a monitor, an LCD, a TFT, a loudspeaker, a multichannel sound system, an LED pattern, or a further visualization device. It may further be connected or incorporate at least one of a communication device or communication interface, a connector or a port, capable of sending encrypted or unencrypted information using one or more of email, text messages, telephone, Bluetooth, Wi-Fi, infrared or internet interfaces, ports or connections. It may further be connected or incorporate at least one of a processor, a graphics processor, a CPU, an Open Multimedia Applications Platform (OMAP™), an integrated circuit, a system on a chip such as products from the Apple A series or the Samsung S3C2 series, a microcontroller or microprocessor, one or more memory blocks such as ROM, RAM, EEPROM, or flash memory, timing sources such as oscillators or phase-locked loops, counter-timers, real-time timers, or power-on reset generators, voltage regulators, power management circuits, or DMA controllers. Individual units may further be connected by buses such as AMBA buses or be integrated in an Internet of Things or Industry 4.0 type network.
The evaluation device and/or the data processing device may be connected by or have further external interfaces or ports such as one or more of serial or parallel interfaces or ports, USB, Centronics Port, FireWire, HDMI, Ethernet, Bluetooth, RFID, Wi-Fi, USART, or SPI, or analogue interfaces or ports such as one or more of ADCs or DACs, or standardized interfaces or ports to further devices such as a 2D-camera device using an RGB-interface such as CameraLink. The evaluation device and/or the data processing device may further be connected by one or more of interprocessor interfaces or ports, FPGA-FPGA-interfaces, or serial or parallel interfaces ports. The evaluation device and the data processing device may further be connected to one or more of an optical disc drive, a CD-RW drive, a DVD+RW drive, a flash drive, a memory card, a disk drive, a hard disk drive, a solid state disk or a solid state hard disk.
The evaluation device and/or the data processing device may be connected by or have one or more further external connectors such as one or more of phone connectors, RCA connectors, VGA connectors, hermaphrodite connectors, USB connectors, HDMI connectors, 8P8C connectors, BCN connectors, IEC 60320 C14 connectors, optical fiber connectors, D-subminiature connectors, RF connectors, coaxial connectors, SCART connectors, XLR connectors, and/or may incorporate at least one suitable socket for one or more of these connectors.
Possible embodiments of a single device incorporating one or more of the detectors according to the present invention, the evaluation device or the data processing device, such as incorporating one or more of the optical sensor, optical systems, evaluation device, communication device, data processing device, interfaces, system on a chip, display devices, or further electronic devices, are: mobile phones, personal computers, tablet PCs, televisions, game consoles or further entertainment devices. In a further embodiment, the 3D-camera functionality which will be outlined in further detail below may be integrated in devices that are available with conventional 2D-digital cameras, without a noticeable difference in the housing or appearance of the device, where the noticeable difference for the user may only be the functionality of obtaining and or processing 3D information.
Specifically, an embodiment incorporating the detector and/or a part thereof such as the evaluation device and/or the data processing device may be: a mobile phone incorporating a display device, a data processing device, the optical sensor, optionally the sensor optics, and the evaluation device, for the functionality of a 3D camera. The detector according to the present invention specifically may be suitable for integration in entertainment devices and/or communication devices such as a mobile phone.
A further embodiment of the present invention may be an incorporation of the detector or a part thereof such as the evaluation device and/or the data processing device in a device for use in automotive, for use in autonomous driving or for use in car safety systems such as Daimler's Intelligent Drive system, wherein, as an example, a device incorporating one or more of the optical sensors, optionally one or more optical systems, the evaluation device, optionally a communication device, optionally a data processing device, optionally one or more interfaces, optionally a system on a chip, optionally one or more display devices, or optionally further electronic devices may be part of a vehicle, a car, a truck, a train, a bicycle, an airplane, a ship, a motorcycle. In automotive applications, the integration of the device into the automotive design may necessitate the integration of the optical sensor, optionally optics, or device at minimal visibility from the exterior or interior. The detector or a part thereof such as the evaluation device and/or the data processing device may be especially suitable for such integration into automotive design.
As used herein, the term light generally refers to electromagnetic radiation in one or more of the visible spectral range, the ultraviolet spectral range and the infrared spectral range. Therein, the term visible spectral range generally refers to a spectral range of 380 nm to 780 nm. The term infrared spectral range generally refers to electromagnetic radiation in the range of 780 nm to 1 mm, preferably in the range of 780 nm to 3.0 micrometers. The term ultraviolet spectral range generally refers to electromagnetic radiation in the range of 1 nm to 380 nm, preferably in the range of 100 nm to 380 nm. Preferably, light as used within the present invention is visible light, i.e., light in the visible spectral range.
The term light beam generally may refer to an amount of light emitted and/or reflected into a specific direction. Thus, the light beam may be a bundle of the light rays having a predetermined extension in a direction perpendicular to a direction of propagation of the light beam. Preferably, the light beams may be or may comprise one or more Gaussian light beams such as a linear combination of Gaussian light beams, which may be characterized by one or more Gaussian beam parameters, such as one or more of a beam waist, a Rayleigh-length or any other beam parameter or combination of beam parameters suited to characterize a development of a beam diameter and/or a beam propagation in space.
The detector according to the present invention may further be combined with one or more other types of sensors or detectors. Thus, the detector may further comprise at least one additional detector. The at least one additional detector may be adapted for detecting at least one parameter, such as at least one of: a parameter of a surrounding environment, such as a temperature and/or a brightness of a surrounding environment; a parameter regarding a position and/or orientation of the detector; a parameter specifying a state of the object to be detected, such as a position of the object, e.g., an absolute position of the object and/or an orientation of the object in space. Thus, generally, the principles of the present invention may be combined with other measurement principles in order to gain additional information and/or in order to verify measurement results or reduce measurement errors or noise.
As outlined above, the human-machine interface may comprise a plurality of beacon devices which are adapted to be at least one of directly or indirectly attached to the user and held by the user. Thus, the beacon devices each may independently be attached to the user by any suitable means, such as by an appropriate fixing device. Additionally or alternatively, the user may hold and/or carry the at least one beacon device or one or more of the beacon devices in his or her hands and/or by wearing the at least one beacon device and/or a garment containing the beacon device on a body part.
The beacon device generally may be an arbitrary device which may be detected by the at least one detector and/or which facilitates detection by the at least one detector. Thus, as outlined above or as will be outlined in further detail below, the beacon device may be an active beacon device adapted for generating the at least one light beam to be detected by the detector, such as by having one or more illumination sources for generating the at least one light beam. Additionally or alternatively, the beacon device may fully or partially be designed as a passive beacon device, such as by providing one or more reflective elements adapted to reflect a light beam generated by a separate illumination source. The at least one beacon device may permanently or temporarily be attached to the user in a direct or indirect way and/or may be carried or held by the user. The attachment may take place by using one or more attachment means and/or by the user himself or herself, such as by the user holding the at least one beacon device by hand and/or by the user wearing the beacon device.
Additionally or alternatively, the beacon devices may be at least one of attached to an object and integrated into an object held by the user, which, in the sense of the present invention, shall be included into the meaning of the option of the user holding the beacon devices. Thus, as will be outlined in further detail below, the beacon devices may be attached to or integrated into a control element which may be part of the human-machine interface and which may be held or carried by the user, and of which the orientation may be recognized by the detector device. Thus, generally, the present invention also refers to a detector system comprising at least one detector device according to the present invention and which, further, may comprise at least one object, wherein the beacon devices are one of attached to the object, held by the object and integrated into the object. As an example, the object preferably may form a control element, the orientation of which may be recognized by a user. Thus, the detector system may be part of the human-machine interface as outlined above or as outlined in further detail below. As an example, the user may handle the control element in a specific way in order to transmit one or more items of information to a machine, such as in order to transmit one or more commands to the machine.
Alternatively, the detector system may be used in other ways. Thus, as an example, the object of the detector system may be different from a user or a body part of the user and, as an example, may be an object which moves independently from the user. As an example, the detector system may be used for controlling apparatuses and/or industrial processes, such as manufacturing processes and/or robotics processes. Thus, as an example, the object may be a machine and/or a machine part, such as a robot arm, the orientation of which may be detected by using the detector system.
The human-machine interface may be adapted in such a way that the detector device generates at least one item of information on the position of the user or of at least one body part of the user. Specifically in case a manner of attachment of the at least one beacon device to the user is known, by evaluating the position of the at least one beacon device, at least one item of information on a position and/or an orientation of the user or of a body part of the user may be gained.
The beacon device preferably is one of a beacon device attachable to a body or a body part of the user and a beacon device which may be held by the user. As outlined above, the beacon device may fully or partially be designed as an active beacon device. Thus, the beacon device may comprise at least one illumination source adapted to generate at least one light beam to be transmitted to the detector, preferably at least one light beam having known beam properties. Additionally or alternatively, the beacon device may comprise at least one reflector adapted to reflect light generated by an illumination source, thereby generating a reflected light beam to be transmitted to the detector.
The object, which may form part of the detector system, may generally have an arbitrary shape. Preferably, the object being part of the detector system, as outlined above, may be a control element which may be handled by a user, such as manually. As an example, the control element may be or may comprise at least one element selected from the group consisting of: a glove, a jacket, a hat, shoes, trousers and a suit, a stick that may be held by hand, a bat, a club, a racket, a cane, a toy, such as a toy gun. Thus, as an example, the detector system may be part of the human-machine interface and/or of the entertainment device.
As used herein, an entertainment device is a device which may serve the purpose of leisure and/or entertainment of one or more users, in the following also referred to as one or more players. As an example, the entertainment device may serve the purpose of gaming, preferably computer gaming. Thus, the entertainment device may be implemented into a computer, a computer network or a computer system or may comprise a computer, a computer network or a computer system which runs one or more gaming software programs.
The entertainment device comprises at least one human-machine interface according to the present invention, such as according to one or more of the embodiments disclosed above and/or according to one or more of the embodiments disclosed below. The entertainment device is designed to enable at least one item of information to be input by a player by means of the human-machine interface. The at least one item of information may be transmitted to and/or may be used by a controller and/or a computer of the entertainment device. The at least one item of information preferably may comprise at least one command adapted for influencing the course of a game. Thus, as an example, the at least one item of information may include at least one item of information on at least one orientation of the player and/or of one or more body parts of the player, thereby allowing for the player to simulate a specific position and/or orientation and/or action required for gaming. As an example, one or more of the following movements may be simulated and communicated to a controller and/or a computer of the entertainment device: dancing; running; jumping; swinging of a racket; swinging of a bat; swinging of a club; pointing of an object towards another object, such as pointing of a toy gun towards a target.
The entertainment device as a part or as a whole, preferably a controller and/or a computer of the entertainment device, is designed to vary the entertainment function in accordance with the information. Thus, as outlined above, a course of a game might be influenced in accordance with the at least one item of information. Thus, the entertainment device might include one or more controllers which might be separate from the evaluation device of the at least one detector and/or which might be fully or partially identical to the at least one evaluation device or which might even include the at least one evaluation device. Preferably, the at least one controller might include one or more data processing devices, such as one or more computers and/or microcontrollers.
As further used herein, a tracking system is a device which is adapted to gather information on a series of past positions of the at least one object and/or at least one part of the object. Additionally, the tracking system may be adapted to provide information on at least one predicted future position and/or orientation of the at least one object or the at least one part of the object. The tracking system may have at least one track controller, which may fully or partially be embodied as an electronic device, preferably as at least one data processing device, more preferably as at least one computer or microcontroller. Again, the at least one track controller may fully or partially comprise the at least one evaluation device and/or may be part of the at least one evaluation device and/or may fully or partially be identical to the at least one evaluation device.
The tracking system comprises at least one detector according to the present invention, such as at least one detector as disclosed in one or more of the embodiments listed above and/or as disclosed in one or more of the embodiments below. The tracking system further comprises at least one track controller. The track controller is adapted to track a series of positions of the object at specific points in time, such as by recording groups of data or data pairs, each group of data or data pair comprising at least one position information and at least one time information.
The tracking system may further comprise the at least one detector system according to the present invention. Thus, besides the at least one detector and the at least one evaluation device and the optional at least one beacon device, the tracking system may further comprise the object itself or a part of the object, such as at least one control element comprising the beacon devices or at least one beacon device, wherein the control element is directly or indirectly attachable to or integratable into the object to be tracked.
The tracking system may be adapted to initiate one or more actions of the tracking system itself and/or of one or more separate devices. For the latter purpose, the tracking system, preferably the track controller, may have one or more wireless and/or wire-bound interfaces and/or other types of control connections for initiating at least one action. Preferably, the at least one track controller may be adapted to initiate at least one action in accordance with at least one actual position of the object. As an example, the action may be selected from the group consisting of: a prediction of a future position of the object; pointing at least one device towards the object; pointing at least one device towards the detector; illuminating the object; illuminating the detector.
As an example of application of a tracking system, the tracking system may be used for continuously pointing at least one first object to at least one second object even though the first object and/or the second object might move. Potential examples, again, may be found in industrial applications, such as in robotics and/or for continuously working on an article even though the article is moving, such as during manufacturing in a manufacturing line or assembly line. Additionally or alternatively, the tracking system might be used for illumination purposes, such as for continuously illuminating the object by continuously pointing an illumination source to the object even though the object might be moving. Further applications might be found in communication systems, such as in order to continuously transmit information to a moving object by pointing a transmitter towards the moving object. The proposed devices and methods provide a large number of advantages over known detectors of this kind. Thus, the detector generally may avoid the shortcomings of the known prior art systems disclosed above. Specifically, the detector may avoid the use of FiP sensors, thereby allowing for e.g., using simple and cheap and commercially available semiconductor sensors such as silicon photodiodes. These photodiodes generally do not show a luminance dependency, and the method disclosed above is generally independent from the brightness of the scenery and/or the brightness of the light spot on the light beam. Consequently, a range of measurement in terms of luminance or total power of the light beam entering the detector is generally larger in the present invention as compared to many of the devices disclosed above. Further, the measurement by using the detector according to the present invention is generally independent from the target spot size, i.e., from either the size of the object, the size of a light spot projected onto the object or the size of a beacon device being one or more of attached to the object, integrated into the object or held by the object.
The detector according to the present invention may be realized as a simple device combining the functionality of distance measurement or measurement of z-coordinates, with the additional option of measuring transversal coordinates, thereby integrating the functionality of a PSD.
The result of the measurement, i.e., the position determined by the detector such as the at least one longitudinal coordinate, may be rendered widely independent from the brightness of the scenery and/or the brightness of the object, the brightness of the at least one Beacon device, the brightness of the at least one illumination source or the total power of the light beam propagating from the object to the detector. Further, due to this independence and wide range of measurement in terms of brightness, reflective objects or non-reflective objects may be used.
When mentioning a range of measurement, the range of measurement may both refer to a range of brightness which may be used with the detector according to the present invention, such as a range of total powers of the light beam, or may refer to a range of distances between the detector and the object which may be measured. Conventional detectors, such as according to one or more of the documents listed above, are typically limited in both ranges of measurement. The use of the quotient signal, as mentioned above, contrarily, provides a wide range of a continuously and monotonically decreasing or increasing function which may be used to determine the longitudinal coordinate from the quotient signal. Consequently, a very wide range of measurement in terms of distance between the object and the detector is given. Similarly, due to the general independence of the quotient signal from the total power of the light beam, at least as long as no saturation of one or both of the optical sensors is reached, also a very wide range of measurement in terms of brightness may be provided, i.e., in terms of total power of the light beam.
Overall, in the context of the present invention, the following embodiments are regarded as preferred:
Embodiment 1: A detector for determining a position of at least one object, the detector having
Embodiment 2: The detector according to the preceding embodiment, wherein the optical sensors are located in the same plane.
Embodiment 3: The detector according to any one of the preceding embodiments, wherein the optical sensors are partial diodes of a segmented diode, with a center of the segmented diode being off-centered from the optical axis of the detector.
Embodiment 4: The detector according to any one of the preceding embodiments, wherein the optical sensors are part of a sensor array, such as an array comprising 2-4 optical sensors such as a quadrant photodiode.
Embodiment 5: The detector according to the preceding embodiment, wherein a geometrical center of the sensor array is offset from the optical axis.
Embodiment 6: The detector according to any one of the two preceding embodiments, wherein the sensor array is movable, preferably automatically, relative to the optical axis.
Embodiment 7: The detector according to the preceding embodiment, wherein the evaluation device is configured for, firstly, determining the at least one transversal position of a light spot generated by the light beam on the sensor array by using the sensor signals and for, secondly, moving the sensor array relative to the optical axis until the light spot is off-centered.
Embodiment 8: The detector according to any one of the preceding embodiments, wherein the optical sensors are partial diodes of a quadrant diode, with a geometrical center of the quadrant diode being off-centered from the optical axis of the detector.
Embodiment 9: The detector according to any one of the preceding embodiments, wherein the light-sensitive areas of the optical sensors are equal.
Embodiment 10: The detector according to any one of the preceding embodiments, wherein the detector comprises at least one lens, wherein the optical axis of the detector is an axis of symmetry of the lens.
Embodiment 11: The detector according to any one of the preceding embodiments, wherein the evaluation device is configured for deriving the quotient signal Q by
wherein x and y are transversal coordinates, A1 and A2 are areas of the beam profile at the sensor position, and E(x,y,zo) denotes the beam profile given at the object distance zo.
Embodiment 12: The detector according to the preceding embodiment, wherein the light-sensitive areas are arranged such that a first sensor signal comprises information of a first area of the beam profiles and a second sensor signal comprises information of a second area of the beam profile, wherein the first area of the beam profile and the second area of the beam profile are one or both of adjacent or overlapping regions.
Embodiment 13: The detector according to the preceding embodiment, wherein the evaluation device is configured to determine the first area of the beam profile and the second area of the beam profile, wherein the first area of the beam profile comprises essentially edge information of the beam profile and the second area of the beam profile comprises essentially center information of the beam profile, wherein the edge information comprises information relating to a number of photons in the first area of the beam profile and the center information comprises information relating to a number of photons in the second area of the beam profile.
Embodiment 14: The detector according to any one of the three preceding embodiments, wherein the light beam propagating from the object to the detector illuminates the optical sensors with at least one line pattern, wherein A1 corresponds to an area with a full line width of the line pattern on the optical sensors, wherein A2 is a central area of the line pattern on the optical sensors.
Embodiment 15: The detector according to any one of the three preceding embodiments, wherein the light beam propagating from the object to the detector illuminates the sensor element with at least one point pattern, wherein A1 corresponds to an area with a full radius of a point of the point pattern on the optical sensors, wherein A2 is a central area of the point on the point pattern on the optical sensors.
Embodiment 16: The detector according to any one of the preceding embodiments, wherein the evaluation device is configured for deriving a quotient signal Q by one or more of dividing the sensor signals, dividing multiples of the sensor signals or dividing linear combinations of the sensor signals, and wherein the evaluation device is further configured for determining the longitudinal coordinate by evaluating the quotient signal Q.
Embodiment 17: The detector according to the preceding embodiment, wherein the evaluation device is configured for using at least one predetermined relationship between the quotient signal Q and the longitudinal coordinate.
Embodiment 18: The detector according to the preceding embodiment, wherein the predetermined relationship is one or more of an empiric relationship, a semi-empiric relationship and an analytically derived relationship.
Embodiment 19: The detector according to any one of the two preceding embodiments, wherein the evaluation device comprises at least one data storage device for storing the predetermined relationship.
Embodiment 20: The detector according to any one of the four preceding embodiments, wherein the evaluation device comprises at least one divider, wherein the divider is configured for deriving the quotient signal.
Embodiment 21: The detector according to the preceding embodiment, wherein the divider is fully or partially embodied as one or both of a software divider or a hardware divider.
Embodiment 22: The detector according to any one of the preceding embodiments, wherein the optical sensors are arranged in one and the same beam path of the detector.
Embodiment 23: The detector according to any one of the preceding embodiments, wherein the optical sensors each are semiconductor sensors, preferably inorganic semiconductor sensors, more preferably photodiodes and most preferably silicon photodiodes.
Embodiment 24: The detector according to any one of the preceding embodiments, wherein the optical sensors each are uniform sensors having a single light-sensitive area each.
Embodiment 25: The detector according to any one of the preceding embodiments, wherein the optical sensors each have, at least within a range of measurement, a linear signal characteristic such that the sensor signals are dependent on the total power of illumination of the respective optical sensor and are independent from a diameter of a light spot of the illumination.
Embodiment 26: The detector according to any one of the preceding embodiments, wherein the detector further comprises an illumination source for illuminating the object, e.g, an illumination source comprising at least one laser, such as an illumination source providing at least one illuminating light beam being on the optical axis or being not on the optical axis.
Embodiment 27: The detector according to the preceding embodiment, wherein the illumination source is configured for generating an illuminating light beam for illuminating the object, wherein the detector is configured such that the illuminating light beam propagates from the detector towards the object along an optical axis of the detector.
Embodiment 28: The detector according to the preceding embodiment, wherein the detector comprises at least one reflective element, preferably at least one prism, for deflecting the illuminating light beam onto the optical axis.
Embodiment 29: The detector according to any one of the three preceding embodiments, wherein the illumination source is adapted to generate at least one illumination pattern for illumination of the object, wherein the illumination pattern comprises at least one pattern selected from the group consisting of: at least one point pattern; at least one pattern comprising at least one pre-known feature; at least one line pattern comprising at least one line; at least one line pattern comprising at least two lines such as parallel or crossing lines.
Embodiment 30: The detector according to any one of the preceding embodiments, wherein the detector further comprises at least one transfer device, the transfer device being adapted to guide the light beam onto the optical sensors.
Embodiment 31: The detector according to the preceding embodiment, wherein the transfer device comprises one or more of: at least one lens, preferably at least one focus-tunable lens; at least one beam deflection element, preferably at least one mirror; at least one diffractive optical element; at least one Fresnel lens; at least one beam splitting element, preferably at least one of a beam splitting cube or a beam splitting mirror; at least one multi-lens system.
Embodiment 32: The detector according to any one of the preceding embodiments, wherein the detector comprises the at least one illumination source, wherein the illumination source is adapted to illuminate the object with at least one illumination light beam, wherein the illumination source is arranged such that a direction of propagation of the illumination light beam is essentially parallel to the optical axis.
Embodiment 33: The detector according to any one of the preceding embodiments, wherein the detector comprises the at least one illumination source, wherein the illumination source and the optical axis are separated by a small baseline.
Embodiment 34: A detector system for determining a position of at least one object, the detector system comprising at least one detector according to any one of the preceding embodiments, the detector system further comprising at least one beacon device adapted to direct at least one light beam towards the detector, wherein the beacon device is at least one of attachable to the object, holdable by the object and integratable into the object.
Embodiment 35: A human-machine interface for exchanging at least one item of information between a user and a machine, wherein the human-machine interface comprises at least one detector system according to the preceding embodiment, wherein the at least one beacon device is adapted to be at least one of directly or indirectly attached to the user and held by the user, wherein the human-machine interface is designed to determine at least one position of the user by means of the detector system, wherein the human-machine interface is designed to assign to the position at least one item of information.
Embodiment 36: An entertainment device for carrying out at least one entertainment function, wherein the entertainment device comprises at least one human-machine interface according to the preceding embodiment, wherein the entertainment device is designed to enable at least one item of information to be input by a player by means of the human-machine interface, wherein the entertainment device is designed to vary the entertainment function in accordance with the information.
Embodiment 37: A tracking system for tracking a position of at least one movable object, the tracking system comprising at least one detector system according to any one of the preceding embodiments referring to a detector system, the tracking system further comprising at least one track controller, wherein the track controller is adapted to track a series of positions of the object at specific points in time.
Embodiment 38: A scanning system for determining a depth profile of a scenery, the scanning system comprising at least one detector according to any of the preceding embodiments referring to a detector, the scanning system further comprising at least one illumination source adapted to scan the scenery with at least one light beam.
Embodiment 39: A camera for imaging at least one object, the camera comprising at least one detector according to any one of the preceding embodiments referring to a detector.
Embodiment 40: A readout device for optical storage media, the readout device comprising at least one detector according to any one of the preceding embodiments referring to a detector.
Embodiment 41: A method for determining a position of at least one object by using at least one detector, the method comprising the following steps:
Embodiment 42: The method according to the preceding embodiment, wherein evaluating of the sensor signals comprises deriving a quotient signal Q by one or more of dividing the sensor signals, dividing multiples of the sensor signals or dividing linear combinations of the sensor signals, and wherein the determining of the longitudinal coordinate comprises evaluating the quotient signal Q.
Embodiment 43: A use of the detector according to any one of the preceding embodiments relating to a detector, for a purpose of use, selected from the group consisting of: a position measurement in traffic technology; an entertainment application; an optical data storage application; a security application; a surveillance application; a safety application; a human-machine interface application; a tracking application; a photography application; a logistics application; a machine vision application; a robotics application; a quality control application; a manufacturing application; a use for calibration of a distance measurement; a use in combination with optical data storage and readout.
Further optional details and features of the invention are evident from the description of preferred exemplary embodiments which follows in conjunction with the dependent claims. In this context, the particular features may be implemented in an isolated fashion or in combination with other features. The invention is not restricted to the exemplary embodiments. The exemplary embodiments are shown schematically in the figures. Identical reference numerals in the individual figures refer to identical elements or elements with identical function, or elements which correspond to one another with regard to their functions.
Specifically, in the figures:
In
The light beam 116, as an example, may propagate along an optical axis 126 of the detector 110. Other embodiments, however, are feasible. The optical detector 110, further, may comprise at least one transfer device 128, such as at least one lens or a lens system, specifically for beam shaping. Consequently, the light beam 116 is focused, such as in one or more focal points 130, and a beam width of the light beam 116 may depend on a longitudinal coordinate z of the object 112, such as on a distance between the detector 110 and the beacon device 114 and/or the object 112. For details of this beam width dependency on the longitudinal coordinate, reference may be made to one or more of the above-mentioned prior art documents, such as to one or more of WO 2012/110924 A1 and/or WO 2014/097181 A1.
As can be seen in
As will be shown in further detail below, the detector 110 may be configured for automatically establishing the off-centered position of the light spot 186 on the array 174. For this purpose, firstly, the detector 110 may be configured for determining whether the sensor signals generated by the optical sensors 176 of the array 174 are equal. If this should be the case, the detector 110 may be configured to determine that the light spot 186 is centered in the array 174 and, consequently, may shift the light spot 186 out of the geometrical center 180 of the array 174, such as by shifting the whole array 174 in a plane perpendicular to the optical axis 126. For this purpose, as will be shown in further detail below with respect to
Turning back to the setup of
The sensor signals s1 and s2 are provided to an evaluation device 132 of the detector 110. The evaluation device 110, as symbolically shown in
The detector 110, in combination with the at least one beacon device 114, may be referred to as a detector system 134, as will be explained in further detail below with reference to
In
The setup shown in
The light beam 116, specifically, may travel along the optical axis 126 of the detector 110. As shown in
Apart from these modifications, the setup of the embodiment in
It shall further be noted that the embodiments shown in
As outlined above, an exemplary embodiment of the detector 110 which may be used in the setup of
In
The detector 110 as symbolically shown in the exemplary embodiment of
In the exemplary embodiment shown in
As outlined above, the detector 110 comprises at least the optical sensors 176, including at least the first optical sensor 118 and the second optical sensor 120. The optical sensors 176 may be located inside the housing 158 of the detector 110. Further, the at least one transfer device 128 may be comprised, such as one or more optical systems, preferably comprising one or more lenses.
An opening 164 inside the housing 158, which, preferably, is located concentrically with regard to the optical axis 126 of the detector 110, preferably defines a direction of view 166 of the detector 110. A coordinate system 168 may be defined, in which a direction parallel or anti-parallel to the optical axis 126 may be defined as a longitudinal direction, whereas directions perpendicular to the optical axis 126 may be defined as transversal directions. In the coordinate system 128, symbolically depicted in
The detector 110 may comprise the optical sensors 118, 120 as well as, optionally, further optical sensors. The optical sensors 118, 120 preferably are located in one and the same beam path, one behind the other, such that the first optical sensor 118 covers a portion of the second optical sensor 120. Alternatively, however, a branched beam path may be possible, with additional optical sensors in one or more additional beam paths, such as by branching off a beam path for at least one transversal detector or transversal sensor for determining transversal coordinates of the object 112 and/or of parts thereof.
One or more light beams 116 are propagating from the object 112 and/or from one or more of the beacon devices 114, towards the detector 110. The detector 110 is configured for determining a position of the at least one object 112. For this purpose, as explained above in the context of
As outlined above, the determination of the position of the object 112 and/or a part thereof by using the detector 110 may be used for providing a human-machine interface 148, in order to provide at least one item of information to a machine 170. In the embodiments schematically depicted in
As outlined above,
Similarly, as outlined above, the human-machine interface 148 may form part of an entertainment device 150. The machine 170, specifically the computer, may also form part of the entertainment device 150. Thus, by means of the user 162 functioning as the object 112 and/or by means of the user 162 handling a control device 160 functioning as the object 112, the user 162 may input at least one item of information, such as at least one control command, into the computer, thereby varying the entertainment functions, such as controlling the course of a computer game.
In the setup of the detectors 110 as shown in
As can be seen, in this alternative setup, two or more optical sensors 176 are present, comprising at least one first optical sensor 118 and at least one second optical sensor 120 located in different planes which are offset in a direction of the optical axis 126, also referred to as the z-direction. Further, as can also be seen, the optical sensors 118, 120 may overlap, whereas in the previous embodiments, preferably, no overlap between the optical sensors 176 is given. Apart from these modifications, the functionality and the evaluation of the sensor signals generally corresponds to the embodiment of
As discussed above, for evaluating the at least two sensor signals of the at least two optical sensors 176 and for deriving an information on the longitudinal position of the object 112 thereof, such as a distance between the detector 110 and the object 112 and/or a z-coordinate of the object 112, preferably, at least one combined sensor signal is generated by the evaluation device 132. The combined sensor signal, as long as this combined sensor signal provides, at least over a measurement range, a unique function of the distance, may be used for deriving the longitudinal coordinate. As an example, the combined sensor signal may be or may comprise at least one quotient signal Q. In
In all experiments, a setup as shown in
In
In
In
The experiments shown in
In
In
Contrarily, in
In
Consequently, by this movement of the light spot 186, the z-dependency of a combined sensor signal taking into account at least two sensor signals of the optical sensors 176 may be increased. As an example, the four diodes of the array 174, in
As shown in
In the situation shown in
Contrarily, in the situation of
FIG. 9D shows experimental results of the comparison of the two experimental setups using a detector setup according to
In
Alternatively to using a mask 119,
The geometrical centers of the first optical sensor 118 and the second optical sensor 120 are spaced apart from the optical axis 126 of the detector 110.
Referring to
wherein x and y are transversal coordinates, A1 and A2 are areas of the beam profile at the sensor position, and E(x,y,zo) denotes the beam profile given at the object distance zo. A1 may correspond to a full or complete area of a feature point on the optical sensors. A2 may be a central area of the feature point on the optical sensors 118, 120. The central area may be a constant value. The central area may be smaller compared to the full area of the feature point. For example, in case of a circular feature point, the central area may have a radius from 0.1 to 0.9 of a full radius of the feature point, preferably from 0.4 to 0.6 of the full radius.
In the embodiment shown in
| Number | Date | Country | Kind |
|---|---|---|---|
| 16199397 | Nov 2016 | EP | regional |
| 16199398 | Nov 2016 | EP | regional |
| 16199399 | Nov 2016 | EP | regional |
| 17161334 | Mar 2017 | EP | regional |
| 17195398 | Oct 2017 | EP | regional |
| 17195427 | Oct 2017 | EP | regional |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/EP2017/079558 | 11/17/2017 | WO |
| Publishing Document | Publishing Date | Country | Kind |
|---|---|---|---|
| WO2018/091638 | 5/24/2018 | WO | A |
| Number | Name | Date | Kind |
|---|---|---|---|
| 3035176 | George et al. | May 1962 | A |
| 3112197 | Neugebauer et al. | Nov 1963 | A |
| 3384752 | Odone | May 1968 | A |
| 3562785 | Craig | Feb 1971 | A |
| 3564268 | Bayne et al. | Feb 1971 | A |
| 3873823 | Northrup et al. | Mar 1975 | A |
| 3937950 | Hosoe et al. | Feb 1976 | A |
| 3954340 | Blomqvist et al. | May 1976 | A |
| 4023033 | Bricot et al. | May 1977 | A |
| 4053240 | Aizawa et al. | Oct 1977 | A |
| 4079247 | Bricot et al. | Mar 1978 | A |
| 4256513 | Yoshida | Mar 1981 | A |
| 4286035 | Nishizima et al. | Aug 1981 | A |
| 4346293 | Fetzer | Aug 1982 | A |
| 4469945 | Hoeberechts et al. | Sep 1984 | A |
| 4524276 | Ohtombe | Jun 1985 | A |
| 4565761 | Katagiri et al. | Jan 1986 | A |
| 4584704 | Ferren | Apr 1986 | A |
| 4593187 | Grotts et al. | Jun 1986 | A |
| 4602158 | Barrett | Jul 1986 | A |
| 4603258 | Sher et al. | Jul 1986 | A |
| 4647193 | Rosenfeld | Mar 1987 | A |
| 4653905 | Farrar et al. | Mar 1987 | A |
| 4675517 | Shiomi | Jun 1987 | A |
| 4675535 | Tsunekawa et al. | Jun 1987 | A |
| 4694172 | Powell et al. | Sep 1987 | A |
| 4760004 | Rochat et al. | Jul 1988 | A |
| 4760151 | Rochat et al. | Jul 1988 | A |
| 4767211 | Munakata et al. | Aug 1988 | A |
| 4773751 | Matsuda et al. | Sep 1988 | A |
| 4927721 | Gratzel et al. | May 1990 | A |
| 4952472 | Baranyi et al. | Aug 1990 | A |
| 5082363 | Nakanishi et al. | Jan 1992 | A |
| 5216476 | Lanckton | Jun 1993 | A |
| 5227985 | Dementhon et al. | Jul 1993 | A |
| 5235377 | Ide et al. | Aug 1993 | A |
| 5291066 | Neugebauer et al. | Mar 1994 | A |
| 5323222 | Kunishige | Jun 1994 | A |
| 5343291 | Ohwada et al. | Aug 1994 | A |
| 5350644 | Graetzel et al. | Sep 1994 | A |
| 5355241 | Kelley | Oct 1994 | A |
| 5375008 | Guerreri | Dec 1994 | A |
| 5512997 | Ogawa | Apr 1996 | A |
| 5576975 | Sasaki et al. | Nov 1996 | A |
| 5581094 | Hara et al. | Dec 1996 | A |
| 5589928 | Babbitt et al. | Dec 1996 | A |
| 5856844 | Batterman et al. | Jan 1999 | A |
| 6061122 | Hoshino et al. | May 2000 | A |
| 6118119 | Ruschin | Sep 2000 | A |
| 6163371 | Kato et al. | Dec 2000 | A |
| 6191881 | Tajima | Feb 2001 | B1 |
| 6229913 | Nayar et al. | May 2001 | B1 |
| 6266142 | Junkins et al. | Jul 2001 | B1 |
| 6359211 | Spitler et al. | Mar 2002 | B1 |
| 6417836 | Kumar et al. | Jul 2002 | B1 |
| 6507392 | Richards et al. | Jan 2003 | B1 |
| 6512233 | Sato et al. | Jan 2003 | B1 |
| 6785028 | Atsuumi et al. | Aug 2004 | B1 |
| 6930297 | Nakamura | Aug 2005 | B1 |
| 6947459 | Kurtz et al. | Sep 2005 | B2 |
| 6995445 | Forrest et al. | Feb 2006 | B2 |
| 7022966 | Gonzo et al. | Apr 2006 | B2 |
| 7049601 | Agano | May 2006 | B2 |
| 7196317 | Meissner et al. | Mar 2007 | B1 |
| 7247851 | Okada et al. | Jul 2007 | B2 |
| 7301608 | Mendenhall et al. | Nov 2007 | B1 |
| 7417716 | Nagasaka et al. | Aug 2008 | B2 |
| 7626569 | Lanier | Dec 2009 | B2 |
| 7677742 | Hillmer et al. | Mar 2010 | B2 |
| 7768498 | Wey | Aug 2010 | B2 |
| 7773070 | Trisnadi et al. | Aug 2010 | B2 |
| 7855778 | Yung et al. | Dec 2010 | B2 |
| 7939932 | Martin | May 2011 | B2 |
| 8013901 | Fukuhara et al. | Sep 2011 | B2 |
| 8019166 | Cheng et al. | Sep 2011 | B2 |
| 8107056 | Riza | Jan 2012 | B1 |
| 8144173 | Baba | Mar 2012 | B2 |
| 8228299 | Maloney et al. | Jul 2012 | B1 |
| 8231809 | Pschirer et al. | Jul 2012 | B2 |
| 8345003 | Trisnadi et al. | Jan 2013 | B1 |
| 8363526 | Hotta et al. | Jan 2013 | B2 |
| 8390793 | Yamaguchi et al. | Mar 2013 | B2 |
| 8411289 | Takahashi | Apr 2013 | B2 |
| 8477580 | Yamamoto et al. | Jul 2013 | B2 |
| 8563855 | Pschirer et al. | Oct 2013 | B2 |
| 8593565 | Shuster | Nov 2013 | B2 |
| 8902354 | Shuster | Dec 2014 | B2 |
| 8908157 | Eisele et al. | Dec 2014 | B2 |
| 9104910 | Huang | Aug 2015 | B2 |
| 9385326 | Wonneberger et al. | Jul 2016 | B2 |
| 9389315 | Bruder et al. | Jul 2016 | B2 |
| 9428518 | Wonneberger et al. | Aug 2016 | B2 |
| 9557856 | Send et al. | Jan 2017 | B2 |
| 9646365 | Hinkel et al. | May 2017 | B1 |
| 9665182 | Send et al. | May 2017 | B2 |
| 9741954 | Bruder et al. | Aug 2017 | B2 |
| 9787899 | Hinkel et al. | Oct 2017 | B1 |
| 9829564 | Bruder et al. | Nov 2017 | B2 |
| 9919999 | Koenemann et al. | Mar 2018 | B2 |
| 9958535 | Send et al. | May 2018 | B2 |
| 9989623 | Send et al. | Jun 2018 | B2 |
| 10012532 | Send et al. | Jul 2018 | B2 |
| 10094927 | Send et al. | Oct 2018 | B2 |
| 10120078 | Bruder et al. | Nov 2018 | B2 |
| 10290817 | Battagliarin et al. | May 2019 | B2 |
| 10412283 | Send et al. | Sep 2019 | B2 |
| 20010025938 | Imai | Oct 2001 | A1 |
| 20020011576 | Cho et al. | Jan 2002 | A1 |
| 20020075471 | Holec | Jun 2002 | A1 |
| 20030017360 | Tai et al. | Jan 2003 | A1 |
| 20030094607 | Guenther et al. | May 2003 | A1 |
| 20030128351 | Schmidt | Jul 2003 | A1 |
| 20030132391 | Agano | Jul 2003 | A1 |
| 20030227635 | Muller | Dec 2003 | A1 |
| 20040178325 | Forrest et al. | Sep 2004 | A1 |
| 20040190117 | Kubaink | Sep 2004 | A1 |
| 20040216625 | Birnstock et al. | Nov 2004 | A1 |
| 20050052120 | Gupta et al. | Mar 2005 | A1 |
| 20050061957 | Kase | Mar 2005 | A1 |
| 20050122308 | Bell et al. | Jun 2005 | A1 |
| 20050184301 | Nagasaka et al. | Aug 2005 | A1 |
| 20050217720 | Rey-Mermet et al. | Oct 2005 | A1 |
| 20050227390 | Shtein et al. | Oct 2005 | A1 |
| 20050227406 | Shtein et al. | Oct 2005 | A1 |
| 20050268957 | Enomoto et al. | Dec 2005 | A1 |
| 20050269616 | Andriessen | Dec 2005 | A1 |
| 20060044546 | Lewin et al. | Mar 2006 | A1 |
| 20060049397 | Pfeiffer et al. | Mar 2006 | A1 |
| 20060065833 | Craig et al. | Mar 2006 | A1 |
| 20060075585 | Krieger et al. | Apr 2006 | A1 |
| 20060082546 | Wey | Apr 2006 | A1 |
| 20070008515 | Otani et al. | Jan 2007 | A1 |
| 20070010924 | Otani et al. | Jan 2007 | A1 |
| 20070046625 | Yee | Mar 2007 | A1 |
| 20070080925 | Radivojevic et al. | Apr 2007 | A1 |
| 20070109558 | Harding | May 2007 | A1 |
| 20070122927 | Li et al. | May 2007 | A1 |
| 20070176165 | Forrest et al. | Aug 2007 | A1 |
| 20070183047 | Phillips et al. | Aug 2007 | A1 |
| 20070206181 | Arenberg et al. | Sep 2007 | A1 |
| 20080013005 | Deane | Jan 2008 | A1 |
| 20080080789 | Marks | Apr 2008 | A1 |
| 20080106746 | Shpunt et al. | May 2008 | A1 |
| 20080118143 | Gordon et al. | May 2008 | A1 |
| 20080130005 | Waslowski et al. | Jun 2008 | A1 |
| 20080157965 | Shahar | Jul 2008 | A1 |
| 20080170750 | Gordon | Jul 2008 | A1 |
| 20080240502 | Freedman et al. | Oct 2008 | A1 |
| 20080259310 | Wada | Oct 2008 | A1 |
| 20080269482 | Pschirer et al. | Oct 2008 | A1 |
| 20080284925 | Han | Nov 2008 | A1 |
| 20080297487 | Hotelling et al. | Dec 2008 | A1 |
| 20090009747 | Wolf | Jan 2009 | A1 |
| 20090046543 | De Hoog et al. | Feb 2009 | A1 |
| 20090066929 | Tropf | Mar 2009 | A1 |
| 20090009701 | Yamaguchi | Apr 2009 | A1 |
| 20090153841 | Ophey et al. | Jun 2009 | A1 |
| 20090185158 | Wolf | Jul 2009 | A1 |
| 20090188547 | Hayashi et al. | Jul 2009 | A1 |
| 20090225319 | Lee | Sep 2009 | A1 |
| 20090231582 | Aebischer | Sep 2009 | A1 |
| 20090322677 | Lee et al. | Dec 2009 | A1 |
| 20100073462 | Lee et al. | Mar 2010 | A1 |
| 20100091263 | Kumagai et al. | Apr 2010 | A1 |
| 20100118123 | Freedman et al. | May 2010 | A1 |
| 20100141927 | Hashimoto et al. | Jun 2010 | A1 |
| 20100141964 | Horsch | Jun 2010 | A1 |
| 20100194942 | Wada | Aug 2010 | A1 |
| 20100231513 | Deliwala | Sep 2010 | A1 |
| 20100258179 | Wieting | Oct 2010 | A1 |
| 20100279458 | Yeh | Nov 2010 | A1 |
| 20100282309 | Pschirer et al. | Nov 2010 | A1 |
| 20100283868 | Clark et al. | Nov 2010 | A1 |
| 20100297405 | Flores et al. | Nov 2010 | A1 |
| 20110032365 | Yett | Feb 2011 | A1 |
| 20110055846 | Perez et al. | Mar 2011 | A1 |
| 20110096319 | Otani et al. | Apr 2011 | A1 |
| 20110099105 | Mennie et al. | Apr 2011 | A1 |
| 20110103215 | Hotta et al. | May 2011 | A1 |
| 20110122287 | Kunishige et al. | May 2011 | A1 |
| 20110123188 | Cardwell et al. | May 2011 | A1 |
| 20110127788 | Nakanishi | Jun 2011 | A1 |
| 20110181553 | Brown et al. | Jul 2011 | A1 |
| 20110194097 | Yamaguchi et al. | Aug 2011 | A1 |
| 20110248151 | Holcombe et al. | Oct 2011 | A1 |
| 20110284756 | Miko et al. | Nov 2011 | A1 |
| 20110286661 | Lee et al. | Nov 2011 | A1 |
| 20110297235 | Bergmann | Dec 2011 | A1 |
| 20110306413 | Bickerstaff et al. | Dec 2011 | A1 |
| 20110317146 | Gu et al. | Dec 2011 | A1 |
| 20120013885 | Yang et al. | Jan 2012 | A1 |
| 20120061587 | Wu | Mar 2012 | A1 |
| 20120062517 | Lai et al. | Mar 2012 | A1 |
| 20120063287 | Yamamoto et al. | Mar 2012 | A1 |
| 20120105690 | Waqas et al. | May 2012 | A1 |
| 20120016241 | Vaillant | Jun 2012 | A1 |
| 20120160298 | Kanamoto et al. | Jun 2012 | A1 |
| 20120206336 | Bruder | Aug 2012 | A1 |
| 20120242867 | Shuster | Sep 2012 | A1 |
| 20120249998 | Eisele et al. | Oct 2012 | A1 |
| 20120250137 | Maxik et al. | Oct 2012 | A1 |
| 20120262365 | Mallinson | Oct 2012 | A1 |
| 20120262696 | Eisele et al. | Oct 2012 | A1 |
| 20120289672 | Kastler et al. | Nov 2012 | A1 |
| 20120293651 | Kawamata et al. | Nov 2012 | A1 |
| 20120320160 | Drazic | Dec 2012 | A1 |
| 20120328906 | Kwon et al. | Dec 2012 | A1 |
| 20130033579 | Wajs | Feb 2013 | A1 |
| 20130076695 | Gomez et al. | Mar 2013 | A1 |
| 20130135604 | Gogolla et al. | May 2013 | A1 |
| 20130201492 | Takahashi | Aug 2013 | A1 |
| 20130222551 | Shamir et al. | Aug 2013 | A1 |
| 20130235390 | Holzapfel et al. | Sep 2013 | A1 |
| 20130242283 | Bailey et al. | Sep 2013 | A1 |
| 20130266210 | Morgan-Mar et al. | Oct 2013 | A1 |
| 20130271818 | Maxik et al. | Oct 2013 | A1 |
| 20130320302 | Park et al. | Dec 2013 | A1 |
| 20140015242 | Forrest | Jan 2014 | A1 |
| 20140043610 | Engel et al. | Feb 2014 | A1 |
| 20140066656 | Bruder et al. | Mar 2014 | A1 |
| 20140078376 | Shuster | Mar 2014 | A1 |
| 20140124782 | Jung et al. | May 2014 | A1 |
| 20140132724 | Choi et al. | May 2014 | A1 |
| 20140209789 | Hu | Jul 2014 | A1 |
| 20140211295 | Maxik et al. | Jul 2014 | A1 |
| 20140217329 | Hayoz et al. | Aug 2014 | A1 |
| 20140233028 | Englund | Aug 2014 | A1 |
| 20140291480 | Bruder et al. | Oct 2014 | A1 |
| 20140347442 | Wang et al. | Nov 2014 | A1 |
| 20140368726 | Gladnick | Dec 2014 | A1 |
| 20150029326 | Backman et al. | Jan 2015 | A1 |
| 20150085166 | Shuster | Mar 2015 | A1 |
| 20150111337 | Welker et al. | Apr 2015 | A1 |
| 20150124241 | Eisele et al. | May 2015 | A1 |
| 20150124268 | Bruder et al. | May 2015 | A1 |
| 20150132887 | Welker et al. | May 2015 | A1 |
| 20150170400 | Seitz et al. | Jun 2015 | A1 |
| 20150286340 | Send et al. | Oct 2015 | A1 |
| 20150346911 | Christiansson | Dec 2015 | A1 |
| 20150372046 | Kim et al. | Dec 2015 | A1 |
| 20160084650 | Hsu et al. | Mar 2016 | A1 |
| 20160099429 | Bruder et al. | Apr 2016 | A1 |
| 20160124074 | Wonneberger et al. | May 2016 | A1 |
| 20160127664 | Bruder et al. | May 2016 | A1 |
| 20160139243 | Send et al. | May 2016 | A1 |
| 20160140786 | Wang et al. | May 2016 | A1 |
| 20160155575 | Yamato et al. | Jun 2016 | A1 |
| 20160177177 | Koenemann et al. | Jun 2016 | A1 |
| 20160211464 | Tanabe et al. | Jul 2016 | A1 |
| 20160218302 | Hermes et al. | Jul 2016 | A1 |
| 20160224110 | Massonneau et al. | Aug 2016 | A1 |
| 20160248021 | Sundarraj et al. | Aug 2016 | A1 |
| 20160255323 | Wajs | Sep 2016 | A1 |
| 20160266257 | Bruder et al. | Sep 2016 | A1 |
| 20160286199 | Wajs et al. | Sep 2016 | A1 |
| 20160320489 | Send et al. | Nov 2016 | A1 |
| 20160364015 | Send et al. | Dec 2016 | A1 |
| 20160377417 | Jovanovski et al. | Dec 2016 | A1 |
| 20170039793 | Send et al. | Feb 2017 | A1 |
| 20170074652 | Send et al. | Mar 2017 | A1 |
| 20170082426 | Bruder et al. | Mar 2017 | A1 |
| 20170082486 | Send et al. | Mar 2017 | A1 |
| 20170123593 | Send et al. | May 2017 | A1 |
| 20170183295 | Koenemann et al. | Jun 2017 | A1 |
| 20170205230 | Send et al. | Jul 2017 | A1 |
| 20170219694 | Send et al. | Aug 2017 | A1 |
| 20170219709 | Send et al. | Aug 2017 | A1 |
| 20170237926 | Bruder et al. | Aug 2017 | A1 |
| 20170250334 | Hermes et al. | Aug 2017 | A1 |
| 20170263868 | Tanabe et al. | Sep 2017 | A1 |
| 20170309828 | Tanabe et al. | Oct 2017 | A1 |
| 20170363465 | Send et al. | Dec 2017 | A1 |
| 20170363741 | Send et al. | Dec 2017 | A1 |
| 20180003993 | Send et al. | Jan 2018 | A1 |
| 20180007343 | Send et al. | Jan 2018 | A1 |
| 20180017679 | Valouch et al. | Jan 2018 | A1 |
| 20180031672 | Bruder et al. | Feb 2018 | A1 |
| 20180044357 | Spielmann et al. | Feb 2018 | A1 |
| 20180067213 | Send et al. | Mar 2018 | A1 |
| 20180136319 | Send et al. | May 2018 | A1 |
| 20180182980 | Lennartz et al. | Jun 2018 | A1 |
| 20180210064 | Send et al. | Jul 2018 | A1 |
| 20180231376 | Send et al. | Aug 2018 | A1 |
| 20180238993 | Send et al. | Aug 2018 | A1 |
| 20180243045 | Franjic et al. | Aug 2018 | A1 |
| 20180249051 | Send et al. | Aug 2018 | A1 |
| 20180276843 | Send et al. | Sep 2018 | A1 |
| 20180329024 | Send et al. | Nov 2018 | A1 |
| 20180356501 | Send et al. | Dec 2018 | A1 |
| 20190129035 | Valouch et al. | May 2019 | A1 |
| 20190129036 | Valouch et al. | May 2019 | A1 |
| 20190140129 | Valouch et al. | May 2019 | A1 |
| 20190157470 | Send et al. | May 2019 | A1 |
| 20190170849 | Hermes et al. | Jun 2019 | A1 |
| 20190172964 | Hermes et al. | Jun 2019 | A1 |
| 20190198206 | Ter Maat et al. | Jun 2019 | A1 |
| 20190277703 | Valouch et al. | Sep 2019 | A1 |
| 20190339356 | Schildknecht et al. | Nov 2019 | A1 |
| 20190353767 | Eberspach et al. | Nov 2019 | A1 |
| 20200000119 | Send et al. | Jan 2020 | A1 |
| Number | Date | Country |
|---|---|---|
| 1065054 | Oct 1979 | CA |
| 2119330 | Sep 1995 | CA |
| 2196563 | Dec 1996 | CA |
| 1270706 | Oct 2000 | CN |
| 1677053 | Oct 2005 | CN |
| 1723564 | Jan 2006 | CN |
| 1777859 | May 2006 | CN |
| 1809801 | Jul 2006 | CN |
| 1894976 | Jan 2007 | CN |
| 1896686 | Jan 2007 | CN |
| 101129074 | Feb 2008 | CN |
| 101290348 | Oct 2008 | CN |
| 101449181 | Jun 2009 | CN |
| 101650173 | Feb 2010 | CN |
| 101655350 | Feb 2010 | CN |
| 101859439 | Oct 2010 | CN |
| 102096962 | Jun 2011 | CN |
| 201897828 | Jul 2011 | CN |
| 102435136 | May 2012 | CN |
| 102506754 | Jun 2012 | CN |
| 102549380 | Jul 2012 | CN |
| 102549381 | Jul 2012 | CN |
| 102737435 | Oct 2012 | CN |
| 102833569 | Dec 2012 | CN |
| 103106411 | May 2013 | CN |
| 103322910 | Sep 2013 | CN |
| 103403494 | Nov 2013 | CN |
| 103492835 | Jan 2014 | CN |
| 103649677 | Mar 2014 | CN |
| 103650478 | Mar 2014 | CN |
| 2 417 854 | Oct 1974 | DE |
| 25 01 124 | Aug 1975 | DE |
| 32 03 613 | Aug 1982 | DE |
| 32 25 372 | Feb 1983 | DE |
| 42 11 875 | Oct 1993 | DE |
| 196 04 856 | Aug 1997 | DE |
| 10146752 | Apr 2002 | DE |
| 10 2005 043 627 | Mar 2007 | DE |
| 10 2005 053 995 | May 2007 | DE |
| 10 2007 037 875 | Feb 2009 | DE |
| 10 2010 042 278 | Apr 2012 | DE |
| 20 2012 009 070 | Jan 2013 | DE |
| 10 2014 108 353 | Dec 2014 | DE |
| 0 112 169 | Jun 1984 | EP |
| 0 185 450 | Jun 1986 | EP |
| 0 309 631 | Apr 1989 | EP |
| 0 754 930 | Jan 1997 | EP |
| 1 176 646 | Jan 2002 | EP |
| 1 191 819 | Mar 2002 | EP |
| 1 330 117 | Jul 2003 | EP |
| 1 373 272 | Jan 2004 | EP |
| 1 667 246 | Jun 2006 | EP |
| 1 832 910 | Sep 2007 | EP |
| 1 947 477 | Jul 2008 | EP |
| 2040097 | Mar 2009 | EP |
| 2 205 657 | Jul 2010 | EP |
| 2 220 141 | Aug 2010 | EP |
| 2 507 286 | Oct 2012 | EP |
| 2 527 866 | Nov 2012 | EP |
| 2 725 617 | Apr 2014 | EP |
| 2 735 542 | May 2014 | EP |
| 2 781 931 | Sep 2014 | EP |
| 2 813 324 | Dec 2014 | EP |
| 2 818 493 | Dec 2014 | EP |
| 15 153 215 | Jan 2015 | EP |
| 2 831 180 | Feb 2015 | EP |
| 15 157 363 | Mar 2015 | EP |
| 15 164 653 | Apr 2015 | EP |
| 2 884 303 | Jun 2015 | EP |
| 15 177 275 | Jul 2015 | EP |
| 15 180 353 | Aug 2015 | EP |
| 15 180 354 | Aug 2015 | EP |
| 15 185 005 | Sep 2015 | EP |
| 15 191 960 | Oct 2015 | EP |
| 15 196 238 | Nov 2015 | EP |
| 15 196 239 | Nov 2015 | EP |
| 15 197 744 | Dec 2015 | EP |
| 15 155 835 | Feb 2016 | EP |
| 16 155 834 | Feb 2016 | EP |
| 16 155 845 | Feb 2016 | EP |
| 3 008 421 | Apr 2016 | EP |
| 3 008 757 | Apr 2016 | EP |
| 3 036 503 | Jun 2016 | EP |
| 3 045 935 | Jul 2016 | EP |
| 2 432 723 | May 2007 | GB |
| S59-50579 | Mar 1984 | JP |
| 59-79805 | May 1984 | JP |
| 61-89501 | May 1986 | JP |
| S61-135280 | Jun 1986 | JP |
| 61-186804 | Aug 1986 | JP |
| 64-17485 | Jan 1989 | JP |
| H02-170004 | Jun 1990 | JP |
| 04-240817 | Aug 1992 | JP |
| 5-48833 | Feb 1993 | JP |
| 05-240640 | Sep 1993 | JP |
| 6-133321 | May 1994 | JP |
| 7-146113 | Jun 1995 | JP |
| 7-318630 | Dec 1995 | JP |
| 8-159714 | Jun 1996 | JP |
| 8-292586 | Nov 1996 | JP |
| 10-26513 | Jan 1998 | JP |
| 10-221064 | Aug 1998 | JP |
| H11-230860 | Aug 1999 | JP |
| 11-257917 | Sep 1999 | JP |
| 11-325825 | Nov 1999 | JP |
| 3110095 | Nov 2000 | JP |
| 2001-516150 | Sep 2001 | JP |
| 2002-176191 | Jun 2002 | JP |
| 2003-307407 | Oct 2003 | JP |
| 2004-508691 | Mar 2004 | JP |
| 2005-509909 | Apr 2005 | JP |
| 2005-189087 | Jul 2005 | JP |
| 2005-241340 | Sep 2005 | JP |
| 2005-296268 | Oct 2005 | JP |
| 2006-514366 | Apr 2006 | JP |
| 2006-337254 | Dec 2006 | JP |
| 2007-521559 | Aug 2007 | JP |
| 2007-530978 | Nov 2007 | JP |
| 2008-522418 | Jun 2008 | JP |
| 2008-164538 | Jul 2008 | JP |
| 2009-257890 | Nov 2009 | JP |
| 2010-081002 | Apr 2010 | JP |
| 2010-218770 | Sep 2010 | JP |
| 2010-531520 | Sep 2010 | JP |
| 2011-503673 | Jan 2011 | JP |
| 2011-027707 | Feb 2011 | JP |
| 2012-522248 | Sep 2012 | JP |
| 2012-229964 | Nov 2012 | JP |
| 2012-231154 | Nov 2012 | JP |
| 2013-051674 | Mar 2013 | JP |
| 2011-40111 | Nov 2011 | TW |
| 99-09603 | Feb 1999 | WO |
| WO 0129576 | Apr 2001 | WO |
| WO 02076988 | Oct 2002 | WO |
| WO 02101838 | Dec 2002 | WO |
| 03012371 | Feb 2003 | WO |
| WO 03098617 | Nov 2003 | WO |
| WO 2004072909 | Aug 2004 | WO |
| WO 2004114112 | Dec 2004 | WO |
| WO 2005106965 | Nov 2005 | WO |
| WO 2007006717 | Jan 2007 | WO |
| WO 2007054470 | May 2007 | WO |
| WO 2008122531 | Oct 2008 | WO |
| WO 2008145172 | Dec 2008 | WO |
| WO 2009013282 | Jan 2009 | WO |
| WO 2009021859 | Feb 2009 | WO |
| WO 2009053291 | Apr 2009 | WO |
| WO 2009058115 | May 2009 | WO |
| WO 2009105801 | Sep 2009 | WO |
| WO 2010-063521 | Jun 2010 | WO |
| WO 2010088032 | Aug 2010 | WO |
| WO 2010094636 | Aug 2010 | WO |
| WO 2010118409 | Oct 2010 | WO |
| WO 2010118450 | Oct 2010 | WO |
| WO 2011067192 | Jun 2011 | WO |
| WO 2011083722 | Jul 2011 | WO |
| WO 2011091967 | Aug 2011 | WO |
| WO 2012001628 | Jan 2012 | WO |
| WO 2012046181 | Apr 2012 | WO |
| WO 2012049038 | Apr 2012 | WO |
| WO 2012085803 | Jun 2012 | WO |
| WO 2012091814 | Jul 2012 | WO |
| WO 2012110924 | Aug 2012 | WO |
| WO 2012115593 | Aug 2012 | WO |
| WO 2012139354 | Oct 2012 | WO |
| 2012152812 | Nov 2012 | WO |
| 2012168395 | Dec 2012 | WO |
| WO 2006134370 | Dec 2012 | WO |
| WO 2013009676 | Jan 2013 | WO |
| WO 2013090960 | Jun 2013 | WO |
| WO 2013091016 | Jun 2013 | WO |
| 2013118037 | Aug 2013 | WO |
| WO 2013116883 | Aug 2013 | WO |
| WO 2013144177 | Oct 2013 | WO |
| WO 2013156101 | Oct 2013 | WO |
| 2013170982 | Nov 2013 | WO |
| 2014086722 | Jun 2014 | WO |
| 2014097489 | Jun 2014 | WO |
| WO 2014097181 | Jun 2014 | WO |
| WO 2014123522 | Aug 2014 | WO |
| 2014178923 | Nov 2014 | WO |
| 2014198623 | Dec 2014 | WO |
| 2014198625 | Dec 2014 | WO |
| 2014198626 | Dec 2014 | WO |
| 2014198629 | Dec 2014 | WO |
| 2015024871 | Feb 2015 | WO |
| WO 2015081362 | Jun 2015 | WO |
| WO 2015091607 | Jun 2015 | WO |
| WO 2015150203 | Oct 2015 | WO |
| WO 2015150324 | Oct 2015 | WO |
| WO 2015150989 | Oct 2015 | WO |
| WO 2015159192 | Oct 2015 | WO |
| WO 2015161989 | Oct 2015 | WO |
| WO 20151162528 | Oct 2015 | WO |
| WO 2015176981 | Nov 2015 | WO |
| WO 2015177784 | Nov 2015 | WO |
| 2015193804 | Dec 2015 | WO |
| 2016005893 | Jan 2016 | WO |
| WO 2016012274 | Jan 2016 | WO |
| WO 2016046034 | Mar 2016 | WO |
| WO 2016046350 | Mar 2016 | WO |
| 2016051323 | Apr 2016 | WO |
| WO 2016066494 | May 2016 | WO |
| 2016092449 | Jun 2016 | WO |
| 2016092454 | Jun 2016 | WO |
| WO 2016083914 | Jun 2016 | WO |
| WO 2016120392 | Aug 2016 | WO |
| WO 2016146725 | Sep 2016 | WO |
| WO 2017089553 | Jun 2017 | WO |
| WO 2017144401 | Aug 2017 | WO |
| WO 2018077868 | May 2018 | WO |
| WO 2018091640 | May 2018 | WO |
| WO 2018096083 | May 2018 | WO |
| WO 2018115073 | Jun 2018 | WO |
| WO 2018146138 | Aug 2018 | WO |
| WO 2018146146 | Aug 2018 | WO |
| WO 2018167215 | Sep 2018 | WO |
| WO 2018193045 | Oct 2018 | WO |
| WO 2019002199 | Jan 2019 | WO |
| WO 2019011803 | Jan 2019 | WO |
| WO 2019038354 | Feb 2019 | WO |
| WO 2019042956 | Mar 2019 | WO |
| WO 2019042959 | Mar 2019 | WO |
| WO 2019072965 | Apr 2019 | WO |
| WO 2019096986 | May 2019 | WO |
| WO 2019115594 | Jun 2019 | WO |
| WO 2019115595 | Jun 2019 | WO |
| WO 2019115596 | Jun 2019 | WO |
| Entry |
|---|
| EP2040097 Horsch et al Description Generated Jun. 22, 2022. 31 pp. (Year: 2022). |
| Ikeoka, et al., “Real-Time Depth Estimation with Wide Detectable Range Using Horizontal Planes of Sharp Focus Proceedings”, ACIVS 2011: Advanced Concepts for Intelligent Vision Systems, pp. 669-680 (with English Abstract) (https://link.springer.com/chapter/10.1007/978-3-642-23687-7_60). |
| U.S. Appl. No. 16/484,369, filed Aug. 7, 2019, Lungenschmied, et al. |
| U.S. Appl. No. 16/478,907, filed Jul. 18, 2019, Valouch, et al. |
| U.S. Appl. No. 16/483,231, filed Aug. 2, 2019, Send, et al. |
| International Preliminary Report on Patentability and Written Opinion dated Aug. 22, 2019 in PCT/EP2018/053057. |
| U.S. Appl. No. 16/636,148, filed Feb. 3, 2020, Schindler, et al. |
| U.S. Appl. No. 16/638,946, filed Feb. 13, 2020, Schindler, et al. |
| J. Robertson, “High Dielectric Constant Oxides”, European Physical Journal Applied Physics, vol. 28, No. 3, pp. 265-291, 2004. |
| J.A. Kittl et al., “High-k Dielectrics for Future Generation Memory Devices”, Microelectronic Engineering, Vo. 86 (2009) 1789-1795. |
| Serap Günes, et al., “Hybrid Solar Cells”, Inorganica Chimica Acta 361, (2008), p. 581-588. |
| John E. Anthony, et al., “n-Type Organic Semiconductors in Organic Electronics”, Adv. Mater. 2010, 22, pp. 3876-3892. |
| Tian-yi Li, et al., “Small Molecule Near-Infrared Boron Dipyrromethene Donors for Organic Tandem Solar Cells”, J. Am. Chem. Soc. 2017, 139, 13636-13639. |
| Christian Ulrich et al., “Organic Thin-Film Photovoltaic Cells Based on Oligothiophenes with Reduced Bandgap”, Adv. Funct. Mater. 2007, 17, pp. 2991-2999. |
| Ronald Gresser, et al., “Synthesis and Characterization of Near-Infrared Absorbing Benzannulated Aza-BODIPY Dyes”, Chem. Eur. J. 2011, 17, pp. 2939-2947. |
| Amaresh Mishra, et al., “Small Molecule Organic Semiconductors on the Move: Promises for Future Solar Energy Technology”, Angew. Chem. Int. Ed. 2012, 51, 2020-2067. |
| Huifeng Yao et al., “Molecular Desion of Benzodithiophene-Based Organic Photovoltaic Materials”, Chem. Rev. 2016, 116, 7397-7457. |
| Moritz Riede, et al., “Efficient Organic Tandem Solar Cells based on Small Molecules”, Adv. Funct. Mater. 2011, 21, pp. 3019-3028. |
| Rico Schueppel, et al., “Controlled Current Matching in Small Molecule Organic Tandem Solar Cells Using Doped Spacer Layers”, J. Appl. Phys. 107, 044503, 2010. |
| Jan Meiss et al., “Fluorinated Zinc Phthalocyanine as Donor for Efficient Vacuum-Deposited Organic Solar Cells,” Adv. Funct. Mater. 2012, 22, pp. 405-414. |
| Erwin Bacher, et al., “Synthesis and Characterization of Photo-Cross-Linkable Hole-Conducting Polymers”, Macromolecules, vol. 38, 2005, pp. 1640-1647. |
| H. Bruce Goodbrand, et al., “Ligand-Accelerated Catalysis of the Ullmann Condensation: Application to Hole Conducting Triarylamines”, J. Org. Chem., vol. 64, 1999, pp. 670-674. |
| Felix E. Goodson, et al., “Palladium-Catalyzed Synthesis of Pure, Regiodefined Polymeric Triarylamines”, J. Am. Chem. Soc., vol. 121, 1999, pp. 7527-7539. |
| John F. Hartwig, “Transition Metal Catalyzed Synthesis of Arylamines and Aryl Ethers from Aryl Halides and Triflates: Scope and Mechanism”, Angew. Chem. Int. Ed., vol. 37, 1998, pp. 2046-2067. |
| Sheila I. Hauck, et al., “Tetraazacyclophanes by Palladium-Catalyzed Aromatic Amination. Geometrically Defined, Stable, High-Spin Diradicals”, Organic Letters, vol. 1, No. 13, 1999, pp. 2057-2060. |
| Ping-Hsin Huang, et al., “Synthesis and Characterization of new fluorescent two-photon absorption chromophores”, J. Mater. Chem., vol. 16, 2006, pp. 850-857. |
| Qinglan Huang, et al., “Molecularly ‘Engineered’ Anode Adsorbates for Probing OLED Interfacial Structure-Charge Injection/Luminance Relationships: Large, Structure-Dependent Effects”, J. Am. Chem. Soc., vol. 125, 2003, pp. 14704-14705. |
| A. Balionyte, et al., “Carbazolyl-substituted triphenyldiamine derivatives as novel photoconductive amorphous molecular materials”, Journal of Photochemistry and Photobiology A: Chemistry, vol. 162, 2004, pp. 249-252. |
| G. R. A. Kumara, et al., “Fabrication of Dye-Sensitized Solar Cells Using Triethylamine Hydrothiocyanate as a Cul Crystal Growth Inhibitor”, Langmuir, vol. 18, 2002, pp. 10493-10495. |
| Lukas Schmidt-Mende, et al., “Organic Dye for Highly Efficient Solid-State Dye-Sensitized Solar Cells”, Adv. Mater., vol. 17, No. 7, 2005, pp. 813-815. |
| James Lindley, “Copper Assisted Nucleophilic Substitution of Aryl Halogen”, Tetrahedron, vol. 40, No. 9, 1984, pp. 1433-1456. |
| Yunqi Liu, et al., “Synthesis and characterization of a novel bipolar polymer for light-emitting diodes”, Chem. Commun., vol. 24, 1998, pp. 2747-2748. |
| Narukuni Hirata, et al., “Interface engineering for solid-state dye-sensitised nanocrystalline solar cells: the use of an organic redox cascade”, Chem. Commun., vol. 5, 2006, pp. 535-537. |
| Qingjiang Yu, et al., “High-Efficiency Dye-Sensitized Solar Cells: The Influence of Lithium Ions on Exciton Dissociation, Charge Recombination, and Surface States”, ACS Nano, vol. 4, No. 10, 2010, pp. 6032-6038. |
| Bin Peng, et al., “Systematic investigation of the role of compact TiO.SUB.2 .solar cells”, Coordination Chemistry Reviews, vol. 248, 2004, pp. 1479-1489. |
| Jiun Yi Shen, et al., “High Tg blue emitting materials for electroluminescent devices”, J. Mater. Chem., vol. 15, 2005, pp. 2455-2463. |
| Tobat P. I. Saragi, et al., “Comparison of Charge-Carrier Transport in Thin Films of Spiro-Linked Compounds and Their Corresponding Parent Compounds”, Adv. Funct. Mater., vol. 16, 2006, pp. 966-974. |
| V. P. S. Perera, et al., “Dye-Sensitized Solid-State Photovoltaic Cells Based on Dye Multilayer-Semiconductor Nanostructures”, J. Phys. Chem. B, vol. 107, 2003, pp. 13758-13761. |
| U. Bach, et al., “Solid-state dye-sensitized mesoporous TiO.SUB.2 .solar cells with high photon-to-electron conversion efficiencies”, Nature, vol. 395, 1998, pp. 583-585. |
| John P. Wolfe, et al., “Rational Development of Practical Catalysts for Aromatic Carbon-Nitrogen Bond Formation”, Acc. Chem. Res. vol. 31, 1998, pp. 805-818. |
| Bryant H. Yang, et al., “Palladium-Catalyzed amination of aryl halides and sulfonates”, Journal of Organometallic Chemistry, vol. 576, 1999, pp. 125-146. |
| Zhong Hui Li, et al., “Synthesis and Functional Properties of Strongly Luminescent Diphenylamino End-Capped Oligophenylenes”, J. Org. Chem., vol. 69, 2004, pp. 921-927. |
| Brian O'Regan, et al., “A low-cost, high-efficiency solar cell based on dye-sensitized colloidal TiO2 films”, Letters to Nature, vol. 353, 1991, pp. 737-740. |
| International Search Report dated Sep. 24, 2014 in PCT/EP2014/061682. |
| International Preliminary Report on Patentability and Written Opinion dated Dec. 15, 2015 in PCT/EP2014/061682. |
| International Preliminary Report on Patentability and Written Opinion dated Dec. 23, 2015 in PCT/EP2014/061695. |
| Supplementary European Search Report dated Nov. 19, 2014, issued in corresponding European Patent Application No. EP 12 74 6808. |
| Volker Viereck, et al., Large-area applications of optical MEMS: micromirror arrays guide daylight, optimize indoor illumination, Optical Components, Photonik International 2, 2009, pp. 48-49. |
| C.U. Murade, et al., “High speed adaptive liquid microlens array”, Optics Express, vol. 20, No. 16, Jul. 30, 2012, pp. 18180-18187. |
| Jason Heikenfeld, et al., “Recent Progress in Arrayed Electrowetting Optics”, Optics & Photonics News, vol. 20, No. 1, Jan. 1, 2009, pp. 20-26. |
| Tao Peng, “Algorithms and models for 3-D shape measurement using digital fringe projections”, Dissertation, University of Maryland (College Park, Md.), Jan. 16, 2007, 268 pages (http://drum.lib.umd.edu//handle/1903/6654; http://en.wikipedia.org/wiki/Gray_code; http://en.wikipedia.org/wiki/Structured-light_3D_scanner). |
| Jie-Ci Yang et al., “An Intelllgent Automated Door Control System Based on a Smart”, Sensors, 2013, 13(5), pp. 5923-5936; doi: 10.3390/s130505923 www.mdpi.com/journal/sensors. |
| Tomas Leijtens, et al., “Hole Transport Materials with Low Glass Transition Temperatures and High Solubility for Application in Solid-State Dye-Sensitized Solar Cells”, ACS Nano, vol. 6, No. 2, 2012, pp. 1455-1462 www.acsnano.org. |
| International Search Report and Written Opinion dated Oct. 31, 2014 in PCT/EP2014/067466 filed Aug. 15, 2014. |
| Paul Pargas, “Phenomena of Image Sharpness Recognition of CdS and CdSe Photoconductors” Journal of the Optical Society of America, vol. 54, No. 4, Apr. 1964, pp. 516-519. |
| Paul Pargas, “A Lens Measuring Method Using Photoconductive Cells” Journal of the SMPTE, vol. 74, Jun. 1965, pp. 501-504. |
| Jack T. Billings, “An Improved Method for Critical Focus of Motion-Picture Optical Printers” Journal of the SMPTE, vol. 80, Aug. 1971, pp. 624-628. |
| International Search Report dated Sep. 9, 2014 in PCT/EP2014/061688 filed Jun. 5, 2014. |
| Written Opinion of the International Searching Authority dated Sep. 9, 2014 in PCT/EP2014/061688 filed Jun. 5, 2014. |
| Written Opinion of the International Searching Authority dated Mar. 12, 2015 in PCT/EP2014/078155 Filed Dec. 17, 2014. |
| International Search Report and Written Opinion of the International Searching Authority dated May 16, 2014 in PCT/IB2013/061095. |
| Seigo Ito, et al., “High-Efficiency Organic-Dye-Sensitized Solar Cells Controlled by Nanocrystalline-TiO2 Electrode Thickness”, Adv. Mater., vol. 18, 2006, pp. 1202-1205. |
| Atte Haapalinna, et al., “Measurement of the Absolute Linearity of Photodetectors with a Diode Laser,” Meas. Sci. Technol., 10, (1999) 1075-1078. |
| M. R. Andersen, et al., “Kinect Depth Sensor Evaluation for Computer Vision Applications”,Electrical and Computer Engineering, Technical Report ECE-TR-6, Aarhus University, 2012, 39 pages. |
| Takumi Kinoshita, et al., “Wideband dye-sensitized solar cells employing a phosphine-coordinated ruthenium sensitizer”, Nature Photonics, vol. 7, 2013, pp. 535-239. |
| Office Action dated Apr. 22, 2015 in Chinese Patent Application No. 201280018328.5 (submitting English translation only). |
| International Search Report and Written Opinion dated Sep. 3, 2014 in PCT/EP2014/061691. |
| International Preliminary Report on Patentability dated Sep. 25, 2015 in PCT/EP2014/061691. |
| Kuthirumal, S., et al., “Flexible Depth of Field Photography,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, No. 1 (2011), pp. 58-71. |
| Hiura Shinsaku et al., “Depth Measurement by the Multi-Focus Camera,” Computer Vision and Pattern Recognition, 1998. Proceedings. 1998 IEEE Computer Society Conference on Santa Barbara, CA, USA, Jun. 23-25, 1998, pp. 953-959. |
| International Preliminary Report on Patentability and Written Opinion dated Dec. 23, 2015 in PCT/EP2014/061688. |
| Denis Klimentjew, et al., “Multi Sensor Fusion of Camera and 3D Laser Range Finder for Object Recognition” 2010 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems, 2010, pp. 236-241. |
| International Search Report and Written Opinion dated Nov. 21, 2014 in PCT/EP2014/067465. |
| “So funktioniert die DLP-Technologie” DLP-Technologie—www.dlp.com/de/technology/how-dlp-works, 2014, 2 Pages. |
| “NEW—Ultra-Compact Pockels Cells with Brewster Polarizer and Waveplate for Laser Q-Switching” Leysop Ltd, Manfacturers and Suppliers of Electro-Optic Components—http://www.leysop.com/integrated_pockels_cell.htm, Aug. 4, 2013, 2 Pages. |
| D. Scaramuzza, et al., “Extrinsic Self Calibration of a Camera and a 3D Laser Range Finder from Natural Scenes” 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2007, (4164-4169) 8 pages. |
| International Search Report dated Aug. 28, 2015, in PCT/IB2015/052769, filed Apr. 16, 2015. |
| International Search Report and Written Opinion dated Jun. 30, 2015 in PCT/IB15/052233 Filed Mar. 26, 2015. |
| International Search Report dated Sep. 22, 2015, in Application No. PCT/IB2015/052785, filed on Apr. 16, 2015. |
| International Search Report dated Nov. 27, 2015, in PCT/IB2015/055121 , filed Jul. 7, 2015. |
| International Search Report and Written Opinion dated Mar. 29, 2016, in PCT/IB2015/054536, filed Jun. 16, 2015. |
| International Search Report dated Jan. 18, 2016 in PCT/IB2015/057412 filed Sep. 28, 2015. |
| Wang, Feng-Peng et al., “Distance Measurement using Digital Cameras Based on Laser Spot Detection”, published on Jun. 30, 2011, School of Physics and Electronic Information, Gannan Normal University, Ganzhou 341000, China (with English Abstract). |
| Bahaa E. A. Saleh, et al., “Fundamentals of Photonics” John Wiley & Sons, Inc., Chapter 3, 1991, pp. 80-107 (with Cover Page). |
| International Search Report dated Mar. 21, 2016, in PCT/IB2015/059406. |
| International Search Report in PCT/IB2015/059411 dated Mar. 16, 2016 filed Dec. 7, 2015. |
| Nam-Trung Nguyen, “Micro-optofluidic Lenses: A review”, Biomicrofluidics, 2010, vol. 4, 031501-15. |
| Uriel Levy et al., “Tunable optofluidic devices”, Microfluid Nanofluid, 2008, vol. 4, pp. 97-105. |
| International Search Report dated Mar. 22, 2016 in PCT/IB2015/059408 filed Dec. 7, 2015. |
| International Search Report dated Mar. 15, 2016 in PCT/IB2015/059404 filed Dec. 7, 2015. |
| International Search Report and Written Opinion dated Mar. 21, 2016, in PCT/IB2015/059403, filed Dec. 7, 2015. |
| International Search Report and Written Opinion dated May 27, 2016, in PCT/EP2016/051817, filed Jan. 28, 2016. |
| International Preliminary Report on Patentability dated Aug. 1, 2017, in PCT/EP2016/051817. |
| Linyi Bian, et al., “Recent Progress in the Design of Narrow Bandgap Conjugated Polymers for High-Efficiency Organic Solar Cells”, Progress in Polymer Science, vol. 37, 2012, pp. 1292-1331. |
| Antonio Facchetti, “Polymer donor-polymer acceptor (all-polymer) solar Cells”, Materials Today, vol. 16 No. 4, Apr. 2013, pp. 123-132. |
| Graham H. Carey, et al., “Colloidal Quantum Dot Solar Cells”, Chemical Reviews, vol. 115 No. 23, 2015, pp. 12732-12763. |
| Jason P. Clifford, et al., “Fast, Sensitive and Spectrally Tunable Colloidal Quantum-Dot Photodetectors”, Nature Nanotechnology, Jan. 2009, pp. 1-5. |
| Kotaro Fukushima, et al., “Crystal Structures and Photocarrier Generation of Thioindigo Derivatives”, Journal of Physical Chemistry B, vol. 102 No. 31, 1998, pp. 5985-5990. |
| Serap Günes, et al., “Hybrid Solar Cells”, Inorganica Chimica Acta, vol. 361, 2008, pp. 581-588. |
| R. S. Mane, et al., “Chemical Deposition Method for Metal Chalcogenide Thin Films”, Materials Chemistry and Physics, vol. 65, 2000, pp. 1-31. |
| Wilfried Hermes, “Emerging Thin-Film Photovoltaic Technologies”, Chemie Ingenieur Technik, 2015, vol. 87 No. 4, pp. 376-389. |
| Paul H. Keck, “Photoconductivity in Vacuum Coated Selenium Films”, Journal Optical Society of America, vol. 42 No. 4, Apr. 1952, pp. 221-225, with cover page. |
| Frank H. Moser, et al., “Phthalocyanine Compounds”, Reinhold Publishing Corporation, 1963, p. 69-76 with cover pages. |
| M. Popescu, “Disordered Chalcogenide Optoelectronic Materials: Phenomena and Applications”, Journal of Optoelectronics and Advanced Materials, vol. 7 No. 4, Aug. 2005, pp. 2189-2210. |
| Friedrich Andreas Sperlich, “Electron Paramagnetic Resonance Spectroscopy of Conjugated Polymers and Fullerenes for Organic Photovoltaics”, Julius-Maximilians-Universität Würzburg, 2013, pp. 1-127. |
| F. Stöckmann, “Superlinear Photoconductivity”, Physica Status Solidi, vol. 34, 1969, pp. 751-757. |
| Evangelos Theocharous, “Absolute Linearity Measurements on a PbS Detector in the Infrared”, Applied Optics, vol. 45 No. 11, Apr. 10, 2006, pp. 2381-2386. |
| Evangelos Theocharous, et al., “Absolute Linearity Measurements on HgCdTe Detectors in the Infrared Region”, Applied Optics, vol. 43 No. 21, Jul. 20, 2004, pp. 4182-4188. |
| Arthur L.Thomas, “Phthalocyanine Research and Applications”, CRC Press, 1990, pp. 253-271 with cover pages. |
| International Search Report and Written Opinion dated Jul. 19, 2016 in PCT/EP2016/058487 filed Apr. 18, 2016. |
| International Preliminary Report on Patentability and Written Opinion dated Feb. 1, 2018, in PCT/EP2016/066783. |
| http://www.plenoptic.info/pages/refocusing.html, 2014. |
| C. Hahne, A. Aggoun, S. Haxha, V. Velisavljevic, and J. Fernández, “Light field geometry of a standard plenoptic camera,” Opt. Express 22, 26659-26673 (2014). |
| C. Hahne, A. Aggoun, S. Haxha, V. Velisavljevic, and J. Fernández, “Baseline of virtual cameras acquired by a standard plenoptic camera setup,” in 3D-TV-Conference: The True Vision—Capture, Transmission and Display of 3D Video (3DTV-CON), Jul. 2-4, 2014. |
| C. Hahne, A. Aggoun, and V. Velisavljevic, “The refocusing distance of a standard plenoptic photograph,” in 3D-TV-Conference: The True Vision—Capture, Transmission and Display of 3D Video (3DTV-CON), Jul. 8-10, 2015. |
| C. Hahne and A. Aggoun, “Embedded FIR filter design for real-time refocusing using a standard plenoptic video camera,” Proc. SPIE 9023, in Digital Photography X, 902305 (Mar. 7, 2014). |
| Office Action dated Jan. 3, 2018, in Chinese Patent Application No. 201610552144.7 parallel to U.S. Appl. No. 15/364,680. |
| International Preliminary Report on Patentability and Written Opinion dated Feb. 22, 2018 in PCT/EP2016/069049). |
| Office Action dated Mar. 5, 2018, in corresponding Chinese Patent Application No. 201480056299.0. |
| R. M. Schaffert, “A New High-Sensitivity Organic Photoconductor for Electrophotography”, IBM J. Res. Develop., 1971, p. 75-89. |
| P. Gregory, Ed., Chemistry and Technology of printing and imaging systems, Chapman & Hall, 1996, Chap. 4, R.S. Gairns, Electrophotography, p. 76-112. |
| “Telezentrisches Objektiv” Retrieved from the Internet: https://de.wikipedia.org/wiki/Telezentrisches_Objektiv. Date of retrieval: Sep. 11, 2015, 3 Pages. |
| Benjamin F. Grewe, et al., “Fast Two-Layer Two-Photon Imaging of Neuronal Cell Populations Using an Electrically Tunable Lens,” Biomedical Optics Express, vol. 2, No. 7, Jul. 1, 2011 (pp. 2035-2046). |
| Petr Bartu et al, “Conformable Large-AreaPposition-Sensitive Photodetectors Based on Luminescence-Collecting Silicone Waveguides,” Journal of Applied Physics, 107, 123101 (2010). |
| Roland Stolarski, “Fluorescent Naphthalimide Dyes for Polyester Fibres,” Fibres & Textiles in Eastern Europe, vol. 17, No. 2 (73) pp. 91-95 (2009). |
| Ayse Aktas et al., “Synthesis, Characterization, Electrochemical and Spectroelectrochemical Properties of Peripherally Tetra-Substituted Metal-Free and Metallophthalocyanines,” Dyes and Pigments, 99, (2013) 613-619. |
| Hairong Li, et al., “Syntheses and Properties of Octa-, Tetra-, and Di- Hydroxy-Substituted Phthalocyanines,” Tetrahedron, 65 (2009) 3357-3363. |
| “Methine Dyes and Pigments,” Ullmann's Encyclopedia of Industrial Chemistry, vol. 23 (2012). |
| Jing Liu, et al., “Sulfone-Rhodamines: A New Class of Near-Infrared Fluorescent Dyes for Bioimagin,” ACS Applied Materials& Interfaces, 8, 22953-22962 (2016). |
| T. Nedelcev et al., “Preparation and Characterization of a New Derivative of Rhodamine B with an Alkoxysilane Moiety,” Dyes and Pigments, 76 (2008), 550-556. |
| Aurore Loudet et al., “BODIPY Dyes and Their Derivatives: Syntheses and Spectroscopic Properties,” Chem.Rev., 107 (2007) 4981-4932. |
| Weili Zhao, et al., “Conformationally Restricted Aza-Bodipy: A Highly fluorescent, Stable, Near-Infrared-Absorbing Dye”, Angew. Chem. Int. Ed., 44 (2005) 1677-1679. |
| Georg M. Fischer, et al., “Near-Infrared Dyes and Fluorophores Based on Diketopyrrolopyrroles,” Angew. Chem. Int. Ed. 46 (2007) 3750-3753. |
| Amaresh Mishra et al., “Small Molecule Organic Semiconductors on the Move: Promises for Future Solar Energy Technology,” Angew. Chem. Int. Ed., 51, (2012), 2020-2067. |
| G. Seybold et al., “New Perylene and Violanthrone Dyestuffs for Fluorescent Collectors,” Dyes and Pigments, 11 (1989) 303-317. |
| Nam-Trung Nguyen, “Micro-Optofluidic Lenses: A Review”, Biomicrofluidics, 4, (2010) 031501. |
| Uriel Levy et al., “Tunable Optofluidic Devices,” Microfluid Nanofluid (2008) 4: 97-105. |
| Robert Koeppe et al., “Video-Speed Detection of the Absolute Position of a Light Point on a Large-Area Photodetector Based on Luminescent Waveguides,” Optics Express, vol. 18, No. 3, (Feb. 1, 2010), 2209. |
| Office Action dated Jul. 9, 2018, in Japanese Patent Application No. 2017-007544. |
| Xing Lin, et al., “Coded focal stack photography”, Computational Photography (ICCP), 2013 IEEE International Conference On, Apr. 19, 2013, XP032424246, pp. 1-9. |
| Nabeel A. Riza, et al., “Noncontact distance sensor using spatial signal processing”, Optics Letters, Optical Society of America, vol. 34, No. 4, Feb. 15, 2009, XP001522006, pp. 434-436. |
| Nabeel A. Riza, et al., “Smart agile lens remote optical sensor for three-dimensional object shape measurements”, Applied Optics, Optical Society of America, vol. 49, No. 7, Mar. 1, 2010, XP001552714, pp. 1139-1150. |
| International Search Report dated Nov. 7, 2017, in corresponding PCT/EP2017/057867. |
| International Search Report dated Jul. 7, 2017, in corresponding PCT/EP2017/057825. |
| International Search Report dated Aug. 17, 2017, in corresponding PCT/EP2017/060057. |
| International Search Report dated Aug. 1, 2017, in corresponding PCT/EP2017/060058. |
| Walter Fuhs, “Hydrogenated Amorphous Silicon—Material Properties and Device Applications”, in S. Baranovski, Charge Transport in Disordered Solids, Wiley, p. 97-147, 2006. |
| Office Action dated Dec. 18, 2018, in Japanese Patent Application No. 2016-518930. |
| A.G. Pattantyus-Abraham, I.J. Kramer, A.R. Barkhouse, X. Wang, G. Konstantatos, R. Debnath, L. Levina, I. Raabe, M.K. Nazeeruddin, M. Grätzel, and E.H. Sargent, Depleted-Heterojunction Colloidal Quantum Dot Solar Cells, ACS Nano 4 (6), May 24, 2010. |
| R. Martins and E. Fortunato, Thin Film Position Sensitive Detectors: from 1D to 3D Applications, Chap. 8 in R. A. Street (Ed.), Technology and Applications of Amorphous Silicon, Springer, 2010. |
| International Search Report dated Oct. 20, 2017 in PCT/EP2017/068956 filed on Jul. 27, 2017. |
| Leskela, M. et al., “Preparation of lead sulfide thin films by the atomic layer epitaxy process,” Pergamon Press plc, Vacuum/vol. 41/Nos. 4-6, pp. 1457-1459 (1990). |
| Dasgupta, N. et al., “Fabrication and Characterization of Lead Sulfide Thin Films by Atomic Layer Deposition,” The Electrochemical Society, ECS Transactions, 16 (4) 29-36 (2008), Total 8 pages. |
| Dasgupta, N. et al., “Design of an atomic layer deposition reactor for hydrogen sulfide compatibility,” Review of Scientific Instruments 81, 044102 (2010), Total 6 pages. |
| Xu, J. et al., “Atomic layer deposition of absorbing thin films on nanostructured electrodes for short-wavelength infrared photosensing,” AIP Publishing, Applied Physics Letters 107, 153105 (2015), Total 5 pages. |
| Blount, G., et al., “Photoconductive properties of chemically deposited PbS with dielectric overcoatings,” AIP Publishing, Journal of Applied Physics 46, 3489 (1975), Total 12 pages. |
| Groner, M. et al., “Low-Temperature Al2O3 Atomic Layer Deposition,” American Chemical Society, Chem. Mater., vol. 16, No. 4, 2004, pp. 639-645. |
| Yoon, W. et al., “Electrical Measurement Under Atmospheric Conditions of PbSe Nanocrystal Thin Films Passivated by Remote Plasma Atomic Layer Deposition of Al2O3,” IEEE Transactions on Nanotechnology, vol. 12, No. 2, Mar. 2013, pp. 146-151. |
| Hu, C., et al., “Air-stable short-wave infrared PbS colloidal quantum dot photoconductors passivated with Al2O3 atomic layer deposition,” AIP Publishing, Applied Physics Letters 105, 171110 (2014), Total 5 pages. |
| Liu, Y., et al., “Robust, Functional Nanocrystal Solids by Infilling with Atomic Layer Deposition,” ACS Publications, American Chemical Society, Nano Letters 2011, 11, pp. 5349-5355. |
| Liu, Y., et al., “PbSe Quantum Dot Field-Effect Transistors with Air-Stable Electron Mobilities above 7 cm2 V−1 s−1,” ACS Publications, American Chemical Society, Nano Letters 2013, 13, pp. 1578-1587. |
| George, S., “Atomic Layer Deposition: An Overview,” American Chemical Society, Chem. Rev. 2010, 110, pp. 111-131. |
| Konstantatos, G., et al., “Engineering the Temporal Response of Photoconductive Photodetectors via Selective Introduction of Surface Trap States,” American Chemical Society, Nano Letters 2008, vol. 8, No. 5, pp. 1446-1450. |
| Soci, C., et al., “ZnO Nanowire UV Photodetectors with High Internal Gain,” American Chemical Society, Nano Letters 2007, vol. 7, No. 4, pp. 1003-1009. |
| List of integrated circuit packaging types (Wikipedia article, downloaded Jan. 15, 2019 from https://en.wikipedia.org/wiki/List_of_integrated_circuit_packaging_types). |
| List of integrated circuit packaging types, Dimension Reference (Wikipedia article, downloaded Jan 15. 2019 from https://en.wikipedia.org/wiki/List_of_integrated_circuit_packaging_types#PIN-PITCH). |
| Wikipedia, Article denoted, “Thermocouple”, retrieved Jul. 20, 2016. |
| Pekkola et al., “Focus-Induced Photoresponse: a novel way to measure distances with photodetectors”, Scientific Reports (2018) 8:9208, 8 pages. |
| L. Pintilie, et al., “Field-effect-assisted photoconductivity in PbS films deposited on silicon dioxide”, J. Appl. Phys. 91, p. 5782, 2002. |
| Kurt Konolige et al., “A Low-Cost Laser Distance Sensor”, 2008 IEEE International Conference on Robotics and Automation, Pasadena, CA, May 19-23, 2008. |
| U.S. Appl. No. 15/309,631, filed Nov. 8, 2016, U.S. Pat. No. 9,919,999, Koenemann, et al. |
| U.S. Appl. No. 14/908,740, filed Jan. 29, 2016, U.S. Pat. No. 10,290,817, Battagliarin et al. |
| U.S. Appl. No. 15/512,116, filed Mar. 17, 2017, US 2017-0250334, Hermes, et al. |
| U.S. Appl. No. 15/557,178, filed Sep. 11, 2017, US 2018-0044357, Spielmann, et al. |
| U.S. Appl. No. 16/328,912, filed Feb. 27, 2019, US 2019-0198206, Ter Maat, et al. |
| U.S. Appl. No. 16/461,670, filed May 16, 2019, Eberspach, et al. |
| U.S. Appl. No. 15/578,763, filed Dec. 1, 2017, US 2018-0182980, Lennartz et al. |
| U.S. Appl. No. 13/357,206, filed Jan. 24, 2012, U.S. Pat. No. 9,001,029, Bruder, et al. |
| U.S. Appl. No. 14/132,570, filed Dec. 18, 2013, U.S. Pat. No. 9,389,315, Bruder, et al. |
| U.S. Appl. No. 14/460,529, filed Aug. 15, 2014, U.S. Pat. No. 9,665,182, Send, et al. |
| U.S. Appl. No. 14/460,540, filed Jan. 31, 2017, U.S. Pat. No. 9,557,856, Send, et al. |
| U.S. Appl. No. 14/598,432, filed Jan. 16, 2015, US 2015-0124268, Bruder, et al. |
| U.S. Appl. No. 14/787,909, filed Oct. 29, 2015, U.S. Pat. No. 9,741,954, Bruder, et al. |
| U.S. Appl. No. 14/897,981, filed Dec. 11, 2015, US 2016-0124074, Wonneberger, et al. |
| U.S. Appl. No. 14/897,467, filed Dec. 10, 2015, U.S. Pat. No. 9,989,623, Send, et al. |
| U.S. Appl. No. 14/896,958, filed Dec. 9, 2015, U.S. Pat. No. 9,829,564, Bruder, et al. |
| U.S. Appl. No. 15/099,717, filed Apr. 15, 2016, U.S. Pat. No. 10,120,078, Bruder, et al. |
| U.S. Appl. No. 15/105,489, filed Jun. 16, 2016, US 2016-0320489, Send, et al. |
| U.S. Appl. No. 15/301,112, filed Sep. 30, 2016, US 2017-0140786, Send, et al. |
| U.S. Appl. No. 15/304,328, filed Oct. 14, 2016, US 2017-0039793, Send, et al. |
| U.S. Appl. No. 15/305,379, filed Oct. 20, 2016, US 2017-0074652, Send, et al. |
| U.S. Appl. No. 15/319,156, filed Dec. 15, 2016, US 2017-0123593, Send, et al. |
| U.S. Appl. No. 15/364,680, filed Nov. 30, 2016, US 2017-0082426, Bruder, et al. |
| U.S. Appl. No. 15/367,213, filed Dec. 2, 2016, U.S. Pat. No. 10,012,532, Send, et al. |
| U.S. Appl. No. 15/324,223, filed Jan. 5, 2017, US 2017-0205230, Send, et al. |
| U.S. Appl. No. 15/514,830, filed Mar. 28, 2017, U.S. Pat. No. 10,094,927, Send, et al. |
| U.S. Appl. No. 15/492,007, filed Apr. 20, 2017, U.S. Pat. No. 9,958,535, Send, et al. |
| U.S. Appl. No. 15/534,335, filed Jun. 8, 2017, US 2017-0363465, Send, et al. |
| U.S. Appl. No. 15/534,294, filed Jun. 8, 2017, US 2018-0276843, Send, et al. |
| U.S. Appl. No. 15/533,572, filed Jun. 6, 2017, US 2018-0003993, Send, et al. |
| U.S. Appl. No. 15/534,343, filed Jun. 8, 2017, US 2018-0007343, Send, et al. |
| U.S. Appl. No. 15/587,420, May 5, 2017, US 2017-0237926, Bruder, et al. |
| U.S. Appl. No. 15/534,041, filed Jun. 8, 2017, US 2017-0363741, Send, et al. |
| U.S. Appl. No. 15/547,664, filed Jul. 31, 2017, US 2018-0017679, Valouch, et al. |
| U.S. Appl. No. 15/554,496, filed Aug. 30, 2017, US 2018-0067213, Send, et al. |
| U.S. Appl. No. 15/727,733, filed Oct. 9, 2017, US 2018-0031672, Bruder, et al. |
| U.S. Appl. No. 15/567,885, filed Oct. 19, 2017, US 2018-0136319, Send, et al. |
| U.S. Appl. No. 15/744,334, filed Jan. 12, 2018, US 2018-0210064, Send, et al. |
| U.S. Appl. No. 15/751,283, filed Feb. 8, 2018, US 2018-0231376, Send, et al. |
| U.S. Appl. No. 15/758,135, filed Mar. 7, 2018, US 2018-0249051, Send, et al. |
| U.S. Appl. No. 15/775,424, filed May 11, 2018, US 2018-0329024, Send, et al. |
| U.S. Appl. No. 15/778,454, filed May 23, 2018, US 2018-0356501, Send, et al. |
| U.S. Appl. No. 15/960,581, filed Apr. 24, 2018, US 2018-0238993, Send, et al. |
| U.S. Appl. No. 16/090,990, filed Oct. 3, 2018, US 2019-0140129, Valouch, et al. |
| U.S. Appl. No. 16/091,409, filed Oct. 4, 2018, Send, et al. |
| U.S. Appl. No. 16/094,402, filed Oct. 17, 2018, US 2019-0129036, Valouch, et al. |
| U.S. Appl. No. 16/095,846, filed Oct. 23, 2018, Valouch, et al. |
| U.S. Appl. No. 16/096,361, filed Oct. 25, 2018, US 2019-0129035, Valouch, et al. |
| U.S. Appl. No. 16/321,143, filed Jan. 28, 2019, Hermes, et al. |
| U.S. Appl. No. 16/321,054, filed Jan. 28, 2019, Hermes, et al. |
| U.S. Appl. No. 16/344,538, filed Apr. 24, 2019, Valouch, et al. |
| U.S. Appl. No. 16/347,364, filed May 3, 2019, Eberspach, et al. |
| U.S. Appl. No. 16/344,511, filed Apr. 24, 2019, Valouch, et al. |
| E. Noelting et al., “Derichte der deutschen chemischen Gesellschaft”, Band, 38, C. 3516-3527 (1005). |
| Chinese Office Action dated Jan. 29, 2019 in Chinese Patent Application No. 201580036919.9 (English Translation). |
| International Search Report and Written Opinion dated May 31, 2012 in PCT/IB2012/050592 filed on Feb. 9, 2012. |
| International Search Report and Written Opinion of the International Searching Authority dated Mar. 12, 2015 in PCT/EP2014/078155 filed Dec. 17, 2014. |
| International Search Report and Written Opinion dated Jan. 18, 2016 in PCT/IB2015/057412 filed Sep. 28, 2015. |
| International Search Report and Written Opinion in PCT/IB2015/059411 dated Mar. 16, 2016 filed Dec. 7, 2015. |
| International Search Report and Written Opinion dated Mar. 10, 2016 in PCT/IB2015/059404. |
| International Preliminary Report and Written Opinion dated Jun. 13, 2017 in PCT/IB2015/059404. |
| International Search Report and Written Opinion dated May 20, 2016 in PCT/EP2016/054532. |
| International Search Report and Written Opinion dated Nov. 17, 2016 in PCT/EP2016/071628, filed Sep. 14, 2016. |
| Baeg et al., “Organic Light Detectors: Photodiodes and Phototransistors”, Advanced Materials, vol. 25, No. 31, Mar. 11, 2013, pp. 4267-4295. |
| Number | Date | Country | |
|---|---|---|---|
| 20190339356 A1 | Nov 2019 | US |
