This disclosure, and the exemplary embodiments described herein, relates generally to autonomous vehicles, and more particularly to encoding and transmitting information using rotating reflective barcodes encoding time-varying information in reflection patterns scanned by lidar systems. While the exemplary embodiments described are related to autonomous vehicles, it is to be understood that the scope of this disclosure is not limited to such application.
For autonomous vehicles of any scale (whether self-driving automobiles or small mobile robots), it is necessary to obtain information about the surrounding environment in order to maneuver safely and accomplish desired tasks, such as arriving at a desired location at a precise time. Vehicles typically obtain this data through a combination of external information sources, such as the satellite-based Global Positioning System (GPS), and onboard sensors, such as radar, LIDAR (light detection and ranging), and cameras. External sources such as GPS are extremely useful but can have limited availability in certain contexts, such as underground tunnels or in the event of accidental disruption or adversarial disabling of GPS satellites. With regards to onboard sensors, radar and LIDAR provide range measurements and are therefore typically used for object avoidance and maintaining position in the roadway. Monocular cameras don't have range measurements (though stereoscopic setups can yield some depth information), but their high resolution and ability to see color make them useful for understanding the important detailed meaning of a scene, such as traffic signs and lane markings. However, processing of the camera data is typically accomplished through machine learning and artificial intelligence (AI) algorithms, which can impose a significant computational expense. Since cameras typically have a limited field-of-view, a 360-degree surround view can only be obtained by using multiple cameras and digitally stitching together their individual views.
Most self-driving vehicles and autonomous robots are equipped with LIDAR sensors to provide a high-resolution map of the surrounding environment in real-time. The LIDAR data that is most typically used for this application is the point cloud consisting of detected range to the closest object at each bearing. However, many LIDAR systems also provide a measure of the return intensity (also known as reflectance or reflectivity) at each bearing angle. US Patent Application Publication No. 2020/0292735 introduced the idea of using specially designed retroreflectors to encode data in the LIDAR intensity data, analogous to a traditional printed barcode. These retroreflectors (which we will refer to as “static Lidar barcodes”) can be placed in the traffic infrastructure to provide data to LIDAR-equipped vehicles by means of the LIDAR intensity patterns. While appealing in its simplicity, it is severely limited in the amount of data that can be encoded.
Described herein is a system and method for encoding information in a relatively dense and time-varying manner. In exemplary embodiments described, a retroreflector or reflector is wrapped around a rotating cylinder (also referred to as “Rotational LIDAR Barcodes”), which can encode significantly longer data messages which can be detected and decoded from every direction, even when partially obstructed.
The following publications are incorporated by reference in their entirety.
U.S. 2020/0292735, patent application Ser. No. 16/754,116, filed Oct. 5, 2018, and entitled RETROREFLECTORS PROVIDING INFORMATION ENCODED IN REFLECTED NON-VISIBLE LASER WHILE RETAINING VISIBLE LIGHT SAFETY PROPERTIES.
U.S. Pat. No. 10,147,469, patent application Ser. No. 15/588,661, granted Sep. 17, 2019, and filed May 7, 2017, and entitled NAVIGATION USING SELF-DESCRIBING FIDUCIALS.
U.S. Pat. No. 10,145,993, patent application Ser. No. 15/862,408, granted Dec. 4, 2018, and filed Jan. 4, 2018, and entitled RETROREFLECTORS PROVIDING INFORMATION ENCODED IN REFLECTED NON-VISIBLE LASER WHILE RETAINING VISIBLE LIGHT SAFETY PROPERTIES.
In accordance with one embodiment of the present disclosure, disclosed is a rotational LIDAR barcode operatively associated with a LIDAR barcode detection system comprising: a cylindrical member including an outside face, the outside face including a vertical height and a horizontal width, the horizontal width equal to a circumference of the cylinder face; a barcode pattern operatively attached to the cylindrical member outside face, the barcode pattern including a pattern of reflective areas and nonreflective areas representing a binary barcode data message; a rotator operatively coupled to the cylindrical member, the rotator rotating the cylindrical member and operatively attached barcode pattern, and a mount configured to align the barcode pattern, as it rotates, to reflect light received from the LIDAR barcode detecting system back to the LIDAR barcode detecting system.
In accordance with another embodiment of the present disclosure, disclosed is a method of encoding and decoding data represented as a barcode using a rotational LIDAR barcode and a vehicle mounted LIDAR system comprising: wrapping a cylinder face of a cylinder with a barcode pattern including a pattern of reflective areas and nonreflective areas representing a binary barcode data message to be decoded by the vehicle mounted LIDAR system; rotating the cylinder and wrapped barcode pattern at a predetermined angular speed or range of speeds, a rotational plane of the cylinder and wrapped barcode pattern providing for reflecting at least one beam transmitted from the vehicle mounted LIDAR system; and the vehicle mounted LIDAR system scanning the rotating cylinder and wrapped barcode pattern and decoding the wrapped barcode pattern to determine the data message associated with the wrapped barcode pattern.
In accordance with another embodiment of the present disclosure, disclosed is a vehicle mounted LIDAR and rotational LIDAR barcode system comprising: a vehicle mounted LIDAR system; and a rotational LIDAR barcode including: a cylindrical member including an outside face, the outside face including a vertical height and a horizontal width, the horizontal width equal to a circumference of the cylinder face; a barcode pattern operatively attached to the cylindrical member outside face, the barcode pattern including a pattern of reflective areas and nonreflective areas representing a binary barcode data message; a rotator operatively coupled to the cylindrical member, the rotator rotating the cylindrical member and operatively attached barcode pattern; and a mount configured to align the barcode pattern, as it rotates, to reflect light received from the vehicle mounted LIDAR system back to the vehicle mounted LIDAR system.
For a more complete understanding of the present disclosure, reference is now made to the following descriptions taken in conjunction with the accompanying drawings, in which:
Described herein is a system and method for encoding information in a relatively dense and time-varying manner. In exemplary embodiments, a retroreflector is wrapped around a rotating cylinder (also referred to as “Rotational LIDAR Barcodes”), which can encode significantly longer data messages which can be detected and decoded from every direction, even when partially obstructed.
Rotational LIDAR Barcodes can be constructed as patterns of alternating strips of reflective and non-reflective material of varying widths, wrapped around a rotating cylinder. They are designed to produce barcode-like reflection patterns when scanned by LIDAR units on mobile vehicles, with the rotation speed of the barcode designed for the optimal tradeoff between scan accuracy and scan time. The particular patterns can be detected in the LIDAR intensity data received at the vehicle and present a method of encoding data in a traffic infrastructure for LIDAR-equipped vehicles to detect and decode. Rotational LIDAR Barcodes could be used for a wide array of applications. First, they could be used to mark important aspects of the traffic infrastructure such as lane closures, traffic signs, parking lanes, handicapped parking spots, store fronts, and other landmarks. Second, they could be used to certify the authenticity of traffic signs (detected by camera-based systems) to counter possible spoofing attacks. Third, they could be mounted on emergency vehicles as a visual cue (in addition to the typical flashing lights) to help self-driving vehicles detect approaching emergency vehicles approaching from any direction. For mobile robots, Rotational LIDAR Barcodes could be used in laboratory or warehouse environments to indicate prescribed traffic patterns or points of interest, or in any other context where automatic detection of landmarks or other vehicles would be helpful (e.g., amusement parks, autonomous golf carts, etc.).
The foregoing and other features of this disclosure and the exemplary embodiments described herein, are hereinafter described in greater detail with reference to the accompanying drawings.
U.S. Pat. No. 10,145,993 introduced a method to encode data into an environment using specially designed retroreflectors, which can be detected by means of LIDAR sensors already installed on most self-driving vehicles and autonomous robots and can be processed and decoded at a low computational cost. These systems can augment existing camera-based methods for understanding the surrounding environment, providing a low-cost method for encoding data into the traffic infrastructure. However, this method based on static retroreflectors is severely limited in the density of data which can be effectively encoded.
With reference to
U.S. Pat. No. 10,417,469 is similar in theme but distinctly different in implementation and purpose. U.S. Pat. No. 10,417,469 revolves around the use of “self-describing fiducials” (such as barcodes) which can encode navigation information which mobile agents can detect with cameras. The main difference is that U.S. Pat. No. 10,417,469 uses only passive camera-based systems for detection, rather than active laser scanners (e.g., Lidar) to illuminate the targets.
LIDAR scanners operate by using a laser to transmit ultraviolet, visible light, or near infrared to illuminate the surrounding environment and then analyzing the reflected energy to determine range to detected objects. Automotive LIDAR typically operates in the infrared range, at wavelengths of either 905 nm or 1550 nm. According to exemplary embodiments of this disclosure, Rotational LIDAR Barcodes refers to signs (or “targets”) made of alternating strips of materials that are either reflective or non-reflective (i.e., absorbent) of energy at those wavelengths, with the sign mounted on a rotating cylinder.
While the static planar LIDAR barcodes described in the prior art cited above are appealing in their simplicity to implement, they suffer from several significant drawbacks. First, they are by definition directional in nature and therefore can only be observed and decoded from a particular field of view. Within that field of view, parallax effects may present challenges to detection and decoding from certain angles as well. Secondly, the entire barcode must be in view for successful decoding, and therefore even partial obstructions to the view will prevent successful detection and decoding. Lastly, and most significantly, these static barcodes are severely limited in the density of data which they can effectively encode at longer ranges, due to the spreading of sample points on the barcode based on LIDAR angular resolution. Again, as shown in
As an example, for a TURTLEBOT3 robot as used for experimental implementations of the disclosed rotational LIDAR barcode as described herein, the LIDAR unit has an angular resolution of 1 degree which results in sample points separation of approx. 5 cm when the robot is 3 m from the barcode. For the static barcode case, to obtain at least one LIDAR scan point on each information bit in a 32-bit message would require a target face 168 cm wide, and most likely successful detection and decoding would require more than one sample per bit. While LIDAR units with better angular resolution are certainly available on the market, many of these are intended to be operated at longer ranges (i.e., larger values of ρ), which lead to the same issue.
Due to these limitations for static planar LIDAR barcodes, described herein is a rotational LIDAR barcode based on time-varying versions of a retroreflector(s) wrapped around a rotating cylinder. The rotational LIDAR barcode can encode significantly longer data messages which can be detected and decoded from every direction, even when partially obstructed.
With reference to
It is important to note that the rotation of the barcode effectively adds an additional temporal dimension to the data. With reference to
While the spatial sampling resolution is completely constrained by the angular resolution of the LIDAR unit, the time-based sampling resolution can be controlled by selection of the rotation rate for the barcode. Thus, the limitation on bit density for static retroreflectors described in U.S. Pat. No. 10,145,993 can be overcome with Rotational LIDAR Barcodes by selection of the time-based sampling rate.
The rotation of the barcode also creates a unique spatiotemporal signature in the sensor data, which enables detection in a cluttered environment by analysis of the statistics of the intensity data. Also, based on the radius of the Rotational LIDAR Barcode, the angular resolution of the LIDAR, and the distance between the LIDAR-equipped vehicle and the barcode target, it is possible that multiple LIDAR beams (along different neighboring bearing angles) will strike the target on a given sweep.
With reference to
An example of the sampled bitstream received along a particular bearing during this experiment is shown in
As mentioned previously, one advantage of Rotational LIDAR barcodes is that their time-based variation produces a unique signature which can be exploited for detection purposes. In what follows, we will describe a two-stage approach that provides reliable detection from reasonable ranges in experimental trials.
The first stage of our detection process is based on windowed standard deviation of the LIDAR return intensity measurements along each bearing angle. Bearings which reach a prescribed threshold for minimum variation during the observed window are returned as candidates to pass to the second stage.
The second stage makes use of the fact that neighboring bearing angles which view the barcode will provide bit streams that are time-delayed versions of the neighbor angle bit streams. We can therefore compare a correlation of shifted versions of neighbor bearings to see if a particular shift meets a prescribed minimum correlation.
As previously described, rotational LIDAR barcodes can be used to encode a variety of data, from identification of key landmarks to navigation instructions, to information about upcoming traffic routing, and even advertising information. Rotational LIDAR Barcodes can also be mounted on vehicles themselves to enable vehicle identification by other LIDAR-equipped vehicles. This would be particularly useful for identifying emergency vehicles. Also, Rotational LIDAR Barcodes can be attached to traffic signs to certify their authenticity. Spoofing of traffic signs is an increasing concern for camera-based identification methods for self-driving vehicles, and an attached Rotational LIDAR Barcode can be used to encode a serial number authenticating the validity of the sign.
Although any appropriate encoding scheme is possible, the main two types of encoding are simple binary, or UPC barcode encoding. In all cases, since the message repeats itself, there should be some type of “start-of-frame” indicator. This can be encoded with a particularly distinct pattern of white/black bars, or with a bar that is much more reflective than the rest (similar to sync separators used in early analog TV).
With reference to
As shown, the system includes a rotational LIDAR barcode 804 and a LIDAR-equipped vehicle 802, such as an autonomous automobile or robot, etc. The vehicle LIDAR scanning system further includes a LIDAR transmitter/receiver 806, a barcode detection processor 808, a barcode decoding processor 810 and a vehicle control unit 812. In operation, the rotational LIDAR barcode 804 rotates at a predetermined speed and the LIDAR transmitter/receiver 806 scans the rotating LIDAR barcode 804 as the vehicle travels or at stationary points. The reflective/nonreflective light intensity signal received at the LIDAR transmitter/receiver 806 is further processed to provide intensity and range data to the barcode detection processor 808.
The barcode detection processor 808 further processes the received intensity and range data to 1) remove own-vehicle motion effects and 2) detect barcodes based on variance in LIDAR return intensity along bearing.
The barcode decoding processor 810 receives the data representing the bearing to the detected barcodes and 1) finds the start sequence, 2) translates perceived bar widths to number of bits and generates a decoded message.
The decoded message and bearing to barcode data are transmitted to the vehicle control unit 812 for further processing depending on the significance of the decoded message.
Some advantages and novel features of the disclosed exemplary embodiments include the following:
Some alternative representations and uses of time-varying LIDAR barcode technology includes the following.
Possible implementation with natural processes (e.g., wind, wave motion, etc.) driving the rotation of the barcode.
The methods illustrated throughout the specification, may be implemented in a computer program product that may be executed on a computer. The computer program product may comprise a non-transitory computer-readable recording medium on which a control program is recorded, such as a disk, hard drive, or the like. Common forms of non-transitory computer-readable media include, for example, floppy disks, flexible disks, hard disks, magnetic tape, or any other magnetic storage medium, CD-ROM, DVD, or any other optical medium, a RAM, a PROM, an EPROM, a FLASH-EPROM, or other memory chip or cartridge, or any other tangible medium from which a computer can read and use.
Alternatively, the method may be implemented in transitory media, such as a transmittable carrier wave in which the control program is embodied as a data signal using transmission media, such as acoustic or light waves, such as those generated during radio wave and infrared data communications, and the like.
It will be appreciated that variants of the above-disclosed and other features and functions, or alternatives thereof, may be combined into many other different systems or applications. Various presently unforeseen or unanticipated alternatives, modifications, variations or improvements therein may be subsequently made by those skilled in the art which are also intended to be encompassed by the following claims.
The exemplary embodiment has been described with reference to the preferred embodiments. Obviously, modifications and alterations will occur to others upon reading and understanding the preceding detailed description. It is intended that the exemplary embodiment be construed as including all such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.
This application claims the benefit of U.S. Provisional Application No. 63/246,337, filed Sep. 21, 2021, and entitled ROTATING REFLECTIVE BARCODES ENCODING TIME-VARYING INFORMATION IN REFLECTION PATTERNS SCANNED BY VEHICLE-MOUNTED LIDAR SYSTEMS, which is hereby incorporated in its entirety by reference.
Number | Date | Country | |
---|---|---|---|
63246337 | Sep 2021 | US |