Patent Grant
12205483
As compared to outdoor operations, operating an unmanned aerial vehicle, or drone, within indoor spaces presents a unique set of challenges for the unmanned aerial vehicle, and creates unique risks for contents of the indoor spaces. For example, an unmanned aerial vehicle that operates outdoors may typically climb above any encountered hazards by increasing altitude, in order to elevate above ground-based or airborne obstacles, which may include humans or other animals, or other stationary or moving objects. An unmanned aerial vehicle that operates indoors, however, is commonly constrained by ceilings at eight to ten foot elevations, or approximately one to five feet above the heights of most humans.
An unmanned aerial vehicle operating outdoors may capture data regarding its positions and orientations using one or more onboard sensors and interpret the data to generate commands or instructions for operating motors or control surfaces to avoid such obstacles. Because indoor spaces are commonly constrained by narrow hallways or other passageways, and feature limited operating areas between floors and ceilings, an unmanned aerial vehicle that operates indoors must often make tight turns or changes in altitude, or travel well below its maximum speeds, in order to avoid contacting objects such as walls, ceilings, floors or others during normal operations.
When an unmanned aerial vehicle operating in an indoor space detects or encounters an object of any kind, the unmanned aerial vehicle must take care to promptly reduce its speed, thereby minimizing a rate at which a range between the unmanned aerial vehicle and the object is decreasing, and select an appropriate course in order to avoid a collision with the object. Given that unmanned aerial vehicles operating in indoor spaces typically travel in close proximity to one or more objects, a course for avoiding a collision with one object may place an unmanned aerial vehicle at risk of colliding with another object.
As is set forth in greater detail below, the present disclosure is directed to systems and methods for selecting routes for obstacle avoidance by unmanned aerial vehicles. The present disclosure describes, inter alia, unmanned aerial vehicles that are configured to determine distances to walls, fixtures or other features surrounding the aerial vehicles during flight operations using time-of-flight sensors, LIDAR (viz., light detection and ranging) sensors, imaging devices, radar sensors, sonar sensors, or other sensors. When an unmanned aerial vehicle encounters objects that pose a collision risk, the unmanned aerial vehicle may select a safe direction for deceleration by executing a braking maneuver based on the determined distances. For example, the unmanned aerial vehicle may determine positions of surrounding objects detected during flight operations, and define sectors, areas or zones around the unmanned aerial vehicle that may safely accommodate the unmanned aerial vehicle while decelerating or executing the braking maneuver. The unmanned aerial vehicle may select one of the sectors, areas or zones, and decelerate in a direction corresponding to the selected sector, area or zone.
An unmanned aerial vehicle may generate and execute any number of commands or instructions (which may be executed by a control system or another component of the unmanned aerial vehicle), in order to cause the unmanned aerial vehicle to remain on a desired course, at a desired speed, or at a desired altitude in accordance with the trajectory. The unmanned aerial vehicles may further continuously or repeatedly monitor for objects within predetermined ranges of their respective directions of travel using one or more range sensors. Upon determining that an object is detected within a predetermined range, an unmanned aerial vehicle may immediately slow, e.g., by one or more braking maneuvers, to decelerate or otherwise reduce a speed of the unmanned aerial vehicle in a direction corresponding to a sector, an area or a zone that has been determined to be clear of objects. Such a braking maneuver may include, for example, reducing a rotational speed or adjusting an operation of one or more rotors or engines, such as by operating one or more motors or propellers to cease thrust or to effect thrust in a direction opposite to a current direction of motion, such as to sharply reduce a speed of the unmanned aerial vehicle, or to bring the unmanned aerial vehicle to a stop.
Referring to
In some implementations, the range sensor 130-1 may be configured to calculate distances to objects at an angular range of three hundred sixty degrees about the range sensor 130-1, and at any maximum measuring radius, e.g., twelve meters (12 m) or more. For example, in some implementations, the range sensor 130-1 may be configured to transmit light at any measurement frequency, e.g., up to 4,500 Hertz (or 4,500 measurements per second). The transmitted light may be infrared light, or light at any other wavelength or frequency. In some implementations, the range sensor 130-1 may transmit light in a single direction, and capture or receive reflections of the light from one or more objects. A difference between a time at which light is transmitted, and a time at which reflections of the light are received, is commonly called a “time-of-flight,” and may be used with the speed of light to calculate distances to objects from which the light was reflected. In some implementations, the range sensor 130-1 may be configured to emit light at a wavelength of approximately 105 nanometers, e.g., within an infrared band, and at a power of approximately twenty-five milliwatts (25 mW).
The range sensor 130-1 may further combine distances calculated based on times-of-flight with information regarding angles at which the light was emitted and received to generate a point cloud or another representation of positions of objects, in any local or global coordinate system, which may be stored by the aerial vehicle 110 and processed or transmitted to one or more external devices or systems for processing. The range sensor 130-1 may also include one or more motor drivers or other components for controlling a start, a stop or a speed of a motor, and may generate and transmit packets of data including any information regarding transmissions or returns, e.g., rotational speeds, start or end angles, time stamps, or other related data.
As is also shown in
As is shown in
The aerial vehicle 110 may be configured to capture information or data using the range sensors 130-1, 130-2 while traveling within the facility 160. As is further shown in
As is further shown in
As is shown in
In some other implementations, a position or an orientation of the aerial vehicle 110 may be determined based on information or data captured by one or more other sensors, e.g., the range sensors 130-1, 130-2, which may detect one or more objects having known positions within the facility 160, e.g., according to a local or global coordinate system, and may determine a position or an orientation of the aerial vehicle 110 with respect to such objects.
During flight operations, the aerial vehicle 110 may capture information or data using one or more onboard sensors, e.g., the range sensors 130-1, 130-2, or any imaging devices, radar sensors, sonar sensors, or others, and determine whether any objects are present within a predetermined range or distance of the aerial vehicle 110, e.g., about an entire perimeter or circumferential range around the aerial vehicle 110, or within a portion of the perimeter or circumferential range. For example, as is shown in
As is shown in
Upon receiving data regarding one or more returns of reflections of the transmitted light, the range sensor 130-1 may determine an angle from the aerial vehicle 110 (e.g., from the range sensor 130-1) to an object (e.g., an obstacle), based on the angular interval by which the returns of the reflected light were received, as well as a distance to the object, based on an elapsed time between the transmission and the return, or in any other manner.
As is also shown in
Alternatively, the aerial vehicle 110 may be configured to capture data using the range sensors 130-1, 130-2, and interpret the data to determine whether any objects are present within a first zone or area defined by a predetermined range or distance forward of the aerial vehicle 110, and within a first arc or sector that has a radius of the predetermined range or distance and subtends an angle θ1 on either side of the flight direction of approximately twenty to thirty degrees, or any other angle. In some implementations, the aerial vehicle 110 may be further configured to interpret the data to determine whether any objects are present within a second area or zone defined by a predetermined range or distance aft of the aerial vehicle 110, or behind the aerial vehicle 110, and within a second arc or a sector that has a radius of the predetermined range or distance and subtends an angle θ2 on either side opposite the flight direction of approximately sixty degrees, or any other angle. Alternatively, the aerial vehicle 110 may be configured to determine whether any objects are present within any other area or zone defined by any range or distance and at any angle or direction with respect to the flight direction of the aerial vehicle 110.
As is shown in
The aerial vehicle 110 may determine a position of the ladder 175-2 in three-dimensional space, e.g., based on the navigation map 165, or in any other manner, and store the position of the ladder 175-2 in one or more data stores. In some implementations, a position of the ladder 175-2 may be determined based on times associated with reflections or returns of light from the ladder 175-2 by one or both of the range sensor 130-1 or the range sensor 130-2. For example, a two-dimensional point cloud or other representation determined by the range sensor 130-1 may be fused or combined with a three-dimensional depth image or other representation determined by the range sensor 130-2, to determine a position in three-dimensional space of the ladder 175-2, where a pose (e.g., a position or an orientation) of the aerial vehicle 110 are known.
As is shown in
In some implementations, a sector or a bearing to the sector may be defined relative to a location that is not coincident with a centroid of the aerial vehicle 110. For example, in some implementations, data generated by a range sensor that defines a location or bearing relative to the range sensor may be converted, transformed, or translated to define a location or bearing relative to a centroid of the aerial vehicle 110, or relative to any configured portion of an aerial vehicle.
In some implementations, the aerial vehicle 110 may determine that the set S is free and clear of objects beyond a predetermined range based on data received by the range sensor 130-1. For example, where the range sensor 130-1 is a LIDAR sensor, the aerial vehicle 110 may collect beams received by the range sensor 130-1 in the form of reflections from objects around the aerial vehicle 110, such as behind the aerial vehicle 110, or within the arc or sector subtending the angle θ2 opposite the flight direction as shown in
As is shown in
The aerial vehicle 110 executes the braking maneuver to safely avoid contacting not only the ladder 175-2 but also any other objects that are located at other bearings or in other directions with respect to the aerial vehicle 110, such as the lamp 175-1. The aerial vehicle 110 may then slow to a stop, e.g., to hover or land, or select another destination, and calculate a trajectory for traveling to the other destination.
The systems and methods of the present disclosure are directed to the indoor navigation and obstacle avoidance by aerial vehicles (e.g., unmanned aerial vehicles, or drones). The aerial vehicles of the present disclosure may be of any type or form, and may include but need not be limited to low-power drones that may be configured for efficient path planning, adaptive speed modification, and obstacle avoidance techniques during operations within indoor spaces.
In some implementations, an aerial vehicle traveling on a selected course, at a selected attitude and at a selected speed may continuously determine distances to objects around a perimeter of the aerial vehicle, e.g., using a continuously rotating two-dimensional LIDAR sensor, one or more time-of-flight sensors, or any other sensors, e.g., imaging devices, radar sensors, sonar sensors. The aerial vehicle may calculate or otherwise determine distances at angles about the perimeter of the aerial vehicle, e.g., based on returns received by a LIDAR sensor, a time-of-flight sensor, or others, and aggregate sets of the distance measurements for angular sectors about the perimeter. Such sectors may have any angular width, e.g., five degrees, or others. Minimum distances to objects within each of the angular sectors may be determined and stored in one or more memory components of the aerial vehicle. Where a set of adjacent angular sectors having minimum distances above a predetermined threshold are identified, a bearing associated with the set of adjacent angular sectors (e.g., a bearing to an angular center of the set) and the minimum distance measurement of the set are stored in a memory component of the aerial vehicle. Any number of sets of adjacent angular sectors may be identified in this manner, and bearings and minimum distance measurements of each of such sets may be stored by the aerial vehicle.
Subsequently, when an obstacle is detected within a flight path of the aerial vehicle, or it is otherwise determined that traveling on a current trajectory will cause the aerial vehicle to contact the obstacle, the aerial vehicle may automatically execute a braking maneuver, and select one of the sets of adjacent angular sectors on any basis. The aerial vehicle may generate or execute one or more instructions for decelerating and turning toward a bearing associated with the selected one of the sets of adjacent angular sectors, e.g., by inserting an intervening waypoint along the bearing, thereby reducing a risk that the aerial vehicle will contact the obstacle or any other objects during the braking maneuver. Distances to surrounding objects may be continuously determined while the aerial vehicle is engaged in flight operations, and sets of adjacent angular sectors may be identified or updated accordingly based on such distances.
In some implementations, an aerial vehicle may determine whether a path or a trajectory would cause the aerial vehicle to come into contact with one or more objects while traveling between waypoints. If the aerial vehicle determines that the path or the trajectory could cause the aerial vehicle to come into contact with or pass unacceptably close to any objects between any pair of waypoints, the aerial vehicle may insert an additional waypoint (e.g., an intervening waypoint at a midpoint between the pair of waypoints), and recalculate a trajectory that causes the aerial vehicle to pass through the additional waypoint. If the recalculated trajectory would cause the aerial vehicle to pass the obstacle at a safe distance, the aerial vehicle may proceed along the recalculated trajectory.
In some implementations, an aerial vehicle may be outfitted or equipped with one or more modules, e.g., hardware components or software applications to be executed by one or more hardware components. Such modules may include an obstacle detection module that may assess relative positions of obstacles in a given environment according to any local or global coordinate system, using temporal information to fuse data received from range sensors provided aboard the aerial vehicle, e.g., a rotating two-dimensional LIDAR sensor and time-of-flight sensors provided in fixed orientations with respect to a fuselage or other component of the aerial vehicle. Additionally, a range map, a point cloud or another representation may be subject to pixelization to determine whether any obstacles are present, and to minimize computation resources and processing power.
Additionally, in some implementations, an aerial vehicle may be configured to determine a state of risk that the aerial vehicle may collide with one or more objects, and to select a speed of the aerial vehicle based on the state of risk. For example, where the aerial vehicle does not detect any obstacles within a first predetermined range in a direction of travel of the aerial vehicle (or on a heading of the aerial vehicle), the aerial vehicle may travel at a maximum speed, or at any other safe speed, along the direction of travel, consistent with a state of low or insignificant risk. Alternatively, where the aerial vehicle detects one or more obstacles within the first predetermined range in the direction of travel, the aerial vehicle may execute an immediate braking maneuver, and may slow to a predetermined speed that may be a fraction of the maximum speed, consistent with a state of high or significant risk. For example, in some implementations, in order to immediately brake, an aerial vehicle may insert an intervening waypoint that is directly behind a position of the aerial vehicle, and calculate a trajectory that would cause the aerial vehicle to attempt to return to the intervening waypoint, e.g., by operating one or more motors or control surfaces, and to effectively brake as a result.
The aerial vehicle may recalculate a trajectory for the aerial vehicle at the reduced speed, and continue to determine whether any obstacles are present along the trajectory of the aerial vehicle within a second predetermined range that may be shorter than the first predetermined range, subject to any adjustments compensating for the motion of the aerial vehicle along the direction of travel. If an obstacle is present within the second predetermined range, the aerial vehicle may execute an emergency maneuver, e.g., to brake or to otherwise avoid the obstacle, such as by inserting an intervening waypoint or any other position immediately behind the aerial vehicle or in any other location with respect to a direction of travel or flight path of the aerial vehicle. If no obstacles are detected within the second predetermined range, then the aerial vehicle may continue to proceed at the predetermined reduced speed before accelerating to the maximum speed again upon confirming that the aerial vehicle is at a state of acceptable risk.
In accordance with some embodiments of the present disclosure, an aerial vehicle may be programmed or configured to generate one or more environment maps of spaces within a facility using images or other data captured by one or more onboard sensors. Such sensors may include digital cameras (e.g., visual or depth cameras) or other imaging devices, which may be aligned with fields of view or axes of orientation extending in any direction, e.g., forward, aft, port, starboard, up, down, or in any other direction as well as radar sensors, sonar sensors or others. For example, the imaging devices may have fields of view or axes of orientation that are aligned along or parallel to yaw, pitch or roll axes of an aerial vehicle, e.g., principal axes of the aerial vehicle, or at any other angle.
In some embodiments, an aerial vehicle may include one or more depth cameras or range sensors, such as LIDAR sensors or time-of-flight sensors, that are aligned at any angle with respect to an orientation or configuration of an aerial vehicle. For example, the aerial vehicle may include depth cameras or range sensors with fields of view or orientations pointing vertically upward or downward, as well as depth cameras or range sensors having fields of view or orientations that vary with respect to an orientation or configuration of the aerial vehicle. In some embodiments, an aerial vehicle may include one or more imaging devices such as depth cameras or range sensors (e.g., LIDAR sensors or other time-of-flight sensors) that are configured to determine ranges based on any number of pixels of data, e.g., grids of four pixels by four pixels, or eight pixels by eight pixels, or the like. In some embodiments, an aerial vehicle may include one or more imaging devices such as depth cameras or range sensors that are configured to determine ranges based on single pixels of data, e.g., a single-pixel LIDAR sensor or other sensor. Moreover, the aerial vehicle may include one or more depth cameras, range sensors or other systems that are aligned with fixed orientations, or with variable orientations (e.g., rotating or shifting orientations).
In some implementations, an aerial vehicle may include one or more time-of-flight sensors that are configured to generate distance data in a plurality of zones in a grid layout, e.g., a 4×4 grid of zones or an 8×8 grid of zones. For example, in such implementations, a time-of-flight sensor may comprise a VCSEL and diffractive optical elements that allow a square field of view to be projected onto a scene, as well as a SPAD array, and a receiver lens that focuses light onto the SPAD array.
The aerial vehicles of the present disclosure may be outfitted with one or more processors, components, transceivers, sensors or other systems for engaging in communications with aspects of a facility (e.g., appliances, lighting, environmental or other systems), as well as any persons within the facility. For example, an aerial vehicle may include any number of transceivers for communicating with aspects of the Internet or one or more other networks, including but not limited to any wired or wireless routers within a facility, or any other computer devices therein, as well as any number of sensors or readers for communicating via any wired or wireless systems or protocols, including but not limited to wireless fidelity (“Wi-Fi”), Bluetooth, radio frequency identification (or “RFID”), near-field communication (or “NFC”) readers, or any other type of systems or protocols. For example, the aerial vehicles may further include any number of audio or video sensors, including but not limited to one or more imaging devices (e.g., digital cameras) and/or microphones, or any other type of sensors, embedded or incorporated therein.
Additionally, an aerial vehicle may further include any number of sensors, such as imaging devices (e.g., cameras configured to capture visual or depth data), temperature sensors, magnetometers, Wi-Fi receivers, Bluetooth receivers, or others, and may be programmed or configured to travel throughout one or more spaces of a facility and to capture data using such sensors. Based on the captured data, an environment map of such spaces or the facility may be generated. The environment map may identify or depict one or more boundaries (e.g., walls, ceilings, floors) or other aspects of such spaces, as well as the respective dimensions of such spaces, or the respective surfaces or textures of such boundaries. In some embodiments, an aerial vehicle may autonomously travel throughout one or more spaces of a facility in order to capture data using one or more sensors, and such data may be utilized in generating an environment map of the spaces of the facility. In some other embodiments, an aerial vehicle may be transported (e.g., carried) or escorted by a human actor throughout such spaces, and may capture data using one or more sensors as the aerial vehicle is transported or escorted throughout such spaces. Data captured as the aerial vehicle is escorted may be utilized in generating an environment map of the spaces of the facility. Additionally, in some embodiments, the aerial vehicle may selectively operate one or more propulsion motors as the aerial vehicle is transported or otherwise escorted throughout such spaces, in order to maintain altitude and/or tilt control. Furthermore, in some embodiments, data captured by the aerial vehicle as the aerial vehicle travels throughout the spaces of the facility may be adjusted to account for presence of one or more body parts of a human actor that is transporting or otherwise escorting the aerial vehicle through the facility.
In accordance with some embodiments of the present disclosure, an aerial vehicle may be configured to operate along with one or more stations, e.g., base components, charging docks (or charging stations or docking stations), or other intermediary devices. Such stations may have openings, cavities or spaces configured to accommodate one or more portions of an aerial vehicle, and may include one or more surfaces that are aligned to come into contact with corresponding surfaces of the aerial vehicle, thereby enabling electrical power, information or data to be conveyed between the aerial vehicle and such stations. In some embodiments, a base component, a charging dock, or another intermediary device may include an opening, a cavity or another space that is sized and shaped to receive or accommodate a specific portion of an aerial vehicle, e.g., a fuselage of the aerial vehicle, and to enable the aerial vehicle to be aligned in one of a plurality of alignments or orientations with respect to the base component, the charging dock or intermediary device.
Moreover, aerial vehicles of the present disclosure may be configured for use or operation within facilities of any kind. As used herein, the term “facility” shall refer to any building, region, structure or other space (e.g., covered or uncovered), such as a home of any type, kind, shape or form, including but not limited to a house, an apartment, a condominium, a dormitory, a barracks, or any other defined or undefined structure having one or more living spaces. A facility may also be a business-related structure such as a building, an office, a shopping center, a restaurant, a post office, a grocery store, a department store, a materials handling facility, or any other defined or undefined structure having one or more commercial areas. A facility may also be any other type of facility including but not limited to stadiums, ballfields, transportation centers or financial institutions (e.g., banks). In some embodiments, the facility may be or include an island or a space station.
Referring to
The aerial vehicle 210 includes a processor 212, a memory 214 and a transceiver 216. The aerial vehicle 210 further includes a control system 220, a plurality of propulsion motors 222, an inertial sensor 224, a position sensor 226 and an imaging device 228. The aerial vehicle 210 also includes one or more range sensors 230 that may be mounted or coupled to surfaces of the aerial vehicle 210.
The processor 212 may be configured to perform any type or form of computing function associated with the operation of the aerial vehicle 210, including but not limited to the execution of one or more algorithms or techniques, such as for calculating delay times for the transmission and receipt of one or more signals by the transmitter, or for determining bearings or angles from which reflected energy originated based on one or more signals received. The processor 212 may also be configured to execute any other algorithms or techniques (e.g., object detection or recognition algorithms or techniques) associated with any number of applications, purposes or functions, or to select at least one of a course, a speed or an altitude for the safe operation of the aerial vehicle 210. For example, the processor 212 may be configured to control any aspects of the operation of the aerial vehicle 210 and the one or more computer-based components thereon, including but not limited to the propulsion motors 222, the inertial sensor 224, the position sensor 226 and/or the imaging device 228.
The processor 212 may also control the operation of one or more control systems or modules, such as the control system 220, for generating instructions for conducting operations of one or more of the propulsion motors 222, the inertial sensor 224, the position sensor 226 and/or the imaging device 228, or for interpreting information or data captured by one or more onboard sensors, e.g., the inertial sensor 224, the position sensor 226 and/or the imaging device 228, or others (not shown). Such control systems (or modules) 220 may be associated with one or more other computing devices or machines, and may communicate with the data processing system 280 or one or more other computer devices or aerial vehicles (not shown) over the network 290, through the sending and receiving of digital data, as indicated by line 218.
The processor 212 may be a uniprocessor system including one processor, or a multiprocessor system including several processors (e.g., two, four, eight, or another suitable number), and may be capable of executing instructions. For example, in some embodiments, the processor 212 may be a general-purpose or embedded processor unit such as a central processing unit (“CPU”), graphics processing unit (“GPU”) or a neural processing unit (“NPU”) having any number of instruction set architectures (ISAs), such as the x86, PowerPC, SPARC, or MIPS ISAs, or any other suitable ISA. Where the processor 212 is a multiprocessor system, each of the processors within the multiprocessor system may operate the same ISA, or different ISAs.
Additionally, the aerial vehicle 210 further includes one or more memory or storage components 214 (such as databases or data stores) for storing any type of information or data, e.g., instructions for operating the aerial vehicle 210, or information or data captured during operations of the aerial vehicle 210. For example, the memory 214 may be configured to store information or data regarding energy transmitted by the range sensors 230, information or data regarding energy captured by the range sensors 230, or any other information or data. The memory 214 may be configured to store executable instructions, imaging data, flight paths, flight control parameters and/or other data items accessible by or to the processor 212. The memory 214 may be implemented using any suitable memory technology, such as random-access memory (or “RAM”), static RAM (or “SRAM”), synchronous dynamic RAM (or “SDRAM”), nonvolatile/Flash-type memory, or any other type of memory. In some embodiments, program instructions, imaging data, flight paths, flight control parameters and/or other data items may be received or sent via the transceiver 216, e.g., by transmission media or signals, such as electrical, electromagnetic, or digital signals, which may be conveyed via a communication medium such as a wired and/or a wireless link.
The transceiver 216 may be configured to enable the aerial vehicle 210 to communicate through one or more wired or wireless means, e.g., wired technologies such as Universal Serial Bus (or “USB”) or fiber optic cable, or standard wireless protocols such as Bluetooth® or any Wi-Fi protocol, such as over the network 290 or directly. The transceiver 216 may further include or be in communication with one or more input/output (or “I/O”) interfaces, network interfaces and/or input/output devices, and may be configured to allow information or data to be exchanged between one or more of the components of the aerial vehicle 210, or to one or more other computer devices or systems (e.g., other aerial vehicles, not shown) via the network 290. For example, in some embodiments, the transceiver 216 may be configured to coordinate I/O traffic between the processor 212 and one or more onboard or external computer devices or components, e.g., the propulsion motors 222, the altimeter 224, the position sensor 226 and/or the imaging device 228. The transceiver 216 may perform any necessary protocol, timing or other data transformations in order to convert data signals from a first format suitable for use by one component into a second format suitable for use by another component. In some embodiments, the transceiver 216 may include support for devices attached through various types of peripheral buses, e.g., variants of the Peripheral Component Interconnect (or “PCI”) bus standard or the USB standard. In some other embodiments, functions of the transceiver 216 may be split into two or more separate components, or integrated with the processor 212.
The control system 220 may include one or more electronic speed controls, power supplies, navigation systems and/or payload engagement controllers for controlling aspects of the operation of the aerial vehicle 210, as desired. For example, the control system 220 may be configured to cause or control the operation of one or more of the propulsion motors 222, the altimeter 224, the position sensor 226 and/or the imaging device 228, such as to cause one or more of the propulsion motors 222 to rotate propellers at desired speeds, to capture information or data regarding altitudes, positions and/or speeds, and to cause one or more of the imaging devices 228 to capture any imaging data (e.g., still or moving images) as well as any associated audio data and/or metadata. The control system 220 may also operate the one or more propulsion motors 222 to cause such propellers to be aligned in selected positions or angles. The control system 220 may further control any other aspects of the aerial vehicle 210, including but not limited to the operation of one or more control surfaces (not shown) such as wings, rudders, ailerons, elevators, flaps, brakes, slats or other features within desired ranges, or the enactment with or release of one or more items by one or more engagement systems (not shown). In some embodiments, the control system 220 may be integrated with one or more of the processor 212, the memory 214 and/or the transceiver 216, and configured to receive commands or generate and provide status updates of propeller speeds, as well as times and lengths of any adjustments.
The propulsion motors 222 may be any type or form of motor (e.g., electric, gasoline-powered or any other type of motor) capable of generating sufficient rotational speeds of one or more propellers or other components to provide lift and/or thrust forces to the aerial vehicle 210 and any payload engaged thereby, to aerially transport the engaged payload thereby. In some embodiments, one or more of the propulsion motors 222 may be a brushless DC multi-phase motor such as an outrunner brushless motor or an inrunner brushless motor.
The aerial vehicle 210 may include any number of such propulsion motors 222 of any kind. For example, one or more of the propulsion motors 222 may be aligned or configured to provide forces of lift to the aerial vehicle 210, exclusively, while one or more of the propulsion motors 222 may be aligned or configured to provide forces of thrust to the aerial vehicle 210, exclusively. Alternatively, one or more of the propulsion motors 222 may be aligned or configured to provide forces of lift and forces of thrust to the aerial vehicle 210, as needed. For example, the propulsion motors 222 may be fixed in their orientation on the aerial vehicle 210, or configured to vary their respective orientations, e.g., a tilt-rotor aircraft. Moreover, the propulsion motors 222 may be aligned or configured to operate with different capacities or ratings, or at different speeds, or coupled to propellers having different sizes and shapes. Additionally, one or more of the propulsion motors 222 may be an electric motor, e.g., a brushless DC multi-phase motor, and one or more of the propulsion motors 222 may be a gasoline-powered motor.
Each of the propulsion motors 222 may be coupled to one or more propellers (or rotors or rotatable systems) having a plurality of shaped blades joined to a hub or boss. For example, each of such propellers may be rotatably mounted to a mast or shaft associated with a respective one of the propulsion motors 222 and may be configured to generate forces of thrust when rotated within a fluid. Each of such propellers may include any number of blades, and may be fixed pitch, adjustable pitch or variable pitch in nature. Moreover, one or more of such propellers may be banded or shielded in any manner. In some embodiments, one or more propellers may be configured to rotate about a vertical axis, and to provide forces of lift in a vertical direction (e.g., upward) accordingly. In some other embodiments, one or more of the propellers may be configured to rotate about a horizontal axis, and to provide forces of thrust in a horizontal direction (e.g., forward) accordingly. In still other embodiments, one or more of the propellers may be configured to rotate about axes that are neither horizontal nor vertical, and to provide forces of lift and/or thrust in directions corresponding to such axes accordingly.
The inertial sensor 224 may include one or more components for measuring linear and/or angular motion of the aerial vehicle 210. The inertial sensor 224 may include one or more gyroscopes (e.g., mechanical or electrical components or instruments for determining an orientation), one or more accelerometers (e.g., mechanical or electrical components or instruments for sensing or measuring accelerations), one or more compasses or other magnetometers (e.g., mechanical or electrical components or instruments for determining one or more directions with respect to a frame of reference that is fixed with respect to the surface of the Earth), or other components.
The position sensor 226 may be any device, component, system or instrument adapted to receive signals (e.g., trilateration data or information) relating to a position of the aerial vehicle 210 according to any local or global coordinate system, from one or more GPS satellites of a GPS network, from one or more towers or beacons from a cellular telephone network, or from any other source (not shown). In some embodiments, the position sensor 226, or position data received thereby, may be used to determine an airspeed of the aerial vehicle 210 over time. In some other embodiments, the aerial vehicle 210 may include one or more devices, components, systems, or instruments for determining a speed or velocity of the aerial vehicle 210, and may include related components (not shown) such as pitot tubes, accelerometers, or other features.
The imaging device 228 may be any form of optical recording devices that may be aligned with respect to any expected or ordinary operating orientation of the aerial vehicle 210, and are configured to photograph or otherwise record imaging data of objects or any other elements within fields of view forward of, aft of, lateral to, above or below the aerial vehicle 210, or for any other purpose. The imaging device 228 may include one or more processors, one or more memory or storage components, and one or more image sensors, e.g., color sensors, grayscale sensors, black-and-white sensors, depth sensors, or the like, and may further include one or more photosensitive surfaces, filters, chips, electrodes, clocks, boards, timers, power sources, connectors or any other relevant features (not shown). The imaging device 228 may capture imaging data in the form of one or more still or moving images of any kind or form, as well as any relevant audio signals or other information during the operation of the aerial vehicle 210.
The imaging device 228 may be mounted, fixed, embedded or otherwise joined to one or more external surfaces of the aerial vehicle 210 in any manner and in any orientation or alignment to capture imaging data from above the aerial vehicle 210. For example, the imaging device 228 may be coupled to any form of support system or structure for maintaining the lenses or other optical elements of the imaging device 228 at a selected orientation or configuration. Alternatively, the imaging device 228 may be mounted, fixed, embedded or otherwise joined to external surfaces of the aerial vehicle 210 in any other manner.
The imaging device 228 may communicate with the processor 212 and/or the control system 220, or with one another, by way of a wired or wireless connection that may be dedicated or comprise all or part of an internal network (not shown), e.g., an internal communications bus. Additionally, the imaging device 228 may be adapted or otherwise configured to communicate with the data processing system 280 by way of the network 290. The imaging device 228 may be of any type or form in accordance with the present disclosure, including but not limited to one or more digital cameras, depth sensors or range cameras, infrared cameras, radiographic cameras or other optical sensors.
The range sensors 230 may be any devices or systems for determining ranges or distances between the aerial vehicle 210 and one or more objects. In some implementations, the range sensors 230 may include one or more time-of-flight sensors, which may comprise illuminators or other light sources (e.g., modulated light sources) that are configured to transmit light along axes or directions that are normal to and extend radially outward from surfaces of the aerial vehicle 210, and receivers that are configured to capture reflections of the light off one or more of such surfaces. Reflected light captured by the range sensors 230 may be interpreted to generate a depth image or range profile of distances between the range sensors 230 and the one or more surfaces, several times per second.
In some embodiments, one or more of the range sensors 230 may be configured to rotate an illuminator, or an axis or a direction along which the range sensors 230 transmit light, e.g., about an axis along which a geometric center or centroid of the aerial vehicle 210 or a fuselage thereof are coaligned. By rotating the illuminator or the transmissions of light about the axis, one or more of the range sensors 230 may be used to generate depth images, range profiles or other sets of distances covering three hundred sixty degrees around the aerial vehicle 210.
The range sensors 230 may also include illuminators or other light sources (e.g., modulated light sources) that are configured to transmit light along axes that are parallel to the axis along which the geometric center or centroid of the aerial vehicle 210 or a fuselage thereof are coaligned, and in opposite directions, e.g., up and down. The range sensors 230 may be further configured to capture reflections of the light transmitted thereby off one or more surfaces above or below the aerial vehicle 210, and to interpret such reflections in order to generate depth images, range profiles or other sets of distances between the range sensors 230 and the one or more surfaces, several times per second.
Alternatively, the range sensors 230 may be configured to emit acoustic signals in a single direction, or in multiple directions. For example, in some embodiments, the range sensors 230 may be configured to transmit continuous-wave (or “CW”) pulses, e.g., pings, which may take the form of phase-coded transmit signals having bandwidths determined by the phase coding or frequency spread of the signals. Alternatively, in some other embodiments, the range sensors 230 may be configured to emit electromagnetic signals in one or more waves or pulses. The range sensors 230 may be configured to transmit signals at any power level, frequency (e.g., center frequency), wavelength or bandwidth, and within any total frequency range. The range sensors 230 may further include one or more receivers (or receiving elements) for receiving energy (e.g., acoustic energy or, alternatively, electromagnetic energy), or capturing information or data relating to received energy, that are configured to be mounted to a surface, e.g., one or more surfaces of the aerial vehicle 210. The range sensors 230 may be configured to receive energy at any power level, frequency (e.g., center frequency), wavelength or bandwidth, and within any total frequency range.
In addition to the inertial sensors 224, the position sensor 226, the imaging device 228, or the range sensors 230, the aerial vehicle 210 may also include any number of other sensors, components or other features for controlling or aiding in the operation of the aerial vehicle 210, including but not limited to one or more environmental or operational sensors for determining one or more attributes of an environment in which the aerial vehicle 210 is operating, or may be expected to operate, including extrinsic information or data or intrinsic information or data. For example, the aerial vehicle 210 may include one or more compasses, speedometers, thermometers, barometers, hygrometers, gyroscopes, air monitoring sensors (e.g., oxygen, ozone, hydrogen, carbon monoxide or carbon dioxide sensors), ozone monitors, pH sensors, magnetic anomaly detectors, metal detectors, radiation sensors (e.g., Geiger counters, neutron detectors, alpha detectors), attitude indicators, depth gauges, accelerometers, or sound sensors (e.g., microphones, piezoelectric sensors, vibration sensors or other transducers for detecting and recording acoustic energy from one or more directions).
Although the block diagram of the system 200 shown in
The data processing system 280 includes one or more physical computer servers 282 having one or more computer processors 284 and any number of data stores 286 (e.g., databases) associated therewith, as well as provided for any specific or general purpose. For example, the data processing system 280 of
The servers 282 and/or the computer processors 284 may also connect to or otherwise communicate with the network 290, as indicated by line 288, through the sending and receiving of digital data. For example, the data processing system 280 may include any facilities, stations or locations having the ability or capacity to receive and store information or data in one or more data stores, e.g., from the aerial vehicle 210, from one or more other aerial vehicles, or from one or more other external computer systems (not shown) via the network 290. In some embodiments, the data processing system 280 may be provided in a physical location. In other such embodiments, the data processing system 280 may be provided in one or more alternate or virtual locations, e.g., in a “cloud”-based environment. In still other embodiments, the data processing system 280 may be provided onboard one or more aerial vehicles, including but not limited to the aerial vehicle 210.
The network 290 may be any wired network, wireless network, or combination thereof, and may comprise the Internet in whole or in part. In addition, the network 290 may be a personal area network, local area network, wide area network, cable network, satellite network, cellular telephone network, or combination thereof. The network 290 may also be a publicly accessible network of linked networks, possibly operated by various distinct parties, such as the Internet. In some embodiments, the network 290 may be a private or semi-private network, such as a corporate or university intranet. The network 290 may include one or more wireless networks, such as a Global System for Mobile Communications (GSM) network, a Code Division Multiple Access (CDMA) network, a Long-Term Evolution (LTE) network, or some other type of wireless network. Protocols and components for communicating via the Internet or any of the other aforementioned types of communication networks are well known to those skilled in the art of computer communications and thus, need not be described in more detail herein.
The computers, servers, devices and the like described herein have the necessary electronics, software, memory, storage, databases, firmware, logic/state machines, microprocessors, communication links, displays or other visual or audio user interfaces, printing devices, and any other input/output interfaces to provide any of the functions or services described herein and/or achieve the results described herein. Also, those of ordinary skill in the pertinent art will recognize that users of such computers, servers, devices and the like may operate a keyboard, keypad, mouse, stylus, touch screen, or other device (not shown) or method to interact with the computers, servers, devices and the like, or to “select” an item, link, node, hub or any other aspect of the present disclosure.
In some embodiments, the processor 212, the servers 282 and/or the processors 284 may be configured to generate two-dimensional or three-dimensional maps or other representations of locations of objects, as determined based on angles at which signals are transmitted by the range sensors 230, or bearings or angles to objects from which energy including reflections of the signals are captured by the range sensors 230. In some embodiments, the processor 212, the servers 282 and/or the processors 284 may be configured to determine an optimal path or route between two locations for the execution of a given task by the aerial vehicle 210 or one or more other aerial vehicles (not shown). The processor 212, the servers 282 and/or the processors 284 may determine an optimal path or route based on any factor or element, including but not limited to times required to travel on any paths of an optimal route, any costs associated with traveling on the paths, or any other intrinsic or extrinsic factors, such as according to one or more traditional shortest path or shortest route algorithms.
The aerial vehicle 210 and/or the data processing system 280 may use any applications, features, or techniques to connect to the network 290, or to communicate with one another. For example, the aerial vehicle 210 may be adapted to transmit information or data in the form of synchronous or asynchronous messages to the data processing system 280 or to any other computer device (e.g., to one or more other aerial vehicles) in real time or in near-real time, or in one or more offline processes, via the network 290. Those of ordinary skill in the pertinent art would recognize that the aerial vehicle 210 or the data processing system 280 may operate or be operated by any of a number of computing devices that are capable of communicating over the network, including but not limited to set-top boxes, mobile devices, laptop computers, desktop computers, and the like. The protocols and components for providing communication between such devices are well known to those skilled in the art of computer communications and need not be described in more detail herein.
The data and/or computer-executable instructions, programs, firmware, software and the like (also referred to herein as “computer-executable” components) described herein may be stored on a computer-readable medium that is within or accessible by computers or computer components such as the processor 212, the servers 282 and/or the processors 284, or any other computers or control systems utilized by the aerial vehicle 210 or the data processing system 280 (e.g., by one or more other aerial vehicles), and having sequences of instructions which, when executed by a processor (e.g., a CPU, a GPU or an NPU), cause the processor to perform all or a portion of the functions, services and/or methods described herein. Such computer-executable instructions, programs, software, and the like may be loaded into the memory of one or more computers using a drive mechanism associated with the computer-readable medium, such as a floppy drive, CD-ROM drive, DVD-ROM drive, network interface, or the like, or via external connections.
Some embodiments of the systems and methods of the present disclosure may also be provided as a computer-executable program product including a non-transitory machine-readable storage medium having stored thereon instructions (in compressed or uncompressed form) that may be used to program a computer (or other electronic device) to perform processes or methods described herein. The machine-readable storage media of the present disclosure may include, but is not limited to, hard drives, floppy diskettes, optical disks, CD-ROMs, DVDs, ROMS, RAMs, erasable programmable ROMs (“EPROM”), electrically erasable programmable ROMs (“EEPROM”), flash memory, magnetic or optical cards, solid-state memory devices, or other types of media/machine-readable medium that may be suitable for storing electronic instructions. Further, embodiments may also be provided as a computer-executable program product that includes a transitory machine-readable signal (in compressed or uncompressed form). Examples of machine-readable signals, whether modulated using a carrier or not, may include, but are not limited to, signals that a computer system or machine hosting or running a computer program can be configured to access, or including signals that may be downloaded through the Internet or other networks.
Any of the functions, calculations, determinations or other processing steps described herein may be performed locally, e.g., by one or more computer processors provided aboard an aerial vehicle, or remotely, e.g., by one or more computer systems in communication with an aerial vehicle. For example, in some implementations, one or more processors provided aboard an aerial vehicle, e.g., the processors 212, may interpret acoustic data captured by one or more microphones provided aboard the aerial vehicle, and calculate distances, bearings or ranges to objects based on the acoustic data. In some other implementations, one or more processors provided in association with an external computer system in communication with an aerial vehicle, e.g., the servers 282, may interpret distances, bearings or ranges to objects and construct an environment map (or internal representation) of a space or area in which the aerial vehicle operated based on such distances, bearings or ranges. Alternatively, any of such functions, calculations, determinations or other processing steps may be performed aboard the aerial vehicle or by one or more back-end systems.
Referring to
As is shown in
The fuselage 313 comprises a housing or chamber having a shape that is also defined by a plurality of sides (e.g., four sides) and a common height. For example, as is shown in
As is also shown in
The frame 311 and the fuselage 313 are mounted to one another in a manner that causes a geometric center or centroid of a cross-section of the frame 311 to be aligned along a common axis with a geometric center or centroid of a cross-section of the fuselage 313. As is shown in
The time-of-flight sensor module 330-1 may comprise an illuminator or other light source (e.g., a modulated light source) that is configured to transmit light along axes or directions that are normal to, and extend radially outward from, the axis along which the geometric center or centroid of the frame 311 and the geometric center or centroid of the fuselage 313 are coaligned. The time-of-flight sensor module 330-1 is further configured to capture reflections of the light off one or more surfaces. Reflected light captured by the time-of-flight sensor module 330-1 may be interpreted to generate a depth image or range profile of distances between the time-of-flight sensor module 330-1, and the one or more surfaces, several times per second.
In some embodiments, the time-of-flight sensor module 330-1 may be configured to rotate an illuminator, or the axis or the direction along which the time-of-flight sensor module 330-1 transmits light, about the axis along with the geometric center or centroid of the frame 311 and the geometric center or centroid of the fuselage 313 are coaligned. By rotating the illuminator or the transmissions of light about the axis along with the geometric center or centroid of the frame 311, and the geometric center or centroid of the fuselage 313 are coaligned, the time-of-flight sensor module 330-1 may be used to generate depth images, range profiles or other sets of distances covering three hundred sixty degrees around the aerial vehicle 310.
The time-of-flight sensors 330-2, 330-3 also include illuminators or other light sources (e.g., modulated light sources) that are configured to transmit light along axes that are parallel to the axis along which the geometric center or centroid of the frame 311 and the geometric center or centroid of the fuselage 313 are coaligned, and in opposite directions, e.g., up and down, with respect to the orientation of the aerial vehicle 310 as shown in
The aerial vehicle 310 includes a visual camera 330-4 or other imaging device embedded or installed within the housing or chamber defined by the fuselage 313. The visual camera 330-4 has a lens and an image sensor defining a field of view extending normal to one of the sides of the fuselage 313. Alternatively, the aerial vehicle may include any number of the visual cameras 330-4, which may be provided on the same side of the fuselage 313, or on different sides of the fuselage 313. In some embodiments, the aerial vehicle 310 may include at least one visual camera 330-4 provided on each of the sides of the fuselage 313, thereby covering three hundred sixty degrees around the aerial vehicle 310. Alternatively, in some other embodiments, one or more visual cameras 330-4 may be mounted in association with the frame 311 or in any other location on the aerial vehicle 310. The housing or chamber defined by the fuselage 313 may further include any number of other components therein, including but not limited to processors, control systems, memory components, batteries or other power sources, transceivers, circuitry, or other sensors.
As is shown in
As is also shown in
As is also shown in
As is further shown in
In accordance with embodiments of the present disclosure, the top cover 315-1 and the bottom cover 315-2 may be mounted to the upper and lower edges of the frame 311 in any manner, such as by clips, screws, bolts, snap-fit connections, or any other features. Alternatively, or additionally, the covers 315-1, 315-2 may be mounted to the fuselage 313, the struts 317, the propulsion motors 325, the time-of-flight sensor modules 330-1, 330-2, 330-3 or any other portion of the aerial vehicle 310 in any manner. Moreover, as is further shown in
As is further shown in
Furthermore, the frame 311, the fuselage 313, the covers 315-1, 315-2 and the struts 317, or external portions of housings of the propulsion motors 325, or any other components of the aerial vehicle 310, may be formed as single pieces, or as two or more pieces, that may be joined together or combined to form such components. For example, as is shown in
In some embodiments, the top cover 315-1, the bottom cover 315-2 and one or more portions of the fuselage 313 may be mated together in a manner that defines a cavity or volume that includes the propulsion motors 325 and any rotating propellers therein. For example, as is shown in
The unmanned aerial vehicles of the present disclosure may be formed from materials of any type or form. For example, in some embodiments, one or more of the frame 311, the fuselage 313, the covers 315-1, 315-2 and the struts 317, or external portions of housings of the propulsion motors 325, may be formed from one or more polycarbonates, or from one or more styrenes, e.g., polystyrenes or acrylonitrile butadiene styrenes (or “ABS”), or blends thereof, including but not limited to blends of polycarbonates and ABS. Alternatively, or additionally, one or more of the frame 311, the fuselage 313, the covers 315-1, 315-2 and the struts 317, or external portions of housings of the propulsion motors 325, may be formed from any other lightweight, durable materials such as plastics (e.g., thermosetting plastics such as epoxy or phenolic resins, polyurethanes or polyesters, as well as polyethylenes, polypropylenes or polyvinyl chlorides, and also recycled plastics), wood (e.g., woods with sufficient strength properties such as ash), metals (e.g., lightweight metals such as aluminum, or metals of heavier weights including alloys of steel), composites (e.g., carbon fiber), rubbers, or any other combinations of materials.
Aerial vehicles of the present disclosure may be operated in any manner to capture information or data, e.g., distances or ranges to surfaces of objects above, below or around the aerial vehicles, as well as images of their surroundings, or any other information or data. Referring to
As is shown in
The range sensor 430-1 extends above the aerial vehicle 410 and is configured to transmit light on axes or directions that may rotate at any angular velocity @, such as ten revolutions per second, relative to the fuselage 413. The range sensors 430-2, 430-3 are configured to transmit light in fixed axes or directions above and below the aerial vehicle 410, respectively. The range sensor 430-4 extends radially outward from the fuselage 413, and is configured to transmit light on a fixed axis or direction relative to the fuselage 413. In some implementations, one or more of the range sensors 430-1, 430-2, 430-3, 430-4 may be or include additional sensors or components. Alternatively, one or more of the range sensors 430-1, 430-2, 430-3, 430-4 may be replaced by other sensors or components.
In some implementations, fields of view of the range sensors 430-2, 430-3, 430-4 may define any angles with respect to their axes of orientation. For example, in some implementations, the fields of view may extend from the respective range sensors 430-2, 430-3, 430-4 at angles of twenty to thirty degrees on either side of their respective axes of orientation. The range sensors 430-2, 430-3, 430-4 may thus define image planes in the shapes of squares having sides of approximately sixteen centimeters (16 cm) each at a distance of twenty centimeters (20 cm) from the respective sensors, or sides of approximately twenty-eight centimeters (28 cm) each at a distance of thirty-five centimeters (35 cm) from the respective sensors. Alternatively, the sensors 430-2, 430-3, 430-4 may define fields of view having any other angles, and with any other dimensions, in accordance with implementations of the present disclosure.
Reflections of light captured by the range sensors 430-1, 430-2, 430-3, 430-4 may be interpreted to construct depth images, range profiles or other sets of distances between the range sensors 430-1, 430-2, 430-3, 430-4 and one or more surfaces above, below or around the aerial vehicle 410, or forward of the aerial vehicle 410, several times per second.
Thus, as is shown in
Referring to
At box 510, an aerial vehicle travels in flight on a selected course, at a selected altitude and at a selected speed. For example, the aerial vehicle may be programmed with a plurality of waypoints for traveling within one or more indoor spaces, and such waypoints may be represented in any manner, e.g., by waypoint data or spatial point data including one or more sets of coordinates in three-dimensional space. The waypoints may be associated with any mission to be executed by aerial vehicles within the indoor spaces in general, or by the aerial vehicle in particular.
The aerial vehicle may calculate a trajectory for traveling along paths between the plurality of waypoints, e.g., to minimize snap at a safe speed. For example, the aerial vehicle may calculate a trajectory that calls for travel at a safe speed, beginning from a velocity of zero at a position in three-dimensional space corresponding to an initial waypoint and ending at a velocity of zero at a position in three-dimensional space corresponding to the final waypoint. A trajectory may also include, for each defined point in three-dimensional space forming part of the trajectory, a time value indicating a time at which the trajectory passes through that point (e.g., a time at which an aerial vehicle following that trajectory should pass through that point). Thus, a trajectory may be characterized as defining points in four-dimensional space-time.
A control system or another processor-driven system aboard the aerial vehicle may then generate and transmit one or more instructions or commands for causing the aerial vehicle to travel on a calculated trajectory, e.g., at one or more selected courses, speeds or altitudes. In some implementations, the aerial vehicle may generate commands or instructions at any rate or frequency, e.g., ten times per second, or at ten Hertz (10 Hz), and execute such commands or instructions to operate one or more motors or control surfaces accordingly.
At box 515, distance measurements around the aerial vehicle are determined based on data captured by the aerial vehicle in flight. In some implementations, the aerial vehicle may be outfitted or equipped with one or more range sensors, such as LIDAR sensors, imaging devices, radar sensors, sonar sensors, or others, that are configured to calculate distances to objects at an angular range of three hundred sixty degrees about the respective sensors, and at any maximum measuring radii, e.g., twelve meters (12 m) or more. Such range sensors may be configured to transmit light at any measurement frequency, e.g., up to 4,500 Hertz (or 4,500 measurements per second), and the transmitted light may be infrared light, or light at any other wavelength or frequency. The range sensors may transmit light in single directions, and capture or receive reflections of the light from one or more objects. Differences between times at which light is transmitted, and times at which reflections of the light are received, are commonly called “times-of-flight,” and may be used with the speed of light to calculate distances to objects from which the light was reflected. Such range sensors may emit light at wavelengths of approximately 905 nanometers, e.g., within an infrared band, and at a power of approximately twenty-five milliwatts (25 mW), or at any other wavelengths or power levels.
The distance measurements may be stored in one or more memory components of the aerial vehicle in association with a state of the aerial vehicle, e.g., a velocity at which the aerial vehicle is traveling when the distance measurements are determined, or a position where the aerial vehicle is located when the distance measurements are determined. In some implementations, the distance measurements may be determined for an entire perimeter or circumference around the aerial vehicle. Alternatively, the distance measurements may be determined for less than the entire perimeter or circumference around the aerial vehicle, e.g., for one or more sectors, areas or zones that may be located or defined with respect to a direction of travel of the aerial vehicle, or with respect to one or more principal axes of the aerial vehicle, e.g., a range sensor access generally parallel to a yaw axis of the aerial vehicle. In some implementations, the distance measurements may be determined for an area or zone defined by a predetermined range or distance forward or aft of the aerial vehicle, and within an arc or a sector having a radius of the predetermined range or distance that subtends a predetermined angle on either side of a direction of travel of the aerial vehicle, or with respect to one or more principal axes of the aerial vehicle, e.g., a roll axis or a pitch axis of the aerial vehicle. For example, in some implementations, distance measurements may be determined for an area or zone that extends behind the aerial vehicle and is defined by a distance within a range of twenty-five to fifty centimeters (25-50 cm), and an arc or sector subtending an angle of one hundred twenty degrees (120°), or sixty degrees (60°) on either side of a predetermined axis extending opposite a direction of travel of the aerial vehicle.
Alternatively, the aerial vehicle may be configured to determine or calculate distance measurements around the aerial vehicle in any other manner. For example, in some implementations, the aerial vehicle may include one or more speakers (or other acoustic emitters) and microphones (or other acoustic sensors) provided in any location. The speakers may be configured to convert electrical signals into acoustic energy of any intensity and across any or all frequencies, which may be emitted as sound or other acoustic energy, and reflected or scattered on any surfaces of objects, while the microphones may capture and interpret reflections of the acoustic energy. Distances between the aerial vehicle and objects from which the reflections are received may be determined by dividing a product of one half of a speed of sound in air, and a difference between times at which sounds were emitted and times at which reflections of the sounds were captured. The aerial vehicle may include any number of other systems or sensors for determining distances to objects in accordance with implementations of the present disclosure.
In some implementations, reflections of light captured by a range sensor may be determined and clustered based on their respective ranges and angles from positions of the aerial vehicle. A position of an object may be determined based on the clusters, with respect to their positions according to any local or global coordinate system. Where distance measurements are determined based on returns captured by a sensor (e.g., a LIDAR sensor) having a field of view that rotates relative to the aerial vehicle, clusters of returns that are detected in one pass, sweep or other operation of the sensor may be compared to clusters of returns that are detected in subsequent passes, sweeps or operations of the sensor. Where clusters of returns are substantially similar, e.g., in both range and angle, the clusters may be identified as representing a common object. Moreover, where clusters of returns identified in consecutive passes, sweeps or operations of a sensor have substantially similar shapes or sizes, the clusters may be identified as representing a common object. In some implementations, an object from which the clusters of returns are received may be classified or identified based on a shape or a size of returns received from the object. For example, where returns from an object have a shape or a size consistent with a shape or a size of a body part of a human or another animal, the object may be classified accordingly. Conversely, where a single return or a cluster of fewer than a predetermined number of returns detected in one pass, sweep or operation of a sensor is not detected in a subsequent pass, sweep or operation of the sensor, the return or returns may be disregarded as corresponding to a “phantom” object.
At box 520, sets of distance measurements for angular sectors about the aerial vehicle are aggregated. The angular sectors may be defined as having any widths or resolutions, such as one degree (1°), five degrees (5°), ten degrees (10°), or any other width or resolution. For example, where the aerial vehicle is outfitted with a LIDAR sensor having a planar field of view with a fixed angular width that may rotate at any angular velocity relative to one or more principal axes of the aerial vehicle, returns of reflected light may be downsampled or decimated into bins or other angular sectors having five-degree (5°) widths.
In some implementations, the sets of distance measurements for angular sectors may be determined or aggregated following a single pass, sweep or operation of a LIDAR sensor. Alternatively, in some other implementations, sets of distance measurements for angular sectors may be determined or aggregated following a predetermined number of passes, sweeps or operations of a LIDAR sensor, e.g., based on a common set of consecutive returns for each of such sectors.
At box 525, minimum distance measurements are determined for each of the angular sectors about the aerial vehicle. For example, where distance measurements are determined based on returns of reflected light captured by a LIDAR sensor, a minimum distance measurement within each of the respective bins or other angular sectors may be identified and stored in association with the respective bins or angular sectors, e.g., in one or more memory compartments of the aerial vehicle.
At box 530, one or more sets of adjacent angular sectors having minimum distance measurements above a predetermined threshold are identified. For example, in some implementations, sets of adjacent angular sectors or bins may be defined as having minimum distances in excess of a single predetermined threshold, e.g., thirty centimeters (30 cm), or any other thresholds. Alternatively, sets of adjacent angular sectors or bins may be defined as having minimum distances in excess of other predetermined thresholds, e.g., not only thirty centimeters (30 cm) but also twenty centimeters (20 cm), ten centimeters (10 cm), or any other threshold. Moreover, based on the distances associated with the respective angular sectors or bins, positions of any number of objects may be determined with respect to a position of the aerial vehicle, e.g., ranges and angles, according to any local or global coordinate system.
At box 535, bearings and minimum distance measurements for the sets of adjacent angular sectors identified at box 530 are stored in one or more memory components of the aerial vehicle. For example, a set of adjacent angular sectors or bins may be defined between a bearing corresponding to a first angular sector or bin of the set, and a bearing corresponding to a last angular sector of the set. A bearing for a set of adjacent angular sectors or bins may be defined based on a center or middle angular sector or bin of the set, or a line bisecting an angle between bearings corresponding to first and last angular sectors of the set. Moreover, a minimum distance measurement of a set of adjacent angular sectors may be identified as the smallest minimum distance of any of the angular sectors or bins of the set.
At box 540, whether an object is detected in a flight path of the aerial vehicle is determined. An aerial vehicle may detect one or more stationary or mobile objects in any manner. For example, in some implementations, an object may be detected where a predetermined number of returns from a range sensor indicate that the object is within a predetermined range of the aerial vehicle. A range sensor by which an object is detected may be the same sensor from which distance measurements were determined at box 515, or a different sensor. In some implementations, an object may be detected where a predetermined number of consecutive returns indicates the presence of the object within the predetermined range, e.g., ten returns at a ten Hertz (10 Hz) frequency, or for one second. Alternatively, the object may be detected using an imaging device, or any other sensor.
In some implementations, an object may be detected in accordance with an algorithm that considers or defines multiple states of risk or readiness for the aerial vehicle. For example, the aerial vehicle may be determined to be in a first state, e.g., a low-risk state, where no objects are detected within a first range or distance, e.g., a maximum range or distance, and may operate in an unrestricted manner while in the first state. Alternatively, the aerial vehicle may be determined to be in a second state, e.g., a medium-risk state, where one or more objects are detected within the first range or distance, but beyond a second range or distance, e.g., a minimum range or distance, for a predetermined number of consecutive passes, sweeps or operations of a sensor. The aerial vehicle may operate subject to one or more restrictions on speed or altitude, or any other aspect of operations, while in the second state. The aerial vehicle may be determined to be in a third state, a high-risk state, where one or more objects are within the second range or distance, which may be defined based on a maximum speed and latency of the aerial vehicle. The aerial vehicle may operate only as necessary, and with strict restrictions on speed or altitude, or any other aspect of operations, while in the third state. Alternatively, any number of states may be defined with respect to any levels of risk or readiness for the aerial vehicle, and the aerial vehicle may operate subject to any restrictions associated with such states.
Additionally, where a position and a velocity of the aerial vehicle are known at a time that an object is detected, a position of the object may be determined based on a distance or range and angle to the object. Positions of objects may be stored in one or more memory components provided aboard the aerial vehicle, or transmitted to one or more external systems, which may be provided in one or more alternate or virtual locations, e.g., in a “cloud”-based environment. Furthermore, aspects of motions of an object, e.g., velocities or accelerations, or others, may be determined where the object is represented in a cluster of returns and detected within consecutive or multiple passes, sweeps or operations of a sensor.
If an object is not detected in the flight path of the aerial vehicle, then the process advances to box 550, where whether the aerial vehicle has arrived at a destination is determined. If the aerial vehicle has arrived at the destination, then the process ends. If the aerial vehicle has not arrived at the destination, however, then the process returns to box 510, where the aerial vehicle continues to travel on a selected course, at a selected altitude and at a selected speed, and to box 515, where distance measurements around the aerial vehicle are determined based on data captured by the aerial vehicle in flight.
If an object is detected in the flight path of the aerial vehicle, then the process advances to box 545, where the aerial vehicle brakes and turns to a bearing of the set of adjacent angular sectors having a greatest minimum distance measurement, and the process ends. Where information regarding the sets of adjacent angular sectors or bins including their respective minimum distances and bearings is known, and the aerial vehicle detects an object within its flight path, the aerial vehicle may select one of the sets of adjacent angular sectors or bins having the greatest minimum distance, and execute one or more instructions for causing the aerial vehicle to execute a braking maneuver in order to decelerate and turn in a direction of a bearing of the selected one of the sets of adjacent angular sectors or bins. For example, in some implementations, the aerial vehicle may insert an intervening waypoint along the bearing of the selected one of the sets of adjacent angular sectors and calculate a new trajectory that calls for traveling to or through the intervening waypoint. The aerial vehicle may then execute any number of instructions for causing one or more propulsion motors, control surfaces or other features of the aerial vehicle to operate to cause the aerial vehicle to travel along the newly calculated trajectory.
Alternatively, a set of adjacent angular sectors or bins may be selected on any other basis. For example, in some implementations, a set of adjacent angular sectors may be selected based on an angular width, such that the aerial vehicle is caused to travel on a bearing toward a widest one of the sets of adjacent angular sectors.
In some implementations, the aerial vehicle may select the set of adjacent angular sectors or bins in an iterative manner. For example, where information regarding the sets of adjacent angular sectors or bins including their respective minimum distances and bearings is known, and the aerial vehicle detects an object within its flight path, the aerial vehicle may determine whether any of the sets of adjacent angular sectors has a minimum distance measurement greater than a first threshold, e.g., thirty centimeters (30 cm), and select the one of the sets having the greatest minimum distance measurement of such sets. If none of the sets has a minimum distance measurement greater than the first threshold, however, then the aerial vehicle may identify a second threshold, smaller than the first threshold, e.g., twenty-five centimeters (25 cm), and determine whether any of the sets of adjacent angular sectors has a minimum distance measurement greater than the second threshold. The thresholds may be selectively reduced, e.g., by an iterative step amount, such as five centimeters (5 cm), ten centimeters (10 cm), or others, until one of the sets of adjacent angular sectors in excess of a threshold is identified.
In some implementations, one or more steps of the flow chart 500 of
Referring to
As is shown in
The aerial vehicle 610 may be configured to execute any number of instructions upon detecting an object from any number of sensors. For example, upon determining that an object is detected within a predetermined range (or distance) D1, the aerial vehicle 610 may begin determining distance measurements for each of a plurality of sectors within the area or zone behind the aerial vehicle 610, and identifying sets of adjacent sectors having distances above a predetermined threshold. The aerial vehicle 610 may also execute one or more instructions for reducing its speed, or for taking any other actions. Upon determining that an object is detected within a predetermined range (or distance) D2 of the aerial vehicle 610, the aerial vehicle 610 may be configured to select one of the sets of adjacent sectors and execute one or more instructions for braking and turning toward a bearing of the selected one of the sets of adjacent sectors.
As is shown in
As is further shown in
As is shown in
As is shown in
Subsequently, as is shown in
Referring to
At box 710, an aerial vehicle travels in flight on a selected course, at a selected altitude and at a selected speed. For example, the aerial vehicle may be programmed with a plurality of waypoints for traveling within one or more indoor spaces and configured to calculate a trajectory for traveling along paths between such waypoints. The aerial vehicle may further include a control system or another processor-driven system configured to generate and transmit one or more instructions or commands for causing the aerial vehicle to travel on a trajectory, and the aerial vehicle may execute such commands or instructions to operate one or more motors or control surfaces accordingly.
At box 715, distance measurements around the aerial vehicle are determined based on data captured by the aerial vehicle in flight. In some implementations, the aerial vehicle may be outfitted or equipped with one or more range sensors that are configured to calculate measurements of distances to objects around the aerial vehicle, and such distance measurements may be stored in one or more memory components of the aerial vehicle, e.g., in association with a state of the aerial vehicle. In some implementations, the distance measurements may be determined for an entire perimeter or circumference around the aerial vehicle. Alternatively, the distance measurements may be determined for less than the entire perimeter or circumference around the aerial vehicle, e.g., for one or more sectors, areas or zones that may be located or defined with respect to a direction of travel of the aerial vehicle, or with respect to one or more principal axes of the aerial vehicle, e.g., a yaw axis of the aerial vehicle.
At box 720, sets of distance measurements for angular sectors about the aerial vehicle are aggregated. The angular sectors may be defined as having any widths or resolutions, such as one degree (1°), five degrees (5°), ten degrees (10°), or any other width or resolution. For example, where the aerial vehicle is outfitted with a LIDAR sensor having a planar field of view with a fixed angular width that may rotate at any angular velocity relative to one or more principal axes of the aerial vehicle, returns of reflected light may be downsampled or decimated into bins or other angular sectors having five-degree (5°) widths.
In some implementations, the sets of distance measurements for angular sectors may be determined or aggregated following a single pass, sweep or operation of a LIDAR sensor. Alternatively, in some other implementations, sets of distance measurements for angular sectors may be determined or aggregated following a predetermined number of passes, sweeps or operations of a LIDAR sensor, e.g., based on a common set of consecutive returns for each of such sectors.
At box 725, minimum distance measurements are determined for each of the angular sectors about the aerial vehicle. For example, where distance measurements are determined based on returns of reflected light captured by a LIDAR sensor, a minimum distance measurement within each of the respective bins or other angular sectors may be identified and stored in association with the respective bins or angular sectors, e.g., in one or more memory compartments of the aerial vehicle.
At box 730, whether an object is detected in a flight path of the aerial vehicle is determined. An aerial vehicle may detect one or more stationary or mobile objects in any manner, such as where a predetermined number of returns from a range sensor indicate that the object is within a predetermined range of the aerial vehicle and in a direction of the flight path of the aerial vehicle.
If an object is not detected in the flight path of the aerial vehicle, then the process advances to box 735, where whether the aerial vehicle has arrived at a destination is determined. If the aerial vehicle has arrived at the destination, then the process ends. If the aerial vehicle has not arrived at the destination, however, then the process returns to box 710, where the aerial vehicle continues to travel on a selected course, at a selected altitude and at a selected speed, and to box 715, where distance measurements around the aerial vehicle are determined based on data captured by the aerial vehicle in flight.
If the arial vehicle has detected an object in the flight path, however, then the process advances to box 740, where whether a set of angular sectors having minimum distance measurements in excess of a threshold has been identified is determined. For example, the minimum distance measurements determined at box 725 may be compared to one another to determine whether minimum distance measurements of any adjacent angular sectors, e.g., two or more of such sectors, exceed the threshold.
If any sets of angular sectors having minimum distance measurements in excess of the threshold are identified, then the process advances to box 745, where the aerial vehicle decelerates in a direction associated with a set of the angular sectors having minimum distance measurements beyond the threshold. For example, the aerial vehicle may execute a braking maneuver or otherwise decelerate in a direction associated with the set of the angular sectors having minimum distance measurements beyond the threshold, such as by identifying an intervening waypoint or another position along a bearing associated with the set and calculating a trajectory for traveling to or through the waypoint or the other position. Where two or more of such sets of angular sectors are identified, one of the sets may be identified or selected in any manner or on any basis. For example, a set of sectors having a greatest minimum distance, or a set defining a widest angle, may be identified and selected.
If no sets of angular sectors having minimum distance measurements in excess of the threshold are identified, however, then the process advances to box 750, where whether the threshold is at a minimum value is determined. For example, in some implementations, a minimum distance required for the aerial vehicle to brake or otherwise decelerate, or turn to a direction associated with a predetermined bearing, may be identified, and set as a minimum value of the threshold.
If the threshold is at the minimum value of the threshold, then the process advances to box 755, where the aerial vehicle decelerates in a direction associated opposite the flight path of the aerial vehicle. For example, upon failing to identify any sets of angular sectors having minimum distance measurements in excess of the threshold, the aerial vehicle may be configured to turn to or travel toward a default bearing or position, e.g., opposite the flight path of the aerial vehicle, such as by executing one or more braking maneuvers.
If the threshold is not at the minimum value of the threshold, however, then the process advances to box 760, where the aerial vehicle decreases (or decrements) the threshold, before returning to box 740, where whether a set of angular sectors having minimum distance measurements in excess of a threshold has been identified is determined. For example, in some implementations, upon determining that an object is present within a flight path of the aerial vehicle, the aerial vehicle may be programmed or configured to determine whether any sets of adjacent angular sectors having minimum distance measurements beyond a first threshold, e.g., a comparatively larger threshold, such as thirty centimeters (30 cm). In the event that no such sets are identified, the aerial vehicle may be configured to decrease or decrement the first threshold, e.g., by a predetermined amount, such as five centimeters (5 cm) or ten centimeters (10 cm), to a second threshold. Subsequently, whether any sets of adjacent angular sectors having minimum distance measurements beyond the second threshold may be determined and, in the event that no such sets are identified, the second threshold may be further decreased or decremented until the minimum value is reached.
In some implementations, reflections of light captured by a range sensor may be determined and clustered based on their respective ranges and angles from positions of the aerial vehicle. A position of an object may be determined based on the clusters, with respect to their positions according to any local or global coordinate system. Where distance measurements are determined based on returns captured by a LIDAR sensor having a field of view that rotates relative to the aerial vehicle, clusters of returns that are detected in one pass or sweep of the LIDAR sensor may be compared to clusters of returns that are detected in subsequent passes or sweeps of the LIDAR sensor. Where the clusters are substantially similar, the clusters may be identified as representing a common object. Conversely, where a single return or a cluster of fewer than a predetermined number of returns detected in one pass or sweep of the LIDAR sensor is not detected in a subsequent pass or sweep of the LIDAR sensor, the return or returns may be disregarded as corresponding to a “phantom” object.
In some implementations, a fast-clustering algorithm may be utilized to cluster data, such as LIDAR data, which may include one or more returns or distance measurements. For example, a sensor may be characterized as performing scans comprising a transmission of plurality of beams of light, e.g., approximately 430 beams, any of which may produce a return from one or more objects. A system may determine, for each beam or return, a distance measurement and an angle associated with the beam or return. In accordance with one or more preferred implementations, a clustering algorithm may determine, for each beam or return, a distance difference (or distance delta) representing an absolute value of a difference between a distance measurement for that beam or return and a distance measurement for the immediately preceding beam or return, and an angle difference (or angle delta) representing an absolute value of a difference between an angle associated with that beam or return and an angle associated with the immediately previous beam or return. The clustering algorithm compares the determined distance difference to a first distance threshold and compares the determined angle difference to a second angle threshold. If each of the distance difference and the angle difference is below (e.g., is less than or equal to) a corresponding threshold, the beam or return is assigned to the same cluster as the immediately previous beam or return. Otherwise, if either of the distance difference or the angle difference is not below (e.g., is greater than or equal to) a corresponding threshold, a new cluster is started, and the beam or return is assigned to the new cluster.
In accordance with one or more preferred implementations, an algorithm is utilized to determine that a cluster for a current scan corresponds to a cluster from a previous scan (e.g., to track or monitor an object that the cluster represents). In accordance with one or more preferred implementations, an algorithm operates, for one or more clusters in a scan (e.g., in a brute force approach, on each cluster in a scan), to compare the one or more clusters to stored clusters from previous scans (e.g., in a brute force approach, on each stored cluster from one or more previous scans).
In accordance with one or more preferred implementations, for each cluster, a center distance is determined representing a distance from an aerial vehicle or a sensor (e.g., a LIDAR sensors) to a center of a cluster. In accordance with one or more preferred implementations, for each cluster, a center angle is determined representing an angle from an aerial vehicle or sensor to a center of a cluster. In accordance with one or more preferred implementations, an algorithm compares a respective current cluster in a scan and a respective previous cluster of a previous scan by determining a center distance difference (or center distance delta) representing an absolute value of a difference between a center distance for the respective current cluster and a center distance for the respective previous cluster, and a center angle difference (or center angle delta) representing an absolute value of a difference between a center angle for the respective current cluster and a center angle for the respective previous cluster. The center distance difference is compared to a center distance threshold and the center angle difference is compared to a center angle threshold. If each of a center distance difference and a center angle difference is below (e.g., is less than or equal to) a corresponding threshold, a respective current cluster is determined to be the “same” cluster as a respective previous cluster, and one or more records may be updated to indicate that the respective current cluster is the respective previous cluster.
In accordance with one or more preferred implementations, determining whether a cluster for a current scan corresponds to a cluster from a previous scan may also consider a comparison of shapes of each of the clusters.
In accordance with one or more preferred implementations, a distance variance value for each cluster is determined (e.g., as a greatest difference, a total difference, or an average distance between two or more distance measurements in a cluster). In accordance with one or more preferred implementations, an angular sweep or field of view for each cluster is determined (e.g., a difference between a maximum angle for a distance measurement in the cluster and a minimum angle for a distance measurement in the cluster). In accordance with one or more preferred implementations, a scale value for a respective current cluster in a scan as compared to a respective previous cluster of a previous scan is determined by dividing a center distance for the respective current cluster by a center distance for the respective previous cluster.
In accordance with one or more preferred implementations, a shape comparison algorithm compares a respective current cluster in a scan and a respective previous cluster of a previous scan by determining a distance variance difference (or distance variance delta) representing an absolute value of a difference between a distance variance value for the respective current cluster and a distance variance value for the respective previous cluster, and a field of view difference (or field of view delta) representing an absolute value of a difference between an angular sweep or field of view for the respective current cluster multiplied by a determined scale value for the respective current cluster as compared to the respective previous cluster, and an angular sweep or field of view for the respective previous cluster. If each of these difference values is below (e.g., is less than or equal to) a corresponding threshold, the respective current cluster and the respective previous cluster are determined to have the same shape.
In accordance with one or more preferred implementations, an algorithm compares a respective current cluster in a scan and a respective previous cluster of a previous scan by determining a center distance difference (or a center distance delta) representing an absolute value of a difference between a center distance for the respective current cluster and a center distance for the respective previous cluster, a center angle difference (or a center angle delta) representing an absolute value of a difference between a center angle for the respective current cluster and a center angle for the respective previous cluster, and determines whether the respective current cluster and the respective previous cluster have the same shape. The center distance difference is compared to a center distance threshold and the center angle difference is compared to a center angle threshold. If each of these difference values is below (e.g., is than or equal to) a corresponding threshold and the clusters are determined to have the same shape, a respective current cluster is determined to be the “same” cluster as a respective previous cluster, and one or more records may be updated to indicate that the respective current cluster is the respective previous cluster.
In accordance with one or more preferred implementations of a clustering algorithm, one or more data structures are utilized to reduce time complexity, e.g., using a K-dimensional tree approach.
The contents of U.S. patent application Ser. Nos. 16/584,721 and 17/029,688, and International Patent Application No. PCT/US2020/052268, are incorporated by reference herein in their entireties.
Although some embodiments of the present disclosure show the use of unmanned aerial vehicles in support of one or more applications or functions at a facility such as a home or a like structure, those of ordinary skill in the pertinent arts will realize that the systems and methods of the present disclosure are not so limited. Rather, the systems and methods of the present disclosure may be utilized in connection with any facility, including but not limited to homes, in support of any type of application or function. Likewise, the systems and methods of the present disclosure may be utilized in connection with any aerial vehicle, including aerial vehicles that are manned or unmanned during operations.
It should be understood that, unless otherwise explicitly or implicitly indicated herein, any of the features, characteristics, alternatives or modifications described regarding a particular embodiment herein may also be applied, used, or incorporated with any other embodiment described herein, and that the drawings and detailed description of the present disclosure are intended to cover all modifications, equivalents and alternatives to the various embodiments as defined by the appended claims. Moreover, with respect to the one or more methods or processes of the present disclosure described herein, including but not limited to the flow chart shown in
Conditional language, such as, among others, “can,” “could,” “might,” or “may,” unless specifically stated otherwise, or otherwise understood within the context as used, is generally intended to convey in a permissive manner that certain embodiments could include, or have the potential to include, but do not mandate or require, certain features, elements and/or steps. In a similar manner, terms such as “include,” “including” and “includes” are generally intended to mean “including, but not limited to.” Thus, such conditional language is not generally intended to imply that features, elements and/or steps are in any way required for one or more embodiments or that one or more embodiments necessarily include logic for deciding, with or without user input or prompting, whether these features, elements and/or steps are included or are to be performed in any particular embodiment.
The elements of a method, process, or algorithm described in connection with the embodiments disclosed herein can be embodied directly in hardware, in a software module stored in one or more memory devices and executed by one or more processors, or in a combination of the two. A software module can reside in RAM, flash memory, ROM, EPROM, EEPROM, registers, a hard disk, a removable disk, a CD-ROM, a DVD-ROM or any other form of non-transitory computer-readable storage medium, media, or physical computer storage known in the art. An example storage medium can be coupled to the processor such that the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium can be integral to the processor. The storage medium can be volatile or nonvolatile. The processor and the storage medium can reside in an ASIC. The ASIC can reside in a user terminal. In the alternative, the processor and the storage medium can reside as discrete components in a user terminal.
Disjunctive language such as the phrase “at least one of X, Y, or Z,” or “at least one of X, Y and Z,” unless specifically stated otherwise, is otherwise understood with the context as used in general to present that an item, term, etc., may be either X, Y, or Z, or any combination thereof (e.g., X, Y, and/or Z). Thus, such disjunctive language is not generally intended to, and should not, imply that certain embodiments require at least one of X, at least one of Y, or at least one of Z to each be present.
Unless otherwise explicitly stated, articles such as “a” or “an” should generally be interpreted to include one or more described items. Accordingly, phrases such as “a device configured to” are intended to include one or more recited devices. Such one or more recited devices can also be collectively configured to carry out the stated recitations. For example, “a processor configured to carry out recitations A, B and C” can include a first processor configured to carry out recitation A working in conjunction with a second processor configured to carry out recitations B and C.
Language of degree used herein, such as the terms “about,” “approximately,” “generally,” “nearly” or “substantially” as used herein, represent a value, amount, or characteristic close to the stated value, amount, or characteristic that still performs a desired function or achieves a desired result. For example, the terms “about,” “approximately,” “generally,” “nearly” or “substantially” may refer to an amount that is within less than 10% of, within less than 5% of, within less than 1% of, within less than 0.1% of, and within less than 0.01% of the stated amount.
Although the invention has been described and illustrated with respect to illustrative embodiments thereof, the foregoing and various other additions and omissions may be made therein and thereto without departing from the spirit and scope of the present disclosure.
| Number | Name | Date | Kind |
|---|---|---|---|
| 3417729 | Gilday | Dec 1968 | A |
| 3575527 | Watanabe | Apr 1971 | A |
| 3805723 | Bernaerts | Apr 1974 | A |
| 4865248 | Barth | Sep 1989 | A |
| 4954962 | Evans, Jr. | Sep 1990 | A |
| 5040116 | Evans, Jr. | Aug 1991 | A |
| 5283739 | Summerville | Feb 1994 | A |
| 5371581 | Wangler | Dec 1994 | A |
| 5386462 | Schlamp | Jan 1995 | A |
| 5452374 | Cullen | Sep 1995 | A |
| 5497236 | Wolff | Mar 1996 | A |
| 5731884 | Inoue | Mar 1998 | A |
| 5847522 | Barba | Dec 1998 | A |
| 5901253 | Tretter | May 1999 | A |
| 5995898 | Tuttle | Nov 1999 | A |
| 6031612 | Shirley | Feb 2000 | A |
| 6266577 | Popp | Jul 2001 | B1 |
| 6344796 | Ogilvie | Feb 2002 | B1 |
| 6374155 | Wallach | Apr 2002 | B1 |
| 6426699 | Porter | Jul 2002 | B1 |
| 6507670 | Moed | Jan 2003 | B1 |
| 6543983 | Felder | Apr 2003 | B1 |
| 6636781 | Shen | Oct 2003 | B1 |
| 6676460 | Motsenbocker | Jan 2004 | B1 |
| 6690997 | Rivalto | Feb 2004 | B2 |
| 6694217 | Bloom | Feb 2004 | B2 |
| 6705523 | Stamm | Mar 2004 | B1 |
| 6804607 | Wood | Oct 2004 | B1 |
| 6919803 | Breed | Jul 2005 | B2 |
| 6954290 | Braudaway | Oct 2005 | B1 |
| 6961711 | Chee | Nov 2005 | B1 |
| 6965440 | Nakagiri | Nov 2005 | B1 |
| 6970838 | Kamath | Nov 2005 | B1 |
| 7006952 | Matsumoto | Feb 2006 | B1 |
| 7016536 | Ling | Mar 2006 | B1 |
| 7031519 | Elmenhurst | Apr 2006 | B2 |
| 7129817 | Yamagishi | Oct 2006 | B2 |
| 7133743 | Tilles | Nov 2006 | B2 |
| 7145699 | Dolan | Dec 2006 | B2 |
| 7149611 | Beck | Dec 2006 | B2 |
| 7171879 | Gass | Feb 2007 | B2 |
| 7188513 | Wilson | Mar 2007 | B2 |
| 7335071 | Motsenbocker | Feb 2008 | B1 |
| 7337686 | Sagi-Dolev | Mar 2008 | B2 |
| 7337944 | Devar | Mar 2008 | B2 |
| 7339993 | Brooks | Mar 2008 | B1 |
| 7459880 | Rosen | Dec 2008 | B1 |
| 7639386 | Siegel | Dec 2009 | B1 |
| 7668404 | Adams | Feb 2010 | B2 |
| 7673831 | Steele | Mar 2010 | B2 |
| 7685953 | Giles | Mar 2010 | B2 |
| 7693745 | Pomerantz | Apr 2010 | B1 |
| 7894939 | Zini | Feb 2011 | B2 |
| 7925375 | Schininger | Apr 2011 | B2 |
| 7946526 | Zimet | May 2011 | B2 |
| 7946530 | Talmage, Jr. | May 2011 | B1 |
| 7966093 | Zhuk | Jun 2011 | B2 |
| 8015023 | Lee | Sep 2011 | B1 |
| 8078317 | Allinson | Dec 2011 | B2 |
| 8126642 | Trepagnier | Feb 2012 | B2 |
| 8131607 | Park | Mar 2012 | B2 |
| 8145351 | Schininger | Mar 2012 | B2 |
| 8195328 | Mallett | Jun 2012 | B2 |
| 8286236 | Jung | Oct 2012 | B2 |
| 8412588 | Bodell | Apr 2013 | B1 |
| 8418959 | Kang | Apr 2013 | B2 |
| 8429754 | Jung | Apr 2013 | B2 |
| 8473189 | Christoph | Jun 2013 | B2 |
| 8511606 | Lutke | Aug 2013 | B1 |
| 8577538 | Lenser et al. | Nov 2013 | B2 |
| 8599027 | Sanchez | Dec 2013 | B2 |
| 8602349 | Petrov | Dec 2013 | B2 |
| 8639400 | Wong | Jan 2014 | B1 |
| 8736820 | Choe | May 2014 | B2 |
| 8752166 | Jung | Jun 2014 | B2 |
| 8791790 | Robertson | Jul 2014 | B2 |
| 8874301 | Rao | Oct 2014 | B1 |
| 8899903 | Saad | Dec 2014 | B1 |
| 8948914 | Zini | Feb 2015 | B2 |
| 8956100 | Davi | Feb 2015 | B2 |
| 8989053 | Skaaksrud | Mar 2015 | B1 |
| 9033285 | Iden | May 2015 | B2 |
| 9051043 | Peeters | Jun 2015 | B1 |
| 9079587 | Rupp | Jul 2015 | B1 |
| 9139310 | Wang | Sep 2015 | B1 |
| 9163909 | Chengalva | Oct 2015 | B2 |
| 9193452 | Carreker | Nov 2015 | B2 |
| 9195959 | Lopez | Nov 2015 | B1 |
| 9216587 | Ando | Dec 2015 | B2 |
| 9216857 | Kalyan | Dec 2015 | B1 |
| 9230236 | Villamar | Jan 2016 | B2 |
| 9235213 | Villamar | Jan 2016 | B2 |
| 9244147 | Soundararajan | Jan 2016 | B1 |
| 9256852 | Myllymaki | Feb 2016 | B1 |
| 9261578 | Im | Feb 2016 | B2 |
| 9321531 | Takayama | Apr 2016 | B1 |
| 9336506 | Shucker et al. | May 2016 | B2 |
| 9336635 | Robertson | May 2016 | B2 |
| 9358975 | Watts | Jun 2016 | B1 |
| 9373149 | Abhyanker | Jun 2016 | B2 |
| 9381916 | Zhu | Jul 2016 | B1 |
| 9397518 | Theobald | Jul 2016 | B1 |
| 9404761 | Meuleau | Aug 2016 | B2 |
| 9409644 | Stanek | Aug 2016 | B2 |
| 9411337 | Theobald | Aug 2016 | B1 |
| 9412280 | Zwillinger | Aug 2016 | B1 |
| 9436183 | Thakur | Sep 2016 | B2 |
| 9436926 | Cousins | Sep 2016 | B2 |
| 9448559 | Kojo | Sep 2016 | B2 |
| 9489490 | Theobald | Nov 2016 | B1 |
| 9510316 | Skaaksrud | Nov 2016 | B2 |
| 9535421 | Canoso | Jan 2017 | B1 |
| 9545852 | Streett | Jan 2017 | B2 |
| 9561941 | Watts | Feb 2017 | B1 |
| 9568335 | Thakur | Feb 2017 | B2 |
| 9582950 | Shimizu | Feb 2017 | B2 |
| 9600645 | Fadell | Mar 2017 | B2 |
| 9619776 | Ford | Apr 2017 | B1 |
| 9623553 | Theobald et al. | Apr 2017 | B1 |
| 9623562 | Watts | Apr 2017 | B1 |
| 9650136 | Haskin | May 2017 | B1 |
| 9652912 | Fadell | May 2017 | B2 |
| 9656805 | Evans | May 2017 | B1 |
| 9671791 | Paczan | Jun 2017 | B1 |
| 9677564 | Woodworth | Jun 2017 | B1 |
| 9682481 | Lutz et al. | Jun 2017 | B2 |
| 9697730 | Thakur | Jul 2017 | B2 |
| 9718564 | Beckman | Aug 2017 | B1 |
| 9720414 | Theobald | Aug 2017 | B1 |
| 9731821 | Hoareau | Aug 2017 | B2 |
| 9733646 | Nusser | Aug 2017 | B1 |
| 9746852 | Watts | Aug 2017 | B1 |
| 9746853 | Scheepjens | Aug 2017 | B2 |
| 9778653 | McClintock | Oct 2017 | B1 |
| 9786187 | Bar-Zeev | Oct 2017 | B1 |
| 9796529 | Hoareau | Oct 2017 | B1 |
| 9828092 | Navot | Nov 2017 | B1 |
| 9858604 | Apsley | Jan 2018 | B2 |
| 9886035 | Watts | Feb 2018 | B1 |
| 9896204 | Willison | Feb 2018 | B1 |
| 9959771 | Carlson | May 2018 | B1 |
| 9959773 | Raptopoulos et al. | May 2018 | B2 |
| 9974612 | Pinter | May 2018 | B2 |
| 10022753 | Chelian | Jul 2018 | B2 |
| 10022867 | Saboo et al. | Jul 2018 | B2 |
| 10048697 | Theobald | Aug 2018 | B1 |
| 10108185 | Theobald | Oct 2018 | B1 |
| 10137984 | Flick | Nov 2018 | B1 |
| 10558226 | Bigdeli | Feb 2020 | B1 |
| 10745132 | Kimchi | Aug 2020 | B1 |
| 10780988 | Buchmueller | Sep 2020 | B2 |
| 10860115 | Tran | Dec 2020 | B1 |
| 11164149 | Williams | Nov 2021 | B1 |
| 11619952 | Kwon | Apr 2023 | B2 |
| 11628932 | Seung | Apr 2023 | B2 |
| 11649050 | Miller | May 2023 | B1 |
| 11726184 | Ferreira | Aug 2023 | B2 |
| D1008873 | Lin | Dec 2023 | S |
| 11851162 | Daube et al. | Dec 2023 | B1 |
| 11868146 | Yasunaga | Jan 2024 | B2 |
| 11933613 | Michini | Mar 2024 | B2 |
| 20010045449 | Shannon | Nov 2001 | A1 |
| 20020016726 | Ross | Feb 2002 | A1 |
| 20020035450 | Thackston | Mar 2002 | A1 |
| 20020072979 | Sinha | Jun 2002 | A1 |
| 20020087375 | Griffin | Jul 2002 | A1 |
| 20020107751 | Rajagopalan | Aug 2002 | A1 |
| 20020111914 | Terada | Aug 2002 | A1 |
| 20020116289 | Yang | Aug 2002 | A1 |
| 20020123930 | Boyd | Sep 2002 | A1 |
| 20020156645 | Hansen | Oct 2002 | A1 |
| 20030040980 | Nakajima | Feb 2003 | A1 |
| 20030072031 | Kuwata | Apr 2003 | A1 |
| 20030121968 | Miller | Jul 2003 | A1 |
| 20030141411 | Pandya | Jul 2003 | A1 |
| 20040002898 | Kuhlmann | Jan 2004 | A1 |
| 20040068416 | Solomon | Apr 2004 | A1 |
| 20040112660 | Johansson | Jun 2004 | A1 |
| 20040160335 | Reitmeier | Aug 2004 | A1 |
| 20040162638 | Solomon | Aug 2004 | A1 |
| 20040257199 | Fitzgibbon | Dec 2004 | A1 |
| 20050068178 | Lee | Mar 2005 | A1 |
| 20050093865 | Jia | May 2005 | A1 |
| 20050102240 | Misra | May 2005 | A1 |
| 20050244060 | Nagarajan | Nov 2005 | A1 |
| 20050285934 | Carter | Dec 2005 | A1 |
| 20060053534 | Mullen | Mar 2006 | A1 |
| 20060118162 | Saelzer | Jun 2006 | A1 |
| 20060136237 | Spiegel | Jun 2006 | A1 |
| 20060287829 | Pashko-Paschenko | Dec 2006 | A1 |
| 20070016496 | Bar | Jan 2007 | A1 |
| 20070073552 | Hileman | Mar 2007 | A1 |
| 20070102565 | Speer | May 2007 | A1 |
| 20070150375 | Yang | Jun 2007 | A1 |
| 20070170237 | Neff | Jul 2007 | A1 |
| 20070210953 | Abraham | Sep 2007 | A1 |
| 20070233337 | Plishner | Oct 2007 | A1 |
| 20070244763 | Williams | Oct 2007 | A1 |
| 20070246601 | Layton | Oct 2007 | A1 |
| 20070262195 | Bulaga | Nov 2007 | A1 |
| 20070293978 | Wurman | Dec 2007 | A1 |
| 20080012697 | Smith | Jan 2008 | A1 |
| 20080027591 | Lenser | Jan 2008 | A1 |
| 20080100258 | Ward | May 2008 | A1 |
| 20080109246 | Russell | May 2008 | A1 |
| 20080111816 | Abraham | May 2008 | A1 |
| 20080141921 | Hinderks | Jun 2008 | A1 |
| 20080150679 | Bloomfield | Jun 2008 | A1 |
| 20080154659 | Bettes | Jun 2008 | A1 |
| 20080167817 | Hessler | Jul 2008 | A1 |
| 20080184906 | Kejha | Aug 2008 | A1 |
| 20080189012 | Kaufmann | Aug 2008 | A1 |
| 20080301009 | Plaster | Dec 2008 | A1 |
| 20090027253 | van Tooren | Jan 2009 | A1 |
| 20090062974 | Tamamoto | Mar 2009 | A1 |
| 20090063166 | Palmer | Mar 2009 | A1 |
| 20090079388 | Reddy | Mar 2009 | A1 |
| 20090086275 | Liang | Apr 2009 | A1 |
| 20090091435 | Bolourchi | Apr 2009 | A1 |
| 20090106124 | Yang | Apr 2009 | A1 |
| 20090149985 | Chirnomas | Jun 2009 | A1 |
| 20090164379 | Jung | Jun 2009 | A1 |
| 20090165127 | Jung | Jun 2009 | A1 |
| 20090236470 | Goossen | Sep 2009 | A1 |
| 20090254457 | Folsom | Oct 2009 | A1 |
| 20090254482 | Vadlamani | Oct 2009 | A1 |
| 20090299903 | Hung | Dec 2009 | A1 |
| 20090303507 | Abeloe | Dec 2009 | A1 |
| 20090314883 | Arlton | Dec 2009 | A1 |
| 20100007479 | Smith | Jan 2010 | A1 |
| 20100030608 | Kaminsky | Feb 2010 | A1 |
| 20100031351 | Jung | Feb 2010 | A1 |
| 20100088163 | Davidson | Apr 2010 | A1 |
| 20100088175 | Lundquist | Apr 2010 | A1 |
| 20100100269 | Ekhaguere | Apr 2010 | A1 |
| 20100169185 | Cottingham | Jul 2010 | A1 |
| 20100206145 | Tetelbaum | Aug 2010 | A1 |
| 20100287065 | Alivandi | Nov 2010 | A1 |
| 20100299067 | McCollough | Nov 2010 | A1 |
| 20100299222 | Hamilton, IV | Nov 2010 | A1 |
| 20110035149 | McAndrew | Feb 2011 | A1 |
| 20110074570 | Feldstein | Mar 2011 | A1 |
| 20110087350 | Fogel | Apr 2011 | A1 |
| 20110112761 | Hurley | May 2011 | A1 |
| 20110153052 | Pettibone | Jun 2011 | A1 |
| 20110166707 | Romanov | Jul 2011 | A1 |
| 20110178711 | Christoph | Jul 2011 | A1 |
| 20110210866 | David | Sep 2011 | A1 |
| 20110227435 | Maeda | Sep 2011 | A1 |
| 20110246331 | Luther | Oct 2011 | A1 |
| 20110253831 | Cheng | Oct 2011 | A1 |
| 20110264311 | Lee | Oct 2011 | A1 |
| 20110282476 | Hegemier | Nov 2011 | A1 |
| 20110301787 | Chaperon | Dec 2011 | A1 |
| 20110313878 | Norman | Dec 2011 | A1 |
| 20120039694 | Suzanne | Feb 2012 | A1 |
| 20120078592 | Sims, Jr. | Mar 2012 | A1 |
| 20120091260 | Callou | Apr 2012 | A1 |
| 20120109419 | Mercado | May 2012 | A1 |
| 20120219397 | Baker | Aug 2012 | A1 |
| 20120221438 | Cook, Jr. | Aug 2012 | A1 |
| 20120227389 | Hinderks | Sep 2012 | A1 |
| 20120229325 | Dutruc | Sep 2012 | A1 |
| 20120234969 | Savoye | Sep 2012 | A1 |
| 20120235606 | Takeuchi | Sep 2012 | A1 |
| 20120323365 | Taylor | Dec 2012 | A1 |
| 20130006739 | Horvitz | Jan 2013 | A1 |
| 20130073477 | Grinberg | Mar 2013 | A1 |
| 20130081245 | Vavrina | Apr 2013 | A1 |
| 20130093582 | Walsh | Apr 2013 | A1 |
| 20130126611 | Kangas | May 2013 | A1 |
| 20130148123 | Hayashi | Jun 2013 | A1 |
| 20130206915 | Desaulniers | Aug 2013 | A1 |
| 20130218446 | Bradley | Aug 2013 | A1 |
| 20130218799 | Lehmann | Aug 2013 | A1 |
| 20130261792 | Gupta | Oct 2013 | A1 |
| 20130262251 | Wan et al. | Oct 2013 | A1 |
| 20130262252 | Lakshman | Oct 2013 | A1 |
| 20130262276 | Wan | Oct 2013 | A1 |
| 20130262336 | Wan | Oct 2013 | A1 |
| 20130264381 | Kim | Oct 2013 | A1 |
| 20130320133 | Ratti | Dec 2013 | A1 |
| 20130324164 | Vulcano | Dec 2013 | A1 |
| 20130332062 | Kreitmair-Steck | Dec 2013 | A1 |
| 20140010656 | Nies | Jan 2014 | A1 |
| 20140022055 | Levien et al. | Jan 2014 | A1 |
| 20140025230 | Levien | Jan 2014 | A1 |
| 20140030444 | Swaminathan | Jan 2014 | A1 |
| 20140031964 | Sidhu | Jan 2014 | A1 |
| 20140032034 | Raptopoulos | Jan 2014 | A1 |
| 20140040065 | DuBois | Feb 2014 | A1 |
| 20140052661 | Shakes | Feb 2014 | A1 |
| 20140058959 | Isbjornssund | Feb 2014 | A1 |
| 20140081445 | Villamar | Mar 2014 | A1 |
| 20140089073 | Jacobs | Mar 2014 | A1 |
| 20140136282 | Fedele | May 2014 | A1 |
| 20140136414 | Abhyanker | May 2014 | A1 |
| 20140149244 | Abhyanker | May 2014 | A1 |
| 20140156053 | Mahdavi | Jun 2014 | A1 |
| 20140180914 | Abhyanker | Jun 2014 | A1 |
| 20140200697 | Cheng | Jul 2014 | A1 |
| 20140214684 | Pell | Jul 2014 | A1 |
| 20140244433 | Cruz | Aug 2014 | A1 |
| 20140254896 | Zhou | Sep 2014 | A1 |
| 20140271200 | Sutton | Sep 2014 | A1 |
| 20140283104 | Nilsson | Sep 2014 | A1 |
| 20140309813 | Ricci | Oct 2014 | A1 |
| 20140325218 | Shimizu | Oct 2014 | A1 |
| 20140330456 | Lopez Morales | Nov 2014 | A1 |
| 20150006005 | Yu | Jan 2015 | A1 |
| 20150066178 | Stava | Mar 2015 | A1 |
| 20150069968 | Pounds | Mar 2015 | A1 |
| 20150098819 | Tourin | Apr 2015 | A1 |
| 20150102154 | Duncan | Apr 2015 | A1 |
| 20150112837 | O'Dea | Apr 2015 | A1 |
| 20150112885 | Fadell | Apr 2015 | A1 |
| 20150120094 | Kimchi | Apr 2015 | A1 |
| 20150120602 | Huffman | Apr 2015 | A1 |
| 20150127712 | Fadell | May 2015 | A1 |
| 20150129716 | Yoffe | May 2015 | A1 |
| 20150153175 | Skaaksrud | Jun 2015 | A1 |
| 20150154545 | Skaaksrud et al. | Jun 2015 | A1 |
| 20150158599 | Sisko | Jun 2015 | A1 |
| 20150175276 | Koster | Jun 2015 | A1 |
| 20150183528 | Walsh | Jul 2015 | A1 |
| 20150185034 | Abhyanker | Jul 2015 | A1 |
| 20150191255 | Zolich | Jul 2015 | A1 |
| 20150202770 | Patron | Jul 2015 | A1 |
| 20150227882 | Bhatt | Aug 2015 | A1 |
| 20150246727 | Masticola | Sep 2015 | A1 |
| 20150253777 | Binney | Sep 2015 | A1 |
| 20150254611 | Perez | Sep 2015 | A1 |
| 20150259078 | Filipovic | Sep 2015 | A1 |
| 20150286216 | Miwa | Oct 2015 | A1 |
| 20150314881 | Tsaliah | Nov 2015 | A1 |
| 20150317597 | Shucker | Nov 2015 | A1 |
| 20150332206 | Trew | Nov 2015 | A1 |
| 20150367850 | Clarke | Dec 2015 | A1 |
| 20150370251 | Siegel | Dec 2015 | A1 |
| 20160001877 | Paulos | Jan 2016 | A1 |
| 20160009413 | Lee | Jan 2016 | A1 |
| 20160019495 | Kolchin | Jan 2016 | A1 |
| 20160033966 | Farris | Feb 2016 | A1 |
| 20160051110 | Cao | Feb 2016 | A1 |
| 20160058181 | Han | Mar 2016 | A1 |
| 20160068267 | Liu | Mar 2016 | A1 |
| 20160070265 | Liu | Mar 2016 | A1 |
| 20160085238 | Hayes | Mar 2016 | A1 |
| 20160104099 | Villamar | Apr 2016 | A1 |
| 20160104113 | Gorlin | Apr 2016 | A1 |
| 20160107750 | Yates | Apr 2016 | A1 |
| 20160114488 | Mascorro Medina | Apr 2016 | A1 |
| 20160117931 | Chan | Apr 2016 | A1 |
| 20160125746 | Kunzi | May 2016 | A1 |
| 20160129592 | Saboo | May 2016 | A1 |
| 20160130015 | Caubel | May 2016 | A1 |
| 20160132059 | Mason | May 2016 | A1 |
| 20160144734 | Wang | May 2016 | A1 |
| 20160144982 | Sugumaran | May 2016 | A1 |
| 20160180618 | Ho | Jun 2016 | A1 |
| 20160196755 | Navot | Jul 2016 | A1 |
| 20160196756 | Prakash | Jul 2016 | A1 |
| 20160200438 | Bokeno | Jul 2016 | A1 |
| 20160207627 | Hoareau | Jul 2016 | A1 |
| 20160214717 | De Silva | Jul 2016 | A1 |
| 20160214728 | Rossi | Jul 2016 | A1 |
| 20160221671 | Fisher et al. | Aug 2016 | A1 |
| 20160235236 | Byers | Aug 2016 | A1 |
| 20160236778 | Takayama et al. | Aug 2016 | A1 |
| 20160239789 | Hanks | Aug 2016 | A1 |
| 20160239803 | Borley | Aug 2016 | A1 |
| 20160257401 | Buchmueller | Sep 2016 | A1 |
| 20160258775 | Santilli | Sep 2016 | A1 |
| 20160266578 | Douglas | Sep 2016 | A1 |
| 20160282126 | Watts | Sep 2016 | A1 |
| 20160299233 | Levien | Oct 2016 | A1 |
| 20160321503 | Zhou | Nov 2016 | A1 |
| 20160334229 | Ross | Nov 2016 | A1 |
| 20160340021 | Zhang | Nov 2016 | A1 |
| 20160364660 | Brown | Dec 2016 | A1 |
| 20160364679 | Cao | Dec 2016 | A1 |
| 20160364823 | Cao | Dec 2016 | A1 |
| 20160364989 | Speasl | Dec 2016 | A1 |
| 20160371984 | Macfarlane | Dec 2016 | A1 |
| 20170011333 | Greiner | Jan 2017 | A1 |
| 20170011340 | Gabbai | Jan 2017 | A1 |
| 20170032315 | Gupta | Feb 2017 | A1 |
| 20170087999 | Miller | Mar 2017 | A1 |
| 20170096222 | Spinelli | Apr 2017 | A1 |
| 20170098378 | Soundararajan et al. | Apr 2017 | A1 |
| 20170100837 | Zevenbergen | Apr 2017 | A1 |
| 20170101017 | Streett | Apr 2017 | A1 |
| 20170113352 | Lutz | Apr 2017 | A1 |
| 20170147975 | Natarajan | May 2017 | A1 |
| 20170152060 | Morisawa | Jun 2017 | A1 |
| 20170154347 | Bateman | Jun 2017 | A1 |
| 20170164319 | Skaaksrud et al. | Jun 2017 | A1 |
| 20170167881 | Rander | Jun 2017 | A1 |
| 20170193442 | Ekkel | Jul 2017 | A1 |
| 20170199522 | Li | Jul 2017 | A1 |
| 20170255896 | Van Dyke | Sep 2017 | A1 |
| 20170286905 | Richardson | Oct 2017 | A1 |
| 20170300855 | Lund | Oct 2017 | A1 |
| 20170305526 | Thomassey | Oct 2017 | A1 |
| 20170308098 | Yu et al. | Oct 2017 | A1 |
| 20170316379 | Lepek | Nov 2017 | A1 |
| 20170330145 | Studnicka | Nov 2017 | A1 |
| 20170345245 | Torresani | Nov 2017 | A1 |
| 20170372256 | Kantor | Dec 2017 | A1 |
| 20180024554 | Brady | Jan 2018 | A1 |
| 20180088586 | Hance | Mar 2018 | A1 |
| 20180127211 | Jarvis | May 2018 | A1 |
| 20180137454 | Kulkarni | May 2018 | A1 |
| 20180203464 | Yu et al. | Jul 2018 | A1 |
| 20180257775 | Baek et al. | Sep 2018 | A1 |
| 20190135403 | Perry | May 2019 | A1 |
| 20190161190 | Gil | May 2019 | A1 |
| 20200272144 | Yang et al. | Aug 2020 | A1 |
| 20200284883 | Ferreira | Sep 2020 | A1 |
| 20200324898 | Youmans | Oct 2020 | A1 |
| 20200369384 | Kelly | Nov 2020 | A1 |
| 20210089055 | Tran | Mar 2021 | A1 |
| 20210094686 | Metzner | Apr 2021 | A1 |
| 20220009647 | Johannesson | Jan 2022 | A1 |
| 20220144424 | Metzner et al. | May 2022 | A1 |
| 20220162001 | Gherardi | May 2022 | A1 |
| 20220163980 | Beer | May 2022 | A1 |
| 20230298268 | Oleynikova | Sep 2023 | A1 |
| Number | Date | Country |
|---|---|---|
| 101385059 | Mar 2009 | CN |
| 103224026 | Jul 2013 | CN |
| 203039518 | Jul 2013 | CN |
| 103365297 | Oct 2013 | CN |
| 104760704 | Jul 2015 | CN |
| 109118826 | Jan 2019 | CN |
| 102011086497 | May 2013 | DE |
| 3415436 | Dec 2018 | EP |
| 2692064 | Dec 1993 | FR |
| 2455374 | Jun 2009 | GB |
| S48088255 | Oct 1973 | JP |
| S56048952 | Nov 1981 | JP |
| H0712088 | Jan 1995 | JP |
| 2004126800 | Apr 2004 | JP |
| 2009297449 | Dec 2009 | JP |
| 2010095246 | Apr 2010 | JP |
| 2011211025 | Oct 2011 | JP |
| 2012037204 | Feb 2012 | JP |
| 2007052246 | May 2007 | WO |
| 2008147484 | Dec 2008 | WO |
| 2008147484 | Feb 2009 | WO |
| 2009153588 | Dec 2009 | WO |
| 2010070717 | Jun 2010 | WO |
| 2012012752 | Jan 2012 | WO |
| 2012012752 | Jun 2012 | WO |
| 2013140085 | Sep 2013 | WO |
| 2013148123 | Oct 2013 | WO |
| 2014064431 | May 2014 | WO |
| 2014068982 | May 2014 | WO |
| 2014080409 | May 2014 | WO |
| 2014064431 | Jul 2014 | WO |
| 2017064202 | Apr 2017 | WO |
| 2018156991 | Aug 2018 | WO |
| Entry |
|---|
| Bullock et al., “Analysis of the Use of Digital Road Maps in Vehicle Navigation,” 1994, IEEE, p. 494-501 (Year: 1994). |
| DHL Trend Research, “Self-Driving Vehicles in Logistics,” Dec. 2014, Markus Kückelhaus et al. (downloaded from http://www.dhl.com/content/dam/downloads/g0/about_us/logistics_insights/dhl_self_driving_vehicles.pdf with an archived Web version available on https://web.archive.org/web/20151018154844/http://www.dhl.com/content/dam/downloads/g0/about_us/logistics_insights/dhl_self_driving_vehicles.pdf), 39 pages. |
| DHL Trend Research, “Unmanned Aerial Vehicles in Logistics: A DHL perspective on implications and use cases for the logistics industry,” 2014, Markus Kückelhaus et al., URL: http://www.dhl.com/content/dam/downloads/g0/about_us/logistics_insights/dhl_trend_report_uav.pdf with a Web Archive version available at: https://web.archive.org/web/20150923080141/http://www.dhl.com/en/about_us/logistics_insights/dhl_trend_research/uav.html, 24 pages. |
| Hawas et al., “Infrastructureless Inter-Vehicular Real-Time Route Guidance,” 2008, IEEE, p. 1213-1219 (Year: 2008). |
| Kais, Mikael et al., “An Intelligent architecture for automated transportation in our cities”, 2001 European Control Conference (ECC), Porto, Portugal, Sep. 4-7, 2001, pp. 277-282 (Year: 2001). |
| Kladis et al., “Event-Based Energy Optimum Route Planning in the Context of Unmanned Aerial Vehicles for Multi-Objective Exploration Missions,” 2009, IEEE, p. 1281-1286 (Year: 2009). |
| Marcus Wohlsen, “The Next Big Thing You Missed: Amazon's Delivery Drones Could Work—They Just Need Trucks,” Wired: Business, Jun. 10, 2014, URL: https://www.wired.com/2014/06/the-next-big-thing-you-missed-delivery-drones-launched-from-trucks-are-the-future-of-shipping/, 4 pages. |
| Mike Murphy, “Google wants to deliver packages from self-driving trucks,” published Feb. 9, 2016, URL: https://qz.com/613277/google-wants-to-deliver-packages-from-self-driving-trucks/, 4 pages. |
| Nanohawk Aeraccess: “Unmanned Aerial Vehicle—Aeraccess—Nanohawk—Webarchive Jun. 16, 2019”, Jun. 16, 2019 (Jun. 16, 2019), pp. 1-3, XP055759013, URL: https://web.archive.org/web/20190616235755/http://www.aeraccess-group.com/product/indoor/nanohawk [Retrieved from the Internet: Dec. 10, 2020]; the whole document. |
| Navaravong et al., “Formation Reconfiguration for Mobile Robots with Network Connectivity Constraints,” 2012, IEEE, p. 18-24 (Year: 2012). |
| Parent, Michel et al., “Intelligent Transportation in Cities with CTS”, The IEEE 5th International Conference on Intelligent Transportation Systems, Sep. 3-6, 2002, Singapore, pp. 826-830 (Year: 2002). |
| Sandoval, “Google patents secure rolling box to receive packages from drones,” Geekwire.com, Jan. 27, 2016, URL: http://www.geekwire.com/2016/google-pondering-drone-delivery-even-about-boxes-it-flies-to-front-doors/, 11 pages. |
| Smith, Randall C., and Peter Cheeseman, “On the Representation and Estimation of Spatial Uncertainty,” The Int'l Journal of Robotics Research, vol. 5, No. 4 (Winter 1986), Copyright 1986 Massachusetts Institute of Technology, pages. |
| URL: https://web.archive.org/web/20160804001046/https://www.starship.xyz/, download date: Aug. 4, 2016, 21 pages. |
| Wikipedia, “Ramer-Douglas-Peucker Algorithm,” www.wikipedia.com, Wikipedia: The Free Encyclopedia, 5 pages, Aug. 18, 2022, URL: https://en.wikipedia.org/wiki/Ramer%E2%80%93Douglas%E2%80%93Peucker_algorithm, obtained via web.archive.org on Jan. 6, 2023. |
| Andrew Amato, “Parrot Announces New Bobop Drone”, Dronelife.com, May 12, 2014; http://dronelife.com/2014/05/12/parrot- announces-new-bebop-drone/, accessed Dec. 21, 2016. |
| Ivan Barajas, “Parrot Unveils New Drone and Controller,” May 14, 2014; https://blog.newegg.com/parrot-unveils-drone-controller-oculus-compatible/, accessed Dec. 21, 2016. |
| Microchip Technology Inc. AT42QT1010 Data Sheet. Copyright © 2017 Microchip Technology Inc. 30 pages. |
| MicroPilot home page, https://www.micropilot.com/; archived on Jan. 3, 2013 by Internet Archive, http://web.archive.org/web/2013103180733/http://www.micropilot.com/?, accessed May 2, 2016. |
| Norman Chan, “Hands-On with Parrot's Bebop Drone Quadcopter”, tested.com, May 12, 2014; http://www.tested.com/tech/robots/461148-parrots-bebop-drone-what-you-should-know/, accessed Dec. 22, 2016. |
| rchelicopterfun.com, “Parrot AF Drone Quadcopter Review”, http://www.rchelicopterfun.com/parrot-ar-drone.html; archived on Mar. 27, 2012 by Internet Archive, http://web.archive.org/web/20120327140509/http://www.rchelicopterfun.com/parrot-ar-drone.html, accessed Apr. 27, 2016. |
| Wang, David. Application Report: FDC1004: Basics of Capacitive Sensing and Applications. SNOA927A—Dec. 2014—Revised Jun. 2021. Copyright © 2021 Texas Instruments Incorporated. 12 pages. |
| Wikipedia, “Apollo Abort Modes”, https://en.wikipedia.org/wiki/Apollo_abort_modes; archived by Internet Archive on Aug. 16, 2012, https://web.archive.org/web/20120816115115/https://en .wikipedia.org/wiki/Apollo_abort_modes; accessed Jul. 9, 2019 (Year: 2012). |
| Wikipedia, “Space Shuttle Abort Modes,” https://en.wikipedia.org/wiki/Space_Shuttle_abort_modes; archived by Internet Archive on Aug. 15, 2012, https://web.archive.org/web/20120815031221/https://en.wikipedia.org/wiki/Space_Shuttle_abort_modes; accessed Jul. 9, 2019 (Year: 2012). |
