This application is related by subject matter to U.S. patent application Ser. Nos. 13/257,378 and 13/257,369, filed Sep. 19, 2011, which published as 2012/0008542 and 2012/0011365, respectively, on Jan. 12, 2012. Other related patents include U.S. Pat. Nos. 7,119,676; 7,983,685; 8,244,260; and 8,385,322; and other related patent applications include International Patent Application Nos. PCT/US04/33051, which published as WO/2005/043930 on May 12, 2005; PCT/US08/71346, which published as WO/2009/018212 on Feb. 5, 2009; PCT/US09/37755, which published as WO/2010/107441 on Sep. 23, 2010; and PCT/US09/37753, which published as WO/2010/107440 on Sep. 23, 2010. Each of the above-identified patents, patent applications, and published applications is hereby incorporated by reference herein in its entirety.
Field of the Invention
The present disclosure relates to a safety system and method to avoid collision between vehicles and other assets in an underground mine based upon computer vision information, asset tracking system information, and motion information.
Description of the Related Art
As background, the following U.S. patents and published patent applications are hereby incorporated by reference herein in their entireties: U.S. Pat. No. 8,527,172; U.S. Pat. No. 8,280,621; US 20070021915, U.S. Pat. No. 6,553,130; U.S. Pat. No. 6,226,389; US 20040022416; and US 20130278440.
Prior art vehicle safety systems for detecting and communicating dangers are commonly comprised of one or more camera sensors, visual monitoring displays, and short range proximity sensors such as ultrasonic or infrared sensors. The camera sensors provide a view of areas not otherwise seen by the driver, and can be visible light sensors, short wave infrared sensors, or long wave infrared sensors, also referred to as thermal cameras. The visual monitoring display is positioned so that the driver can view what the video camera is capturing, and determine whether there is a danger of colliding with an object if the driver proceeds in the current direction. Proximity sensors can be paired with a camera system to detect objects within range and generate audible and visual warnings to the driver. These safety systems, whether limited to only a camera or proximity sensor or encompassing a combination of camera and proximity sensor, provide good localized coverage for detecting objects within a defined proximity, and communicating a level of danger to the driver to avoid collision between objects within a defined line-of-sight distance. Some prior art camera systems add infrared illuminators and sensors to obtain a visible picture of an otherwise visibly dark object. Similarly, some prior art camera systems use thermal imaging to provide a visible picture of naturally emitted infrared energy. Proximity sensors are often paired with these systems to detect and alert of dangers, while a small number of camera systems pair with asset tracking systems to communicate collision dangers of assets rather than generic objects or heat sources. In addition, some of these systems provide automated controls for slowing or stopping or providing audible and/or visual notification to multiple levels of danger.
Prior art tracking systems based on radio signal strength measurements between fixed and mobile objects can also determine positions of equipment and personnel. These systems measure distance between fixed radio transceivers at known locations and mobile radio transceivers using radio frequency (RF) signal strength. For example, a position for the mobile units can be determined using trilateration algorithms based on multiple received signal strength indicators (RSSI). Precision of the calculated location is subject to such factors as the quantity and bearings of the fixed transmitters, interference of the RF signals, and RF signal-to-noise measurements. At best, these systems provide an estimation of the distance to a tracked object and are insufficient as a collision avoidance system on their own.
The prior art systems described above are not reliable for determining, communicating, and providing collision avoidance safety response to predetermined assets and infrared emitting objects in non-line-of-sight conditions such as underground mines and tunnels. None of these systems comprise a collision avoidance system that detects and responds to moving object threats determined from a combination of thermal computer vision information, asset tracking system information, and motion information of individual objects.
The present disclosure provides a collision avoidance system comprising computer vision which may include thermal infrared or other types of video cameras and video analysis software that identifies and monitors important objects, an asset tracking system that provides asset location information using fixed and mobile radio units, and motion information from each mobile unit.
In embodiments of this disclosure, outputs from the computer vision system, asset tracking system, and motion information from each mobile unit are analyzed to achieve collision avoidance between known recognized assets and familiar objects identified as threats. In an exemplary scenario, humans and vehicles are considered assets and are given a unique radio unit for tracking. The assets are tracked using the wireless communications and tracking component. The vehicles are equipped with infrared video or other types of cameras and provide imaging to the computer vision component that identifies objects to the operator such as humans, vehicles, and electrical infrastructure. When the asset tracking data, motion data, and object identification data are combined, the system analyzes collision threats and provides multiple levels of warnings and actions to avoid collision between those assets.
In embodiments, as data is collected by both the computer vision and asset tracking software components, the collision avoidance algorithms evaluate their respective inputs, provide a confidence level for the output, and publish a message to be collected by the collision avoidance software component. The Collision Avoidance Component (CAC) combines all input, weighted by confidence, to form a current threat rating value (TRV). These TRVs are compared against specific thresholds, each with a specific, configurable function or action. The lower the TRV, the less operator action is required to avoid collision with another asset. For example, a low TRV may result in a simple audible beep that warns the asset operator of their proximity to another asset, while a higher TRV may result in a longer, louder tone to reduce speed, and an even higher TRV may automatically trigger a vehicle to stop in order to avoid colliding with another asset. Similarly, the TRV may trigger visible warnings according to the severity of the threat, alternatively or in addition or audible warnings.
One embodiment of the present disclosure provides a collision avoidance system for an underground mine environment where the system may comprise a computer vision component, an asset tracking component, and a motion detection component for calculating a directional velocity component for the asset tracking component. The collision avoidance component may be configured to receive inputs from the computer vision component, asset tracking component, and motion detection component and combine the inputs into an algorithm to produce an output that determines a warning or action for avoidance of a collision between a first asset and a second asset.
In any embodiment of this disclosure, the first asset or second asset may be a vehicle or human.
In any embodiment of this disclosure, the computer vision component may comprise a video imaging component, a computer processor, and a memory, wherein the computer processor receives or is configured to receive video image frames from the video imaging component and analyze them to identify assets according to a set of computer-executable instructions stored in the memory. The video imaging component may comprise one or more thermal infrared cameras which provide or are configured to provide a thermal image capture of an asset. The computer-executable instructions may identify or be configured to identify assets according to object classification algorithms configured to detect human forms, identify objects, and track motion based on one or more heat signature outlines contained in the thermal image capture. The object classification algorithms may determine or be configured to determine the position of a human selected from the group consisting of prone, supine, crouched, standing, sitting, crawling, squatting, and kneeling. Further, the object classification algorithms may recognize or be configured to recognize human faces or features such as eyes, ears, and mouth or recognize heat emitting infrastructure comprising batteries, motors, wires, communications and tracking components, and electrical power supplies.
In any embodiment of this disclosure, the computer-executable instructions may calculate or be configured to calculate the proximity of the human form by the size and heat intensity of the thermal image capture whereby a brighter image pixel indicates warmer areas and closer and larger humans span a greater number of pixels than do farther and smaller humans. The computer-vision component may be trained or be configured to be trained with positive and negative samples of assets in memory. The computer-vision component may be trained or be configured to be trained with positive samples by manually calibrating measurements using the formula F=P*D/S. The computer-vision component may be trained or be configured to be trained such that as the asset's pixel size changes in the image, the distance to the object is calculated with a new pixel value P using the formula D=S*F/P. The computer vision component may provide an output or be configured to provide an output that is a value based on the type and confidence of the asset determined in the video image frame.
In any embodiment of this disclosure, the asset tracking component may be based on fixed mesh radio nodes and mobile mesh radio nodes. A mobile mesh radio node may be placed on an asset and the asset tracking component may be configured such that a Received Signal Strength Indication (RSSI) between the mobile mesh radio node and surrounding fixed mesh radio nodes determines the location of the mobile mesh radio node. Further, the accuracy of the determined location of the mobile mesh radio node may be dependent on the placement and density of the fixed mesh radio nodes and the relative distance between the mobile mesh radio node and its surrounding fixed mesh radio nodes. The distance between fixed mesh radio nodes may be approximately 1500 feet and the asset tracking component may provide a tracking accuracy of better than 450 feet at least 98% of the time. The asset tracking component may include transmit-only beacon nodes which form a grid with spacing between beacon nodes of approximately 200 feet and the asset tracking component may provide a tracking accuracy of approximately 50 feet. The asset tracking component may assign or be configured to assign a confidence value to the determined location of the mobile mesh radio node based on its distance from another mobile mesh radio node.
In any embodiment of this disclosure, the motion detection component may comprise an accelerometer-based motion sensor device placed on the first asset and the motion detection component may determine or be configured to determine a directional velocity component comprising a speed and direction of travel of the first asset based on the accelerometer-based motion sensor. The motion detection component may further comprise an accelerometer-based motion sensor device placed on the second asset and the asset tracking component may determine or be configured to determine a directional velocity component comprising a speed and direction of travel of the second asset based on the accelerometer-based motion sensor. Further, the asset tracking component and motion detection component may determine or be configured to determine an intersection point and time to intersection point of the first asset and second asset.
In any embodiment of this disclosure, the collision avoidance component may produce an output or be configured to produce an output which is a proximity value between the first asset and the second asset, or which is a Threat Rating Value that determines the warning or action based on the inputs.
Another embodiment of the present disclosure provides a collision avoidance system for an underground mine environment where the system may comprise a computer vision component comprising a video imaging component comprising one or more thermal infrared cameras configured to provide video image frames comprising a thermal image capture of one or more assets, a computer processor, and a memory comprising a set of computer-executable instructions configured for instructing the computer processor to analyze video image frames received from the video imaging component to identify assets present in the thermal image capture. The system may further comprise an asset tracking component based on fixed mesh radio nodes and mobile mesh radio nodes where a mobile mesh radio node is placed on a first asset and the asset tracking component is configured to determine the location of the mobile mesh radio node based on a Received Signal Strength Indication (RSSI) between the mobile mesh radio node and surrounding fixed mesh radio nodes. The system may further comprise a motion detection component which is configured to determine a directional velocity component for the asset tracking component based on an accelerometer-based motion sensor device placed on the first asset, wherein the directional velocity component comprises a speed and direction of travel. In addition, the system may comprise a collision avoidance component which is configured to receive inputs from the computer vision component, asset tracking component, and motion detection component and combine the inputs into an algorithm programmed in a set of computer-executable instructions which instruct a processor to calculate a Threat Rating Value that determines a warning or action for the first asset to avoid collision with a second asset.
Another embodiment of the present disclosure provides a method for avoiding collisions in an underground mine environment. The method may comprise one or more of providing a thermal image of a first asset, identifying the type of asset in the thermal image using a computer processor directed by a set of computer-executable instructions to employ object classification algorithms, tracking the location of the first asset based on a Received Signal Strength Indication (RSSI) between a mobile mesh radio node placed on the first asset and surrounding fixed mesh radio nodes, and tracking the speed and direction of travel of the first asset based on an accelerometer placed on the first asset. Further, based on the type of asset and pixel characteristics in the thermal image, location of the first asset, and speed and direction of the first asset, the method may further comprise determining proximity information of the first asset in comparison to a second asset or determining a threat rating based on a rating algorithm, whereby the proximity information or threat rating determine a warning or action to avoid collision of the first asset with the second asset.
Another embodiment of the present disclosure provides a method for avoiding collisions in an underground mine environment. The method may comprise providing a thermal image of a first asset with a computer vision component and identifying the type of asset with the computer vision component. The computer vision component may comprise a video imaging component comprising one or more thermal infrared cameras which provide video image frames comprising a thermal image capture of an object, a computer processor, and a memory comprising a set of computer-executable instructions which instruct the computer processor to analyze video image frames received from the video imaging component to identify assets present in the thermal image capture, wherein the set of computer-executable instructions employ object classification algorithms to identify the asset. Further, the method may further comprise tracking the location of the first asset with an asset tracking component comprising fixed mesh radio nodes and mobile mesh radio nodes, wherein a mobile mesh radio node is placed on a first vehicle or a human and the asset tracking component determines the location of the mobile mesh radio node based on a Received Signal Strength Indication (RSSI) between the mobile mesh radio node and surrounding fixed mesh radio nodes. In addition, the method may further comprise tracking the speed and direction of travel of the first asset with a motion detection component which determines a directional velocity component for the asset tracking component based on an accelerometer-based motion sensor device placed on the first asset, wherein the directional velocity component comprises a speed and direction of travel. Still further, the method may comprise determining a Threat Rating Value through a collision avoidance component which receives inputs from the computer vision component, asset tracking component, and motion detection component and combines the inputs into an algorithm programmed in a set of computer-executable instructions which instruct a processor to calculate the Threat Rating Value. Further, the method may further comprise issuing a warning or action for the first asset to avoid collision with a second asset based on the Threat Rating Value.
The foregoing features and advantages will be apparent from the following, more detailed description of preferred embodiments of the present disclosure as illustrated in the accompanying drawings in which reference characters refer to the same parts throughout the various views.
The accompanying drawings illustrate certain aspects of some of the embodiments of the present disclosure, and should not be used to limit or define the disclosure. Together with the written description the drawings serve to explain certain principles of the disclosure.
It is to be understood by persons of ordinary skill in the art that the following descriptions are provided for purposes of illustration and not for limitation. An artisan understands there are many variations that lie within the spirit of the disclosure and the scope of the appended claims. Unnecessary detail of known functions and operations may be omitted from the current description so as not to obscure the present disclosure.
As used herein, the term “asset” refers to a vehicle or human in an underground mine environment. The vehicle may be any vehicle employed in underground mining, including but not limited to a personnel transport vehicle, a rescue vehicle, a utility vehicle, a loader, a truck, a shearer, a drill, a crane, a flat bed, a lift, a plow, a roof support carrier, and may be powered by any source including electric, battery, diesel, and gasoline. The human may be any person in the underground mining environment, including but not limited to miners, managers, foremen, supervisors, and support personnel.
As used herein, the term “approximately” applied to a value refers to a value that ranges from minus 10% of the value to plus 10% of the value. Thus, “approximately” 100 would refer to any number from 90 to 110.
Human operators diligently avoid collision of their vehicles with other vehicles and walkers when a threat is perceived; however, many situations, such as underground mining, do not provide workers with sufficient sensory inputs to perceive threats of collision. In underground mining, vehicle drivers may be located in a small space facing a wall, such as displayed in
Computer Vision Component
The computer vision component, an embodiment of which is displayed in
In embodiments, the means for video imaging 5 or cameras 5, in
The use of a thermal camera allows for humans to be identified even in dusty situations and heavy clothing as shown in
Multiple image frames are taken to form a continual live video feed for analysis. The number of image frames per second (fps) may range from 10 fps to 30 fps, 5 fps to 50 fps, or 1 fps to 100 fps in various embodiments. Video analysis is performed by the computer image software on each frame in real-time as the image frames are acquired from the camera. Previous frame analysis results are stored for comparison with incoming frames as they are captured. Real-time image processing and comparison with previous images gives the computer vision component the ability to confidently determine short range line-of-sight collision threats, distance of threats, and speed of threats as they appear and progress.
Object classification algorithms perform the role of computer vision and automated machine perception to identify the objects in the analyzed image frames. These algorithms have the ability to detect human forms, identify objects, and track motion. Using the heat signature, visible, or ultraviolet image outline contained in the images, the algorithms determine if the object is a human form, even if the person is in a prone, supine, crouched, upright/standing, sitting, crawling, squatting, kneeling, or other natural human position. In one embodiment, the proximity of the human is calculated by the size and heat intensity whereby warmer areas are indicated by a brighter image pixel and closer and larger humans span a greater number of pixels than do farther and smaller human. Given a recognized body position of the human, comparisons are made to subsequent findings and distance is measured. In other embodiments, other types of proximity sensors may be used in substitution of or in addition to the heat intensity measurements, including sensors based on LIDAR (light detection and ranging), RADAR (radio detection and ranging), SONAR (sound navigation and ranging), ultrasonic, or other infrared sensors.
Confident, acceptable object recognition is achieved by training the computer vision software with positive samples of objects that should be detected, and by providing negative samples whereby no desired objects to be detected exist. When training and calibrating the computer vision software with positive samples, measurements are first manually calibrated using the formula:
F=P·D/S
Where:
F is the focal length of the camera
P is the number of pixels wide of the subject
D is the distance from the camera to the subject
S is the size of the subject
As the subject's pixel size changes in the image, the distance to the object is calculated with a new pixel value P using the formula:
D=S·F/P
The output of the video analysis is a value based on the type and confidence of the object determined in the scene and multiple values that are continuously fed to the collision avoidance component in real-time. Two distinct values are provided, one for human forms and another for fixed and mobile objects. The higher the value, the greater the confidence that the video analysis correctly identified the particular shape. Each value is also accompanied by an amplitude level which represents how close the object is to the camera. A large outline and/or a high heat signature would result in a large amplitude level.
Asset Tracking Component
Another embodiment of this disclosure is a wireless communication and tracking system (i.e. asset tracking component) for an underground mine environment, shown in
The Fixed Mesh Radio Nodes and Mobile Mesh Radio Nodes form part of a Wireless Mesh Network (WMN). The Fixed Mesh Node FMN is a stationary dual-transceiver mesh radio unit operates on the WMN. Multiple units operate together to form the semi-static infrastructure for the WMN. Each FMN has the capability to coordinate individual clusters within the wireless mesh network WMN and route data through the network between mobile nodes and to a Gateway Node. An FMN can also communicate through a wired backbone headend with a wired backbone, such as a leaky feeder system, as well as form the core links for WMN.
Mobile Mesh Radio (MMR) is a portable device carried by personnel that allows them to have voice and data communication with a Network Operations Center and/or other personnel equipped with an MMR. MMR can also be a relay link between another MMR and an FMN, or between a sensor mesh node (SMN) and an FMN.
The accuracy may be enhanced to approximately 50 feet by adding transmit only Beacon Nodes (BCN) to form a grid with spacing of approximately 200 feet.
To improve the average tracking accuracy over a time interval, a maximum speed is applied to the estimated position. If the newly calculated position is farther from the previous position than which can be reached by traveling at the maximum speed, the new position's distance is capped by the maximum distance allowed given the time difference of the two calculations. The bearing between the two positions is unaffected. The maximum speed value is set by the tracking system and is based whether the tracked mobile object is identified as human or a vehicle.
The tracking confidence value assigned to a tracked device is determined by several factors. A strong RSSI value indicates that two tracked objects are close to each other. If the RF signal is from another mobile (non-fixed) device, the position calculation accuracy is reduced accordingly based on the distance of the transmitting device. Fixed infrastructure devices have very high position accuracy and are algorithmically favored in the mobile device's position calculation.
The bearing or direction of travel is also a factor in the tracking analysis. Based on an accelerometer-based motion sensor device in the mobile objects, the speed and direction of travel is measured and transmitted over the wireless Asset Tracking Component along with its unique ID. The collision avoidance tracking algorithm determines if the direction of travel of other mobile objects will intersect with its current path (forward or reverse). If the two paths do not intersect, the directional velocity component will be near zero. If the paths intersect, the intersection point will result in a higher directional velocity factor. Time to the intersection point (based on the calculated speed) will also affect the directional velocity component.
Persons with a tracked device riding in the vehicle present a special case for the vehicle's collision avoidance alarms. The collision avoidance algorithm detects that they are co-located and over-rides the alarms. The de-activation period is extended if the RSSI values are very high and shortened when the RSSI fall below a configured threshold.
Motion Component
Aiding the Collision Avoidance Component in determining when and how to react to a collision threat is the Motion component. This component provides two valuable metrics in determining if a collision could occur: direction of travel and speed. Without adequate knowledge of these two metrics, proper collision avoidance would not be possible.
In embodiments, the motion component may also be considered distributed within the underground mine environment, as it includes sensors placed on one or more mobile objects (i.e. vehicles) within the underground mind environment. Both direction of travel and speed are acquired from the motion sensor of each mobile object. When a change in gravitational force is detected, the motion component evaluates whether or not the change is useful for collision avoidance. For instance, if a coal shuttle vehicle receives a heavy load of coal, the sudden downward thrust of the vehicle will trigger the accelerometer and produce data that is not associated with avoiding collision since the shuttle vehicle is not moving forward or backward. However, if the coal shuttle vehicle moves forward or backward, it will produce useful gravitational force metrics for determining direction and speed to avoid collision with nearby humans and vehicles. The accelerometer may be any type of accelerometer known in the art, analog or digital, including without limitation capacitive, piezoelectric, piezoresistive, Hall Effect, and magnetoresistive accelerometers.
Collision Avoidance
The Collision Avoidance Component (CAC) 8 embodiment shown in
Embodiments of the high level algorithms are demonstrated below:
Threat rating value for tracking system input 1:
TRVTS1=CTS1·DTS1·VTS1
Threat rating value for tracking system input n:
TRVTSn=CTSn·DTSn·VTSn
Threat rating for Collision Avoidance Component:
TRV=(KVH·AVH·VVH)+(KVO·AVO·VVO)+(KTS·max[TRVTS1 . . . TRVTSn])
Where:
KVH=weight constant for the host computer vision component input
AVH=amplitude level for the host computer vision component input
VVH=value of the host computer vision component input
KVO=weight constant for the object computer vision component input
AVO=amplitude level for the object computer vision component input
VVO=value of the object computer vision component input
KTS=weight constant for the asset tracking component input
CTS1=confidence level for the first asset tracking component input
DTS1=directional velocity component for the first asset tracking component input
VTS1=value of the first asset tracking component input
TRVTS1=threat rating value for the nth asset tracking component input
CTSn=confidence level for the nth asset tracking component input
DTSn=directional velocity component for the nth asset tracking component input
VTSn=value of the nth asset tracking component input
TRVTSn=threat rating value for the nth asset tracking system input
TRV=threat rating value for the Collision Avoidance Component
In the first scenario, the computer vision component of vehicle 201 identifies the vehicle ahead, and the computer vision component of vehicle 202 identifies the vehicle behind. The vehicle objects are recognized, and their speeds and distances are continuously calculated while travelling. The asset tracking component likewise determines that the vehicle assets are nearby, and determines the location of each vehicle. Last, the motion component on each vehicle determines its own speed and direction of travel. On each vehicle, the values are then passed to the collision avoidance component, and in this scenario, vehicle 202 is alerted of immediate danger of collision, and vehicle 201 slows to a safe speed of four MPH to allow vehicle 202 to distance itself farther from potential collision.
In the second scenario, all of the collision determination steps are taken as they were in the first scenario. The difference here is that both vehicle 202 and vehicle 203 alert their drivers and reduce to a safe speed below five MPH to allow the drivers to carry on in a safe manner.
The human 307 is in the front camera field of view 308, and travelling the same direction as the vehicle. In this instance, the human can safely travel in the same path as the vehicle as long as the vehicle isn't travelling too close or approaching too fast. Sudden changes in the perceived danger of collision will alert both the vehicle and the human.
Human 311 is travelling nearby the vehicle, but not directly in its path. This scenario provides warning of nearby danger, but does not stop the vehicle as no collision is imminent. As human 311 enters the field of view 308 at close proximity, however, the vehicle will be caused to halt and avoid colliding with the human.
Human 312 is travelling perpendicular to the vehicle, such as in an underground mine crosscut. As the human approaches the vehicle, an audible and/or visible warning is provided for both parties, and the vehicle is halted if the human gets too close. This scenario is quite common in underground mines since crosscuts provide safe shelter for humans while vehicles pass. The vehicle's speed, however, will be reduced as a chance exists that the vehicle may turn into the crosscut where the human is located.
Human 313, similar to human 311, is travelling nearby the vehicle, and into a camera's field of view. The difference is that human 313 is approaching the rear of the vehicle travelling in a similar direction as the vehicle. The human is not in high danger of collision with the vehicle, so human 313 will be alerted of nearby danger, but the vehicle will not be halted.
Human 314 is directly in the vehicle's rear camera field of view 309 but is travelling away from the vehicle. Similar to human 313, human 314 will be warned of nearby danger, but the vehicle will continue on at a safe speed, since there is not an imminent risk of collision with human 314.
It will be understood that the various processes, operations, and/or algorithms of described and/or depicted in this disclosure may be carried out by a group of computer-executable instructions that may be organized into routines, subroutines, procedures, objects, methods, functions, or any other organization of computer-executable instructions that is known or becomes known to a skilled artisan in light of this disclosure, where the computer-executable instructions are configured to direct a computer or other data processing device to perform one or more of the specified processes, operations, and/or algorithms. Embodiments of this disclosure include one or more computers or devices loaded with a set of the computer-executable instructions described herein wherein the one or more computers or devices are instructed and configured to carry out the processes, operations, and/or algorithms of the disclosure. The computer or device performing the specified processes, operations, and/or algorithms may comprise at least one processing element such as a central processing unit and a form of computer-readable memory which may include random-access memory (RAM) or read-only memory (ROM). In embodiments, the computer or device may be positioned on one or more vehicles as one, several, or all of the components of a Collision Avoidance System described in this disclosure. The computer-executable instructions can be embedded in computer hardware or stored in the computer-readable memory such that the computer or device may be directed to perform one or more of the processes, operations, and/or algorithms depicted and/or described herein. Embodiments of this disclosure also include a computer program product comprising one or more computer files comprising a set of computer-executable instructions for performing one or more of the processes, operations, and/or algorithms described and/or depicted herein. In exemplary embodiments, the files may be stored contiguously or non-contiguously on a computer-readable medium, in computer-readable memory on a single computer, or distributed across multiple computers. Embodiments of this disclosure also include a computer readable medium comprising one or more computer files comprising a set of computer-executable instructions for performing one or more of the calculations, processes, operations, and/or algorithms described and/or depicted herein. Further, embodiments of the disclosure include a computer program product comprising the computer files, either in the form of the computer-readable medium comprising the computer files and, optionally, made available to a consumer through packaging, or alternatively made available to a consumer through electronic distribution. As used herein, a “computer-readable medium includes any kind of computer memory such as floppy disks, conventional hard disks, CD-ROMS, Flash ROMS, non-volatile ROM, electrically erasable programmable read-only memory (EEPROM), and RAM.
A skilled artisan will further appreciate, in light of this disclosure, how the processes, operations, and/or algorithms can be implemented, in addition to software, using hardware or firmware. As such, as used herein, the operations in this disclosure can be implemented in a system comprising any combination of software, hardware, or firmware.
Embodiments of this disclosure may include a user interface which may be used in conjunction with the computer-executable instructions. For example, the user interface may include a graphical user interface configured to allow a user to access a camera image or video feed, display a Received Signal Strength Indication of a nearby fixed or mobile mesh radio node, display one or more warnings or alarms to take action to avoid a collision, including a message. The message may indicate the level of threat of collision with an asset and/or instructions designed to avoid collision such as to decelerate or execute an evasive maneuver. The graphical user interface may also communicate that such an action was automatically taken by the collision avoidance component. The graphical user interface may allow a user to perform these tasks through the use of text fields, check boxes, pull-downs, command buttons, and the like. A skilled artisan will appreciate how such graphical features may be implemented for performing the tasks of this disclosure.
Such graphical controls and components are reusable class files that are delivered with a programming language. For example, pull-down menus may be implemented in an object-oriented programming language wherein the menu and its options can be defined with program code. Further, some programming languages integrated development environments (IDEs) provide for a menu designer, a graphical tool that allows programmers to develop their own menus and menu options. The menu designers provide a series of statements behind the scenes that a programmer could have created on their own. The menu options may then be associated with an event handler code that ties the option to specific functions. Text fields, check boxes, and command buttons may be implemented similarly through the use of code or graphical tools. A skilled artisan can appreciate that the design of such graphical controls and components is routine in the art.
The present disclosure has been described with reference to particular embodiments having various features. In light of the disclosure provided above, it will be apparent to those skilled in the art that various modifications and variations can be made in the practice of the present disclosure without departing from the scope or spirit of the disclosure. One skilled in the art will recognize that the disclosed features may be used singularly, in any combination, or omitted based on the requirements and specifications of a given application or design. For example, any of the methods described can be implemented in systems according to the disclosure, while any of the systems described can be configured to operate any of the inventive methods. Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure.
It is noted in particular that where a range of values is provided in this specification, each value between the upper and lower limits of that range is also specifically disclosed. The upper and lower limits of these smaller ranges may independently be included or excluded in the range as well. The singular forms “a,” “an,” and “the” include plural referents unless the context clearly dictates otherwise. It is intended that the specification and examples be considered as exemplary in nature and that variations that do not depart from the essence of the disclosure fall within the scope of the disclosure. Further, all of the references cited in this disclosure including published patents, published patent applications, and non-patent literature are each individually incorporated by reference herein in their entireties and as such are intended to provide an efficient way of supplementing the enabling disclosure as well as provide background detailing the level of ordinary skill in the art.
Number | Name | Date | Kind |
---|---|---|---|
6226389 | Lemelson et al. | May 2001 | B1 |
6281806 | Smith | Aug 2001 | B1 |
6526352 | Johnson | Feb 2003 | B1 |
6553130 | Lemelson et al. | Apr 2003 | B1 |
7102496 | Ernst, Jr. | Sep 2006 | B1 |
7119676 | Silverstrim et al. | Oct 2006 | B1 |
7983685 | Silverstrim et al. | Jul 2011 | B2 |
8244260 | Silverstrim et al. | Aug 2012 | B2 |
8280621 | Edwards et al. | Oct 2012 | B2 |
8385322 | Colling et al. | Feb 2013 | B2 |
8527172 | Moshchuk et al. | Sep 2013 | B2 |
8885559 | Schmidt et al. | Nov 2014 | B2 |
9179475 | Koleszar et al. | Nov 2015 | B2 |
9258722 | Schmidt et al. | Feb 2016 | B2 |
20040022416 | Lemelson et al. | Feb 2004 | A1 |
20050004723 | Duggan | Jan 2005 | A1 |
20050100192 | Fujimura | May 2005 | A1 |
20070021915 | Breed | Jan 2007 | A1 |
20100208244 | Earhart | Aug 2010 | A1 |
20120008542 | Koleszar et al. | Jan 2012 | A1 |
20120011365 | Schmidt et al. | Jan 2012 | A1 |
20120025964 | Beggs | Feb 2012 | A1 |
20120268262 | Popovic | Oct 2012 | A1 |
20130278440 | Rubin et al. | Oct 2013 | A1 |
20150036482 | Schmidt et al. | Feb 2015 | A1 |
Number | Date | Country |
---|---|---|
2005043930 | May 2005 | WO |
2009018212 | Feb 2009 | WO |
2010107440 | Sep 2010 | WO |
2010107441 | Sep 2010 | WO |
Number | Date | Country | |
---|---|---|---|
20150254985 A1 | Sep 2015 | US |