Patent Grant
11625510
This application references the Non Provisional U.S. patent application Ser. No. 16/504,919, filed Jul. 8, 2019 and entitled METHOD AND APPARATUS FOR POSITION BASED QUERY WITH AUGMENTED REALITY HEADGEAR, the entire contents of which are hereby incorporated by reference. This application references the Non Provisional patent application Ser. No. 16/297,383, filed Jul. 5, 2019 and entitled SYSTEM FOR CONDUCTING A SERVICE CALL WITH ORIENTEERING, the entire contents of which are hereby incorporated by reference. This application references the Non Provisional patent application Ser. No. 16/249,574, filed Jan. 16, 2019 and entitled ORIENTEERING SYSTEM FOR RESPONDING TO AN EMERGENCY IN A STRUCTURE, the entire contents of which are hereby incorporated by reference. This application references the Non Provisional patent application Ser. No. 16/176,002, tiled Oct. 31, 2018 and entitled SYSTEM FOR CONDUCTING A SERVICE CALL WITH ORIENTEERING, the entire contents of which are hereby incorporated by reference. This application references the Non Provisional patent application Ser. No. 16/171,593, filed Oct. 26, 2018 and entitled SYSTEM FOR HIERARCHICAL ACTIONS BASED UPON MONITORED BUILDING CONDITIONS, the entire contents of which are hereby incorporated by reference. This application references the Non Provisional patent application Ser. No. 16/165,517, filed Oct. 19, 2018 and entitled BUILDING VITAL CONDITIONS MONITORING, the entire contents of which are hereby incorporated by reference. This application references the Non Provisional patent application Ser. No. 16/161,823, filed Oct. 16, 2018 and entitled BUILDING MODEL WITH CAPTURE OF AS BUILT FEATURES AND EXPERIENTIAL DATA, the entire contents of which are hereby incorporated by reference. This application references the Non Provisional patent application Ser. No. 15/887,637, filed Feb. 2, 2018 and entitled BUILDING MODEL WITH CAPTURE OF AS BUILT FEATURES AND EXPERIENTIAL DATA, the entire contents of which are hereby incorporated by reference. This application references the Non Provisional patent application Ser. No. 15/716,133, filed Sep. 26, 2017 and entitled BUILDING MODEL WITH VIRTUAL CAPTURE OF AS BUILT FEATURES AND OBJECTIVE PERFORMANCE TRACKING, the entire contents of which are hereby incorporated by reference. This application references the Non Provisional patent application Ser. No. 15/703,310, filed Sep. 5, 2017 and entitled BUILDING MODEL WITH VIRTUAL CAPTURE OF AS BUILT FEATURES AND OBJECTIVE PERFORMANCE TRACKING, the entire contents of which are hereby incorporated by reference. This application references the Non Provisional patent application Ser. No. 16/528,104, filed Jul. 31. 2019 and entitled SMART CONSTRUCTION WITH AUTOMATED DETECTION OF ADVERSE STRUCTURE CONDITIONS AND REMEDIATION, the entire contents of which are hereby incorporated by reference. This application references the Non-Provisional U.S. patent application Ser. No. 16/657,660, filed Oct. 18, 2019 and entitled METHOD AND APPARATUS FOR CONSTRUCTION AND OPERATION OF CONNECTED INFRASTRUCTURE, the entire contents of which are hereby incorporated by reference.
The present invention relates to storage areas with automated environmental condition tracking and control. More specifically, the present invention relates to methods and apparatus for quantifying thermal conditions and other environmental variables in a cold storage facility and tracking the locations of products in the cold storage facility and environmental conditions experienced by the product during the product's cold storage life cycle.
Cold storage facilities are used in a variety of industries with differing needs and product storage life cycles. Cold storage points may vary depending on a type of goods to be stored and a length of a storage period. For example, storage temperatures may be as high as about 5°C for non-frozen goods such as fresh meats, dairy, fresh produce and flowers that are stored for relatively short periods of time. Frozen meat and seafood may be stored at temperatures ranging between—10 to 18°C; and industrial storage, or blood, and other biological supplies may need to be stored at less than −30°C.
Each temperature range and type of good may have unique needs that may affect how a cold storage facility is optimally cooled and how important control of other environmental factors, such as moisture, light, and vibration, may be to maintaining product quality during storage.
Accurate, uniform, and efficient control of the environmental conditions in cold storage facilities may be difficult to achieve, particularly in large facilities. Without such control, the quality of goods may be negatively affected. Additionally, due to lack of uniformity in environmental control, it may be difficult to assess which goods have suffered a decrease in product quality. As such, there remains a need for improved control and monitoring of environmental conditions in cold storage facilities.
Accordingly, the present invention provides apparatus and methods for quantifying and alerting a user as to what conditions exist in a cold storage facility and what goods or other items are present in the cold storage facility and where respective items are located and what conditions the items experienced while in the cold storage facility.
Locations are determined based upon wireless communications between transceivers located within specified areas of the cold storage facility. The wireless communications may indicate a location of a condition quantified b a automated sensor. Quantified conditions may include, for example, one or more of: a temperature currently present in the facility, a temperature experienced by an item stored in the facility, conditions of air introduced by an HAVC; humidity, barometric pressure, air flow, door position, presence of humans in the cold storage space, presence of a pallet, presence of a particular item, length of presence in the facility; whether an item has been dropped or otherwise jarred; location of an item, pallet or person; and a location and position of almost any condition quantifiable via electronic sensor. Notifications and/or alerts may be transmitted indicating one or both of a condition present and a location of the condition.
Provided according to embodiments of the invention, are methods, apparatus, devices, and systems for tracking products throughout the storage life cycle at a cold storage facility. Tracking includes both location tracking and tracking of the environmental conditions in which the product has been stored at each location in the facility.
Further provided according to embodiments of the invention are systems and methods for control of environmental conditions within a cold storage facility, in some embodiments, a three-dimensional (3D) profile of temperature, humidity, and/or other environmental conditions may be created and analyzed over time. The information regarding environmental conditions over time may be correlated with the information regarding the location of products over time to determine the environmental conditions of a particular product throughout its time in the cold storage facility. Additionally, monitoring changes in the 3D environmental profiles in the facility over time may allow for optimization of cooling or heating cycles, air input and extraction, and other HVAC operations in view of the specific, events occurring at the facility, the specific configuration of the facility, and the type of products in the facility. For example, changes in temperature may be correlated with specific actions at certain locations, and the system may use this information to optimize heating or cooling in the facility, either automatically or with user input.
Further provided according to embodiments of the invention are methods of tracking and locating the facility's employees, infrastructure, equipment, and sensors. This tracking combined with orienteering methodology and devices may allow for facile location of persons or objects in the facility. Additionally, in the event of an adverse event with such persons or objects, appropriate aid or resources may be routed to the appropriate location using the devices and methodology herein.
Also provided according to embodiments of the invention are user interfaces, devices, systems, and apparatus for tracking products, employees, infrastructure, equipment, and sensors. Such devices, systems, and apparatus may perform the methods described herein, Additionally, the devices, systems, and apparatus may be in logical communication with warehouse management systems (WMS) or other databases that track and control processes in the cold storage facility.
The accompanying drawings, that are incorporated in and constitute a part of this specification, illustrate several embodiments of the disclosure and, together with the description, serve to explain the principles of the disclosure:
As used in the description of the invention and the appended claims, the singular forms “a,” “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise.
A number of embodiments of the present disclosure are described herein. While this specification contains many specific implementation details, they should not be construed as limitations on the scope of any disclosures or of what may be claimed, but rather as descriptions of features specific to particular embodiments of the present disclosure. While embodiments of the present disclosure are described herein by way of example using several illustrative drawings, those skilled in the art will recognize the present disclosure is not limited to the embodiments or drawings described. It should be understood the drawings and the detailed description thereto are not intended to limit the present disclosure to the form disclosed, but to the contrary, the present disclosure is to cover all modification, equivalents and alternatives falling within the spirit and scope of embodiments of the present disclosure as defined by the appended claims.
The headings used herein are for organizational purposes only and are not meant to be used to limit the scope of the description or the claims. As used throughout this application, the word “may” is used in a permissive sense (i.e., meaning having the potential to), rather than the mandatory sense (i.e., meaning must), Similarly, the words “include”, “including”, and “includes” mean including but not limited to. To facilitate understanding, like reference numerals have been used, where possible, to designate like elements common to the figures.
The phrases “at least one”, “one or more”, and “and/or” are open-ended expressions that are both conjunctive and disjunctive in operation. For example, each of the expressions “at least one of A, 13 and C”, “at least one of A, B, or C”. “one or more of A, B, and C”, “one or more of A, B, or C” and “A, B, and/or C” means A alone, B alone, C alone, A and B together, A and C together, B and C together, or A, B and C together.
“Agent” as used herein refers to a person or automation capable of supporting a Smart Device at a geospatial location relative to a ground plane. As used herein, the term “Cold Storage Employee” or “Employee” may be used interchangeably with Agent as a specific example, but it should be understood that any suitable Agent may be used in lieu of an Employee, including an automated Agent. One example of an automated Agent is a transport such as a forklift that is used to transport products in the cold storage facility.
“As Built,” as used herein, includes data that quantifies details of how a specific physical structure was actually constructed, In the present invention, a cold storage facility may be designed and modeled in a 3D virtual setting. As Built data is combined with a design model in a virtual setting to generate an Augmented Virtual Model. As Built data may reflect one or more of: fabrication of the Facility; repair; maintenance; upgrades; improvements; and work order execution associated with the Facility.
An “Augmented Virtual Model” or “AVM” of a Property or Structure may include a CAD model (or similar virtual model) of the Property or Structure, augmented with relevant data about the Property or Structure. This data may include As Built or Experiential data about rooms, features, utilities, or other facts about the Property or Structure.
“Experiential Data,” as used herein, may be generated and entered into the AVM virtual model of the structure. Experiential Data may include data indicative of a factor that may be tracked and/or measured in relation to the Facility. Experiential data is typically generated by Sensors in or proximate to the Facility and may include, by way of non-limiting example, one or more sensors, such as: vibration Sensors (such as accelerometers and piezo electro devices); force transducers; electronic temperature sensing device (e.g. thermocouple, thermistor, resistance based, etc.); amp meters, ohmmeters, switches, motion detectors; light wavelength capture (such as infrared temperature profile devices), water flow meters; air flow meters: and the like. Some examples of Experiential Data may include: details of operation of equipment or machinery in the Processing Facility; vibration measurements; electrical current draws; machine run times, machine run rates, machine run parameters; interior and/or exterior temperatures; opening and closings of doors and windows; weight loads; preventive maintenance; cleaning cycles; air circulation; mold contents; thermal profiles and the like.
“Image Capture Device” as used herein refers to an apparatus for capturing digital or analog image data. An Image Capture Device may be a two- or three-dimensional camera and may include a charge-coupled device (known as a “CCD”).
A “Modality,” as used in conjunction with a transceiver, transmitter, and/or receiver, refers to one or both of a bandwidth of wireless communication and a protocol associated with a bandwidth. By way of non-limiting example, a modality, as used in relation to a transceiver, transmitter, and/or receiver may include: Wi-Fi; Wi-Fi RTT, Bluetooth; UWB; ultrasonic; sonic; infrared; or other logical communication medium.
“Node” as used herein means a device including at least a processor, a digital storage, and a wireless transceiver. A “Base Node” may have additionally functionality including the ability to aggregate data over time, perform controller functions, transmit data via one or more wireless modality, be powered by utility-based alternating current, and/or communicate via a hardwired medium (e.g., via ethernet cabling).
“Property,” as used herein, means a Structure/Facility or piece of real property to be navigated in accordance with the orienteering methods and apparatus described herein. In general, the term “Property” will refer to the cold storage Facility and/or the real property on which it sits.
“Ray” as used herein refers to a straight line including a starting point and extending indefinitely in a direction.
“Reference Point Transceiver” as used herein means an electronic device capable of one or both of transmitting or receiving data, and whose position within a Property is known or capable of being known. An example of a Reference Point Transceiver is a Node.
“Smart Device” as used herein includes an electronic device including, or in logical communication with, a processor, controller, and digital storage, and capable of executing logical commands.
“Structure” as used herein refers to a manmade assembly of parts connected in an ordered way. Examples of a Structure in this disclosure include a building that includes a Cold Storage Facility. “Cold Storage Facility” or “Facility,” as used herein, refers to one or more enclosed spaces, which may include doors, bays, vents, or other methods of ingress/egress, but are generally temperature controlled via an HVAC or other cooling system. Examples of Cold Storage Structures include vaults, lockers, rooms, warehouses, and the like. A cold storage warehouse or Facility may be described with reference to many embodiments herein, but it is to be understood that the same systems and methods could be used with other cold storage Structures.
“Environmental Control System” includes at least one processing device in Communication with an environmental sensor. In some embodiments, the processing device acts as or is in communication with a controller that performs an action based on environmental data in the Facility obtained by a sensor, wherein such actions include creating visual displays, generating alarms, and modifying a heating or cooling cycle by the cooling system. An environmental control system may include a temperature control system, but may also include other non-temperature sensors including humidity, light, motion, vibration, weight, volume, LIDAR, other location positioning sensors/devices. A “Temperature Control System” is one type of “Environmental Control System,” in which the environmental data is temperature data.
As used herein, a “3D thermal detection apparatus” refers to any apparatus or device that can create a 3D temperature profile. In some embodiments, the apparatus may include thermal imaging software with depth of field parameters. Examples include LIDAR relief map with thermal infrared imaging overlaid thereon. In some cases, the 3D thermal detection apparatus may include a series of temperature sensors, such that a 3D profile can be obtained by extrapolating between sensors.
As used herein, a “3D temperature profile” refers to a display or readout that provides temperature measurements throughout a 3D space, whereby temperature can be measured on X, Y, and Z axes or using polar coordinates. The sensitivity and mechanism of display may vary depending on the apparatus used to create the 3D temperature profile. Other types of environmental data may monitored in a 3D space to create 3D profiles. Examples include 3D humidity profiles, 3D air flow profiles, and 3D radiation profiles.
Methods of tracking position of persons, objects, equipment, sensors, and other infrastructure will first be described. Such methods, devices, and systems may be used in various embodiments described herein.
Orienteering methods and apparatus may provide accurate determination of the position of a Smart Device, and/or an Employee or Agent in possession of the Smart Device, relative to a destination or feature of interest. In some cases, highly accurate location position may be determined via automated apparatus and multiple levels of increasingly accurate location determination. A first level may include use of a GPS device (i.e., within the Smart Device) providing a reading to first identify the location of a Property on which a Cold Storage Facility sits (i.e., the cold storage warehouse). The GPS device may further be associated with a high-level, general description of the Property, such as an address, lot number, or other recorded designator.
A second level may use position transmitters located within, or proximate to, the Cold Storage Facility to execute triangulation processes in view of on-site location references, such as a Reference Position Transceiver, near field radio communication beacons at known X-Y position reference points; line of sight with physical reference markers; coded via ID such as bar code, hash tag, and alphanumeric or other identifier. In some embodiments, triangulation may calculate a position within a boundary created by the reference points to within millimeter range. In some embodiments, differential GPS may be used to accurately determine a location of a Smart Device with a sub centimeter accuracy. This location may be determined with reference to two- or three-dimensional Cartesian or polar coordinates.
In some examples, a reference point Node may be placed in a fixed location and function as a transceiver of signals. For example, a Node may receive and transmit signals in a radio frequency band of the electromagnetic spectrum. In a simple form, a Node may receive an incoming wireless radio frequency communication and also broadcast a radio frequency wireless communication. Radio frequencies utilized for wireless communication may include those within the electromagnetic spectrum radio frequencies used in UWB, Wi-Fi and Bluetooth modalities, as well as IR, bisible and UV light as examples.
In some embodiments, a Node is operative to communicate timing data and data generated by a sensor. Such communications may provide identifying information unique to the Node, data related to the synchronization of timing with reference point transceivers, and/or data received from other Nodes.
A triangulation calculation of the position of a Smart Device or a Node may result from a system of multiple reference position Nodes communicating timing signals to or from the Smart Device or Node. Methods of calculating positions via wireless communications may include one or more of: RTT, RSSI, AoD, AoA, timing signal differential and the like. Triangulation or other mathematical techniques may also be employed in determining a location.
The process of determination of a position based upon triangulation with the reference points may be accomplished, for example via executable software interacting with a controller in a Node, a Smart Device, or server.
Referring now to
The Reference Position Transceivers 101-104 may include devices such as a radio transmitter, radio receiver, a light generator, or an image-recognizable device (i.e., an apparatus set out in a distinctive pattern recognizable by a camera). A radio transmitter may include a router or other Bluetooth or other communication device for entering into logical communication with a controller. In some embodiments, Reference Position Transceivers 101-104 may include a Wi-Fi router that additionally provides access to a distributed network, such as the Internet. Cartesian coordinates (including Cartesian coordinates generated relative to a GPS or other reference point), or any other coordinate system, may be used as data that may be utilized for one or more of: locating one or both of an Employee 100; indicating a direction of interest; and identifying a Structure or defined area 106. A radio transmitter may include a router or other Wi-Fi device. The radio transmitter may include transmissions on traditional Wi-Fi frequencies (300 MHz-60 GHz),, including ultra-wideband frequencies (a bandwidth above 500 MHz, e.g. between 3.1 and 10.6 GHz or 24 GHz), some specific products may utilize bandwidths of between 6 to 8.5 GHz; 6 to 9 GHz, 3.5 to 6,5 GHz, 6 to 8 GHz, and the like. The light generator may distribute light at human-safe intensities and at virtually any frequency known in the art. Such frequencies include, without limitation, infrared, ultraviolet, visible, or nonvisible light. Further, the light beacon may comprise a laser, which may transmit light at any of the aforementioned frequencies in a coherent beam.
Positions of the Reference Point Transceivers 101-104, which are fixed in respective locations 108(a-d) within or proximate to the defined area 106, define a wireless communication, area (WCA) 112. Essentially the WCA 112 defines an area in which a location of a person may be tracked. A sensor may continue to quantify a physiological state of the person supporting an appropriate sensor whether or not the person is within the WCA 112.
In some embodiments, data quantifying a physiological state of a person may be stored within a Node or Tag supported by the person and transmitted via wireless communication once a Node or Tag is within wireless communication range of a Reference Point Transceiver 101-104. The Reference Point Transceivers 101-104 may include devices capable of wireless communication via a same modality as that utilized by the Agent-supported Transceiver 105. A radio frequency transceiver included in one or both of the Reference Point Transceivers 101-104 and the Agent-supported Transceiver 105 may therefore transmitters and receivers operative to communicate via wireless modalities that include, for example; Wi-Fi, Bluetooth, Ultra-wideband (“UWB”), ultrasonic, infrared, or other communication modality capable of logical communication between Transceivers 101-105.
In some embodiments, a Reference Point Transceiver 101-104 may include a multi-modality transceiver, that communicates more locally via a first modality, such as Ultrawideband (“UWB”) Bluetooth, Wi-Fi, ANT, Zigbee, BLE, Z Wave, 6LoWPAN, Thread, Wi-Fi-ah, NFC (near field communications), Dash 7, Wireless HART or similar modality; and to a greater distance via a second modality, such as a cellular communication modality (e.g. 3G, 4G, 5G and the like), sub GHz modality, Internet Protocol modalities and the like which may provide access to a distributed network, such as the Internet. Other modalities are also within the scope for the present invention.
Wireless communications between Transceivers 101-105 may engage in logical communications to provide data capable of generating one or wore of: Cartesian coordinates, polar coordinates, vector values, AoA, AoD, RTT, RSS, a GPS position, or other data that may be utilized for one or more of: locating one or both of an Agent 100; indicating a direction of interest; and identify a defined area 106.
A precise location may be determined via logical processes, such as triangulation; trilateration; and/or angle phase change; based upon timing values or other mechanism to generate a distance from one or more antennas in the multiple Reference Point Transceivers 101-104 to one or more antennas in an Agent-supported Transceiver(s) 105.
For example, a radio transmission or light transmission be measured and compared from three Reference Point Transceivers 101-103. Measurement may include, one more of; a timing value, a received transmission strength, received transmission amplitude, and received transmission quality.
Other embodiments may include a device recognizable via image analysis via a sensor, LiDAR, Image Capture Device, CCD device, and the like which may capture an image of three or more recognizable features. Image analysis may identify three or more of the recognizable features and a size ratio of the respective image captured recognizable features may be utilized to calculate a distance from each and thereby a position of the Agent 100. Similarly, a height designation may be made via triangulation using the position identifiers as reference to a known height or a reference height
A plurality of modalities may allow for increased accuracy because each modality may have a different degree of reliability. For example, a Smart Device may measure the timing signal transmitted by a Wi-Fi router within a different error tolerance than it may measure the receipt into a photodetector of infrared laser light This has at least two principle benefits. First, the location calculation may, in some embodiments, be a weighted average of the location calculated from each modality. Second, outliers may be shed. For example, if the standard location calculation comprises a weighted average of the location as calculated by five modalities, but one modality yields a location greater than two standard deviations from the average computed location, then that modality may not be considered for future weighted location calculations.
Additionally, the radio transmitters and nor transceiver in the Smart Device may comprise multiple antennas that transmit signals in a staggered fashion to reduce noise. By way of non-limiting example, if there are three antennas, then the may transmit a signal in intervals of 20 milliseconds. Given this rate of transmission, a detected time of arrival may be used to determine the distance between the transmitter and the antenna (i.e., the Smart Device). Moreover, the antennas may comprise varying lengths to accommodate desirable wavelengths. Further, dead reckoning may be used to measure location, using traditional methods of numerical integration.
A precise location may be determined based upon wireless transmissions between Nodes, such as between an Smart Device and the Reference Position Transceivers Timing determinations—as well as signal qualities like angle of arrival, angle of departure, transmission strength, transmission noise, and transmission interruptions—may be considered in generating relative positions of Nodes. Additional considerations may include Al and unstructured queries of transmissions between Nodes and triangulation logic based upon a measured distance from three or more Reference Position Transceivers 101-104. For example, a radio transmission or light emission may be measured, and timing associated with the radio transmission or light to determine a distance between Nodes. Distances from three Reference Position Transceivers 101-103 may be used to generate a position of a Node in consideration. Other methodologies include determination of a distance from one or more Nodes and a respective angle of arrival and/or angle of departure of a radio or light transmission between the Node in consideration and another Node (Reference Point Node or dynamic position Node).
Other embodiments may include a device recognizable via analysis of an image from an Image Capture Device which may include a sensor. LiDAR, CCD device or MOS device camera. The Image Capture Device may capture an image of three or more recognizable features which may in an example occupy positions of Reference Position Transceivers. Image analysis may recognize the identification of each of three or more of the recognizable features and a size ratio of the respective image-captured recognizable features (compared, in some embodiments, to a known absolute size of the subject) may be utilized to calculate a precise position. Similarly, a height designation may be made via triangulation using the position identifiers as reference to a known height or a reference height. Discernment of a physical artifact may, for example, be based upon topographical renditions of physical aspects included in the Structure, such as those measured using LIDAR, a magnetic force, image data (or a point cloud derived from image data). A pattern on a surface may convey a reference point by a recognizable pattern (which may be unique to the setting). Vernier or three-dimensional structure as non-limiting examples. A Smart Device ascertaining a physical reference mark and a distance of the Smart Device to the mark may determine a relative location in space to a coordinate system of the marks.
In some embodiments of the invention, position determination in a Cold Storage Facility or on the Property contemplates determination of a geospatial location using triangulation, trilateration, or multilateration techniques. In some embodiments, a geospatial location relative to one or more known reference points is generated. The geospatial location in space may be referred to as having an X,Y position indicating a planar designation (e.g. a position on a flat floor), and a Z position (e.g. a level within a Structure, such as a second floor) may be generated based upon indicators of distance from reference points. Indicators of distance may include a comparison of timing signals received from wireless references. A geospatial location may be generated relative to the reference points. In some embodiments, a geospatial location with reference to a larger geographic area is associated with the reference points, however, in many embodiments, a controller will generate a geospatial location relative to the reference point(s) and it is not relevant where the position is located in relation to a greater geospatial area, In addition to these Cartesian coordinates, polar coordinates may be used, as further described below.
A geospatial location based upon triangulation may be generated based upon a controller receiving a measurement of angles between the position and known points at either end of a fixed baseline. A point of a geospatial location may be determined based upon generation of a triangle with one known side and two known angles. Referring again to
A geospatial location based upon trilateration may be generated based upon a controller receiving wireless indicators of distance and geometry of geometric shapes, such as circles, spheres, triangles and the like.
A geospatial location based upon multilateration may be generated based on a controller receiving a measurement of a difference in distance to two reference positions, each reference position being associated with a known location. Wireless signals may be available at one or more of: periodically, within determined timespans, and continually. The determination of the difference in distance between two reference positions provides multiple potential locations at the determined distance. A controller (such as one in the Smart Device) may be used to generate a plot of potential locations. In some embodiments, the potential determinations generally form a curve. Specific embodiments may generate a hyperbolic curve.
The controller may be programmed to execute code to locate a relatively exact position along a generated curve, which is used to generate a geospatial location. The multilateration system thereby receives as input multiple measurements of distance to reference points, wherein a second measurement taken to a second set of stations (which may include one station of a first set of stations) is used to generate at second curve. A point of intersection of the first curve and the second curve is used to indicate a specific location.
In examples described thus far, a position of an agent or user may be determined as discussed. It may be equally possible for a transceiver to be associated with a package, storage element or other physical entity as shall be described further herein, utilizing the techniques as have been described relating to a user or agent.
In exemplary embodiments, as described herein, the distances may be triangulated based on measurements of UWB, Wi-Fi or sub GHz strength at two points. Transceiver signals propagate outward as a wave, ideally according to an inverse square law. Ultimately, a crucial feature of the present invention relies on measuring relative distances between two points. In light of the speed of Wi-Fi waves and the real-time computations involved in orienteering; these computations need to be as computationally simple as possible. Thus, depending upon a specific application and mechanism for quantifying a condition or location, such as a measurement, various coordinate systems may be desirable. In particular, if the Smart Device moves only in a planar direction while the elevation is constant, or only at an angle relative to the ground, the computation is more simple.
Cartesian coordinates (including Cartesian coordinates generated relative to a GPS or other reference point), or any other coordinate system, may be used as data that may be utilized for one or more of: locating one or both of an Agent 100; indicating a direction of interest; and identifying a defined area 106. A radio transmitter may include a router or other Wi-Fi device. The radio transmitter may include transmissions via a Ultra-Wideband (“UWB”) frequencies including, for example, frequencies above 500 MHz, 3.5-6.5 GHz or other frequencies usable with a UWB modality; on Wi-Fi frequencies (300 MHz-60 GHz), sub GHz frequencies or other modality. A light generator may distribute light at human-safe intensities and at virtually any frequency known in the art. Such frequencies may include, without limitation: infrared, ultraviolet, visible, or nonvisible light. Further, a light beacon may comprise a laser, which may transmit light at any of the aforementioned frequencies in a coherent beam.
One exemplary coordinate system includes a polar coordinate system. One example of a three-dimensional polar coordinate system is a spherical coordinate system. A spherical coordinate system typically comprises three coordinates: a radial coordinate, a polar angle, and an azimuthal angle (r, θ, and φ, respectively, though θ and φ are occasionally swapped conventionally).
By way of non-limiting example, suppose Point 1 is considered the origin for a spherical coordinate system (i.e., the point (0, 0. 0)). Each Wi-Fi emitter e-1, e2, e3 can be described as points (r1, θ1, φ1), (r2, θ2, φ2), and (r3, θ3, φ3), respectively. Each of the ri's (1<i<3) represent the distance between the Wi-Fi emitter and the Wi-Fi receiver on the Smart Device.
It is understood that in some embodiments, an azimuth may include an angle, such as a horizontal angle determined in an arcuate manner from a reference plane or other base direction line, such as an angle formed between a reference point or reference direction; and line (ray or vector) such as a ray or vector generated from or continuing to a Smart Device. In preferred embodiments, the ray or vector may be generally directed from a Reference Position Transceiver towards, and/or intersect one or more of: an item of interest; a point of interest; an architectural aspect (such as a wall, beam, header, corner, arch, doorway, window, etc.); an installed component that may act as a reference in an augmented virtual model (AVM) (such as, for example, an electrical outlet, a light fixture, a plumbing fixture, an architectural aspect; an item of equipment; an appliance; a multimedia device, etc.); another Reference Position Transceiver or Other identifiable destination.
Accordingly, in some embodiments, a spherical coordinate system may include Reference Position Transceiver that is capable of determining an angle of departure of a location signal and a Transceiver that is capable of determining an angle of arrival of the location signal; one or both of which may be used to facilitate determination of an applicable azimuth.
According to various embodiments of the present invention, one or both of an angle of departure and an angle of arrival may therefore be registered by a Transceiver that is transmitting and/or receiving wireless signals (e.g. radio frequency, Bluetooth 5.1, sonic frequency, or light. frequency).
In another, aspect, multiple timing signals that have been wirelessly communicated may be mathematically processed to increase an overall accuracy, such as, for example, combined into a weighted average, average, mean, weighted mean. In situations with multiple timing signals communicated via different modalities a weighted average may be determined to be most beneficial. In some embodiments, conditions in an environment through which the wireless communication of values for variables enabling determination of a location of a Transceiver 105 may be quantified by sensors and the quantified conditions may be used to further ascertain a beneficial mathematical process. For example, if sensors indicate significant electrical interference in a particular bandwidth of the electromagnetic spectrum, such as a spectrum utilized by UWB and/or Bluetooth modalities, a mathematical process may give a higher weight to a value (for a variable useful in determining the position of a Transceiver 105) transmitted via an infrared modality which is not effected by the electrical interference. Likewise, if a sensor reading quantifies a significant amount of particulate or other interference in an atmosphere through which a wireless communication is transmitted, a lower mathematical weight may be allocated to an infrared transmission (or other light beam effected by particulate) value.
Also, processing may allow for outliers of values for variables useful in determining a location to be shed. For example, if a standard location calculation comprises a weighted average of multiple values for variable useful for determining a location, which may include values generated based upon wireless communication using multiple modalities, but a particular modality yields a location greater than two standard deviations from an average computed location, then that modality may not be considered a recalculation and/or in future weighted location calculations. Similarly, values generated using a single modality that fall outside a designated deviation (e.g. two standard deviations or three standard deviations) may be excluded from a value generated via mathematical processing (e.g. average, weighted average, mean, median, mode, etc.).
Additionally, the radio transmitters, receivers, and/or transceivers in a Tag, Node and/or Smart Device may include multiple antennas that transmit and/or receive electromagnetic transmissions. In some embodiments, the multiple antennas may transmit and/or receive in a staggered fashion suitable to reduce noise.
Additional considerations may include AI and unstructured queries of transmissions between Tags and triangulation logic based upon a measured distance from three or more Reference Point Transceivers 101-104. For example, a radio transmission or light emission may be measured, and timing associated with the radio transmission or light emission to determine a distance between disparate transceivers included in Tags. Nodes, Smart Devices and the like. Distances from three Reference Point Transceivers 101-103 may be used to generate a position of a Transceiver in consideration. Other methodologies include determination of a distance from one or more Transceiver and a respective angle of arrival and/or angle of departure of a radio or light transmission between the Node in consideration and another Transceiver (Reference Point Transceiver or dynamic position Transceiver, e.g. an Agent supported lag or Node).
In some embodiments, locating an Agent occurs in or proximate to a Cold Storage Facility in which Reference Position Transceivers, (including, for example, one or more of: Wi-Fi Transceivers, UWB Transceivers, Bluetooth Transceivers, infrared Transceivers and ultrasonic Transceivers) may be located above and/or below an Employee. In these embodiments, a cylindrical coordinate system may be more appropriate. A cylindrical coordinate system typically comprises three coordinates: a radial coordinate, an angular coordinate, and an elevation (r, θ, and z, respectively). A cylindrical coordinate system may be desirable where, for example, all Wi-Fi emitters have the same elevation. Angles may be determined as described above.
In some embodiments of the present invention, sound waves may be used to perform one or more of: location determination, movement tracking in interior or exterior locations, and distance calculation from a position to an Employee, which may be accomplished based upon transmission and receipt of sonic transmission. Sound wave transmissions include several significant attributes, which may be beneficial for a given set of circumstances when used for radiofrequency-based location determination. According to the present invention, sonic waves may be deployed independently, or in combination with, transmissions and reception of logical communications utilizing other bandwidths, such as bandwidths associated with Wi-Fi, Bluetooth, ANT, infrared or almost any wavelength in the Industrial, Scientific and Medical bands (sometimes referred to as “ISM Bands”). Methods, devices, and apparatus for using sound waves in location determination may be found, for example, in U.S. Pat. No. 10,628,617, the contents of which are incorporated herein.
In certain embodiments of the invention, a direction of interest of an Agent or Smart Device may be determined. In some cases, a direction determination may be based upon a movement of a device. For example, a device with a controller and an accelerometer, such as mobile Smart Device, may include a user display that allows a direction to be indicated by movement of the device from a determined location acting as a base position towards an As Built feature in an extended position. In some implementations, the Smart Device may first determine a first position based upon triangulation with the reference points and a second position (extended position) also based upon triangulation with the reference points. These position determinations may proceed as described above. The process of determination of a position based upon triangulation with the reference points may be accomplished, for example via executable software interacting with the controller in the Smart Device, such as, for example via running an app on the Smart Device.
FIG. FIG. Referring now to
The Agent supported Transceivers 204-212 are co-located with the Agent resulting from being supported by the Agent, therefore a position of the person 201 may be designated as being the same as the position of one or more of the Agent supported Transceivers 204-212 or sonic mathematical correlation of the respective positions of the Agent supported Transceivers 204-212, such as for example: an average, a weighted average, a mean, a median, other function or algorithm involving the respective positions of two or more to the Agent supported Transceivers 204-212.
In some embodiments, a person 201 (or other Agent or mammal) will support two or more Transceivers 204-212 and each of the Agent supported Transceivers 204-212 will enter into communication with the Reference Point Transceivers 202A-D in a manner conducive to generating a respective position of the two or more Agent supported Transceivers 204-212 supported by the person 201. A directional vector and/or ray may be calculated based upon respective positions of two or more of the Agent supported Transceivers 204-212 supported by the person 201. The directional vector and/or ray may be used, for example, to designate a forward facing position of the person 201 or a direction of interest associated with the person 201. For example (and discussed in more detail below) headgear 251 may include multiple headgear mounted transceivers 211, such as a transceiver 211 along a front portion of a headgear (front brim or front headband portion or from of eyeglasses) and a second headgear transceiver along a rear portion of the headgear (not shown in
Other wearable items 252-257 or subcutaneously implanted items, which may also be considered a wearable item 258, may also include Transceivers 204-212, such as those supported at areas of the person 201 other than the person's head. By way of example, Transceivers 204-212 may be attached to, incorporated into, mourned on, held by, or otherwise supported by items wearable by a person 201 or items that may be held in the person's hand.
A subcutaneously implanted item, such as wearable item 258 may include sensors of various kinds to measure physiological measurements as well as communication and identification devices. A subcutaneously implanted item ma have art outer casing consistent with insertion under the skin such as, in a non-limiting sense, glass or hydrogels. The implant may measure temperatures, pressures, electrical signals and the like. In sonic examples, the subcutaneously implanted item may include antennas for the reception of signals and energy. The subcutaneously implanted item may be paired with other devices such as transponders and sensors to provide sensed measurements, location and orientation information. For purposes of discussion, some examples of subcutaneously implanted sensors and devices may be classified as a type of wearable item in the discussion herein.
The present invention additionally provides for Sensors 204-212 (with additional sensors illustrated and discussed in subsequent figures) that are supported by the person 201 or located within the structure 200. The sensors quantify a condition within the structure 200. For example, the condition may be a physical state present in the structure or a physiological state present in the person 201.
In some preferred embodiments, Sensors may be placed on an inner surface of a wearable item such the sensor has access to a skin surface on the person 201. Access to a skin surface may allow, or improve quantification of a physiological condition (sometimes referred to herein as a Biometric). For example, access to a skin surface may allow for electronic quantification of a body temperature, a heart rate, a blood oxygen level, a blood sugar level, blood pressure, intracranial pressure and other conditions present and quantifiable via electronic sensors supported by the person 201.
In other embodiments, a structure sensor 213 may be positioned, such as mounted on an architectural aspect or equipment item, and receive energy from an environment around the sensor in a manner that allows the structure sensor 213 to quantify a condition within the environment. For example, a structure sensor 213 may receive environmental input 241 such as infrared energy into the sensor and based upon the receipt of environmental input 241, such as for example infrared energy, the structural sensor 213 may quantify surface temperatures of items within the structure 200. The items within the structure may include person(2) and the structural sensor may quantify a respective surface temperature of each person 201-201A. if a structural sensor 213 is positioned to receive environmental input, such as infrared energy 241 from a known area, the position of a person 201-201A may be determined via wireless communications (as discussed herein) and a quantified temperature may be correlated to a particular person based upon the location of the person 201-201A and the known area monitored and quantified by the structural sensor 213.
In another aspect, a location of Agent supported Transceivers 204-212 may be determined via wireless communications with Reference Point Transceivers 202A-D and used to designate a forward facing direction of a person 201 at a particular instance in time. Therefore, one or more of: triangulation, trilateration and determination of an angle of transceiving may be used to designate a forward facing position. For example, an AoD and/or AoA may be used to designate a frontward facing direction (or rearward, sideways facing direction) at a particular instance in time (and/or timeframe) while triangulation is used to generate a geospatial position during the same timeframe (or an overlapping timeframe).
As illustrated in
Sensors in other wearable items, may quantify similar or variant conditions. For example, an accelerometer or piezo device in a footwear 256 may quantify steps. A Transceiver in a ring 212 may quantify hand movement, such as a natural swing movement during walking or running, a rotational movement while operating a steering wheel, a vertical movement while lifting or climbing a ladder, etc., and in some embodiments may work in correlation with an image capture device, such as stereoscopic cameras in a head gear 251 to quantify hand and finger movements that may be processed and translated into control commands.
An eye covering 257 type item supported by the person may include, for example, eye glasses, goggles, an augmented reality (A/R) headset, a virtual reality (V/R) headset, facemask or other item that is generally placed in position in front of a person's 201 eyes may include one or multiple transceivers 205 that may transceive to help determine a position of the person 201 wearing the eye covering 257. The one or multiple transceivers 205 may also transceive in a manner that allows for determination of a forward direction of the eye covering 257 such that a designation of a direction the person 201 is facing may be correlated with the forward direction of the eye covering 257 The forward direction of the eye covering may be determined according to the methods and devices discussed herein for determining a direction, such as a direction of interest.
In another aspect, one or more sensors 237 mounted on the eye covering 257 may monitor and quantify eye movement and a direction of eye focus. A direction of eye focus quantified by the sensor 237 may assist in determining one or more of: a forward facing direction, what the eye is focused on, whether the person 201 is looking down or up, nystagmus movement or her health and/or performance condition.
As discussed above, Transceivers 212-236 may be mounted on a wearable item 251-258 such that as a wearable items with one or more Transceivers 212-236 and a Transceiver 204-212 may quantify a condition experienced by the person 201 wearing the wearable item 251-258. A condition experienced by the person may include one or more physiological state(s) of the person 201, such as, for example, a heartrate, a body temperature, a breathing rate, breathing pause, eye movement, body conductivity, body capacitance, or other biophysical and/or biochemical state or condition.
Sensors incorporated into a wearable item may therefore provide quantification of bodily conditions present in the wearer. The quantified body conditions may be transmitted via a wireless communication to a processor that is operative to identify a bodily condition. For example, the processor may check to ascertain if a quantified body condition is within thresholds determined to indicate a healthy condition of the body; or if the quantified body condition indicates a transitory state that may ultimately result in an unhealthy state; or if the quantified body condition indicates a present state of a human body that is in an unhealthy state.
As further discussed herein, a condition quantified by a sensor may precede execution of a remedial action based upon a value of a variable used to represent the condition quantified by the sensor. In those embodiments that include a body condition being quantified by a sensor worn in a wearable item 251-258 by a person 201, a remedial action may include, by way of non-limiting, one or more of: alerting the person 201 wearing the wearable item 251-258. An alert to the person 201 wearing the wearable item 251-258 may include, for example, a human audible alert, a kinetic action, such as a vibration or TENS (Transcutaneous electrical nerve stimulation), a visual indication (such as a light), or activation of another device perceptible by the person 201, An alert may also be generated that is communicated to a person other than the person 201 wearing the wearable item 251-258, or a server, processor, controller or other automation. For example, a supervisor in a processing plant may receive one or both of an email and a text message indicating that a person 201 wearing a wearable item 251-258 and working in the processing plant has a body temperature that exceeds 100° F. and an elevated heartrate. The supervisor may recognize that a fever and elevated heartrate may be indicative of a viral infection and therefore contact the person 201 wearing the wearable item 251-258 and direct the person 201 to be further assessed. In addition, because the person will also be associated with wireless position tracking via a. transceiver included in a wearable wrist band, the present invention may determine where the person 201 has moved about while the person 201 has generated an indication of a fever, e.g. an elevated body temperature, Moreover, a position of a first person 201 may be determined relative to a position of a second person 201A during one or more time intervals. Additional remedial actions may be implemented if the first person 201 has been located at a position within a threshold distance 240 from the second person 201A. For example, both the first person 201 and the second person 201A may be placed in quarantine until such time as it is determined that both the first person 201 and the second person 201A are free of viral infection. Alternatively, in some embodiments, a comparison of a first pattern of wireless positions indicative of a path travelled by a first person 201 and a second pattern of wireless positions indicative of a path travelled by a second person 201A may indicate that the second person 201A is not likely to contract a virus from the first person 201 because the second person 201A did not come within a distance 240 determined to be statistically favorable to transfer of the virus.
In some embodiments, Transceivers 212-236 may quantify almost any condition present in an area defined by a range of a Reference Point Transceiver 202A-202D communicating with a Transceivers 204-212 associated with a Transceivers 212-236. Embodiments will include an area of communication between the Transceiver 204-212 and the Reference Point Transceivers 202A-202D that includes an interior of a facility or other structure 200, and an area immediately surrounding the structure. As discussed above, a condition quantified by sensors within range of the Reference Point Transceivers 202A-202D may be a physiological state of a person 201. In addition, the condition quantified by sensors within range of the Reference Point Transceivers 202A-202D may include a condition in an environment within the area, such as an atmospheric environment and/or a condition in a structure 200 or an item of machinery.
As such, in some embodiments, a wearable item 251-258 that includes a wristband, headgear, vest, gown, footwear, or other capable of assessing a condition of a person 201 and an environment item, may include sensors 214-236 that quantify a condition present in the person 201 that wears the wearable item 251-258 or a condition ambient to the person 201 wearing the wearable, item 251-258. Non-limiting examples of conditions ambient to a person include, temperature, humidity, visible light, IR light, UV light, other electromagnetic energy within a defined bandwidth, movement, vibration, impact, noise, motion or other quantifiable condition.
In some embodiments, one or more sensors 213 may be positioned other than as part of a wearable item 251-258. For example, a sensor 213 may be fixedly attached to a structure 200. The structural sensor 213 attached to the structure 200 may receive environmental input 241 descriptive of an environment surrounding a person 201-201A. The present invention is able to generate a position of a person 201-201A, quantify a condition in an environment 238 surrounding the person 201-201A, quantify a condition present as a physiological state of the person 201-201A at multiple instances in time and store each position and quantified condition. In addition, the present invention may generate a user interface indicating a present status of the above listed conditions, a chronology of states of conditions and positions, a summary (including mathematical analysis, such as average, mean, weighted average, median etc.).
The present invention may also provide a user interface or other human discernable device, such as a light, audio Instruction, mechanical actuation, video, narrative or other communication medium to indicate what actions the controller deems to be appropriate based upon a particular set of conditions quantified by the sensors 214-236 and algorithmic analysis, such as unstructured queries and artificial intelligence. For example, for a viral contagion, such as, for example, COVID-19, a close contact may be defined as anyone or any tracked item that Nus within six feet of an infected person for at least 15 minutes starting from 48 hours before the person began feeling sick, if it is known that the virus propagates well in an environment with certain temperature and humidity conditions and does not propagate well in an environment with other temperature and humidity conditions, then the present invention may provide an interface indicating a first value for remedial action, such as a distance 240 between persons 201-201A to maintain, or what should be considered “close contact” based upon a first set of environmental conditions that are quantified, and a second value for that should be considered “close contact” for a second set of environmental conditions.
More specifically, if an exemplary virus propagates well in an environment of less than about 10° C. and in humidity of less than about 75%; and the same virus has limited propagation in environments of over 20° C. and atmospheric humidity of about 80% or more, then the present invention may provide an interface or other indicator of a distance 240 defining a first value for a “close contact” based upon a temperature and humidity of an environment in which a people are situated; e.g. a first value defining a distance 240 between persons 201-201A for close contact as 6 feet or less and a second value defining a distance 240 between persons 201-201A for close contact as 12 feet or less. A choice of the first value for dos contact and the second value for close contact will be contingent upon measured quantifications of a temperature and humidity of an environment 238 in which a person 201-201A is situated. Other quantified environmental conditions may become a factor upon which user interface values are based.
In addition to a suggested distance 240 to maintain between persons 201-201A, the present invention provides for sequential position determination for persons 201-201A based upon wireless communication (position based upon wireless communication is discussed in more detail herein). According to the present invention, wireless communication including values of variables capable of determining a position of a transceiver supported by an Agent, such as a person 201-201A is conducted on a periodic and/or episodic basis. Periodic basis includes a times period between wireless communications, such as every t seconds, or every minute, or every five minutes or other time period appropriate toile circumstances, such as a likelihood of movement and power consumption. A wireless communication based upon an episodic basis may include wireless communicating as a response to an event, such as movement detected by a tower powered device, such as accelerometers, solid state magnetic sensors, gimbals and the like, or other relatively low powered device that is capable of detecting movement. Other events may also precede a wireless communication, such as a detection of another person within a defined distance 240, a change in another variable, such as temperature, carbon dioxide level, inert gas level, oxygen level, presence of a chemical, or almost any other measurable quantity. Essentially, if a threshold level of a quantifiable condition is reached, an onboard controller is programmed to conduct wireless communication and calculation of a position of a transceiver conducting the wireless communication.
In another aspect, environmental conditions and person conditions may be measured by sensors included in wearable items worn by a person, or by sensors located in a position conducive to quantifying a condition of one or both of an environment 238 in which the person 201-201A is located and a position conducive to quantifying a condition inherent in the person 201-201A. For example, a sensor 213, such as an infrared (IR), thermistor, thermocouple, or other sensor type may receive environmental input 241 that the sensor 213 may use to quantify a condition, such as, a temperature of the environment 238 or a condition of a person 201-201A, The sensor 214-236 may be located in a wearable item 251-258, or a sensor 213 may be fixed or removably fixed, in a position in a structure 200. Either way a sensor 213-236 may be operative to quantify a temperature condition of a person 201-201A or an environment 240. A value generated by a sensor 213-236 may be stored and or processed by a controller. Processing may include a mathematical combination of a sensor reading with other sensor readings. A sensor reading may include a sensor generated value of a variable. Combining of values of variables may include aggregating and mathematically processing the sensor generated values of variables.
As illustrated, a wearable item 251-258 may include, by way of non-limiting example, headgear 251, such as a hat, hardhat, headband, skull cap, cap, A/R headset, V/R headset, surgical cap, scrub cap; an earring 252; an wrist strap 253, including an arm band, a watch, a bracelet, or other item that may be positioned around an arm; a torso covering 254, such as a vest, lab coat, scrub; hazmat suit, smock; a ring 255, footwear 256, such as a shoe, boot, sandal, slipper, cleat, sneaker, or other article fitted to a foot. As illustrated, each of the wearable items illustrated 251-258 has a Transceivers 204-212 associated with it. The Transceivers 204-212 may be fixedly attached, removably attached or incorporated into the wearable item.
In some embodiments, a wearable item 251-258, such as for example a ring 255, an earring 252, or any of the other wearable items may include a transceiver that engages in low power, short distance communication, such as for example, one or more of: LPWAN, WhiskerRF, ANT, Near Field Communications (NFC), Zigby, Bluetooth Low Energy, or other low energy type communication. The wearable item may then communicate sensor data to a higher powered smart device 239, such as one or more of: a communications Tag, a Node, a Smart Phone, a Smart headset, a smart watch and the like, wherein the higher powered smart device 239 may engage in wireless communication with one or more Reference Point Transceivers 202A-202D to determine a position of a person 201-201A and also convey through the wireless communications some or all of the digital content generated by sensors 214-236 included with the wearable item 251-258.
For example, in some embodiments a higher powered smart device 239 may receive from the lower powered sensor multiple values of a variable quantifying a condition present in a structure 200, each value of the variable quantifying a condition present in the structure associated with a chronological time value. Variables, and values for variables will depend upon a type of sensor made operative to quantify a condition. Sensors may be chosen based upon which conditions will be quantified. Similarly, a transceiver 202A-213 may vary based upon an environment in which the transceiver will be deployed.
Sensors 214-236 may include an accelerometer to quantify motion (movement, acceleration); magnetic sensor, which may include magnetic field sensors, Hall Effect Sensors and/or Anisotropic Magneto-Resistive (AMR) sensors, magnetometer (3-axis); ambient temperature sensor; ambient light sensor; specific wavelength of light sensor; viii) ambient humidity; Magnet detector (maintenance button, “docked” indication, etc.); ix) sound pressure level (SPL); optical proximity sensor; power source such as a rechargeable lithium polymer battery; USB connector for charging and/or data transfer; NFC (near field communications) (tag, or reader); Battery voltage sensing; audio alert, such as a Beeper; Vibrator, LED indicator(s); Manual switch such as a tactile button; Inductive charging; vibration detection; air quality sensing such as CO2, VOC, and the like; specific gas detection (CO, CO2, H2S, particulate); Gesture sensing (sense hand swipes over device); tap and double tap detection; Agent fail detection; free-fail detection; shock and/or impact detection; IR (infrared) beacon TX and/or RX; pedometer; activity meter; processor enhanced audio monitoring (e.g. listen tier fire alarm siren); heartrate monitor; blood oxygen sensing; OLED display; electronic ink display; and most any other electronic sensor of a quantifiable condition.
By way of non-limiting example Transceivers 202-212 may be operative to perform wireless communications suitable for real time location functionality and/or data transfer, transmitting and receiving capabilities using one or more modalities such as: UWB; Bluetooth, including BLE; WiFi RTT; infrared; ultrasonic; sonic; sub GHz; or other modality.
In some embodiments, communication between transceiver, such as those included in Tags, Nodes and/or Smart Devices may be achieved by a burst transmission protocol, such as low-power derivative of Gaussian pulses. In exemplary embodiments, the burst transmission protocol may transmit radio waves having a frequency of 3.0 GHz-11.0 GHz, with a high bandwidth of approximately 500 MHz-1.3 GHz, in some embodiments, the bandwidth may be based on a tolerance from an arithmetic central frequency; for example, if the central frequency for the burst transmissions is 6.0 GHz, then the bandwidth may be a relative tolerance, such as 20% of 6.0 GHz (i.e., 1.2 GHz). An interval between individual pulses may be uniform or variable, based on an implemented encoding scheme. In some exemplary embodiments, burst transmissions may lack carrier waves. Transceiving modalities that may be implemented using burst transmission protocol according to the present invention, include, ultra wideband, sub-GHz, and industrial, scientific and medical (“ISM”) radio bands within the frequency range of 6.7 MHz (megahertz) and 250 GHz.
In sonic exemplary embodiments, one or more pulses of wireless communication may be relatively short in duration (i.e., 0.01 ns-10 ns). The total pulse train may be written as the sum of individual, time-shifted pulses; i.e., s(t)=Σ_(n=0){circumflex over ( )}∞a_np(t−τ_n)
where s(t) is the pulse train signal, p(t) is a characteristic pulse shape, an is the amplitude of the nth pulse, and τn is the time-offset of the nth pulse.
When two nodes exchange burst transmissions, they may be able to detect a range or distance between the two nodes. This may be accomplished through a “time of flight” (ToF) measurement. The ToF measurement may be made by or supplemented with a time difference of arrival (TDOA) or time delay estimation (TDE). TDOA may rely on a measurement of the amount of time it takes for a signal transmitted by one node to reach a plurality of other nodes. Using multilateration techniques, a location of the transmitting node may be estimated.
Referring now to
Each wearable item 251-256 may include at least one wireless transceiver 204-212 and one or more sensors 213-236. As discussed herein, the wireless transceiver 204-212 may include a solid state components operable to transmit and/or receive wireless communications. The wireless communications may utilize any of the modalities discussed herein. The transceivers may wirelessly communicate with one or more Reference Point Transceivers 202A-202D and/or a smart device 239, such as a smart phone, smart tablet, smart headgear and the like. Wireless communications may include relatively higher powered and longer range modality, such as UWB, Bluetooth, WiFi, cellular or satellite communications and/or lower powered and shorter range communication modality, such as Zigby, ANT, BLE, or other low power protocol.
Those wearable items 251-256 that are capable of supporting more than one wireless transceiver may be operative to engage in wireless communications with Reference Point Transceivers 202A-202D and generate positional coordinates for each transceiver 204-212 (as illustrated wearable items 251, 254, 256 include multiple transceivers, but any wearable item with a large enough area may include multiple transceivers 204-212). Two or more sets of positional coordinates may be referenced to generate a direction of interest. In the present invention a direction of interest may include a direction a person 201-201A is facing. Accordingly, a headgear 251, a vest 254 and/or a shoe 256 may include multiple transceivers 211A-211D, 207-209 and 204-204A respectively for which corresponding positional coordinates may be generated based upon values of variables transceived according to the methods an apparatus described herein. The multiple positional coordinates may be used in turn to generate a direction of interest, such as, a forward facing direction. A forward facing direction may be important in certain safety situations, such as in the case of a contagion that may be spread by a cough, sneeze of exhale. It may be important to know a separation distance 203 between a first person 201 to a second person 201A in the forward direction. A separation distance 203 in a rearward direction may not have as short a critical distance.
In some embodiments, Sensors 213-236 are hardwired or otherwise in physical logical communication with connected to transceivers 204-212 included in a same wearable device. In physical logical communication may include, by way of non-limiting example, on a same printed circuit board (PCB); in a same semiconductor chip; connected by a circuit tracer or wired connection; or other physical logic pathway capable of conveying a digital value.
Transceivers 213-236 may be operative to engage in wireless communication with one or more of: Reference Point Transceivers 202A-202D; a smart device 239, such as a Smart Phone, Smart Tablet or Smart Headgear; a cellular network; a satellite network; a mesh network or other logical network.
Referring now to
Ongoing determination of a location 263-264 of an Agent, such as a person 201-201A may also be used to ensure that the person 201-201 A does not traverse and area designated as a contaminated area, such as a raw food area, or a chemically contaminated area, and then proceed to a controlled area, such as a non-contaminated area, such as an area with food or other product ready for distribution to the public.
Devices involved in the generation and/or storage and/or transmitting of digital content, such as, one or more of: agent supported transceivers 207-208, reference point transceivers 202A-202D, gateways 270, and servers 271 may each engage in direct logical communications and/or in logical communications with an intermediary device For example, a reference point transceiver 202A-202D may engage in logical communication with a gateway 270 or other intermediary device and the gateway 270 may engage in logical communication 269 with a server 271; or the reference point transceiver 202A-202D may engage in logical communication 267 directly with the server 271. There may be numerous paths of logical communication 265-269 between various device types. In various embodiments of the present invention, any or all of the devices (e.g. sensors, transceivers, gateways, servers) involved in generation and/or storage and/or transmitting of digital content, may aggregate, store and or process digital content that the device has access to.
Also, variations on a location of a sensor and a transceiver are within the scope of the present invention. As discussed above, a Sensor 204-213 may be located on a wearable item 251-258 or a sensor 213 may be positioned at a point extraneous to a person 201-201A, such as mounted on an architectural aspect (such as, for example, a doorway, a hall entrance, a turnstile or other crowd movement control device; and receive sensor input 242 from a person 201 useful to quantify a condition of the person 201. Transceivers 207-208 may determine a location of a person 201-201A that correlates with a sensor 213 quantification of a condition present with the person 201-201A based upon input 242 received from the person 201-201A. Based upon a location 263 of a person 201, a processor (such as those present in a gateway 270, server 271 or smart device 239) which person is associated with a condition quantified by the sensor 213 mounted on an architectural aspect.
Some embodiments may also include a sensor 213 capable of receiving input 242 sufficient to quantify biometric based identification of the person 201, and/or a physiological state present in the person 201. For example, sensor 213 may receive input enabling facial recognition of the person 201 and correlate input from a sensor 204-213 to quantify a physiological condition and a facial recognition identification. Transceivers 207-208 may engage in wireless communications sufficient to determine a location 263 of the person 201 when the physiological condition and a facial recognition identification took place via other sensors than those co-located with the transceivers supported by the Agent.
Some modalities, such as those modalities that adhere to the Bluetooth 5.1 or BL5.1 standards, allow a Node to determine an angle of arrival (AoA) or an angle of departure (AoD) for a wireless transmission. An array of antennas may be used to measure aspects of the Bluetooth signaling that may be useful to calculate these AoA and AoD parameters. By calibrating an antenna system, the system may be used to determine angles in one or two dimensions depending on the design of the antenna. The result may be significant improvement in pinpointing the location of origin of to signal.
An array of antennas may be positioned relative to each other and a transmitting transceiver to allow for extraction of an AoA/AoD. Such an as may include a rectangular array; a polar or circular array; a linear array; and a patterned array, where a number of antennas are deployed in a pattern conducive to a particular environment for transceiving. Antennas may be separated by characterized distances from each other, and in some examples, a training protocol for the antenna array results in antenna positioning incorporating superior angle and location precision, Some transceivers may transceive in 2.4-2.482 GHz frequency bands, and thus the radiofrequency transmissions may have wavelengths in the roughly 125 mm as length scale. A collection of antennas separated by significantly less than the wavelength may function by comparing a phase of RF transmissions arriving at the antennas. An accurate extraction of phase differences can yield a difference in path length that when accumulated can lead to a solution for the angles involved. In some embodiments, Nodes may include antenna arrays combined with batteries and circuitry to form complete self-contained devices. Antenna arrays and methods of using the same for determining position and direction of a Smart Device or other Node are described in U.S. Ser. No. 16/775,223, the contents of which are incorporated herein by reference.
Referring now to
The recording of data representing the interactions between agents and items in cold storage could have numerous important functions. For example, the various sensors on the agent could store data relevant to the health status of the agent. The health status indicators can be coupled to the position and orientation information to indicate if it is likely that the agent came sufficiently close to an item in cold storage to effect it is some manners.
The physical item 300 in a cold storage location may also record ambient conditions such as temperature, particulate level and air pressures. Variations in the measurement of temperature may form a record that could be used to disposition the integrity of the physical item kept in cold storage. The sensor data may also be useful in determining likelihood of contamination events between the agent and the physical item. More generally, the sensors on the physical item 300 may record ambient condition data and communicate that with the controlling systems of the cold storage location.
Retelling now to
In some embodiments, the vector may have a length determined by a controller that is based upon a distance to a feature of interest in space as represented in a model on the controller in the direction of the generated vector. Tile vector may represent a distance 353 from the second position 352 to the space of interest 375 along the axis defined by a line between the first position 351 and the second position 352. In contrast, a ray may include just a starting point and a direction.
In still other embodiments, a device with a controller and an accelerometer, such as mobile phone, tablet or other Smart Device that includes a Transceiver 355, may include a user display that allows a direction to be indicated by movement of the device from a determined location acting as a base position towards an point in a direction of interest or representing a center of an RTA of the device. The movement may occur to a second location in an extended position. In some implementations, the Smart Device determines a first position 351 based upon triangulation with the reference points. The process of determination of a position based upon triangulation with the reference points may be accomplished, for example via executable software interacting with a controller in the Smart Device, such as, for example by running an app on the Transceiver 355.
An array of antennas positioned at a user reference point may allow for the accurate receipt of orientation information from a transmitter. As discussed earlier a combination devices with arrays of antennas may be used to calculation a position. A single Node with an array of antennas can be used for orienteering and determining a direction of interest. Each of the antennas in such an array receiving a signal from a source may have different phase aspects of the received signal at the antennas due to different distances that the emitted signal passes through. The phase differences can be turned into a computed angle that the source makes with the antenna array.
Referring to
In some embodiments, see
An array of antennas positioned at a reference point may allow for the accurate receipt of orientation information from a transmitter. As discussed earlier a combination devices with arrays of antennas may be used to calculation a position. A single Node with an array of antennas can be used for orienteering and determining a direction of interest. Each of the antennas in such an array receiving a signal from a source will have different phase aspects of the received signal at the antennas due to different distances that the emitted signal passes through. The phase differences can be turned into a computed angle that the source makes with the antenna array.
Referring now to
Referring now to
Referring to
Referring now to
In some preferred embodiments, the Smart Device will be fixedly attached to a smart receptacle 802. The smart receptacle 802 may include an appearance of a passive case, such as the type typically used to protect the Smart Device 801 from a damaging impact. However, according to the present invention, the smart receptacle 802 will include digital and/or analog logical components, such as wireless position devices 803-810. The wireless position devices 803-810 include circuitry capable of receiving wireless transmissions from multiple wireless Reference Position Transceivers 811-814. The wireless transmissions will I tide one or both of analog and digital data suitable for calculating a distance from each respective reference point 811-814.
In some embodiments, the smart receptacle 802 will include a connector 815 for creating an electrical path for carrying one or both of electrical power and logic signals between the Smart Device 801 and the smart receptacle 802, For example, the connector 815 may include a mini-USB connector or a lightning connector. Additional embodiments may include an inductive coil arrangement for transferring power.
Embodiments may also include wireless transmitters and receivers to provide logical communication between the wireless position devices 803-810 and the Smart Device 801. Logical communication may be accomplished, for example, via one or more of: UWB, Bluetooth, ANT, and infrared media.
Reference Position Transceivers 811-814 provide wireless transmissions of data that may be received by wireless position devices 803-810. The wireless transmissions are utilized to generate a position of the respective wireless position devices 803-810 in relation to the Reference Position Transceivers 811-814 providing the wireless transmissions to the wireless position devices 803-840. The wireless position devices 803-810 are associated with one or more of: a position in a virtual model; a geographic position; a geospatial position in a defined area, such as Structure; and a geospatial position within a defined area (such as, for example a Property).
According to the present invention, a Smart Device may be placed into a case, such as a smart receptacle 802 that includes two or more wireless position devices 803-810. The wireless position devices 803-810 may include, for example, one or both of: a receiver and a transmitter, in logical communication with an antenna configured to communicate with Reference Position Transceivers 811-814. Communications relevant to location determination may include, for example, one or more of: timing signals; SIM information; received signal strength; GPS data; raw radio measurements; Cell-ID; round trip time of a signal; phase; and angle of received/transmitted signal: time of arrival of a signal; a time difference of arrival; and other data useful in determining a location.
The Nodes 803-810 may be located strategically in the case 802 to provide intuitive direction to a user holding the case 802, and also to provide a most accurate determination of direction. Accordingly, a forward Node 803 may be placed at a top of a Smart Device case and a rearward Node 804 may be placed at a bottom of a Smart Device case 802. Some embodiments each of four corners of a case may include a Node 805, 806, 807, 808. Still other embodiments may include a Node 609 and 810 on each lateral side.
Referring now to
A Smart Device 901A with integrated Nodes 903A-910A and a Smart Device 901A with Nodes 903A-910A in a smart receptacle 902A may provide a directional indication, such as a directional Vector 917A, without needing to move the Smart Device from a first position to a second position.
Referring now to
Referring to
In some embodiments of the invention, determining the location of objects in a Cold Storage Facility may benefit from an Augmented Virtual Model. The present invention references prior applications and issued patents owned by the applicant relating to automated apparatus and methods for generating improved Augmented Virtual Models (sometimes referred to herein as an “AVM”) of a Structure. An AVM may include As Built elements, including the locations of Reference Position Transceivers and items of interest to which an Agent may wish to be guided by methods and apparatus described herein.
An AVM may include original design data matched to As Built data captured via highly accurate geolocation, direction and elevation determination. As Built data is matched with a time and date of data acquisition and presented in two- and three-dimensional visual representations of the Property. The augmented models additionally include data relating to features specified in a Property design and data collected during building, deployment, maintenance, and modifications to the Property. In some embodiments, a fourth dimension of time may also be included. As a result, an Employee or other Agent may be guided to, for example, a former location of an item of interest.
The experience of the physical Structure is duplicated in the virtual Augmented Virtual Model, The AVM may commence via an electronic model generated via traditional CAD software or other design type software. In addition, the AVM may be based upon values for variables, including one or more of: usage of a Structure; usage of components within the Structure; environmental factors encountered during a build stage or deployment stage; and metrics related to Performance of the Structure. The metrics may be determined, for example, via measurements performed by sensors located in and proximate to Structures located on the Property.
Referring now to
Similarly, a direction may be calculated via a prescribed movement of the Smart Device 1206 during execution of code that will record a change in position relative to the Reference Position Transceivers 1224A. For example, a user Smart Device 1206, such as a smart phone or tablet may be directed towards a wall or other Structure portion and upon execution of executable code, the Smart Device may be moved in a generally perpendicular direction towards the wall. The change in direction of the user device 1206 relative to the Reference Position Transceivers 1221A may be used to calculate a direction. Based upon a recorded position within the Structure 1202A and the calculated direction, a data record may be accessed in the AVM 1200 and a specific portion of the AVM 1200 and/or the virtual structure 1202B may be presented on the Smart Device 1206. In other embodiments, a direction may be chosen, or verified via a mechanism internal to the Smart Device 1206, such as a compass or accelerometer.
Embodiments may include models generated using, for example, standard modeling software such as BIM 360™ field which may support the display of a Structure design in a very complete level of detail. Additional events for scanning may occur during the construction process to capture accurate, three-dimensional as-built point-cloud information. Point cloud may include an array of points determined from image capture and/or laser scanning or other data collection technique of as-built features. In some examples, captured data may be converted into a 3D model, and saved within a cloud-based data platform.
In some examples, other methods of capturing spatially accurate information may include the use of drones and optical scanning techniques, which may include high-resolution imagery obtained from multiple viewpoints. Scanning may be performed with light-based methods such as a CCD camera or LIDAR. Other methods may include infrared, ultraviolet, acoustic, and magnetic and electric-field mapping techniques may be utilized.
Structure-related information may include physical features generally associated with an exterior of a Structure such as geolocation, elevation, surrounding trees and large landscaping features, underground utility locations (such as power, water, sewer, sprinkler system, and many other possible underground utility features), paving, and pool or patio areas. Structure-related information mini also include features generally related to a Structure such as underground plumbing locations, stud locations, electrical conduit and wiring, vertical plumbing piping, and HVAC systems or other duct work. The acquisition of the data may allow the model system to accurately locate these interior and exterior features. Acquisition of as-built data during different points of the construction completion allows measurements to be taken prior to aspects involved in a measurement process being concealed by concrete, drywall or other various building materials.
Referring to
Referring to
Referring to
Referring to
Referring to
In some non-limiting examples of a wireless Node, a Bluetooth communications hub compatible with a standard such as, for example BLE5.1 (Bluetooth Low Energy 5.1) or Wi-Fi RTT may be fixedly attached to a structural component, such as a door header 1336 as Node 1350 acting as a Reference Position Transceiver. In another example, a Node 1351 may act as a Reference Position Transceiver and be attached to a wall stud, preferentially one that has electrical conduit 1335 running along it. In some embodiments, the electrical conduit 1335 may supply power to the Node 1351. Alternatively, a Node 1352 may be configured as part of a receptacle box. In some examples, one or more Nodes 1350-1351 may be battery powered. One or more Nodes 1350-1351 may be powered via electrical supply wiring 1353 from a nearby power conduit 1335 so that the Node 1350-1351 may be tied into a centrally powered electrical system. Moreover, the Nodes may be adapted to de-power and de-couple from a network based on a power supply status or a power drain change.
Referring to
With the methods and apparatus described above, the location of an individual with a Smart Device or other Node may be determined. Additionally, the methods and apparatus may be used to determine the individual's direction of interest. Similarly, the location of any object tagged with a transceiver, Node, or Smart Device may be determined, whether stationary or in transit. In a Cold Storage Facility, equipment, sensors, pallets, transports, and employees may hold or be “tagged” with a transceiver, Node, or Smart Device, and each location may be transmitted to monitoring system and/or identified on a user display to monitor the location of each item. Additionally, the item or persons may transmit to a system information about the item/person, either automatically or when queried. For example, the sensors may transmit location and also data for the sensor over any desired time period. Equipment and sensors may be able to transmit information about maintenance history or part numbers. Movement of pallets and transports may be tracked throughout the Facility. Employee movements may also be tracked and information about the Employee, including hours, authorizations, and the like, may be transmitted to a monitoring system. These are non-limiting examples and other types of information may be transmitted from the Node or Smart Device to a monitoring system.
In some cases, it may be prohibitively inefficient or expensive to “tag” or associate a Node or Smart Device with every item to be monitored. For example, it may be cost and time prohibitive to assign each product being stored in the Facility with a Node or Smart Device. As such, other methods of monitoring the products and other items may be used. In some embodiments of the invention, an item may be assigned a “Virtual Tag,” in which a Smart Device can identify an object in its field of view and capture its location coordinates based on the position of the Smart Device. A “Virtual Tag” or “icon” may be generated in the Smart Device's field of view at those coordinates and saved in the digital memory. Information regarding the object may also be saved with the spatial coordinates. This information can be transmitted to the monitoring system so that others may find the object by routing to the particular coordinates. In addition, when that Smart Device, or another Smart Device in logical communication with the monitoring system, views the particular location, the Smart Device will identify that a virtual tag exists at that location with, for example, an icon. The icon may also be interactive so that information associated with the object may be known. Thus, the monitoring system may be in logical communication with both physical tags such as Smart Devices or Nodes and with Virtual Tags created by a Smart Device. This will be described in further detail below.
Creating of Virtual Tags may, in sonic embodiments, performed using the following steps, Referring to
At step 1407, the position of any Nodes or Reference Point Transceivers is verified, and in step 1408, the direction of the Image Capture Device is determined based on the methods described herein, using the Smart Device and any Nodes or Reference Point Transceivers nearby.
At step 1409, the Image Capture Device captures an image. This may be done by capture of wireless energy of a first wavelength is received into a wireless receiver in the Smart Device. In exemplary embodiments, this step may comprise receiving image data based on visible light into a camera of the smart device. The wireless energy may be dispersed over a one-, two-, or three-dimensional space in a defined physical area, and may be received into a one-, two-, or three-dimensional array in the receiver. The wireless energy may take the form of electromagnetic radiation, such as light in the human-visible light spectrum (generally having a wavelength between 380 nm-740 nm), ultraviolet light (generally having a wavelength between 10.0 nm-400 nm), or infrared light (generally having a wavelength between 740 nm-2.00 mm). The set of wireless energy available to the wireless receiver is the smart device's Field of View. In step 1410, the Field of View may change as the angle or position of the Smart Device changes.
In step 1411, the Field of View is quantified. In some embodiments, a pattern of digital values is generated based upon receipt of wireless energy into the wireless receiver. This pattern of digital values may be based on one or more qualities of the received wireless energy, including its intensity, spatial dispersion, wavelength, or angle of arrival. The pattern may be placed into an appropriate array. For example, if the display of the smart device is a two-dimensional display, then the pattern of digital values may comprise a two-dimensional representation of the image data received. In sonic embodiments, the pattern of digital values may be based on an aggregated set of values from an array of receivers. For example, if the basis of the digital values is the intensity of the wireless energy received into the receiver, then the digital value assigned to a given entry in the array may be based on a weighted average of intensity of wireless energy received at a plurality of the receivers in the array. The set of digital values within the Field of View is the Digital Field of View. In step 1412, optionally, the coordinates of the space in the Digital Field of View may also be determined. Any suitable method may be used, but in some embodiments, LIDAR may be used to determine the presence and distance from the Smart Device of various items in the Field of View and therefore determine the position of items. In step 1413, items or space in the Field of View may be “tagged” such as via a touch screen or cursor. Using similar steps for step 1412, the location coordinates for the Virtual Tags may be known and transmitted, for example, to a monitoring system or controller.
Referring to step 1414 when a Smart Device that is part of the monitoring system approaches the location of the Virtual Tag, the Smart Device will identify that the Virtual Tag is within the Smart Device's Field of View. In step 1415, When this occurs, the Smart Device will generate an icon in the position of the Virtual Tag. This icon may be interactive so that information regarding the location or Virtual Tag may be ascertained. For example, the Virtual Tag may provide a location and environmental history of the product at the coordinates, as will be discussed in further detail below.
Next, additional details regarding how the Field of View may be determined will be discussed. Referring now to
As discussed extensively herein, the location of the Smart Device 1501 is determined based upon wireless coordinates to and/or from the Smart Device 1501; and described via a coordinate system, such as Cartesian coordinates, or other X,Y,Z coordinates, polar coordinates, spherical coordinates, and cylindrical coordinates. Aspects of wireless communications that may be referenced to generate location coordinates may include one or more of: RTT (round trip time), time of flight, RSSI (received signal strength indicator); angle of an angle of departure, and other aspects described herein.
With the location of the Smart Device 1501 determined, a location of the WCA 1502 may be extrapolated based upon the location of the Smart Device and a transceiving distance the Smart Device is capable of. According to the present invention, a portion of the WCA 1512 is selected as a radio target area (RTA) 1512 from which the Smart Device 1501 may receive specific bandwidths of electromagnetic radiation. In preferred embodiments, the RTA 1512 will include a frustum expanding outward in a conical shape from one or more energy receivers 1509 included in the Smart Device 1501. The frustum Shaped RTA 1512 will overlap a portion of the WCA 1502 sphere. Other shapes for a radio target area 1502 are also within the scope of the invention. In some embodiments, a shape of the RTA 1512 will be based upon receiving capabilities of the one or more energy receivers 1509. For example, an energy receiver 1502 with a charge coupled device (CCD) or complementary metal oxide semiconductor (CMOS) receiver may have a single plane receiving surface and be best matched with a frustum of a generally conical shape, an energy receiver 1509 with multiple receiving surfaces (e.g. with multiple CCD and/or CMOS devices) may be arranged to enable a more complex shaped RTA 1512.
In some preferred embodiments, a direction of interest 1511 win intersect the RTA 1512. As discussed herein, the direction of interest 1512 may be represented by a ray or vector 1511. In addition, the direction of interest may be represented as a direction of interest area, such as a frustum 1511C defined by multiple rays or vectors 1511, 1511A, 1511B. In various embodiments, the direction of interest 1511 area may encompass the RTA 1512 or be a subset of the RTA 1512.
A direction of interest may be determined for example via the methods and devices described herein and may be associated with a direction based upon a ray or vector indicative of a direction of interest 1511, a direction based upon a magnetic field sensor, an accelerometer, a light beam, correlation between two tags or nodes, Agent gestures, or other Smart Device recognized apparatus and/or method.
One or more transceivers 1503-1504 (typically included within a Smart Device, Tag, or Node) will be located within an area defined by the RTA 1512. According to the present invention, a position of the transceiver 1503-04 will be determined and a user interactive mechanism will be generated at a position of the transceiver 1503-04 within a graphical user interface emulating aspects of the RTA 1512 on the Smart Device 1501 or another user interactive interface screen (not shown, and maybe at a site remote to the RTA 1512).
According to the present invention, some portion of the RTA 1512 (which may include the entirety of the RTA 1512) may be portrayed on an Agent interface 1510, including, in some embodiments, a human readable graphical user interface (GUI), The interface 1510 may include a representation of a particular level of electromagnetic energy received via the energy receiver 1509 and associated with a particular area of the RTA 1512. For example, energy levels of an infrared wavelength that has emanated from or reflected off of an item in the WTA 1512 and received via an infrared receiver in the Smart Device 1512 may be used to generate a heat map type interface display, similarly energy that has emanated from or reflected off of an item in the RTA 1512 in the 400 nM to 700 nM range and been received via a charged couple device in the Smart Device 1501 may be portrayed as a human visible image of items in the area included in the RTA 1512. Other embodiments may include a point cloud derived from electromagnetic energy bouncing off of or emanating from items included in the RTA 1512 or a series of polygons generated based upon a LIDAR receiver in the Smart Device 1512. An Agent interface 1510 may be presented in a modality understandable to an Agent type. For example, an interface presented to a UAV air UGV may include a digital pattern and an interface presented to a human Agent may include multiple pixels or voxels generating a pattern visible to a human being.
The wireless location methods and apparatus described herein may be deployed in conjunctions with one or more Tags and/or Nodes 1503-08 located with the WCA 1502 to generate location coordinates for the Tags and/or Nodes 1503-08. A controller or other device operating a processor may determine which Tags and/or Nodes 1503-08 are located within the three dimensional space included in the RTA 1512 based upon a) the location of the Tags and/or Nodes 1503-08; and b) the location of area included in the RTA 1512.
In another aspect of the present invention, in some embodiments, some energy levels may not be represented in the Agent interface 1510. For example, in some embodiments, energy levels reflected off of a particular item may not be included in the Agent interface 1510 which would a simulation that the item is not within the RTA 1512. Other embodiments may only represent energy levels that have reflected off of selected items within the RTA 1512 thereby emphasizing the presence of the selected items and ignoring the presence of other items within the RTA 1512.
In some embodiments of the invention, sensors throughout the facility may be virtually tagged via a Virtual Tag method described herein or from an AVM using As Built data. As such, throughout the facility, when a Smart Device views the sensor, an icon may be generated. When a user interacts with the icon, the icon may generate information about the sensor or equipment. For example, the following equipment could be virtually tagged: refrigeration compressors; refrigeration pumps; refrigeration valve groups; refrigeration evaporative condensers and fan coil units; air compressors; boilers; and electrical panels. Examples of sensors that could be virtually tagged include sensors that detect temperature, humidity, air flow, current/amperage, pressure, water flow, water sensor, and ammonia levels. When icons associated with such equipment or sensors are generated, a user may interact with the icon, and information about the items may be displayed on the smart device. For example, when touched by a user, the icon may bring up a menu of the information about the equipment or sensor, including a user's manual, parts list, maintenance history, and/or history of data generated by the sensor or equipment over a certain time period. The icons may also provide an indication or warning if there is a problem with the equipment or sensor.
As a non-limiting example, ammonia sensors may be installed in various locations in the cold storage facility, including the refrigeration machine room, along the ammonia piping, near valve groups, at the fan coils, within ductwork (if applicable), and the like. A smart device may provide an AVM of the facility that identifies each ammonia sensor, and if the icon for each sensor is opened, information about the particular sensor may be obtained, including the ammonia detection over time (for a predetermined time period), maintenance history for the sensor, part numbers, and user manual. In addition, if an ammonia sensor shows a reading outside a predefined ranged deemed acceptable, a warning or alert may be generated and the Agent/Employee may be notified. In addition, the icon itself may generate a label identifying the sensor with the readings outside the predefined concentration range. Additionally, the icon may allow for various alerts to be generated on the icon so that the urgency of the problem may be understood by the Employee. For example, different colors may denote different levels of urgency, such as yellow denoting a mild increase about the predefined concentration range, orange denoting a higher concentration, and red indicating a dangerously high level of ammonia detected. Of course, any color combination may be used, and other methods of identifying the urgency with the problem may be used including increase in intensity of an icon, increase in sound or frequency of an alert, or a flashing icon (with an increase in flashing frequency indicating increasing urgency). Additionally, the orienteering methods may allow for an Agent to locate the problematic sensor, either via the smart device or a heads up display associated therewith.
Similarly, this technology may be useful with detectors for carbon monoxide, fire protection leaks, fire protection water tank storage levels and/or water quality, and hazardous materials in areas such as chemical storage rooms. Many other types of sensors and processes can be monitored with using smart devices by such methods. In addition to sensors and equipment in the facility, virtual tags may allow for an Employee to locate key underground features in U/G piping, stormwater, process waste, sanitary, etc., such as turns in pipes, lift stations, manholes, etc. This would allow for isolation of the 3D model of the underground systems and display using augmented reality using a smart device or head's up display and using the location sensors to “anchor” the model, so that it's always accurate.
Another example is the use of virtual or physical tags for moisture detection, such as moisture sensors in duct and pipe insulation (underside, low points, etc.), including those installed indoors or outside, that are subject to condensation and leaks. Based on active ‘warnings’ from the smart device, an Agent can locate the condensation issues using the orienteering methods described herein using a smart device or head's up display.
Referring now to
According to the present invention, a WCA and RTA may be used to provide an augmented reality user interface indicating a location of various persons within the WCA and RTA and indications of sensor readings related to those persons as well as links to further information.
Referring now to
As illustrated, a portion 1601a of the RTA 1601 may flatten out in response to a ground plane, wall, partition, or other obstruction encountered. A Node 1606 may be located on or within a surface that makes up a relevant obstruction and the Node 1606 may appear to be along a perimeter of the RTA 1601. Similarly, a Virtual Tag may be associated with location coordinates that appear on or within a floor, wall, partition, or other article acting as a radio frequency obstruction and thereby appear to be a part of the obstruction, however, since it is virtual, the Virtual Tag will not effect the physical properties of the obstruction. Essentially, a Virtual Tag may have location coordinates that correspond to anywhere in the physical real-world. In some examples, a software limit or setting may limit location coordinates of Virtual Tags to some distance from a base position or a distance from a designated position, such as a location of a designated Physical Tag, Reference Point Transceiver or other definable position.
In addition to obstructions, a topography of an environment within an RTA 1601 may also limit wireless conveyance of energy within an RTA 1601 to an energy-receiving Sensor 1604. Topography artifacts may include, for example, a terrain, buildings, infrastructure, machinery, shelving or other items and/or other structures that may create impediments to the receipt of wireless energy.
Energy received 1603 into the energy-receiving Sensor 1604 may be used to create aspects of a user interface that is descriptive of the environment within the RTA 1601. According to the present invention, environmental aspects, Nodes 1606, Tags (both physical Tags and Virtual Tags) and the like may be combined with user interactive mechanisms, such as switches or other control devices built into a user interactive device, and included in a user interactive interface. For example, energy levels received into an energy-receiving Sensor 1604 may be combined with location coordinates of Physical Tags and/or Virtual Tags and a user interactive device may be positioned in an interactive user interface at a position correlating with the position coordinates and be surrounded with a visual indicator or the received energy levels.
In this manner, a single user interface will include a static image representative of received energy levels at an instance in time; a visual representation of a location(s) of Physical and/or Virtual Tag(s), and devices with user interactive functionality. In sonic embodiments, the devices with user interactive functionality may be positioned at a location in the user interactive interface correlating with the position(s) of the Physical and/or Virtual Tag(s).
This disclosure will discuss RTAs 1601 that are frustums of a generally conical shape, however, RTAs 1601 of other volume shapes are within the scope of the invention. For example, if an energy-receiving Sensor 1604 included a receiving surface that was a shape other than round, or had multiple receiving surfaces, each of a round or other Shape, the RTA 1601 associated with such an energy-receiving Sensor 1601 may have a shape other than a frustum of generally conical shape.
Referring now to
Referring now to
Some embodiments include a RTA 1607 that varies according to a type of energy receiving device 1604 receiving a corresponding type of energy. For example, an energy-receiving Sensor 1604 that receives energy in a lower bandwidth may have an RTA 1607 that extends a greater distance than an energy-receiving Sensor 1604 that receives energy in a higher bandwidth. Similarly, some energy-receiving Sensors 1604 may be effected by farces outside of the RTA 1607, such as a magnetometer which may be sensitive to signal interactions around all of the WCA 1600, and a RTA 1607 associated with a magnetometer may accordingly be the same as the WCA 1600.
By way of non-limiting example, a RTA 1607 for a CCD-type energy receiver may be represented as a frustum with an expansion angle of approximately 60 degrees in shape. Accordingly, the RTA 1607 subtends only a portion of the universal WCA 1620.
Referring now to
In
A user interface based upon quantified energy in an RTA 1607, 1607A, may present a representation of energy within the respective RTA 1607, 1607A as quantified by an energy-receiving Sensor 1602. Energy levels of other three-dimensional spaces within the WCA 1600 may be quantified by energy receivers and presented in a user interface by directing energy from a selected three-dimensional space into the energy receivers and thereby defining a different RTA. In this manner, energy levels may be quantified from essentially any area within the WCA 1620 and represented as part of a user interface. Quantified energy levels may vary based upon a receiving Sensor. For example, a CCD Sensor may quantify visible light spectrum energy, and a LIDAR receiver a broad spectrum, an infrared receiver may quantify infrared energy levels, and energy-receiving Sensors. A particular feature present in a particular portion of the electromagnetic spectrum quantified by an energy-receiving Sensor may have a unique physical shape which characterizes it and which may be associated with a corresponding virtual-world aspect and Tag associated with the location.
In some examples, as has been described, quantification of energy levels associated with aspects of the physical world may be for one or more of: characterizing an RTA 1607, 1607A by quantifying energy levels and patterns existing at an instance in time, determining a location and/or orientation of a Smart Device 1602 or other Node, such as node 1606; and verifying a location and/or orientation of a Smart Device. In some examples, energy levels associated with aspects of the physical world may be communicated by the Smart Device to a remote controller for further processing, and the remote controller may communicate information back to the Smart Device or to another user interface. Information communicated from the controller may include, for example, an orientation of physical and/or virtual aspects located within the universal RTA in relation to the Smart Device; quantified energy indicating of or more of: a topographical feature, a surface temperature, a vibration level, information associated with a Virtual Tag, information associated with a physical Tag, sensor data, or other information associated with the RTA 1607A.
A view of a Radio Target Area 1607A may be a relatively small portion of the entire WCA that surrounds a Smart Device. An area of energy to be quantified by a sensor (sometimes referred to herein as the Radio Target Area) may be displayed surrounded by the WCA 1630.
Referring now to
Referring to
Referring to
A location of a real-world Tag or Virtual Tag may be in either RTA 1661, the WCA 1641 or external to both the RTA 1661 and the WCA 1641. Examples of Tags outside the RTA 1661 and within the WCA 1641 include Tags 1662-1666. An example of a Tag in the device RTA is Tag 1661. A Tag located external to of the WCA 1641 and the RTA 1661 includes Tag 1667.
In some examples, a display on the user's Smart Device may illustrate image data captured via a CCD included in a Smart Device. Portions of the image data captured via a CCD may be removed and replaced with an icon at position correlating to a position in space within the RTA 1661. The icon may indicate of a Tag 1661 located within the RTA 1661, or at least the direction in the RTA 1664 along which the Tag 1661 may be located at an instance in time. In addition, an area of a user interface portraying the Icon may user interactive device such that when the device is activated, the Smart Device is operative to perform an action.
The actual positions of the Tags in real-world space (or the digital equivalent in the real-world space) may be stored and maintained in a database. Positions of physical Tags may be determined via techniques based upon wireless communication and be updated periodically. A period of update may be contingent upon variables including, user preference, Tag movement, change in environmental conditions, User query or other variable that may be converted into a programmable command. In another example of some embodiment, an Agent may interact with a user interface and understand the presence of Tags that are outside of the RTA 1661 and adjust one or both of a position and direction that the Smart Device to cause the Smart Device to be positioned such that the RTA 1661 encompasses a position of the Tag of interest.
Referring to illustration
Smart Device 1701 may have its position and direction determined using the orienteering methods described herein, with reference to Reference Point Transceiver 1731. The position may be determined relative to a Base Node, which may operate as an origin in a coordinate system associated with Structure 1721 and its surroundings. The position-determination step may be aided with reference to transmitter 1722, which in some embodiments, may be a Reference Point Transceiver. In this example, transmitter 1722 is positioned proximate to the Structure 1721.
A receiver on Smart Device 1701 may be operative to receive a wireless logical communication from transmitter 1722. This communication may be in one of a variety of modalities, such as Bluetooth, ultra-wideband, radiofrequency, etc. Based upon the signal, Smart Device 1701 may transmit to a server, a database query based upon a determined set of coordinates of transmitter 1722.
the database contains an entry comprising, as a data structure, a set of coordinates proximate to the set of coordinates of transmitter 1722, then Smart Device display 1702 may display icon 1712 proximate to the location of transmitter 1722, as displayed on Smart Device display 1702, or otherwise on the virtual representation of the shop 1711. In this way, a user of Smart Device 1701 may be alerted to the presence of information associated with structure 1721 in which the user may be interested.
In sonic embodiments, icon 1712 may be displayed on Smart Device display 1702 only if Smart Device 1701 can transmit appropriate permissions to the database. For example, icon 1712 may only display if Smart Device 1701 is on a certain Wi-Fi network or if the user of Smart Device 1701 has input the appropriate credentials, In other embodiments, icon 1712 may display on any Smart Device display 1702 (if the Radio Target Area Cone 1713 includes transmitter 1722), but further functionality may be based upon inputting a password (or, in some embodiments, correctly input the answer to a question).
In some embodiments, the appearance of icon 1712 may change based upon an identity of the user or based upon some other dynamic. For example, if the user has a certain UUID, and the database includes a message specifically intended for a user with that UUID, then the icon may flash to indicate the presence of a message. This message may be displayed textually, visually, audibly, or by a hologram. Similarly, the database may record each instance in which it is accessed by a query from a Smart Device. Such a record may include a time stamp. If data related to structure 1721 has changed since the last time stamp, then icon 1712 may turn red (for example) to indicate such a change. In addition, digital content may be appended to any content already in the database, such as additional alphanumeric annotation, an audio file, an image file, a video file or a story file.
Activation of an interactive user device encompassing icon 1712, additional functionality may be provided to the user of the Smart Device 1701. For example, selecting icon 1712 may display information about Structure 1721, such as shop hours or discounts. Alternatively, activating the interactive user device associated with icon 1712 may generate a control panel, which may allow the user to control aspects relating to sensors or other electronics within structure 1721. For example, upon confirmation that Smart Device 1701 has the appropriate permissions, selecting icon 1712 may allow the user to turn off the lights within structure 1721.
The Smart Device 1701 may also display other functional buttons on its display 1702. In some examples, one such function may be to show displays of the sensor RTA 1713 in the context of the universal Radio Target Area surrounding the user, By activating the functional button, the user may be presented with a set of options to display the universal Radio Target Area.
Referring to illustration
Referring to
The foregoing has been illustrative of the location, position, and direction of interest being determined by the methods and apparatus above. Additionally, the objects or persons may be identified and located by a physical tag or a Virtual Tag. However, it may also be desirable to know the environmental conditions throughout the facility, and as such be able to correlate object location with the environmental conditions at that location.
A first aspect of the Environmental Control System is the creation of a three dimensional (3D) environmental profile, whereby an environmental parameter of interest is detected throughout a 3D space in the Facility. Any suitable environmental parameter can be monitored in this fashion, but as an example, the creation and monitoring of a 3D temperature profile will be discussed. Creating a 3D temperature profile in the cold storage space may be accomplished by any suitable means.
One approach for creating a 3D temperature profile is to install temperature sensors throughout the space, thus creating a “temperature mesh” which can be extrapolated to create a 3D temperature profile. Referring to
Another approach to creating a 3D temperature profile is to overlay a positioning or visualization rendering (e.g., from LIDAR) with a temperature profile obtained with a scanning technology such as thermal infrared imaging. Any suitable way of creating the positioning or visualization rendering, including video methods, may be used. However, as an example, LIDAR will be discussed. In some embodiments, referring to
An exemplary method is shown in
In some cases, in order to better visualize the change in temperature at certain locations, “temperature sensitive materials,” or TSM, which are materials having a low specific heat capacity, may be added to portions of the warehouse structures, including on walls, scaffolding, pallets, and the like. The low specific heat materials change temperature relatively easily and so smaller changes in heat may more greatly affect the temperature of such materials. Thermal imaging devices measures the amount of infrared radiation emitted from an object, and an increase in temperature will increase the amount of emitted radiation that the thermal imaging sensors may detect. Thus, materials that change temperature more easily make it easier for the scans to determine smaller fluctuations in heat at a particular location. This addition of TSM into portions of the warehouse, pallets, and the like, may be particularly useful when the product packaging has a high specific heat and so does not significantly register changes in heat fluctuations in the warehouse.
A number of configurations would allow for the LIDAR and thermal imaging scans to be obtained throughout the Facility. LIDAR and thermal imaging devices may be placed throughout the Facility in addition to or as part of a Node or Smart Device. The information from each scan may be transmitted via Nodes, Smart Devices, or other transceivers to the environmental control system so that the 3D temperature profiles may be generated and combined to provide a 3D profile of the space as a whole. Since sonic of the transmission and reception frequencies may overlap, may be close to overlapping or may have harmonics or subharmonics that overlap with other base transmission frequencies, it may be desirable that a transmission controller may multiplex the signals to the various components to ensure that the transmissions do not interfere with each other.
The 3D temperature (or other environmental parameter) profile allows every part of the Facility (that is monitored using the system) to have a temperature history. Using the methods described above using physical and virtual tags, the location of each object, including products, can be determined over time. Combining the two sets of information, the Environmental Control System can determine the environmental condition of each object over time.
While the temperature control system may be used to track the temperature of each product as it proceeds through its storage lifecycle, the temperature control system may also have additional features that allow for the control system to optimize cooling within the cold storage space, For example, the 3D temperature profile of the Facility, or a portion of the same, may be monitored before, after, or during certain Action Events, which are actions in or concerning the Facility that may have an effect on the environmental conditions in the Facility.
In sonic embodiments, the environmental control system identifies an Action Event related to the Facility and correlates the Action Event with an increase or decrease in temperature in one or more areas (also referred to as zones) in the space. For example, the environmental control system may be in electronic communication with a sensor at a bay door and determine via the sensor that the bay door has been opened. The environmental control system may monitor changes in that area, or throughout the Facility, to assess the effect of the bay door opening on environmental conditions in the Facility, As another example, the environmental control system may compare changes in humidity in the Facility based on the type or amount of goods being stored at a specific time. Correlating Action Events with changes in the environment in the Facility may allow for optimization of heating, cooling, or other cycles in the Facility. In addition, the HVAC system may intermittently run cycles in the cold storage space for heating/cooling the room. The temperature control system may be in electronic communication with the HVAC system to determine that effect of any particular cycle or time of cycling. Such correlation may allow for optimization of timing and cycling of the HVAC system. Another Action Event is a change in the position of pallets. A concentration of colder items in a particular space may decrease the temperature near those items, or the configuration may change the air flow in the space.
In some embodiments of the invention, if a temperature in a particular area of the Facility exceeds a certain temperature, the temperature control system will initiate an influx of cooler (lower temperature) air in or near that area. The amount and position of the influx of cooler air may depend on both magnitude of temperature increase and the location of the temperature increase. For example, an influx of cooler air may be provided directly to a position within or adjacent to the detected warmer temperature. Alternatively, because heat rises, the influx of cooler air may be provided to an area above where the increase in temperature is located. The temperature control system may alternatively or additionally extract (via vacuum or other methods) a portion of air (e.g., a portion of warmer air) in the room. Again, the system may extract air in the area of the warehouse that has the temperature above a certain threshold or may extract air in another portion of the cold storage space. This may likewise be performed if a temperature is too low, whereby an influx of warmer air may be provided to the area or cooler air may be extracted. The environmental monitoring may allow for experimentation as to where the influx of air is most efficient, and where a return duct is best located. Additionally, determining where to locate of fans in the Facility may be determined.
In addition to analysis and optimization, the environmental control system may identify problematic areas so that timely action may be taken to protect product quality. For example, if a temperature exceeds a certain temperature (or, for example, moisture levels exceed a certain amount), an alarm may be generated to at least one Smart Device or Node. Alarms include both visual alarms (such as on user interface) and audible alarms. In sonic cases, an alarm may be generated if a certain area exceeds a specific temperature for more than a predetermined number of minutes. Other environmental parameters falling outside predetermined ranges, such as light or vibration, may also act to initiate alarms.
As described above, the location and position of any Agent or item in the Cold Storage Facility can be known provided that a physical tag (e.g., Node or Smart Device) and/or Virtual Tag is correlated with each. Additionally, the environmental history of the Agent or item can be determined according to methods described herein. Such use of location and environmental tracking will now be described to explain how the location and environmental conditions of products may be monitored throughout their storage life cycle in the Facility.
First, the arrival of a product to a cold storage facility will be described. A delivery driver may locate the Property on which the Cold Storage Facility sits and be routed to the Property using conventional methods such as GPS. Once approaching the facility, more specific positioning methods may be used to identify the appropriate location for drop off of the contents (e.g., one or more pallets of products). Referring to
Referring to
The pallets 2324 and/or products 2326 may then be transported to a secondary location within the Cold Storage Facility 2302, for example via a transport (not shown) such as the forklift. The orienteering methods described herein may direct the driver of the transport to the appropriate location within the Cold Storage Facility 2302 using, for example, a Smart Device 2318 and/or a heads-up display (not shown). When a pallet 2324 or product 2326 is loaded onto a transport, a Smart Device 2318 on the transport may identify the pallet 2324 or product 2326 based on identifying the Virtual Tag 2300 in its Field of View, interacting with a Node on a pallet or on the delivery truck, or by scanning a physical tag 2314. The Smart Device 2318 on the transport may then correlate the movements of that product/pallet 2324/2326 with that of the transport as it moves throughout the Facility 2302 until the product/pallet 2324/2326 is relocated to a second location. The Smart Device 2318 may transmit the information about the pallet/product 2324/2326 as it is transported through the Facility 2302 to the Warehouse Management System or a separate database, thereby providing information about where the pallet/product 2324/2326 moved through the Facility 2302 and the environmental conditions throughout, as will be discussed in further detail below. The Smart Device 2318 on the transport may also include an accelerometer or vibration sensors thereon so that any jarring movements/dropping of products during transport is also detected and transmitted to the appropriate database or VMS.
When a transport moves through the warehouse space with a Node, it may pass through different temperature zones within the Facility 2302. However, the transport's Node may “pick up” the temperature or other environmental conditions of each zone in a warehouse as it passes through the building. For example, a transport may move a pallet/product 2324/2326 from its original location and update temperature conditions as it moves throughout the building, such as by correlating the location with the temperature periodically at defined intervals) such as after a certain time period or after the transport has moved a certain distance, The Node in the transport may transmit a pallet/product 2324/2326 location and transmit it to the control system, whereby its location can be correlated with the temperature at that location and time. This can be repeated sequentially and preferably automatically as the transport moves through the Facility 2302. Alternatively, the Facility 2302 may be divided into certain temperature zones and once the transport leaves a particular zone (based on the orienteering methods or GPS methods), the temperature of the pallet/product 2324/2326 updates to that temperature identified within the new zone. This could be updated at an interval and the temperature zones could be very small (less than 5 feet) or larger (such as every 5-10, 10-20, 30-40, and 50 or more feet) depending on the particular system and warehouse.
When the pallet/product arrives at a second (or third, etc.) location, a new virtual tag may be created at that location by a Smart Device, e.g., a smart device on the transport. The new coordinates of the products with the identifying information are transmitted to the appropriate database. Thus, the environmental conditions at those coordinates will be associated with the products at that location for the time that the pallet/product is at that location.
Referring to
One method according to an embodiment of the invention is shown in
When the products are to be moved to another location, the same process of tracking the products during transport and storage described above may be used. Mixing of products and pallets may also be possible where the specific products are identified as they are mixed. For example, referring to
Originally, each pallet may be tracked through the warehouse via the combination of virtual tags and association with a transmitter/receiver on a transport (or other means of tracking the movements of the products) as discussed above. As such, the pallet 2610 products have a location and environmental history associated therewith, and the pallet 2620 products have a location and environmental history associated therewith. While mixing the pallets, each product may be scanned and assigned to a new pallet. The controller or other database processor may then assign a location and environmental history to each product in the new pallet. Thus, in this example, the database would include a location/environmental history for the A products and a location/environmental history the B products while in the separate pallets 2610, 2620. Thereafter, the A and B products will have the same location and environmental data (while in the same pallet such as pallet 2630) but each product A in pallet 2630 will have a database listing that includes the location and environmental conditions for pallet 2610 prior to mixing and then the location and environmental conditions for pallet 2630 after mixing. Thus, environmental tracking may be possible even with pallet mixing.
In some embodiments, a product may be tracked from manufacture to final sale through the systems and methods. Once a product is manufactured, the product may be assigned position tags (virtual or physical) that identify the location of the products while in the manufacturing facility. Then, the products may be tracked during transport through the facility by being associated with a transport Node or Smart Device. Once the products are placed onto a delivery truck or other means of transportation, the location and environmental conditions of the truck/transport may be known by GPS or other location tracking technologies (location), and via sensors in the transport (environmental conditions) and the information may be transmitted to a database by a Node or Smart Device on the transport. Then, processing and storage at the Cold Storage Facility may be tracked as discussed above. Once ready to distribute the products at a sales location, the product may be tracked through the Cold Storage Facility to a truck/transport, as described above, and then tracked during transportation, as further described above. If desired, the product can continue to be tracked at the sales location. Physical and/or virtual tags may be created in the store (at one or multiple locations) to identify the location of the product in the store. Thus, from manufacture to final sale, the location and environmental conditions of a product may be known. This may be beneficial for product tracking and quality assurance efforts.
As with the products and pallets, Employees be tracked with Nodes or Smart Devices carried on the Employee. The movements of the Employee may be tracked throughout the Cold Storage Facility. Additionally, the Employee's access to particular areas of the Facility may be based on transmissions from a Node, Smart Device, or transmitter on the Employee, and if the Employee is not in the appropriate area of the warehouse, an alarm or notification may be generated. Additionally, one or more Employees may have a temperature or other environmental sensor on his/her person, including on a hat or electronic device carried with the person, and the temperature of the air at the employee's location over time may also be tracked to provide further information about the temperature in the Facility. Tracking the location of each employee with the transmitter, Node, or Smart Device may also allow for counting and/or locating employees in the event of an emergency. Positional sensors on the Employee's hat (e.g., a sensor on the front or back) may also be useful to provide information about the employee's orientation for the purpose of orienteering or direction of interest as described herein.
Provided according to embodiments of the invention are user interfaces that may be used with the methods, devices, and apparatus described herein. A user interface may displayed on a tablet, Smart Device, or other flat screen, or in some embodiments be presented in a virtual reality environment, such as via a virtual reality headset. It should be noted that although a Smart Device is generally operated by a human user, some embodiments of the present invention include a controller, accelerometer, data storage medium, Image Capture Device, or an infrared capture device being available in a handheld or unmanned vehicle or other Agent.
In some embodiments of the invention, a user interface may provide an overview of operations in the Cold Storage Facility. In some cases, a user interface tracks sensors, pallets, employees, and equipment, such as via a touch screen. Referring to
For example, in
The following section describes various apparatus that may be suitable to perform the methods described herein.
Referring now to
The processor 2802 is also in communication with a storage device 2803. The storage device 2803 may comprise any appropriate information storage device, including combinations of digital storage devices (e.g. an SSD), optical storage devices, and/or semiconductor memory devices such as Random Access Memory (RAM) devices and Read Only Memory (ROM) devices.
The storage device 2803 can store a software program 2804 with executable logic for controlling the processor 2802. The processor 2802 performs instructions of the software program 2804, and thereby operates in accordance with the present invention. The processor 2802 may also cause the communication device 2801 to transmit information, including, in some instances, timing transmissions, digital data and control commands to operate apparatus to implement the processes described above. The storage device 2803 can additionally store related data in a database 2805 and database 2806, as needed.
Referring now to
The BLE5.1-consistent Node 2910 may include a processing element 2923 which may have its own memory of different types as well as capabilities for encryption of data. The BLE5.1 consistent Node 2910 may also have transceiver 2924. This circuitry may include baseband and RF functions as well as control the AoA functions and the self-verifying array functions. The Bluetooth communications 2924 may receive signals through an on-module antenna 2925 or an external antenna or array of antennas ma provide external RF input 2926, The BLE5.1-consistent Node 2910 may include functional circuitry blocks for control of security functions 2927, crypto-generations, random number generation and the like, The BLE5.1-consistent Node 2910 may include functional blocks for power management 2928.
The BLE5.1-consistent Node 2910 may be operative for quantification of temperature aspects of the Node 2910, battery-control functions and power-conversion functions. An external power source 2933 may be included to provide electrical energy to a power management unit 2928 which, in some examples, may be from a battery unit, or a grid connected power supply source in other examples. The BLE5.1-consistent Node 2910 may include functions for control of timing and triggering 2929. In a related sense, the BLE5.1-consistent Node 2910 may include function: for clock management 2930 within the module, The BLE5.1-consistent Node 2910 may also include circuit elements that are always-on 2931 to allow external connections 2932 to interact with the device and perhaps awake it from a dormant state. There may also be other customized and/or generic functions that are included in a BLE5.1-consistent Node 2910 and/or multichip module.
Referring now to
A Node 2950 may, include multiple antennas or antenna arrays 2951-2956. As described previously, the Node 2950 may include a transceiver module 2910, and in some examples, the transceiver module may include Bluetooth-adherent aspects. Communications received via an antenna 2951-2956 may be directly ported into the transceiver module 2910. Embodiments may also include routing particular antenna/antenna array outputs to the transceiver module 2910 in a controlled and timed sequence. A processing module 2970 may coordinate a connection of the Node 2950 to external peripherals.
In some examples, circuitry 2980 to logically communicate with one or more of: a Peripheral, a data connection, cameras and sensors controllers, and components to perform data and image acquisition of various kinds, or it may interface external components with the Node 2950.
The Node 2950 may also include its own power management unit 2960 which may take connected power or battery power or both and use it to prove the various power needs of the components of the assembly. The Node 2950 may have its own processing modules 2970 or collections of different types of processing functions which may have dedicated memory components 2971. In some examples, specialized processing chips of various kinds such as Graphical Processing Units and fast mathematics function calculators as well as dedicated artificial intelligence processing chips may be included to allow the Node 2950 to perform various computational functions including location determination of wirelessly connected devices amongst other functions. There may be numerous other functions to include in a Node 2950 and alternatives types of devices to perform the functions presented herein.
Referring now to
A microphone 3010 and associated circuitry may convert the sound of the environment, including spoken words, into machine-compatible signals. Input facilities may exist in the form of buttons, scroll wheels, or other tactile sensors such as touch-pads: in some embodiments, input facilities may include a touchscreen display.
Visual feedback to the user is possible through a visual display, touchscreen display, or indicator lights. Audible feedback 3034 may come from a loudspeaker or other audio transducer. Tactile feedback may come from a vibrate module 3036.
A magnetic force sensor 3037 may sense the magnetic field environment of the smart device and support direction determinations.
A motion sensor 3038 and associated circuitry convert the motion of the mobile device 3002 into machine-compatible signals. The motion sensor 3038 may comprise an accelerometer that may be used to sense measurable physical acceleration, orientation, vibration, and other movements. In some embodiments, motion sensor 3038 may include a gyroscope or other device to sense different motions.
A location sensor 3040 and associated circuitry may be used to determine the location of the device. The location sensor 3040 may detect GPS radio signals from satellites or may also use assisted GPS where the mobile device may use a cellular network to decrease the time necessary to determine location. In some embodiments, the location sensor 3040 may use radio waves to determine the distance from known radio sources such as cellular towers to determine the location of the mobile device 3002. In some embodiments these radio signals may be used in addition to GPS.
The mobile device 3002 comprises logic 3026 to interact with the various other components, possibly processing the received signals into different formats and/or interpretations. Logic 3026 may be operable to read and write data and program instructions stored in associated storage or memory 3030 such as RAM, ROM, flash, or other suitable memory. It may read a time signal from the clock unit 3028. In some embodiments, the mobile device 3002 may have an on-board power supply 3032. In other embodiments, the mobile device 3002 may be powered from a tethered connection to another device, such as a Universal Serial Bus (USB) connection.
The mobile device 3002 also includes a network interface 3016 to communicate data to a network and/or an associated computing device. Network interface 3016 may provide two-way data communication. For example, network interface 3016 may operate according to the internet protocol. As another example, network interface 3016 may be a local area network (LAN) card allowing a data communication connection to a compatible LAN. As another example, network interface 3016 may be a cellular antenna and associated circuitry which may allow the mobile device to communicate over standard wireless data communication networks. In some implementations, network interface 3016 may include a Universal Serial Bus (USB) to supply power or transmit data. In some embodiments, other wireless links may also be implemented.
As an example of one use of mobile device 3002, a reader may scan some coded information from a location marker in a Structure with the mobile device 3002. The coded information may be included on apparatus such as a hash code, bar code, RFID or other data storage device. In some embodiments, the scan may include a bit-mapped image via the optical capture device 3008. Logic 3026 causes the bit-mapped image to be stored in memory 3030 with an associated time-stamp read from the dock unit 3028. Logic 3026 may also perform optical character recognition (OCR) or other post-scan processing on the bit-mapped image to convert it to text. The reader may then upload the bit-mapped image (or text or other signature, if post-scan processing has been performed by logic 3026) to an associated computer via network interface 3016.
As an example of another use of mobile device 3002, a reader may capture some text from an article as an audio file by using microphone 3010 as an acoustic capture port. Logic 3026 causes audio file to be stored in memory 3030. Logic 3026 may also perform voice recognition or other post-scan processing on the audio file to convert it to text. As above, the reader may then upload the audio file (or text produced by post-scan processing performed by logic 3026) to an associated computer via network interface 3016.
A directional sensor 3041 may also be incorporated into the mobile device 3002. The directional sensor may be a compass and produce data based upon a magnetic reading or based upon network settings.
A LiDAR sensing system 3051 may also be incorporated into Smart Device 3002. The LiDAR system may include a scannable laser light (or other collimated) light source which may operate at nonvisible wavelengths such as in the infrared. An associated sensor device, sensitive to the light of emission may be included in the system to record time and strength of returned signal that is reflected off of surfaces in the environment of Smart Device 3002. Aspects relating to capturing data with LiDAR and comparing it to a library of stored data (which may be obtained at multiple angles to improve accuracy) are discussed above.
Various embodiments of the present invention may include one or more systems including a controller and electronic and/or solid state sensors and electromechanical and optical devices can be configured to perform particular operations or actions by virtue of having software, firmware, hardware, or a combination of them installed on the system that in operation causes or cause the system to perform the actions. One or more computer programs can be configured to perform particular operations or actions by virtue of including instructions that, when executed by data processing apparatus, cause the apparatus to perform the actions.
One general aspect includes a method of monitoring a cold storage structure including: generating a representation of a surface topography within a specified area within a structure. The method of monitoring also includes wirelessly communicating between a transceiver collocated with an item in the specified area and one or more reference point transceivers.
In some embodiments, the method of monitoring also includes based upon the wirelessly communicating between a transceiver collocated with an item in the specified area and one or more reference point transceivers, generating positional coordinates for the items with the specified area with the structure. The method of monitoring may also include with a thermal energy detection apparatus, quantifying amounts of thermal energy at disparate positions included in the specified area within the structure. The method of monitoring also includes generating a user interface including a 3d temperature profile of the items within a specified area within the structure, the user interface including at least portions of the representation of the surface topography and the quantified amounts of thermal energy at disparate positions included in the specified area within the structure. The method of monitoring also includes at selected time intervals repeat the step of quantifying amounts of thermal energy at disparate positions included in the specified area within the structure. The method of monitoring also includes with a controller monitoring the thermal energy at disparate positions included in the specified area within the structure for an increase or a decrease in thermal energy outside a predetermined temperature range; and if an amount of thermal energy at the disparate positions included in the specified area within the structure fall outside the predetermined temperature range, with a controller initiate one or both of an influx of air to a predetermined position in the structure or initiate an extraction of air from a predetermined position in the structure. Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.
Implementations may include one or more of the following features. The method where the step of quantifying an amount of thermal energy includes quantifying infrared energy as a digital value using a microbolometer. The method where the sensor quantifying a thermal state includes operation of an electronic temperature sensing device including at least one of thermocouple, a thermistor, and a resistance based sensor. The method where if amounts of thermal energy quantified include a temperature greater than the predetermined temperature range, initiating with the controller one or both of: an influx of air with a lower amount of thermal energy than the amount quantified; and extracting air with an amount of thermal energy higher than the predetermined temperature range. The method where the influx of air is provided to a location of the increase in temperature. The method where an influx of air is provided to an airflow vent above a location including thermal energy higher than the predetermined temperature range, as determined by the step of quantifying amounts of thermal energy. The method where if the thermal energy at a location is less than the predetermined temperature range, initiating with the controller one or both of an influx of air with a higher amount of thermal energy than the amount quantified; and extracting air with an amount of thermal energy lower than the predetermined temperature range. The method where the influx of air is provided to the location of the increase in temperature.
In some embodiments, implementations may include one or more of the following aspects, identifying action events related to the cold storage structure and the controller correlates the action events with an increase or decrease in temperature outside of the predetermined temperature range. The method where the action event is one or both of: an opening of a door and a closing of the door in the cold storage structure. The method where the action event is one of a heating cycle and a cooling cycle by a HVAC unit. The method additionally including the steps of: with a lidar unit in electronic communication with the controller, generating positional data points of items in the area of the structure; and with the controller creating a 3d position map of the cold storage structure at selected intervals to identify objects in the cold storage structure. The method where the step of quantifying amounts of thermal energy includes measuring a temperature of an item in the specified area within the structure.
The method additionally may include the step of: with the controller correlating a change in temperature of the item in the specified area within the structure. The method additionally including the step of with the controller determining at least one of: a change in a position of the item in the specified area within the structure, and removal of the item from the specified area within the structure. The method where the item is a worker in the cold storage structure.
Still other aspects of some embodiments may include measuring an amount of moisture in the specified area with a humidity detection apparatus in electronic communication with the controller. The method additionally including the step of initiating an alarm if the amount of moisture in the specified area is outside a predetermined humidity range. The method where the wirelessly communicating between a transceiver collocated with an item in the specified area and one or more reference point transceivers includes a ultra-wideband transceiving. The method where the wirelessly communicating between a transceiver collocated with an item in the specified area and one or more reference point transceivers includes a bluetooth transceiving. Implementations of the described techniques may include hardware, a method or process, or computer software on a computer-accessible medium.
Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in combination in multiple embodiments separately or in any suitable sub-combination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a sub-combination or variation of a sub-combination.
Moreover, the separation of various system components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.
Thus, particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order show, or sequential order, to achieve desirable results. In certain implementations, multitasking and parallel processing may be advantageous. Nevertheless, it will be understood that various modifications may be made without departing from the spirit and scope of the claimed disclosure.
This application claims priority to the Non-Provisional patent application Ser. No. 17/374,225, filed Jul. 13, 2021, and entitled METHOD AND APPARATUS FOR INTERACTING WITH A NODE IN A STORAGE AREA, which is a continuation in part application of Ser. No. 17/134,824 filed Dec. 28, 2020 and entitled METHOD AND APPARATUS FOR INTERACTING WITH A TAG IN A COLD STORAGE AREA as a continuation in part application, which in turn claims priority Non-Provisional patent application Ser. No. 16/943,750 filed Jul. 30, 2020 and entitled COLD STORAGE ENVIRONMENTAL CONTROL AND PRODUCT TRACKING as a continuation in pan application, which in turn claims priority to the Non-Provisional patent application Ser. No. 16./915,15,5 filed Jun. 29, 2020 and entitled METHOD OF WIRELESS DETERMINATION OF A POSITION OF A NODE as a continuation in part application, which in turn claims priority to the Non Provisional Patent Application Ser. No. 16/775,223, filed Jan. 28, 2020 and entitled SPATIAL SELF-VERIFYING ARRAY OF NODES as a continuation in part; which in turn claims priority to the Non Provisional patent application Ser. No. 16/688,775, filed Nov. 19, 2019 and entitled METHOD AND APPARATUS FOR WIRELESS DETERMINATION OF POSITION AND ORIENTATION OF A SMART DEVICE as a continuation in part and to Non Provisional patent application Ser. No. 16/721,906, filed. Dec. 19, 2019 and entitled METHOD AND APPARATUS FOR DETERMINING A DIRECTION OF INTEREST as a continuation in part; the present application also claims priority to the Non-Provisional patent application Ser. No. 16/900,753 filed Jun. 12, 2020 and entitled METHOD AND APPARATUS FOR AUTOMATED SITE AUGMENTATION as a continuation in part application; and to the Non-Provisional patent application Ser. No. 16/898,602 filed Jun. 11, 2020 and entitled METHOD AND APPARATUS FOR ENHANCED POSITION AND ORIENTATION DETERMINATION as a continuation application, which in turn claims priority to Non-Provisional patent application Ser. No. 16/817,926 filed Mar. 13, 2020 and entitled APPARATUS FOR OPERATION OF CONNECTED INFRASTRUCTURE as a continuation in part application; and to Non-Provisional patent application Ser. No. 16/905,048 filed Jun. 18, 2020 and entitled APPARATUS FOR DETERMINING A DIRECTION OF INTEREST as a continuation application; and to Non-Provisional patent application Ser. No. 16/503,878 filed Jul. 05, 2019 and entitled METHOD AND APPARATUS FOR ENHANCED AUTOMATED WIRELESS ORIENTEERING as a continuation in part application: and to Non-Provisional patent application Ser. No. 16/142,275 filed Sep. 26, 2018 and entitled METHODS AND APPARATUS FOR ORIENTEERING as a continuation in part application, which in turn claims priority to Provisional application Ser. No. 62/769,133 filed Nov. 19, 2018 and entitled METHODS AND APPARATUS FOR ORIENTEERING. The contents of each of the heretofore claimed matters are relied upon and incorporated herein by reference.
| Number | Name | Date | Kind |
|---|---|---|---|
| 5884202 | Arjomand | Mar 1999 | A |
| 5933479 | Michael et al. | Aug 1999 | A |
| 6006021 | Tognazzini | Dec 1999 | A |
| 6292108 | Straser et al. | Sep 2001 | B1 |
| 6321158 | DeLorme et al. | Nov 2001 | B1 |
| 6853958 | Turin et al. | Feb 2005 | B1 |
| 6963282 | Yeates et al. | Nov 2005 | B1 |
| 7057557 | Lee | Jun 2006 | B2 |
| 7994981 | Farrokhi et al. | Aug 2011 | B1 |
| 8280404 | Roskind | Oct 2012 | B1 |
| 8818758 | Singh et al. | Aug 2014 | B1 |
| 8843350 | Jacobi et al. | Sep 2014 | B2 |
| 8862415 | Adams | Oct 2014 | B1 |
| 8965741 | McCulloch et al. | Feb 2015 | B2 |
| 8996156 | Melzer-Jokisch et al. | Mar 2015 | B2 |
| 9008979 | Miura | Apr 2015 | B2 |
| 9037124 | Ledet | May 2015 | B1 |
| 9064219 | Hall et al. | Jun 2015 | B2 |
| 9342928 | Rasane et al. | May 2016 | B2 |
| 9529072 | Matzner | Dec 2016 | B2 |
| 9668106 | Lin et al. | May 2017 | B2 |
| 9703517 | Andolina | Jul 2017 | B2 |
| 9772396 | Liao et al. | Sep 2017 | B2 |
| 9792020 | Kelley et al. | Oct 2017 | B1 |
| 9892559 | Koshida et al. | Feb 2018 | B2 |
| 9934335 | Pettersson | Apr 2018 | B2 |
| 9952569 | Brissman | Apr 2018 | B2 |
| 10025887 | Santarone et al. | Jul 2018 | B1 |
| 10042341 | Jacob | Aug 2018 | B1 |
| 10054914 | Vartiainen et al. | Aug 2018 | B2 |
| 10130285 | Singamsetty et al. | Nov 2018 | B1 |
| 10149141 | Stamatakis et al. | Dec 2018 | B1 |
| 10210353 | Li et al. | Feb 2019 | B1 |
| 10222301 | Silva et al. | Mar 2019 | B2 |
| 10278016 | Bitra et al. | Apr 2019 | B2 |
| 10295647 | Hulvey | May 2019 | B1 |
| 10355351 | Cummings et al. | Jul 2019 | B2 |
| 10444324 | Dackefjord et al. | Oct 2019 | B2 |
| 10824774 | Santarone et al. | Nov 2020 | B2 |
| 10825247 | Vincent et al. | Nov 2020 | B1 |
| 10872179 | Wodrich | Dec 2020 | B2 |
| 11080439 | Wodrich | Aug 2021 | B2 |
| 11087039 | Duff et al. | Aug 2021 | B2 |
| 11100260 | Wodrich | Aug 2021 | B2 |
| 11194938 | Kincart | Dec 2021 | B2 |
| 11429761 | Wodrich | Aug 2022 | B2 |
| 11436388 | Santarone et al. | Sep 2022 | B2 |
| 11468209 | Wodrich | Oct 2022 | B2 |
| 20020095269 | Natalini et al. | Jul 2002 | A1 |
| 20020181405 | Ying | Dec 2002 | A1 |
| 20030110001 | Chassin et al. | Jun 2003 | A1 |
| 20030135324 | Navab | Jul 2003 | A1 |
| 20030163440 | Tonack | Aug 2003 | A1 |
| 20030195008 | Mohi et al. | Oct 2003 | A1 |
| 20040002786 | Sasaki | Jan 2004 | A1 |
| 20040119662 | Dempski | Jun 2004 | A1 |
| 20040122628 | Laurie | Jun 2004 | A1 |
| 20040162084 | Wang | Aug 2004 | A1 |
| 20050165576 | Jesmonth | Jul 2005 | A1 |
| 20050208951 | Annunziato et al. | Sep 2005 | A1 |
| 20050275525 | Ahmed | Dec 2005 | A1 |
| 20060028345 | Lee | Feb 2006 | A1 |
| 20060084436 | Green et al. | Apr 2006 | A1 |
| 20060084463 | Yoo et al. | Apr 2006 | A1 |
| 20060164382 | Kulas et al. | Jul 2006 | A1 |
| 20070004449 | Sham | Jan 2007 | A1 |
| 20070266395 | Lee et al. | Nov 2007 | A1 |
| 20070296571 | Kolen | Dec 2007 | A1 |
| 20080103835 | Corcoran et al. | May 2008 | A1 |
| 20080180246 | Malik | Jul 2008 | A1 |
| 20080246734 | Tsui et al. | Oct 2008 | A1 |
| 20080319654 | Shintani et al. | Dec 2008 | A1 |
| 20090097710 | Sroka et al. | Apr 2009 | A1 |
| 20090189810 | Murray | Jul 2009 | A1 |
| 20090216438 | Shafer | Aug 2009 | A1 |
| 20090293051 | Krywaniuk | Nov 2009 | A1 |
| 20100103036 | Malone et al. | Apr 2010 | A1 |
| 20100107635 | Tsubone | May 2010 | A1 |
| 20100271263 | Moshfeghi | Oct 2010 | A1 |
| 20100296075 | Hinderling et al. | Nov 2010 | A1 |
| 20100309044 | Ische et al. | Dec 2010 | A1 |
| 20110047516 | Pavan et al. | Feb 2011 | A1 |
| 20110068906 | Shafer et al. | Mar 2011 | A1 |
| 20110075835 | Hill | Mar 2011 | A1 |
| 20110115816 | Brackney | May 2011 | A1 |
| 20110153101 | Thomas et al. | Jun 2011 | A1 |
| 20110182202 | Olofsson et al. | Jul 2011 | A1 |
| 20110199479 | Waldman | Aug 2011 | A1 |
| 20110251787 | Gupta et al. | Oct 2011 | A1 |
| 20120087212 | Vartanian et al. | Apr 2012 | A1 |
| 20120127306 | Oh et al. | May 2012 | A1 |
| 20120127976 | Lin et al. | May 2012 | A1 |
| 20120173456 | Hirl | Jul 2012 | A1 |
| 20120188847 | Miyamoto et al. | Jul 2012 | A1 |
| 20120204646 | Lee et al. | Aug 2012 | A1 |
| 20120214507 | Vartanian et al. | Aug 2012 | A1 |
| 20120240077 | Vaittinen et al. | Sep 2012 | A1 |
| 20120259594 | Khan et al. | Oct 2012 | A1 |
| 20120296610 | Hailemariam et al. | Nov 2012 | A1 |
| 20130010103 | Ihara et al. | Jan 2013 | A1 |
| 20130073059 | Brian et al. | Mar 2013 | A1 |
| 20130084886 | Crilly, Jr. et al. | Apr 2013 | A1 |
| 20130120630 | Kim et al. | May 2013 | A1 |
| 20130169681 | Rasane et al. | Jul 2013 | A1 |
| 20130169801 | Martin et al. | Jul 2013 | A1 |
| 20130197685 | Matsunaga et al. | Aug 2013 | A1 |
| 20130201311 | Hirakawa | Aug 2013 | A1 |
| 20130223261 | Aggarwal et al. | Aug 2013 | A1 |
| 20130283529 | Hayes et al. | Oct 2013 | A1 |
| 20130288719 | Alonzo | Oct 2013 | A1 |
| 20130297555 | Fadell et al. | Nov 2013 | A1 |
| 20130345975 | Vulcano et al. | Dec 2013 | A1 |
| 20140058572 | Stein et al. | Feb 2014 | A1 |
| 20140084909 | Pagani | Mar 2014 | A1 |
| 20140107828 | Zhu et al. | Apr 2014 | A1 |
| 20140129599 | Boccon-Gibod | May 2014 | A1 |
| 20140146038 | Kangas et al. | May 2014 | A1 |
| 20140156455 | Atwood et al. | Jun 2014 | A1 |
| 20140188394 | Febonio et al. | Jul 2014 | A1 |
| 20140210856 | Finn et al. | Jul 2014 | A1 |
| 20140225814 | English et al. | Aug 2014 | A1 |
| 20140244160 | Cragun et al. | Aug 2014 | A1 |
| 20140266755 | Arensmeier et al. | Sep 2014 | A1 |
| 20140267420 | Schowengerdt et al. | Sep 2014 | A1 |
| 20140274151 | Pattabiraman et al. | Sep 2014 | A1 |
| 20140277594 | Nixon et al. | Sep 2014 | A1 |
| 20140278065 | Ren | Sep 2014 | A1 |
| 20140368373 | Crain et al. | Dec 2014 | A1 |
| 20150005903 | Worek et al. | Jan 2015 | A1 |
| 20150094081 | Gupta | Apr 2015 | A1 |
| 20150094865 | Choi et al. | Apr 2015 | A1 |
| 20150116132 | Nohra et al. | Apr 2015 | A1 |
| 20150121222 | Lacaze et al. | Apr 2015 | A1 |
| 20150137967 | Wedig et al. | May 2015 | A1 |
| 20150142179 | Ito et al. | May 2015 | A1 |
| 20150154803 | Meier et al. | Jun 2015 | A1 |
| 20150156423 | Lundberg | Jun 2015 | A1 |
| 20150168154 | Boerger | Jun 2015 | A1 |
| 20150177718 | Vartiainen et al. | Jun 2015 | A1 |
| 20150178633 | EIBsat et al. | Jun 2015 | A1 |
| 20150192656 | Werner et al. | Jul 2015 | A1 |
| 20150206348 | Koreeda | Jul 2015 | A1 |
| 20150207316 | Saussele et al. | Jul 2015 | A1 |
| 20150227123 | Laycock et al. | Aug 2015 | A1 |
| 20150262421 | Bell et al. | Sep 2015 | A1 |
| 20150287318 | Nair et al. | Oct 2015 | A1 |
| 20150294506 | Bare et al. | Oct 2015 | A1 |
| 20150327010 | Gottschalk et al. | Nov 2015 | A1 |
| 20150347854 | Bare et al. | Dec 2015 | A1 |
| 20150356786 | Bare et al. | Dec 2015 | A1 |
| 20150356789 | Komatsu et al. | Dec 2015 | A1 |
| 20160004805 | Drees et al. | Jan 2016 | A1 |
| 20160019721 | Bare et al. | Jan 2016 | A1 |
| 20160026253 | Bradski et al. | Jan 2016 | A1 |
| 20160026729 | Gil et al. | Jan 2016 | A1 |
| 20160048200 | Kim et al. | Feb 2016 | A1 |
| 20160063671 | Piippo et al. | Mar 2016 | A1 |
| 20160066068 | Schultz et al. | Mar 2016 | A1 |
| 20160069675 | Bando et al. | Mar 2016 | A1 |
| 20160071196 | Joshi et al. | Mar 2016 | A1 |
| 20160077532 | Lagerstedt et al. | Mar 2016 | A1 |
| 20160084936 | Smith et al. | Mar 2016 | A1 |
| 20160091217 | Verberkt et al. | Mar 2016 | A1 |
| 20160095188 | Verberkt et al. | Mar 2016 | A1 |
| 20160132046 | Beoughter et al. | May 2016 | A1 |
| 20160135006 | Fjeldsoe-Nielsen | May 2016 | A1 |
| 20160178383 | Mays et al. | Jun 2016 | A1 |
| 20160183057 | Steiner | Jun 2016 | A1 |
| 20160202678 | Aggarwal et al. | Jul 2016 | A1 |
| 20160216879 | Park et al. | Jul 2016 | A1 |
| 20160238692 | Hill et al. | Aug 2016 | A1 |
| 20160258760 | Beaumont et al. | Sep 2016 | A1 |
| 20160284075 | Phan et al. | Sep 2016 | A1 |
| 20160284128 | Michalscheck et al. | Sep 2016 | A1 |
| 20160285416 | Tiwari et al. | Sep 2016 | A1 |
| 20160323149 | Hu et al. | Nov 2016 | A1 |
| 20160335731 | Hall | Nov 2016 | A1 |
| 20160343093 | Riland et al. | Nov 2016 | A1 |
| 20160343243 | Rabb et al. | Nov 2016 | A1 |
| 20160345137 | Ruiz | Nov 2016 | A1 |
| 20160356593 | Huang et al. | Dec 2016 | A1 |
| 20160360429 | Li et al. | Dec 2016 | A1 |
| 20160379083 | Sala et al. | Dec 2016 | A1 |
| 20170006487 | Baldwin et al. | Jan 2017 | A1 |
| 20170023659 | Bruemmer et al. | Jan 2017 | A1 |
| 20170055126 | O'Keeffe | Feb 2017 | A1 |
| 20170079001 | Lewis | Mar 2017 | A1 |
| 20170115022 | Grosshart et al. | Apr 2017 | A1 |
| 20170131426 | Sgarz et al. | May 2017 | A1 |
| 20170169683 | Ryder | Jun 2017 | A1 |
| 20170200312 | Smith et al. | Jul 2017 | A1 |
| 20170212668 | Shah et al. | Jul 2017 | A1 |
| 20170234962 | Yang et al. | Aug 2017 | A1 |
| 20170235290 | Weber et al. | Aug 2017 | A1 |
| 20170237892 | Sakai | Aug 2017 | A1 |
| 20170255450 | Mullins et al. | Sep 2017 | A1 |
| 20170286568 | Dean et al. | Oct 2017 | A1 |
| 20170289344 | Greenberger et al. | Oct 2017 | A1 |
| 20170363504 | Winant et al. | Dec 2017 | A1 |
| 20180018826 | Maier et al. | Jan 2018 | A1 |
| 20180031618 | Friedlander et al. | Feb 2018 | A1 |
| 20180035263 | Titus et al. | Feb 2018 | A1 |
| 20180048693 | Gulbinas et al. | Feb 2018 | A1 |
| 20180075168 | Tiwari et al. | Mar 2018 | A1 |
| 20180084623 | Joppi et al. | Mar 2018 | A1 |
| 20180101803 | Tiwari et al. | Apr 2018 | A1 |
| 20180102858 | Tiwari et al. | Apr 2018 | A1 |
| 20180108079 | Traub | Apr 2018 | A1 |
| 20180113506 | Hall | Apr 2018 | A1 |
| 20180130260 | Schmirler et al. | May 2018 | A1 |
| 20180131907 | Schmirler et al. | May 2018 | A1 |
| 20180159904 | Hu et al. | Jun 2018 | A1 |
| 20180206096 | Sharma et al. | Jul 2018 | A1 |
| 20180239313 | Santarone et al. | Aug 2018 | A1 |
| 20180239840 | Santarone et al. | Aug 2018 | A1 |
| 20180242907 | Bonomi | Aug 2018 | A1 |
| 20180285482 | Santos et al. | Oct 2018 | A1 |
| 20180295526 | Wills et al. | Oct 2018 | A1 |
| 20180316885 | Reis | Nov 2018 | A1 |
| 20180328753 | Stenning et al. | Nov 2018 | A1 |
| 20180357823 | Koniki et al. | Dec 2018 | A1 |
| 20180374269 | Smith | Dec 2018 | A1 |
| 20190025905 | Godina et al. | Jan 2019 | A1 |
| 20190041972 | Bae | Feb 2019 | A1 |
| 20190057169 | Santarone et al. | Feb 2019 | A1 |
| 20190096232 | Wedig et al. | Mar 2019 | A1 |
| 20190145648 | Sinha et al. | May 2019 | A1 |
| 20190156576 | Ndolo | May 2019 | A1 |
| 20190216333 | Lai et al. | Jul 2019 | A1 |
| 20190228370 | Lien | Jul 2019 | A1 |
| 20190268062 | Josefiak | Aug 2019 | A1 |
| 20190281573 | Tyagi | Sep 2019 | A1 |
| 20190294834 | Mountz | Sep 2019 | A1 |
| 20190355177 | Manickam et al. | Nov 2019 | A1 |
| 20190392088 | Duff et al. | Dec 2019 | A1 |
| 20190392640 | Qian et al. | Dec 2019 | A1 |
| 20200057825 | Motahar | Feb 2020 | A1 |
| 20200072538 | Woolf et al. | Mar 2020 | A1 |
| 20200108926 | Smith et al. | Apr 2020 | A1 |
| 20200151450 | Hishinuma et al. | May 2020 | A1 |
| 20200286300 | Shih | Sep 2020 | A1 |
| Number | Date | Country |
|---|---|---|
| 102147597 | Aug 2011 | CN |
| 2726817 | Sep 2018 | EP |
| 2498177 | Jul 2013 | GB |
| 101354688 | Jan 2014 | KR |
| 2008124713 | Oct 2008 | WO |
| 2011019810 | Feb 2011 | WO |
| 2014204753 | Dec 2014 | WO |
| 2016130571 | Aug 2016 | WO |
| 2016192916 | Dec 2016 | WO |
| 2017149526 | Sep 2017 | WO |
| 2017223008 | Dec 2017 | WO |
| Entry |
|---|
| “Wireless Indoor Localization Systems and Techniques: Survey and Comparative Study” Indonesian Journal of Electrical Engineering and Computer Science, vol. 3, No. 2, Aug. 2016, pp. 392-409 (Year: 2016). |
| Amekudzi, Adjo A., Rebecca Shelton, and Tim R. Bricker. “Infrastructure Rating Tool: Using Decision Support Tools to EnhanceASCE Infrastructure Report Card Process.” Leadership and Management in Engineering 13.2 (2013): 76-82. (Year: 2013). |
| ASCE 2014 Report Card of Montana's Infrastructure, accessed at https://www.infrastructurereportcard.org/wp-contentluploads/2014/11/2014-Report-Card-for-Montanas-Infrastructure.pdf (Year: 2014). |
| Aukstakalnis, Steve. Practical augmented reality: A guide to the technologies, applications, and human factors for AR and VR.Addison-Wesley Professional, 2016. (Year: 2016). |
| Azeez et al., “Wireless Indoor Localization Systems and Techniques: Survey and Comparative Study” Indonesian Journal of Electrical Engineering and Computer Science, vol. 3, No. 2, Aug. 2016, pp. 392-409 (Year: 2016). |
| Brainergiser, “Large holographic projector—a real use case”. Sep. 19, 2015, https://www.youtube.com/watch?v=JwnS-EKTW2A&feature=youtu.be. |
| Carmelo Scuro et al., IoT for Structural Health Monitoring, Dec. 2018, IEEE Instrumentation & Measurement Magazine, pp. 4-14. |
| Dong et ai, “Evaluation of the Reliability of RSSI for Indoor localization”, 2012, pp. 6. downloaded from the internet https://ieeexplore.ieee.org/documentl6402492 (Year: 2012). |
| Edwards, Luke, “Holograms are finally here: Plasma lasers used to create images in mid-air.” Pocket-Lint.com, Nov. 5, 2014, https://www.pocket-lint.com/gadgets/news/131622-holograms-are-finally-here-plasma-lasers-used-to-create-images-in-mid-air. |
| Gifford, Matthew, “Indoor Positioning with Ultrasonic/Ultrasound”, Oct. 19, 2018, 7 pages, https://www.leverege.com/blogpost/ultrasonic-indoor-positioning. |
| Hexamite, “HX19V2 RFID Ultrasonic Positioning System”, 1999, https://www.hexamite.com/hx19.htm. |
| Hu. Wi-Fi Based Indoor Positioning System Using Smartphones. Nov. 2013. [retrieved on Apr. 23, 2018]. Retrieved [from the Internet: (91 pages total). |
| Wikipedia article “Building Information Modeling”, archive data Jan. 15, 2016 on the Wayback machine (Year: 2016). |
| Zou et al., “Smartscanner: Know More in Walls with Your Smartphone!” IEEE Transactions on Mobile Computing, vol. 15, No. 11, Nov. 2016, pp. 2865-2877 (Year: 2016). |
| Zhihua Wang et al., A Review of Wearable Technologies for Elderly Care that Can Accurately Track Indoor Position, Recognize Physical Activities and Monitor Vital Signs in Real Time, Feb. 10, 2017, Sensors (Basel), 17(2), 341, p. 1-36; doi: 10.3390/s 17020341 (Year: 2017). |
| Yang et al, “Wi Fi-Based Indoor Positioning”, 2015, pp. 150-157 downloaded from the internet I EEE.com databases. (Year: 2015). |
| Khemapech et al., Bridge Structural Monitoring and Warning System Aplication in Thailand—Experiences Learned, 2017, TRON Forum ISBN 978-4-89362-330-0, pp. 1-8. |
| Kyle, “Property Management” Dearborn Real Estate, 2000—Business & Economics, pp. 40-41. |
| Liu et al.,“A Hybrid Smartphone Indoor Positioning Solution for Mobile LBS” Sensors 2012, issue 12, pp. 17208-17233 (Year: 2012). |
| Liu, “Survey of Wireless Based Indoor Localization Technologies” accessed at http://www.cse.wustl.edu/-jain/cse574-14/ftp/indoor/index.html, May 5, 2014, 17 pg printout (Year: 2014). |
| Mordue, Stefan, Paul Swaddle, and David Philp. Building information modeling for dummies. John Wiley & Sons, 2015. (Year: 2015). |
| Qi, J.; Liu, G.-P. A Robust High-Accuracy Ultrasound Indoor Positioning System Based on a Wireless Sensor Network. Sensors 2017, 17, 2554. |
| Suermann, Patrick C. Evaluating the impact of building information modeling (BIM) on construction. Florida Univ GainesvilleGraduate School, 2009. (Year: 2009). |
| Thomson, C. P. H. From Point Cloud to Building Information Model: Capturing and Processing Survey Data Towards Automation forHigh Quality 3D Models to Aid a BIM Process. Diss. UCL (University College London), 2016. (Year: 2016). |
| Wang et al. A wireless structural health monitoring system with multithreaded sensing devices: design and validation. ⋅ In: Structure and Infrastructure Engineering. Feb. 16, 2007 (Feb. 16, 2007) Retrieved on Jul. 21, 2019 (Jul. 21, 2019) from entire document. |
| Wang et al. Performance Evaluation of Automatically Generated BIM from Laser Scanner Data for Sustainability Analyses. 2015. [retrieved on Apr. 23, 2018]. Retrieved from the Internet: . (8 pages total). |
| Wang, Siqi, Jinsheng Du, and Jianyong Song. “A Framework of BIM-Based Bridge Health Monitoring System.” 2016 InternationalConference on Civil, Transportation and Environment. Atlantis Press, 2016. (Year: 2016). |
| Number | Date | Country | |
|---|---|---|---|
| 20220414281 A1 | Dec 2022 | US |
| Number | Date | Country | |
|---|---|---|---|
| 62531955 | Jul 2017 | US | |
| 62531975 | Jul 2017 | US | |
| 62462347 | Feb 2017 | US | |
| 62909061 | Oct 2019 | US | |
| 62712714 | Jul 2018 | US | |
| 62793714 | Jan 2019 | US | |
| 62871499 | Jul 2019 | US | |
| 62769133 | Nov 2018 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 17374225 | Jul 2021 | US |
| Child | 17897645 | US | |
| Parent | 17134824 | Dec 2020 | US |
| Child | 17374225 | US | |
| Parent | 16943750 | Jul 2020 | US |
| Child | 17134824 | US | |
| Parent | 16161823 | Oct 2018 | US |
| Child | 16597271 | US | |
| Parent | 15887637 | Feb 2018 | US |
| Child | 16597271 | Oct 2019 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 16900753 | Jun 2020 | US |
| Child | 16943750 | US | |
| Parent | 16721906 | Dec 2019 | US |
| Child | 16900753 | US | |
| Parent | 16817926 | Mar 2020 | US |
| Child | 16721906 | US | |
| Parent | 16688775 | Nov 2019 | US |
| Child | 16817926 | US | |
| Parent | 16657660 | Oct 2019 | US |
| Child | 16688775 | US | |
| Parent | 16597271 | Oct 2019 | US |
| Child | 16657660 | US | |
| Parent | 15716133 | Sep 2017 | US |
| Child | 16161823 | US | |
| Parent | 15703310 | Sep 2017 | US |
| Child | 16161823 | Oct 2018 | US |
| Parent | 15716133 | Sep 2017 | US |
| Child | 15887637 | US | |
| Parent | 15703310 | Sep 2017 | US |
| Child | 15716133 | US | |
| Parent | 16549503 | Aug 2019 | US |
| Child | 16657660 | Oct 2019 | US |
| Parent | 16528104 | Jul 2019 | US |
| Child | 16549503 | US | |
| Parent | 16504919 | Jul 2019 | US |
| Child | 16528104 | US | |
| Parent | 16503878 | Jul 2019 | US |
| Child | 16504919 | US | |
| Parent | 16297383 | Mar 2019 | US |
| Child | 16503878 | US | |
| Parent | 16176002 | Oct 2018 | US |
| Child | 16297383 | US | |
| Parent | 16171593 | Oct 2018 | US |
| Child | 16176002 | US | |
| Parent | 16165517 | Oct 2018 | US |
| Child | 16171593 | US | |
| Parent | 16161823 | Oct 2018 | US |
| Child | 16165517 | US | |
| Parent | 16142275 | Sep 2018 | US |
| Child | 16161823 | US | |
| Parent | 15887637 | Feb 2018 | US |
| Child | 16142275 | US | |
| Parent | 16249574 | Jan 2019 | US |
| Child | 16503878 | Jul 2019 | US |
| Parent | 16176002 | Oct 2018 | US |
| Child | 16249574 | US | |
| Parent | 16503878 | Jul 2019 | US |
| Child | 16688775 | Nov 2019 | US |
