Patent Application
20220187844
Datacenter interiors are complex, varied, and dynamic. This family of attributes encompasses construction, configuration, maintenance, and repairs of the datacenter and devices (servers, racks, etc.) within datacenters. Often, the actual arrangement and content of these attributes is somewhere between estimated and historical. Continual monitoring and cataloging of these datacenter attributes to mitigate the gap between theory and practice has been difficult to achieve.
The state of datacenter rack contents is theorized based on various tools and processes, but the knowledge of the rack contents is not always entirely accurate. Conventional tools may store the state of datacenter rack contents using information that is manually entered, based on barcode asset tag information. However, if such rack content information is incorrect or only partially accurate, there may not be an easy way to correct such inaccuracies.
The present disclosure provides for automated imaging systems and methods of imaging a physical location, such as a datacenter, with an automated guided vehicle (AGV). The AGV may be equipped with a distance detection tool, such as a laser, to detect distances between the AGV and different surfaces within the physical location. The imaging system includes a plurality of cameras configured to capture a plurality of images from within the physical location. Moreover, the imaging system may correlate the distances with the plurality of images, and may generate a map of the physical location using the plurality of images and the distances.
One aspect of the disclosure provides an automated imaging system, including an automated guided vehicle having a propulsion system configured to move the vehicle, and a processor configured to control the motor and the steering system, an optical imaging system, the optical imaging system including a plurality of cameras, the cameras each being configured to have a respective field of view, the fields of view being at least partially non-overlapping with one another, a laser imaging system, the laser imaging system configured to scan a physical area to obtain respective distances between the vehicle and a plurality of locations within the physical area, and one or more processors configured to correlate a plurality of images taken by the cameras with the respective distances taken by the laser imaging system.
According to some examples, the cameras are arranged in an array, and the array is moveable in a vertical direction. The cameras in the array may be spaced apart from one another in a circular pattern that extends in a plane parallel to the vertical direction. The plurality of images may include a first set of the images and a first set of the distances taken at a first vertical position and a second set of the images and a second set of the distances taken at a second vertical position, the first vertical position and the second vertical position being spaced apart from one another in the vertical direction.
According to some examples, the one or more processors are further configured to combine the plurality of images taken by the cameras with the respective distances taken by the laser imaging system into a map. A wireless communication component may be configured to communicate with a remote computing system. The one or more processors may be configured to navigate the vehicle to a particular location based on communication from the remote computing system, the particular location corresponding to at least one of the plurality of images captured by the imaging system. The propulsion system may include a plurality of wheels rotatably mounted to the housing, a driving element configured to rotate the wheels, a steering system configured to pivot the wheels and navigate the housing along predetermined paths, and a processor configured to control the motor and the steering system.
Another aspect of the disclosure provides a method of imaging a physical area with an automated guided vehicle. The method may include moving the automated guided vehicle to a plurality of locations within the physical area by a propulsion system that navigates the vehicle along predetermined paths, taking a plurality of images of the physical area with an optical imaging system coupled to the vehicle, the optical imaging system including a plurality of cameras, the cameras each being configured to have a respective field of view, the fields of view being at least partially non-overlapping with one another, scanning the physical area with a laser imaging system coupled to the vehicle to obtain a plurality of distances between the vehicle and a plurality of locations within the physical area, and correlating the plurality of images taken by the cameras with the plurality of distances taken by the laser imaging system.
According to some examples, the method may further include locating the plurality of images and the plurality of distances relative to a known coordinate system of the physical area.
The cameras may be arranged in an array, and the array is moveable in a vertical direction perpendicular to a floor of the physical area. The cameras in the array may be spaced apart from one another in a circular pattern that extends in a plane parallel to the vertical direction. The taking of the plurality of images may include taking a first set of the images and a first set of the distances at a first vertical position, moving the array from the first vertical position to a second vertical position spaced apart from the first vertical position in the vertical direction, and taking a second set of the images and a second set of the distances at the second vertical position. The imaging system may include a fixture having a rail extending in the vertical direction and a carriage coupled to the rail and movable relative to the rail in the vertical direction, the cameras being affixed to the carriage, and the moving of the array includes moving the carriage along the rail from the first vertical position to the second vertical position. Moving the carriage along the rail may be driven by a servo motor coupled to the carriage.
According to some examples, the method may further include wirelessly sending the plurality of images and the plurality of distances to an image processor, the image processor being part of a computer that is remote from the automated guided vehicle. Moving the automated guided vehicle to the first target location may include moving the automated guided vehicle in a horizontal direction perpendicular to the vertical direction, and the fields of view extend from the plurality of cameras in a depth direction perpendicular to both the horizontal direction and the vertical direction.
Yet another aspect of the disclosure provides a method of imaging a physical area with an automated guided vehicle, the method including moving the automated guided vehicle to a plurality of locations within the physical area by a propulsion system that navigates the vehicle along predetermined paths, taking an initial plurality of images of the physical area with an imaging system coupled to the vehicle, the imaging system including a plurality of cameras, the cameras each being configured to have a respective field of view, the fields of view being at least partially non-overlapping with one another, the taking of the initial plurality of images including taking a first set of the images at a first vertical position, moving the plurality of cameras from the first vertical position to a second vertical position spaced apart from the first vertical position in the vertical direction, and taking a second set of the images at the second vertical position, combining the initial plurality of images taken by the cameras into an initial single mosaic image using an image processor, and locating the initial plurality of images relative to a known coordinate system of the physical area.
The method may further include receiving a selection of a location within the initial single mosaic image to move the automated guided vehicle to one of the plurality of locations and to take an updated plurality of images, and combining the updated plurality of images with the initial plurality of images taken by the cameras into an updated single mosaic image using the image processor.
The technology relates generally to automated 3D imaging systems and methods of imaging a physical location using an automated guided vehicle (AGV). The physical location may be, for example, a datacenter or other indoor space. For example, the automated imaging system may be configured to take a plurality of high resolution images of a datacenter rack, and combine the plurality of images into a single mosaic image. The system may parse location and barcode data from the combined mosaic image, thereby providing accurate current location and data information of the contents of the datacenter in an automated fashion. The automated imaging system may also provide real-time inventory location and data for commercial and retail racks of products that may not otherwise be obtained in an automated fashion.
Referring to
The propulsion system 16 may include a motor and/or another driving element that is configured to rotate a plurality of wheels 18 that are rotatably mounted to the AGV 12, a steering system (not shown) that is configured to pivot the wheels and navigate the AGV 12 along predetermined paths, and a processor or control system 100 (described below with respect to
The AGV 12 may also include sensors (not shown) for detecting conditions surrounding the AGV. The example configuration of the AGV 12 that is shown in
While the AGV 12 is in use, the location of the AGV may be known relative to a known 3D coordinate system of the physical area in which the AGV is navigated, such as a datacenter. To obtain its position within the known coordinate system as it moves, the AGV 12 may read barcodes on the floor that are at known locations within the coordinate system. As it travels, the AGV 12 may synchronize its location relative to the barcodes.
The automated imaging system 10 may also include an imaging system 20 coupled to the AGV 12. The imaging system 20 may include a plurality of cameras 30. The cameras 30 each may be configured to have a respective field of view 32. The fields of view 32 may be at least partially non-overlapping with one another, although in some examples, the fields of view may be entirely non-overlapping with one another. The fields of view 32 of the cameras 30 may extend away from the cameras in a depth direction D between an object, such as a datacenter rack 1, and the cameras.
The automated imaging system 10 may include an image processor (e.g., the processor 111 shown in
The cameras 30 may include one or more optical cameras and a laser imaging system, such as one or more LIDAR scanners. While the cameras 30 are shown in an arrangement of several rows stacked vertically, it should be understood that other arrangements are possible. For example, the cameras may be positioned in a 360 degree configuration about a vertical axis or a horizontal axis. As another example, the cameras may be arranged in a rosette such that the cameras collectively capture every angle around the AGV. In yet further examples, the cameras may be a single 360 degree camera.
The optical cameras may obtain images that may be combined to obtain a visual map of the physical location. For example, captured images may be combined to generate a map of an entire datacenter, such that individual rack components may be visually identified.
The dimensional accuracy of the LIDAR scanning may be within approximately plus or minus 10 mm. In some examples, a single LIDAR scanner may be used, in which an internal electro-mechanical mirror system may move a single laser beam around, permitting approximately one thousand distances to be collected while the AGV is at a single location. In other examples, a fixed array of lasers may be used. Although the system 10 is described as including LIDAR scanners, some examples only use optical images, while other examples combine optical and LIDAR data.
In the example illustrated, the imaging system 20 may include a fixture 40 affixed to the housing, the fixture having a rail 42 extending in a vertical direction V perpendicular to the depth direction D, and a carriage 44 extending in a horizontal direction H perpendicular to the vertical direction and the depth direction. The cameras 30 may be affixed to the carriage 44. The cameras 30 may be arranged along the carriage 44 in an array extending in the horizontal direction H. The cameras 30 in the array may be equally spaced apart from one another by a distance D1 in the horizontal direction. The carriage 44 may be coupled to the rail 42 such that the carriage and the array of cameras 30 affixed thereto are moveable relative to the rail in the vertical direction V. The imaging system 20 may include a servo motor (not shown) configured to move the carriage 44 in the vertical direction V along the rail 42. It should be understood that in other examples, the imaging system 20 may include other arrangements of the cameras 30, such as by implementing other mounting fixtures or mechanisms.
Although the cameras 30 are described above as arranged along the carriage 44 in an array extending in the horizontal direction H, in alternative examples, the cameras may be arranged in a circular ring around a vertical axis and extending in a plane that is perpendicular to the depth direction D. The circular ring may be movable in the vertical direction V along the rail 42. According to further examples, the cameras may be arranged in a ring extending around a longitudinal axis. In further examples, the cameras may be arranged in a rosette or sphere shaped, thereby capturing substantially all of the surroundings of the AGV. In yet further examples, the cameras may be a single 360 degree camera.
The carriage 44 may be moveable in the vertical direction V to any position along the rail 42. First, second, and Nth vertical positions P1, P2, and PN of the carriage 44 are shown in
As shown in
The LIDAR scanner may be one or more of the cameras 30 and may contain a mirror therein to direct a laser beam along any of one thousand orientations relative to the scanner. Alternatively, the LIDAR scanner may be an array of laser beams that are fixed relative to the AGV 12 or the scanner may be movable in the vertical direction V along the rail 42.
Each of these optical images may be correlated with dimensional data that is taken by the LIDAR scanner. For example, each of the images 50 may be overlaid with corresponding LIDAR data at one thousand points, such that when each image is correlated with the corresponding LIDAR data, one thousand locations within each image will be known relative to the location of the AGV 12, within a known coordinate system of the datacenter.
In the example shown, the images 50 are arranged in a four-by-six grid, with six rows of four images each extending in the horizontal direction H. However, it should be understood that the number and arrangement of images may be varied. As shown in
In the example shown in
To make a full single mosaic image 60 with no horizontal or vertical gaps, there may be a small amount of overlap between the field of view recorded in adjacent ones of the images 50. However, in other examples, there may be no overlap between the field of view recorded in adjacent ones of the images 50, since a complete imaging of the datacenter may not be necessary to identify the specific components or types of components and their locations within the datacenter, or to parse the single mosaic image to extract asset location information and asset tag barcode information. The example image 60 may be a single image out of a set of 20, 50, 100, or 500 images of the entire datacenter. Any number of images 60 may be combined to permit the entire datacenter to be imaged.
According to other examples, the 360 image of
Referring to
The map 62 may also be used to calculate a distance between features that are visible on the map. For example, a user may calculate a distance by placing two virtual pins on the map, and the map may show the distance between the pins, within plus or minus 10 mm. The map 62 may also be used to determine the size of the objects within the datacenter, such as the width of a column or the width of a rack, to an accuracy of plus or minus 10 mm.
The map 62 may be periodically updated via collection of new data by the AGV 12. To update the map 62, a user may command the AGV 12 to travel to one more desired locations within the datacenter to collect new optical and LIDAR data, and such data may replace the portions of the map that may have become outdated due to repositioning of items within the datacenter. Each set of optical and LIDAR data may be correlated with the date and time that the data was collected, to assist a user in knowing whether or not portions of the map 62 have become outdated and need to be updated.
Referring to
The one or more processors 111 may be any conventional processors, such as commercially available microprocessors. Alternatively, the one or more processors may be a dedicated device such as an application specific integrated circuit (ASIC) or other hardware-based processor. In some examples, the one or more processors may include a graphics processing unit (GPU) and/or a tensor processing unit (TPU), for example, for machine-learning based perception. Although
Memory 112 may store information that is accessible by the processors 111, including instructions 113 that may be executed by the processors, and data 114 such as asset location information and/or asset tag barcode information. Such instructions 113 may include detecting camera positioning based on sensor feedback, adjusting positions of the imaging system, capturing images of the datacenter rack, and combining images and parsing the mosaic image. The memory 112 may be of a type of memory operative to store information accessible by the processors 111, including a non-transitory computer-readable medium, or other medium that stores data that may be read with the aid of an electronic device, such as a hard-drive, memory card, read-only memory (“ROM”), random access memory (“RAM”), digital versatile disc (“DVD”) or other optical disks, as well as other write-capable and read-only memories. The subject matter disclosed herein may include different combinations of the foregoing, whereby different portions of the instructions 113 and data 114 are stored on different types of media.
Data 114 may be retrieved, stored or modified by processors 111 in accordance with the instructions 113. For instance, although the present disclosure is not limited by a particular data structure, the data 114 may be stored in computer registers, in a relational database as a table having a plurality of different fields and records, XML documents, or flat files. The data 114 may also be formatted in a computer-readable format such as, but not limited to, binary values, ASCII or Unicode. By further way of example only, the data 114 may be stored as bitmaps comprised of pixels that are stored in compressed or uncompressed, or various image formats (e.g., JPEG), vector-based formats (e.g., SVG) or computer instructions for drawing graphics. Moreover, the data 114 may comprise information sufficient to identify the relevant information, such as numbers, descriptive text, proprietary codes, pointers, references to data stored in other memories (including other network locations) or information that is used by a function to calculate the relevant data.
In one example, computing device 110 may be an AGV computing system incorporated into control system 100, such as an AGV 12 having an imaging system 20. The computing device may be capable of communicating with various components of the system autonomously, or without continuous input from a human operator. For example, computing device 110 may be in communication with various components 120-125 of the control system 100, which operate in accordance with the instructions 113 of memory 112 in an autonomous mode which does not require or need continuous or periodic input from an operator. Although these systems are shown as external to computing device 110, in other examples these systems may also be incorporated into the computing device.
The instructions 113 may be executed to perform various operations using one or more of the components 120-125 or other components not shown. For example, the AGV navigation system 120 may include a GPS or other system which directs the AGV to a target datacenter rack 1 for imaging. The positioning system 121 may be instructed to respond to feedback provided by the perception system 123, which may include one or more cameras, sensors, etc. For example, the positioning system 121 may include one or more motors or other devices used to control positioning of the AGV 12 and/or cameras 30. As the perception system 123 provides data regarding proximity of the target location within the datacenter to the cameras 30, the positioning system 121 may adjust a position of one or more of the AGV 12 or carriage 44 so as to maintain a position where the cameras are in close proximity to the target location.
Lighting system 122 may include one or more LEDs or other illuminating devices. The lighting system 122 may be instructed to illuminate a target location for imaging of a portion of the datacenter any time the AGV 12 is within a predetermined distance from the target location, when environmental conditions are such that natural light is insufficient, or under any other circumstances. In this regard, the lighting system 122 may also be responsive to input from the perception system 123, such as feedback from light or proximity sensors. Similarly, cleaning components 124 may also respond to feedback from the perception system 123. For example, the one or more processors 111 may determine, based on images captured by the perception system 123, that the target location is dirty. As such dirt may be obstructing information, such as an asset tag barcode that needs to be read to determine which component is in a particular datacenter rack 1, the cleaning components 124 may be instructed to spray air or perform some other operation to clear the debris. The power system 125 may include, for example, a battery for powering the control system 100.
While the components 110-115 and 120-125 are described above in reference to an example of a datacenter imaging mode, it should be understood that the components may also operate in response to operator input or other instructions. For example, the computing device 110 may provide information to a remote computing device, such as an operator control unit, through transmitter/receiver 115. Likewise, the computing device 110 may receive instructions from the remote operator control unit. In other examples, the control system 100 may operate in an autonomous imaging mode, but still provide feedback to a remote computing device using the transmitter/receiver 115.
The design of the automated datacenter imaging system 10 shown in
The automated datacenter imaging system 10 may also be used in environments other than datacenters. For example, the automated imaging system 10 may be used for manufacturing or retail inventory tracking. For example, the imaging system 20 may scan barcodes of products on a rack in a warehouse in the images 50 and in the single mosaic image 60, and such barcodes can be parsed from the single mosaic image by the image processor. In some examples, the imaging system 20 may scan any two-dimensional space, and information may be parsed from the single mosaic image 60 to provide inventory information either based on parsing barcodes from the single mosaic image or based on the shape and arrangement of the individual inventory items.
Referring to
According to some examples, the first set of images may be combined into a single mosaic image of the first location. For example, the images may be combined using an image processor, which may be located on the AGV or at a remote location. The single mosaic image may provide, for example, an angular view of the physical location, such as a 360 degree view. According to some examples, multiple sets of images may be captured from multiple locations to generate multiple mosaic images of different locations and having different perspectives.
In block 420, distances between the AGV and various objects in the physical location are detected. The distances may be detected by, for example, laser such as LIDAR or other distance detection mechanisms. The distances may be correlated with the captured images. For example, in some implementations the distances may be computed at substantially a same time as the images are captured, and the distance corresponding to a point in each image may be stored in association with the image. For example, the distance may be a distance between the camera and a center of a field of view of a captured image. As another example, the distance may be the distance between a point on the AGV and an object in the captured image, such as a datacenter rack or a computing device located on the rack, or an identifier such as a barcode tagging the equipment on the rack.
In block 430, the AGV may move from the first location to a second location, for example, by navigating to a second set of coordinates within the physical location. The cameras may capture a second set of one or more images at the second location.
In block 440, distances between the AGV and various objects at the second location are detected.
In block 450, the first and second images and the detected distances may be used to generate a map of the physical location. According to some examples, a known coordinate system of the physical location may further be used to generate the map. The map may be configured such that when a user clicks on a location on the map, a corresponding one of the captured images may be displayed, and dimensions of objects and distances between locations in the map may be provided.
Using the generated map, the automated imaging system may receive a selection of a location within the datacenter map, and the AGV may move to and the selected location to take an updated plurality of images and an updated set of LIDAR data. The updated plurality of images and the updated set of LIDAR data may replace the corresponding images and data within the map.
Unless otherwise stated, the foregoing alternative examples are not mutually exclusive, but may be implemented in various combinations to achieve unique advantages. As these and other variations and combinations of the features discussed above can be utilized without departing from the subject matter defined by the claims, the foregoing description of the examples should be taken by way of illustration rather than by way of limitation of the subject matter defined by the claims. In addition, the provision of the examples described herein, as well as clauses phrased as “such as,” “including” and the like, should not be interpreted as limiting the subject matter of the claims to the specific examples; rather, the examples are intended to illustrate only one of many possible examples. Further, the same reference numbers in different drawings can identify the same or similar elements.
