Patent Application
20210404808
The present application claims priority under 35 U.S.C. § 119 to Japanese Patent Application No. 2020-109966 filed Jun. 25, 2020. The contents of this application are incorporated herein by reference in their entirely.
The present invention relates to an eyewear display system, and more specifically to an eyewear display system for assisting point cloud data observation using a ground-mounted scanner.
Conventionally, point cloud data observation using a ground-mounted scanner has been known (for example, refer to Patent Literature 1). In point cloud data observation, point cloud density must be secured to meet required observation accuracy. The point cloud density depends on a measurement distance from the scanner and a rotation speed of the scanner. That is, in a region at a short distance from the scanner, the point cloud density is high, but the point cloud density becomes lower with increasing distance from the scanner.
Although the point cloud density is high when the rotation speed of the scanner is low, the point cloud density is low when the rotation speed is high. Therefore, in order to secure required point cloud density, by setting a plurality of scanner installation points (instrument installation points) so that measurement regions overlap to some degree, point cloud data is acquired.
Therefore, in point cloud observation, measurements are made by setting scanner installation points (instrument installation points) in order so that measurement regions overlap.
Patent Literature 1 Japanese Published Unexamined Patent Application No. 2018-28464
However, an excessive amount of effort is required to install a scanner at the next instrument installation points in order and perform observation while being aware of this overlapping on-site. In addition, there is a risk of omission in measurement results.
The present invention has been made in view of these circumstances, and an object thereof is to provide a technology for facilitating on-site observation without omission in point cloud data observation using a ground-mounted scanner.
In order to achieve the object described above, a point cloud data observation system according to one aspect of the present invention includes a scanner including a measuring unit configured to irradiate pulsed distance-measuring light and acquire three-dimensional coordinates of an irradiation point by measuring a distance and an angle to the irradiation point, and a point cloud data acquiring unit configured to acquire point cloud data as observation data by rotationally irradiating distance-measuring light in the vertical direction and the horizontal direction by the measuring unit; an eyewear device including a display, a relative position detection sensor configured to detect a position of the device, and a relative direction detection sensor configured to detect a direction of the device; a storage device configured to store an observation route calculated in consideration of three-dimensional CAD design data of an observation site and instrument information of the scanner; and an information processing device including a coordinate synchronizing unit configured to synchronize information on a position and a direction of the eyewear device and a coordinate space of the observation route, and an eyewear display control unit configured to control display of the observation route on the display, wherein the scanner, the eyewear device, the storage device, and the information processing device are capable of inputting and outputting information to each other, and the eyewear device superimposes and displays the observation route on a landscape of the site.
In the aspect described above, it is also preferable that the observation route is calculated to enable acquisition of point cloud data at required point cloud density by a minimum number of instrument installation points in the entire observation site in consideration of the instrument information of the scanner and a three-dimensional structure in the three-dimensional CAD design data, and the instrument information of the scanner includes coordinates of an instrument center of the scanner, pulse interval setting of the distance-measuring light, and rotation speed setting of the scanner.
In the aspect described above, it is also preferable that the observation route is calculated to be a shortest route connecting all instrument installation points in the observation site.
In the aspect described above, it is also preferable that the eyewear device is configured to update display of the observation route according to observation progress information.
In the aspect described above, it is also preferable that the information processing device is configured to recalculate the order of the instrument installation points in the observation route when point cloud observation by the scanner deviates from the observation route.
By the eyewear display system according to the aspect described above, in point cloud data observation using a ground-mounted scanner, observation without omission can be easily performed.
Hereinafter, a preferred embodiment of the present invention will be described with reference to the drawings, however, the present invention is not limited to this embodiment. The same components common to the embodiment and respective modifications are provided with the same reference signs, and overlapping descriptions are omitted as appropriate.
The scanner 2 is installed at an arbitrary point via a leveling base mounted on a tripod. The scanner 2 includes a base portion 2α provided on the leveling base, a bracket portion 2β that rotates horizontally about an axis H-H on the base portion 2α, and a light projecting portion 2γ that rotates vertically at the center of the bracket portion 2β. The eyewear device 4 is worn on the head of a worker. The processing PC 6 is installed at an observation site.
Scanner
The distance-measuring unit 21 includes a light transmitting unit, a light receiving unit, a light transmitting optical system, a light receiving optical system sharing optical elements with the light transmitting optical system, and a turning mirror 21α. The light transmitting unit includes a light emitting element such as a semiconductor laser, and emits pulsed light as distance-measuring light. The emitted distance-measuring light enters the turning mirror 21α through the light transmitting optical system, and is deflected by the turning mirror 21α and irradiated onto a measuring object. The turning mirror 21α rotates about a rotation axis V-V by being driven by the vertical rotation driving unit 22.
The distance-measuring light retroreflected by the measuring object enters the light receiving unit through the turning mirror 21α and the light receiving optical system. The light receiving unit includes a light receiving element such as a photodiode. A part of the distance-measuring light enters the light receiving unit as internal reference light, and based on the reflected distance-measuring light and internal reference light, a distance to an irradiation point is obtained by the arithmetic processing unit 26.
The vertical rotation driving unit 22 and the horizontal rotation driving unit 24 are motors, and are controlled by the arithmetic processing unit 26. The vertical rotation driving unit 22 rotates the turning mirror 21α about the axis V-V in the vertical direction. The horizontal rotation driving unit 24 rotates the bracket portion 2β about the axis H-H in the horizontal direction.
The vertical angle detector 23 and the horizontal angle detector 25 are encoders. The vertical angle detector 23 measures a rotation angle of the turning mirror 21α in the vertical direction. The horizontal angle detector 25 measures a rotation angle of the bracket portion 2β in the horizontal direction. As a result, the vertical angle detector 23 and the horizontal angle detector 25 constitute an angle-measuring unit that measures an angle of an irradiation direction of the distance-measuring light.
The arithmetic processing unit 26 is a microcontroller configured by mounting, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), etc., on an integrated circuit.
The arithmetic processing unit 26 calculates a distance to an irradiation point of each one-pulse light of the distance-measuring light based on a time difference between a light emission timing of the light transmitting unit and a light receiving timing of the light receiving unit (a reflection time of the pulsed light). In addition, the arithmetic processing unit 26 calculates an irradiation angle of the distance-measuring light at this time, and calculates an angle of the irradiation point.
The arithmetic processing unit 26 includes a point cloud data acquiring unit 261 configured by software. By scanning the entire circumference (360°) with distance-measuring light by controlling the distance-measuring unit 21, the vertical rotation driving unit 22, and the horizontal rotation driving unit 24, the point cloud data acquiring unit 261 acquires three-dimensional coordinates of each irradiation point, and acquires point cloud data of the entire circumference as observation data at an instrument installation point. Observation of the point cloud data of the instrument entire circumference of the scanner 2 in this way is referred to as full dome scanning.
The display unit 27 is, for example, a liquid crystal display. The operation unit 28 includes a power key, numeric keys, a decimal key, plus/minus keys, an enter key, and a scroll key, etc., and is configured to enable a worker to operate the scanner 2 and input information into the scanner 2.
The storage unit 29 is, for example, a hard disk drive, and stores programs for executing functions of the arithmetic processing unit 26.
The external storage device 30 is, for example, a memory card, etc., and stores various data acquired by the scanner 2.
The communication unit 31 enables communication with an external network, and for example, connects to the Internet by using an Internet protocol (TCP/IP) and transmits and receives information to and from the eyewear device 4 and the processing PC 6.
Eyewear Device
The display 41 is a goggles-lens-shaped transmissive display that covers the eyes or an eye of a worker when the worker wears it. As an example, the display 41 is an optical see-through display using a half mirror, and is configured to enable observation of a video image obtained by superimposing and synthesizing a virtual image received by the control unit 42 on a real image of a landscape of the site (hereinafter, referred to as “actual landscape”).
The control unit 42 includes an eyewear arithmetic processing unit 3, a communication unit 44, a relative position detection sensor (hereinafter, simply referred to as “relative position sensor”) 45, a relative direction detection sensor (hereinafter, simply referred to as “relative direction sensor”) 46, an eyewear storage unit 47, and an operation switch 48.
The eyewear arithmetic processing unit 43 is a microcomputer configured by mounting at least a CPU and a memory (RAM, ROM) on an integrated circuit. The eyewear arithmetic processing unit 43 outputs information on a position and a direction of the eyewear device 4 detected by the relative position sensor 45 and the relative direction sensor 46 to the processing PC 6.
In addition, the eyewear arithmetic processing unit 43 superimposes and displays an observation route 661 described later as a virtual image on a landscape of the site on the display 41 according to control of the eyewear display control unit 602. In the present embodiment, three-dimensional CAD design data is a three-dimensional design drawing of the observation site, created with an absolute coordinate system by using CAD.
The communication unit 44 enables communication with an external network, and connects to the Internet by using an Internet protocol (TCP/IP) and transmits and receives information to and from the processing PC 6.
The relative position sensor 45 performs wireless positioning from a GPS antenna, a Wi-Fi (registered trademark) access point, and an ultrasonic oscillator, etc., installed at the observation site, to detect a position of the eyewear device 4 in the observation site.
The relative direction sensor 46 consists of a combination of a triaxial accelerometer or a gyro sensor and a tilt sensor. The relative direction sensor 46 detects a tilt of the eyewear device 4 by setting the up-down direction as a Z-axis direction, the left-right direction as a Y-axis direction, and the front-rear direction as an X-axis direction.
The eyewear storage unit 47 is, for example, a memory card. The eyewear storage unit 47 stores programs that enable the eyewear arithmetic processing unit 43 to execute functions.
The operation switch 48 includes, for example, a power button 48α for turning ON/OFF a power supply of the eyewear device 4 as illustrated in
Processing PC
The PC communication unit 63 is a communication control device such as a network adapter, a network interface card, a LAN card, a modem, etc., and is a device for connecting the processing PC 6 to the Internet and a communication network such as a WAN, LAN, etc., by wire or wirelessly. The PC arithmetic processing unit 60 transmits and receives (inputs and outputs) information to and from the scanner 2 and the eyewear device 4 through the PC communication unit 63.
The PC display unit 64 is, for example, a liquid crystal display. The PC operation unit 65 is, for example, a keyboard, a mouse, etc., and enables various inputs, selections, and determinations, etc.
The PC storage unit 66 is a computer readable recoding medium that stores, describes, saves, and transmits information in a computer-processable form. For example, a magnetic disc such as an HDD (Hard Disc Drive), an optical disc such as a DVD (Digital Versatile Disc), and a semiconductor memory such as a flash memory are applicable as the PC storage unit 66. The PC storage unit 66 stores at least an observation route 661 calculated in advance in the observation site by the scanner 2.
The PC arithmetic processing unit 60 is a control unit configured by mounting at least a CPU and a memory (RAM, ROM, etc.) on an integrated circuit. In the PC arithmetic processing unit 60, a coordinate synchronizing unit 601 and an eyewear display control unit 602 are configured software-wise by installing a program.
The coordinate synchronizing unit 601 receives information on a position and a direction of the scanner 2 and information on a position and a direction of the eyewear device 4, and manages a coordinate space of the scanner 2, a coordinate space of the eyewear device 4, and a coordinate space of the observation route 661 by converting these into data in the same coordinate space. Such converted observation route 661 is referred to as an observation route 661a.
The eyewear display control unit 602 transmits the observation route 661a converted so as to conform to the coordinate spaces of the eyewear device 4 and the scanner 2 by the coordinate synchronizing unit 601 to the eyewear device 4, and controls display of the observation route 661a on the display 41.
In other words, the PC storage unit 66 stores programs and data for executing the processing of the PC arithmetic processing unit 60 including the coordinate synchronizing unit 601 and the eyewear display control unit 602. The PC arithmetic processing unit 60 can read and execute those programs and data to execute the method of this embodiment. The PC arithmetic processing unit 60 can read and execute those programs and data to execute the method according to this embodiment. In addition, all or part of the method of the present embodiment may be realized by any hardware (processor, storage device, input/output device, etc.), software, or a combination thereof.
Calculation of Observation Route
Here, the observation route 661 is described. The observation route 661 is calculated in advance by an information processing device such as a personal computer based on instrument information (coordinates of the instrument center, pulse interval setting, and rotation speed setting of scanning) of the scanner 2 and three-dimensional CAD design data of the observation site, and is stored in the PC storage unit 66.
A method for calculating the observation route 661 is described with reference to
The scanner 2 acquires point cloud data by performing rotational scanning (full dome scanning) with distance-measuring light by a predetermined angle (for example, 360°) in the vertical direction and a predetermined angle (for example, 180°) in the horizontal direction from the instrument center. Therefore, as illustrated in
Point cloud density acquired by the scanner 2 becomes higher as the pulse interval of the distance-measuring light becomes narrower, becomes lower as the rotation speed of the scanner 2 becomes higher, and becomes lower with increasing distance from the instrument center of the scanner 2. In this way, the point cloud density depends on the pulse interval setting of the distance-measuring light of the scanner 2, rotation speed setting of the scanner 2, and a distance from the instrument center of the scanner.
In the point cloud data observation, according to the purpose of observation and a request from a client, required point cloud density is set. An observation range of the scanner 2 includes a region A1 in which the required point cloud density can be met by one measurement, and a region A2 in which the required point cloud density cannot be met by one measurement, but can be met by overlapping with measurements from other points. Therefore, when designing the observation route, for example, as illustrated in
However, point cloud data observation is a survey for acquiring three-dimensional data of three-dimensional structures in an observation site, so that three-dimensional structures (for example, three-dimensional structures S1, S2, and S3 in
Therefore, by using three-dimensional CAD data of the observation site, by performing a computer simulation in three-dimensional consideration of pulse interval setting of the distance-measuring light of the scanner 2, rotation speed setting of the scanner 2, a distance from the instrument center of the scanner, and a positional relationship with the three-dimensional structures, positions of the fewest instrument installation points P which can cover the entire observation range at the required point cloud density are calculated as illustrated in
Further, as illustrated in
As described above, the observation route 661 is converted into a coordinate system of a coordinate space with an origin set at a reference point by the coordinate synchronizing unit 601, and transmitted to the eyewear device 4 by the eyewear display control unit 602, and for example, as illustrated in
Point Cloud Observation Method Using Display System 1
Next, an example of a point cloud observation method using the display system 1 will be described.
First, in Step S101, a worker sets a reference point and a reference direction in the observation site. Specifically, known points and arbitrary points in the site are selected, and a position of the eyewear device 4 in a state where the worker stands while wearing the eyewear device is set as the reference point. In addition, in the site, a characteristic point (for example, a corner of the structure) different from the reference point is arbitrarily selected, and a direction from the reference point to the characteristic point is defined as the reference direction.
Next, in Step S102, by using the scanner 2, the reference point and the reference direction are synchronized with an absolute coordinate system. Specifically, the worker installs the scanner 2 at an arbitrary point in the site and grasps absolute coordinates of the scanner 2 by using a publicly known method such as backward intersection, and grasps absolute coordinates of the reference point and the characteristic point selected in Step S101. The scanner 2 transmits the acquired absolute coordinates of the scanner 2, the reference point, and the characteristic point to the processing PC 6.
The coordinate synchronizing unit 601 of the processing PC 6 converts the absolute coordinates of the reference point into (x, y, z)=(0, 0, 0), and recognizes the reference direction as a horizontal angle of 0°, and thereafter, enables management of information from the scanner 2 in a space with an origin set at the reference point.
Next, in Step S103, the worker synchronizes the eyewear device 4. Specifically, in a state where the worker stands at the reference point and faces the reference direction while wearing the eyewear device, the worker installs the eyewear device 4 at the reference point, matches the center of the display 41 with the reference direction, and sets (x, y, z) of the relative position sensor 45 to (0, 0, 0) and sets (roll, pitch, yaw) of the relative direction sensor 4 to (0, 0, 0).
Synchronization of the eyewear device 4 is not limited to the method described above, and may be performed by, for example, a method in which the eyewear device 4 is provided with a laser device for indicating the center and the directional axis of the eyewear device 4, and by using a laser as a guide, the center and the directional axis are matched with the reference point and the reference direction.
Next, in Step S104, the coordinate synchronizing unit 601 obtains an observation route 661a (
Steps S101 to S104 are initial settings for using the display system 1. Accordingly, in Step S105, the observation route 661a can be superimposed and displayed on a landscape of the site on the display 41.
Then, in Step S106, the worker installs the scanner 2 according to this display of the observation route, and executes point cloud data observation at the installation point.
In this way, in the present embodiment, by using the scanner 2, an observation route that meets the required point cloud density and enables observation of the entire observation site with the minimum number of instrument installation points can be superimposed and displayed on a landscape of the site. As a result, the worker can easily perform point cloud observation without omission only by installing the scanner 2 according to the display and performing observation. In particular, in the present embodiment, the observation route is calculated in consideration of information on three-dimensional structures included in three-dimensional CAD design data of the observation site, so that not just two-dimensional but also three-dimensional observation without omission is possible, and determination by checking the structures in the height direction can be easily made.
In the present embodiment, a shortest route connecting all instrument installation points is calculated, so that point cloud observation can be particularly efficiently performed.
Modification 1
Specifically, in three-dimensional CAD design data, an observation route predicted from current three-dimensional CAD data by using a reinforcement learning model created by performing reinforcement learning in which, on the assumption that a scanner 2 having a set distance-measuring light pulse interval, a set scanner rotation speed, and set instrument center coordinates (instrument height) has been installed at an arbitrary instrument installation point P, next instrument installation points are set so that arrangement of the points which can finally meet required point cloud density with a minimum number of instrument installation points maximizes a reward, is used as the observation route 661A.
As a reinforcement learning method, a publicly known method such as the Monte Carlo method and Q-learning method can be used.
Modification 2
The processing PC 6B includes an eyewear display control unit 602B in place of the eyewear display control unit 602.
The coordinate synchronizing unit 601 is the same as in the display system 1, and synchronizes the observation route 661 with the eyewear device 4 and converts it into an observation route 661a (not illustrated).
The eyewear display control unit 602B updates the observation route 661a synchronized by the coordinate synchronizing unit 601 according to observation progress information received from the scanner 2 as described later.
Specifically, each time point cloud data observation at each point is completed, display for prompting shifting to a next instrument installation point, such as display with a change in color of a point at which point cloud observation has been completed, and flashing display of an arrow toward a next instrument installation point, is performed. Further, it is also possible that, from the scanner 2, acquired point cloud data is received together with the observation progress information, and the acquired point cloud data is displayed on the display 41. The point cloud data observation is performed while the display is updated each time the observation at each point is completed.
With this configuration, the worker can grasp an observation progress status without any special awareness, and can smoothly move to work at a next point.
The present modification responds to a case where the instrument installation points cannot be set in the order displayed in the observation route 661a in the observation site. When the instrument installation points cannot be set in the order in the displayed observation route 661a, the observation route recalculating unit 603 installs the scanner 2 at a different point and acquires installation information of the scanner 2, and by defining this point as a start point, recalculates the setting order of instrument installation points.
With the configuration described above, it becomes possible to respond to the case where instrument installation points cannot be set in the order in an initial observation route.
Although the information processing device and the storage device are configured as the processing PC 6 in the embodiment, as a still another modification, the information processing device and the storage device may be configured as follows.
(1) The eyewear storage unit 47 of the eyewear device 4 may be caused to function as the storage device, and the eyewear arithmetic processing unit 43 may be caused to function as the information processing device. This configuration is realized by an improvement in performance and downsizing of a storage medium and a control unit constituting the eyewear storage unit and the eyewear arithmetic processing unit. Accordingly, the configuration of the display system 1 can be simplified.
(2) A server device capable of communicating with the processing PC 6, the scanner 2, and the eyewear device 4 is further provided, and the server device is configured to serve as the storage device and the information processing device. With this configuration, the burden of processing on the processing PC 6 can be reduced, and a processing time can be significantly shortened.
Accordingly, the burden on the PC storage unit 66 of the processing PC 6 can be reduced.
Although a preferred embodiment and the modifications thereof according to the present invention have been described above, the embodiment and the modifications described above are examples of the present invention, and the embodiment and the modifications can be combined with other embodiments and modifications based on the knowledge of a person skilled in the art, and such combined embodiments are also included in the scope of the present invention.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2020-109966 | Jun 2020 | JP | national |
