Patent Application
20240028750
The present invention relates to a managing system of an edge device in a blockchain network.
In recent years, data generation using deep learning techniques, such as a generative adversarial network (GAN), has become possible, and it is demanded to ensure the authenticity of data. WO 2020/079766 discloses a method for preventing the alteration of data in a distributed ledger identity verification system by recording acquired data in an edge device in a distributed ledger.
In the identity verification system according to WO 2020/079766, the identity verification is performed by extracting feature values of a target, and if it is determined that the identity is verified, the acquisition time and location are recorded in the distributed ledger, so as to prevent alteration.
However in the technique disclosed in WO 2020/079766, alteration of the acquired data itself cannot be prevented, and the authenticity of data, correct linking of feature values and acquired data, and the like may not be ensured.
With the foregoing in view, it is an object of a technique of the present disclosure to ensure the recognition result using data in an edge device constituting a blockchain network and the authenticity of the data.
According to some embodiments, a management system including an edge device, and a blockchain network connected to the edge device, wherein the edge device includes a processor, and a memory storing a program which, when executed by the processor, causes the edge device to execute recognition processing on acquired image data, hold attribute information related to capturing of the image data, and convert the image data into a hash value, and the management system links, as linked data, the hash value converted by the edge device, the result of the recognition processing by the edge device, and the attribute information held by the edge device with one another, and registers the linked data in a distributed ledger in the blockchain network.
According to some embodiments, a control method for a management system of an edge device connected to a blockchain network, including a recognition step of causing the edge device to execute recognition processing on acquired image data, a holding step of causing the edge device to hold attribute information related to capturing of the image data, a conversion step of causing the edge device to convert the image data into a hash value, and a registration step of causing the management system to link, as linked data, the hash value converted in the conversion step, the result of the recognition processing in the recognition step, and the attribute information held in the holding step with one another, and register the linked data in a distributed ledger in the blockchain network. In addition, according to some embodiments, a non-transitory computer-readable storage medium which stores programs causing a computer to execute a control method for a management system of an edge device connected to a blockchain network, wherein the control method includes a recognition step of causing the edge device to execute recognition processing on acquired image data, a holding step of causing the edge device to hold attribute information related to capturing of the image data, a conversion step of causing the edge device to convert the image data into a hash value, and a registration step of causing the management system to link, as linked data, the hash value converted in the conversion step, the result of the recognition processing in the recognition step, and the attribute information held in the holding step with one another, and register the linked data in a distributed ledger in the blockchain network.
Further features of the present invention will become apparent from the following description of exemplary embodiments with reference to the attached drawings.
Embodiments of the present disclosure will be described with reference to the drawings. The present disclosure, however, is not limited to the following embodiments, and may be appropriately changed within a scope of not departing from the spirit thereof In the drawings to be described below, composing elements having a same function are denoted with a same reference sign, and description thereof may be omitted or simplified in some cases.
A management system of each embodiment will be described with reference to the drawings. A management system according to Embodiment 1 is a management system that is configured assuming that a visual inspection is performed on a product (object) at a production plant. In Embodiment 1, means for acquiring image data on a product is a camera, for example, and an edge device performs determination processing to determine whether a product is non-defective/defective using image data on an image of the product captured by the camera.
The image data acquired by the imaging unit 100 is converted into a hash value by the hash conversion unit 120, which is conversion means for converting the image data into a hash value. This conversion processing is processing to be executed to prevent alteration of the image data and to ensure the authenticity of the image data, using a characteristic of a hash function in which the hash value completely changes if data before the conversion is altered even slightly. For the hash conversion means to prevent alteration of the image data, MD5, SHA-1, SHA-224, SHA-256, SHA-384, SHA-512, SHA3-512, SHAKE128, SHAKE256, or the like is used, for example. The image data acquired by the imaging unit 100 is inputted to the edge recognition unit 110. The edge recognition unit 110 is recognition means for executing recognition processing on the inputted image data using an image recognition algorithm (e.g. machine vision) and artificial intelligence (AI). For the recognition processing, image processing techniques used for visual inspection, such as scoring a defect of a subject in the image data based on such a method as a convolutional neural network (CNN), and detecting a contamination position based on blob processing, can be used. The edge recognition unit 110 outputs the result of the recognition processing for the image data.
For the edge recognition unit 110, a graphic processing unit (GPU), a field programmable gate array (FPGA) or the like is used. In the edge device 10 of Embodiment 1, the imaging unit 100 is configured as the data acquisition unit, but a sensor to acquire data, such as a line sensor and temperature sensor, may be used as the data acquisition unit instead of the imaging unit 100.
The attribute information acquisition unit 150 acquires, for example, attribute information, including the acquisition date/time and acquisition steps, of the image data acquired by the imaging unit 100. Then the hash value of the image data converted by the hash conversion unit 120, the result of the recognition processing by the edge recognition unit 110 and the attribute information acquired by the attribute information acquisition unit 150, are outputted to the transaction registration unit 140. The transaction registration unit 140 links the hash value, the recognition processing result and the attribute information with each other, and register the linked data in the storage unit 170 as the transaction data on the blockchain network 30.
Here the acquired data hash value 411A is a hash value of the image data acquired from the edge device 10. The recognition result 412A includes information to indicate the result of determining whether the work 530 in the image data acquired by the edge device 10 is non-defective or defective. Furthermore, the recognition result 412A includes: information on foreign substance detection result which indicates the presence/absence of a scratch, a foreign substance, or the like, information on coordinates and the seriousness of the scratch, foreign substance, or the like on the work if any; information to indicate the size of the work 530; and digitized information on sensory evaluation (e.g. tinge).
The attribute information 413A is, for example, information acquired from the edge device 10, and is process information including: a management number and production plant name of the visual inspection line 500; a product name and serial number of the work 530, a production lot number; and a shipment destination of the conveying device 520. The process information also includes the information on the inspection machine, inspection date/time, shipment destination of product, and the like. The attribute information 413A is information on capturing of images acquired by the edge device 10, and includes information on various apparatuses used for capturing images, and information on the environment in which images are captured.
The hash value 411A, the recognition result 412A and the attribute information 413A of the series of data acquired by the edge device 10 are combined and registered in the block 40A as the transaction data 410A. Thereby a blockchain of the transaction data is configured.
It is a characteristic of a blockchain that alteration of a block influences subsequent blockchains, hence alteration of a block is difficult. Therefore the traceability of the hash value, the recognition result, and the attribute information of the image data on the work 530 acquired by the edge device 10 is ensured. In Embodiment 1, the acquired data itself is not managed, but the hash value is managed in the block 40A, so as to conserve block capacity. Therefore, the traceability of the image data, the recognition result, and the attribute information is ensured by matching the information included in the block 40A and the information included in the image data which the acquired data saving unit 130 stored separately. To match the information, the inspection data/time information and the like included in the attribute information 413 A can be used. In this case, if unique metadata is embedded in a head portion or the like of the image data, as indicated in
As described above, according to the management system 1 of Embodiment 1, the traceability and authenticity of the image data in the visual inspection of the work are ensured, whereby the evidential capability of information on problems related to the shipment of defective products can be enhanced.
A management system according to Embodiment 2 will be described next. In the following description, a composing element the same as Embodiment 1 is denoted with a same reference sign, and detailed description thereof will be omitted. The management system according to Embodiment 2 is a management system assuming an autonomous driving system of automobiles. In Embodiment 2, it is assumed that a distance sensor of a camera, an LiDAR, a radar sensor, or the like is used as means for acquiring data, for example, and recognition processing is performed on the acquired data by the edge device 10, and the recognition result is used for autonomous driving.
The transaction registration unit 140 links the recognition result by the edge recognition unit 110, vehicle operation result including steering and acceleration/deceleration performed by the vehicle operation unit 180, hash values of the acquired data by the sensors, and attribute information acquired by the attribute information acquisition unit 150. Then the transaction registration unit 140 registers the linked data in the storage unit 170 of the edge device 10 as the transaction data on the blockchain network 30.
As indicated in
The transaction registration unit 140 stores information acquired by the edge device 10A, such as speed information of the vehicle, GPS coordinates, time, traffic control information, weather information and the vehicle information, in the storage unit 170 as the attribute information 653 of the transaction data 650. The recognition result 652 and the attribute information 653 registered in this way, along with the hash value 651 of the image data acquired by the imaging unit 100, are registered in a block as the transaction data 650.
Just like Embodiment 1, the block registered in the edge device 10A forms a series of blockchains as mentioned above. Thereby the traceability and authenticity of the transaction data, including the image data and the recognition result acquired by the edge device, are ensured. According to the management system 1 of Embodiment 2, the traceability and authenticity of the image data and recognition result, which are critical for autonomous driving of a vehicle, are ensured, hence the evidential capacity of information in an accident caused by the autonomous driving vehicle can be enhanced.
A management system according to Embodiment 3 will be described next. In the following description, a composing element the same as Embodiments 1 and 2 is denoted with the same reference sign, and detailed description thereof will be omitted. The management system according to Embodiment 3 is a management system constructed assuming a monitoring camera system and an access management system. In the management system 1 of Embodiment 3, a plurality of edge devices constituting the blockchain network 30 perform the recognition processing in cooperation, whereby the recognition accuracy of the data recognition processing can be enhanced.
As indicated in the example in
The transaction registration unit 140 registers the installation information that can be acquired by the edge devices 10B and 10C, such as installation location of each edge device, image capturing time, and ID to uniquely identify the monitoring camera, in the storage unit 170 as the attribute information 713 of the transaction data 710. The recognition result 712 and the attribute information 713 registered in this way are registered, along with a hash value 711 of the image data acquired by the imaging unit 100, as the transaction data 710.
Therefore according to Embodiment 3, accuracy of the personal recognition result and access records of the monitoring cameras performed by the edge devices 10B and 10C and evidential capacity thereof can be enhanced.
A management system according to Embodiment 4 will be described next. In the following description, a composing element the same as Embodiments 1 to 3 is denoted with the same reference sign, and detailed description thereof will be omitted. The management system according to Embodiment 4 is a management system constructed assuming a case of using the edge drive as a digital camera for general consumers. Specifically, in a management system 200 according to Embodiment 4, the edge device 10 forms a public blockchain network 31. A difference from the management system 1 of Embodiment 1 is that the management system 200 performs Proof of Work (PoW) or Proof of Stake (PoS). The edge device 10 may include a microphone device and use a moving image capturing function, just like a standard digital camera, and in the following description, it is assumed that a moving image is captured by the edge device 10.
In this description, it is assumed that the edge device 10 is a digital camera that can capture moving image data, but the edge device 10 may be a smartphone, a tablet, or the like. The data acquired by the edge device 10 is not limited to the moving image data, and the edge device 10 may be Internet of Things (IoT) sensor or a microphone having a configuration different from the imaging unit of the edge device 10.
The edge device 10 links a hash value of moving image data based on the captured moving image, the recognition result and the attribute information, and registers the linked data in the storage unit 170 of the edge device 10 as transaction data on the public blockchain network 31. In Embodiment 4, the public blockchain network 31 is used as the blockchain network. Therefore, the edge device 10 ensures the authenticity of the moving image data by executing processing using such a consensus algorithm as Proof of Work (PoW) and Proof of Stake (PoS). It is also possible that the edge device 10 itself performs mining as a node device, and ensure the authenticity of the moving image data thereby.
The hash value of the moving image data of the moving image captured by a digital camera, which is the edge device 10, is included in the transaction data 810A as an acquired data hash value 811A. Specifically, the hash conversion unit 120 converts image data and audio data included in a moving image in a predetermined period, such as predetermined seconds or minutes, out of the captured moving image, into a hash value, and the converted hash value becomes the acquired data hash value 811A. A person recognition result and an audio recognition result, which are the result of the recognition processing which the edge device 10 executed using the moving image data, are included in the transaction data 810A as a recognition result 812A. Furthermore, a product ID and owner information of the digital camera, which is the edge device 10, and attribute information of the camera, such as location information to indicate the moving image capturing location, and the moving image capturing date and time, are also included in the transaction data 810A as camera information 813A.
For example, in an interview of a celebrity, a hash value of a moving image data of a moving image captured by the edge device 10, a person recognition result, audio recognition result, location information and image capturing data and time when the moving image was captured, owner information of the camera, and the like, are managed by the public blockchain network 31. Thereby the authenticity of the moving image data is ensured, and ownership of the moving image data can be verified. Further, the edge device 10 may link the transaction data and the moving image data of the captured moving image, whereby a non-fungible token (NFT) is issued, and ownership of the moving image data can be verified.
Therefore according to the management system 200 of Embodiment 4, authenticity of the moving image data of the moving image captured by the digital camera, which is the edge device 10, is ensured, and ownership thereof is verified, whereby the digital copyright of the moving image data can be protected.
Whereas the management system of the present disclosure has been described in detail based on the preferred embodiments thereof, the present disclosure is not limited to these specific embodiments, and various other modes within a scope not departing from the technical spirit of the present disclosure are also included in the present disclosure. The plurality of embodiments described above may be combined when necessary.
For example, in the above embodiments, the edge device is configured as a node of the blockchain network, but an electronic apparatus, to store the transaction data, may be used as the node of the blockchain network instead of the edge device. In this case, in the above embodiments, the edge device need not include the storage unit, and may be configured as an external electronic apparatus of the blockchain network. In other words, in the above embodiments, the camera may be configured as a node of the blockchain network, or as an external electronic apparatus of the blockchain network. Furthermore, in the above embodiment, a target to be converted into a hash value included in the transaction data may be image data acquired by the edge device or may be moving image data.
Embodiment(s) of the present invention can also be realized by a computer of a system or apparatus that reads out and executes computer executable instructions (e.g., one or more programs) recorded on a storage medium (which may also be referred to more fully as a ‘non-transitory computer-readable storage medium’) to perform the functions of one or more of the above-described embodiment(s) and/or that includes one or more circuits (e.g., application specific integrated circuit (ASIC)) for performing the functions of one or more of the above-described embodiment(s), and by a method performed by the computer of the system or apparatus by, for example, reading out and executing the computer executable instructions from the storage medium to perform the functions of one or more of the above-described embodiment(s) and/or controlling the one or more circuits to perform the functions of one or more of the above-described embodiment(s). The computer may comprise one or more processors (e.g., central processing unit (CPU), micro processing unit (MPU)) and may include a network of separate computers or separate processors to read out and execute the computer executable instructions. The computer executable instructions may be provided to the computer, for example, from a network or the storage medium. The storage medium may include, for example, one or more of a hard disk, a random-access memory (RAM), a read only memory (ROM), a storage of distributed computing systems, an optical disk (such as a compact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™), a flash memory device, a memory card, and the like.
According to the present disclosure, the authenticity of data acquired by an edge device connected to a blockchain network can be ensured.
While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.
This application claims the benefit of Japanese Patent Application No. 2022-116659, filed on Jul. 21, 2022, which is hereby incorporated by reference herein in its entirety.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2022-116659 | Jul 2022 | JP | national |
