The present disclosure relates to communication control for data collection.
User terminals, sensors, and in-vehicle systems have been developed, and the number of sensors and the data amount generated from each sensor have been greatly increased. When the data is collected in a conventional communication method, there is a possibility that processing of communication on a reception side may become a bottleneck.
As a high-speed data transfer method, remote direct memory access (RDMA) has been studied. In RDMA, data is transferred by DMA (data is directly transferred between, for example, a peripheral device and a main memory (RAM) without passing through a CPU) from a memory of a local computer to a memory of a different remote computer. For this reason, RDMA does not require CPU processing for data transfer, and thus can avoid the bottleneck on the reception side.
A remote transfer technique using this RDMA has been proposed (see, for example, Non Patent Literature 1). However, in Non Patent Literature 1, it is necessary to set a communication path for each flow in advance in order to transfer data by RDMA in a wide area communication network. Therefore, in a case of collecting data from a large number of data sources, a large amount of network resources is required.
Non Patent Literature 1: “Hitori 1-hacho” de 100 Gbps datsu TCP/IP de cho-teichien (100 Gbps by “1 person 1 wavelength”, ultra-low delay by de-TCP/IP), Nikkei Electronics, 2021-03, P40-47
An object of the present disclosure is to provide a system that does not require a large amount of network resources even in a case of collecting data from a large number of data sources.
A system and a method according to the present disclosure are a system for collecting data from a plurality of data sources to a server and a method executed by the system, in which a path management controller that manages a communication path between the plurality of data sources and the server is included, and the path management controller adjusts a section length of the communication path from each of the plurality of data sources to the server so as to shorten a time for generating the communication path for transmitting the data from the plurality of data sources to the server.
A path management controller device and a method according to the present disclosure are a path management controller device included in a system for collecting data from a plurality of data sources to a server, and a method executed by the path management controller device, in which the path management controller device manages a communication path between the plurality of data sources and the server, and adjusts a section length of the communication path generated at a time of data transmission from each of the plurality of data sources to the server so as to shorten a generation time of the communication path at the time of data transmission from the plurality of data sources to the server.
Each of the devices such as the controller and the data sources of the present disclosure can also be implemented by a computer and a program, and the program can be recorded in a recording medium or provided through a communication network. A program of the present disclosure is a program for causing a computer to be implemented as each functional unit included in the devices according to the present disclosure, and is a program for causing a computer to execute each step included in a method to be executed by the devices according to the present disclosure.
According to the present disclosure, it is possible to provide a system that does not require a large amount of network resources even in a case of collecting data from a large number of data sources.
Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. The present disclosure is not limited to the embodiments to be described below. These embodiments are merely examples, and the present disclosure can be implemented in a form with various modifications and improvements based on the knowledge of those skilled in the art. Note that components having the same reference numerals in the present specification and the drawings indicate the same components.
In the present embodiment, an example in which the transmission control controller 20 and the path management controller 30 are separately arranged is illustrated, but these controllers may be provided in a common device or may be configured by a plurality of devices arranged in a distributed manner. Each device of the present invention can also be implemented by a computer and a program, and the program can be recorded in a recording medium or provided through a communication network.
Each of the network devices 81A to 81E knows to which network device data should be transferred when the data addressed to the server 40 is received from the data sources 10A to 10C. Therefore, even if it is unknown which of the network devices 81 is to be used, the data sources 10A to 10C specify the server 40 and transmit data, whereby the data is transmitted to the server 40. Accordingly, the present disclosure can transfer data in a lossless and broadband manner when a communication path is generated.
The present disclosure proposes a method of allocating a lossless and broadband communication path to the necessary data source 10 at necessary timing in order to implement data transfer by RDMA. In the system of the present disclosure, when data to be transmitted to the server 40 is generated, each of the plurality of data sources 10 generates a communication path using the network device 81 set in the local device at transmission timing set in a transmission policy of the local device. As a result, each data source 10 can autonomously transmit the sensing data to the server 40 on the basis of the transmission policy of the local device.
The transmission policy includes the following examples.
First transmission policy: A communication path is generated and data transfer is performed immediately after data generation, and the communication path is released immediately after data transfer completion.
Second transmission policy: A communication path is generated and data transfer is performed after a certain amount of data is accumulated, and the communication path is released immediately after data transfer completion. Third transmission policy: A communication path is generated and data transfer is performed immediately after data generation, and the communication path is continuously held for a certain period of time after data transfer completion.
Fourth transmission policy: A communication path is generated and data transfer is performed after a certain amount of data is accumulated, and the communication path is continuously held for a certain period of time after data transfer completion.
In the present embodiment, with the provision of the transmission control controller 20, the transmission policy that matches a data transfer requirement of the data source 10 is distributed. Then, each data source 10 autonomously transmits the sensing data according to the transmission policy in consideration of the sensor characteristic and the data transfer requirement for each application. As a result, the present embodiment controls trade-off between immediacy of data transmission and reduction in communication path use time.
When the communication path is established between the data source 10 and the server 40, communication quality can be secured and data transfer with high reliability can be performed, but it takes time to generate the communication path. Therefore, in the present disclosure, the path management controller 30 adjusts the length of each communication path so as to shorten the generation time of the communication path. Details will be described below.
The data reception function 41 receives sensing data from each data source 10.
The memory 42 stores sensing data from a sensor 50.
The application 43 collects the sensing data from the sensor 50. The application 43 is any application that collects any sensing data detected or generated by a user terminal, a sensor, an in-vehicle system, or the like.
The generated data storage function 17 stores the sensing data from the sensor 50.
The requirement notification function 11 reads the data transfer requirement from the requirement table 18 and notifies the transmission control controller 20 of the read data transfer requirement.
The transmission policy reception function 12 receives the transmission policy from the transmission control controller 20.
The data transmission timing control function 13 controls the communication path setting function 14, the data transmission function 15, and the communication path release function 16 in accordance with the transmission policy table 19.
The communication path setting function 14 transmits a connection request of a path to the path management controller 30.
The data transmission function 15 transmits the sensing data stored in the generated data storage function 17 to the server 40.
The communication path release function 16 transmits a release request of a path to the path management controller 30.
The requirement table 18 stores the data transfer requirement for each data source 10.
The transmission policy table 19 stores a policy of when the sensing data is transmitted from the data source 10 to the server 40.
Each data source 10 notifies the transmission control controller 20 of the data transfer requirement on the basis of the information of the requirement table 18 (S101). The data transfer requirement includes the allowable delay time and the data generation frequency of each data source 10 as illustrated in
The requirement reception function 21 of the transmission control controller 20 receives the data transfer requirement from the data source 10. The transmission policy determination function 22 determines the transmission policy for each data source 10 according to the data transfer requirement in accordance with the transmission policy determination rule 24 (S102).
The transmission policy distribution function 23 of the transmission control controller 20 distributes the determined transmission policy to each data source 10 (S103).
When acquiring data transfer requirement from one of the data source 10 (S11), the transmission control controller 20 determines a requirement for the allowable delay time (S12). In a case where the requirement for the allowable delay time is 100 ms or less (Yes in S12), a method of generating a path and transmitting the data immediately after data generation is determined (S13). On the other hand, in a case where the requirement for the allowable delay time exceeds 100 ms (No in S12), a method of storing a certain amount of sensing data and transmitting the sensing data is determined (S14).
Next, the transmission control controller 20 determines the data generation frequency (S15). In a case where the data generation frequency is 4 times/s or less (Yes in S15), a method of releasing a path immediately after data transmission is determined (S16). On the other hand, in a case where the data generation frequency exceeds 4 times/s (No in S15), a method of waiting for a certain period of time after data transmission completion and releasing a path is determined (S17).
For example, in the case of the data transfer requirement illustrated in
Since a set time of the allowable delay time is 1000 ms, and a set value of the data generation frequency is 4 times, the transmission control controller 20 determines the second transmission policy as the transmission policy.
Since a set time of the allowable delay time is 200 ms, and a set value of the data generation frequency is 10 times, the transmission control controller 20 determines the fourth transmission policy as the transmission policy.
Since a set time of the allowable delay time is 100 ms, and a set value of the data generation frequency is 5 times, the transmission control controller 20 determines the third transmission policy as the transmission policy. As a result, the transmission policies as illustrated in
When receiving the transmission policy from the transmission control controller 20, the data source 10 stores the transmission policy in the transmission policy table 19. As a result, as illustrated in
Here, the data transfer requirement may include application information. In this case, the transmission control controller 20 sets a threshold in consideration of the requirements included in the application information in steps S12 and S15. Further, the transmission policy determined by the transmission policy determination function 22 is transmitted to the path management controller 30.
The path setting request reception function 31 receives the connection request or the release request of the communication path from each data source 10.
The path setting function 32 generates or releases the communication path in accordance with the connection request or the release request of the communication path from each data source 10.
The path setting location table 33 manages setting information (generation/release) of the communication path from each data source 10 to the server 40.
The path design function 34 designs each communication path so that the generation time of the communication path is shortened in a case where the transmission policy from the transmission control controller 20 is executed. The design here is reflected in the path setting location table 33.
The path management table 35 manages the data amount required to execute the transmission policy from the transmission control controller 20 for each path.
In the present embodiment, according to the transmission policy of the data source 10A, data is transmitted after 15 MB of data is accumulated. Since the data generation frequency of the data source 10A is 4 times/s, 15 MB of data is transmitted at a frequency of 1 time/s. Therefore, as illustrated in
The path design function 34 designs the communication path on the basis of the information in the path management table 35. For example, the path design function 34 determines resources to be allocated to each communication path. Here, in a case where a section length of the communication path is short, the time required to generate the communication path becomes short. Therefore, the section length is shortened for a section with a high generation frequency of the communication path. As a result, the path generation time is shortened, and thus data can be immediately transmitted at the time of data generation.
The path design function 34 lengthens the section length for a section with a low generation frequency of the communication path. Since this communication path has a low data generation frequency and a low possibility of contention, the communication path may be exclusively used in a plurality of flows. As a result, new path generation can be reduced, and the generation time of the communication path can be shortened. In addition, the data amount of the communication path may be able to be adjusted depending on the communication path. In such a case, the path design function 34 may allocate a large number of resources to the communication path through which a flow having a large generated data amount passes.
The generation frequency and the data amount of the communication path are read from the path management table 35 (S31), and whether the generation frequency of the communication path is a preset threshold (6 times/s in the present embodiment) or more (S32). In a case where the generation frequency is 6 times/s or more, the section length is maintained (S33). On the other hand, in a case where the generation frequency is less than 6 times/s, the communication path is extended by one to the next section (S34).
Whether the data amount of the communication path added in step S33 or S34 is a preset threshold (1 Gbps in the present embodiment) or more (S35). Then, in a case where the data amount is 1 Gbps or more (Yes in S35), the allocated resources of the added communication path are increased (S36).
For example, since the generation frequency of the communication path of the data source 10B is 10 times/s, step S33 is executed to divide the communication path by the network devices 81C and 81A. Meanwhile, since the generation frequency of the communication path of the data source 10C is 5 times/s, step S34 is executed to extend the communication path from the network device 81E to the network device 81D to the communication path from the network device 81E to the server 40. Further, at this point of time, there is no communication path with the data amount of 1 Gbps or more. As a result, the communication path as illustrated in
In the present embodiment, the sensing data from the data sources 10A and 10B join at the network device 81A. In this case, the network device 81A divides the communication path. For the communication path from the network device 81A to the server 40, the data sources 10A and 10B may individually generate communication paths, but may generate a communication path to use in cooperation in advance. In this case, when a data source that uses the communication path appears, the communication path is allocated to the data source.
An example of setting the communication path in a case where the data sources 10A to 10C transmit the sensing data will be described with reference to
The data source 10B accumulates the sensing data generated in the sensor 50, and transmits the connection request of the communication path to the path management controller 30 when the data reaches 20 MB (S201).
When receiving the connection request of the communication path, the path management controller 30 sets the communication paths of the network devices 81A and 81C and generates the communication paths (S202). When the generation of the communication paths is completed, the path management controller 30 transmits a communication path generation completion notification to the data source 10B (S203).
When receiving the communication path generation completion notification, the data source 10B transmits the sensing data to the server 40 (S204-1 and S204-2). Transmission of the release request of the communication path is waited for subsequent 50 ms. When sensing data is generated during that time, the sensing data is transmitted each time generated (S204-3).
When 50 ms elapses from the last data transmission (step S204-3), the data source 10B transmits the release request of the communication paths to the path management controller 30 (S205).
When receiving the release request of the communication path, the path management controller 30 releases the communication paths of the network devices 81A and 81C, and transmits a communication path release completion notification indicating that the release of the communication paths is completed to the data source 10B (S206).
The data source 10A accumulates the sensing data generated in the sensor 50, and transmits the connection request of the communication path to the path management controller 30 when the data reaches 15 MB (S301).
When receiving a generation request of the communication path, the path management controller 30 generates the communication path from the network device 81E to the server 40 (S302). When the generation of the communication path is completed, the path management controller 30 transmits the communication path generation completion notification to the data source 10A (S303).
When receiving the communication path generation completion notification (S303), the data source 10A collectively transmits the accumulated sensing data to the server 40 (S304-1, S304-2, and S304-3). Thereafter, the data source 10A immediately transmits the release request of the communication path to the path management controller 30 (S305).
When receiving the release request of the communication paths, the path management controller 30 releases the communication paths of the network devices 81A and 81B, and transmits the communication path release completion notification indicating that the release of the communication paths is completed to the data source 10A (S306).
When data is generated in the sensor 50, the data source 10C transmits the generation request of the communication path to the path management controller 30 (S401).
When receiving the communication path generation completion notification (S403), the data source 10C transmits the sensing data to the server 40 (S404-1), and then waits for transmission of the release request of the communication path for the subsequent 20 ms. When sensing data is generated during that time, the sensing data is transmitted each time the sensing data is generated (S404-2 and S404-3).
When 20 ms elapses from the last data transmission (step S404-3), the data source 10C transmits the release request of the communication paths to the path management controller 30 (S405).
When receiving the release request of the communication paths, the path management controller 30 releases the communication paths of the network devices 81D and 81E, and transmits the communication path release completion notification indicating that the release of the communication paths is completed to the data source 10C (S406).
As illustrated in
The path management controller 30 has a function to monitor the data generation amount of each data source 10 and a generation status of the communication path, and may change the design of the communication path according to a monitoring result. In this case, when acquiring the data amount of each communication path, the path management controller 30 executes steps S31 to S36 again. The monitoring method is arbitrary, and for example, providing a counter for recording the data amount in the network devices 81A to 81E, and reading a value of the counter can be illustrated as an example.
For example, in a case where the generation frequency of the communication path of the data source 10C increases from 5 times/s to 10 times/s, the communication path is divided by the network devices 81E to 81D as illustrated in
In the present embodiment, a transmission control controller 20 determines a transmission policy in consideration of entire NW resources. When the transmission policy is determined, a resource use amount of a case where each of the data sources A to C executes the transmission policy can be calculated. Although a method of calculating the resource use amount is arbitrary, for example, the following equations can be used.
The transfer time=a transfer data amount/a communication band
The path generation frequency=the data generation frequency*the data generation amount/an accumulation amount Here, the path generation overhead is a head portion indicating the connection request of a communication path, and may be any value. In the present embodiment, an example of 0.03 s is given as an example. The path release standby time corresponds to path release timing in the transmission policy.
It is the transmission policy in which the data generation frequency is 4 times/s, the data generation amount is 5 MB/time, and the communication path is generated after 15 MB is accumulated.
The path generation frequency: 1.3 times/s=(5 [MB/time]×4 [times/s])/15 [MB] The transfer time: 0.012 s=15 [MB]/(10 [Gb/s]/8) The resource use amount: 0.056=(0.03+0.012+0)*1.3
It is the transmission policy in which the data generation frequency is 10 times/s, the data generation amount is 10 MB/time, the communication path is generated after 20 MB is accumulated, and release of the communication path is waited for 0.05 s.
The path generation frequency: 5 times/s=(10 [MB/time]×10 [times/s])/20 [MB] The transfer time: 0.016 s=20 [MB]/(10 [Gb/s]/8) The resource use amount: 0.48=(0.03+0.016+0.05)*5
It is the transmission policy in which the data generation frequency is 5 times/s, the data generation amount is 10 MB/time, the communication path is immediately generated, and release of the communication path is waited for 0.02 s.
The path generation frequency: 5 times/s
The transfer time: 0.008 s=10 [MB]/(10 [Gb/s]/8)
The resource use amount: 0.29=(0.03+0.008+0.02)*5
In a case where the resource use amounts of the respective data sources are 0.056, 0.48, and 0.29, a total resource use amount of a case where the data sources A, B, and C execute the transmission policies is 0.826.
The transmission policy determination function 22 adjusts the transmission policy with reference to the network resource DB 25.
The resource use amount is calculated using the set transmission policy (S21).
Next, the network resources are read from the network resource DB 25, and the total resource use amount is calculated (S22).
Next, whether the total resource use amount falls within a set threshold is determined (S23).
In a case where the total resource use amount falls within the set threshold (Yes in S23), the determination of the set transmission policy is terminated.
On the other hand, in a case where the total resource use amount does not fall within the set threshold (No in S23), the transmission policy of each data source is adjusted such that the total resource use amount becomes equal to or less than the set threshold. Specifically, in order of looseness of the transmission policy (S24), path release timing is advanced (S25), or the accumulation amount at the path generation timing is increased (S26). This can reduce a time in which the data source 10 with a loose transmission policy consumes network resources, and reduce the total resource use amount.
According to the requirement table of
A case where the set threshold is 0.8, and the adjustment is performed by reducing the path release standby time of the transmission policy by 10% in step S25, and increasing the data accumulation amount by one time of data generation amount in step S26 will be described in detail.
Since the path release timing of the data source A is after data transmission completion and the path release standby time is 0, step S26 is executed. In step S26, the data accumulation amount is increased by 5 MB corresponding to one time of data generation amount. As a result, as illustrated in
The transmission policy of the data source A is the transmission policy of generating the communication path after 20 MB is accumulated. The resource use amount of the data source A at this time can be calculated as follows. The path generation frequency: 1.0 times/s=(5 [MB/time]×4 [times/s])/20 [MB] The transfer time: 0.016 s=20 [MB]/(10 [Gb/s]/8) The resource use amount: 0.046=(0.03+0.016+0)*1.0
The transmission policy determination function 22 calculates the total resource use amount of the case where the data sources A, B, and C execute the transmission policies on the basis of the transmission policy of the data source A after adjustment, and compares the total resource use amount with a threshold. For example, since the resource use amount of the data source A after adjustment is 0.046, the total resource use amount of the case where the data sources A, B, and C execute the transmission policies is 0.816. Since the set threshold is 0.8, the transmission policy of the data source B with the next longest allowable delay time is adjusted.
Since the path release timing of the data source B is release when there is no data generation for 50 ms, step S25 is executed. In step S25, the path generation time is shortened by 5 ms, which is 10% of 50 ms. As a result, as illustrated in
The transmission policy of the data source B is the transmission policy of releasing the path when there is no data generation for 45 ms. The resource use amount of the data source B at this time is 0.455=(0.03+0.016+0.455)*5.
The transmission policy determination function 22 calculates the total resource use amount of the case where the data sources A, B, and C execute the transmission policies on the basis of the transmission policy of the data source B after adjustment, and compares the total resource use amount with a threshold. For example, since the resource use amount of the data source B after adjustment is 0.455, the total resource use amount of the case where the data sources A, B, and C execute the transmission policies is 0.719. Since the set threshold is 0.8 and the calculated value is below the threshold, the adjustment is completed.
In this manner, the transmission policy determination function 22 repeats steps S24 to S26 until the total resource use amount falls within the set threshold. Here, the processing in step S25 may be executed for the transmission policy in which the path generation timing is immediately after data generation, and step S26 may be executed for other transmission policies. Further, step S25 may be further executed for the transmission policy in which step S26 has been executed.
When the total resource use amount falls within the set threshold (Yes in S23), the transmission policy determination function 22 terminates the determination of the set transmission policy. When the determining of the transmission policy in the transmission policy determination function 22 ends, the transmission policy distribution function 23 distributes the transmission policy to each data source 10.
When receiving the transmission policy from the transmission control controller 20, the data source 10 stores the transmission policy in the transmission policy table 19. As a result, as illustrated in
As described above, the present disclosure determines the transmission policy for each data source 10 and generates the communication path for each data source 10 in accordance with the transmission policy, and thus can allocate the communication paths in a time-division manner at necessary timing. Therefore, the present disclosure can perform control to reduce the communication path use time from each data source 10 to the server 40, facilitate reuse of the communication paths, and reduce the required number of communication paths for the entire communication network 80 with respect to the trade-off between immediacy of data transmission and reduction in communication path use time.
Therefore, the present disclosure can implement the lossless and broadband communication network 80 with the communication paths, and enable RDMA communication with a large number of data sources without requiring a large amount of network resources. Furthermore, the present disclosure can prevent data loss in the communication network 80, and thus can implement reliable data transfer using RDMA.
Note that, in the present embodiment, an example in which data collected by the server 40 is sensing data has been described. However, the present disclosure is applicable to any data required to be collected, such as data in user terminals, sensors, or in-vehicle systems.
Furthermore, in the above-described embodiments, an example of setting the lossless and broadband communication path between the data source 10 and the server 40 has been described. However, a transfer route of data of which quality is not guaranteed may be set as the communication path.
The present disclosure can be applied to the information communication industry.
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/JP2022/007873 | 2/25/2022 | WO |