The present invention relates to a data transfer apparatus and a data transfer method.
A structured P2P (Peer to Peer) network has been attracting attention as a network for realizing high-speed data transfer between nodes. In a data distribution system using the structured P2P network, a plurality of data transfer apparatuses cooperate with each other to realize high-speed distribution of large quantities of data. Data to be distributed are input from a data transmitter to any of the data transfer apparatuses configuring the data distribution system and are distributed from the data transfer apparatus or another data transfer apparatus to a data receiver.
In such data distribution system, each of the data transfer apparatuses can communicate with the other data transfer apparatuses via a network connected by a transmission medium such as a LAN (Local Area Network) cable. A plurality of data transmitters can transmit data to any of the data transfer apparatuses via the network. In the data distribution system, the data transfer apparatuses cooperate with each other to distribute the data, which are received from the data transmitters, to a data receiver that is in charge of processing the data. Therefore, each data transfer apparatus previously stores therein the addresses of some of the other data transfer apparatuses, and determines an address to transfer the data to, from among the group of stored addresses, in accordance with a predetermined procedure, so that the data eventually reach the data receiver that is in charge of processing the data.
In the system disclosed in Non-Patent Document 1, the ID of each node (data transfer apparatus) is configured by a value (key) of m-bit length. Each data transfer apparatus transfers data, which is received from a data transmitter or another data transfer apparatus, to a successor corresponding to a hash value (key) of the data. A successor corresponding to a key k is one of the data transfer apparatuses configuring the data distribution system, the ID of which indicates the key closest to the key k.
According to Non-Patent Document 1, key (m bits) is expressed by a factor ring modulo 2m, and a key that is distant from any key k by d is expressed as (k+d)mod 2m, as shown in
In the system disclosed in Non-Patent Document 1, each data transfer apparatus holds the addresses of some of the other data transfer apparatuses in order to transfer the data to these data transfer apparatuses. These addresses correspond to the addresses of the successors having keys s (base point keys) that are distant from the ID of the data transfer apparatus by 2i−1 (i=1, 2, . . . , m). According to the example shown in
According to the configurations shown in
Non-Patent Document 1: Ion Stoica, and three others, “Chord: A Scalable Peer-to-peer Look up Service for Internet Applications,” ACM SIGCOMM Computer Communication Review 31(4), U.S.A., ACM Press, 2001, p. 149-160
Patent Document 1: Patent Publication JP-A-2007-328813
Incidentally, in the configurations shown in
An example of a deadlock is now described. Suppose that large quantities of data are input from the data transmitter to the data transfer apparatus 0. For example, the transfer unit of the data transfer apparatus 0 determines to transfer the data received from the data transmitter, to the data transfer apparatus 1, and inserts the data into the transmission queue s0-3. In so doing, suppose that the process for extracting the data from the transmission queue s0-3 in order to allow the data transfer apparatus 0 to transmit the data to the data transfer apparatus 1 is performed later than the process for inserting the data from the transfer unit of the data transfer apparatus 0 into the transmission queue s0-3, and that the transmission queue s0-3 is filled with data. In such a case, a back pressure is applied from the transmission queue to the transfer unit, preventing the transfer unit from extracting the data from the receiving queue. Consequently, the receiving queue r0-3 becomes too full to hold the data received from the data transfer apparatus 1, and a back pressure is applied from the data transfer apparatus 0 to the data transfer apparatus 1.
On the other hand, suppose that large quantities of data are input from a data transmitter to the data transfer apparatus 1. For instance, the transfer unit of the data transfer apparatus 1 determines to transfer the data received from the data transmitter, to the data transfer apparatus 0, and inserts the data into the transmission queue s1-1. In this case, as with the case of the data transfer apparatus 0, the receiving queue r1-1 becomes filled with data, and consequently a back pressure is applied from the data transfer apparatus 1 to the data transfer apparatus 0. In such circumstances, the data transfer apparatus 0 and the data transfer apparatus 1 apply a back pressure to each other and remain unable to transmit data. In other words, a deadlock state occurs.
Patent Document 1 discloses an example of a method for avoiding the occurrence of deadlocks. In the method disclosed in Patent Document 1, a message communication buffer, large enough for a node corresponding to a data transfer apparatus to constantly process input messages, is secured for the purpose of avoiding a deadlock. This method relates to increasing the sizes of the transmission queues and receiving queues shown in
According to the method disclosed in Patent Document 1, however, the size of the message communication buffer is proportionate to the number of messages that are likely to be held unprocessed. When the number of messages that are likely to be held unprocessed is extremely high, the size of the message communication buffer needs to be extremely large, which is not efficient in terms of the utilization of the memory resources. In addition, when not being able to specify the number of messages that are likely to be held unprocessed, an unlimited number of regions need to be provided for the message communication buffer, which is not unrealistic.
The present invention was contrived in view of such circumstances, and an object thereof is to prevent the occurrence of deadlocks regardless of the size of a message communication buffer in a structured P2P network or without securing a large message communication buffer.
A data transfer apparatus according to one aspect of the present invention has: a routing table storage unit configured to store a routing table including an identifier, a first address and a second address of one or more other data transfer apparatuses connected to one another via a network; a first transfer unit configured to: once receiving data sent to a first address of the own apparatus (data transfer apparatus), determine another data transfer apparatus as a data transfer destination, based on a relationship between a key calculated from the data and identifiers of the one or more other data transfer apparatuses included in the routing table; and
transmit the data to a first address of the another data transfer apparatus when a magnitude relationship between the identifier of the own apparatus and the identifier of the another data transfer apparatus is in one condition, and transmit the data to a second address of the another data transfer apparatus when the magnitude relationship is in the other condition; and a second transfer unit configured to: once receiving data sent to a second address of the own apparatus, determine another data transfer apparatus as a data transfer destination, based on a relationship between a key of the data and identifiers of the one or more data transfer apparatuses included in the routing table, and transmit the data to a second address of the another data transfer apparatus.
A data transfer method according to one aspect of the present invention for causing each of a plurality of data transfer apparatuses connected to one another via a network in a data distribution system, to perform: storing a routing table including an identifier, first address and second address of one or more other data transfer apparatuses; once receiving data sent to a first address of an own apparatus: determining a data receiver or another data transfer apparatus as a data transfer destination, based on a relationship between a key calculated from the data and the identifier of the one or more data transfer apparatuses included in the routing table; transmitting the data to the data receiver when the data receiver is determined as the data transfer destination; transmitting the data to the first address of the another data transfer apparatus when the another data transfer apparatus is determined as the data transfer destination and when a magnitude relationship between an identifier of the own apparatus and the identifier of the another data transfer apparatus is in one condition; and transmitting the data to the second address of the another data transfer apparatus when one the magnitude relationship is in the other condition; once receiving data sent to a second address of the own apparatus: determining the data receiver or another data transfer apparatus as a data transfer destination, based on a relationship between a key calculated from the data and the identifier of the one or more data transfer apparatuses included in the routing table; transmitting the data to the data receiver when the data receiver is determined as the data transfer destination; and transmitting the data to the second address of another data transfer apparatus when the another data transfer apparatus is determined as the data transfer destination.
Note that the term “unit” described in the present invention does not only represent physical means but also connotates means for realizing its function by means of software. A function of a single “unit” or apparatus may be realized by two or more physical means or devices, and functions of two or more “units” or apparatuses may be realized by single physical means or device.
The present invention can prevent the occurrence of deadlocks regardless of the size of a message communication buffer in a structured P2P network or without securing a large message communication buffer.
An embodiment of the present invention is described hereinafter with reference to the drawings.
==Configuration==
The first and second receiving queues 40 and 42 each have a memory region for temporarily holding data received from the data transmitter or another data transfer apparatus. Note that the data transfer apparatus 20 has a first address and second address. Data transmitted to the first address is stored in the first receiving queue 40, and data transmitted to the second address is stored in the second receiving queue 42.
The first and second transmission queues 44 and 46 each have a memory region for temporarily holding data to be transmitted to another data transfer apparatus or the data receiver. Note that the data stored in the first and second transmission queues 44 and 46 are sequentially transmitted to a transfer destination determined by the first transfer unit 50 or the second transfer unit 52.
The first transfer unit 50 extracts the data from the first receiving queue 40 and determines another data transfer apparatus or the data receiver as a transfer destination to transfer the data to, based on a routing table stored in the routing table storage unit 62. The first transfer unit 50 then transfers the data to a transmission queue that transmits the data to the determined transfer destination. Similarly, the second transfer unit 52 extracts the data from the second receiving queue 42 and determines a transfer destination to transfer the data to, based on the routing table. The second transfer unit 52 then transfers the data to a transmission queue that transmits the data to the determined transfer destination. In the present embodiment, the first and second transfer units 50 and 52 can search the routing table to determine the transfer destinations by using hash values of the data as keys.
The routing table management unit 60 generates routing tables when the data transfer apparatuses 20 are added to the data distribution system 10 and stores the generated routing tables in the routing table storage unit 62. The routing table management unit 60 also updates the routing tables when another data transfer apparatus is added to the data distribution system 10 or when the configuration of the data distribution system 10 is changed.
When there exists a data transfer apparatus having the same ID as the key k, the distance between this data transfer apparatus and the key k is “0,” and, therefore, this data transfer apparatus becomes the successor of the key k.
Furthermore, the routing table management unit 60 stores an ID of a predecessor of the data transfer apparatus 20 and first and second addresses of the predecessor in the predecessor storage unit 64. Here, the predecessor means another data transfer apparatus of the data distribution system 10, which has, as its ID, a key that is farthest from the ID of the data transfer apparatus 20.
Returning to
The data receiver 20 that receives data through the first receiving queue 40 causes the first transfer unit 50 to determine a transfer destination to transfer the received data to, and then inserts the data into a transmission queue of the first transmission queue 44 that corresponds to the transfer destination. For instance, when the transfer destination determined by the first transfer unit of the data transfer apparatus 0 is the data transfer apparatus 1, the data is inserted into a transmission queue s0-3 of the first transmission queue 44 that corresponds to the data transfer apparatus 1.
Each data transfer apparatus 20 has its ID in the own apparatus information storage unit 66. When transferring the data received through the first receiving queue 40, and when, for example, the ID of the transfer-destination data transfer apparatus is greater than the ID of the own apparatus (when the magnitude relation ship is in one condition), the data is transmitted to the first address of the transfer destination. When, on the other hand, the ID of the transfer-destination data transfer apparatus is smaller than the ID of the own apparatus (when the magnitude relationship is in the other condition), each data transfer apparatus 20 transmits the data to the second address of the transfer destination. When transferring the data received through the second receiving queue 42 to another data transfer apparatus, each data transfer apparatus 20 transmits the data to the second address of the transfer destination.
When the determined transfer destination is the data receiver, the first transfer unit 50 or the second transfer unit 52 inserts the data into the transmission queue of the second transmission queue 46 that corresponds to the data receiver, and then transmits the data to the data receiver. However, in the data transfer apparatus having the largest ID number (the data transfer apparatus 3 in the example shown in
The data distribution system 10 according to the present embodiment described above uses the following two features of the structured P2P network. The first feature is that the data transfer apparatuses are marked with numbers and ordered into a ring, and data are transferred in this order. Depending on the routing protocol, some of the ordered data transfer apparatuses are skipped when the data are transferred. The second feature is that the data are transferred no more than one cycle.
According to the first feature, the data transfer apparatuses are marked with numbers and ordered into a ring in an order of: data transfer apparatus 0, data transfer apparatus 1, data transfer apparatus 3, data transfer apparatus 0, and the like, as shown in the example in
In view of these two features described above, the present embodiment prevents the occurrence of deadlocks by considering the ring shown in
==Processes==
Next are described examples of processes performed in the data distribution system 10. First, data transfer processes performed in the first and second transfer units 50 and 52 are described.
When the present data transfer apparatus is the successor of the key k (S0802: Yes), the first transfer unit 50 inserts the key k and data into a queue of the second transmission queue 46, which holds the data to be transferred to the data receiver (S0803).
When the present data transfer apparatus is not the successor of the key k (S0802: No), the first transfer unit 50 executes a search process for searching the routing table for a transfer-destination data transfer apparatus, to which the received key k and data are transferred (S0804). As a result of this search process, a record including the ID of the transfer-destination data transfer apparatus is specified. Hereinafter, the ID, first address, and second address of this specified record are referred to as the ID, first address, and second address of the transfer-destination data transfer apparatus.
Once the transfer-destination data transfer apparatus is searched, the first transfer unit 50 determines whether the ID of the transfer-destination data transfer apparatus is greater than the ID of the present data transfer apparatus or not (S0805). When the ID of the transfer-destination data transfer apparatus is greater than the ID of the present data transfer apparatus (when the magnitude relationship is in one condition) (S0805: Yes), the first transfer unit 50 inserts the key k and data into a queue of the first transmission queue 44, which holds the data to be sent to the first address of the transfer-destination data transfer apparatus (S0806). However, when the ID of the transfer-destination data transfer apparatus is smaller than the ID of the present data transfer apparatus (when the magnitude relationship is in the other condition) (S0805: No), the first transfer unit 50 inserts the key k and data into a queue of the first transmission queue 44, which holds the data to be sent to the second address of the transfer-destination data transfer apparatus (S0807).
When the present data transfer apparatus is the successor of the key k (S1002: Yes), the second transfer unit 52 inserts the key k and data into a queue of the second transmission queue 46, which holds the data to be transferred to the data receiver (S1003).
When the present data transfer apparatus is not the successor of the key k (S1002: No), the second transfer unit 52 executes a search process for searching the routing table for the transfer-destination data transfer apparatus, to which the received key k and data are transferred (S1004). This search process is same as the one performed by the first transfer unit 50 (S0804).
Once the transfer-destination data transfer apparatus is searched, the second transfer unit 52 inserts the key k and data into a queue of the second transmission queue 46, which holds the data to be sent to the second address of the transfer-destination data transfer apparatus (S1005).
A process for generating/managing the routing table is described next. FIG. 11 is a flowchart showing an example of the process for generating/managing the routing table. First, the routing table management unit 60 acquires the ID=k, a first address a, and a second address a′ of the present data transfer apparatus from the own apparatus information storage unit 66 (S1101). The routing table management unit 60 also determines whether the present data transfer apparatus is the first data transfer apparatus of the data distribution system 10 or not, and, when the present data transfer apparatus is not the first data transfer apparatus, acquires a first address ae of any of the data transfer apparatuses configuring the data distribution system (S1101).
Note that whether the present data transfer apparatus is the first data transfer apparatus of the data distribution system 10 or not can be determined by, for example, storing, beforehand, information on each of the data transfer apparatuses configuring the data distribution system 10 in a predetermined storage unit provided in the data distribution system 10. This determination can also be made by, for example, transmitting a predetermined message for confirming the presence/absence of the other data transfer apparatuses, in the data distribution system 10.
When the present data transfer apparatus is the first data transfer apparatus of the data distribution system 10 (S1102: Yes), the routing table management unit 60 stores k, a, and a′ in the predecessor storage unit 64 (S1103). The routing table management unit 60 then registers, in the routing table, the records indicating a key that is distant from k by 2i−1, k, a, and a′, where i=1 to m, as the base point key, ID, first address, and second address (S1104, S1105). Note that the alphabet m represents the number of bits in a key.
When the present data transfer apparatus is not the first data transfer apparatus of the data distribution system 10 (S1102: No), the routing table management unit 60 executes a routing table initializing process (S1106) and then executes an update process for updating the routing tables of the other data transfer apparatuses (S1107).
With the key distant from the key k by 2i−1 as a base point key si (S1203), the routing table management unit 60 requests the data transfer apparatus identified by ae for the ID and addresses of a successor of si (S1204). This request message describes not only the base point key si but also the address a indicating a response destination.
After receiving this request message, the other data transfer apparatus transfers the request message to the successor of si by performing the processes shown in
In response to the request, the routing table management unit 60 receives the ID=ki, first address si, and second address a′i, of the successor from the successor of the base point key si (S1205). The routing table management unit 60 then registers, in the routing table, the records indicating si, ki, ai, and a′i as the base point key, ID, first address, and second address (S1206). However, when the distance between the base point key si and the ID=k of the present data transfer apparatus is shorter than the distance between the base point key si and the ID=ki, the routing table management unit 60 registers, in the routing table, the records indicting si, k, a, and a′ as the base point key, ID, first address, and second address.
After receiving this request message, the other data transfer apparatus transfers the request message to the successor of k by performing the processes shown in
In response to the request, the routing table management unit 60 receives the ID=k1, first address a1, and second address a′1 of the successor from the successor of the k (S1302). The routing table management unit 60 then registers, in the routing table, the records indicting the key k1 that is distant from k by 1, a1, and a′1 as the base point key, ID, first address, and second address (S1303).
The routing table management unit 60 further acquires the ID=kp, first address ap, and second address a′p of the data transfer apparatus, which are stored in the predecessor storage unit 64 of this successor, from the successor identified by the address a1 (S1304), and then stores the acquired ID and first and second addresses in the predecessor storage unit 64 of the present data transfer apparatus (S1305).
Moreover, the routing table management unit 60 updates the ID, first address, and second address that are stored in the predecessor storage unit 64 of the successor identified by the address a1, to k, a, and a′, respectively (S1306).
The routing table management unit 60 calculates a key k-i that is distant from k by 2i−1 (S1402) and requests the other data transfer apparatus for the addresses of the predecessor of k-i (S1403). Note that the request message describes not only k but also the address a indicating the response destination.
After receiving this request message, the other data transfer apparatus transfers the request message to the predecessor of k-i by similarly performing the processes shown in
In response to the request, the routing table management unit 60 receives the first address a-1 of the predecessor of k-i from the predecessor of k-i (S1404). The routing table management unit 60 transmits k, a, a′ and i to the address a-1 and updates the routing table (S1405).
When the key k is the ID of the present data transfer apparatus (S1502: Yes), the routing table updating process is ended. When the key k is not the ID of the present data transfer apparatus (S1502: No), the routing table management unit 60 acquires, from the routing table, the record indicating, as the base point key si, a key that is distant from the ID of the present data transfer apparatus by 2i−1 (S1503).
The routing table management unit 60 determines whether the distance between si and k is shorter than the distance between si and the ID in the record (S1504). When the distance between si and k is longer than the distance between si and the ID in the record (S1504: No), the routing table updating process is ended. When the distance between si and k is shorter than the distance between si and the ID in the record (S1504: Yes), the routing table management unit 60 updates the ID, first address, and second address in the record to k, a, and a′, respectively (S1505). The routing table management unit 60 then transmits k, a, a′ and i to the first address stored in the predecessor storage unit 64, and updates the routing table (S1506).
Next, operations of the data distribution system 10 are described using specific examples. In the following description, suppose that the data transfer apparatuses 20 having ID numbers 0, 3, 1 are added sequentially to the data distribution system 10, as shown in
First, the data transfer apparatus 0 with ID number 0 is activated. Once the data transfer apparatus 0 is activated, the routing table management unit 60 executes the initializing process shown in
Next, the data transfer apparatus 3 with ID number 3 is activated. In the data transfer apparatus 3 or a data transfer apparatus that is activated subsequently, the process proceeds to the step followed by “No” of step S1102. In the routing table initializing process of step S1106, the successor/predecessor initializing process S1201 is executed first. In the successor/predecessor initializing process, steps S1301, S1302 are executed to acquire the ID, first address, and second address of the data transfer apparatus 0 as those of the successor of the key 3. As a result of step S1303, the routing table of the data transfer apparatus 3 is updated as shown in
Furthermore, in step S1304 the data transfer apparatus 3 acquires, from the data transfer apparatus 0, the ID, first address, and second address of the data transfer apparatus 0 that are stored in the predecessor storage unit 64, as those of the predecessor of the data transfer apparatus 3. Then, as a result of step S1305, the predecessor storage unit 64 of the data transfer apparatus 3 becomes the one shown in
Thereafter, through the processes subsequent to step S1202 shown in
Next, the process S1107 for updating the routing table of the other data transfer apparatus is executed (
The data transfer apparatus 0 updates the routing table (
The data transfer apparatus 3 that receives the information from the data transfer apparatus 0 determines in step S1502 that the received key 3 is the ID of the data transfer apparatus 3, and ends the routing table updating process.
In the process for updating the routing table of the other data transfer apparatus (
Finally, the data transfer apparatus 1 with ID number 1 is activated. Once the data transfer apparatus 1 completes the routing table initializing step S1106 and the step S1107 of updating the routing table of the other data transfer apparatus, the routing table storage unit 62 and the predecessor storage unit 64 of each data transfer apparatus are updated as shown in
Next are described examples of operations that are performed in the data distribution system 10 until data transmitted from the data transmitter 24 to each data transfer apparatus 20 reaches the data receiver 26.
First of all, suppose that the data transfer apparatus 3 receives data from the data transmitter 24. The data transfer apparatus 3 inserts the received data into the first receiving queue 40. The first transfer unit 50 extracts the data from the first receiving queue 40 and calculates a hash value of the data as a key. Note that this key calculation is executed only when the data transfer apparatus receives data from the data transmitter 24. Here, suppose that the calculated key is 1.
Because the data transfer apparatus 3 is not a successor corresponding to the key=1 in step S0802 of the process shown in
The data transfer apparatus 0 inserts the key=1 and data that are received at the second address into the second receiving queue 42 (queue r0-2 shown in
The data transfer apparatus 1 inserts the key=1 and data that are received at the second address into the second receiving queue 42 (queue r1-3 shown in
The above has described the present embodiment. The present embodiment can prevent the occurrence of deadlocks because each data flow is not configured into a ring. Moreover, the present embodiment can prevent the occurrence of deadlocks regardless of the size of a message communication buffer or without securing a large message communication buffer.
The present embodiment is described in order to facilitate understanding of the present invention and should not be construed as limiting the present invention. The present invention can be changed/improved without departing from the spirit thereof and includes the equivalents thereof.
According to the present embodiment, when, for example, the ID of the transfer-destination data transfer apparatus is greater than the ID of the own apparatus, it is determined that the magnitude relationship is in one condition, and when the ID of the transfer-destination data transfer apparatus is smaller than the ID of the own apparatus, it is determined that the magnitude relationship is in the other condition. However, the IDs to be prioritized can be reversed.
In addition, in the present embodiment, a ring is configured in ascending order of the IDs of the data transfer apparatuses; however, the ring may be configured in descending order of the IDs of the data transfer apparatuses.
This application claims priority based on Japanese Patent Application No. 2011-068486 filed in Japan Patent Office on Mar. 25, 2011, the entire contents of which are hereby incorporated by reference.
The invention of the present application was described above with reference to the embodiment. However, the invention of the present application is not limited thereto. Various modifications that can easily be understood by a person skilled in the art can be made to the configurations and details of the invention of the present application, within the scope of the invention of the present application.
The present embodiment can partially or entirely be described in the form of the notes described hereinafter but is not limited thereto.
Number | Date | Country | Kind |
---|---|---|---|
2011-068486 | Mar 2011 | JP | national |
Filing Document | Filing Date | Country | Kind | 371c Date |
---|---|---|---|---|
PCT/JP2012/057564 | 3/23/2012 | WO | 00 | 3/18/2013 |