This application is based upon and claims the benefit of priority from Japanese Patent Application No. 2019-047969, filed on Mar. 15, 2019; the entire contents of which are incorporated herein by reference.
Embodiments described herein relate generally to a processing apparatus, a processing method, and a computer program product.
There is known a technique that aggregates a plurality of machines to one host and operates the machines by using a virtual machine technique. Virtualization has prevailed in a field, such as an industrial system, and it is considered that the identical host operates a virtual machine executing a process that needs to be real-time (real-time process) and a virtual machine executing a process that does not need to be real-time (non-real-time process).
However, in the conventional technique, it is difficult to control data access to a memory while considering the real-time process executed by a host processor.
According to an embodiment, a processing apparatus includes a memory and a processor coupled to the memory. The processor is configured to: execute data access that is at least one of data writing to the memory and data reading from the memory; receive access control information for controlling timing of the data access to be executed; and control the timing of the data access based on the received access control information.
The following describes embodiments of a processing apparatus, a processing method, and a computer program product in detail with reference to the attached drawings.
Example of Hardware Structure
The memory 1 is connected to the host processor 2 via a memory controller in the host processor 2. The memory 1 is implemented by, for example, Dynamic Random Access Memory (DRAM) or the like.
The host processor 2 is connected to the storage controller 5 using a bus, such as PCI Express (registered trademark). Similarly, the host processor 2 is connected to the network interface controller 4 using a bus, such as PCI Express (registered trademark).
The host processor 2 develops an image of an execution program stored in the storage 3 to the memory 1 and executes a process while reading an instruction and data on the memory 1. The process is executed by one or a plurality of cores included in the host processor 2. The host processor 2 has a hardware virtualization assistance function and can effectively execute a virtual machine by virtualization-compatible instruction set, Input/Output Memory Management Unit (IOMMU), and the like.
The storage 3 (and the storage controller 5) and the network interface controller 4 are peripheral apparatuses, peripheral devices, and peripherals connected to the host processor 2.
The storage 3 is, for example, implemented by a Hard Disk Drive (HDD), a Solid State Drive (SSD), and the like. The storage 3 is connected to the storage controller 5 in a specified standard, such as SATA, SAS, and U.2 (SFF-8639). The storage 3 and the storage controller 5 may be integrated.
The network interface controller 4 connects the host processor 2 to a network 200.
The network 200 is, for example, Ethernet (registered trademark). Specifically, the network 200 is a network supporting a standard defined in IEEE 802.1. The standard defined in IEEE 802.1 is, for example, the Audio Video Bridging (AVB) standard, the Time-Sensitive Networking (TSN) standard and the like. Any kind of the network 200 may be used. The network 200 is, for example, an office network, a network inside a data center, a vehicle-mounted network, an in-plant network, a network of a portable base station, and the like.
The network interface controller 4 and the storage controller 5 are each implemented by, for example, an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), and the like. The network interface controller 4 and the storage controller 5 each may be implemented by a combination of ASIC, FPGA, and a processor. The network interface controller 4 and the storage controller 5 each may incorporate a memory differing from the above-mentioned memory 1. The network interface controller 4 and the storage controller 5 each may be mounted as chips differing from the host processor 2 or may be mounted as one chip as a System-on-a-Chip (SoC).
[Example of Functional Configuration]
In the host processor 2, virtual machines A to F operate. In
The network interface controller 4 executes transmission and reception processes of a frame over the network 200 in accordance with an instruction from the host processor 2. The frame is, for example, an Ethernet (registered trademark) frame. The network interface controller 4 is compatible with Single Root I/O Virtualization (SR-IOV). Each of the virtual machines A to F is able to directly access the network interface controller 4 by means of a PCI Passthrough function. A notification unit 405 and a data access unit 406 in the network interface controller 4 may include interfaces independently accessible from the respective virtual machines of the virtual machines A to F. That is, a register access interface, a DMA interface, and an interrupt notification interface may be included in each virtual machine.
The memory 1 includes a virtual machine schedule information storage unit 101, task schedule information storage units 102a and 102b, and data storage units 103a and 103b.
The host processor 2 includes a virtual machine control unit 201, the virtual machines A to F, a time management unit 204, and an access control information generation unit 205. The virtual machine A includes a task control unit 202a and a task execution unit 203a. Similarly, the virtual machine B includes a task control unit 202b and a task execution unit 203b.
The network interface controller 4 includes a time management unit 401, an output timing information storage unit 402, an access control information reception unit 403, an access timing control unit 404, the notification unit 405, the data access unit 406, a data buffer unit 407, an output timing control unit 408, a data input/output unit 409, an input data classification unit 410, and a network information management unit 411.
The virtual machine control unit 201 provides management and control functions of computer resources to implement the virtual machines A to F. The virtual machine control unit 201 is, for example, a virtual machine monitor, a hypervisor, a virtualization OS, and the like. The virtual machine control unit 201 can logically separate and integrate the computer resources. For example, the virtual machine control unit 201 can logically separate one computer into the plurality of virtual machines A to F as if the one computer were a plurality of computers. This allows each of the virtual machines A to F to independently operate an OS and an application. The virtual machine control unit 201 controls the virtual machines A to F based on virtual machine scheduling information stored in the virtual machine schedule information storage unit 101. The virtual machine control unit 201 also manages and controls an interrupt notification notified from the peripheral device or the like and transmits the interrupt notification to the task control units 202a and 202b described later.
The virtual machine schedule information storage unit 101 stores the virtual machine scheduling information, which is provided to each CPU core.
Example of Virtual Machine Scheduling Information
While the example of ARINC 653 is given here, the scheduling method given to each CPU is not limited to this. For example, the scheduling method given to each CPU may be a scheduling method, such as credit base. Earliest Deadline First (EDF) base, and Real-Time-Deferrable-Server (RTDS) base. The scheduling operations by these scheduling methods are executed based on time information supplied by the time management unit 204.
Returning to
The task control unit 202a manages and controls the entire virtual machine A. The task control unit 202a provides a function of a so-called Operating System (OS). The task may be a process, a thread, and a job and means a unit of a process controlled by the OS. The task control unit 202a manages and controls resources, such as a processor and a memory, required to execute the task. Specifically, the task control unit 202a assigns an execution period of the processor and the memory, switches the execution period, maps a virtual memory and a physical memory, and the like. Additionally, the task control unit 202a manages and controls the interrupt notification notified from the virtual machine control unit 201. The task control unit 202a assigns and controls a task execution period based on task schedule information stored in the task schedule information storage unit 102a.
Since the description of the task control unit 202b in the virtual machine B is similar to the description of the task control unit 202a in the virtual machine A, the description as to the task control unit 202b is omitted. Similarly, the following describes the similar descriptions of the virtual machines A and B with an example of the virtual machine A, while the description of the virtual machine B is omitted.
Example of Task Schedule Information
Returning to
The access control information includes, for example, information for controlling access based on the schedule information, such as the above-mentioned virtual machine schedule information and task schedule information. Specifically, the access control information includes, for example, information such as an access-prohibited period calculated based on a time at which the host processor 2 executes the real-time process. In this case, the access timing control unit 404 prohibits, for example, data access from the data access unit 406 to the memory 1 during the access-prohibited period. The access control information includes, for example, a real-time task process execution period specified according to the time information. In this case, the access timing control unit 404 limits or prohibits, for example, the data access from the data access unit 406 to the memory 1 during the real-time task process execution period. A specific example of the access control information will be described later.
In an environment that is not the virtual machine environment, the access control information may be generated by using only the task schedule information. When only the virtual machine scheduling information is acquired, the access control information may be generated using only the virtual machine scheduling information.
The data storage unit 103a is used to store the program executed by the task execution unit 203a and data held by the task execution unit 203a. Although not illustrated, the data storage unit 103a may store the data in the task control unit 202a. The task execution unit 203a executes the program while reading the program from the data storage unit 103a on the memory 1. The data storage unit 103a stores the data held in the task execution unit 203a. The data storage unit 103a is used to store data regarding the frame when the frame transmitted/received over the network interface controller 4 is transmitted. For example, the data storage unit 103a stores a reception descriptor ring to receive the frame, a transmission descriptor ring to transmit the frame, and data of the frame. The data storage unit 103a may be configured to house a plurality of network interfaces used by the virtual machine A, or the data storage units 103a may be provided for each of the network interfaces used by the virtual machine A.
Each entry (each descriptor) of the transmission descriptor ring includes a transfer source address, a length, and a status. The transfer source address indicates a start address indicative of a start position of a storage area in the data storage unit 103a storing the data of the frame target for transmission. The length indicates a length of the frame target for transmission. The status stores information indicative of the state of the transmission process.
In view of the data access to the memory 1, the transmission descriptor ring is a descriptor ring managing data reading (reading of transmission data) from the memory 1.
Each entry (each descriptor) of the reception descriptor ring includes a transfer destination address, a length, and a status. The transfer destination address indicates a start address indicative of a start position of a storage area in the data storage unit 103a storing the data of the frame target for reception. The length indicates a length of the frame target for reception. The status stores information indicative of the state of the reception process.
The above-mentioned statuses include, for example, error bits, DONE bits, and the like. The error bit indicates presence/absence of a transfer error. The DONE bit indicates termination of a process in the network interface controller 4. When the DONE bit of the transmission descriptor is 1, it means termination of the transmission process. When the DONE bit of the reception descriptor is 1, it means termination of the reception process. The network interface controller 4 writes 1 to the respective bits (the error bit and the DONE bit). After the task control unit (for example, a task control unit 202a) or the task execution unit (for example, the task execution unit 203a) confirms the respective bits, 0 is written to the respective bits to clear the respective hits.
From the viewpoint of the data access to the memory 1, the reception descriptor ring is a descriptor ring managing data writing (writing of received data) to the memory 1.
Returning to
The time management unit 401 in the network interface controller 4 manages the time information in the network interface controller 4. Examples of the time information include time information synchronized in a time synchronization protocol, such as NTP, IEEE 1588, and IEEE 802.1AS, which may be used over the network 200. The time information may be time information synchronized with time information acquired from a GPS or the like. This time information provided by the time management unit 401 is preferably synchronized with the above-mentioned time information provided by the time management unit 204 in the host processor 2. For example, when the time synchronization is executed in IEEE 1588 and IEEE 802.1AS, the data input/output unit 409 may give time stamps to the transmitted and received frames using a counter of the time management unit 401 and calculate a time offset from a ground master to correct the time information of the time management unit 401. Furthermore, the network interface controller 4 may correct a clock of the time management unit 204 on the host processor 2 based on a clock corrected by the data input/output unit 409.
The input data classification unit 410 is a functional unit that classifies the received frames.
The network information management unit 411 records correspondence information of the MAC addresses with the virtual machines A to C. Note that the network interface controller 4 connected to one virtual machine (for example, the virtual machine A) needs not to be one, and a plurality of the network interface controllers 4 may be connected. In this case as well, the correspondence is executed with the MAC addresses. First, using the transmission destination MAC address of the received frame, the input data classification unit 410 determines that the received frame is addressed to which virtual machine address. Next, the input data classification unit 410 acquires a PCP value indicative of a priority included in a VLAN tag of the received frame. Then, the input data classification unit 410 acquires the traffic class of the received frame from mapping information of the priority with the traffic class provided by the network information management unit 411, classifies the received frame, and stores the received frame in the data buffer unit 407.
The data buffer unit 407 includes, for each of input (reception) and output (transmission), queues (FIFOs) to store the frames for respective combinations of the network interface controllers 4 connected to the virtual machines A to F and the traffic classes. Since respective entries in the queues store frames, whole data of a frame and a length of the frame are both stored in each entry.
When input (reception) is performed, the received frames classified by the input data classification unit 410 as described above are stored in the queues corresponding to respective data types classified for the combinations of the transmission destination MAC addresses of the network interface controllers 4 connected to the virtual machines A to F and the traffic classes and the like.
The data buffer unit 407 stores the state of the start frame in each queue. Specifically, the data buffer unit 407 stores information representing whether the reading of the descriptor has been completed, whether the transfer of the frame has been completed, and whether the writing of the descriptor has been completed.
The output timing control unit 408 controls timing at which the data (frame) is output to the network 200. The output timing is controlled to follow the TSN standard, such as IEEE 802.1Qbv, for example. The output timing control unit 408 determines timing at which the frames received from the respective virtual machines A to F are transmitted with a gate control list stored in the output tuning information storage unit 402, the mapping information recorded in the network information management unit 411, and the time information provided by the time management unit 401. Then, the output timing control unit 408 transmits the frames to the network 200 via the data input/output unit 409.
The access control information reception unit 403 receives the access control information generated by the access control information generation unit 205. A method for delivery and reception of the access control information may be a method, for example, of setting from the host processor 2 using a register interface or the like. Additionally, the method for delivery and reception of the access control information may be a method that includes an access control information storage unit in the memory 1, notifies a network interface controller 4 of an address of the access control information storage unit from the host processor 2, and reads the access control information from this access control information storage unit by the network interface controller 4.
The access control information is transmitted to the access timing control unit 404 and is used for timing control of data access described later. The information on the data type in the access control information may be transmitted to the input data classification unit 410 to switch the operation of the data classification by the input data classification unit 410.
The access timing control unit 404 controls, for example, timing at which the data access unit 406 accesses the data of the frame and the descriptor in the data storage unit 103a. The access timing control unit 404 performs the control such that the data storage unit 103a is not accessed during the access-prohibited period, which is included in the access control information received by the access control information reception unit 403. At this time, the access timing control uses the time information provided by the time management unit 401 in the network interface controller 4.
Specifically, in the reception, the access timing control unit 404 prohibits the data reading and the data writing to the reception descriptor and the writing of the frame data. In the transmission, the access timing control unit 404 prohibits the data reading and the data writing to the transmission descriptor and the reading of the frame data. The access timing control unit 404 determines whether the data access to the target data type is completed within an access-permitted period and instructs the data access unit 406 to access the data. In this determination, a transfer time of a bus between the memory 1 and the network interface controller 4 and a frequency of the memory access by the host processor 2 are taken into consideration. In order to perform this determination with time to spare, a guard band may be provided so as not to instruct the data access from the access timing control unit 404 to the data access unit 406 in the case where the remaining time until the termination of the access-permitted period falls below a constant value.
The data access unit 406 accesses the data of the frame and the data of the descriptor stored in the data storage units 103a and 103b and the data buffer unit 407. The data access unit 406 then transfers those data by DMA. The data access unit 406 executes the data access notified from the access timing control unit 404 with respect to each data type such as the MAC address and the traffic class.
In the reception, the data access unit 406 reads a transfer destination address from the reception descriptor and writes the data of the received frame read from the corresponding queue in the data buffer unit 407 to an area in the data storage unit (for example, the data storage unit 103a) designated by the transfer destination address. The data access unit 406 writes the length and the status to the reception descriptor in the data storage unit (for example, the data storage unit 103a).
In the transmission, the data access unit 406 reads the transfer source address and the length from the transmission descriptor, reads data by the amount designated by this length from an area in the data storage unit (for example, the data storage unit 103a) designated by the transfer source address and writes the data of the frame to the corresponding queue in the data buffer unit 407. Then, the data access unit 406 writes the status to the transmission descriptor in the data storage unit (for example, the data storage unit 103a).
The notification unit 405 executes notification between the host processor 2 and the network interface controller 4. Specifically, the notification unit 405 executes the notification using a register interface and an interrupt notification. The register interface provides an interface to read and write the variables Head and Tail, which manage the respective descriptor rings for transmission and reception in the data storage unit (for example, the data storage unit 103a), to the host processor 2. The host processor 2 is notified of the completion of reception of the frame and the completion of transmission of the frame by interrupt.
The network information management unit 411 manages the correspondence information of the MAC addresses with the virtual machines A to F and the above-mentioned mapping information (see
Next, the following describes setting of the access control information with reference to
There are some methods of calculating the real-time task execution period.
For example, like access control information 10a, access during execution of a real-time task may be equally prohibited. Determining the access-prohibited period like the access control information 10a allows data access without hindering the process of the real-time task. For example, at a period of 0 to 799 us, the access to the memory 1 by the data access unit 406 is prohibited.
Alternatively, like access control information 10b, access from the virtual machines other than the virtual machine executing the real-time task may be prohibited. Frames (data) of the virtual machines A to F are classified according to Media Access Control (MAC) addresses included in the frames. Determining the access-prohibited period like the access control information 10b allows restraining a communication delay of the virtual machine executing the real-time process while restraining communications of the virtual machines executing non-real-time processes. For example, at a period of 0 to 399 us, regarding the data of the virtual machine D executing the task a, which needs to be real-time, the data access unit 406 is allowed to access the memory 1. Additionally, at a period of 400 to 799 us, regarding the data of the virtual machine D executing the task b, which needs to be real-time, the data access unit 406 is allowed to access the memory 1.
Alternatively, like access control information 10c, accesses other than access to the designated traffic class (priority) may be further prohibited. Determining the access-prohibited period like the access control information 10c allows restraining a delay of communications of the real-time task in execution while eliminating an influence to the real-time task from communications of another task in units of tasks. For example, at a period of 0 to 399 us, regarding data of a traffic class 7 of the virtual machine D used to execute the task a, which needs to be real-time, the data access unit 406 is allowed to access the memory 1. Additionally, at a period of 400 to 799 us, regarding data of a traffic class 1 of the virtual machine D used to execute the task b. which needs to be real-time, the data access unit 406 is allowed to access the memory 1.
Alternatively, like access control information 10d, communications of the task whose priority is lower than the task in execution may be prohibited. In other words, communications of the task whose priority is higher than that of the task in execution may be permitted. The larger the value of the traffic class is, the higher the priority is. For example, at a period of 0 to 399 us, regarding the data of the traffic class 7 of the virtual machine D used to execute the task a, which needs to be real-time, the data access unit 406 is allowed to access the memory 1. At a period of 400 to 799 us, regarding the data of the traffic class 1 of the virtual machine D used to execute the task b, which needs to be real-time, and the data of the traffic class 7 of the virtual machine D used to execute the task a, which needs to be real-time, the data access unit 406 is allowed to access the memory 1.
Alternatively, like access control information 10e, the access-permitted period may be shifted forward in units of one period (200 us) such that the communications are completed before the execution of the real-time task. For example, the access-permitted period of 0 to 199 us of the access control information 10e corresponds to the access-permitted period of 200 to 399 us of the access control information 10c. For example, the access-permitted period of 200 to 399 us of the access control information 10e corresponds to the access-permitted period of 400 to 599 us of the access control information 10c. For example, the access-permitted period of 400 to 599 us of the access control information 10e corresponds to the access-permitted period of 600 to 799 us of the access control information 10c.
Note that an access-limited period, in which a throughput is set to equal to or less than a constant amount, may be employed instead of the above-mentioned access-prohibited period in which the access is completely prohibited. That is, the access timing control unit 404 may limit the data access during the access-limited period.
The access control information generation unit 205 generates the access control information (for example, the access control information 10d) (Step S2). Next, the access control information generation unit 205 sets the access control information generated by the process of Step S2 to the access control information reception unit 403 (Step S3).
The following describes a flow of the frame reception process with reference to
Example of Frame Reception Process
The input data classification unit 410 classifies the data based on the received frames (Step S13). At Step S13, as illustrated in the above-mentioned
Next, the input data classification unit 410 stores the received frames classified by the process of Step S13 in the data buffer unit 407 with respect to each data type (Step S14). The data buffer unit 407 can store the received frames in the queues with respect to each data type.
The access timing control unit 404 selects the received frame from the queues of the respective data types recorded in the data buffer unit 407 based on the time information provided by the time management unit 401 in the network interface controller 4 (Step S15). Specifically, the access timing control unit 404 selects the received frame from the queue in which the received frame is present during the access-permitted period of the queue. The access timing control unit 404, for example, may confirm whether the received frames are present in the order of the queues storing the received frames with the higher values of the traffic classes. For example, when access to the plurality of virtual machines is permitted, the access timing control unit 404 may confirm whether the received frames addressed to these virtual machines are present in order of round robin method.
Next, while the access timing control unit 404 determines whether the data transfer of the received frames is completed within the access-permitted period, the data access unit 406 reads the descriptor from, writes the frame to, and writes the descriptor to the data storage unit (for example, the data storage unit 103a). When the descriptor is read or written, the access timing control unit 404 executes the determination considering the length of the data of the descriptor. When the received frame is read or written, the access timing control unit 404 executes the determination considering the length of this received frame and a size of DMA transfer. Those relationships are assumed to be preliminarily given by measurement.
Specifically, first, the access timing control unit 404 confirms a state of the selected queue (Step S16). The state of queue includes three states: waiting for descriptor reading, waiting for frame transfer, and waiting for descriptor writing. The initial state is the waiting for descriptor reading. In the case where the state of the selected queue is the waiting for descriptor reading, the access timing control unit 404 determines whether the reading of the descriptor is completed within the access-permitted period before the reading (Step S17).
When the reading is not completed within the access-permitted period (Step S17: No), processing is returned to Step S15. When the reading is completed within the access-permitted period (Step S17: Yes), the access timing control unit 404 instructs the data access unit 406 to read the transfer destination address of the reception descriptor managed by the value of Head and updates this state of queue in the data buffer unit 407 to the waiting for frame transfer (Step S18). It is assumed that the task control unit (for example, the task control unit 202a) or the task execution unit (for example, the task execution unit 203a) preliminarily designates the transfer destination address for the area for storing the received frame.
When the state of queue enters the waiting for frame transfer, an access timing control unit 404 determines whether the writing of the received frame is completed within the access-permitted period before the writing (Step S19). This determination may be executed using the frame length.
When the writing is not completed (Step S19: No), processing is returned to Step S15. When the writing is completed (Step S19: Yes), the access timing control unit 404 instructs the data access unit 406 to read the data of the received frame from this queue in the data buffer unit 407 and write the data of the received frame in the area in the data storage unit (for example, the data storage unit 103a) designated by the transfer destination address, and updates this state of queue in the data buffer unit 407 to the waiting for descriptor writing (Step S20).
When the state of queue enters the waiting for descriptor writing, the access timing control unit 404 determines whether the writing of the descriptor is completed within the access-permitted period before the writing (Step S21).
When the writing is not completed (Step S21: No), processing is returned to Step S15. When the writing is completed (Step S21: Yes), the access timing control unit 404 instructs the data access unit 406 to write the length and the status to the reception descriptor, delete the entry of the received frame that has already been transferred from the queue, and updates this state of queue in the data buffer unit 407 to the waiting for descriptor reading (Step S22). When the process is normally terminated, data of 1 is written to the DONE bit in the status and the value of Head is increased by 1.
An area, in which the transfer destination address of the descriptor is cached, may be provided in each queue in the data buffer unit 407. When the transfer of the received frame is completed, the task execution unit (for example, the task execution unit 203a) reads the received frame triggered by reception completion interrupt notification from the notification unit 405 and polling (Step S23). The notification from the notification unit 405 is transmitted to the task execution unit (task execution unit 203a) via the virtual machine control unit 201 and the task control unit (for example, the task control unit 202a). An interrupt number different depending on each descriptor ring may be used for the interrupt. The task execution unit (for example, the task execution unit 203a) reads the status bit in the descriptor indicated by the value of Head in the descriptor ring notified by the interrupt, confirms that the DONE bit is 1, and receives the frame data by the amount designated by the length. By clearing (setting 0 to all bits) the status and writing a value of adding 1 to the value of Tail to a register notifying Tail, the network interface owns the used descriptor.
By the above-mentioned frame reception process of
Next, the following describes a flow of the frame transmission process with reference to
Example of Frame Transmission Process
Next, the access timing control unit 404 selects one in which the transmittable frame is present (one on which the process has not been terminated yet after receiving the transmission request, that is, one with the descriptor owned by the network interface controller 4 is present) from the descriptors of the data types to which the access is permitted, and selects the queue in the data buffer unit 407 corresponding to the descriptor ring (Step S32).
The access timing control unit 404 confirms the state of the queue selected by the process of Step S32 (Step S33). The state of queue includes three states: waiting for descriptor reading, waiting for frame transfer, and waiting for descriptor writing. The initial state is a state of waiting for descriptor reading.
When the state of the selected queue is the waiting for descriptor reading, the access timing control unit 404 determines whether the reading of the descriptor is completed within the access-permitted period before the reading (Step S34).
When the reading of the descriptor is not completed within the access-permitted period (Step S34: No), processing is returned to Step S32. When the reading of the descriptor is completed within the access-permitted period (Step S34: Yes), the access timing control unit 404 instructs the data access unit 406 to read the transfer source address and the length in the transmission descriptor and updates this state of queue in the data buffer unit 407 to the waiting for frame transfer (Step S35).
When the state of queue enters the waiting for frame transfer, the access timing control unit 404 determines whether the writing of the transmission frame is completed within the access-permitted period before the writing (Step S36). This determination may be executed using the frame length and the DMA transfer size.
When the writing of the transmission frame is not completed (Step S36: No), processing is returned to Step S32. When the writing of the transmission frame is completed (Step S36: Yes), the access timing control unit 404 adds an entry to store the transmission frame to the queue in the data buffer unit 407, instructs the data access unit 406 to read the transmission frame data by the amount designated by the length from the area designated by the transfer source address in the data storage unit 103a and write the transmission frame data to the data buffer unit 407, and updates this state of queue in the data buffer unit 407 to the waiting for descriptor writing (Step S37).
When the state of the queue enters the waiting for descriptor writing, the access timing control unit 404 determines whether the writing of the descriptor is completed within the access-permitted period before the writing (Step S38).
When the writing of the descriptor is not completed (Step S38: No), processing is returned to Step S32. When the writing of the descriptor is completed (Step S38: Yes), the access timing control unit 404 instructs the data access unit 406 to write the status in the transmission descriptor and updates this state of queue in the data buffer unit 407 to the waiting for descriptor reading (Step S39). When the transfer is normally executed, the DONE bit is set to 1 and the writing is executed. An area in which the transfer source address of the descriptor is cached may be provided in each queue in the data buffer unit 407. When the transfer of the transmission frame is completed, a transmission completion interrupt notification is transmitted to the task execution unit 203a via the notification unit 405 through the virtual machine control unit 201 and the task control unit 202a.
The frame put in the queue in the data buffer unit 407 is transmitted to the network 200 via the data input/output unit 409 based on the output timing information by the output timing control unit 408 (Step S40) and the entry that has been transmitted is deleted from the queue (Step S41). Afterwards, the value of Head is updated to a value of adding 1 to the value of Head, and notification is executed to the task execution unit 203a by interrupt. The task execution unit 203a that has received the notification clears the status in the descriptor and executes a release process of the storage area of the frame indicated by the transfer source address and the length.
By the above-mentioned frame transmission process of
As described above, in the first embodiment, the data access unit 406 executes the data access indicative of at least one of the data writing to the memory and the data reading from the memory. The reception unit (access control information reception unit 403) receives the access control information for controlling the timing of the data access. The control unit (access timing control unit 404) controls the timing of the data access based on the access control information.
According to the first embodiment, the data access to the memory 1 can be controlled considering the real-time process executed by the host processor 2. Specifically, it is capable of eliminating the need of considering the collision of the memory accesses by the host processor 2 and the network interface controller 4 and allowing the transmission and the reception of the frame without hindering the execution of the process that needs to be real-time. Moreover, this allows facilitating the calculation of the worst delay to ensure being real-time.
The conventional technique was not able to control memory access considering the operation of the real-time task. Additionally, the conventional technique was not able to control the timing of writing the data of the received frame to the memory 1 when the frame is received. In view of this, when the process that needs to be real-time is executed, the memory access by the host processor 2 and the memory access for transmission and reception of the frame are simultaneously executed, causing a problem of a delay in execution of the real-time process.
Modifications of First Embodiment
The following describes the modification of the first embodiment. In this modification, the description similar to the description of the first embodiment is omitted.
While the access control information generation unit 205 is provided in the host processor 2 in the original first embodiment, the access control information generation unit 205 may be provided in, for example, the network interface controller 4. In this case, the virtual machine schedule information and the task schedule information may be configured to be set to the network interface controller 4. In this case, the host processor 2 includes a schedule information setting unit that sets the schedule information, and the network interface controller 4 includes a schedule information reception unit. The schedule information includes at least one of the virtual machine schedule information and the task schedule information.
The original first embodiment gives the example of generating the access control information by using both of the virtual machine schedule information and the task scheduling information. Alternatively, when the virtual machine is not used, only the task scheduling information may be used or the access control information may be generated from only the virtual machine scheduling information.
The original first embodiment gives the example in which the network interface controller 4 is compatible with SR-IOV, and the respective virtual machines A to F can directly access the network interface controller 4. Alternatively, instead of the direct access from the respective virtual machines A to F to the network interface controller 4, bridges that connect the respective virtual machines A to F may be created in the virtual machines A to F or in the virtual machine control unit 201 (hypervisor), and the frame aggregated in those bridges may be transmitted and received.
While the original first embodiment gives the example in which the data of the frame is not separated and is transferred at once, the data of the frame may be separately transferred. In such a case, a data buffer unit 407 stores information indicating which frames have been transferred, and the transfer may be resumed at the access-permitted period.
The original first embodiment gives the example in which only the host processor 2 and the network interface controller 4 are connected to the memory 1. Alternatively. another peripheral processing apparatus, such as the storage controller 5, may be connected. In this case, a delay caused by another processing apparatus may be considered in the determination of the completion of the data access. For example, the completion determination may be executed assuming that a delay occurs by a value found by multiplying the maximum lengths of the data accesses by the respective processing apparatuses by the number of respective processing apparatuses.
In the original first embodiment, a guard band may be inserted at a time point when the access-permitted period switches to the access-prohibited period. By this configuration, even when there is a difference between the time information provided by the time management unit 204 in the host processor 2 and the time information provided by the time management unit 401 in the network interface controller 4, the control can be executed without collision of the memory accesses.
The following describes the second embodiment. In the second embodiment, the description similar to that of the first embodiment is omitted. The second embodiment is described with an example of causing a processing apparatus to function as a recording device.
Example of Functional Configuration
The storage controller 5 includes a time management unit 501, an access control information reception unit 503, an access timing control unit 504, a notification unit 505, a data access unit 506, a data buffer unit 507, a data input/output unit 509, an input data classification unit 510, and a namespace management unit 511.
Although the storage controller 5 is approximately identical to the network interface controller 4, the storage controller 5 differs in that, the output timing information storage unit 402 and the output timing control unit 408 are not provided, and that the namespace management unit 511 is provided instead of the network information management unit 411.
In the data input/output unit 509 and the input data classification unit 510 in the storage controller 5, an input corresponds to reading from the storage 3 and an output corresponds to writing to the storage 3.
Configurations of the descriptors for the data reading and the data writing from the host processor 2 to the storage controller 5 are similar to those of the first embodiment (see
Access control information 10g is an example in the case of equally prohibiting the access during execution of the real-time task. Determining the access-prohibited period like the access control information 10g allows data access without hindering the process of the real-time task. For example, at a period of 0 to 800 us. an access to the memory 1 by the data access unit 406 is prohibited.
Access control information 10h is an example of prohibiting access by the virtual machines other than the virtual machine executing the real-time task. Determining the access-prohibited period like the access control information 10h allows restraining a communication delay of the virtual machine executing the real-time process while restraining communications of the virtual machines executing the non-real-time processes. For example, at a period of 0 to 399 us, regarding the data of the virtual machine D executing the task a, which needs to be real-time, the data access unit 406 is allowed to access the memory 1. Additionally, at a period of 400 to 799 us, regarding the data of the virtual machine D executing the task b, which needs to be real-time, the data access unit 406 is allowed to access the memory 1.
Example of Data Reading Process
Next, the access timing control unit 504 selects a queue of a data type recorded in the data buffer unit 507 based on the time information provided by the time management unit 501 in the storage controller 5 (Step S52).
The access timing control unit 504 confirms the state of the selected queue (Step S53). The state of queue includes three states: waiting for descriptor reading; waiting for data transfer; and waiting for descriptor writing. The waiting for descriptor reading is an initial state. In the case where a state of the selected queue is the waiting for descriptor reading, the access timing control unit 504 determines whether the reading of the descriptor is completed within the access-permitted period before the reading (Step S54).
When the reading of the descriptor is not completed within the access-permitted period (Step S54: No), processing is returned to Step S52. When the reading of the descriptor is completed within the access-permitted period (Step S54: Yes), the access timing control unit 504 instructs the data access unit 506 to read the transfer destination address of the read descriptor, issues a read command to the storage 3, and updates the state of queue in the data buffer unit 507 to the waiting for data transfer (Step S55).
The input data classification unit 510 classifies the read data into the queues in the data buffer unit 507 based on the namespaces (Step S56) and stores the read data in these queues (Step S57).
When the state of queue enters the waiting for data transfer, an access timing control unit 504 determines whether the writing of the read data is completed within the access-permitted period before the writing (Step S58). This determination may be executed using the data length and the size of DMA transfer.
When the writing of the read data is not completed (Step S58: No), processing is returned to Step S52. When the writing of the read data is completed (Step S58: Yes), the access timing control unit 504 instructs the data access unit 506 to read the read data from this queue in the data buffer unit 507 and write the read data in an area in the data storage unit 103a designated by the transfer destination address, and updates this state of queue in the data buffer unit 507 to the waiting for descriptor writing (Step S59).
When the state of the queue enters the waiting for descriptor writing, the access timing control unit 504 determines whether the writing of the descriptor is completed within the access-permitted period before the writing (Step S60).
When the writing of the descriptor is not completed (Step S60: No), processing is returned to Step S52. When the writing of the descriptor is completed (Step S60: Yes), the access timing control unit 504 instructs the data access unit 506 to write the length and the status to the read descriptor and delete an entry of the read data that has already been transferred from the queue, and updates this state of queue in the data buffer unit 507 to the waiting for descriptor reading (Step S61).
An area, in which the transfer destination address of the descriptor is cached, may be provided in each queue in the data buffer unit 507. When the transfer of the read data is completed, the task execution unit 203a reads the read data triggered by reading completion interrupt notification from the notification unit 505 and polling (Step S62). The notification from the notification unit 505 is transmitted to the task execution unit 203a via the virtual machine control unit 201 and the task control unit 202a.
By the above-mentioned data reading process of
The following describes a flow of the data writing process with reference to
Example of Data Writing Process
Next, the access timing control unit 504 selects a descriptor, in which the write data is present, among the descriptors of the data types to which access are allowed, and selects a queue in the data buffer unit 407 corresponding to the descriptor ring (Step S72).
The access timing control unit 504 confirms the state of queue selected by the process of Step S72 (Step S73). The state of queue includes three states: waiting for descriptor reading; waiting for data transfer; and waiting for descriptor writing. The waiting for descriptor reading is an initial state.
When the state of the selected queue is the waiting for descriptor reading, the access timing control unit 504 determines whether the reading of the descriptor is completed within the access-permitted period before the reading (Step S74).
When the reading of the descriptor is not completed within the access-permitted period (Step S74: No), processing is returned to Step S72. When the reading of the descriptor is completed within the access-permitted period (Step S74: Yes), the access timing control unit 504 instructs the data access unit 506 to read the transfer source address and the length of the write descriptor, and updates this state of queue in the data buffer unit 507 to the waiting for data transfer (Step S75).
When the state of queue enters the waiting for data transfer, the access timing control unit 504 determines whether the reading of the write data is completed within the access-permitted period before the writing (Step S76). This determination may be executed using the data length and the size of DMA transfer.
When the reading of the write data is not completed (Step S76: No), processing is returned to Step S72. When the reading of the write data is completed (Step S76: Yes), the access timing control unit 504 adds an entry to store the write data in the queue in the data buffer unit 507, instructs the data access unit 506 to read the write data by the amount designated by the length from the area designated by the transfer source address in the data storage unit 103a, write the write data to the data buffer unit 507, and updates this state of queue in the data buffer unit 507 to the waiting for descriptor writing (Step S77).
When the state of queue enters the waiting for descriptor writing, the access timing control unit 504 determines whether the writing of the descriptor is completed within the access-permitted period before the writing (Step S78).
When the writing of the descriptor is not completed (Step S78: No), processing is returned to Step S72. When the writing of the descriptor is completed (Step S78: Yes), the access timing control unit 504 instructs the data access unit 506 to write the length and the status of the write descriptor, updates this state of queue of the data buffer unit 507 to the waiting for descriptor reading, and notifies data writing completion (Step S79). An area, in which the transfer source address of the descriptor is cached, may be provided in each queue in the data buffer unit 507. When the transfer of the write data is completed, a transmission completion interrupt notification is transmitted to the task execution unit 203a via the notification unit 505 through the virtual machine control unit 201 and the task control unit 202a.
The write data put in the queue in the data buffer unit 507 is written onto the storage 3 by the data input/output unit 509, and the entry that has already been written is deleted from the queue (Step S80).
By the above-mentioned data writing process of
As described above, the processing apparatus 100-2 of the second embodiment can acquire the effects similar to those of the first embodiment even when operated as the recording device. Specifically, prohibition (or limitation) of the memory access from the storage controller 5 eliminates the need for considering the memory access collision with the host processor 2 and allows executing the real-time process. Moreover, this allows further facilitating the calculation of the worst delay to ensure being real-time.
Function blocks of the above-mentioned network interface controller 4 and storage controller 5 may be implemented by computer-readable programs causing a computer to function.
The program executed by the computer is recorded as a file in an installable format or an executable format in a computer-readable storage medium, such as a CD-ROM, a memory card, a CD-R, a Digital Versatile Disc (DVD), and the like and provided as a computer program product.
The program executed on the computer may be stored on the computer connected to a network such as the Internet and provided through downloading over the network. Alternatively, the program executed on the computer is not downloaded and may be provided over the network such as the Internet.
Additionally, the program executed by the computer may be preliminarily incorporated into a ROM or the like and provided.
The program executed by the computer has a module configuration including the function blocks which can be implemented by the program among the above-mentioned functional configuration (function blocks). Some or all of the above-mentioned respective function blocks may be implemented by hardware, such as an Integrated Circuit (IC), not being implemented by software.
In the case where the functions are implemented by using two or more processors, the respective processors may implement one function or may implement two or more functions.
While certain embodiments have been described, these embodiments have been presented by way of example only, and are not intended to limit the scope of the inventions. Indeed, the novel embodiments described herein may be embodied in a variety of other forms; furthermore, various omissions, substitutions and changes in the form of the embodiments described herein may be made without departing from the spirit of the inventions. The accompanying claims and their equivalents are intended to cover such forms or modifications as would fall within the scope and spirit of the inventions.
Number | Date | Country | Kind |
---|---|---|---|
JP2019-047969 | Mar 2019 | JP | national |
Number | Name | Date | Kind |
---|---|---|---|
6182197 | Dias et al. | Jan 2001 | B1 |
8151265 | Ben-Yehuda | Apr 2012 | B2 |
8245229 | Green | Aug 2012 | B2 |
20020112131 | Fong | Aug 2002 | A1 |
20160048405 | Lang | Feb 2016 | A1 |
20160112182 | Karnes | Apr 2016 | A1 |
20190045037 | Sukhomlinov | Feb 2019 | A1 |
20190065386 | Park | Feb 2019 | A1 |
20190087245 | Yamaura | Mar 2019 | A1 |
20190121659 | van Riel | Apr 2019 | A1 |
20200326980 | Prantner | Oct 2020 | A1 |
Number | Date | Country |
---|---|---|
2013-061857 | Apr 2013 | JP |
2016-045563 | Apr 2016 | JP |
2019-53591 | Apr 2019 | JP |
WO 2015029249 | Mar 2015 | WO |
Entry |
---|
Yun, H, et al. “MemGuard: Memory bandwidth reservation system for efficient performance isolation in multi-core platforms.” DOI: 10. 1109/RTAS. 2013. 6531079, 10 Pages. |
“Intel Ethernet Controller 1210 Datasheet” Intel Corp. Networking Division. Jun. 2018, 872 Pages. |
Number | Date | Country | |
---|---|---|---|
20200293353 A1 | Sep 2020 | US |