Patent Application
20240012690
This application claims the benefit under 35 USC § 119(a) of Korean Patent application No. 10-2022-0084124, filed on Jul. 8, 2022, in the Korean Intellectual Property Office, the entire disclosure of which is incorporated herein by reference for all purposes.
The following description relates to a device and method for accelerator partitioning and batch scheduling.
With increasing demand for artificial intelligence (AI) technology, demand for increasing throughput of a system processing an AI model has increased. Techniques for partitioning an accelerator that performs an AI operations and scheduling a query including a batch to a partition are used.
This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.
In one general aspect, an electronic device includes one or more processors, and a memory storing instructions configured to cause the one or more processors to, for a first partitioning of an accelerator into partitions of different sizes, based on resource utilization of the partitions batch of different sizes input to the partition, determine correspondences between the sizes of the batches and the sizes of the partitions in the first partitioning, determine numbers of partitions for the respective determined sizes of the partitions based on the correspondences between the sizes of the batches and the sizes of the partitions in the first partitioning, and partition the accelerator into a second partitioning based on the determined numbers of the respective sizes of the partitions.
The resource utilization may be determined based on a neural network (NN) model executed by the partitions.
The instructions may be further configured to cause the one or more processors to determine the correspondences between the sizes of the batches and the sizes of the partitions based on a size of a batch corresponding to when the resource utilization of a partition corresponds to a preset threshold resource utilization.
The instructions may be further configured to cause the one or more processors to determine the numbers of partitions of respective sizes according to throughput based on the sizes of the partitions and the sizes of the batches.
The instructions may be further configured to cause the one or more processors to determine the numbers of partitions of respective sizes such that the number of batches by size corresponds to the number of batches by size which are processible based on the numbers of partitions of respective sizes.
The instructions may be further configured to cause the one or more processors to, when a batch is scheduled to the accelerator, calculate predicted execution times processing the batch by each of the respective partition sizes, based on a processing time determined based on the size of one or more partitions obtained by partitioning the accelerator and the size of the batch input to the partitions, and assign the batch to one of the partitions by comparing the predicted execution times with an execution time requirement that is associated with the batch.
The processing times may be determined based on a neural network (NN) model executed by the partitions.
The instructions may be further configured to cause the one or more processors to schedule the batch to a smallest partition among partitions for which the respectively corresponding predicted execution times meet the execution time requirement.
In one general aspect, an electronic device includes one or more processors and a memory storing instructions executable by the processor, wherein, in response to the instructions being executed by the one or more processors, the one or more processors: based on a processing time determined based on a size of one or more partitions obtained by partitioning an accelerator and a size of a batch input to each of the partitions, when the batch is scheduled to the accelerator, calculate predicted execution times for completing processing of the batch for each of the respective sizes of the partitions, and schedule the batch to one of the partitions by comparing the predicted execution times with an execution time requirement associated with the batch.
The processing times may be determined based on a neural network (NN) model executed by the partitions.
The instructions may further configure the one or more processors to calculate the predicted durations based on a remaining processing time of a batch that is currently processed by one of the partitions, a processing time of a batch that is already scheduled to one of the partitions, and a processing time of the batch.
The instructions may further configure the processor to schedule the batch to a smallest partition size among partition sizes for which the corresponding predicted execution times are earlier than the execution time requirement.
In one general aspect, a method manages an accelerator device that can be reconfigured to have different partitions that are capable of executing batches, and the method includes: providing associations between batch sizes and respective partition sizes, the batch sizes including amounts of data in the corresponding batches, the partition sizes including amounts of processing resources of the corresponding partitions, instantiating a number of instances of partitions for each of the respective partition sizes based on information about frequencies of batches for the respective batch sizes, and assigning batches to the instantiated instances of partitions, wherein the batches are assigned to instances of partitions that have partition sizes associated with, according to the associations, the sizes of the batches.
The associations may be determined based on a resource utilization threshold.
The information about frequencies of batches for the respective batch sizes may be determined based on statistics of the sizes of partitions and the sizes of batches.
Numbers of instances of partitions for the respective partition sizes may be determined such that the number of instances of a partition of a given size is proportional to a frequency of processing batches of a corresponding size.
The accelerator device may be a multi-instance graphics processing unit (GPU).
A partition size may correspond to a number of graphics processing clusters.
The assigning of a batch to an instantiated instance of a partition may further include predicting execution times of the batch for the respective partition sizes and assigning the batch to an instance of a partition having the lowest predicted execution time.
The associations between the batch sizes and the respective partition sizes may be determined based on historical statistics of executions of previous batches by previous partitions of the accelerator having the partition sizes.
Other features and aspects will be apparent from the following detailed description, the drawings, and the claims.
The following detailed description is provided to assist the reader in gaining a comprehensive understanding of the methods, apparatuses, and/or systems described herein. However, various changes, modifications, and equivalents of the methods, apparatuses, and/or systems described herein will be apparent after an understanding of the disclosure of this application. For example, the sequences of operations described herein are merely examples, and are not limited to those set forth herein, but may be changed as will be apparent after an understanding of the disclosure of this application, with the exception of operations necessarily occurring in a certain order. Also, descriptions of features that are known after an understanding of the disclosure of this application may be omitted for increased clarity and conciseness.
The features described herein may be embodied in different forms and are not to be construed as being limited to the examples described herein. Rather, the examples described herein have been provided merely to illustrate some of the many possible ways of implementing the methods, apparatuses, and/or systems described herein that will be apparent after an understanding of the disclosure of this application.
The terminology used herein is for describing various examples only and is not to be used to limit the disclosure. The articles “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. As used herein, the term “and/or” includes any one and any combination of any two or more of the associated listed items. As non-limiting examples, terms “comprise” or “comprises,” “include” or “includes,” and “have” or “has” specify the presence of stated features, numbers, operations, members, elements, and/or combinations thereof, but do not preclude the presence or addition of one or more other features, numbers, operations, members, elements, and/or combinations thereof.
Throughout the specification, when a component or element is described as being “connected to,” “coupled to,” or “joined to” another component or element, it may be directly “connected to,” “coupled to,” or “joined to” the other component or element, or there may reasonably be one or more other components or elements intervening therebetween. When a component or element is described as being “directly connected to,” “directly coupled to,” or “directly joined to” another component or element, there can be no other elements intervening therebetween. Likewise, expressions, for example, “between” and “immediately between” and “adjacent to” and “immediately adjacent to” may also be construed as described in the foregoing.
Although terms such as “first,” “second,” and “third”, or A, B, (a), (b), and the like may be used herein to describe various members, components, regions, layers, or sections, these members, components, regions, layers, or sections are not to be limited by these terms. Each of these terminologies is not used to define an essence, order, or sequence of corresponding members, components, regions, layers, or sections, for example, but used merely to distinguish the corresponding members, components, regions, layers, or sections from other members, components, regions, layers, or sections. Thus, a first member, component, region, layer, or section referred to in the examples described herein may also be referred to as a second member, component, region, layer, or section without departing from the teachings of the examples.
Unless otherwise defined, all terms, including technical and scientific terms, used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains and based on an understanding of the disclosure of the present application. Terms, such as those defined in commonly used dictionaries, are to be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and the disclosure of the present application and are not to be interpreted in an idealized or overly formal sense unless expressly so defined herein. The use of the term “may” herein with respect to an example or embodiment, e.g., as to what an example or embodiment may include or implement, means that at least one example or embodiment exists where such a feature is included or implemented, while all examples are not limited thereto.
The host processor 110 may be a device configured to control respective operations of components included in the electronic device 100 and may be, for example, a central processing unit (CPU), however the example is not limited thereto. The host processor 110 may control operations performed by the electronic device 100. The host processor 110 may execute a kernel and other operating system components to manage use of hardware resources of the electronic device 100, which may also be referred to as a server, a host, or the like. The host processor 110 may receive one or more requests for processing a neural network in the accelerator 140. Based on the neural network request(s), the host processor 110 may generate a kernel including instructions executable by the accelerator 140 and transfer the generated kernel to the accelerator 140. The request may be made for a neural network-based data inference (e.g., computing a prediction based on applying layers of weights of the neural network to an input to the neural network) and for obtaining a result of the data inference by allowing the accelerator 140 to execute the neural network for object recognition, pattern recognition, computer vision, speech recognition, machine translation, machine interpretation, recommendation services, personalized services, image processing, autonomous driving, or the like.
The off-chip memory 120 may be a memory located (or disposed) outside of the accelerator 140 and be, for example, dynamic random-access memory (DRAM), high bandwidth memory (HBM), and the like that is used as a main memory of the host processor 110, but is not limited to thereto. The off-chip memory 120 may store inference target data and/or parameters of the neural network to be executed by the accelerator 140, and data stored in the off-chip memory 120 may be transferred to the accelerator 140 for inferencing based thereon. The off-chip memory 120 may also be used in a case in which an on-chip memory inside the accelerator 140 is insufficient to execute the neural network in the accelerator 140.
The off-chip memory 120 may have a greater memory capacity than the on-chip memory in the accelerator 140. However, when the neural network is executing, the cost for the accelerator 140 accessing the off-chip memory 120 may be greater than the cost for the accelerator 140 accessing its own internal memory (e.g., on-chip memory). Costs of accessing off-chip memory 120 by the accelerator 140 may be power and/or access time.
The accelerator 140 may be an artificial intelligence (AI) accelerator that performs inferences on input data by executing the neural network, for example, based on a kernel transmitted from the host processor. The accelerator 140 may include a processors separate from the host processor 110. For example, the accelerator 140 may be a neural processing unit (NPU), a graphics processing unit (GPU), a tensor processing unit (TPU), or a digital signal processor (DSP), but is not limited thereto. In some embodiments, as described below, the accelerator 140 may be configured to enable partitioning the accelerator 140 into virtual accelerator devices (partitions) whose resources may be exclusive to respective user processes. For example, the accelerator 140 may be implemented as an NVIDIA (TM) Multi-instance GPU (MIG) and the partitions of the accelerator 140 may be MIG instances. Techniques described herein may be applied to any partitionable accelerator.
The accelerator 140 may process some neural network tasks more efficiently than the general-purpose host processor (the host processor 110) due to characteristics of the operations performed by the neural network. For some such neural network tasks, the on-chip memory and one or more processing elements (PEs) included in the accelerator 140 may be utilized for efficient neural network processing. The on-chip memory included in the accelerator 140 may store data necessary for performing a computation of the accelerator 140 or may include a global shared buffer and/or a local buffer for storing a computation result; the on-chip memory is be distinguished from the off-chip memory 120 located outside of the accelerator 140. For example, the on-chip memory may include a scratchpad memory accessible through an address space, static random-access memory (SRAM), or a system cache, but is not limited thereto.
In general, a neural network may provide an optimal output corresponding to an input by mapping an input and an output that are in a non-linear relationship, based on deep learning encoded into the neural network, e.g., in the form of weights. Deep learning may be for solving problems from a big data set and may optimize a neural network by determining parameters (e.g., weights) and/or structure of a model of the neural network. A neural network may include layers (e.g., an input layer, hidden layers, and an output layer). Each of the layers may include a respective plurality of nodes. Each node may be a calculation unit having one or more inputs and an output (e.g., an activation), and the nodes may be connected to each other. A weight may be set for a connection between two nodes and may be adjusted or changed through a learning or training algorithm. The weight may amplify, reduce, or maintain a relevant data value, thereby determining a degree of influence of the data value on a final result of the neural network. Weighted inputs of nodes included in a previous layer may be input to each node included in the output layer. A process of inputting weighted data from a layer to a next layer is referred to as propagation. For ease of description, neural networks described herein may also be referred to as models.
In some embodiments, the host processor 110 may form partitions (e.g., sizes and numbers of partition instances) by partitioning the accelerator 140. Operation requests processed by the accelerator 140 may have various batch sizes (sizes of units/batches of data to be processed). The resource utilization of the accelerator 140 over many batches may vary depending on the sizes of the batches and/or a neural network (NN) model being processed. For example, the host processor 110 may partition the accelerator 140 by considering a provided NN model and/or an input batch size. By doing so, the host processor 110 may increase resource utilization of the accelerator 140 (decrease the overall amount of idle time of the accelerator) and may also process an input query to satisfy an associated latency constraint (e.g., a requirement specified in a service level agreement (SLA)). The terms “query” and “request” are used interchangeably herein. Note that the terms “query/request” and “batch” are used interchangeably herein as there is usually a one-to-one correspondence between a query/request and a batch (although a query/request may request processing of more than one batch). However, a query generally refers to a request (e.g., a call to an application to an application programming interface or one or more processor instructions) and a batch refers to the actual data for which a request requests processing; a request will generally provide information on the location(s) and extent(s) of its batch data. In terms of partitioning and scheduling/assigning batches, properties of the batches, e.g., size and frequency, may inform partitioning and scheduling.
As mentioned above, the accelerator 140 may be divided into partitions. The host processor 110 may schedule input queries (i.e., assign batches of the input queries) to the partitions obtained based on partitioning of the accelerator 140 and based on the size (or predicted execution times) of the queries/batches. For example, the host processor 110 may calculate a predicted execution time (see
For example, the host processor 110 may form partitions by partitioning the accelerator 140 of the server 200. For example, the host processor 110 may determine a number of partitions (and possibly other partition parameters/settings). That is, the host processor 100 may determine the numbers of instances of partitions of different sizes. For example, the host processor 110 may determine the number of instances of partitions of different sizes and may accordingly partition each of the accelerators 140-1, 140-2, . . . , 140-n.
In one or more embodiments, the host processor 110 may schedule an input query/batch to the accelerator 140. For example, the host processor 110 may schedule an input query by determining which of the accelerator partitions of accelerators the 140-1, 140-2, . . . , 140-n the input query will be assigned to for execution. Scheduling is described further below.
Referring to
For example, when the accelerator 140 is a GPU, a partition size may be deemed to be the number of GPCs in a partition. For example, the size of a first partition 141 including four GPCs may be 4, the size of a second partition 142 including two GPCs may be 2, and the size of a third partition 143 including one GPC may be 1.
In operation 310, the host processor 110 may determine a batch size corresponding to a partition size. That is, the host processor 110 may determine or access information indicating correspondences between batch sizes and respective partition sizes. Such correspondences may be based on the partition size and resource utilization of a partition according to the batch size that is input to (or corresponds to) the partition.
The batch may represent a unit of data that is input to the accelerator 140 and processed at least one time by the accelerator 140. The batch may include pieces of data input to the accelerator 140 and processed. The batch size may correspond to (or be proportional to) the number of pieces of data processed at one time. For example, the batch size may represent the size of total data processed at one time.
For example, the resource utilization of a partition may be a time during which a resource of the partition (e.g., the GPC of
As the batch size processed by a given partition increases, the resource utilization may increase however the latency to process the may increase in proportion to the size of the batch. The host processor 110 may determine the partition size for a corresponding batch size to satisfy a preset constraint (e.g., the SLA) for processing an input query while also increasing the resource utilization based on the partition size.
An increase in the batch size may be an increase in the number of pieces of data to be processed by a partition. Generally, when the number of pieces of data processed by the partition increases, the partition resource may be utilized for a longer period to process the batch in the partition, and thus, the resource utilization of the partition may increase. However, when the number of pieces of data processed by the partition increases, latency until the completion of the processing of the batch may increase since more time is required to process the batch in the partition.
For example, when an input batch is processed by each partition size and the resource utilization of the partition is equal to or greater than predetermined resource utilization (e.g., threshold resource utilization), latency may rapidly increase. The host processor 110 may determine which batch sizes should correspond with which partition sizes based on batch sizes at the threshold resource utilization in each partition size.
In operation 320, the host processor 110 of the electronic device 100 may determine the numbers of partitions of different sizes that are to be instantiated. For example, the host processor 110 may determine the numbers of partition instances of different sizes based on a historical batch size distribution and based on correspondences between the batch sizes and the partition sizes. For example, a batch size distribution may be a number of batches, by size, input to the electronic device 100 per unit time. In other words, a batch size distribution may be historical frequencies of respective batch sizes. For example, a batch size distribution may be statistically calculated by analyzing previously input queries. For example, the batch size distribution may be an average of the numbers of batches by sizes of the batches inputted per second to the electronic device 100.
For example, a batch size distribution may be a distribution based on the batch sizes included in queries processed by the accelerator 140. For example, when a given/predetermined batch size is input many times per unit of time, the host processor 110 may determine the number of partitions of a corresponding size to instantiate in proportion to the frequency of the given/predetermined batch size. Such may be performed for different batch sizes and different respectively corresponding partition sizes.
In operation 330, the host processor 110 of the electronic device 100 may partition the accelerator 140 to have various numbers of partitions for different partition sizes as determined by operation 320.
By operations 310, 320, and 330, the host processor 110 may partition the accelerator 140 such that the latency of processing new batches in the partitions satisfies a preset constraint condition and while increasing utilization of the accelerator 140.
To reiterate, for example, in operation 310, when a batch is to be processed by a partition, the host processor 110 may determine the partition size corresponding to the size of the batch such that the latency of the partition may not exceed the preset constraint condition while increasing the resource utilization. In operation 320, the host processor 110 may partition the accelerator 140 by determining the numbers of partitions of different sizes based on historical size distribution information of input batches such that the input batches may be efficiently processed. In operation 340, when a batch is scheduled to a partition (among available partitions),based on an estimated/predicted processing times for the batch by each of the partition sizes.
The processing times for a given batch may be determined based on the sizes of the partitions obtained by partitioning the accelerator and the size of the batch. As the partition size increases, the corresponding processing time for processing the batch in the partitions may decrease. As batch size increases, the processing time for processing the batch by a given partition size may increase.
For example, as shown in Table 1, the processing time may be determined based on the size of each partition and the size of the batch that is input to each partition. For example, when the partition size is 1 (e.g., 1 GPC of Table 1) and the batch size is 31, the processing time may be 56 ms. In Table 1, the top row lists examples of partition sizes.
In one or more embodiments, the processing times may be determined by a NN model, which may be executed by the partitions or elsewhere. The processing time may be determined based on a NN model executed by the partitions in substantially the same manner as determining resource utilization and latency, which is described with reference to
For example, as described with reference to
In one or more embodiments, in operation 350, the host processor 110 may schedule a batch to one of the partitions by comparing the predicted execution times for the batch by the partition sizes to a constrained execution time (e.g., the SLA) that is associated with the query.
For example, a query may be a request to be processed by the accelerator 140, where the request is received from an external electronic device configured to be connected to the electronic device 100 and communicate with the electronic device 100. For example, the accelerator 140 may be operated by a server to provide a network or cloud service for processing network-based requests for clients. The query may include a batch to be computed in the accelerator 140 and scheduling the network/cloud query to a partition may be substantially the same as scheduling any other batch to a partition. For example, a constrained execution time may be set to the query.
For example, the host processor 110 may schedule a batch to one of the partitions for which the predicted execution time of the batch is less than the constrained/required execution time associated with the batch (e.g., by an SLA or the like).
For example, the host processor 110 may schedule a batch to the smallest partition, among the partitions, for which the predicted execution time is less than the constrained execution time. The host processor 110 may increase resource utilization by scheduling batches to the smallest partitions whose predicted execution times can satisfy the constrained execution times.
In operations 340 and 350, the host processor 110 may schedule a query to a partition such that the query is processed by the constrained execution time that is preset to the query.
For example, as illustrated in the example 400 of
The host processor 110 may determine a batch size that is correspond to a partition size based on resource utilization determined based on the NN model that is executed by the partition. The host processor 110 may partition the accelerator 140 based on the NN model executed by the partition.
Referring to
As illustrated in
As illustrated in
Referring to
As shown in the example 600 of
Referring to
The “batch size corresponding to the partition size” may refer to the batch size for which resource utilization is less than the threshold resource utilization when a batch of the batch size (which corresponds to the size of a partition) is processed by the partition. For example, when a batch of size less than B1 and greater than or equal to B0 is processed by a small size partition, resource utilization may be less than the threshold resource utilization, in other words, the resource utilization may be less than the threshold resource utilization and the latency may be small when the batch is processed by the partition.
In one or more embodiments, the batch sizes corresponding to the partition sizes may be determined based on the performance of the accelerator 140. For example, as illustrated in
For example, referring to
In the batch size frequency distribution illustrated in
For example, the host processor 110 may determine the numbers of partition instances for different respective partition sizes by considering throughputs (frequencies) based of the batch sizes corresponding to the partition sizes.
The throughput characterization table of
By considering throughputs based on the partition sizes and the batch sizes, the host processor 110 may determine the number of small partition instances to be 1, the number of medium partition instances instances to be 2, and the number of large partition instances instances to be 4.
For example, the throughputs represent processing frequencies at which the partitions of different sizes process queries. For example, referring to
Similarly to the small partition, the host processor 110 may determine the number of medium partition instances to be 2 in order to process 20 queries (having a batch size of 2) per second. The host processor 110 may determine the number of large partition instances to be 4 in order to process 40 queries (having a batch size of 3) and 20 queries (having a batch size of 4) per second.
In the example described above, the host processor 110 may determine the numbers of partition instances for different sizes such that the batch size frequency distribution corresponds to the number of instances of partitions by size.
For example, the host processor 110 may partition the accelerator 140 based on the numbers of partition instances for different respective partition sizes based on the throughput/frequency information (e.g., distribution). For example, the partition sizes small, medium, and large may respectively be partitions formed to have 1 GPC, 3 GPCs, and 7 GPCs in the GPU of
For example, referring to
As can be seen, when the accelerator 140 is partitioned in a way that aligns sizes and numbers of instances of partitions with the historical frequencies of different queries/batches, the partitioning of the accelerator may significantly improve the utilization of the resources of the accelerator 140 (if future queries/batches somewhat follow their historical pattern) while potentially satisfying latency requirements or the like.
For example, the waiting time Twait may be calculated as a sum of the remaining processing time Tremaining,current for a currently processed batch and a processing time Σ(Testimated,queued) of a batch pre-scheduled to the partition, as Equation 1.
Twait=Σ(Testimated, queued)+Tremaining,current Equation 1
For example, the host processor 110 may calculate the waiting time Twait and the processing time Tnew of an input batch by using the processing time of Table 1 above.
In one embodiment, the host processor 110 may calculate an SLA slack as Equation 2 shown below. In Equation 2, Testimated,new may denote a processing time of an input batch, Tnew and SLAtarget are respectively constrained execution times preset to a query, and α and β are respective parameters configured to set prediction performance of the SLA slack, based on a unique environment of the electronic device 100 including the accelerator 140 and/or the server 200, respectively.
SLAslacl=SLAtarget−α(Twait+β·Testimated,new) Equation 2
Referring to
For example, referring to
For example, the host processor 110 may schedule a batch to a partition having the smallest size among partitions in which predicted execution times 1002, 1012, 1022 are ahead of (satisfy) the constrained execution time 1031 of the batch. Execution time constraints or the like may vary for different requests, which in some implementations may be associated with different users and different SLAs.
Referring to
The host processor 110 may thus increase resource utilization of the accelerator 140 by scheduling a batch to the smallest partition among partitions for which respective predicted execution times are ahead of the constrained execution time.
In some embodiments, when there are no partitions for which the respective predicted execution times are ahead of the constrained execution time, the host processor 110 may schedule the batch to a partition having the earliest predicted execution time. For example, in the example illustrated in
Although embodiments are described above with reference to batches of neural network data, the techniques for partitioning accelerators and assigning batches to partitions are not limited to any particular type of data. Rather, the techniques may be used regardless of the type of data in the batches.
The computing apparatuses, the electronic devices, the processors, the memories, the displays, the information output system and hardware, the storage devices, and other apparatuses, devices, units, modules, and components described herein with respect to
The methods illustrated in
Instructions or software to control computing hardware, for example, one or more processors or computers, to implement the hardware components and perform the methods as described above may be written as computer programs, code segments, instructions or any combination thereof, for individually or collectively instructing or configuring the one or more processors or computers to operate as a machine or special-purpose computer to perform the operations that are performed by the hardware components and the methods as described above. In one example, the instructions or software include machine code that is directly executed by the one or more processors or computers, such as machine code produced by a compiler. In another example, the instructions or software includes higher-level code that is executed by the one or more processors or computer using an interpreter. The instructions or software may be written using any programming language based on the block diagrams and the flow charts illustrated in the drawings and the corresponding descriptions herein, which disclose algorithms for performing the operations that are performed by the hardware components and the methods as described above.
The instructions or software to control computing hardware, for example, one or more processors or computers, to implement the hardware components and perform the methods as described above, and any associated data, data files, and data structures, may be recorded, stored, or fixed in or on one or more non-transitory computer-readable storage media. Examples of a non-transitory computer-readable storage medium include read-only memory (ROM), random-access programmable read only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random-access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROMs, CD-Rs, CD+Rs, CD-RWs, CD+RWs, DVD-ROMs, DVD- Rs, DVD+Rs, DVD-RWs, DVD+RWs, DVD-RAMs, BD-ROMs, BD-Rs, BD-R LTHs, BD-REs, blue-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), flash memory, a card type memory such as multimedia card micro or a card (for example, secure digital (SD) or extreme digital (XD)), magnetic tapes, floppy disks, magneto-optical data storage devices, optical data storage devices, hard disks, solid-state disks, and any other device that is configured to store the instructions or software and any associated data, data files, and data structures in a non-transitory manner and provide the instructions or software and any associated data, data files, and data structures to one or more processors or computers so that the one or more processors or computers can execute the instructions. In one example, the instructions or software and any associated data, data files, and data structures are distributed over network-coupled computer systems so that the instructions and software and any associated data, data files, and data structures are stored, accessed, and executed in a distributed fashion by the one or more processors or computers.
While this disclosure includes specific examples, it will be apparent after an understanding of the disclosure of this application that various changes in form and details may be made in these examples without departing from the spirit and scope of the claims and their equivalents. The examples described herein are to be considered in a descriptive sense only, and not for purposes of limitation. Descriptions of features or aspects in each example are to be considered as being applicable to similar features or aspects in other examples. Suitable results may be achieved if the described techniques are performed in a different order, and/or if components in a described system, architecture, device, or circuit are combined in a different manner, and/or replaced or supplemented by other components or their equivalents.
Therefore, in addition to the above disclosure, the scope of the disclosure may also be defined by the claims and their equivalents, and all variations within the scope of the claims and their equivalents are to be construed as being included in the disclosure.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10-2022-0084124 | Jul 2022 | KR | national |
