Patent Application
20230289656
The present disclosure relates to distributed training of an ML model in a communication network.
Increased automation in networks implies that autonomy needs to be extended to ever more aspects of the network. In network management, designing algorithms for optimization of the network has proven to be complex. To overcome this challenge, Artificial Intelligence and Machine Learning (AI/ML) approaches, which eliminate the need for human design of the required algorithms, have been proposed. These AI/ML algorithms may be deployed in the network and service management planes where they replace human effort in fault, capacity and performance management for network domains and user services.
Machine learning is a form of Artificial Intelligence that enables a system to learn from data rather than through explicit programming (taken from Machine Learning For Dummies, IBM Limited Edition, Wiley page 4. https://www.ibm.com/downloads/cas/GB8ZMQZ3).
In managing the call sessions, a lot of data on the performance of the session and on the devices involved in such session is available and can be collected. This data can be used for optimizing and troubleshooting the call sessions using AI/ML algorithms. Similarly, the amount of human effort spent on developing optimal algorithms for signal processing should be reduced which can be achieved with the use of AI/ML algorithms able to learn the optimal processing.
The end result is that different areas of the network will be filled with AI/ML functions (herein also called Cognitive Network Functions (CNFs)) with learning being possible at different parts and domains of the network. For some of this learning, a distributed approach may be used (distributed learning).
FL is a form of machine learning where, instead of model training at a single node, different nodes perform the training in a distributed way. In a distributed machine learning model, a single ML model training is performed distributed to use computation powers of different nodes. Federated learning is different from distributed learning in the sense that each distributed node has its own local training data which may not come from the same distribution; each host (also named distributed training host) computes parameters of its local ML model (in short: computes its ML model) and the central host combines the parameters of all the models to generate a main model. The objective of this approach is to keep the training dataset where it is generated and perform the model training locally at each individual learner (DTH) in the federation. After training a local model, each individual learner transfers its local model (i.e., the parameters thereof) to an aggregating unit (MTH). Optionally, each DTH may also transmit a subset of its own training data to MTH. The MTH utilizes the local model parameters to update an aggregated model (meta-model) which is eventually fed back to the individual local learners for their use. As a result, each local learner (DTH) benefits from the datasets of the other learners only through the aggregated model, shared by the aggregator, without explicitly accessing high volume of training data. AIso, data privacy may be improved in this way because the training data may be privacy-sensitive.
An optimum number of local learners to participate in updating the meta-model, grouping of the local learners, and frequency of local updates and aggregation thereof into the aggregated model, that induce a trade-off between model performance and resource preservation, are application dependent.
Federated learning over wireless networks is very recently investigated topic. Some of the most recent works are:
These references discuss optimization problems related to scheduling problems in FL over wireless networks.
It is an object of the present invention to improve the prior art.
According to a first aspect of the invention, there is provided an apparatus comprising: one or more processors, and memory storing instructions that, when executed by the one or more processors, cause the apparatus to: monitor if a distributed training host receives a request from a meta-training host to provide a machine learning model; check whether a link from the distributed training host to the meta-training host is required for another data communication having higher priority than providing the machine learning model such that the other data communication will block the link for the providing the machine learning model to the meta-training host; inform the meta-training host, in response to the request, that the link is required for the other data communication if the link is required for the other data communication.
According to a second aspect of the invention, there is provided an apparatus comprising: one or more processors, and memory storing instructions that, when executed by the one or more processors, cause the apparatus to: monitor whether, in response to a request from a meta-training host to a distributed training host to provide a machine learning model, a busy information is received that a link from the distributed training host to the meta-training host is required for another data communication than providing the machine learning model; inhibit scheduling a first resource on the link for the providing the machine learning model if the busy information is received.
According to a third aspect of the invention, there is provided a method comprising: monitoring if a distributed training host receives a request from a meta-training host to provide a machine learning model; checking whether a link from the distributed training host to the meta-training host is required for another data communication having higher priority than providing the machine learning model such that the other data communication will block the link for the providing the machine learning model to the meta-training host; informing the meta-training host, in response to the request, that the link is required for the other data communication if the link is required for the other data communication.
According to a fourth aspect of the invention, there is provided a method comprising: monitoring whether, in response to a request from a meta-training host to a distributed training host to provide a machine learning model, a busy information is received that a link from the distributed training host to the meta-training host is required for another data communication than providing the machine learning model; inhibiting scheduling a first resource on the link for the providing the machine learning model if the busy information is received.
Each of the methods of the third and fourth aspects may be a method of distributed training.
According to a fifth aspect of the invention, there is provided a computer program product comprising a set of instructions which, when executed on an apparatus, is configured to cause the apparatus to carry out the method according to any of the third and fourth aspects. The computer program product may be embodied as a computer-readable medium or directly loadable into a computer.
According to some embodiments of the invention, at least one of the following advantages may be achieved:
It is to be understood that any of the above modifications can be applied singly or in combination to the respective aspects to which they refer, unless they are explicitly stated as excluding alternatives.
Further details, features, objects, and advantages are apparent from the following detailed description of the preferred embodiments of the present invention which is to be taken in conjunction with the appended drawings, wherein:
Herein below, certain embodiments of the present invention are described in detail with reference to the accompanying drawings, wherein the features of the embodiments can be freely combined with each other unless otherwise described. However, it is to be expressly understood that the description of certain embodiments is given by way of example only, and that it is by no way intended to be understood as limiting the invention to the disclosed details.
Moreover, it is to be understood that the apparatus is configured to perform the corresponding method, although in some cases only the apparatus or only the method are described.
Some example embodiments of this invention provide a system and mechanism for model collection from the distributed hosts in a federated learning system.
For some network-related machine learning use cases, training may be done in a distributed way but yet the different observations of the distributed units need to be aggregated for good decision at inference. Each instance of the model at the distributed entity has only a subset of the data available for the training because it does not have the data of the other instances of the distributed entity. Thus, only suboptimal performance can be achieved using a model trained separately at each instance of the distributed entity. Better performance may be achieved with a single model that combines the knowledge of the different model instances.
Examples of such use cases include the following:
The concept of distributed learning has not been extensively used in communication networks, although these networks can significantly benefit from its application. There are no mechanisms defined to implement or support such learning, be it between the base stations and UEs, between the core network and UEs, or between OAM or its related automation platforms and the base stations from which the training data is sourced.
Some example embodiments of the invention provide a mechanism to support distributed training of machine learning models in communication networks and the related aggregation of the models through a meta learning process. The entity that undertakes distributed training, herein called a Distributed Training Host (DTH), trains and generates a sub-model while the entity that aggregates the sub-models, here called the Meta Training Host (MTH), generates the final aggregated model, also called a Meta-model.
Some example embodiments of the invention are particularly relevant for interface examples 1 to 3 of the list hereinabove because these interfaces may be involved in data transmission by a user (terminal, e.g. UE) of the network. Note that the list is not exhaustive for the invention. The invention is explained hereinafter in detail for the interface example 1 (interface UE—RAN) because resources on this air interfaces are particularly rare and valuable.
In a wireless network, UEs transmit their local FL models (after the models are trained using their locally collected data) to a base station (gNB) on regular communication links. The gNB, acting as a central controller, generates an aggregated FL model using the received local FL models. gNB may broadcast the aggregated FL model back to all UEs, as illustrated in
The convergence time for federated learning includes not only the computation time on the local learners and the aggregator but also the communication time between them which depends on the wireless channel quality as well as data transmission requirements of the UEs (which delay model transmission). Hence, the FL performance and convergence time will be significantly affected by model collection scheduling. Some example embodiments provide a solution to the problem of: Designing an appropriate model collection scheduling method that helps FL model converge with minimum (or no) loss to QoS of the UEs.
Some example embodiments of the invention provide a method and means to enable the UE to transmit its locally trained model to the gNB at the earliest possibility while also accounting for the UE's own communication requirements. For this reason, the UE, upon receipt of a request from the gNB to transmit the locally trained model, checks if the transmission of the locally trained model to the gNB is possible without jeopardizing another data transmission to the gNB having higher priority. The priority of the other data transmission may be higher e.g. because there is a corresponding QoS requirement for the other data transmission. If the link (uplink) to the gNB will be required for the other data transmission, the UE indicates to the gNB that the uplink will be required for the other transmission, and the UE does not transmit the locally trained model in response to the received request.
In addition, in some example embodiments, it is considered if the locally trained model is sufficiently mature. In some example embodiments, the check whether or not the uplink is required for the another data transmission is performed only if the locally trained model is sufficiently mature. That is, in such example embodiments, the transmission of the trained model from the UE is delayed in cases where the model has been maturely trained but where UE has other pending data to be transmitted on the uplink.
Some example embodiments of the invention provide messages for expressing constraints related to the UE's transmission requirements and options for the course of action from the gNB. Some example embodiments of the invention provide the following feature:
The UEs which require the uplink for another data transmission choose not to immediately transmit their locally trained models. Instead, the UE sends an indication that the link is required for another data transmission (e.g. transmission resources to gNB not available for the model transmission).
In addition, in some example embodiments, each UE indicating the link being required, may inform the gNB on a respective time to be considered by the gNB for receiving the model from that UE. Each of the UEs may send its own respective time, determined independently from any of the other UEs. The time may have one of two different meanings:
In some example embodiments, the UE is equipped with only one of the corresponding algorithms. Hence, UE may indicate only the respective one of these times. In some example embodiments, the UE is equipped with both of these algorithms, and the UE may send either of these times depending on circumstances. In some example embodiments, a UE being equipped with both of these algorithms may send both of these times. If a UE being equipped with the algorithm b) cannot determine the time when it is sure to have finished the pending transmission, it may indicate a time very far in the future (e.g. one day or one month) for this time, or may not indicate any time.
If the uplink from a UE is required for another data transmission, the UE may not immediately transmit its locally trained model(s) to gNB. It may signal to gNB to ping (request) it again after a short time interval. There is a high probability that the UE will have finished its other transmission by that time and will be able to transmit its updated model. In some example embodiments, this time interval may be configured by the network, e.g. based on general considerations. In such example embodiments, the UE may not provide any time indication to the gNB because it is anyway known by the gNB from the configuration.
The UE may also decide not to send the locally trained model because it is not sufficiently mature. In this case, typically, the gNB should ping (request) the UE after a long time interval (relative to the time interval indicated when the uplink is required for another data transmission) because maturing a ML model may take quite some time. By requesting the UE after a long time interval, unnecessary signalling overhead for further unsuccessful attempts may be saved.
The MTH (gNB) may request the UE again when it requests for delayed response due to its other communication requirements. However, if MTH collects enough models from other UEs and the aggregated model converges, it may decide to skip those UEs indicating that their uplink is required for another communication for this round of model updates. There may be more reasons why gNB decides not to request later the locally trained model from a UE with required uplink. E.g., the generation or updating of the aggregated model may be abandoned. AIso, the gNB may be busy with other tasks at that time such that the gNB defers the request to the UE by some additional time interval.
Hereinafter, an example embodiment is described at greater detail, wherein the UE checks both whether the local model is sufficiently mature and whether its uplink is required for another data transmission. An example embodiment where the UE does not check if the local model is sufficiently mature may be derived from this model by assuming that the check for maturity is always affirmative (i.e., the model is assumed to be mature, regardless of whether or not it is actually mature).
ML model maturity defines how mature the ML model is in terms of the training process. For example, it may be based on at least one of:
In this example embodiment, the UE checks first if the model is sufficiently mature, and only then, it checks whether or not its uplink is required for another data transmission. Thus, when the gNB requests for the model, one of three statuses may be determined, as described in Table 1, namely Immature, Mature_available or Mature_Busy. UE may provide the status indications “Immature” and “Mature_busy” to the gNB in response to the request to provide the trained ML model. Instead of or in addition to the status indication “Mature_available”, it may provide the trained ML model to the gNB upon the request. In some example embodiments, instead of or in addition to the status “immature”, the status “Static” may be used. It describes a case that the model parameters are not different (or at least not substantially different, wherein the meaning of the term “substantially” is model dependent) from the ones already provided at a previous update of the local model at the gNB. If only one of the statuses “immature” and “static” is used, it may include the information of the other of these statuses.
The three outcomes represent the combination of the status of the training and the status of the UE's transmission resources. These combinations result in three of four UE's responses, depending on whether or not the check for the transmission resources is performed if the model is not mature:
In an embodiment, instead of using the gNB-configured short_ping_back_timer (T2), the UE computes a wait timer T3, computed e.g. based on the amount of pending other data. The gNB may then wait for the model or indicate that the wait time is too long and that the model is no longer needed.
In some example embodiments, similar to
In some example embodiments, following the wait time, the MTH (gNB) may inform the UE that MTH still expects the model. For example, this information may be send if MTH determines that the transmission of the model from the UE is still required in view of the number, quality, and/or maturity of models received from other UEs. In these cases, the information that the MTH still expects the model may be used as an alternative to not sending the message to the UE to discard (suspend) the transmission of the model. However, in some example embodiments, MTH may inform the UE that MTH still expects the model even if MTH does not check if the transmission of the model from the UE is still required in view of the number, quality, and/or maturity of models received from other UEs. For example, the message may be considered as a kind of a heartbeat between the ML model applications on the UE and on the gNB. In these example embodiments (i.e., in example embodiments, where the MTH may inform the UE after the wait time that MTH still expects the model), UE uploads the model to MTH (gNB) only if it receives the information that MTH still expects the model.
In some example embodiments, MTH may be enabled to send a message to the UE to discard (or suspend) the transmission and to send an information to the UE that MTH still expects the model. Depending on the implementation, UE may upload the model to MTH after the wait time only if UE does not receive the message on discarding and UE receives the information that MTH still expects the model. In other implementations, UE may upload the model to MTH after the wait time if at least one of the above conditions (i.e. UE does not receive the message on discarding, and UE receives the information that MTH still expects the model) is fulfilled.
In some example embodiments, MTH does not send any message to the UE to discard (or suspend) the transmission and does not send any information to the UE that MTH still expects the model. In such example embodiments, UE uploads the model to MTH after the wait time. In such example embodiments, MTH typically does not check if the transmission of the model from the UE is still required in view of the number, quality, and/or maturity of models received from other UEs.
The apparatus comprises means for monitoring 10, means for checking 20, and means for informing 30. The means for monitoring 10, means for checking 20, and means for informing 30 may be a monitoring means, checking means, and informing means, respectively. The means for monitoring 10, means for checking 20, and means for informing 30 may be a monitor, checker, and informer, respectively. The means for monitoring 10, means for checking 20, and means for informing 30 may be a monitoring processor, checking processor, and informing processor, respectively.
The means for monitoring 10 monitors if a distributed training host receives a request from a meta-training host to provide a machine learning model (S10). For example, the distributed training host may be a UE, and the meat-training host may be a gNB.
The means for checking 20 checks whether a link from the distributed training host to the meta-training host is required for another data communication having higher priority than providing the machine learning model such that the other data communication will block the link for the providing the machine learning model to the meta-training host (S20).
S10 and S20 may be performed in any arbitrary sequence. S10 and S20 may be performed fully or partly in parallel. In some example embodiments, S20 is only performed if the distributed training host receives the request (S10=yes).
If the link is required for the other data communication (S20=yes), the means for informing 30 inform the meta-training host, in response to the request (S10), that the link is required for the other data communication (S30).
If the MTH (gNB) receives the busy indication (i.e., that the link is required for another data communication), it should typically not schedule any resource for the transmission of the machine learning model. Nevertheless, in some example embodiments, the apparatus may additionally comprise a means for inhibiting. The means for inhibiting may be an inhibiting means, or an inhibitor, or an inhibiting processor. If the link is required for the other data communication (S20=yes), the means for inhibiting inhibits providing, in response to the request (S10), the machine learning model to the meta-training host on the link. S30 and the inhibiting may be performed in any arbitrary sequence. S30 and the inhibiting may be performed fully or partly in parallel.
The apparatus comprises means for monitoring 110 and means for inhibiting 120. The means for monitoring 110 and means for inhibiting 120 may be a monitoring means and inhibiting means, respectively. The means for monitoring 110 and means for inhibiting 120 may be a monitor and inhibiter, respectively. The means for monitoring 110 and means for inhibiting 120 may be a monitoring processor and inhibiting processor, respectively.
The means for monitoring 110 monitors whether a busy information is received (S110). The busy information is received in response to a request from a meta-training host to the distributed training host to provide a machine learning model. The busy information indicates that a link from the distributed training host to the meta-training host is required for another data communication than providing the machine learning model.
If the busy information is received (S110=yes), the means for inhibiting 120 inhibits scheduling a resource on the link for the providing the machine learning model (S120).
On the other hand, if the busy information is not received in response to the request to the distributed training host to provide a machine learning model, in some example embodiments, the apparatus may allow the meta-training host (gNB) to schedule a resource for the transmission of the model on the link (uplink) from the distributed training host to the meta-training host. Whether or not the resource is actually scheduled may depend on other conditions, too.
Some example embodiments of the invention are described where the uplink is required for another data transmission with a higher priority than providing the local ML model. However, this is not the only potential reason why the UE does not provide the local ML model on request from the gNB. For example, the processing power and/or the memory of the UE may be exhausted due to other tasks with higher priority. In these cases, the UE may indicate to the gNB that it is busy and does not provide the local ML model on the request from the gNB.
Some example embodiments are explained with respect to a 5G network. However, the invention is not limited to 5G. It may be used in 3G or 4G networks and 3GPP networks of future generations if they support distributed training. It is not even limited to 3GPP networks. It may be used in other wired or wireless communication networks (e.g. WiFi networks).
One piece of information may be transmitted in one or plural messages from one entity to another entity. Each of these messages may comprise further (different) pieces of information.
Names of network elements, network functions, protocols, and methods are based on current standards. In other versions or other technologies, the names of these network elements and/or network functions and/or protocols and/or methods may be different, as long as they provide a corresponding functionality.
A terminal (UE) may be e.g. a mobile phone, a smartphone, a MTC device, a laptop etc. The user may be a human user or a machine (e.g. in machine-type communication (MTC)).
A gNB is an example of a base station to which some example embodiments of the invention are applicable. Another example is an eNB. The MTH may be integrated with the base station or stand-alone therefrom.
If not otherwise stated or otherwise made clear from the context, the statement that two entities are different means that they perform different functions. It does not necessarily mean that they are based on different hardware. That is, each of the entities described in the present description may be based on a different hardware, or some or all of the entities may be based on the same hardware. It does not necessarily mean that they are based on different software. That is, each of the entities described in the present description may be based on different software, or some or all of the entities may be based on the same software. Each of the entities described in the present description may be deployed in the cloud.
According to the above description, it should thus be apparent that example embodiments of the present invention provide, for example, a distributed training-host such as a UE, an MTC device, a RAN network element (such as a gNB or eNB) or a component thereof, an apparatus embodying the same, a method for controlling and/or operating the same, and computer program(s) controlling and/or operating the same as well as mediums carrying such computer program(s) and forming computer program product(s). According to the above description, it should thus be apparent that example embodiments of the present invention provide, for example, an meta-training host such as a RAN, or a component thereof (e.g. eNB or gNB), an OAM, or an automation platform, an apparatus embodying the same, a method for controlling and/or operating the same, and computer program(s) controlling and/or operating the same as well as mediums carrying such computer program(s) and forming computer program product(s).
Implementations of any of the above described blocks, apparatuses, systems, techniques or methods include, as non-limiting examples, implementations as hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controller or other computing devices, or some combination thereof. Each of the entities described in the present description may be embodied in the cloud.
It is to be understood that what is described above is what is presently considered the preferred embodiments of the present invention. However, it should be noted that the description of the preferred embodiments is given by way of example only and that various modifications may be made without departing from the scope of the invention as defined by the appended claims.
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/EP2020/071756 | 8/3/2020 | WO |
