WIRELESS FEDERATED LEARNING FRAMEWORK AND RESOURCE OPTIMIZATION METHOD

Abstract

A wireless federated learning (FL) framework and a resource optimization method are provided to resolve a problem that FL is not suitable for many hardware-constrained Internet of Things (IoT) devices with a small amount of computing resources. In the framework, users with sufficient computing resources upload locally trained model parameters to a base station, and users with limited computing resources only need to send training data to the base station. The base station performs data training and model aggregation to obtain a global model. In this way, the users with limited computing resources and the users with sufficient computing resources cooperatively train the global model. To improve a data transmission rate and reduce an aggregation error of FL, a non-convex optimization problem is constructed to jointly design user transmit power and a reception strategy of the base station, and solves the problem through a successive convex approximation (SCA) method.

Description

CROSS REFERENCE TO THE RELATED APPLICATIONS

This application is based upon and claims priority to Chinese Patent Application No. 202310182495.3, filed on Mar. 1, 2023, the entire contents of which are incorporated herein by reference.

TECHNICAL FIELD

The present disclosure relates to the field of wireless resource management and computer technologies. and in particular, to a wireless federated learning (FL) framework and a resource optimization method.

BACKGROUND

A large quantity of Internet of Things (IoT) devices such as low-end sensors, high-definition cameras, and advanced robots are deployed on a network edge such that the next-generation network can support various emerging applications, such as smart factories, autonomous driving, interactive games, and the metaverse. Success of these applications is based on full use of huge amounts of data.

Traditional centralized learning (CL) needs to acquire data samples from edge users. However, a large amount of data is distributed on geographically isolated terminals, which brings huge communication overheads to CL. In addition, considerable transmission delays affect real-time performance of applications. Furthermore, directly sending raw data poses a risk of privacy leakage for individual users.

To address the main challenge of CL, federated learning (FL) allows users to locally update models, thereby avoiding the transmission of raw data. In comparison with CL, a core idea of FL is to replace the data upload with model sharing. This not only plays an important role in protecting user privacy, but also greatly reduces the communication burden.

Success of FL relies on abundant computing resources on IoT terminals. However, this is impractical for many hardware-constrained IoT devices, such as low-end sensors with limited computing resources. Therefore, it is extremely challenging to directly implement existing (centralized or federated) machine learning frameworks in heterogencous IoT devices with different computing capabilities.

To address these challenges, it is necessary to explore a new machine learning framework to ease the computational requirements for these resource-constrained IoT devices. In addition, to improve the data transmission rate and reduce aggregation errors of FL, it is important to propose a new learning and optimization method to make full use of distributed data and computing resources of an FL server and all clients.

SUMMARY

To overcome defects of the prior art, the present disclosure provides a wireless FL framework and a resource optimization method.

To achieve the foregoing objective, the present disclosure provides the following technical solutions:

The present disclosure provides a wireless FL framework, including:

• • N CL users with limited computing resources, where the CL users send training data to a base station for CL to participate in FL;
  • K FL users with sufficient computing resources, where the FL, users obtain local models through local training data, and upload local model parameters as an aggregation model to the base station; and
  • the base station serving as an FL server and configured to compute a global model, where the base station performs CL on the training data accumulated by the CL users to obtain a CL model, and performs weighted summation on the CL model and the received aggregation model based on a data amount to obtain the global model.

Further, it is assumed that there are T FL cycles in total, which are represented as a set T={1, 2, . . . , T}. In the t^th FL cycle, a local model update formula of the k^th FL user is as follows:

$w_k^{\left(t+1\right)}={\tilde{w}}^{\left(t\right)}-\eta g_k^{\left(t\right)},g_k^{\left(t\right)}=\nabla F_k\left({\tilde{w}}^{\left(t\right)};D_k^{\left(t\right)}\right),\forall k$

A centralized update formula of the base station is as follows:

${\stackrel{\_}{w}}^{\left(t+1\right)}={\tilde{w}}^{\left(t\right)}-\eta {\stackrel{\_}{g}}^{\left(t\right)},$ ${\stackrel{\_}{g}}^{\left(t\right)}=\nabla F\left({\tilde{w}}^{\left(t\right)};D^{\left(t\right)}\right)$

A global model aggregation formula is as follows:

${\tilde{w}}^{\left(t+1\right)}=\frac{❘D^{\left(t\right)}❘{\stackrel{\_}{w}}^{\left(t+1\right)}+{\sum }_{k=N+1}^{N+K}❘D_k^{\left(t\right)}❘w_k^{\left(t+1\right)}}{❘D^{\left(t\right)}❘+{\sum }_{k=N+1}^{N+K}❘D_k^{\left(t\right)}❘}$

where η is a learning rate of a stochastic gradient descent method, D^(t)=Σ_t=1^tΣ_n=1^ND_n^(t) is a training data set accumulated at the base station in t cycles, FR(⋅) and g_k^(t) are respectively a local loss function and a gradient of the k^th FL user, F(⋅) and g^(t) are respectively a loss function and a gradient of the CL at the base station, and {tilde over (w)}^(t+1) is the global model aggregated at the base station.

The present disclosure further provides a resource optimization method for the foregoing wireless FL framework, including the following steps:

• • S1: initializing, by the base station, a training task and the global model, and sending the global model to all users:
  • S2: after receiving the global model, computing, by the FL user, a local loss function and a gradient based on the local training data, and updating the local model;
  • S3: using, by the FL users and the CL users, a same frequency band to respectively upload the local models and send the training data simultaneously; and detecting, by the base station, a received signal through imperfect successive interference cancellation (SIC), and separating the training data and the aggregation model;
  • S4: obtaining data transmission rates of all CL users and a mean square error (MSE) of the aggregation model of all FL users, and detecting signals of all CL users in an order of 1, 2, . . . , N through imperfect SIC;
  • S5: constructing a non-convex optimization problem based on the data transmission rates and the MSE of the aggregation model; and
  • S6: transforming and solving the non-convex optimization problem in S5 through a successive convex approximation (SCA) method, and outputting user transmit power and a reception strategy.

Further, the local loss function in S2 is an MSE loss function or a cross-entropy loss function.

Further, S3 specifically includes:

• • before the user uploads the local model or sends the training data, normalizing a local training data set {D_n} of the CL users into a communication symbol set {s_n} and the local models {w_k} of the FL users into a computation symbol set {s_k}, where a superimposed signal received by the base station is as follows:

$y=\underset{\mathrm{CL}\mathrm{users}}{\underbrace{{\sum }_{n=1}^N{h}_n\sqrt{p_n}{s}_n}}+\underset{\mathrm{FL}\mathrm{users}}{\underbrace{{\sum }_{k=N+1}^{N+K}{h}_k\sqrt{p_k}{s}_k}}+\underset{\mathrm{Noise}}{\underbrace{n_0}}$

• • where h_n(h_k) is a channel coefficient from the n^th (k_th) user to the base station, p_n(p_k) is transmit power of the n^th (k^th) user and falls within an interval [0, P_max], and n₀˜CN(0,σ²) is additive noise in a channel;
  • adjusting the user transmit power such that SIC constraints of different users satisfy:

$❘h_1\sqrt{p_1}❘\ge ❘h_2\sqrt{p_2}❘\ge \dots \ge ❘h_N\sqrt{p_N}❘\ge ❘h_k\sqrt{p_k}❘,\forall k$

• • after receiving the superimposed signal y, the signals {s_n} of all CL users are detected by the base station in the order of 1, 2, . . . , N , where after the signals of all CL users are detected, a residual signal is as follows:

$\hat{y}=\sqrt{\varpi }\sum _{n=1}^N{h}_n\sqrt{p_n}{s}_n+\sum _{k=N+1}^{N+K}{h}_k\sqrt{p_k}{s}_k+n_0$

• • where ω is an interference coefficient of the residual signal, ω=0 corresponds to perfect SIC, and ω=1 corresponds to no SIC; and
  • obtaining, by the base station, an estimated value ŝ=aŷ/K of the aggregation model from the residual signal ŷ by using a receiving factor a, and decoding the signals {s_n} and ŝ into the training data set of all CL users and the aggregation model through denormalization post-processing, to obtain an average FL model.

Further, in S4, a formula of the data transmission rate of the n^th CL user is as follows:

$R_n={\log }_2\left(1+\frac{{❘h_n❘}^2{p}_n}{\varpi \sum _{i=1}^{n-1}{❘h_i❘}^2{p}_i+\sum _{j=n+1}^{N+K}{❘h_j❘}^2{p}_j+{\sigma }^2}\right),\forall n$

• • where ω is the interference coefficient of the residual signal, ω=0 corresponds to perfect SIC, ω=1 corresponds to no SIC, and a sum of the data transmission rates of all CL users is R_sum=Σ_n=1^NR_n.

Further, in S4, a formula of the MSE of the aggregation model is as follows:

$\mathrm{MSE}=\frac{1}{K^2}\left({\vartheta }_{\mathrm{CL}}+{\sum }_{k=N+1}^{N+K}{❘{\mathrm{ah}}_k\sqrt{p_k}-1❘}^2+{❘a❘}^2{\sigma }^2\right)$

• • where

${\vartheta }_{\mathrm{CL}}=\varpi {\sum }_{n=1}^N{❘{\mathrm{ah}}_n\sqrt{p_n}❘}^2$

• •  is interference of the signals of the CL users.

Further, in S5, the non-convex optimization problem is constructed as follows:

$\underset{p,a}{\max }{R}_{\mathrm{sum}}-\lambda \mathrm{MSE}$ $s.t.R_n\ge R_{\min },\forall n$ $p_n,p_k\in \left[0,P_{\max }\right],\forall n,k$ $\mathrm{MSE}\le {ò}_0,$ $❘h_1\sqrt{p_1}❘\ge ❘h_2\sqrt{p_2}❘\ge \dots \ge ❘h_N\sqrt{p_N}❘\ge ❘h_k\sqrt{p_k}❘,\forall k$

• • where p=[p₁, p₂, . . . , p_N, p_n+1, . . . , P_N+K] is the transmit power, R_min is a minimum data transmission rate required by the CL user, P_max is maximum transmit power of all users, is a maximum aggregation model error that the FL users can tolerate, and λ is a constant used to strike a balance between the sum rate R_sum and the MSE.

Further, a convex optimization problem obtained through transformation in S6 is as follows:

$\underset{p,a,\gamma ,b}{\max }{\sum }_{n=1}^N{\log }_2\left(1+{\gamma }_n\right)-\lambda \mathrm{MSE}$ $s.t.h_1^2{p}_1\ge h_2^2{p}_2\ge \dots h_N^2{p}_N\ge h_k^2{p}_k,\forall k\in K$ ${\log }_2\left(1+{\gamma }_n\right)\ge R_{\min },\forall n\in N$ $\mathrm{MSE}\le {?}_0,$ $p_n,p_k\in \left[0,P_{\max }\right],\forall n,k$ $h_n^2{p}_n\ge \varpi {\sum }_{i=1}^{n-1}{h}_i^2\left(\frac{{\tau }_{\mathrm{in}}}{2}{p}_i^2+\frac{1}{2{\tau }_{\mathrm{in}}}{\gamma }_n^2\right)+{\sum }_{j=n+1}^{N+K}{h}_j^2\left(\frac{{\tau }_{\mathrm{jn}}}{2}{p}_j^2+\frac{1}{2{\tau }_{\mathrm{jn}}}{\gamma }_n^2\right)+{\sigma }^2{\gamma }_n,$ ${\left(a^{\left(\ell \right)}\right)}^2+2a^{\left(\ell \right)}\left(a-a^{\left(\ell \right)}\right)\ge b_i^2/p_i,\forall i$

• • where γ=[γ₁, γ₂, . . . , γ_N] and b=[b₁, b₂, . . . , b_N, b_N+1, b_N+2, . . . , b_N+K] are introduced auxiliary vectors, γ_n=2^Rn−1,∀n, b_i=a√{square root over (p_i)},∀i∈N∪K, and τ_in and τ_jn are convex upper bound (CUB) coefficients.

Further, solving the convex optimization problem in S6 includes:

• • S6.1: initializing p⁽⁰⁾, a⁽⁰⁾, γ⁽⁰⁾, b⁽⁰⁾, ω, a maximum quantity L of iterations, and a threshold ε, and setting an iteration index =0;
  • S6.2: computing an objective function value =−λMS;
  • S6.3: given , , , and , updating and by using =/ and =/.
  • S6.4: given and , solving the convex optimization problem through a mathematical toolkit CVX to obtain , , , and ;
  • S6.5: computing an objective function value =−λMS;
  • S6.6: updating =+1 and computing Δ_U=|−|; and
  • S6.7: repeating S6.3 to S6.6 until Δ_U≤ε or ≥and outputting user transmit power and a receiving factor .

Compared with the prior art, the present disclosure has the following beneficial effects:

• • (1) The wireless FL framework provided in the present disclosure can allow the users with limited computing resources and the users with sufficient computing resources to participate in FL together, and make full use of data of heterogeneous users to improve performance of a machine learning model.
  • (2) In the present disclosure, all users use the same frequency band to upload the models or send the data simultaneously such that spectrum resources can be effectively saved and communication delay can be reduced. The base station detects the signals through imperfect SIC. This is more in line with an actual situation
  • (3) In the resource optimization method provided in the present disclosure, the non-convex optimization problem is constructed based on the user data transmission rates and the MSE of the aggregation model to jointly design the user transmit power and the reception strategy of the base station. The problem is solved through the SCA method such that the sum rate can be effectively maximized and the MSE can be effectively minimized.

BRIEF DESCRIPTION OF THE DRAWINGS

To describe the technical solutions in embodiments of the present application or in the prior art more clearly; the following briefly describes the accompanying drawings required for describing the embodiments. Apparently, the accompanying drawings in the following description show merely some embodiments of the present disclosure, and persons of ordinary skill in the art may still derive other accompanying drawings from these accompanying drawings.

FIG. 1 is a structural diagram of a wireless FL framework and a transceiver according to an embodiment of the present disclosure; and

FIG. 2 is an algorithm flowchart of a resource optimization method for a wireless FL framework according to an embodiment of the present disclosure.

DETAILED DESCRIPTION OF THE EMBODIMENTS

Success of FL relies on abundant computing resources of local users. However, this is impractical for many hardware-constrained IoT devices, such as low-end sensors with small amounts of computing resources. To resolve the foregoing problem, the present disclosure proposes a wireless FL framework such that users with limited computing resources and users with sufficient computing resources cooperatively train a global model. In the framework, the users with sufficient computing resources upload locally trained model parameters to a base station, and the users with limited computing resources only need to send training data to the base station. The base station performs data training and model aggregation to obtain the global model. In addition, to improve a data transmission rate and reduce an aggregation error of FL, the present disclosure constructs a non-convex optimization problem to jointly design user transmit power and a reception strategy of the base station, and solves the problem through an SCA method.

To better understand the technical solutions, the foregoing describes in detail a method in the present disclosure with reference to the accompanying drawings.

Referring to FIG. 1, this embodiment includes the following steps:

• • S1: A wireless FL framework is proposed such that users with limited computing resources and users with sufficient computing resources cooperatively train a global model. In the framework, the users with sufficient computing resources upload trained model parameters to a base station, and the users with limited computing resources only need to send training data to the base station.

As shown in FIG. 1, in the framework, it is considered that there are N users with limited computing resources and K users with sufficient computing resources, and the base station serves as an FL server and is configured to compute the global model. The users with limited computing resources cannot meet a requirement of local data training due to limited computing power and can only send the training data to the base station for CL to participate in FL, which are also referred to as CL users and are represented as a set N{1, 2, . . . , N}. The users with sufficient computing resources can obtain local models from local training data and upload local model parameters to the base station without sending training data, which are also referred to as FL users and are represented as a set K{N+1, N+2, . . . , N+K}.

• • S2. CL is performed by the base station on accumulated data to obtain a CL model, and weighted summation is performed on the CL model and the received aggregation model based on a data amount to obtain the global model. It is assumed that there are T FL cycles in total, which are represented as a set T={1, 2 . . . , T}. In the t^th cycle, the training data of the n^th CL user is denoted as D_n^(t), the local model of the k^th FL user is denoted as w_k^(t), and a stochastic gradient descent method is used for training. In this case, local model update by the user, centralized update by the base station, and global model aggregation are expressed as follows:
  • Local model update: w_k^(t+1)={tilde over (w)}^(t)−ηg_k^(t), ∇F_k({tilde over (w)}^(t);D_k^(t),∀k.
  • Centralized update by the base station: w_k^(t+1)={tilde over (w)}^(t)−ηg^(t), g^(t)=∇F({tilde over (w)}^(t);D^(t)).
  • Global model aggregation:

${\tilde{w}}^{\left(t+1\right)}=\frac{❘D^{\left(t\right)}❘{\stackrel{\_}{w}}^{\left(t+1\right)}+{\sum }_{k=N+1}^{N+K}❘D_k^{\left(t\right)}❘w_k^{\left(t+1\right)}}{❘D^{\left(t\right)}❘+{\sum }_{k=N+1}^{N+K}❘D_k^{\left(t\right)}❘}.$

• • η is a learning rate of the stochastic gradient descent method. D^(t)=Σ_t=1^tΣ_n=1^ND_n^(t) is a training data set accumulated at the base station in t cycles. F_k(⋅) and g_k^(t) are respectively a local loss function and a gradient of the k^th FL user. F(⋅) and g^(t) are respectively a loss function and a gradient of the CL at the base station. {tilde over (w)}_k^(t+1) is the global model aggregated at the base station.
  • S3: A training task and the global model are initialized by the base station, and the global model is sent to all users.
  • S4: The local models are updated by the users with sufficient computing resources based on the local training data After receiving the global model. the user with sufficient computing resources, namely, the FL user, computes a local loss function and a gradient based on the local training data. The loss function may be an MSE loss function or a cross-entropy loss function. This depends on the specific learning task. Then, the FL user updates the local model through the local model update formula in S2.
  • S5: A same frequency band is used by the users with sufficient computing resources and the users with limited resources to respectively upload the local models and send the training data simultaneously. A received signal is detected by the base station through imperfect SIC, and the training data and the aggregation model are separated.

Considering a contradiction between limited bandwidth and a large quantity of users in actual IoT, to save communication bandwidth and reduce delay, all users use the same timeslot and frequency band to upload the local models or send the training data in the present disclosure. Before the user uploads the local model or sends the training data, a local training data set {D_n} of the CL users is normalized into a communication symbol set {s_n} and the local models {w_k} of the FL users are normalized into a computation symbol set {s_k}. A superimposed signal received by the base station is as follows:

$y=\underset{\mathrm{CL}\mathrm{users}}{\underbrace{{\sum }_{n=1}^N{h}_n\sqrt{p_n}{s}_n}}+\underset{\mathrm{FL}\mathrm{users}}{\underbrace{{\sum }_{k=N+1}^{N+K}{h}_k\sqrt{p_k}{s}_k}}+\underset{\mathrm{Noise}}{\underbrace{n_0}}$

• • h_n(h_k) is a channel coefficient from the n^th (k_th) user to the base station, p_n(p_k) is transmit power of the n^th (k^th) user and falls within an interval [0, P_max], and n₀˜CN(0,σ²) is additive noise in a channel. After receiving the superimposed signal, the base station detects the signal through imperfect SIC due to a decoding error, and separates the training data and the aggregation model. User transmit power is adjusted such that SIC constraints of different users satisfy:

$❘h_1\sqrt{p_1}❘\ge ❘h_2\sqrt{p_2}❘\ge \dots \ge ❘h_N\sqrt{p_N}❘\ge ❘h_k\sqrt{p_k}❘,\forall k$

After receiving the superimposed signal y, the base station detects the signals {s_n} of all CL users in an order of 1, 2, . . . , N . After the signals of all CL users are detected, a residual signal is as follows:

$\hat{y}=\sqrt{\varpi }\sum _{n=1}^N{h}_n\sqrt{p_n}{s}_n+\sum _{k=N+1}^{N+K}{h}_k\sqrt{p_k}{s}_k+n_0$

• • ω is an interference coefficient of the residual signal, ω=0 corresponds to perfect SIC, and ω=1 corresponds to no SIC. To obtain an average FL model, the base station obtains an estimated value ŝ=aŷ/K of the aggregation model from the residual signal ŷ by using a receiving factor a, and decode the signals {s_n} and ŝ through post-processing such as denormalization into the training data set of all CL users and the aggregation model.
  • S6: Data transmission rates of all users with limited computing resources and an MSE of the aggregation model of all users with sufficient computing resources are obtained. The signals of all CL users are detected in the order of 1, 2, . . . , N through imperfect SIC based on the expression of the superimposed signal received by the base station in S5. The data transmission rate of the n^th CL user is as follows:

$R_n={\log }_2\left(1+\frac{{❘h_n❘}^2{p}_n}{\varpi \sum _{i=1}^{n-1}{❘h_i❘}^2{p}_i+\sum _{j=n+1}^{N+K}{❘h_j❘}^2{p}_j+{\sigma }^2}\right),\forall n$

• • ω is an interference coefficient of the residual signal, ω=0 corresponds to perfect SIC, and ω=1 corresponds to no SIC. A sum (sum rate) of the data transmission rates of all CL users is R_sum=Σ_n=1^NR_n.

It is assumed that an aggregation model obtained by the base station is ideally

$s={\sum }_{k=N+1}^{N+K}{s}_k/K.$

The actual aggregation model obtained in S5 is ŝ=aŷ/K. An aggregation error is measured by the MSE. The MSE is expressed as follows:

$\mathrm{MSE}=\frac{1}{K^2}\left({\vartheta }_{\mathrm{CL}}+{\sum }_{k=N+1}^{N+K}{❘{\mathrm{ah}}_k\sqrt{p_k}-1❘}^2+{❘a❘}^2{\sigma }^2\right)$

${\vartheta }_{\mathrm{CL}}=\varpi {\sum }_{n=1}^N{❘{\mathrm{ah}}_n\sqrt{p_n}❘}^2$

• • is the interference of the signals of the CL users.
  • S7: A non-convex optimization problem is constructed based on the data transmission rates and the MSE of the aggregation model. The communication-centric CL user hopes that the data transmission rate can be maximized, and the computing-centric FL user hopes that the MSE of the aggregation model can be minimized. To meet the requirements of both types of users, the present disclosure jointly designs user transmit power and a reception strategy of the base station. The optimization problem is constructed as follows:

$\begin{array}{cc}\underset{p,a}{\max }& R_{\mathrm{sum}}-\lambda \mathrm{MSE}\\ s.t.& R_n\ge R_{\min },\forall n\\ & p_n,p_k\in \left[0,P_{\max }\right],\forall n,k\\ & \mathrm{MSE}\le {ò}_0,\\ & ❘h_1\sqrt{p_1}❘\ge ❘h_2\sqrt{p_2}❘\ge \dots \ge ❘h_N\sqrt{p_N}❘\ge ❘h_k\sqrt{p_k}❘,\forall k\end{array}$

• • p=[p₁, p₂, . . . , p_N, p_n+1, . . . , P_N+K] is the transmit power, R_min is a minimum data transmission rate required by the CL user. P_max is maximum transmit power of all users. is a maximum aggregation model error that the FL users can tolerate. λ is a constant used to strike a balance between the sum rate and the MSE. Because the optimization variables p,a are coupled to each other in the objective function and the constraints, the foregoing optimization problem is a non-convex optimization problem, and it is difficult to obtain an optimal solution.
  • S8: The non-convex optimization problem in S7 is transformed and solved through an SCA method. In the present disclosure, to solve the non-convex optimization problem constructed in S7, auxiliary variables γ_n=2^Rn−1,∀n, and b_i=a√{square root over (p_i)},∀i∈N∪K are introduced. The data transmission rate R_n can be rewritten as R_n=log₂(1+γ_n). The MSE can be rewritten as follows:

$\mathrm{MSE}=\frac{1}{K^2}\left(\varpi {\sum }_{n=1}^N{h}_n^2{b}_n^2+{\sum }_{k=N+1}^{N+K}{\left(h_k{b}_k-1\right)}^2+a^2{\sigma }^2\right)$

Therefore, the non-convex optimization problem in S7 is transformed into the following optimization problem:

$\begin{array}{cc}\underset{p,a,\gamma ,b}{\max }& {\sum }_{n=1}^N{\log }_2\left(1+{\gamma }_n\right)-\lambda \mathrm{MSE}\\ s.t.& h_1^2{p}_1\ge h_2^2{p}_2\ge \dots h_N^2{p}_N\ge h_k^2{p}_k,\forall k\in K\\ & {\log }_2\left(1+{\gamma }_n\right)\ge R_{\min },\forall n\in N\\ & \frac{h_n^2{p}_n}{{\psi }_n+{\sigma }^2}\ge {\gamma }_n,\forall n\in N\\ & a\sqrt{p_i}\ge b_i,\forall i\in N\bigcup K\\ & \mathrm{MSE}\le {?}_0,\\ & p_n,p_k\in \left[0,P_{\max }\right],\forall n,k\end{array}$

• • γ=[γ₁, γ₂, . . . , γ_N] and b=[b₁, b₂, . . . , b_N, b_N+1, b_N+2, . . . , b_N+K] are introduced auxiliary vectors and are used as optimization variables.

${\psi }_n=\varpi {\sum }_{i=1}^{n-1}{h}_i^2{p}_i+{\sum }_{j=n+1}^{N+K}{h}_j^2{p}_j$

• •  is the interference of the signal of the n^th CL user. In this case, R_n is a concave function with respect to {γ_n}, and MSE is a convex function with respect to a and {b_i}. Therefore, the objective function of the foregoing optimization problem is a concave function with respect to a, {b_i}, and {γ_i}. It can be learned that in the constraints of the foregoing optimization problem, except that

$\frac{h_n^2{p}_n}{{\psi }_n+{\sigma }^2}\ge {\gamma }_n,$

• •  ∀n∈N and a√{square root over (p_i)}≥b_i,∀i∈N∪K are non-convex, the other constraints are convex. The following transforms the two non-convex constraints into convex constraints through CUB and SCA methods.

To solve the non-convex constraint

$\frac{h_n^2{p}_n}{{\psi }_n+{\sigma }^2}\ge {\gamma }_n,$

∀n∈N, it is equivalently transformed into the following expression:

$h_n^2{p}_n\ge \varpi {\sum }_{i=1}^{n-1}{h}_i^2{p}_i{\gamma }_n+{\sum }_{j=n+1}^{N+K}{h}_j^2{p}_j{\gamma }_n+{\sigma }^2{\gamma }_n$

It can be learned that the product terms p_iγ_n and p_jγ_n in the foregoing expression are non-convex. To solve them, the present disclosure lets f(p_iγ_n)=p_iγ_n. A CUB of f(p_iγ_n) is

$g\left(p_i,{\gamma }_n,{\tau }_{\mathrm{in}}\right)=\frac{{\tau }_{\mathrm{in}}}{2}{p}_i^2+\frac{1}{2{\tau }_{\mathrm{in}}}{\gamma }_n^2.$

τ_in is a CUB coefficient. It can be easily proved that when τ_in=γ_n/p_i, f(p_iγ_n)≤g(p_i,γ_n,τ_in) can be taken as equal. Similarly, let f(p_j,γ_n)=p_j,γ_n. The CUB of f(p_j,γ_n) is

$g\left(p_j,{\gamma }_n,{\tau }_{\mathrm{jn}}\right)=\frac{{\tau }_{\mathrm{jn}}}{2}{p}_j^2+\frac{1}{2{\tau }_{\mathrm{jn}}}{\gamma }_n^2.$

After the CUB is replaced by using p_i,γ_n and p_j,γ_n, the non-convex constraint

$\frac{h_n^2{p}_n}{{\psi }_n+{\sigma }^2}\ge {\gamma }_n,\forall n\in N$

is transformed into the following convex constraint:

$h_n^2{p}_n\ge \varpi {\sum }_{i=1}^{n-1}{h}_i^2\left(\frac{{\tau }_{\mathrm{in}}}{2}{p}_i^2+\frac{1}{2{\tau }_{\mathrm{in}}}{\gamma }_n^2\right)+{\sum }_{j=n+1}^{N+K}{h}_j^2\left(\frac{{\tau }_{\mathrm{jn}}}{2}{p}_j^2+\frac{1}{2{\tau }_{\mathrm{jn}}}{\gamma }_n^2\right)+{\sigma }^2{\gamma }_n$

In the foregoing convex constraint, the CUB coefficient τ_in is updated to =/, and are solutions of the optimization problem in a ^th iteration. Similarly, the CUB coefficient τ_jn is updated to =/.

To solve the non-convex constraint a√{square root over (p_i)}≥b_i,∀i∈N∪K , when p₁≠0, it is rewritten as follows:

$a\ge \frac{b_i}{\sqrt{p_i}},\forall ia^2\ge \frac{b_i^2}{p_i},\forall i$

Non-convexity of the foregoing expression originates from the term a² on the left side of the inequality. The present disclosure uses a first-order Taylor expansion of a² at a point for replacement such that the non-convex constraint a√{square root over (p_i)}≥b_i,∀i∈N∪K is transformed into the following convex constraint:

${\left(a^{\left(\ell \right)}\right)}^2+2a^{\left(\ell \right)}\left(a-a^{\left(\ell \right)}\right)\ge b_i^2/p_i,\forall i$

Based on the foregoing approximation, the non-convex optimization problem in S7 can be transformed into the following convex optimization problem:

$\begin{array}{cc}\underset{p,a,\gamma ,b}{\max }& {\sum }_{n=1}^N{\log }_2\left(1+{\gamma }_n\right)-\lambda \mathrm{MSE}\\ s.t.& h_1^2{p}_1\ge h_2^2{p}_2\ge \dots h_N^2{p}_N\ge h_k^2{p}_k,\forall k\in K\\ & {\log }_2\left(1+{\gamma }_n\right)\ge R_{\min },\forall n\in N\\ & \mathrm{MSE}\le {?}_0,\\ & p_n,p_k\in \left[0,P_{\max }\right],\forall n,k\\ & h_n^2{p}_n\ge \varpi {\sum }_{i=1}^{n-1}{h}_i^2\left(\frac{{\tau }_{\mathrm{in}}}{2}{p}_i^2+\frac{1}{2{\tau }_{\mathrm{in}}}{\gamma }_n^2\right)+{\sum }_{j=n+1}^{N+K}{h}_j^2\left(\frac{{\tau }_{\mathrm{jn}}}{2}{p}_j^2+\frac{1}{2{\tau }_{\mathrm{jn}}}{\gamma }_n^2\right)+{\sigma }^2{\gamma }_n,\\ & {\left(a^{\left(\ell \right)}\right)}^2+2a^{\left(\ell \right)}\left(a-a^{\left(\ell \right)}\right)\ge b_i^2/p_i,\forall i\end{array}$

The foregoing convex optimization problem can be solved through a mathematical toolkit CVX to obtain an optimal solution.

As shown in FIG. 2, solving the non-convex optimization problem in S7 includes the following steps:

• • S8.1: p⁽⁰⁾, a⁽⁰⁾, γ⁽⁰⁾, b⁽⁰⁾, ω, a maximum quantity L of iterations, and a threshold ε are initialized, and an iteration index =0 is set.
  • S8.2: An objective function value =−AMS is computed.
  • S8.3: Given , , , and , and are updated by using =/ and =/.
  • S8.4: Given and , the convex optimization problem is solved through the mathematical toolkit CVX to obtain , , , and .
  • S8.5: An objective function value =− is computed.
  • S8.6: =+1 is updated and Δ_U=|−| is computed.
  • S8.7: S8.3 to S8.6 are repeated until Δ_U≤ε or ≥L and user transmit power and a receiving factor are output.

Herein, the user transmit power has an upper bound and the MSE is non-negative. Therefore, the objective function of the optimization problem has an upper bound, and convergence of the foregoing iterative algorithm can be ensured.

• • S9: The same frequency band and the transmit power solved in S8 are used by the users to upload the local models or send the training data simultaneously. User data is detected by the base station through imperfect SIC and decoding is performed by using the reception strategy solved in SS to obtain the aggregation model.

S10: CL is performed by the base station on the accumulated data to obtain the CL model, and weighted summation is performed on the CL model and the received aggregation model based on the data amount to obtain the global model.

• • S11: The global model is broadcast by the base station to all users, and S3 to S10 are repeated until a total cycle quantity requirement is met.

In summary, the wireless FL framework provided in the present disclosure can allow the users with limited computing resources and the users with sufficient computing resources to participate in FL together, and make full use of data of heterogeneous users to improve performance of a machine learning model. In the present disclosure, all users use the same frequency band to upload the models or send the data simultaneously such that spectrum resources can be effectively saved and communication delay can be reduced. The base station detects the signals through imperfect SIC. This is more in line with an actual situation. In addition, in the present disclosure, the non-convex optimization problem is constructed based on the user data transmission rates and the MSE of the aggregation model to jointly design the user transmit power and the reception strategy of the base station. The problem is solved through the SCA method such that the sum rate can be effectively maximized and the MSE can be effectively minimized.

The foregoing embodiments are used only to describe the technical solutions of the present disclosure, and are not intended to limit same. Although the present disclosure is described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions to some technical features therein. These modifications or substitutions do not make the essence of the corresponding technical solutions depart from the spirit and scope of the technical solutions of the embodiments of the present disclosure.

Claims

1. A wireless federated learning (FL) framework, comprising: N centralized learning (CL) users with limited computing resources, wherein the CL users send training data to a base station for CL to participate in FL;K FL users with sufficient computing resources, wherein the FL users obtain local models through local training data, and upload local model parameters as an aggregation model to the base station; andthe base station serving as an FL server and configured to compute a global model, wherein the base station performs CL on the training data accumulated by the CL users to obtain a CL model, and performs weighted summation on the CL model and the aggregation model based on a data amount to obtain the global model.

2. The wireless FL framework according to claim 1, wherein it is assumed that there are T FL cycles in total, which are represented as a set T={1, 2, . . . , T}; and in tth FL cycle, a local model update formula of a kth FL user is as follows:

3. A resource optimization method for the wireless FL framework according to claim 1, comprising the following steps: S1: initializing, by the base station, a training task and the global model, and sending the global model to all users:S2: after receiving the global model, computing, by the FL user, a local loss function and a gradient based on the local training data, and updating the local model;S3: using, by the FL users and the CL users, a same frequency band to respectively upload the local models and send the training data simultaneously; and detecting, by the base station, a received signal through imperfect successive interference cancellation (SIC), and separating the training data and the aggregation model;S4: obtaining data transmission rates of all CL users and a mean square error (MSE) of the aggregation model of all FL users, and detecting signals of all CL users in an order of 1, 2, . . . , N through imperfect SIC;S5: constructing a non-convex optimization problem based on the data transmission rates and the MSE of the aggregation model; andS6: transforming and solving the non-convex optimization problem in S5 through a successive convex approximation (SCA) method, and outputting user transmit power and a reception strategy.

4. The resource optimization method for the wireless FL framework according to claim 3, wherein the local loss function in S2 is an MSE loss function or a cross-entropy loss function.

5. The resource optimization method for the wireless FL framework according to claim 3, wherein S3 comprises: before the user uploads the local model or sends the training data, normalizing a local training data set {Dn} of the CL users into a communication symbol set {sn} and the local models {wk} of the FL users into a computation symbol set {sk}, where a superimposed signal received by the base station is as follows:

6. The resource optimization method for the wireless FL framework according to claim 5, wherein in S4, a formula of the data transmission rate of the nth CL user is as follows:

7. The resource optimization method for the wireless FL framework according to claim 6, wherein in S4, a formula of the MSE of the aggregation model is as follows:

8. The resource optimization method for the wireless FL framework according to claim 6, wherein in S5, the non-convex optimization problem is constructed as follows:

9. The resource optimization method for the wireless FL framework according to claim 3, wherein a convex optimization problem obtained through transformation in S6 is as follows:

10. The resource optimization method for the wireless FL framework according to claim 9, wherein solving the convex optimization problem in S6 comprises: S6.1: initializing p(0), a(0), γ(0), b(0), ω, a maximum quantity L of iterations, and a threshold ε, and setting an iteration index =0;S6.2: computing an objective function value =−;S6.3: given , , , and , updating and by using =/ and =/;S6.4: given and , solving the convex optimization problem through a mathematical toolkit CVX to obtain , , , and ;S6.5: computing an objective function value =−AMS;S6.6: updating =+1 and computing ΔU=|−; andS6.7: repeating S6.3 to S6.6 until ΔU≤ε or ≥L and outputting user transmit power and a receiving factor .

11. A resource optimization method for the wireless FL framework according to claim 2, comprising the following steps: S1: initializing, by the base station, a training task and the global model, and sending the global model to all users;S2: after receiving the global model, computing, by the FL user, a local loss function and a gradient based on the local training data, and updating the local model;S3: using, by the FL users and the CL users, a same frequency band to respectively upload the local models and send the training data simultaneously; and detecting, by the base station, a received signal through imperfect successive interference cancellation (SIC), and separating the training data and the aggregation model;S4: obtaining data transmission rates of all CL users and a mean square error (MSE) of the aggregation model of all FL users, and detecting signals of all CL users in an order of 1, 2, . . . . N through imperfect SIC:S5: constructing a non-convex optimization problem based on the data transmission rates and the MSE of the aggregation model; andS6: transforming and solving the non-convex optimization problem in S5 through a successive convex approximation (SCA) method, and outputting user transmit power and a reception strategy.

12. The resource optimization method for the wireless FL framework according to claim 11, wherein the local loss function in S2 is an MSE loss function or a cross-entropy loss function.

13. The resource optimization method for the wireless FL framework according to claim 11, wherein S3 comprises: before the user uploads the local model or sends the training data, normalizing a local training data set {Dn} of the CL users into a communication symbol set {sn} and the local models {wk} of the FL users into a computation symbol set {sk}, where a superimposed signal received by the base station is as follows:

14. The resource optimization method for the wireless FL framework according to claim 13, wherein in S4, a formula of the data transmission rate of the nth CL user is as follows:

15. The resource optimization method for the wireless FL framework according to claim 14, wherein in S4, a formula of the MSE of the aggregation model is as follows:

16. The resource optimization method for the wireless FL framework according to claim 14, wherein in S5, the non-convex optimization problem is constructed as follows:

17. The resource optimization method for the wireless FL framework according to claim 11, wherein a convex optimization problem obtained through transformation in S6 is as follows:

18. The resource optimization method for the wireless FL framework according to claim 17, wherein solving the convex optimization problem in S6 comprises: S6.1: initializing p(0), a(0), γ(0), b(0), ω, a maximum quantity L of iterations, and a threshold ε, and setting an iteration index =0;S6.2: computing an objective function value =−;S6.3: given , , , and , updating and by using =/ and =/;S6.4: given and , solving the convex optimization problem through a mathematical toolkit CVX to obtain , , , and ;S6.5: computing an objective function value =−AMS;S6.6: updating =+1 and computing ΔU=|−; andS6.7: repeating S6.3 to S6.6 until ΔU≤ε or ≥L and outputting user transmit power and a receiving factor .