Patent Application
20250063406
This application pertains to the field of communication technologies, and specifically, relates to a model obtaining method and a communication device.
A Network Data Analytics Function (NWDAF) automatically perceives and analyzes a network based on network data, and participates in a full life cycle of network planning, construction, operation and maintenance, network optimization, and operations. This makes the network easy to maintain and control, improves utilization of network resources, and improves service experience of a user. The NWDAF may include two parts: an Analytics Logical Function (AnLF) and a Model Training Logical Function (MTLF). The AnLF is responsible for an inference function. The MTLF is responsible for a training function.
When receiving a task initiated by a consumer, the AnLF needs to obtain a model, to use the model to perform inference and derive an analytics result.
Embodiments of this application provide a model obtaining method and a communication device.
According to a first aspect, a model obtaining method is provided, including:
According to a second aspect, a model obtaining method is provided, including: receiving, by a second communication device, a first request sent by a first communication device, where the first request is used to request to obtain information of at least one third communication device, and each third communication device is able to provide model information of at least one model required by the first communication device;
According to a third aspect, a model obtaining method is provided, including:
According to a fourth aspect, a model obtaining method is provided, including:
According to a fifth aspect, a model obtaining apparatus is provided, including:
According to a sixth aspect, a model obtaining apparatus is provided, including:
According to a seventh aspect, a model obtaining apparatus is provided, including:
According to an eighth aspect, a model obtaining apparatus is provided, including:
According to a ninth aspect, a first communication device is provided, where the first communication device includes a processor and a memory, the memory stores a program or instructions capable of running on the processor, and when the program or instructions are executed by the processor, the steps of the method according to the first aspect are implemented.
According to a tenth aspect, a first communication device is provided, including a processor and a communication interface. The communication interface is configured to: send, for the first communication device, a first request to a second communication device, where the first request is used to request to obtain information of at least one third communication device, and each third communication device is able to provide model information of at least one model required by the first communication device; receive information of at least one third communication device that is sent by the second communication device; and obtain model information of a plurality of models based on the information of the at least one third communication device.
According to an eleventh aspect, a second communication device is provided, where the second communication device includes a processor and a memory, the memory stores a program or instructions capable of running on the processor, and when the program or instructions are executed by the processor, the steps of the method according to the second aspect are implemented.
According to a twelfth aspect, a second communication device is provided, including a processor and a communication interface. The communication interface is configured to: receive a first request sent by a first communication device, where the first request is used to request to obtain information of at least one third communication device, and each third communication device is able to provide model information of at least one model required by the first communication device; and send information of the at least one third communication device to the first communication device, where the information of the at least one third communication device is used for the first communication device to obtain model information of a plurality of models. The processor is configured to determine at least one third communication device according to the first request.
According to a thirteenth aspect, a third communication device is provided, where the third communication device includes a processor and a memory, the memory stores a program or instructions capable of running on the processor, and when the program or instructions are executed by the processor, the steps of the method according to the first aspect are implemented.
According to a fourteenth aspect, a third communication device is provided, including a processor and a communication interface. The communication interface is configured to: receive a second request sent by a first communication device, where the second request is used to request to obtain at least one model; and send model information of at least one model to the first communication device according to the second request, where the second request includes at least one of the following: an analytics task identifier, where the analytics task identifier is used to identify a data analytics task to which a required model is applicable; an identifier of a required model; the number of required models; or model attribute information that a model needs to meet.
According to a fifteenth aspect, a fourth communication device is provided, where the fourth communication device includes a processor and a memory, the memory stores a program or instructions capable of running on the processor, and when the program or instructions are executed by the processor, the steps of the method according to the first aspect are implemented.
According to a sixteenth aspect, a fourth communication device is provided, including a processor and a communication interface. The communication interface is configured to: send a task request to a first communication device; and receive data analytics result information sent by the first communication device, where the data analytics result information is obtained by the first communication device by performing analytics based on a plurality of models, the plurality of models are obtained by the first communication device based on information of at least one third communication device, and each third communication device is able to provide model information of at least one model required by the first communication device.
According to a seventeenth aspect, a communication system is provided, including a first communication device, a second communication device, a third communication device, and a fourth communication device, where the first communication device may be configured to perform the steps of the model obtaining method according to the first aspect, the second communication device may be configured to perform the steps of the model obtaining method according to the second aspect, the third communication device may be configured to perform the steps of the model obtaining method according to the third aspect, and the fourth communication device may be configured to perform the steps of the model obtaining method according to the fourth aspect.
According to an eighteenth aspect, a readable storage medium is provided, where the readable storage medium stores a program or instructions, and when the program or instructions are executed by a processor, the steps of the method according to the first aspect are implemented, or the steps of the method according to the second aspect are implemented, or the steps of the method according to the third aspect are implemented, or the steps of the method according to the fourth aspect are implemented.
According to a nineteenth aspect, a chip is provided, where the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is configured to run a program or instructions to implement the method according to the first aspect, or implement the method according to the second aspect, or implement the method according to the third aspect, or implement the method according to the fourth aspect.
According to a twentieth aspect, a computer program or program product is provided, where the computer program or program product is stored in a storage medium, and the computer program or program product is executed by at least one processor to implement the steps of the model obtaining method according to any one of the first aspect to the fourth aspect.
In the embodiments of this application, a first communication device sends a first request to a second communication device, where the first request is used to request to obtain information of at least one third communication device, and each third communication device is able to provide model information of at least one model required by the first communication device. Further, the first communication device may obtain model information of a plurality of models based on information of at least one third communication device that is sent by the second communication device.
The following clearly describes the technical solutions in the embodiments of this application with reference to the accompanying drawings in the embodiments of this application. Clearly, the described embodiments are some but not all of the embodiments of this application. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments of this application shall fall within the protection scope of this application.
The terms “first”, “second”, and the like in this specification and the claims of this application are used to distinguish between similar objects rather than to describe a specific order or sequence. It should be understood that terms used in this way are interchangeable in appropriate circumstances so that the embodiments of this application can be implemented in other orders than the order illustrated or described herein. In addition, “first” and “second” are usually used to distinguish objects of a same type, and do not limit the number of objects. For example, there may be one or more first objects. In addition, in this specification and the claims, “and/or” represents at least one of connected objects, and the character “/” typically represents an “or” relationship between the associated objects.
It should be noted that technologies described in the embodiments of this application are not limited to a Long Term Evolution (LTE)/LTE-Advanced (LTE-A) system, and may also be applied to other wireless communication systems, such as Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Orthogonal Frequency Division Multiple Access (OFDMA), Single-carrier Frequency Division Multiple Access (SC-FDMA), and other systems. The terms “system” and “network” in the embodiments of this application are often used interchangeably, and the technology described herein may be used in the aforementioned systems and radio technologies as well as other systems and radio technologies. In the following descriptions, a New Radio (NR) system is described for an illustration purpose, and NR terms are used in most of the following descriptions, but these technologies may also be applied to applications other than an NR system application, for example, a 6th generation (6G) communication system.
Currently, a Network Data Analytics Function (NWDAF) is introduced in the 3rd generation partnership project (3GPP) to perform some intelligent analytics. The NWDAF has a specific function of artificial intelligence analytics. The NWDAF performs some operation optimization or statistical analytics by collecting some data, obtaining a result through analytics by using a built-in algorithm and an analytics capability, and providing the result for a core network device. The network data analytics function automatically perceives and analyzes a network based on network data, and participates in a full life cycle of network planning, construction, operation and maintenance, network optimization, and operations. This makes the network easy to maintain and control, improves utilization of network resources, and improves service experience of a user.
For example, the NWDAF can provide an observed service experience related network data analytics function. This function network element may initiate a request to another network element to request to collect Quality of Service (QOS) information, such as uplink and downlink rates and a packet loss rate, of a terminal accessing an IP address of a server, and output statistical information, where the information includes user experience of the terminal accessing the server. The NWDAF may also perform prediction based on historical experience of the terminal accessing the server, for example, predict possible user experience of the terminal in a case that the terminal accesses the server in a specific area within a specific time period in the future.
Usually, analytics or prediction content that the NWDAF can provide is distinguished by using an analytic identifier (Analytic ID). Different analytic IDs and some parameters may be indicated to the NWDAF, so that the NWDAF can provide analytics and prediction results corresponding to the analytic identifiers. For example, for observed service experience analytics provided by the NWDAF, an analytic ID is Service Experience. Therefore, the NWDAF can provide corresponding analytics provided that a network element served by the NWDAF is obtained and an analytic ID is entered in a request for analytics.
The NWDAF may include two parts: an analytics logical function AnLF and a model training logical function MTLF. The AnLF is responsible for an inference function. The MTLF is responsible for a training function. Usually, the AnLF performs inference and derives an analytics result by using a model.
The following describes in detail a model obtaining method provided in the embodiments of this application with reference to the accompanying drawings and by using some embodiments and application scenarios thereof.
Step 101: A first communication device sends a first request to a second communication device, where the first request is used to request to obtain information of at least one third communication device, and each third communication device is able to provide model information of at least one model required by the first communication device.
Specifically, the first communication device and the second communication device may be network-side devices (for example, core network devices). The first communication device may be, for example, an AnLF of an NWDAF. The second communication device may be a network repository function (NRF). The third communication device may be an MTLF of the NWDAF or an Analytics Data Repository Function (ADRF).
For example, the first communication device sends the first request to the second communication device when a task request sent by another communication device is received or another trigger condition is met. The first request is used to request to obtain information of at least one third communication device. Each third communication device is able to provide model information of at least one model required by the first communication device. The model information may be the model itself (for example, a file including the model), a download address of the model, or the like.
Step 102: The first communication device receives information of at least one third communication device that is sent by the second communication device.
Specifically, the second communication device determines the at least one third communication device and sends the information of the at least one third communication device to the first communication device according to the first request, and the first communication device receives the information of the at least one third communication device that is sent by the second communication device. For example, information of a third communication device includes an analytics task identifier supported by the third communication device, an identifier or an address of the third communication device, the number of models available for an analytics task corresponding to the supported analytics task identifier, model attribute information of a supported model, or the like.
Step 103: The first communication device obtains model information of a plurality of models based on the information of the at least one third communication device, where the plurality of models are used to derive data analytics result information.
Specifically, each third communication device may provide model information of at least one model, and the model information of the plurality of models may be obtained from the at least one third communication device. Further, the first communication device may obtain the plurality of models, and perform analytics and other operations based on the plurality of models, so that a data analytics result can be more accurate.
According to the method in this embodiment, a first communication device sends a first request to a second communication device, where the first request is used to request to obtain information of at least one third communication device, and each third communication device is able to provide model information of at least one model required by the first communication device. Further, the first communication device may obtain model information of a plurality of models based on information of at least one third communication device that is sent by the second communication device. In this way, a model obtaining method is implemented, with low implementation complexity and high efficiency. In addition, the plurality of models are used to derive data analytics result information, so that a data analytics result can be more accurate.
Optionally, the first request includes: an analytics task identifier, where the analytics task identifier is used to identify a data analytics task to which a required model is applicable. The first request further includes at least one of the following:
The analytics task identifier may indicate a data analytics task to which a model required by the first communication device is applicable.
For example, information of a plurality of third communication devices is sorted in descending order of model quantities. For example, three MTLFs are included: an MTLF 1, an MTLF 2, and an MTLF 3, where the MTLF 3 meets a requirement of the AnLF, and the MTLF 1, the MTLF 2, and the MTLF 3 have two models, one model, and three models respectively. In this case, MTLF information finally returned by the NRF to the AnLF is sorted in an order of the MTLF 3, the MTLF 1, and the MTLF 2.
The number of required models may be a numerical value or a number range. For example, the minimum number and the maximum number are defined. The number of required third communication devices is similar to the number of required models.
Optionally, information of each third communication device includes at least one of the following:
The number of models corresponding to the analytics task identifier supported by the third communication device is the number of models able to process an analytics task corresponding to the analytics task identifier.
Optionally, the model attribute information includes at least one of the following: scope information of model use, a model identifier, result evaluation information of model training, result evaluation information of model use, a model size, model inference duration, or training data information.
Optionally, the scope information of model use includes at least one of the following: a target object of a model, a target time range, a target location range, single network slice selection assistance information S-NSSAI, or a data network name DNN. The target time range is used to indicate a time range within which the model is to be used. The target location range is used to indicate a location range within which the model is to be used.
Optionally, the result evaluation information of model training includes at least one of the following: accuracy, error information, model training duration, or a data amount of model training; and the result evaluation information of model use includes at least one of the following: accuracy, error information, or model running duration.
Specifically, the target object of the model indicates, for example, to perform mobility analytics for a specific terminal; the target time range indicates that the model can be used within the target time range; and the target location range indicates that the model can be used within the target location range, for example, the target object is within the target location range.
The result evaluation information of model training is used to describe result evaluation information of a trained model, for example, accuracy of recognition or decision-making, error information, duration, or a data amount.
For example, the error information may be a Mean Absolute Error (MAE) or a Root Mean Square Error (RMSE).
The result evaluation information of model use is similar to the result evaluation information of model training.
The model size is a size of storage space required for storing the model or running the model.
The model inference duration is used to represent duration within which a model obtains a convergent data inference result during running.
The training data information may include, for example, at least one of the following: source information of training data, for example, source location information of data, or information about a source communication device; or time information of training data, for example, the training data is data within a period from three months ago to two months ago.
The model attribute information that a model needs to meet is, for example, as follows: A target object of the model is a terminal in a specific area, a model size is within 500 MB, accuracy of model use is greater than a specific threshold, a source communication device of training data used by the model is a communication device a, and training data is data within the last week or month.
Optionally, the method further includes:
For example, the model information of the plurality of models is addresses of the models. In this case, the plurality of models may be obtained based on the addresses of the models, and analytics may be performed based on the obtained plurality of models to obtain the data analytics result information.
Optionally, the first communication device performs a model inference operation based on each of the plurality of models to obtain a plurality of data inference results; and the first communication device processes the plurality of data inference results to obtain the data analytics result information.
Final data analytics result information is derived through the foregoing processing, such as aggregation and voting.
For example, the first communication device may process the plurality of data inference results to obtain the data analytics result information in at least one of the following manners:
Specifically, the weighted averaging, the averaging, and the like are aggregation operations, and the accumulation and the accumulation with performance taken into account are specific implementations of cumulative voting. Performance of a model may be taken into account during accumulation. For example, if accuracy of the model is 0.6, a weight in an inference result may be 0.6.
In the foregoing implementation, model inference, analytics, and other processing are performed based on the plurality of models, and an obtained result has good performance and high accuracy, and is not likely to be affected by a change in a network condition or network data.
Optionally, before step 101, the method further includes:
Optionally, after the first communication device performs analytics based on the plurality of models to obtain the data analytics result information, the method further includes:
Specifically, the fourth communication device may be a consumer device, for example, an Application Function (AF). The fourth communication device may send a task request to the first communication device to request to obtain data analytics result information corresponding to some data analytics tasks.
Optionally, content included in the task request may be the same as content included in the first request.
Optionally, step 103 may be implemented in the following manner:
For example, analytics task identifiers supported by a third communication device are an identifier 1 and an identifier 2, the number of models corresponding to the identifier 1 is 5, the number of models corresponding to the identifier 2 is 10, and for three of the five models corresponding to the identifier 1, use scopes are consistent with a use scope in the model attribute information that needs to be met, result evaluation information of model training and use matches result evaluation information in the required model attribute information, and model sizes also match a model size in the model attribute information. Therefore, the third communication device may serve as a target communication device.
Optionally, the model information of the plurality of models may be obtained from the at least one target communication device specifically in the following manner:
Specifically, the first communication device receives the model information of the plurality of models that is sent by the at least one target communication device, and each target communication device may send model information of one or more models.
Optionally, the number of the at least one target communication device may be 1, and that the first communication device sends a second request to the at least one target communication device includes:
Before the first communication device sends the second request to the one of the at least one target communication device, the method further includes:
Optionally, the number of the at least one target communication device may be greater than 1, and that the first communication device sends a second request to the at least one target communication device includes:
Before the first communication device sends the second request to the plurality of target communication devices among the at least one target communication device, the method further includes:
Specifically, in this case, each target communication device may send model information of one or more models. None of the target communication devices is able to provide the model information of all of the plurality of models required by the first communication device. Therefore, the model information needs to be obtained from the plurality of target communication devices.
In the foregoing implementation, the first communication device sends the second request to one or more of the at least one target communication device to obtain the model information of the plurality of models, with low implementation complexity and high efficiency.
For example, refer to
For example, the capability registration message may be carried in Nnrf_NFManagement_NFRegister Register signaling.
For example, the network function type is as follows: NF type=NWDAF type or MTLF type.
The network function instance identifier (NF instance ID) is identification information of a communication device in current registration, for example, Fully Qualified Domain Name (FQDN) information of the communication device for indicating a location of the communication device and connecting to the communication device, or IP address information (another type of identification information).
The supported analytics task identifier (analytic ID) indicates a type of a task that the NWDAF can perform.
Steps 2 and 3: The NRF stores information of the MTLF and sends a response message. For example, the response message may be carried in Nnrf_NFManagement_NFRegister response signaling. The response message is used to notify a third communication device that registration is successful.
Step 4: A consumer device sends a task request to an AnLF (or an NWDAF, or an AnLF including an MTLF). For example, the task request is carried in Nnwdaf_AnalyticsInfo_Request signaling.
Step 5: Send a first request to the NRF. For example, the first request may be carried in Nnrf_NFDiscovery_Request signaling. The first request includes a task analytics identifier, and may further include at least one of the following:
Step 6: The NRF determines, according to the first request from the AnLF, MTLF information to be fed back.
Specifically, after receiving the first request, the NRF searches for all registered MTLFs, selects a final MTLF by matching information, generates an MTLF list, and feeds back MTLF information based on the MTLF list. The first request from the AnLF includes the number of required models and information indicating that information of a plurality of MTLFs is required. In this case, the NRF searches for all MTLFs that support the analytics task identifier included in the first request, and feeds back information about an MTLF that has a plurality of models corresponding to the analytics task identifier. If there are not a plurality of MTLFs meeting the condition, information about other MTLFs that do not meet the model quantity requirement is fed back. In this case, the number of models of a plurality of MTLFs needs to meet the requirement of the number of required models in the first request.
If the first request includes only the number of required MTLFs, the NRF selects information about a plurality of MTLFs that have a model corresponding to the analytics task identifier for feedback.
If the first request includes only the number of required models, the NRF searches, through matching, MTLF information for information about an MTLF that has a plurality of models corresponding to the analytics task identifier for feedback. If no information is found, failure information is returned.
Step 7: The NRF returns information about an MTLF meeting the condition to the AnLF.
Step 8: The AnLF determines a target MTLF from which a model is to be obtained.
Steps 9a and step 9b: The AnLF requests a model that meets a task requirement from the target MTLF. For example, the request includes a task analytics identifier and model attribute information that the model needs to meet.
Step 10: The AnLF performs analytics by using a plurality of models.
Step 11: The AnLF returns data analytics result information to the consumer device. For example, the data analytics result information is carried in Nnwdaf_AnalyticsInfo_Request response signaling.
Optionally, the consumer device may further send the data analytics result information to user equipment, such as a terminal.
The model obtaining method provided in the embodiments of this application may be performed by a model obtaining apparatus. In the embodiments of this application, a model obtaining apparatus provided in the embodiments of this application is described by using an example in which the model obtaining apparatus performs the model obtaining method.
Step 201: A second communication device receives a first request sent by a first communication device, where the first request is used to request to obtain information of at least one third communication device, and each third communication device is able to provide model information of at least one model required by the first communication device.
Step 202: The second communication device determines at least one third communication device according to the first request.
Step 203: The second communication device sends information of the at least one third communication device to the first communication device, where the information of the at least one third communication device is used for the first communication device to obtain model information of a plurality of models, and the plurality of models are used to derive data analytics result information.
Optionally, the first request includes: an analytics task identifier, where the analytics task identifier is used to identify a data analytics task to which a required model is applicable; and the first request further includes at least one of the following:
Optionally, the method further includes:
Optionally, that the second communication device determines the number of the at least one third communication device according to the first request sent by the first communication device includes at least one of the following:
Optionally, the model attribute information includes at least one of the following: scope information of model use, a model identifier, result evaluation information of model training, result evaluation information of model use, a model size, model execution duration, source information of training data, or time information of the training data, where the model size is used to represent a size of storage space required for storing or running a model, and the model inference duration is used to represent duration within which a model obtains a data inference result during running.
Optionally, the result evaluation information of model training includes at least one of the following: accuracy, error information, model training duration, or a data amount of model training; and the result evaluation information of model use includes at least one of the following: accuracy, error information, or model running duration.
Optionally, information of each third communication device includes at least one of the following:
Optionally, the method further includes:
Specific implementation processes and technical effect of the method in this embodiment are the same as those in the method embodiment on the first communication device side. For details, refer to the detailed descriptions in the method embodiment on the first communication device side. Details are not described herein again.
Step 301: A third communication device receives a second request sent by a first communication device, where the second request is used to request to obtain at least one model.
Step 302: The third communication device sends model information of at least one model to the first communication device according to the second request.
The second request includes at least one of the following:
Optionally, the method further includes:
Optionally, the model attribute information includes at least one of the following: scope information of model use, a model identifier, result evaluation information of model training, result evaluation information of model use, a model size, model execution duration, source information of training data, or time information of the training data, where the model size is used to represent a size of storage space required for storing or running a model, and the model inference duration is used to represent duration within which a model obtains a data inference result during running.
Optionally, the result evaluation information of model training includes at least one of the following: accuracy, error information, model training duration, or a data amount of model training; and the result evaluation information of model use includes at least one of the following: accuracy, error information, or model running duration.
Optionally, information of each third communication device includes at least one of the following:
Specific implementation processes and technical effect of the method in this embodiment are the same as those in the method embodiment on the first communication device side. For details, refer to the detailed descriptions in the method embodiment on the first communication device side. Details are not described herein again.
Step 401: A fourth communication device sends a task request to a first communication device.
Step 402: The fourth communication device receives data analytics result information sent by the first communication device, where the data analytics result information is obtained by the first communication device by performing analytics based on a plurality of models, the plurality of models are obtained by the first communication device based on information of at least one third communication device, and each third communication device is able to provide model information of at least one model required by the first communication device.
Optionally, the model attribute information includes at least one of the following: scope information of model use, a model identifier, result evaluation information of model training, result evaluation information of model use, a model size, model execution duration, source information of training data, or time information of the training data, where the model size is used to represent a size of storage space required for storing or running a model, and the model inference duration is used to represent duration within which a model obtains a data inference result during running.
Optionally, the result evaluation information of model training includes at least one of the following: accuracy, error information, model training duration, or a data amount of model training; and the result evaluation information of model use includes at least one of the following: accuracy, error information, or model running duration.
Optionally, information of each third communication device includes at least one of the following:
Specific implementation processes and technical effect of the method in this embodiment are the same as those in the method embodiment on the first communication device side. For details, refer to the detailed descriptions in the method embodiment on the first communication device side. Details are not described herein again.
Optionally, the obtaining module 130 is further configured to:
Optionally, the apparatus further includes: a processing module, configured to perform analytics based on the plurality of models to obtain the data analytics result information.
Optionally, the processing module is specifically configured to:
Optionally, the obtaining module 130 is specifically configured to perform at least one of the following operations:
Optionally, the receiving module 120 is further configured to:
Optionally, the obtaining module 130 is specifically configured to:
Optionally, the first request includes:
Optionally, information of each third communication device includes at least one of the following:
Optionally, the sending module 110 is further configured to:
Optionally, the sending module 110 is specifically configured to:
Optionally, the sending module 110 is specifically configured to:
Optionally, the model attribute information includes at least one of the following: scope information of model use, a model identifier, result evaluation information of model training, result evaluation information of model use, a model size, model inference duration, source information of training data, or time information of the training data, where the model size is used to represent a size of storage space required for storing or running a model, and the model inference duration is used to represent duration within which a model obtains a data inference result during running.
Optionally, the result evaluation information of model training includes at least one of the following: accuracy, error information, model training duration, or a data amount of model training; and the result evaluation information of model use includes at least one of the following: accuracy, error information, or model running duration.
The apparatus in this embodiment may be configured to perform the method in any one of the method embodiments on the first communication device side. Specific implementation processes and technical effect of the apparatus are the same as those in the method embodiment on the first communication device side. For details, refer to the detailed descriptions in the method embodiment on the first communication device side. Details are not described herein again.
Optionally, the first request includes: an analytics task identifier, where the analytics task identifier is used to identify a data analytics task to which a required model is applicable; and the first request further includes at least one of the following:
Optionally, the processing module 220 is further configured to:
Optionally, that the second communication device determines the number of the at least one third communication device according to the first request sent by the first communication device includes at least one of the following:
Optionally, the model attribute information includes at least one of the following: scope information of model use, a model identifier, result evaluation information of model training, result evaluation information of model use, a model size, model execution duration, source information of training data, or time information of the training data, where the model size is used to represent a size of storage space required for storing or running a model, and the model inference duration is used to represent duration within which a model obtains a data inference result during running.
Optionally, the result evaluation information of model training includes at least one of the following: accuracy, error information, model training duration, or a data amount of model training; and the result evaluation information of model use includes at least one of the following: accuracy, error information, or model running duration.
Optionally, information of each third communication device includes at least one of the following:
Optionally, the receiving module 210 is further configured to:
The apparatus in this embodiment may be configured to perform the method in any one of the method embodiments on the second communication device side. Specific implementation processes and technical effect of the apparatus are the same as those in the method embodiment on the second communication device side. For details, refer to the detailed descriptions in the method embodiment on the second communication device side. Details are not described herein again.
Optionally, the model attribute information includes at least one of the following: scope information of model use, a model identifier, result evaluation information of model training, result evaluation information of model use, a model size, model execution duration, source information of training data, or time information of the training data, where the model size is used to represent a size of storage space required for storing or running a model, and the model inference duration is used to represent duration within which a model obtains a data inference result during running.
Optionally, the result evaluation information of model training includes at least one of the following: accuracy, error information, model training duration, or a data amount of model training; and the result evaluation information of model use includes at least one of the following: accuracy, error information, or model running duration.
Optionally, information of each third communication device includes at least one of the following:
Optionally, the sending module 320 is further configured to:
The apparatus in this embodiment may be configured to perform the method in any one of the method embodiments on the third communication device side. Specific implementation processes and technical effect of the apparatus are the same as those in the method embodiment on the third communication device side. For details, refer to the detailed descriptions in the method embodiment on the third communication device side. Details are not described herein again.
The task request includes: an analytics task identifier, where the analytics task identifier is used to identify a data analytics task to which a required model is applicable; and the task request further includes at least one of the following:
Optionally, the model attribute information includes at least one of the following: scope information of model use, a model identifier, result evaluation information of model training, result evaluation information of model use, a model size, model execution duration, source information of training data, or time information of the training data, where the model size is used to represent a size of storage space required for storing or running a model, and the model inference duration is used to represent duration within which a model obtains a data inference result during running.
Optionally, the result evaluation information of model training includes at least one of the following: accuracy, error information, model training duration, or a data amount of model training; and the result evaluation information of model use includes at least one of the following: accuracy, error information, or model running duration.
Optionally, information of each third communication device includes at least one of the following:
The apparatus in this embodiment may be configured to perform the method in any one of the method embodiments on the fourth communication device side. Specific implementation processes and technical effect of the apparatus are the same as those in the method embodiment on the fourth communication device side. For details, refer to the detailed descriptions in the method embodiment on the fourth communication device side. Details are not described herein again.
The model obtaining apparatus in the embodiments of this application may be an electronic device, for example, an electronic device with an operating system; or may be a component in an electronic device, for example, an integrated circuit or a chip. The electronic device may be a terminal or another device other than the terminal. For example, the terminal may include but is not limited to the aforementioned types of the terminal 11, and the another device may be a server, a Network Attached Storage (NAS), or the like. This is not specifically limited in the embodiments of this application.
The model obtaining apparatus provided in the embodiments of this application is capable of implementing the processes implemented in the method embodiments of
Optionally, as shown in
An embodiment of this application further provides a first communication device, including a processor and a communication interface. The communication interface is configured to: send, for the first communication device, a first request to a second communication device, where the first request is used to request to obtain information of at least one third communication device, and each third communication device is able to provide model information of at least one model required by the first communication device; receive information of at least one third communication device that is sent by the second communication device; and obtain model information of a plurality of models based on the information of the at least one third communication device, where the plurality of models are used to derive data analytics result information. The embodiment of the first communication device corresponds to the method embodiment on the first communication device side, and all implementation processes and implementations of the method embodiment are applicable to the embodiment of the first communication device, with the same technical effect achieved.
An embodiment of this application further provides a second communication device, including a processor and a communication interface. The communication interface is configured to: receive a first request sent by a first communication device, where the first request is used to request to obtain information of at least one third communication device, and each third communication device is able to provide model information of at least one model required by the first communication device; and send information of the at least one third communication device to the first communication device, where the information of the at least one third communication device is used for the first communication device to obtain model information of a plurality of models, and the plurality of models are used to derive data analytics result information. The processor is configured to determine at least one third communication device according to the first request. The embodiment of the second communication device corresponds to the method embodiment on the second communication device side, and all implementation processes and implementations of the method embodiment are applicable to the embodiment of the second communication device, with the same technical effect achieved.
An embodiment of this application further provides a third communication device, including a processor and a communication interface. The communication interface is configured to: receive a second request sent by a first communication device, where the second request is used to request to obtain at least one model; and send model information of at least one model to the first communication device according to the second request, where the second request includes at least one of the following: an analytics task identifier, where the analytics task identifier is used to identify a data analytics task to which a required model is applicable; an identifier of a required model; the number of required models; or model attribute information that a model needs to meet. The embodiment of the third communication device corresponds to the method embodiment on the third communication device side, and all implementation processes and implementations of the method embodiment are applicable to the embodiment of the third communication device, with the same technical effect achieved.
An embodiment of this application further provides a fourth communication device, including a processor and a communication interface. The communication interface is configured to: send a task request to a first communication device; and receive data analytics result information sent by the first communication device, where the data analytics result information is obtained by the first communication device by performing analytics based on a plurality of models, the plurality of models are obtained by the first communication device based on information of at least one third communication device, and each third communication device is able to provide model information of at least one model required by the first communication device. The embodiment of the fourth communication device corresponds to the method embodiment on the fourth communication device side, and all implementation processes and implementations of the method embodiment are applicable to the embodiment of the fourth communication device, with the same technical effect achieved.
Specifically, an embodiment of this application further provides a network-side device. As shown in
Specifically, the network-side device 1200 in this embodiment of the present invention further includes instructions or a program stored in the memory 1203 and capable of running on the processor 1201, and the processor 1201 invokes the instructions or program in the memory 1203 to perform the method performed by the modules shown in
Optionally, the structure of the network-side device may be applied to the first communication device, the second communication device, the third communication device, and the fourth communication device.
An embodiment of this application further provides a readable storage medium. The readable storage medium stores a program or instructions. When the program or instructions are executed by a processor, the processes in the model obtaining method embodiments are implemented, with the same technical effect achieved. To avoid repetition, details are not described herein again.
The processor is a processor in the terminal in the foregoing embodiments. The readable storage medium includes a computer-readable storage medium, for example, a computer read-only memory ROM, a random access memory RAM, a magnetic disk, or an optical disc.
An embodiment of this application further provides a chip. The chip includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is configured to run a program or instructions to implement the processes in the model obtaining method embodiments, with the same technical effect achieved. To avoid repetition, details are not described herein again.
It should be understood that the chip provided in this embodiment of this application may also be referred to as a system-level chip, a system on chip, a chip system, a system-on-a-chip, or the like.
An embodiment of this application further provides a computer program or program product. The computer program or program product is stored in a storage medium. The computer program or program product is executed by at least one processor to implement the processes in the model obtaining method embodiments, with the same technical effect achieved. To avoid repetition, details are not described herein again.
An embodiment of this application further provides a communication system, including a first communication device, a second communication device, a third communication device, and a fourth communication device, where the first communication device, the second communication device, the third communication device, and the fourth communication device may be configured to perform the steps of the model obtaining method.
An embodiment of this application further provides a model processing method. As shown in
Step 1302: The first network element receives a registration request response message from the second network element.
The model quantity information is used to indicate the number of models that correspond to an analytics task identifier and that are supported by the first network element. The model information includes at least one of the following information of a model that corresponds to the analytics task identifier and that is supported by the first network element:
In this embodiment of this application, the first network element may be understood as a model training network element, for example, in some embodiments, may be a Model Training logical function (MTLF), also referred to as an MTLF included in a Network Data Analytics Function (NWDAF). The second network element may be understood as an NRF.
It should be understood that the registration request message may further include other information in addition to the capability information, for example, may include:
Optionally, the model performance information may be used to represent accuracy, an error value, and the like of an output result of a model. In some embodiments, the model performance information may include at least one of the following: first performance information, where the first performance information is used to indicate performance that a model is able to achieve in a training stage; or second performance information, where the second performance information is used to indicate performance that a model is able to achieve in an inference stage.
Optionally, after receiving the registration request message, the second network element may store information carried in the registration request message, and send the registration request response message to the first network element after determining that registration is successful.
Optionally, the model size may be represented by a parameter of a model. The source information of training data and the time information of training data may be understood as data information for training a model. Certainly, in other embodiments, the registration request message may alternatively include other data information for training a model. This is not further limited herein.
It should be noted that, when the third network element needs to invoke a model in the first network element to perform a target analytics task, the third network element may first send a second request message to the second network element to request to find a first network element able to perform the target analytics task. Because capability information is stored in the second network element, N first network elements that have models able to be used to perform the target analytics task may be determined based on the capability information. Then the third network element selects a target network element from the N first network elements, and invokes a model in the target network element to perform the target analytics task, to be specific, performs model inference for the target task by using the model in the target network element. Because a first network element matching the target analytics task can be found from the second network element based on the capability information, reliability of subsequent model inference for the target analytics task can be improved. In addition, the following problem can be avoided: A new first network element needs to be found to perform model inference for the target analytics task again due to low accuracy and/or a large error value of model inference for the target analytics task. Therefore, in this embodiment of this application, efficiency of finding a first network element can be further improved.
For example, in some embodiments, when target information includes model quantity information, the second network element may obtain, through matching, a first network element meeting a model quantity requirement of the target analytics task. This can improve reliability of model inference for the target analytics task. In some embodiments, when target information includes model information, the second network element may obtain, through matching, a first network element meeting a model requirement (for example, a model performance requirement) of the target analytics task. This can improve reliability of model inference for the target analytics task.
Optionally, the third network element may be understood as a data Analytics logical function (AnLF), also referred to as an AnLF included in an NWDAF.
In this embodiment of this application, a first network element sends a registration request message to a second network element, where the registration request message includes capability information of the first network element, and the capability information includes at least one of model quantity information and model information; and the first network element receives a registration request response message from the second network element. In this way, because the capability information is carried during registration, the second network element may determine, based on the capability information, a first network element matching a target analytics task, and then may invoke a model in the matching first network element to perform model inference for the target analytics task. Therefore, in this embodiment of this application, reliability of model inference can be improved.
Optionally, in some embodiments, after the first network element receives the registration request response message from the second network element, the method further includes:
In this embodiment of this application, after obtaining N first network elements, the third network element may send the first request message to the first network elements to request to obtain a target model able to perform the target analytics task. The target analytics task may be information received by the third network element from a fourth network element. For example, the fourth network element may send a task request message to the third network element, where the task request message may include an analytics task identifier of the target analytics task, and may further include task filter information. The task filter information may include filter information and an analytic target of a data analytics task model. The filter information of the data analytics task model may be referred to as machine learning model filter information (Machine Learning model filter info). The task filter information may be used to define a task scope, such as an analytics object (for example, mobility analytics for specific UE) of a model, target time of a task, an Area of Interest (AOI), Single Network Slice Selection Assistance Information (S-NSSAI), and a Data Network Name (DNN)
Optionally, after receiving the first request message, the first network element may determine, based on the first request message, a target model able to be used to perform the target analytics task, and then indicate the target model by using the first request response message. After receiving the first request response message, the third network element may perform a model inference operation for the target task by using the target model, derive a target analytics report, and finally send the target analytics report to the fourth network element.
Optionally, in some embodiments, the first request message includes an analytics task identifier of the target analytics task and model filter information, and the model filter information includes at least one of the following:
In this embodiment of this application, the model filter information may be determined based on the task filter information, or may be agreed upon in a protocol (for example, model filter information corresponding to different tasks is agreed upon in the protocol) or determined by the third network element itself. The target model fed back by the first network element should meet the model filter information. Details are as follows:
The filter information of the number of models may be understood as filter information of the number of models corresponding to the analytics task identifier of the target analytics task. In some embodiments, the filter information of the number of models may include a quantity threshold. In a case that the model filter information includes the filter information of the number of models, the number of target models that the first network element needs to feed back to the third network element should be greater than or equal to the quantity threshold, or the number of target models to be fed back should be less than or equal to the quantity threshold.
The filter information of a model identifier may include one or more preset model identifiers. In a case that the model filter information includes the filter information of a model identifier, the first network element needs to feed back, to the third network element, a target model among models corresponding to the preset model identifiers; or the first network element needs to feed back, to the third network element, a target model among models other than the models corresponding to the preset model identifiers.
The filter information of model performance information may include a performance indicator. In a case that the model filter information includes the filter information of performance information, the first network element needs to feed back a target model meeting the performance indicator to the third network element. The filter information of model performance information may include at least one of filter information of the first performance information and filter information of the second performance information.
The filter information of a model size may include a model size threshold. In a case that the model filter information includes the filter information of a model size, the first network element needs to feed back, to the third network element, a target model whose model size is greater than or equal to the model size threshold, or feed back a target model whose model size is less than or equal to the model size threshold.
The filter information of inference duration may include an inference time threshold. In a case that the model filter information includes the filter information of inference duration, the first network element needs to feed back, to the third network element, a target model whose inference duration is greater than or equal to the inference time threshold, or feed back a target model whose inference duration is less than or equal to the inference time threshold.
The filter information of a source of training data may include source information of training data. In a case that the model filter information includes the filter information of a source of training data, the first network element needs to feed back, to the third network element, a target model among models trained by using training data corresponding to the source information of training data; or the first network element needs to feed back, to the third network element, a target model among models other than the models trained by using training data corresponding to the source information of training data.
The filter information of time information of training data may include a time threshold of training data. In a case that the model filter information includes the filter information of time information of training data, a time range may be determined based on the time threshold. The first network element needs to feed back, to the third network element, a target model that uses, in a training stage, training data generated within time that belongs to the time range.
Optionally, the first performance information includes at least one of the following information of the model in training: a first performance indicator, a calculation method for the first performance indicator, first time information, a first value, and a first result, where the first performance indicator includes at least one of accuracy in training and an error value in training, the first time information includes time information corresponding to calculation of the first performance indicator, the first value is used to represent an amount of data used for calculating the first performance indicator, and the first result is a result value obtained through calculation based on a plurality of first performance indicators.
In this embodiment of this application, the first performance indicator may also be understood as performance of the model in training (performance in Training), and is a value obtained based on specific statistical calculation, for example, may be at least one of the accuracy in Training (AiT) and the error value in training (Mean Absolute Error in Training, MAEiT). The accuracy in training may be referred to as accuracy of the model in training, and the accuracy may be obtained by dividing the number of times that the model obtains a correct decision-making result by the total number of times of decision-making. For example, the first network element may set a verification data set for evaluating accuracy of a model. The verification set includes data to be input to a model and actual label data. The first network element inputs verification input data to a trained model to obtain output data. Then the first network element compares the output data with the actual label data to verify consistency, and then obtains a value of accuracy of the model by using the foregoing calculation method.
The calculation method for the first performance indicator may include at least one of the following: a ratio of the number of accurate predictions by a model to the total number of predictions by the model, an MAE, a root mean square error, a recall, an F1 score, and the like.
The first time information is expressed as a time node or a period of time (for example, including start time of calculation of the first performance indicator and end time of calculation of the first performance indicator).
The first result may represent a distribution status of the plurality of first performance indicators, and may be specifically calculated by using a preset calculation method. For example, the first result may be a mean, a median, or a variance.
Optionally, in some embodiments, the second performance information includes at least one of the following information of the model in use: a second performance indicator, a calculation method for the second performance indicator, second time information, a second value, and a second result, where the second performance indicator includes at least one of accuracy in use and an error value in use, and the second time information includes time information corresponding to calculation of the second performance indicator.
In this embodiment of this application, the use may be understood as using the model for model inference, the second performance information corresponds to the first performance information, and the second performance indicator may also be understood as performance of the model in use (performance in use). For example, the calculation method for the second performance indicator may include at least one of the following: a ratio of the number of accurate predictions by a model to the total number of predictions by the model, an MAE, a root mean square error, a recall, an F1 score, and the like. The second time information is expressed as a time node or a period of time (for example, including start time of calculation of the second performance indicator and end time of calculation of the second performance indicator). The second result may represent a distribution status of the plurality of second performance indicators, and may be specifically calculated by using a preset calculation method. For example, the second result may be a mean, a median, or a variance. As shown in
Step 1401: An MTLF sends a registration request message to an NRF, where the registration request message may be referred to as a capability registration message for capability registration.
Optionally, the registration request message may include information such as identification information of the MTLF and a supported analytic ID, and may further include the foregoing capability information, to be specific, include at least one of model quantity information and model information.
Accuracy and an error value in the model information may be used for determining which one of two models is more suitable for the target analytics task. Distribution of accuracy may be used for determining whether performance of the model is stable. Whether a model matches the target analytics task may be determined based on source information and time information of training data of the model (for example, a model trained by using early data is more likely to be affected by a change in network data).
Step 1402: The NRF stores information carried in the registration request message.
Step 1403: The NRF sends a registration request response message.
Step 1404: A task consumer sends a task request message to an AnLF, where the task request message includes an analytics task identifier of the target analytics task, and filter information and an analytic target of a data analytics task model.
Optionally, an order of step 1403 and step 1404 is not agreed upon herein. Usually, step 1403 is performed before step 1404.
Step 1405: The AnLF sends a second request message to the NRF, where the second request message is used for finding a suitable MTLF, and may further include target requirement information in addition to the analytics task identifier of the target analytics task and the filter information of the data analytics task model.
Optionally, the target requirement information includes at least one of the following:
In this embodiment of this application, the target requirement information may be understood as a requirement for a model or a filter requirement for a model. The filter information of the number of models may be a quantity threshold. In this case, the number of models that a suitable MTLF can provide for performing the target analytics task needs to be greater than or equal to the quantity threshold, or less than or equal to the quantity threshold. The requirement information of a model identifier may include at least one model identifier. In this case, a model that a suitable MTLF can provide for performing the target analytics task includes or does not include a model corresponding to the at least one model identifier. The requirement information of model performance information may include a performance indicator (for example, may include at least one of a first performance indicator and a second performance indicator). In this case, a model that a suitable MTLF can provide for performing the target analytics task needs to meet a required performance indicator. The filter information of a model size may include a model size threshold. In this case, a model that a suitable MTLF can provide for performing the target analytics task needs to include a model whose model size is greater than or equal to the model size threshold, or needs to include a model whose model size is less than or equal to the model size threshold. The filter information of a source of training data may include source information of training data. In this case, a model that a suitable MTLF can provide for performing the target analytics task needs to include or not include a model trained by using training data corresponding to the source information of training data. The time information of training data may include a time threshold of training data, and a time range may be determined based on the time threshold. In this case, a suitable MTLF needs to include a model that uses, in a training stage, training data generated within time that belongs to the time range.
Step 1406: The NRF feeds back a second request response message to the AnLF, where the second request response message needs to include determined N MTLFs, and may further include at least one of a task analytics identifier supported by each MTLF, model quantity information, and model information.
It should be understood that the NRF determines the N MTLFs based on the second request message. Optionally, each MTLF includes a model that supports the analytics task identifier of the target analytics task, and the model (or the MTLF) needs to meet the target requirement information and the filter information of the data analytics task model.
Optionally, in some embodiments, the second request response message may further include validity time corresponding to each of the N first network elements. The validity time may be understood as validity time of registered capability information of an MTLF. After the time elapses, the capability information of the MTLF may change. If a model in the MTLF is requested beyond the validity time, reliability of final model inference may be unable to be guaranteed. Therefore, the AnLF may preferentially request a model in an MTLF within the validity time.
Step 1407: The AnLF determines a target MTLF from the N MTLFs.
For example, if the NRF feeds back one MTLF, the MTLF is determined as a target MTLF; or if the NRF feeds back a plurality of MTLFs, one of the plurality of MTLFs further needs to be selected as a target MTLF. Certainly, in other embodiments, the AnLF may alternatively select at least two MTLFs as target MTLFs.
Step 1408: The AnLF sends a first request message (namely, a model obtaining request), for example, Nnwdaf_MLModelInfo_Request or Nnwdaf_MLModelProvision_Request, to the target MTLF.
The first request message may include the analytics task identifier of the target analytics task and model filter information.
After receiving the first request message, the AnLF may find, through matching, a suitable target model based on the analytics task identifier of the target analytics task and the model filter information.
Step 1409: The target MTLF sends a first request response message to the AnLF, where the first request response message includes the target model and address information for obtaining the target model.
That the first request response message includes the target model may be understood as including a configuration file of the model, a description of the model, and the like. The address information of the target model may include a Uniform Resource Locator (URL), FQDN information, an IP address, and the like. After obtaining the address information, the AnLF may directly download the target model.
Step 1410: The AnLF performs model inference for the target analytics task based on the target model to obtain a target task report.
Optionally, the AnLF may obtain an inference result by using one target model, and report the inference result as a target task report. Alternatively, the AnLF may obtain different inference results by using a plurality of target models, and finally derive a target task report based on the plurality of inference results, for example, may perform an aggregation or voting operation on the plurality of inference results to obtain a final inference result as the target task report. The target task report may be understood as data analytics result information.
Step 1411: The AnLF feeds back a task request response message to the task consumer, where the task request response message includes the target analytics report.
In an implementation, the following solutions are protected in this application:
1. A model processing method, including:
2. The method according to 1, where after the receiving, by the first network element, a registration request response message from the second network element, the method further includes:
3. The method according to 2, where the first request message includes an analytics task identifier of the target analytics task and model filter information, and the model filter information includes at least one of the following:
4. The method according to any one of 1 to 3, where the model performance information includes at least one of the following:
5. The method according to 4, where the first performance information includes at least one of the following information of the model in training: a first performance indicator, a calculation method for the first performance indicator, first time information, a first value, and a first result, where the first performance indicator includes at least one of accuracy in training and an error value in training, the first time information includes time information corresponding to calculation of the first performance indicator, the first value is used to represent an amount of data used for calculating the first performance indicator, and the first result is a result value obtained through calculation based on a plurality of first performance indicators.
6. The method according to 4, where the second performance information includes at least one of the following information of the model in use: a second performance indicator, a calculation method for the second performance indicator, second time information, a second value, and a second result, where the second performance indicator includes at least one of accuracy in use and an error value in use, the second time information includes time information corresponding to calculation of the second performance indicator, the second value is used to represent an amount of data used for calculating the second performance indicator, and the second result is a result value obtained through calculation based on a plurality of second performance indicators.
7. A model processing method, including:
8. The method according to 7, where after the storing, by the second network element, the capability information, and sending a registration request response message to the first network element, the method further includes:
9. The method according to 8, where the second request message further includes target requirement information, any one of the N first network elements meets the target requirement information, and the target requirement information includes at least one of the following:
10. The method according to 8, where the second request response message includes at least one of identification information of the N first network elements and address information of the N first network elements.
11. The method according to 8, where the second request response message further includes validity time corresponding to each of the N first network elements.
12. The method according to any one of 7 to 11, where the model performance information includes at least one of the following:
13. The method according to 12, where the first performance information includes at least one of the following information of the model in training: a first performance indicator, a calculation method for the first performance indicator, first time information, a first value, and a first result, where the first performance indicator includes at least one of accuracy in training and an error value in training, the first time information includes time information corresponding to calculation of the first performance indicator, the first value is used to represent an amount of data used for calculating the first performance indicator, and the first result is a result value obtained through calculation based on a plurality of first performance indicators.
14. The method according to 12, where the second performance information includes at least one of the following information of the model in use: a second performance indicator, a calculation method for the second performance indicator, second time information, a second value, and a second result, where the second performance indicator includes at least one of accuracy in use and an error value in use, the second time information includes time information corresponding to calculation of the second performance indicator, the second value is used to represent an amount of data used for calculating the second performance indicator, and the second result is a result value obtained through calculation based on a plurality of second performance indicators.
15. A model processing method, including:
16. The method according to 15, where the first request message includes an analytics task identifier of the target analytics task and model filter information, and the model filter information includes at least one of the following:
17. The method according to 16, where the model performance information includes at least one of the following:
18. The method according to 17, where the first performance information includes at least one of the following information of the model in training: a first performance indicator, a calculation method for the first performance indicator, first time information, a first value, and a first result, where the first performance indicator includes at least one of accuracy in training and an error value in training, the first time information includes time information corresponding to calculation of the first performance indicator, the first value is used to represent an amount of data used for calculating the first performance indicator, and the first result is a result value obtained through calculation based on a plurality of first performance indicators.
19. The method according to 17, where the second performance information includes at least one of the following information of the model in use: a second performance indicator, a calculation method for the second performance indicator, second time information, a second value, and a second result, where the second performance indicator includes at least one of accuracy in use and an error value in use, the second time information includes time information corresponding to calculation of the second performance indicator, the second value is used to represent an amount of data used for calculating the second performance indicator, and the second result is a result value obtained through calculation based on a plurality of second performance indicators.
20. The method according to 15, where the second request message further includes target requirement information, any one of the N first network elements meets the target requirement information, and the target requirement information includes at least one of the following:
21. The method according to 15, where before the sending, by a third network element, a second request message to a second network element, the method further includes:
22. The method according to 21, where after the receiving, by the third network element, a first request response message from the target network element, the method further includes:
23. The method according to 22, where the number of target models is M1, M1 is a positive integer, and the performing, by the third network element, the target analytics task by using the target model to obtain a target analytics report includes:
24. The method according to 15, where the target network element is one first network element.
25. The method according to 15, where the second request response message includes at least one of identification information of the N first network elements and address information of the N first network elements.
26. The method according to 15, where the second request response message further includes validity time corresponding to each of the N first network elements.
27. A model processing apparatus, including:
28. The apparatus according to 27, where the first receiving module is further configured to receive a first request message from a third network element, where the first request message is used to obtain a target model able to be used to perform a target analytics task; and
29. The apparatus according to 28, where the first request message includes an analytics task identifier of the target analytics task and model filter information, and the model filter information includes at least one of the following:
30. The apparatus according to any one of 27 to 29, where the model performance information includes at least one of the following:
31. The apparatus according to 30, where the first performance information includes at least one of the following information of the model in training: a first performance indicator, a calculation method for the first performance indicator, first time information, a first value, and a first result, where the first performance indicator includes at least one of accuracy in training and an error value in training, the first time information includes time information corresponding to calculation of the first performance indicator, the first value is used to represent an amount of data used for calculating the first performance indicator, and the first result is a result value obtained through calculation based on a plurality of first performance indicators.
32. The apparatus according to 30, where the second performance information includes at least one of the following information of the model in use: a second performance indicator, a calculation method for the second performance indicator, second time information, a second value, and a second result, where the second performance indicator includes at least one of accuracy in use and an error value in use, the second time information includes time information corresponding to calculation of the second performance indicator, the second value is used to represent an amount of data used for calculating the second performance indicator, and the second result is a result value obtained through calculation based on a plurality of second performance indicators.
33. A model processing apparatus, including:
34. The apparatus according to 33, where the apparatus further includes a first determining module;
35. The apparatus according to 34, where the second request message further includes target requirement information, any one of the N first network elements meets the target requirement information, and the target requirement information includes at least one of the following:
36. The apparatus according to 34, where the second request response message includes at least one of identification information of the N first network elements and address information of the N first network elements.
37. The apparatus according to any one of 34 to 36, where the second request response message further includes validity time corresponding to each of the N first network elements.
38. The apparatus according to any one of 33 to 37, where the model performance information includes at least one of the following:
39. The apparatus according to 38, where the first performance information includes at least one of the following information of the model in training: a first performance indicator, a calculation method for the first performance indicator, first time information, a first value, and a first result, where the first performance indicator includes at least one of accuracy in training and an error value in training, the first time information includes time information corresponding to calculation of the first performance indicator, the first value is used to represent an amount of data used for calculating the first performance indicator, and the first result is a result value obtained through calculation based on a plurality of first performance indicators.
40. The apparatus according to 39, where the second performance information includes at least one of the following information of the model in use: a second performance indicator, a calculation method for the second performance indicator, second time information, a second value, and a second result, where the second performance indicator includes at least one of accuracy in use and an error value in use, the second time information includes time information corresponding to calculation of the second performance indicator, the second value is used to represent an amount of data used for calculating the second performance indicator, and the second result is a result value obtained through calculation based on a plurality of second performance indicators.
41. A model processing apparatus, including:
42. The apparatus according to 41, where the first request message includes an analytics task identifier of the target analytics task and model filter information, and the model filter information includes at least one of the following:
43. The apparatus according to 42, where the model performance information includes at least one of the following:
44. The apparatus according to 43, where the first performance information includes at least one of the following information of the model in training: a first performance indicator, a calculation method for the first performance indicator, first time information, a first value, and a first result, where the first performance indicator includes at least one of accuracy in training and an error value in training, the first time information includes time information corresponding to calculation of the first performance indicator, the first value is used to represent an amount of data used for calculating the first performance indicator, and the first result is a result value obtained through calculation based on a plurality of first performance indicators.
45. The apparatus according to 43, where the second performance information includes at least one of the following information of the model in use: a second performance indicator, a calculation method for the second performance indicator, second time information, a second value, and a second result, where the second performance indicator includes at least one of accuracy in use and an error value in use, the second time information includes time information corresponding to calculation of the second performance indicator, the second value is used to represent an amount of data used for calculating the second performance indicator, and the second result is a result value obtained through calculation based on a plurality of second performance indicators.
46. The apparatus according to 41, where the second request message further includes target requirement information, any one of the N first network elements meets the target requirement information, and the target requirement information includes at least one of the following:
47. The apparatus according to 46, where before the sending, by a third network element, a second request message to a second network element, the method further includes:
48. The apparatus according to 47, where the number of target models is M1, M1 is a positive integer, and the performing, by the third network element, the target analytics task by using the target model to obtain a target analytics report includes:
49. The apparatus according to 40, where the target network element is one first network element.
50. The apparatus according to 40, where the second request response message includes at least one of identification information of the N first network elements and address information of the N first network elements.
51. The apparatus according to 40, where the second request response message further includes validity time corresponding to each of the N first network elements.
52. A network-side device, including a processor and a memory, where the memory stores a program or instructions capable of running on the processor, and when the program or instructions are executed by the processor, the steps of the model processing method according to any one of 1 to 26 are implemented.
53. A readable storage medium, where the readable storage medium stores a program or instructions, and when the program or instructions are executed by a processor, the steps of the model processing method according to any one of 1 to 26 are implemented.
It should be noted that the terms “include”, “comprise”, or any other variation thereof in this specification are intended to cover a non-exclusive inclusion, so that a process, a method, an object, or an apparatus that includes a list of elements not only includes those elements but also includes other elements that are not expressly listed, or further includes elements inherent to such a process, method, object, or apparatus. In absence of more constraints, an element preceded by “includes a . . . ” does not preclude the existence of other identical elements in the process, method, article, or apparatus that includes the element. In addition, it should be noted that the scope of the method and apparatus in the implementations of this application is not limited to performing functions in the shown or described order, but may also include performing functions in a substantially simultaneous manner or in a reverse order depending on the functions involved. For example, the described method may be performed in an order different from that described, and steps may be added, omitted, or combined. In addition, features described with reference to some examples may be combined in other examples.
According to the foregoing descriptions of the implementations, persons skilled in the art can clearly understand that the methods in the foregoing embodiments may be implemented by using software in combination with a necessary common hardware platform, or certainly may be implemented by using hardware. However, in most cases, the former is a preferred implementation. Based on such an understanding, the technical solutions of this application essentially or the part contributing to the conventional technology may be implemented in a form of a computer software product. The computer software product is stored in a storage medium (for example, a ROM/RAM, a magnetic disk, or a compact disc), and includes several instructions for instructing a terminal (which may be a mobile phone, a computer, a server, an air conditioner, a network device, or the like) to perform the methods in the embodiments of this application.
The foregoing describes the embodiments of this application with reference to the accompanying drawings. However, this application is not limited to the foregoing specific implementations. The foregoing specific implementations are merely examples, but are not limitative. Inspired by this application, persons of ordinary skill in the art may further make many modifications without departing from the purposes of this application and the protection scope of the claims, and all the modifications shall fall within the protection scope of this application.
| Number | Date | Country | Kind |
|---|---|---|---|
| 202210482186.3 | May 2022 | CN | national |
| 202210483893.4 | May 2022 | CN | national |
This application is a continuation of International Patent Application No. PCT/CN2023/092194, filed on May 5, 2023, which claims priorities to Chinese Patent Application No. 202210483893.4, filed on May 5, 2022 and entitled “MODEL OBTAINING METHOD AND COMMUNICATION DEVICE”, and to Chinese Patent Application No. 202210482186.3, filed on May 5, 2022 and entitled “MODEL PROCESSING METHOD AND APPARATUS, NETWORK-SIDE DEVICE, AND READABLE STORAGE MEDIUM”, all of which are incorporated herein by reference in their entireties.
| Number | Date | Country | |
|---|---|---|---|
| Parent | PCT/CN2023/092194 | May 2023 | WO |
| Child | 18937061 | US |
