Patent Application
20220414691
This application relates to the field of computer application technologies, particularly to the field of big data technologies, and more particularly to methods and apparatuses for regional heat prediction model establishment and regional heat prediction.
A regional heat prediction has a great value, enables governments, institutions and individuals to more effectively optimize resource allocation and predict development trends, and even provides reference or suggestions on traffic. The regional heat prediction refers to a prediction of people flow in a region at a specified time, such as a prediction of passenger flow in a certain business district at a specified time, a prediction of passenger flow at a certain station at a specified time, etc.
The regional heat prediction is essentially a time series prediction problem, and an existing time series prediction algorithm includes feature engineering, neural network fitting, or the like. However, the existing time series prediction algorithm has high dependence on a large amount of annotation data, and may have higher prediction accuracy only after trained on data with longer stationarity series history records. However, some small probability events may affect regional heat, resulting in a drastic fluctuation of the regional heat over a period of time compared with ordinary times. However, since such a small probability event has less historical data, the data may not be effectively learned using a traditional method, resulting in poor prediction accuracy of the regional heat in the case of the small probability event.
In a first aspect, the present application provides a method for establishing a regional heat prediction model, including:
pre-training a time series prediction model using first historical regional heat data; and
taking second historical regional heat data as a second support set, and further training the time series prediction model using the second support set to adjust model parameters, so as to obtain the regional heat prediction model;
the regional heat prediction model being configured to predict a second query set, and the second query set including regional heat at a prediction time.
In a second aspect, the present application provides a regional heat prediction method, including:
predicting a second query set using a regional heat prediction model, the second query set including regional heat at a prediction time;
the regional heat prediction model being obtained by pre-training a time series prediction model using first historical regional heat data, then taking second historical regional heat data as a second support set, and further training the time series prediction model using the second support set.
In a third aspect, the present application provides an electronic device, including:
at least one processor; and
a memory connected with the at least one processor communicatively;
where the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the above-mentioned method.
In a fourth aspect, the present application provides a non-transitory computer readable storage medium storing computer instructions, which, when executed by a computer, cause the computer to perform the above-mentioned method.
The drawings are used for better understanding the present solution and do not constitute a limitation of the present application. In the drawings,
The following part will illustrate exemplary embodiments of the present application with reference to the drawings, including various details of the embodiments of the present application for a better understanding. The embodiments should be regarded only as exemplary ones. Therefore, those skilled in the art should appreciate that various changes or modifications can be made with respect to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for clarity and conciseness, the descriptions of the known functions and structures are omitted in the descriptions below.
In the embodiment of the present application, a few-samples-based learning technology, i.e., a few-shot learning technology is adopted, in which a large amount of regularly distributed historical data is used to train a few-shot time series prediction model, and then, by using the trained model in cooperation with a small amount of irregularly distributed historical data, satisfactory prediction accuracy may be obtained based on the small amount of irregularly distributed data, and a more ideal effect is achieved compared with a traditional supervised learning algorithm.
Few-shot learning data is composed of a support set and a query set. The support set contains instances of known categories, the query set contains more than one piece of data whose category is not labeled, and the task of the few-shot learning is predicting the category of the data in the query set.
Based on the above-mentioned theory, a main method according to the first embodiment of the present application has a flow chart shown in
101: pre-training a time series prediction model using first historical regional heat data.
102: taking second historical regional heat data as a second support set, and further training the time series prediction model using the second support set to adjust model parameters, so as to obtain the regional heat prediction model.
In the embodiment of the present application, the first historical regional heat data may include feature data and regional heat of a region at each time point in a first historical period. The second historical regional heat data may include feature data and regional heat of the region at each time point in a second historical period.
The region in the embodiment of the present application may be an area of interest (AOI) or a region of interest (ROI) at a geographical location, or may be a region formed by a plurality of points of interest (POIs), AOIs, ROIs, or the like. For example, the region may be a business district, a campus, a science park, or the like.
The feature data may include at least one of: a number of the POI contained in the region, data of distribution of user visiting time, data of distribution of travel modes of visiting users, or data of distribution of travel distances of the visiting users. The regional heat largely reflects a number of users visiting the region.
Further, after the regional heat prediction model is trained, step 103 may be executed using the trained regional heat prediction model; that is, a second query set is predicted using the regional heat prediction model, and the second query set includes regional heat at a prediction time.
As an example, the above-mentioned first historical regional heat data may be commonly distributed historical regional heat data. The second historical regional heat data may be uncommonly distributed historical regional heat data.
The above-mentioned method according to the present application will be described below in detail in conjunction with two embodiments with business district heat prediction as an example.
It is assumed that regional heat data of a specific business district at each time point of a historical period T is known, features of the business district in the historical period T may be represented by a feature sequence X=(x1, x2, . . . , xT), and xt represents a feature vector of the business district at the time point t. Correspondingly, a passenger flow (i.e., heat) of the business district in the historical period T may be represented by Y=(y1, y2, . . . , yT), and yt represents the heat of the business district at the time point t. For X and Y which are known, a current prediction task is to predict passenger flow =(yT+1, yT+2, . . . , yT+τ) at future τ time points.
xt may be obtained by feature data, such as a number of the POI contained in the business district, data of distribution of user visiting time, data of distribution of travel modes of visiting users, data of distribution of travel distances of the visiting users, or the like. Each feature is converted into a one-hot vector, and the vectors are spliced into a feature vector xt with n dimensions, n being a positive integer.
For example:
a business district is consisted of some POIs of a business place category, and then, the number of the POI contained in the business district may form a one-dimensional vector.
Distribution of visiting time of the business district may be represented by the number of visiting users in each of 24 hours of a day; that is, a 24-dimensional vector is formed.
Data of distribution of travel modes of the visiting users of the business district may be represented by a number of trips of a visiting crowd using various traffic modes, and the feature may be represented as a 4-dimensional vector on an assumption that public traffic, a bicycle, a private car and a walk serve as the traffic modes.
Data of distribution of travel distances of the visiting users of the business district may be represented by a number of different travel distances from departure places to the destination business district. If the travel distance is discretized into 10 levels (i.e.,<0.25 km, 0.25 km-0.5 km, 0.5 km-1 km, 1 km-2 km, 2 km-5 km, 5 km-6 km, 6 km-10 km, 10 km-15 km, 15 km-30 km, >30 km), the vector may be represented as a 10-dimensional vector.
Then, the vectors are spliced to obtain a 39-dimensional feature vector xt. The feature vector obtained by the above-mentioned features may fully reflect the passenger flow of the business district at a specific time point, such that the subsequent prediction of the heat of the business district is more accurate.
As an example, the time series prediction model adopted in this embodiment may be based on a recurrent neural network. A structure of the model may be shown in
For the prediction task, for a given input feature sequence X=(x1, x2, . . . , xT), the feature at each time point is mapped into a m -dimensional hidden-layer state vector ht using the recurrent neural network, which may be represented as:
h
t
=F(xt,ht−1) (1)
where F is a nonlinear function, and may be a recurrent neural network with a sequence modeling capability, such as a long short-term memory (LSTM), a gated recurrent unit (GRU), or the like.
The hidden-layer state vector hT at the last moment in the historical period T is sent into a feedforward neural network to obtain prediction output which is represented as:
Ŷ=Wh
T
+b (2)
where W and b are model parameters, W is a m+τ-dimensional vector, and b is a τ-dimensional vector.
In the training process, the above-mentioned time series prediction model is firstly pre-trained using the feature data and the heat of the business district at each time point in the first historical period T1. Then, the pre-trained time series prediction model is further trained using the feature data and the heat of the business district at each time point in the second historical period T2, so as to adjust the model parameters.
In the pre-training process using the feature data and the heat of the business district at each time point in the first historical period T1, a training set may be constructed using the feature data and the business district heat at each time point in the first historical period T1. A time window Tw may be adopted, Tw has a length less than duration of T1 and duration of T2, and the length of the time window Tw is the same as duration of the historical period T in the above-mentioned prediction task. Usually, the time window is much less than the duration of T1 and the duration of T2; for example, duration of 5 time points is selected as the time window Tw. During construction of the training set, one training set is constructed by the feature data and the business district heat at each time point in the time window Tw and the business district heat of τ time points after the time window Tw. N training sets may be generated by sliding the time window Tw in the first historical period T1, N being a positive integer.
In the training process, the time series prediction model predicts the business district heat of the τ time points after the time window Tw using the feature data and the business district heat at each time point in the time window Tw, with a training target of minimizing a difference value between a prediction result and the business district heat of the τ time points after the time window Tw in the training set. A loss function may be constructed using the above-mentioned training target (i.e., the difference value between the prediction result and the business district heat of the τ time points after the time window Tw in the training set). For example, a mean square error loss function may be constructed using the prediction result and the business district heat of the τ time points after the time window Tw in the training set, and a value of the loss function may be used to perform back propagation to optimize the parameters of the time series prediction model until a training ending condition is met. For example, the value of the loss function is less than or equal to a preset loss function threshold, or a number of iterations reaches a preset number threshold.
For example, the loss function L may adopt the following formula:
where N is a number of the training set, i is a prediction value, by the time series prediction model, of the business district heat of the τ time points after the time window Tw in the ith training set, and Yi is a value of the business district heat of the τ time points after the time window Tw in the ith training set.
After the time series prediction model is pre-trained, when the pre-trained time series prediction model is further trained using the feature data and the business district heat (i.e., the second support set) at each time point in the second historical period T2, a training set is also constructed by means of the above-mentioned time window, with a difference of further optimization and adjustment of the model parameters of the pre-trained time series prediction model.
The pre-trained and adjusted time series prediction model may be configured to predict the business district heat of part of time points in the second query set. The part of time points refer to the prediction time, and the prediction time has duration less than or equal to the above-mentioned τ. During the prediction, the feature data and the business district heat at each time point in the time period T before the prediction time are input into the time series prediction model to obtain the regional heat at the prediction time output by the time series prediction model. The above-mentioned formulas (1) and (2) are adopted in the prediction method.
For example, as shown in
In this embodiment, a further improvement is performed on the basis of the second embodiment, and the time series prediction model is pre-trained using a meta-learning method, thereby obtaining model parameters which may be quickly fitted on the second support set.
In this embodiment, at least one meta-training task is constructed using the first historical regional heat data before the pre-training process. Each meta-training task includes a first support set and a first query set, and the time series prediction model is trained using a meta-learning mechanism.
For example, as shown in
The first support set and the first query set have consistent duration with the second support set and the second query set respectively. As shown in
Unlike the second embodiment, in the meta-training process, for each meta-learning task, a gradient is calculated on the first support set, meta parameters are updated, and a gradient of the loss function of all the meta-learning tasks on the first query set is calculated and used to update the model parameters until the training ending condition is met. For example, the value of the loss function is less than or equal to a preset loss function threshold, or a number of iterations reaches a preset number threshold. So far, the pre-training process is completed to obtain the pre-trained time series prediction model.
Then, the pre-trained time series prediction model is further trained using the second support set (for example, the heat data of the business district in 1st to 8th weeks of 2020 in
In the above-mentioned training process, a training set is constructed in a similar manner as in the second embodiment; that is, one training set is constructed by the feature data and the business district heat at each time point in the time window Tw and the business district heat of τ time points after the time window Tw. A plurality of corresponding training sets may be generated by sliding the time window Tw in the corresponding first support set, first query set and second support set. In this embodiment, the time window Tw, has a length less than duration of the first support set and duration of the first query set. For the training target and the constructed loss function, reference is made to the related description in the second embodiment, which are not repeated herein.
Using the technical solution in the above-mentioned embodiment, when a small probability event occurs, the model may be enabled in well understanding a continuous influence of the small probability event on the region, and the regional heat may be more accurately predicted, thereby assisting a user in more pertinently optimizing a resource allocation and predicting a development trend.
After the above-mentioned prediction of the regional heat at the prediction time, the prediction result may be displayed, stored in a specific format, or further analyzed. The prediction result may also be sent to a user terminal, either actively or in response to a request from the user terminal.
The method according to the present application is described above in detail, and an apparatus according to the present application will be described below in detail in conjunction with an embodiment.
The pre-training module 01 is configured to pre-train a time series prediction model using first historical regional heat data.
The fine adjustment module 02 is configured to take second historical regional heat data as a second support set, and further train the time series prediction model using the second support set to adjust model parameters, so as to obtain the regional heat prediction model.
The regional heat prediction model is configured to predict a second query set, and the second query set includes regional heat at a prediction time.
As an exemplary embodiment, the first historical regional heat data includes feature data and regional heat of a region at each time point in a first historical period; and
the second historical regional heat data includes feature data and regional heat of the region at each time point in a second historical period.
The feature data includes at least one of: a number of POIs contained in the region, data of distribution of user visiting time, data of distribution of travel modes of visiting users, or data of distribution of travel distances of the visiting users.
The time series prediction model has a training target of minimizing a difference value between a prediction result and an expected value, the prediction result is a result of the time series model predicting the regional heat of τ time points after a time window Tw, using the feature data and the regional heat at each time point in the time window Tw, and the expected value is the regional heat of the τ time points after the time window Tw, in a corresponding set; the time window Tw is less than a length of the first historical period and a length of the second historical period, and τ is a positive integer.
As an exemplary embodiment, the pre-training module 01 may construct at least one meta-training task using the first historical regional heat data; and train the time series prediction model using a meta-learning mechanism based on a first support set and a first query set in the meta-training task; the first support set and the first query set have consistent duration with the second support set and the second query set respectively.
When training the time series prediction model using the meta-learning mechanism, the pre-training module 01 may, for each meta-learning task, calculate a gradient of a loss function on the first support set, and update meta parameters; and determine a gradient of the loss function of all the meta-learning tasks on the first query set and use the gradient of the loss function of all the meta-learning tasks on the first query set to update model parameters.
The fine adjustment module 02 is specifically configured to calculate a gradient of the loss function on the second support set and use the gradient of the loss function on the second support set to update the model parameters.
The loss function may be constructed, after the time series model predicts the regional heat of the τ time points after the time window Tw using the feature data and the regional heat at each time point in the time window Tw, using the difference value between the prediction result and the regional heat of the τ time points after the time window Tw in a corresponding set.
The time window Tw has a length less than duration of the first support set and duration of the first query set, and τ is a positive integer.
a predicting module 11 configured to predict a second query set using a regional heat prediction model, the second query set including regional heat at a prediction time.
The regional heat prediction model is obtained by: pre-training a time series prediction model using first historical regional heat data, then taking second historical regional heat data as a second support set, and further training the time series prediction model using the second support set. That is, the regional heat prediction model is pre-trained using the apparatus shown in
The first historical regional heat data includes feature data and regional heat of a region at each time point in a first historical period; and
the second historical regional heat data includes feature data and regional heat of the region at each time point in a second historical period.
As an exemplary embodiment, the predicting module 11 may predict the regional heat at the prediction time using the feature data and the regional heat at each time point in a time window T before the prediction time; the time window T is less than a length of the first historical period and a length of the second historical period.
The above-mentioned time window T actually has a consistent length with the time window Tw used in the process of training the time series prediction model in the fourth embodiment, and total duration of the prediction time is required to be less than or equal to the duration of τ used in the process of training the time series prediction model.
According to an embodiment of the present application, there are also provided an electronic device and a readable storage medium.
As shown in
The memory 702 is configured as the non-transitory computer readable storage medium according to the present application. The memory stores instructions executable by the at least one processor to cause the at least one processor to perform the method for establishing a regional heat prediction model or the regional heat prediction method according to the present application. The non-transitory computer readable storage medium according to the present application stores computer instructions for causing a computer to perform the method for establishing a regional heat prediction model or the regional heat prediction method according to the present application.
The memory 702 which is a non-transitory computer readable storage medium may be configured to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions/modules corresponding to the method for establishing a regional heat prediction model or the regional heat prediction method according to the embodiments of the present application. The processor 701 executes various functional applications and data processing of a server, that is, implements the method for establishing a regional heat prediction model or the regional heat prediction method according to the above-mentioned embodiments, by running the non-transitory software programs, instructions, and modules stored in the memory 702.
The memory 702 may include a program storage area and a data storage area, and the program storage area may store an operating system and an application program required for at least one function; the data storage area may store data created according to use of the electronic device, or the like. Furthermore, the memory 702 may include a high-speed random access memory, or a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid state storage devices. In some embodiments, optionally, the memory 702 may include memories remote from the processor 701, and such remote memories may be connected to the electronic device via a network. Examples of such a network include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The electronic device may further include an input device 703 and an output device 704. The processor 701, the memory 702, the input device 703 and the output device 704 may be connected by a bus or other means, and
The input device 703 may receive input numeric or character information and generate key signal input related to user settings and function control of the electronic device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, a pointing stick, one or more mouse buttons, a trackball, a joystick, or the like. The output device 704 may include a display device, an auxiliary lighting device (for example, an LED) and a tactile feedback device (for example, a vibrating motor), or the like. The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display, and a plasma display. In some implementations, the display device may be a touch screen.
Various implementations of the systems and technologies described here may be implemented in digital electronic circuitry, integrated circuitry, application specific integrated circuits (ASIC), computer hardware, firmware, software, and/or combinations thereof. The systems and technologies may be implemented in one or more computer programs which are executable and/or interpretable on a programmable system including at least one programmable processor, and the programmable processor may be special or general, and may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
These computer programs (also known as programs, software, software applications, or codes) include machine instructions for a programmable processor, and may be implemented using high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. As used herein, the terms “machine readable medium” and “computer readable medium” refer to any computer program product, device and/or apparatus (for example, magnetic discs, optical disks, memories, programmable logic devices (PLD)) for providing machine instructions and/or data for a programmable processor, including a machine readable medium which receives machine instructions as a machine readable signal. The term “machine readable signal” refers to any signal for providing machine instructions and/or data for a programmable processor.
To provide interaction with a user, the systems and technologies described here may be implemented on a computer having: a display device (for example, a cathode ray tube (CRT) or liquid crystal display (LCD) monitor) for displaying information to a user; and a keyboard and a pointing device (for example, a mouse or a trackball) by which a user may provide input for the computer. Other kinds of devices may also be used to provide interaction with a user; for example, feedback provided for a user may be any form of sensory feedback (for example, visual feedback, auditory feedback, or tactile feedback); and input from a user may be received in any form (including acoustic, speech or tactile input).
The systems and technologies described here may be implemented in a computing system (for example, as a data server) which includes a back-end component, or a computing system (for example, an application server) which includes a middleware component, or a computing system (for example, a user computer having a graphical user interface or a web browser through which a user may interact with an implementation of the systems and technologies described here) which includes a front-end component, or a computing system which includes any combination of such back-end, middleware, or front-end components. The components of the system may be interconnected through any form or medium of digital data communication (for example, a communication network). Examples of the communication network include: a local area network (LAN), a wide area network (WAN) and the Internet.
A computer system may include a client and a server. Generally, the client and the server are remote from each other and interact through the communication network. The relationship between the client and the server is generated by virtue of computer programs which run on respective computers and have a client-server relationship to each other.
It should be understood that various forms of the flows shown above may be used and reordered, and steps may be added or deleted. For example, the steps described in the present application may be executed in parallel, sequentially, or in different orders, which is not limited herein as long as the desired results of the technical solution disclosed in the present application may be achieved.
From the above technical solution, in the present application, the time series prediction model pre-trained using the first historical regional heat data is finely adjusted by taking the second historical regional heat data as the support set, thereby establishing the regional heat prediction model capable of predicting the regional heat at the prediction time in the second query set. In this way, even when the second historical regional heat data represents less distribution data, the distribution data may be effectively learned, thereby improving regional heat prediction accuracy.
The above-mentioned implementations are not intended to limit the scope of the present application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made, depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present application all should be included in the extent of protection of the present application.
| Number | Date | Country | Kind |
|---|---|---|---|
| 202011155016.1 | Oct 2020 | CN | national |
This application is the national phase of PCT/CN2021/097892 filed on Jun. 2, 2021, which claims priority to Chinese Patent Application No. 202011155016.1, filed on Oct. 26, 2020, entitled “Methods and Apparatuses for Regional Heat Prediction Model Establishment and Regional Heat Prediction”, which are hereby incorporated in their entireties by reference herein.
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/CN2021/097892 | 6/2/2021 | WO |
