Patent Application
20190180180
The present invention relates to an information processing system, an information processing method, and a program, and more particularly, to an information processing system, an information processing method, and a recording medium that generate an analysis pipeline.
A procedure of data analyzing in machine learning and data mining broadly includes a pre-process on data to be analyzed and learning process performed by inputting the pre-processed data to an analysis engine. In the pre-process, removal of an abnormal value and a deficit value in data, scale conversion such as standardization and normalization, generation of a necessary attribute, and the like are performed. In the engine, a regression analysis, a discriminant analysis, clustering, and the like are performed as learning process depending on a purpose.
The series of processes of data analyzing can be expressed by, for example, a series of processes including removal of a deficit value, standardization, and a regression analysis, namely, a pipeline. Hereinafter, the series of processes of data analyzing is referred to as an analysis pipeline.
Some processes in the analysis pipeline include a parameter that can be adjusted by a person. For example, in an abnormal value removing process, a value to be regarded as abnormal is set as a parameter. Further, in a discriminant analysis process, when a decision tree is used in a discriminant analysis, the tallest height of a tree to be learned is set as a parameter. Hereinafter, a parameter related to a pre-process and a learning process of the analysis pipeline is also referred to as a pipeline parameter.
Setting an appropriate value to a pipeline parameter is important for improving precision of an analysis. For example, when a height of a decision tree is too high, a model generated by learning overfits data, whereas, when a height of a decision tree is too low, a model generated by learning underfits data. Therefore, a parameter needs to be adjusted in such a way that an appropriate value is set to data to be analyzed.
Such adjustment of a pipeline parameter by a person generally takes time. Thus, a system for searching for an appropriate value of a parameter and adjusting the value is used. Grid Search is known as the simplest and general method among methods of searching for a value of a parameter. In Grid Search, a grid is generated based on candidate values of each parameter, all grid points are searched, and a set of optimum values of parameters is obtained. For example, when two parameters a and b respectively have candidate values like a=[1, 10, 100] and b=[1, 0.1, 0.01], nine combinations (3×3 combinations) of values are searched. Although Grid Search is simple, grid points to be searched are likely to be massive, and thus taking time. As a method for solving such a problem of Grid Search, Random Search, a method to which Bayesian optimization is applied, and the like are proposed.
Further, in adjustment of a pipeline parameter, validation of a generated model needs to be performed together with a search for a value of a parameter. As a general validation method of machine learning, a method that divides data to be analyzed into two pieces of data of learning data and test data, generates a model with the learning data, and calculates an evaluation value with the test data is known. In this method, a prediction is performed based on test data by using a generated model, and precision of the prediction is calculated as an evaluation value of the model. Hereinafter, this method is referred to as Single Validation. Furthermore, Cross Validation that repeats generation of a similar model and calculation of an evaluation value while changing pieces of data used as learning data and test data among the same pieces of data to be analyzed is also known.
Systems for adjusting a parameter by using the search method and the validation method are described in the following literatures. For example, NPL 1 describes GridSearchCV using Grid Search and Cross Validation, and RandomSearchCV using Random Search and Cross Validation, as a search method and a validation method, respectively. NPL 2 describes Cross Validator using Grid Search and Cross Validation as a search method and a validation method, respectively.
Further, NPL 3 describes Random Search described above as a search method. NPL 4 describes a method to which Bayesian optimization is applied as a search method.
[NPL 1] “scikit-learn: machine learning in Python”, [online], [Retrieved on May 26, 2016], Internet <URL: http://scikit-learn.org/stable/>
[NPL 2] “Overview: estimators, transformers and pipelines—spark.ml”, [online], [Retrieved on May 26, 2016], Internet <URL: http://spark.apache.org/docs/latest/ml-guide.html>
[NPL 3] James Bergstra, Yoshua Bengio, “Random Search for Hyper-Parameter Optimization”, Journal of Machine Learning Research 13, pages 281-305, 2012
[NPL 4] Jasper Snoek, Hugo Larochelle, Ryan P. Adams, “Practical Bayesian Optimization of Machine Learning Algorithms”, Advances in Neural Information Processing Systems 25 (NIPS 2012), 2012
However, GridSearchCV and RandomSearchCV described in NPL 1 and CrossValidator described in NPL2 have the following problem. That is, since each of these systems has a fixed search method and a fixed validation method, when, for example, a pipeline parameter is adjusted by various validation methods, different systems need to be used according to each validation method. In analysis business, not only Single Validation and Cross Validation described above, but also an original validation method suitable for more actual usage scenes is used as a validation method. For example, in prediction of time-series data, a method of performing a prediction for a year by using a relearned model every three months and obtaining yearly average precision of the model, and the like are used as a validation method. Therefore, preparing a system for adjusting a parameter for each validation method is unrealistic.
An example object of the present invention is to provide an information processing system, an information processing method, and a recording medium that are capable of solving the above-described problem and adjusting a parameter related to an analysis pipeline for various validation methods.
An information processing system for generating an analysis pipeline model by using an analysis pipeline, the analysis pipeline including a pre-process and a learning process for data to be analyzed, a value of a pipeline parameter being a parameter related to at least one of the pre-process and the learning process being applied to the analysis pipeline, the analysis pipeline model including the pre-process and a learned model being learned with the learning process, according to an exemplary aspect of the present invention includes: initialization means for receiving an input of a validation module that generates the analysis pipeline model and calculates an evaluation value of the generated analysis pipeline model by using an input analysis pipeline in accordance with a predetermined validation method, and outputs the generated analysis pipeline model and the calculated evaluation value; and adjustment means for searching for, within a search range of a parameter set including the pipeline parameter and in accordance with a predetermined search method, a value of the parameter set for which the evaluation value is optimized by inputting to the validation module the analysis pipeline to which a value of the pipeline parameter is applied and executing the validation module, and outputting the analysis pipeline model for which the evaluation value is optimized.
An information processing method for generating an analysis pipeline model by using an analysis pipeline, the analysis pipeline including a pre-process and a learning process for data to be analyzed, a value of a pipeline parameter being a parameter related to at least one of the pre-process and the learning process being applied to the analysis pipeline, the analysis pipeline model including the pre-process and a learned model being learned with the learning process, according to an exemplary aspect of the present invention includes: receiving an input of a validation module that generates the analysis pipeline model and calculates an evaluation value of the generated analysis pipeline model by using an input analysis pipeline in accordance with a predetermined validation method, and outputs the generated analysis pipeline model and the calculated evaluation value; and searching for, within a search range of a parameter set including the pipeline parameter and in accordance with a predetermined search method, a value of the parameter set for which the evaluation value is optimized by inputting to the validation module the analysis pipeline to which a value of the pipeline parameter is applied and executing the validation module, and outputting the analysis pipeline model for which the evaluation value is optimized.
A computer readable storage medium recording thereon a program for generating an analysis pipeline model by using an analysis pipeline, the analysis pipeline including a pre-process and a learning process for data to be analyzed, a value of a pipeline parameter being a parameter related to at least one of the pre-process and the learning process being applied to the analysis pipeline, the analysis pipeline model including the pre-process and a learned model being learned with the learning process, the program, according to an exemplary aspect of the present invention causes a computer to perform processes including: receiving an input of a validation module that generates the analysis pipeline model and calculates an evaluation value of the generated analysis pipeline model by using an input analysis pipeline in accordance with a predetermined validation method, and outputs the generated analysis pipeline model and the calculated evaluation value; and searching for, within a search range of a parameter set including the pipeline parameter and in accordance with a predetermined search method, a value of the parameter set for which the evaluation value is optimized by inputting to the validation module the analysis pipeline to which a value of the pipeline parameter is applied and executing the validation module, and outputting the analysis pipeline model for which the evaluation value is optimized.
An advantageous effect of the present invention is to enable adjusting a parameter related to an analysis pipeline for various validation methods.
Example embodiments of the present invention is described in detail with reference to drawings. Note that, similar structural components have the same reference signs in each of the drawings and each of the example embodiments in the specification, and description thereof is appropriately omitted.
A first example embodiment of the present invention is described.
First, an analysis pipeline and an analysis pipeline model in the example embodiment of the present invention are described.
The analysis pipeline has a pipeline parameter related to at least one of the pre-process and the learning process. In the analysis pipeline in
Note that, the analysis pipeline and the analysis pipeline model are programs executed on a central processing unit (CPU), for example.
Next, a configuration of the first example embodiment of the present invention is described.
With reference to
The initialization unit 110 receives, from a user and the like, inputs of data to be analyzed, and an analysis pipeline, a validation module, and a search module to be used in an analysis. The validation module, the search module, and the analysis pipeline are programs executed on the CPU, for example. Note that, the initialization unit 110 may receive inputs of identifiers of the analysis pipeline and the modules to be used among a plurality of analysis pipelines and modules stored in a storage unit (not illustrated) and the like.
As illustrated in
As inputs to the validation module, data to be analyzed and an analysis pipeline to which a value of one or more pipeline parameters are set (applied) are input from the objective function.
The validation module generates an analysis pipeline model and calculates an evaluation value of the generated analysis pipeline model by using the input data and the input analysis pipeline in accordance with a predetermined validation method corresponding to the validation module.
The validation module returns (outputs) the generated analysis pipeline model and the calculated evaluation value to the objective function.
Herein, as the predetermined validation method, Single Validation and Cross Validation described above, and the like are used, for example. Further, as the evaluation value, a root mean squared error (RMSE) calculated from a value predicted by the generated analysis pipeline model and an actual value, and the like are used, for example.
As inputs to the objective function, an argument x is specified (input) from the search module. As the argument x, for a set of one or more parameters (hereinafter also referred to as a parameter set), values of the parameters (hereinafter also described as values of a parameter set) are set. The parameter set includes one or more above-described pipeline parameters.
The objective function returns (outputs) an evaluation value and an analysis pipeline model obtained as a result of the execution of the validation module as a return value to the search module (Step S230).
As inputs to the search module, an objective function is input from the adjustment unit 150. Further, a search range for the argument x (the values of the parameter set) of the objective function is set by the initialization unit 110. As the search range, a range for values in accordance with a searching method of the search module to be used, the analysis pipeline to be used, and the validation module to be used is set. Note that, the search range may be input from the adjustment unit 150 instead of the initialization unit 110. Further, the search range may be set in the search module input from a user and the like, in advance.
The search module specifies a value within the search range as the argument x, and executes the input objective function. The search module searches for the argument x (the values of the parameter set) for which an evaluation value included in a return value of the objective function is optimized (takes minimum or maximum), in accordance with a predetermined search method of the search module. The search module returns (outputs) the return value (the evaluation value and the analysis pipeline model) of the objective function when the evaluation value is optimized, to the adjustment unit 150.
Herein, as the predetermined search method, Grid Search and Random Search described above, and the like are used, for example. Further, as long as the search module can execute the objective function, an input of the objective function may be omitted. Further, the search module may also return (output) the argument x (the values of the parameter set) when the evaluation value is optimized together with the return value from the objective function, to the adjustment unit 150.
By such definitions of the validation module, the objective function, and the search module, the validation module can be realized without depending on the search module. Further, the search module can also be realized without depending on the analysis pipeline and the validation module to be used.
The validation module storage unit 120 stores a validation module to be used.
The search module storage unit 130 stores a search module to be used.
The analysis pipeline storage unit 140 stores an analysis pipeline to be used.
The adjustment unit 150 generates the above-described objective function in accordance with data to be analyzed, an analysis pipeline, and a validation module to be used. The adjustment unit 150 inputs the generated objective function to a search module to be used, and executes the search module. The adjustment unit 150 outputs an analysis pipeline model obtained as a result of the execution of the search module to a user and the like.
Note that, the analysis pipeline adjustment system 100 may be a computer including a CPU and a storage medium that stores a program and operating by control based on the program.
In this case, the analysis pipeline adjustment system 100 includes a CPU 101, a storage device 102 (storage medium) such as a hard disk and a memory, an input-output device 103 such as a keyboard and a display, and a communication device 104 that communicates with another device and the like. The CPU 101 executes a program for realizing the initialization unit 110 and the adjustment unit 150. The storage device 102 stores information of the validation module storage unit 120, the search module storage unit 130, and the analysis pipeline storage unit 140. The input-output device 103 receives inputs of a validation module, a search module, and an analysis pipeline to be used from a user, and outputs an analysis pipeline model to the user. Further, the communication device 104 may receive a validation module, a search module, and an analysis pipeline to be used from another device and the like, or may transmit an analysis pipeline model to another device and the like.
Further, a part or the whole of each of the structural components of the analysis pipeline adjustment system 100 in
When a part or the whole of each of the structural components of the analysis pipeline adjustment system 100 in
Next, operation of the first example embodiment of the present invention is described.
It is assumed herein that data to be analyzed is the data “data1” in
Furthermore, it is assumed that the validation module, the search module, and the analysis pipeline to be used are stored in advance by a user and the like in the validation module storage unit 120, the search module storage unit 130, and the analysis pipeline storage unit 140, respectively.
First, the initialization unit 110 receives inputs of the data to be analyzed, and the validation module, the search module, and the analysis pipeline to be used from a user and the like (Step S110).
For example, the initialization unit 110 receives inputs of the data “data1” to be analyzed, and the validation module “SingleValidation1”, the search module “GridSearch1”, and the analysis pipeline “Pipeline1” to be used.
The initialization unit 110 stores the validation module, the search module, and the analysis pipeline in the validation module storage unit 120, the search module storage unit 130, and the analysis pipeline storage unit 140, respectively (Step S120). Herein, the initialization unit 110 may apply necessary configuration on the validation module and the search module.
For example, the initialization unit 110 configures the validation module “SingleValidation1” in such a way as to calculate an RMSE as an evaluation value and use 80 percent of data for learning and 20 percent of the data for testing as a division ratio of the data.
In
Next, the adjustment unit 150 acquires the analysis pipeline and the validation module to be used from the analysis pipeline storage unit 140 and the validation module storage unit 120, respectively. The adjustment unit 150 generates an objective function for the data to be analyzed, and the analysis pipeline and the validation module to be used (Step S130).
For example, the adjustment unit 150 generates an objective function f1(x) that performs the process as in
Next, the adjustment unit 150 acquires the search module to be used from the search module storage unit 130. The adjustment unit 150 inputs the generated objective function to the search module to be used, and executes the search module (Step S140).
For example, the adjustment unit 150 inputs the objective function f1(x) to the search module “GridSearch1”, and executes the search module “GridSearch1”.
The search module “GridSearch1” executes the objective function f1(x) for each of the six combinations of the values of the parameter set (degree d and regularization parameter λ) specified in the search range “grid1”.
For example, the search module “GridSearch1” sets the values “degree d=2 and regularization parameter λ=10̂-6” of the parameter set included in the search range “grid1” to an argument x, and executes the input objective function f1(x).
The objective function f1(x) sets the values “degree d=2 and regularization parameter λ=10̂-6” of the parameter set specified as the argument x to the analysis pipeline “Pipeline1”. Then, the objective function f1(x) inputs the data “data1” and the analysis pipeline “Pipeline1” to the validation module “SingleValidation1” and executes the validation module “SingleValidation1”.
The validation module “SingleValidation1” generates the analysis pipeline model “PipelineModel1” by using the data “data1” and the analysis pipeline “Pipeline1”. Herein, the validation module “SingleValidation1” generates the analysis pipeline model “PipelineModel1” by using 80 percent of the data “data1” as data for learning. Then, the validation module “SingleValidation1” calculates an evaluation value (RMSE) by using remaining 20 percent of the data “data1” as data for testing. The validation module “SingleValidation1” returns the analysis pipeline model “PipelineModel1” and the evaluation value (RMSE).
The objective function f1(x) returns the evaluation value (RMSE) and the analysis pipeline model “PipelineModel1” obtained as a result of the execution of the validation module “SingleValidation1” as a return value.
The search module “GridSearch1” returns, to the adjustment unit 150, an analysis pipeline model for a combination for which the evaluation value (RMSE) included in the return value is minimum among the six combinations of the values of the parameter set specified in the search range “grid1”.
Next, the adjustment unit 150 outputs the analysis pipeline model returned from the search module to a user and the like (Step S150).
For example, the adjustment unit 150 outputs the analysis pipeline model “PipelineModel1” returned from the search module.
Hereinafter, a user and the like may perform a prediction or an analysis on new data by using the generated analysis pipeline model “PipelineModel1”.
As described above, the operation of the first example embodiment of the present invention is completed.
Note that, a case where the validation module that performs Single Validation and the search module that performs Grid Search are respectively used as a validation module and a search module is described as an example herein. However, the present invention is not limited to this, and another validation module and another search module may be used as long as inputs, outputs, and a process of the validation module and the search module follow the above-described definitions.
For example, “CrossValidation1” that performs Cross Validation and “RandomSearch1” that performs Random Search may be respectively used as a validation module and a search module.
In this case, for example, the validation module “CrossValidation1” divides the data “data1” into 10 data blocks, performs cross-validation for the 10 data blocks, and returns an average of evaluation values (RMSEs) and the analysis pipeline model “PipelineModel1” for which an evaluation value (RMSE) is minimum.
Further, it is described as an example herein that the parameter set includes a parameter (pipeline parameter) related to a pre-process and a learning process in an analysis pipeline. However, the present invention is not limited to this, and the parameter set may include a parameter related to validation process in a validation module.
Note that, the objective function may input a combination of values of a parameter set as a list of “key” and “value” to the validation module, for example. In this case, when there is “key” of a parameter that can be set (applied) to the validation module in the list, the validation module sets (applies) a value of “value” associated with “key”. In this way, even when the validation module is different, behavior of the validation process can be changed with the same interface.
As a value of a parameter related to the validation process, a parameter value for specifying a narrowing ratio of data for learning is used, for example.
In
In this case, there are eight combinations of values in a search range of values for a parameter set (degree d, regularization parameter λ, and narrowing ratio num_train_ratio).
The adjustment unit 150 generates an objective function f2(x) that performs the process as in
The search module “GridSearch2” executes the validation module “SingleValidation1” through the objective function f2(x) for each of the eight combinations of the values of the parameter set specified in the search range “grid2”, and obtains an analysis pipeline model.
Further, as a parameter value related to the validation process, a value of a parameter (Refit flag) that specifies relearning (Refit process) with all pieces of data may be used.
In
In this case, there are eight combinations of values in a search range for values of a parameter set (degree d, regularization parameter λ, and Refit flag).
The search module “GridSearch3” executes the validation module “SingleValidation1” through the objective function f2(x) for each of the eight combinations of the values of the parameter set specified in the search range “grid3”, and obtains an analysis pipeline model.
In this way, a parameter set including a condition related to learning data and a condition related to relearning can be adjusted by including a parameter related to the validation process of the validation module into the parameter set, and thus an analysis pipeline model with a higher degree of precision can be obtained.
Next, a characteristic configuration of the first example embodiment of the present invention is described.
The initialization unit 110 receives an input of a validation module that generates an analysis pipeline model and calculates an evaluation value by using an input analysis pipeline in accordance with a predetermined validation method, and outputs the analysis pipeline model and the evaluation value.
The adjustment unit 150 searches for, within a search range of a parameter set and in accordance with a predetermined search method, a value of the parameter set for which the evaluation value is optimized by executing the validation module inputting to the validation module the analysis pipeline to which a value of the pipeline parameter is applied. The adjustment unit 150 outputs the analysis pipeline model for which the evaluation value is optimized.
Next, an advantageous effect of the first example embodiment of the present invention is described.
According to the first example embodiment of the present invention, a parameter related to an analysis pipeline can be adjusted for various validation methods. The reason is described as follows. That is, the initialization unit 110 receives an input of a validation module that generates an analysis pipeline model and calculates an evaluation value by using an input analysis pipeline in accordance with a predetermined validation method, and outputs the analysis pipeline model and the evaluation value. Then, the adjustment unit 150 searches for, within a search range of a parameter set and in accordance with a predetermined search method, a value of the parameter set for which the evaluation value is optimized by executing the validation module inputting to the validation module the analysis pipeline to which a value of the pipeline parameter is applied.
Further, according to the first example embodiment of the present invention, a parameter related to an analysis pipeline can be adjusted for various combinations of a validation method and a search method. The reason is described as follows. That is, the initialization unit 110 receives an input of a search module that searches for, within a search range of a parameter set and in accordance with a predetermined search method, a value of a parameter set for which an evaluation value is optimized by executing an objective function inputting a value of the parameter set to the objective function. Herein, the objective function is a function that outputs an analysis pipeline model and an evaluation value obtained by executing the validation module inputting to the validation module an analysis pipeline to which a value of a pipeline parameter included in an input parameter set is applied. Then, the adjustment unit 150 generates the objective function and executes the search module.
Next, a second example embodiment of the present invention is described.
The second example embodiment of the present invention is different from the first example embodiment of the present invention in that an analysis pipeline to be used is also specified as a parameter.
First, a configuration of the second example embodiment of the present invention is described.
A block diagram illustrating a configuration of an analysis pipeline adjustment system 100 in the second example embodiment of the present invention is similar to that (
In the second example embodiment of the present invention, an analysis pipeline storage unit 140 stores a plurality of analysis pipelines. Further, a parameter set includes an identifier of an analysis pipeline to be used in the second example embodiment of the present invention.
Note that, the objective function may generate an analysis pipeline of an identifier included in a parameter set, based on information related to the analysis pipeline, instead that the analysis pipeline storage unit 140 stores a plurality of analysis pipelines.
Next, operation of the second example embodiment of the present invention is described.
It is assumed herein that data to be analyzed is the data “data1” in
First, an initialization unit 110 receives inputs of data to be analyzed, and a validation module and a search module to be used, from a user and the like (Step S410).
For example, the initialization unit 110 receives inputs of the data “data1” to be analyzed, and the validation module “SingleValidation1” and the search module “GridSearch4” to be used.
The initialization unit 110 stores the validation module and the search module in a validation module storage unit 120 and a search module storage unit 130, respectively (Step S420). Herein, the initialization unit 110 may apply necessary configuration on the validation module and the search module.
In
In this case, there are four combinations of values as a search range for values of a parameter set (degree d and regularization parameter k), for the analysis pipeline “Pipeline1”. Further, there are four combinations of values as a search range for values of a parameter set (degree d and height h of decision tree), for the analysis pipeline “Pipeline2”. In other words, there are eight combinations as a search range for values of a parameter set.
Next, an adjustment unit 150 acquires a validation module to be used from the validation module storage unit 120. The adjustment unit 150 generates an objective function for the data to be analyzed, and the validation module to be used (Step S430).
For example, the adjustment unit 150 generates an objective function f3(x) that performs the process as in
Next, the adjustment unit 150 acquires the search module to be used from the search module storage unit 130. The adjustment unit 150 inputs the generated objective function to the search module to be used, and executes the search module (Step S440).
For example, the adjustment unit 150 inputs the objective function f3(x) to the search module “GridSearch4”, and executes the search module “GridSearch4”.
The search module “GridSearch4” executes the objective function f3(x) for each of the eight combinations of the values of the parameter set specified in the search range “grid4”.
For example, the search module “GridSearch4” sets values “analysis pipeline=“pipline2”, degree d=3, and height of decision tree h=10″ of a parameter set included in the search range “grid4” to an argument x, and executes the input objective function f3(x).
The objective function f3(x) acquires the analysis pipeline “pipline2” specified as the argument x and sets the values “degree d=3 and height of decision tree h=10” of the parameter set to the analysis pipeline “pipline2”. Then, the objective function f3(x) inputs the data “data1” and the analysis pipeline “pipline2” to the validation module “SingleValidation1” and executes the validation module “SingleValidation1”.
The validation module “SingleValidation1” generates the analysis pipeline model by using the data “data1” and the analysis pipeline “Pipeline2”.
The objective function f3(x) returns the evaluation value (RMSE) and the analysis pipeline model obtained as a result of the execution of the validation module “SingleValidation2” as a return value.
The search module “GridSearch4” returns, to the adjustment unit 150, an analysis pipeline model for a combination for which the evaluation value (RMSE) included in the return value is minimum among the eight combinations of the values of the parameter set specified in the search range “grid4”.
Next, the adjustment unit 150 outputs the analysis pipeline model returned from the search module to a user and the like (Step S450).
As described above, the operation in the second example embodiment of the present invention is completed.
Note that, a parameter set may also include a parameter related to validation process, such as a narrowing ratio of data for learning and a flag indicating relearning by all data, in the second example embodiment of the present invention, similarly to the first example embodiment of the present invention.
Next, an advantageous effect of the second example embodiment of the present invention is described.
According to the second example embodiment of the present invention, an analysis pipeline with a higher degree of precision than that in the first example embodiment of the present invention can be obtained. The reason is that a parameter set further includes an identifier of an analysis pipeline. In this way, a parameter set including a condition related to an analysis pipeline can be adjusted, and an analysis pipeline model with a higher degree of precision can be obtained.
While the present invention has been particularly shown and described with reference to the example embodiments thereof, the present invention is not limited to the embodiments. It will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present invention as defined by the claims.
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/JP2016/003137 | 6/30/2016 | WO | 00 |
