This document relates generally to computer-implemented forecasting and estimation and more particularly to grid generated forecasting and estimation.
Forecasting is a process of making statements about events whose actual outcomes typically have not yet been observed. A commonplace example might be forecasting some variable of interest at some specified future date. Forecasting often involves formal statistical methods employing time series, cross-sectional or longitudinal data, or alternatively to less formal judgmental methods. Forecasts are often generated by providing a number of input values to a predictive model, where the model outputs a forecast. For example, a predictive model may consider a number of inputs including pricing, seasonality, marketing efforts, and other inputs in forecasting demand for a product.
Similarly, a model may be used to generate an estimate of a value based on a number of input values. For example, a property value for a real estate property may be estimated based on a lot size, availability of mortgages, nearby foreclosures, number of bedrooms, a local economy metric, a national economy metric, values of surrounding properties, an interest rate, as well as other inputs.
In accordance with the teachings herein, systems and methods are provided for estimating a value for a target variable. A plurality of known entities are assigned to cells of a grid, where the known entities are assigned to the cells based upon attribute data. A determination is made as to whether each cell has at least a threshold number of assigned known entities. When one of the cells contains fewer than the threshold number of known entities, cells are combined to form a super cell. A model is generated for each cell and super cell based upon target variable values for known entities assigned to that cell or super cell. Data for a target entity is received, and the target entity is assigned to one of the cells. One of the models is selected based upon the cell assignment, and an estimate is generated for the target variable for the target entity using the selected model.
As another example, a system for estimating a value for a target variable may include one or more data processors and one or more computer-readable mediums that include instructions for commanding the one or more data processors to execute steps. In the steps, a plurality of known entities are assigned to cells of a grid, where the known entities are assigned to the cells based upon attribute data. A determination is made as to whether each cell has at least a threshold number of assigned known entities. When one of the cells contains fewer than the threshold number of known entities, cells are combined to form a super cell. A model is generated for each cell and super cell based upon target variable values for known entities assigned to that cell or super cell. Data for a target entity is received, and the target entity is assigned to one of the cells. One of the models is selected based upon the cell assignment, and an estimate is generated for the target variable for the target entity using the selected model.
As a further example, a computer-readable medium may be encoded with instructions for commanding one or more data processors to execute a method for estimating a value for a target variable. In the method, plurality of known entities are assigned to cells of a grid, where the known entities are assigned to the cells based upon attribute data. A determination is made as to whether each cell has at least a threshold number of assigned known entities. When one of the cells contains fewer than the threshold number of known entities, cells are combined to form a super cell. A model is generated for each cell and super cell based upon target variable values for known entities assigned to that cell or super cell. Data for a target entity is received, and the target entity is assigned to one of the cells. One of the models is selected based upon the cell assignment, and an estimate is generated for the target variable for the target entity using the selected model.
Models for providing estimates often have a range of best performance, wherein when the input variable values are within the range of best performance, the model can be relied upon to provide high quality estimations of the target variable value. For example, in an implementation where the model is trained to estimate property values for properties within a city, the model may do a good job of estimating values for mid-value properties but may struggle to provide accurate estimates for high-value and low-value properties. To counter this effect, it may be desirable to provide multiple models for providing estimates, such as a model for estimating high-value property values, a model for estimating mid-value property values, and a model for estimating low-value property values. The incorporation of more models into the estimating process (e.g., a high-to-mid-value estimating model) may further improve performance.
The benefits of continued granularization of the models may be counterbalanced by a limited amount of known data available for training the models. The quality of a model may be largely impacted by the amount of known data provided to the model for training. Thus, where the models are over-granualized, performance of the data estimation system may break down, such that the use of fewer models will offer better performance.
The grid estimation engine 102 offers a tool for balancing the benefits of using multiple models to perform estimation with the loss in performance caused by under training models based on a lack of known data. Known entities having known values for input attributes of the entity as well as known values for the target variable for the known entity are divided amongst cells of a one-or-more dimension grid, where models are generated for different cells of the grid by training those models based upon the known entities that are assigned to the cells. When certain cells do not have enough known entities to perform a quality model training, those certain cells may be combined with other cells having known entities to form a super cell. A model is then generated for the super cell using the combined set of known entities so that the super cell model is sufficiently trained.
When a target entity is encountered, where input attribute values are known for the target entity, and an estimate for the target value for the target entity is sought, the target entity is assigned to one of the grid cells. The model generated for the cell or super cell associated with the assigned target entity cell is then used to generate an estimate for the target variable for the target entity.
The grid estimation engine 102 provides a platform for users 104 to generate estimates for a value for a target variable 106. A user 104 accesses the grid estimation engine 102, which is hosted on one or more servers 108, via one or more networks 110. The one or more servers 108 are responsive to one or more data stores 112. The one or more data stores 112 may contain a variety of data that includes input attribute data 114 and target variable data 106.
Grid estimation recognizes this special relationship between some input variables and the output variables and exploits it to create models that are accurate and robust. Specifically, the grid estimation engine 402 uses these special variables to create an intelligent grid of cells 404, and then uses the rest of the input variables to predict or estimate values for the output target variables by building a model 408 for each cell in the grid 404. The model 408 created for each cell 408 can be a simple model, such as a simple or weighted average, or a simple or multiple regression, or a more complex model, such as an artificial neural network. The models 408 for the cells may be of uniform type or disparate type across the different cells.
Let the special variables be sik i=1, . . . L, k=1, . . . N, where L defines the small number of special input variables and N defines the number of observations. Let xjk j=1, . . . M represent the remaining M input variables and yk represent the output variable. It is possible that only a small number of the N observations have a known output value. Let O≦N represent the number of observations with a known output value. If Di represents the number of groups in dimension i, then the grid will have ΠDi elements.
The grid regression engine 402 begins by taking the O observations with known output values 410 and dividing them into an L dimensional grid 404 such that the observations are distributed among each element of the grid as evenly as possible. For example, if the special variables are latitude and longitude (L=2) with the output variable being home prices, O=10,000 home sales in the past year, the latitude may be divided into six groups, and the longitude may be divided into ten groups so that the grid estimation engine 402 generates a two-dimensional grid having 60 cells.
Once the grid has been created, the grid estimation engine proceeds to build models for each grid cell or super cell having assigned known entities. Models can be simple or weighted average models or the models may be more complex models such as simple or multiple regression models or neural networks.
When the data models 812 have been generated, the grid estimation engine 802 is ready for estimating a target value for a target entity. Data for the target entity is received at 814. The received data 814 includes data for the input attribute variables used to define the grid. Using the attribute values for the target entity, the target entity is assigned to a cell at 816. Based on the cell assignment 816, one or more of the generated models 812 are selected at 818, and the one or more selected models are used to estimate the target value 820 for the target entity.
In
At 1014, data for a target entity is received, where the data for the target entity includes target entity values for the first attribute and the second attribute. At 1016, the target entity is assigned to one of the cells, where the target entity is assigned based upon the target entity values for the first attribute and the second attribute. Based upon the cell assignment, at 1018 one or more of the models are selected, and the one or more selected models are used at 1020 to generate an estimate for the target variable for the target entity.
Target entities can be scored based upon the cell to which they are assigned. Target entities having unknown output values that are assigned to a cell within the borders of model associated cells or super cells are estimated using the models developed for that cell or super cell. Target entities assigned to cells outside the borders of model associated cells or super cells may be assigned a model using missing value imputation techniques, where the target entity cell is assumed to be a missing element that is imputed using the special variables sik and other variables xjk. Once the target variables are estimated using the models, they may be capped using statistics from the assigned cell or some or all neighboring cells to ensure that the errors of cell assignment are minimized towards contribution to producing outliers.
For example, the first attribute value and the second attribute value for target entity 1112 are examined, and the target entity 1112 is assigned to cell 1106. Based on the assignment to cell 1106, the model associated with the group 1 cells is used to estimate the target value for the target entity 1112. The first attribute value, the second attribute value, and/or other variable values for the target entity 1112 are input into the model to estimate the target value.
As another example, the target entity 1114 in cell 1108 has its target value estimated using the models associated with group 1 and group 5. For example, the first attribute value, the second attribute value, and/or other variable values for the target entity 1114 may be individually input into the group 1 and group 5 models, where each of the models generate an independent estimate of the target value for target entity 1114. Those estimates can then be combined, such as via averaging or other method to generate a final estimate for the target value for target entity 1114.
A disk controller 1860 interfaces one or more optional disk drives to the system bus 1852. These disk drives may be external or internal floppy disk drives such as 1862, external or internal CD-ROM, CD-R, CD-RW or DVD drives such as 1864, or external or internal hard drives 1866. As indicated previously, these various disk drives and disk controllers are optional devices.
Each of the element managers, real-time data buffer, conveyors, file input processor, database index shared access memory loader, reference data buffer and data managers may include a software application stored in one or more of the disk drives connected to the disk controller 1860, the ROM 1856 and/or the RAM 1858. Preferably, the processor 1854 may access each component as required.
A display interface 1868 may permit information from the bus 1852 to be displayed on a display 1870 in audio, graphic, or alphanumeric format. Communication with external devices may optionally occur using various communication ports 1872.
In addition to the standard computer-type components, the hardware may also include data input devices, such as a keyboard 1873, or other input device 1874, such as a microphone, remote control, pointer, mouse and/or joystick.
As additional examples, for example, the systems and methods may include data signals conveyed via networks (e.g., local area network, wide area network, internet, combinations thereof, etc.), fiber optic medium, carrier waves, wireless networks, etc. for communication with one or more data processing devices. The data signals can carry any or all of the data disclosed herein that is provided to or from a device.
Additionally, the methods and systems described herein may be implemented on many different types of processing devices by program code comprising program instructions that are executable by the device processing subsystem. The software program instructions may include source code, object code, machine code, or any other stored data that is operable to cause a processing system to perform the methods and operations described herein. Other implementations may also be used, however, such as firmware or even appropriately designed hardware configured to carry out the methods and systems described herein.
The systems' and methods' data (e.g., associations, mappings, data input, data output, intermediate data results, final data results, etc.) may be stored and implemented in one or more different types of computer-implemented data stores, such as different types of storage devices and programming constructs (e.g., RAM, ROM, Flash memory, flat files, databases, programming data structures, programming variables, IF-THEN (or similar type) statement constructs, etc.). It is noted that data structures describe formats for use in organizing and storing data in databases, programs, memory, or other computer-readable media for use by a computer program.
The computer components, software modules, functions, data stores and data structures described herein may be connected directly or indirectly to each other in order to allow the flow of data needed for their operations. It is also noted that a module or processor includes but is not limited to a unit of code that performs a software operation, and can be implemented for example as a subroutine unit of code, or as a software function unit of code, or as an object (as in an object-oriented paradigm), or as an applet, or in a computer script language, or as another type of computer code. The software components and/or functionality may be located on a single computer or distributed across multiple computers depending upon the situation at hand.
It should be understood that as used in the description herein and throughout the claims that follow, the meaning of “a,” “an,” and “the” includes plural reference unless the context clearly dictates otherwise. Also, as used in the description herein and throughout the claims that follow, the meaning of “in” includes “in” and “on” unless the context clearly dictates otherwise. Further, as used in the description herein and throughout the claims that follow, the meaning of “each” does not require “each and every” unless the context clearly dictates otherwise. Finally, as used in the description herein and throughout the claims that follow, the meanings of “and” and “or” include both the conjunctive and disjunctive and may be used interchangeably unless the context expressly dictates otherwise; the phrase “exclusive or” may be used to indicate situation where only the disjunctive meaning may apply.
Number | Name | Date | Kind |
---|---|---|---|
5335291 | Kramer et al. | Aug 1994 | A |
5500513 | Langhans et al. | Mar 1996 | A |
5519319 | Smith et al. | May 1996 | A |
5650722 | Smith et al. | Jul 1997 | A |
5675253 | Smith et al. | Oct 1997 | A |
5677955 | Doggett et al. | Oct 1997 | A |
5761442 | Barr et al. | Jun 1998 | A |
5819226 | Gopinathan et al. | Oct 1998 | A |
5884289 | Anderson et al. | Mar 1999 | A |
5903830 | Joao et al. | May 1999 | A |
5999596 | Walker et al. | Dec 1999 | A |
6021943 | Chastain | Feb 2000 | A |
6029154 | Pettitt | Feb 2000 | A |
6047268 | Bartoli et al. | Apr 2000 | A |
6064990 | Goldsmith | May 2000 | A |
6122624 | Tetro et al. | Sep 2000 | A |
6125349 | Maher | Sep 2000 | A |
6128602 | Northington et al. | Oct 2000 | A |
6170744 | Lee et al. | Jan 2001 | B1 |
6330546 | Gopinathan et al. | Dec 2001 | B1 |
6388592 | Natarajan | May 2002 | B1 |
6422462 | Cohen | Jul 2002 | B1 |
6453206 | Soraghan et al. | Sep 2002 | B1 |
6516056 | Justice et al. | Feb 2003 | B1 |
6549861 | Mark et al. | Apr 2003 | B1 |
6570968 | Marchand et al. | May 2003 | B1 |
6601049 | Cooper | Jul 2003 | B1 |
6631212 | Luo et al. | Oct 2003 | B1 |
6650779 | Vachtesvanos et al. | Nov 2003 | B2 |
6675145 | Yehia et al. | Jan 2004 | B1 |
6678640 | Ishida et al. | Jan 2004 | B2 |
7117191 | Gavan et al. | Oct 2006 | B2 |
7191150 | Shao et al. | Mar 2007 | B1 |
7269516 | Brunner et al. | Sep 2007 | B2 |
7403922 | Lewis et al. | Jul 2008 | B1 |
7455226 | Hammond et al. | Nov 2008 | B1 |
7461048 | Teverovskiy et al. | Dec 2008 | B2 |
7467119 | Saidi et al. | Dec 2008 | B2 |
7480640 | Elad et al. | Jan 2009 | B1 |
7536348 | Shao et al. | May 2009 | B2 |
7562058 | Pinto et al. | Jul 2009 | B2 |
7580798 | Brunner et al. | Aug 2009 | B2 |
7761379 | Zoldi et al. | Jul 2010 | B2 |
7765148 | German et al. | Jul 2010 | B2 |
8015133 | Wu et al. | Sep 2011 | B1 |
8190512 | Subramanian et al. | May 2012 | B1 |
8346691 | Subramanian et al. | Jan 2013 | B1 |
20020099635 | Guiragosian | Jul 2002 | A1 |
20020138417 | Lawrence | Sep 2002 | A1 |
20020194119 | Wright et al. | Dec 2002 | A1 |
20030093366 | Halper et al. | May 2003 | A1 |
20030097330 | Hillmer et al. | May 2003 | A1 |
20030191709 | Elston et al. | Oct 2003 | A1 |
20040039688 | Sulkowski et al. | Feb 2004 | A1 |
20050055373 | Forman | Mar 2005 | A1 |
20050131873 | Fan et al. | Jun 2005 | A1 |
20060020814 | Lieblich et al. | Jan 2006 | A1 |
20060181411 | Fast et al. | Aug 2006 | A1 |
20060218169 | Steinberg et al. | Sep 2006 | A1 |
20070192167 | Lei et al. | Aug 2007 | A1 |
20070239606 | Eisen | Oct 2007 | A1 |
20080133518 | Kapoor et al. | Jun 2008 | A1 |
20080134236 | Iijima et al. | Jun 2008 | A1 |
Entry |
---|
Abdel-Wahhab, O. et al., “Image compression using multi-layer neural networks,”, Proceedings of the Second IEEE Symposium on Computers and Communications, vol. 144, Issue 5, pp. 307-312 (Oct. 1997). |
Almgren, Magnus et al., “Using Active Learning in Intrusion Detection,” Proceedings of the 17th IEEE Computer Security Foundations Workshop, 11 pp. (2004). |
Andras, Peter, “The Equivalence of Support Vector Machine and Regularization Neural Networks,” Neural Processing Letters, 65, pp. 97-104 (2002). |
Brause, Rudiger W., “Cascaded Vector Quantization by Non-Linear PCA Network Layers”, IEEE, pp. 154-160 (1994). |
Chan, Lipchen Alex et al., “Automatic target detection using dualband infrared imager”, Acoustics, Speech, and Signal PICASSP'00. Proceedings, 2000 IEEE International Conference, pp. 2286-2289 (Jun. 9, 2000). |
Chatterjee, Chanchal et al., “Self-Organizing Algorithms for Generalized Eigen-Decomposition”, IEEE Transactions on Neural Networks, vol. 8, No. 6, pp. 1518-1530 (Nov. 1997). |
Chen, Yupeng et al., “High Resolution Adaptive Bearing Estimation Using a Complex-Weighted Neural Network”, IEEE, 0-7803-0532-9/92, pp. II-317-IIII-320 (1992). |
Gabrys, Bogdan et al., “Combining labeled and unlabelled data in the design of pattern classification systems,” International Journal of Approximate Reasoning, 35, pp. 251-273 (2004). |
Gunter, Simon, “Multiple Classifier Systems in Offline Cursive Handwriting Recognition,” cover page and p. 4 (2004). |
Hawkins, Simon et al., “Outlier Detection Using Replicator Neural Networks,” Proceedings of the Fifth International Conference and Data Warehousing and Knowledge Discovery, 10 pp. (2002). |
Hinterseer, Kris, “The Wolfsberg Anti-Money Laundering Principles,” Journal of Money Laundering Control, vol. 5, No. 1 pp. 25-41 (Summer 2001). |
Hodge, Victoria J. et al., “A Survey of Outlier Detection Methodologies,” Artificial Intelligence Review, vol. 22, pp. 85-126 (2004). |
Lazarevic, Aleksandar et al., “Feature Bagging for Outlier Detection,” Research Track Paper, pp. 157-166. |
Mehrotra, Kishan et al., “Elements of Artificial Neural Networks”, MIT Press, pp. 11, 25, 71 76, 85-87, 157, 170-171 (1997). |
Miyazaki, Anthony D. et al., “Internet Privacy and Security: An Examination of Online Retailer Disclosures,” Journal of Public Policy & Marketing, vol. 19, No. 1, pp. 54-61 (Spring 2000). |
Pelleg, Dan et al., “Active Learning for Anomaly and Rare-Category Detection,” School of Computer Science, Carnegie-Mellon University, 8 pp. |
Tsai, Chieh-Yuan, et al., “A Dynamic Web Service based Data Mining Process System”, Proceedings of the 2005 The Fifth International Conference on Computer and Information Technology, pp. 1033-1039 (2005). |
West, David et al., “Neural network ensemble strategies for financial decision applications,” Computers & Operations Research, 32, pp. 2543-2559 (2005). |
Williams, Graham et al., “A Comparative Study of RNN for Outlier Detection in Data Mining,” Proceedings of the 2002 IEEE International Conference on Data Mining 12 pp. (2002). |
Wong, A.S.Y. et al., “A Unified Sequential Method for PCA”, IEEE, pp. 583-586 (1999). |
Zhang, Tiezhu et al., “Study on the Application of Dynamic Balanced Scorecard in the Service Industry”, 2008 International Conference on Intelligent Computation Technology and Automation, Digital Object Identifier: 10.1109/ICICTA.2008.359, pp. 1158-1162 (2008). |
Zhu, Xiaojin, “Semi-Supervised Learning with Graphs,” Language Technologies Institute, School of Computer Science, Carnegie Mellon University, Doctoral Thesis, pp. i-x and 1-164 (May 2005). |
Credit Technologies, Inc.: News and Press Center, http://www.credittechnologies.com/news—center.asp, 19 pp. |
Number | Date | Country | |
---|---|---|---|
20130103617 A1 | Apr 2013 | US |