Patent Application
20190108176
The present invention is generally related to business intelligence systems and more specifically business intelligence systems with geographic data.
The term “business intelligence” is commonly used to refer to techniques for identifying, processing, and analyzing business data. Business intelligence systems can provide historical, current, and predictive views of business operations. Business data, generated during the course of business operations, including data generated from business processes and the additional data created by employees and customers, may be structured, semi-structured, or unstructured depending on the context and knowledge surrounding the data. In many cases, data generated from business processes is structured, whereas data generated from customer interactions with the business is semi-structured or unstructured. Due to the amount of data generally generated during the course of business operations, business intelligence systems are commonly built on top of and/or utilize a data warehouse.
Data warehouses are utilized to store, analyze, and report data such as business data. Data warehouses utilize databases to store, analyze, and harness the data in a productive and cost-effective manner. A variety of databases are commonly utilized including a relational database management system (RDBMS), such as the Oracle Database from the Oracle Corporation of Santa Clara, Calif., or a massively parallel processing analytical database, such as Teradata from the Teradata Corporation of Miamisburg, Ohio. Business intelligence (BI) and analytical tools, such as SAS from SAS Institute, Inc. of Cary, N.C., are used to access the data stored in the database and provide an interface for developers to generate reports, manage and mine the stored data, perform statistical analysis, business planning, forecasting, and other business functions. Most reports created using BI tools are created by database administrators and/or business intelligence specialists, and the underlying database may be tuned for the expected access patterns. A database administrator may index, pre-aggregate or restrict access to specific relations, allow ad-hoc reporting and exploration.
A snowflake schema is an arrangement of tables in a RDBMS, with a central fact table connected to one or more dimension tables. The dimension tables in a snowflake schema are normalized into multiple related tables—for a complex schema there will be many relationships between the dimension tables, resulting in a schema that looks like a snowflake. A star schema is a specific form of a snowflake schema having a fact table referencing one or more dimension tables. However, in a star schema, the dimensions are normalized into a single table—the fact table is the center and the dimension tables are the “points” of the star.
Online transaction processing (OLTP) systems are designed to facilitate and manage transaction-based applications. OTLP may refer to a variety of transactions such a database management system transactions, business, or commercial transactions. OLTP systems typically have low latency response to user requests.
Online analytical processing (OLAP) is an approach to answering multidimensional analytical queries. OLAP tools enable users to analyze multidimensional data utilizing three basic analytical operations: consolidation (aggregating data), drill-down (navigating details of data), and slice and dice (take specific sets of data and view from multiple viewpoints). The basis for many OLAP systems is an OLAP cube. An OLAP cube is a data structure allowing for fast analysis of data with the capability of manipulating and analyzing data from multiple perspectives. OLAP cubes are typically composed of numeric facts, called measures, categorized by dimensions. These facts and measures are commonly created from a star schema or a snowflake schema of tables in a RDBMS.
Systems and methods for interest-driven business intelligence systems including geo-spatial data in accordance with embodiments of the invention are illustrated. An interest-driven business intelligence system including raw data storage configured to contain raw data and perform extract, transform, and load (ETL) processes, a data mart configured to contain metadata that describes the raw data, and an intermediate processing layer, wherein the intermediate processing layer is configured to automatically generate metadata describing the raw data, derive reporting data requirements from at least one report specification based on the metadata, and compile an interest-driven data pipeline based upon the reporting data requirements, where compiling the interest-driven data pipeline includes generating ETL processing jobs to generate geo-spatial data from the raw data by filtering the raw data based on the metadata describing the raw data, determining bounding data based on the metadata describing the raw data, bounding the filtered raw data based on the bounding data, generating geo-spatial data based on the bounded filtered raw data, and storing the geo-spatial data in the data mart, generating reporting data including data satisfying the reporting data requirements based on the geo-spatial data, and storing the reporting data in the data mart for exploration by an interest-driven data visualization system.
In an additional embodiment of the invention, the raw data includes unstructured data.
In another embodiment of the invention, the raw data storage is a data warehouse.
In yet another additional embodiment of the invention, the data warehouse is implemented utilizing a system selected from the group consisting of a distributed computing system, a database management system, and a NoSQL database.
In still another additional embodiment of the invention, the data warehouse is a distributed computing system implemented utilizing Hadoop.
In yet still another additional embodiment of the invention, the data warehouse is configured to store data generated utilizing the intermediate processing layer.
In yet another embodiment of the invention, the intermediate processing layer is configured to generate data warehouse requests.
In still another embodiment of the invention, the data warehouse requests include MapReduce operations.
In yet still another embodiment of the invention, the intermediate processing layer is implemented utilizing a system selected from the group consisting of a distributed computing system, a database management system, and a NoSQL database system.
In yet another additional embodiment of the invention, the system further includes an interest-driven data visualization system wherein the interest-driven data visualization system is configured to receive metadata describing the raw data from the intermediate processing layer and generate a user interface enabling user exploration of the metadata to define at least one report specification, where the user exploration involves selection of additional reporting data based on the metadata.
In still another additional embodiment of the invention, the interest-driven data visualization system is configured to display an indication based upon the interactive exploration of the at least one report.
In yet still another additional embodiment of the invention, the data mart is further configured to contain aggregate data, where aggregate data includes structured data generated using ETL processes from the raw data.
In yet another embodiment of the invention, compiling the interest-driven pipeline further includes generating ETL processing jobs to generate aggregate data from the raw data by filtering the raw data based on the metadata describing the raw data, applying transformations to the raw data based on the metadata describing the raw data, generating aggregate data based on the transformed data, and storing the aggregate data in the data mart.
In still another embodiment of the invention, compiling the interest-driven pipeline further includes generating ETL processing jobs to generate aggregate data from geo-spatial data by identifying at least one dimension within a piece of geo-spatial data, obtaining raw data corresponding to the identified at least one dimension, applying transformations to the obtained raw data based on the metadata describing the obtained raw data, generating aggregate data based on the transformed data, and storing the aggregate data in the data mart.
In yet still another embodiment of the invention, compiling the interest-driven pipeline further includes generating ETL processing jobs to generate geo-spatial data from aggregate data by identifying at least one dimension within a piece of aggregate data, obtaining raw data corresponding to the identified at least one dimension, filtering the obtained raw data based on the metadata describing the obtained raw data, determining bounding data based on the metadata describing the obtained raw data, bounding the filtered obtained raw data based on the bounding data, generating geo-spatial data based on the bounded data, and storing the geo-spatial data in the data mart.
In yet another additional embodiment of the invention, the data mart is contained within the intermediate processing layer.
In still another additional embodiment of the invention, the bounding data includes at least one dimension identified in the metadata describing the raw data.
In yet still another additional embodiment of the invention, bounding the filtered raw data based on the bounding data includes determining pieces of raw data that identifying a location that is within the location described by the bounding data.
In yet another embodiment of the invention, bounding the filtered raw data based on the bounding data further includes mapping at least two dimensions within the filtered raw data to a common set of data and the common set of data is based on the bounding data.
In still another embodiment of the invention, the bounding data includes a latitude and a longitude.
In yet still another embodiment of the invention, the bounding data includes a location.
In yet another additional embodiment of the invention, the bounding data includes a bounding region described relative to the location.
Still another embodiment of the invention includes a method for creating a report utilizing an interest-driven business intelligence system including storing raw data in raw data storage using an interest-driven business intelligence system, where the raw data includes unstructured data, generating metadata describing the raw data using the interest-driven business intelligence system, receiving report definitions generated utilizing the metadata describing the raw data using the interest-driven business intelligence system, automatically generating reporting data requirements derived from report definitions using the interest-driven business intelligence system, automatically compiling an interest-driven data pipeline that is part of the interest-driven business intelligence system using the reporting data requirements and the raw data, obtaining geo-spatial data from the interest-driven data pipeline based on the generated reporting data requirements using the interest-driven business intelligence system, where the geo-spatial data includes a set of dimensions and the geo-spatial data is bounded based on at least one dimension in the set of dimensions, generating reporting data from the reporting data requirements using the interest-driven data pipeline, where the reporting data is based on a portion of the geo-spatial data, and storing the generated reporting data in a data mart within the interest-driven business intelligence system.
Turning now to the drawings, interest-driven business intelligence systems configured to utilize geo-spatial data in accordance with embodiments of the invention are illustrated. Interest-driven business intelligence systems include interest-driven business intelligence server systems configured to create reporting data using raw data retrieved from distributed computing platforms. The interest-driven business intelligence server systems are configured to dynamically compile interest-driven data pipelines to provide analysts with information of interest from the distributed computing platform. The interest-driven business intelligence server system has the ability to dynamically reconfigure the interest-driven data pipeline to provide access to desired information stored in the distributed computing platform. An interest-driven data pipeline is dynamically compiled to create reporting data based on reporting data requirements determined by analysts within the interest-driven business intelligence system. Changes specified at the report level can be automatically compiled and traced backward by the interest-driven business intelligence server system to compile an appropriate interest-driven data pipeline to meet the new and/or updated reporting data requirements. Interest-driven business intelligence server systems further build metadata concerning the data available in the interest-driven business intelligence system and provide the metadata to interest-driven data visualization systems to enable the construction of reports using the metadata. In this way, interest-driven business intelligence server systems are capable of managing huge datasets in a way that provides analysts with complete visibility into the available data. Available data within an interest-driven business intelligence system includes, but is not limited to, raw data, aggregate data, filtered data, and reporting data. Interest-driven business intelligence systems and interest-driven business intelligence server systems that can be utilized in accordance with embodiments of the invention are discussed further in U.S. Pat. No. 8,447,721, titled “Interest-Driven Business Intelligence Systems and Methods of Data Analysis Using Interest-Driven Data Pipelines” and issued May 21, 2013, the entirety of which is incorporated herein by reference.
In many embodiments, the reports are created using interest-driven data visualization systems configured to request and receive data from an interest-driven business intelligence server system. Systems and methods for interest-driven data visualization that can be utilized in accordance with embodiments are described in U.S. patent application Ser. No. 13/791,028, titled “Systems and Methods for Interest-Driven Data Visualization Systems Utilized in Interest-Driven Business Intelligence Systems” and filed Mar. 8, 2013, the entirety of which is hereby incorporated by reference. In order for an interest-driven data visualization system to build reports, a set of reporting data requirements are defined. These requirements specify the reporting data (derived from raw data) that will be utilized to generate the reports. The raw data can be structured, semi-structured, or unstructured. In a variety of embodiments, structured and semi-structured data include metadata, such as an index or other relationships, describing the data; unstructured data lacks any definitional structure. An interest-driven business intelligence server system can utilize reporting data already created by the interest-driven business intelligence server systems and/or cause new and/or updated reporting data to be generated to satisfy the reporting data requirements. In a variety of embodiments, reporting data requirements are obtained from interest-driven data visualization systems based on reporting requirements defined by analysts exploring metadata describing raw data stored in the interest-driven business intelligence system. In many embodiments, reports utilized in interest-driven data visualization systems include a set of datasets determined using reporting data received from an interest-driven business intelligence server system and a set of visualizations. Interest-driven data visualization systems are configured to enable the dynamic association of datasets to visualizations to provide a variety of interactive reports describing the data. In a number of embodiments, multiple datasets within a piece of reporting data (or multiple pieces of reporting data) can be visualized within a single visualization by utilizing a trellised visualization. A trellised visualization includes a plurality of visualizations. In several embodiments, at least one of these visualizations is designated as the master visualization and zero or more slave visualizations can be associated with the master visualization(s). Based on the relationships between the master visualizations and the slave visualizations, interactions with the master visualization(s) are mapped to the slave visualizations. In this way, the slave visualizations can be interacted with in concert with the corresponding master visualizations. Each of the visualizations within the trellised visualization is displayed simultaneously by the interest-driven data visualization system. Systems and methods for interest-driven data visualizations configured to generate trellised visualizations that can be utilized in accordance with embodiments of the invention are disclosed in U.S. patent application Ser. No. 14/140,211, titled “Systems and Methods for Interest-Driven Data Visualization Systems Utilizing Visualization Image Data and Trellised Visualizations” and filed Dec. 24, 2013, the entirety of which is hereby incorporated by reference.
Interest-driven business intelligence server systems are configured to provide reporting data based on one or more reporting data requirements. Reporting data provided by interest-driven business intelligence server systems includes raw data, aggregate data, event-oriented data, geo-spatial data, and/or filtered (e.g. projected) data loaded from raw data storage that has been processed and loaded into a data structure to provide rapid access to the data. It should be noted that any transformation of data loaded from raw data storage can be utilized as appropriate to the requirements of specific embodiments of the invention. In several embodiments, reporting data derived from aggregate data is referred to as aggregate reporting data; similarly, reporting data derived from geo-spatial data can be referred to as geo-spatial reporting data. Event-oriented data includes sets of data aligned along one or more of the dimensions of (e.g. columns of data within) the sets of data. Sets of data include, but are not limited to, fact tables and dimension tables as appropriate to the requirements of specific applications in accordance with embodiments of the invention. In this way, event-oriented data can include a variety of data across multiple sets of data that are organized by ordering data. Systems and methods for business intelligence systems including event-oriented data that can be utilized in accordance with embodiments of the invention are described in U.S. patent application Ser. No. 14/198,039, titled “Systems and Methods for Interest-Driven Business Intelligence Systems Including Event-Oriented Data” and filed Mar. 5, 2014, the entirety of which is hereby incorporated by reference.
Business intelligence systems, including interest-driven business intelligence systems in accordance with embodiments of the invention can be configured to provide segment data that can be explored using interest-driven data visualization systems. In a variety of embodiments, segment data includes data grouped by one or more pieces of segment grouping data. This segment grouping data can be utilized in the exploration of the segment data to quickly identify patterns of interest within the data. The data utilized within the segment data can be sourced from a variety of pieces of data, including source data, aggregate data, event-oriented data, geo-spatial data, and reporting data as appropriate to the requirements of specific applications in accordance with embodiments of the invention. Additionally, multiple segments can be combined together in order to explore patterns existing across multiple segments for one or more pieces of reporting data. Based on patterns identified within the (combined) segment data, specific pieces of reporting data can be generated targeting the identified patterns within the segment data. This reporting data can then be utilized to generate detailed reports for additional analysis and exploration of the patterns located within the (combined) segment data. In a variety of embodiments, metadata describing the (combined) segment data can be stored and utilized to generate updated segment data. This updated segment data can be utilized to further analyze patterns occurring within the reporting data as the underlying reporting data changes. Systems and methods for interest-driven business intelligence systems configured to utilize segment data that can be utilized in accordance with embodiments of the invention are described in U.S. patent application Ser. No. 14/197,150, titled “Systems and Methods for Interest-Driven Business Intelligence Systems including Segment Data” and filed Mar. 5, 2014, the entirety of which is hereby incorporated by reference.
In a number of embodiments, the raw data, aggregate data, event-oriented data, geo-spatial data, and/or filtered data is provided to interest-driven business intelligence server systems as source data. In many embodiments, the source data is described by metadata describing the raw data, aggregate data, event-oriented data, geo-spatial data, and/or filtered data present in the source data. In several embodiments, the source data, aggregate data, event-oriented data, geo-spatial data, and/or reporting data is stored in a data mart or other aggregate data storage associated with the interest-driven business intelligence server system. Interest-driven business intelligence server systems can load source data into a variety of reporting data structures in accordance with a number of embodiments, including, but not limited to, online analytical processing (OLAP) cubes. In a variety of embodiments, the reporting data structures are defined using reporting data metadata describing a reporting data schema. In a number of embodiments, interest-driven business intelligence server systems are configured to combine requests for one or more OLAP cubes into a single request, thereby reducing the time, storage, and/or processing power utilized by the interest-driven business intelligence system in creating source data utilized to create reporting data schemas and/or the reporting data.
In many embodiments, geo-spatial data reporting data is visualized and explored using interest-driven data visualization systems to analyze trends within the regions identified within the geo-spatial data reporting data. In several embodiments, these regions are based on boundary data that defines a particular region within the reporting data. In a number of embodiments, these regions are based on binning data that approximates a region within the reporting data defined based on boundary data. Based on the data associated with the analyzed regions, reporting data requirements identifying aggregate data can be used to create jobs and generate the aggregate data corresponding to the analyzed trends. The aggregate data can then be utilized to generate aggregate reporting data that can be analyzed to gain deeper insights into the regions identified within the geo-spatial data. Similarly, aggregate reporting data can be analyzed to identify potential regions of interest that form the basis for jobs to generate geo-spatial data describing the regions. The geo-spatial data can then be utilized to generate geo-spatial reporting data utilized by interest-driven data visualization systems to analyze the regions identified within the geo-spatial reporting data. For example, the users of a video-on-demand service can have their viewing activity aggregated by the geographic region in which the users live. Based on this geographic data, the relative popularity of particular videos can be quickly analyzed to identify what videos are popular in what areas. This information can then be utilized in a variety of ways, such as providing insights regarding other videos that may be popular in one geographic region but less popular in another. In this way, particular phenomenon within the reporting data based on the geographical distribution of the reporting data can be identified.
Systems and methods for interest-driven business intelligence systems including geo-spatial data in accordance with embodiments of the invention are described below. Although many of the techniques described below are in reference to particular pieces of data being within or outside of a particular bounding area, it should be noted that bins (e.g. aggregations) of data, where a portion of the binned data is within the bounding area and a portion of the binned data is outside of the bounding area, can be included or excluded in a particular bounding area as appropriate to the requirements of specific applications in accordance with embodiments of the invention.
An interest-driven business intelligence system in accordance with an embodiment of the invention is illustrated in
In a variety of embodiments, the distributed computing platform 110 is a cluster of computing devices configured as a distributed computing platform. The distributed computing platform 110 can be configured to act as a raw data storage system and a data warehouse within the interest-driven business intelligence system. In a number of embodiments, the distributed computing platform includes a distributed file system configured to distribute the data stored within the distributed computing platform 110 across the cluster computing devices. In many embodiments, the distributed data is replicated across the computing devices within the distributed computing platform, thereby providing redundant storage of the data. The distributed computing platform 110 is configured to retrieve data from the computing devices by identifying one or more of the computing devices containing the requested data and retrieving some or all of the data from the computing devices. In a variety of embodiments where portions of a request for data are stored using different computing devices, the distributed computing platform 110 is configured to process the portions of data received from the computing devices in order to build the data obtained in response to the request for data. Any distributed file system, such as the Hadoop Distributed File System (HDFS), can be utilized as appropriate to the requirements of specific applications in accordance with embodiments of the invention.
In several embodiments, the interest-driven business intelligence server system 112 is implemented using one or a cluster of computing devices. In a variety of embodiments, alternative distributed processing systems are utilized. Raw data storage is utilized to store raw data, metadata storage is utilized to store data description metadata describing the raw data, and/or report storage is utilized to store previously generated reports including previous reporting data and previous reporting data requirements. Raw data storage, metadata storage, and/or report storage can be a portion of the memory associated with the interest-driven business intelligence server system 112, the distributed computing platform 110, and/or a separate device in accordance with the specific requirements of specific embodiments of the invention. In a variety of embodiments, the interest-driven business intelligence server system 112 and/or distributed computing platform 110 is configured to generate an index for the raw data, metadata, and/or reporting data as appropriate to the requirements of specific applications of the invention. In several embodiments, the interest-driven business intelligence server system 112 and/or distributed computing platform 110 is configured to access data directly without generating and/or referencing an index.
The interest-driven business intelligence server system 112 is configured to communicate via the network 114 with one or more interest-driven data visualization systems, including, but not limited to, mobile devices 116, personal computers 118, presentation devices 120, and tablet devices 122. In many embodiments of the invention, interest-driven data visualization systems include any computing device capable of receiving and/or displaying data. Interest-driven data visualization systems allow users to specify reports including data visualizations that enable the user to explore the raw data stored within the distributed computing platform 110 using reporting data generated by the interest-driven business intelligence server system 112. Reporting data is provided in a variety of forms, including, but not limited to, snowflake schemas and star schemas as appropriate to the requirements of specific applications in accordance with embodiments of the invention. In many embodiments, reporting data is any data that includes fields of data populated using raw data stored within the distributed computing platform 110. The reporting data requested can include aggregate reporting data, event-oriented reporting data, and/or geo-spatial reporting data as appropriate to the requirements of specific applications in accordance with embodiments of the invention.
Based on received reporting data requirements, the interest-driven business intelligence server system 112 automatically compiles one or more interest-driven data pipelines to create or update reporting data to satisfy the received reporting data requirements. The interest-driven business intelligence server system 112 is configured to compile one or more interest-driven data pipelines configured to create and push down jobs to the distributed computing platform 110 to create source data and then applying various filtering, aggregation, alignment, bounding, and/or grouping processes to the source data to produce reporting data to be transmitted to interest-driven data visualization systems. In a number of embodiments, a bounding process includes identifying bounding data based on one or more dimensions across a plurality of pieces of source data and creating geo-spatial data including data taken from the plurality of pieces of source data bounded by the bounding data. In a variety of embodiments, the dimensions and the bounding data are stored using geo-spatial data metadata describing the structure (e.g. a schema) of the geo-spatial data. The fact data corresponding to the geo-spatial data can be combined with the geo-spatial data metadata in response to the received reporting data requirements as appropriate to the requirements of specific applications in accordance with embodiments of the invention.
In many embodiments, the interest-driven business intelligence server system 112 includes reporting data, source data, event-oriented data, geo-spatial data, and/or aggregate data that partially or fully satisfy the reporting data requirements. The interest-driven business intelligence server system 112 is configured to identify the relevant existing reporting data, aggregate data, event-oriented data, geo-spatial data, and/or source data and configure an interest-driven data pipeline to create jobs requesting reporting data minimizing the redundancy between the existing data and the new reporting data requirements. In a variety of embodiments, the interest-driven business intelligence server system 112 is configured to determine redundancies between the requested data and existing data using metadata describing the data available from the distributed computing platform 110. In a number of embodiments, the metadata further describes what form the data is available in, such as, but not limited to, aggregate data, filtered data, source data, reporting data, event-oriented data, and geo-spatial data. In several embodiments, the interest-driven business intelligence server system 112 obtains a plurality of reporting data requirements and creates jobs using the interest-driven data pipeline to create source data containing data fulfilling the union of the plurality of reporting data requirements. In a variety of embodiments, the interest-driven business intelligence server system 112 is configured to identify redundant data requirements in one or more reporting data requirements and configure an interest-driven data pipeline to create jobs requesting source data fulfilling the redundant data requirements. In several embodiments, the interest-driven business intelligence server system 112 is configured to store aggregate data, event-oriented data, geo-spatial data, and/or reporting data in a data mart and utilize the stored data to identify the redundant data requirements. In a number of embodiments, the interest-driven business intelligence server system 112 is configured to identify when reporting data requirements request updated data for existing reporting data and/or source data and configure an interest-driven data pipeline to create jobs to retrieve an updated snapshot of the existing reporting data from the distributed computing platform 110.
The interest-driven business intelligence server system 112 is configured to compile an interest-driven data pipeline to create jobs to be pushed down to the distributed computing platform 110 in order to retrieve data. In a variety of embodiments, the jobs created using the interest-driven data pipeline are tailored to the reporting data requirements. In many embodiments, the jobs created using the interest-driven data pipeline are customized to the hardware resources available on the distributed computing platform 110. In a number of embodiments, the jobs are configured to dynamically reallocate the resources available on the distributed computing platform 110 in order to best execute the jobs. In several embodiments, the jobs are created using performance metrics collected based on the performance of previously executed jobs.
In several embodiments, jobs pushed down to the distributed computing platform 110 by the interest-driven business intelligence server system 112 cannot be executed in a low-latency fashion. In many embodiments, the distributed computing platform 110 is configured to provide a partial set of source data fulfilling the pushed down job and the interest-driven business intelligence server system 112 is configured to create reporting data using the partial set of source data. As more source data is provided by the distributed computing platform 110, the interest-driven business intelligence server system 112 is configured to update the created reporting data based on the received source data. In a number of embodiments, the interest-driven business intelligence server system will continue to update the reporting data until a termination condition is reached. Termination conditions can include, but are not limited to, a certain volume of source data is received, the source data provided is no longer within a particular time frame, and an amount of time to provide the source data has elapsed. In a number of embodiments, a time frame and/or the amount of time to provide the source data is determined based on the time previously measured in the retrieval of source data for similar reporting data requirements.
Although a specific architecture for an interest-driven business intelligence system in accordance with an embodiment of the invention is conceptually illustrated in
Interest-driven business intelligence server systems in accordance with embodiments of the invention are configured to create jobs to request source data from interest-driven business intelligence systems based on received reporting data requirements and to create reporting data using the received source data. The reporting data can be aggregate reporting data event-oriented reporting data, and/or geo-spatial reporting data based on the received reporting data requirements. It should be noted that any data derived from the source data can be utilized as reporting data as appropriate to the requirements of specific embodiments of the invention.
An interest-driven business intelligence server system in accordance with an embodiment of the invention is conceptually illustrated in
The interest-driven business intelligence application 232 configures the processor 210 to perform an interest-driven business intelligence process. In many embodiments, an interest-driven business intelligence process includes creating jobs using an interest-driven data pipeline to retrieve source data in response to reporting data requirements. The source data can then be utilized to generate aggregate data, event-oriented data, and/or geo-spatial data as appropriate to the requirements of specific applications in accordance with embodiments of the invention. In a variety of embodiments, the created jobs are based on redundancies between reporting data requirements and existing source data 234, aggregate data 236, and/or geo-spatial data 238. In a number of embodiments, the interest-driven business intelligence process includes updating reporting data based on incrementally received source data and/or updated source data. In several embodiments, the interest-driven business intelligence process includes obtaining a request for aggregate reporting data and generating the aggregate reporting data based on one or pieces of geo-spatial data. Similarly, the interest-driven business intelligence process can also include obtaining a request for geo-spatial reporting data and generating the geo-spatial reporting data based on one or pieces of aggregate data.
Although a specific architecture for an interest-driven business intelligence server system in accordance with an embodiment of the invention is conceptually illustrated in
In several embodiments, geo-spatial data, event-oriented data, and aggregate data can both be constructed based on a number of facts and dimensions within a particular data schema. In interest-driven business intelligence systems, the data schema can describe source data, aggregate data, event-oriented data, geo-spatial data, and/or reporting data. Other data, including metadata describing raw data available from distributed computing platform, can also be utilized within the data schema to aid in the definition and population of aggregate data, event-oriented data, and/or geo-spatial data as appropriate to the requirements of specific applications in accordance with embodiments of the invention.
Turning now to
The set of data 302 can include any of a variety of data, including raw data, source data, aggregate data, event-oriented data, geo-spatial data, reporting data, and/or any other data (or combination of data) stored and processed by an interest-driven business intelligence system or an interest-driven data visualization systems as appropriate to the requirements of specific application of embodiments of the invention. In a variety of embodiments, the data includes at least one dimension related to location data. The facts associated with the dimension(s) can be stored directly and/or encoded in accordance with several embodiments of the invention. Additionally, the location data can include a single point (e.g. a latitude, a longitude, and/or an elevation), a set of points, a geographic identifier (e.g. a city, state, address, zip code, etc. . . . ), and/or any other data that can be utilized to identify a location in space as appropriate to the requirements of specific applications of the invention. The bounding area 302 is defined based on bounding data describing one or more locations within a space. Bounding data can include, but is not limited to, geographic location data that identifies one or more points in space. The geographic location data can be coordinates that identify a single point, a set of points, and/or a location within a map. Additionally, the bounding data can include range data (e.g. a radius, a census-defined area such as city limits, etc. . . . ) that identifies a region around the point(s) that is included in the bounding area. In this way, data that is proximate to a defined area and/or has a proximity relationship to a set of locations can be included within the defined bounding area. Additionally, multiple sub-bounding areas can be combined together to create a single bounding area, thereby facilitating the generation of arbitrary bounding areas.
In many embodiments, the geographic location data includes facts and/or dimensions identifying location data within the data. The facts and dimensions can include both single dimensions (e.g. a dimension that identifies fact data including encoded location data) and multiple dimensions (e.g. multiple dimensions identifying one or more facts that identify a single location and/or multiple locations) as appropriate to the requirements of specific application of the invention. In many embodiments, the bounding data identifies arbitrary groupings of data that do not necessarily correspond to a geographical location; rather, the bounding data can identify the location of the data within any arbitrary space defined that is relevant to the data being analyzed as appropriate to the requirements of specific applications of the invention. In this way, the bounding data includes one or more points that identify a location and/or a region in space.
The set of included data 306 includes those pieces of data in the set of data 302 that have facts and/or dimensions corresponding to the bounding data defining the bounding area 302. This can include pieces of data that fall within the defined bounding area, pieces of data that fall on the defined bounding area, and/or pieces of data outside of the defined bounding area. For example, the set of data 302 can be aggregated into bins of data and the bounding area utilized to identify bins that fall within or on the bounding area. In this way, a portion of the binned data may actually be outside of the defined bounding area but still be included in the bounding region. Similarly, the bounding area 302 can be utilized to identifying data outside of a particular area. However, it should be noted that any bounding of data based on the bounding data can be utilized as appropriate to the requirements of specific applications of the invention.
Geo-spatial data can be visualized in a variety of ways. Turning now to
Turning now to
Turning now to
While a specific example of geo-spatial data and visualizations of geo-spatial data are conceptually illustrated in
By analyzing data bounded by a set of bounding data, certain patterns of interest can be identified for exploration. Many of these patterns are not apparent until the data is bounded along the dimensions. Geo-spatial reporting data includes data organized in a variety of dimensions and bounded according to at least one of the dimensions.
Interest-driven data visualization systems in accordance with embodiments of the invention are configured to obtain geo-spatial reporting data. A process for obtaining geo-spatial reporting data in accordance with an embodiment of the invention is illustrated in
In many embodiments, source data is obtained (410) utilizing processes similar to those described above. In a number of embodiments, bounding data is determined (412) based on one or more dimensions contained within pieces of the obtained (410) source data. As described above, bounding data identifies one or more dimensions of the source data; in a variety of embodiments, the identified dimensions are related to geographic locations. The bounding data can be determined (412) dynamically by an interest-driven data visualization system and/or based on reporting data requirements determined during the exploration of the source data as appropriate to the requirements of specific applications in accordance with embodiments of the invention. In several embodiments, geo-spatial reporting data is requested (414) by creating geo-spatial request data identifying at least one piece of source data and the determined (412) bounding data. In a variety of embodiments, the request (414) for geo-spatial reporting data is transmitted to an interest-driven business intelligence system. In many embodiments, geo-spatial reporting data is obtained (416) from an interest-driven business intelligence system. In a number of embodiments, geo-spatial reporting data is obtained (416) based on data already present within an interest-driven data visualization system. In several embodiments, a report is generated (418) based on the obtained (416) geo-spatial reporting data.
Although a specific process for obtaining geo-spatial reporting data is described above with respect to
During the exploration of data, particular features of the data may warrant additional inspection. Many of these pieces of data provide a variety of insights into the data when analyzed in conjunction with disparate pieces of data (e.g. pieces of data not included in the data currently being explored) occurring in relation to the pieces of data. For example, data describing a users' interactions with a particular page within a website can provide useful insights regarding users' interactions (or lack thereof) within another page within the website in particular locations, i.e. geographic areas. By analyzing the usage of the pages based on location data, pages within the website that perform well (or poorly) can be identified and additional analysis can be performed with respect to those portions of the web site. This analysis can then be utilized to better target particular pages to particular locations, improving the performance of those pages with respect to a particular audience. In a variety of embodiments, the performance of the pages in a particular location can be tracked over time. As discussed above, geo-spatial data is bounded according to one or more pieces of bounding data. Interest-driven business intelligence server systems in accordance with embodiments of the invention are configured to obtain geo-spatial data by compiling an interest-driven data pipeline to obtain the necessary data bounded according to the bounding data. A process for obtaining geo-spatial data in accordance with an embodiment of the invention is illustrated in
In a variety of embodiments, geo-spatial request data is obtained (510) from an interest-driven data visualization system. Geo-spatial request data includes, but is not limited to, geo-spatial data metadata and/or requested location-based facts. In several embodiments, bounding data is identified (512) within the obtained (510) geo-spatial request data. In a number of embodiments, the generated (514) job data configures a distributed computing platform to create geo-spatial data fulfilling the geo-spatial data and bounded based on the identified (512) bounding data. In many embodiments, the generated (514) job data takes into account source data already present within an interest-driven business intelligence server system that fulfills a portion of the obtained (510) geo-spatial request data. In a variety of embodiments, the job data is transmitted (516) to a distributed computing platform utilizing techniques similar to those described above. The obtained (518) geo-spatial data includes geo-spatial data metadata describing the dimensions of the geo-spatial data and/or the facts (e.g. rows of data) corresponding to the dimensions described in the geo-spatial data metadata and/or the obtained (510) geo-spatial request data. In several embodiments, the obtained (518) geo-spatial data includes additional dimensions and/or facts not directly requested in the obtained (510) geo-spatial request data. In a number of embodiments, geo-spatial reporting data is generated (520) based on the obtained (510) geo-spatial request data and the obtained (518) geo-spatial data and bounded by the identified (512) bounding data. In several embodiments, the obtained (518) geo-spatial data is bounded based on the bounding data utilizing techniques similar to those described above.
In a variety of embodiments, the generated (514) job data and/or the obtained (518) geo-spatial data can be adjusted in order to account for differences between data sources providing the raw data utilized in the creation of the data used throughout the interest-driven business intelligence server system. Adjustments to data include, but are not limited to, accounting for encoding differences between systems and correlating varying location description information across systems. Bounding of data can be performed by shifting the data to a common format and/or by performing mappings of data to a common set of data (e.g. a common dimension). For example, with respect to location-based data, data acquired from multiple sources can all be converted to a common geographic coordinate system in order to account for different techniques utilized to identify a particular location across systems. Similarly, location-based data can be adjusted based on threshold values to account for differences in precision of the stored location data between the variety of systems providing data. Additionally, with respect to identification-based data (e.g. user IDs) that may be associated with the geo-spatial data, a variety of universal tracking information can be utilized to map identification-based data to the universal tracking information in order to account for differences between the identification-based data across the systems providing the data. In this way, users can be identified across disparate systems (and disparate portions within a system) in order to provide the ability to analyze the user's data across the systems and different geographic regions.
A specific process for obtaining geo-spatial data is described above with respect to
In many cases, analysts want to explore trends occurring in data already present in an interest-driven business intelligence system. In this way, the analysts can utilize the existing data to identify interesting features of the data with respect to particular geographic locations. Interest-driven business intelligence server systems in accordance with many embodiments of the invention are configured to determine geo-spatial data metadata based on source data already present within the interest-driven business intelligence system. A process for bounding data in accordance with an embodiment of the invention is illustrated in
In many embodiments, source data metadata describes the pieces of data available within a set of source data. In several embodiments, the source data describes one or more dimensions (e.g. columns) of data within the set of source data. In many embodiments, structured source data metadata is obtained (610) utilizing techniques similar to those described above. In a number of embodiments, bounding data is identified (612) based on the pieces of data available within the source data. In a variety of embodiments, the bounding data (614) is mapped to one or more dimensions described in the source data metadata. In this way, differences between the identified (612) bounding data and the obtained (610) source data metadata can be resolved in order to create geo-spatial data fulfilling a particular request. Additionally, the bounding data can be mapped (614) to source data utilizing processes similar to those described above. In several embodiments, geo-spatial data metadata is determined (616) based on the obtained (610) source data metadata and the identified (612) bounding data using processes similar to those described above. If applicable, the mapped (614) bounding data can also be utilized in the determination (616) of the geo-spatial data metadata. In many embodiments, geo-spatial data is generated (618) based on the geo-spatial data metadata and fact(s) (e.g. rows of data) contained within the source data described by the obtained (610) source data metadata utilizing techniques similar to those described above.
Although a specific process for bounding source data is described above with respect to
During the exploration of geo-spatial data, interesting features of the data can be identified and it is desirable to drill down into those features of the geo-spatial data. In order to support the additional analysis of geo-spatial data, aggregate reporting data can be generated in order to augment the identified features within the data. Returning to the example described above with respect to
An aggregate data reporting request includes, but is not limited to, one or more dimensions and/or facts within a piece of geo-spatial reporting data. Other data can be included in an aggregate reporting data request as appropriate to the requirements of specific applications in accordance with embodiments of the invention. In a variety of embodiments, an aggregate reporting data request is obtained (710) from an interest-driven data visualization system. In several embodiments, the obtained (710) aggregate reporting data requests identifies dimensions and/or facts identified during the exploration of at least one piece of geo-spatial reporting data and/or at least one piece of aggregate reporting data. In a number of embodiments, target data is identified (712) based on the dimensions and/or facts identified in the obtained (710) reporting data request. In a variety of embodiments, the target data is identified (712) based on source data metadata describing the dimensions of data available within source data stored using an interest-driven business intelligence server system. In a number of embodiments, the source data metadata describes facts and/or dimensions that can be obtained from a distributed computing platform. In several embodiments, aggregate reporting data metadata is determined (714) based on the obtained (710) aggregate reporting data request. In a variety of embodiments, aggregate reporting data metadata is determined (714) based on the identified (712) target data. As described above, aggregate reporting data is generated (716) by associating a plurality of facts (e.g. rows of data) that are obtained based on the determined (714) aggregate reporting data metadata. Generating (716) aggregate reporting data can include filtering and/or aggregating source data, event-oriented data, geo-spatial data, and/or aggregate data utilizing processes similar to those described above as appropriate to the requirements of specific applications in accordance with embodiments of the invention. In a number of embodiments, the generated (716) aggregate reporting data is transmitted (718) to one or more interest-driven data visualization systems utilizing techniques similar to those described above.
Specific processes for generating aggregate reporting data are described above with respect to
During the exploration of aggregate data, interesting trends within the data can be identified and it is desirable to drill down into those trends. In order to support the additional analysis of the aggregate data, geo-spatial reporting data can be generated in order to bring in additional context regarding the data being explored for a particular location or locations. Returning now to the example described with respect to
A geo-spatial data reporting request includes, but is not limited to, one or more dimensions and/or facts within a piece of geo-spatial reporting data along with bounding data identifying at least one of the dimensions used to bound the data. Other data can be included in a geo-spatial reporting data request as appropriate to the requirements of specific applications in accordance with embodiments of the invention. In a variety of embodiments, a geo-spatial reporting data request is obtained (760) from an interest-driven data visualization system. In several embodiments, the obtained (760) geo-spatial reporting data request identifies dimensions and/or facts identified during the exploration of at least one piece of geo-spatial reporting data and/or aggregate reporting data. In a number of embodiments, target data is identified (762) utilizing processes similar to those described above. The determined (764) geo-spatial reporting data metadata can include one or more pieces of bounding data. In several embodiments, geo-spatial reporting data metadata is determined (764) based on the obtained (760) geo-spatial reporting data request. In a variety of embodiments, geo-spatial reporting data metadata is determined (764) based on the identified (762) target data. As described above, geo-spatial reporting data is generated (766) by associating a plurality of facts (e.g. rows of data) that are obtained based on the determined (764) geo-spatial reporting data metadata. The generated (766) geo-spatial reporting data includes bounding the plurality of facts within the geo-spatial reporting data according to the bounding data. Generating (766) geo-spatial reporting data can include filtering and/or aggregating source data, geo-spatial data, event-oriented data, and/or aggregate data utilizing processes similar to those described above as appropriate to the requirements of specific applications in accordance with embodiments of the invention. In a number of embodiments, the generated (766) geo-spatial reporting data is transmitted (768) to one or more interest-driven data visualization systems utilizing techniques similar to those described above.
Although a specific process for generating geo-spatial reporting data is described above with respect to
As described above, trends are often identified across multiple geographic locations described within the data being explored. As part of the analysis of the identified trends, it is often beneficial to continue monitoring the identified trends over a period of time. Interest-driven business intelligence server systems in accordance with embodiments of the invention are configured to store geo-spatial data metadata and provide updated geo-spatial data on request. A process for updating geo-spatial data in accordance with an embodiment of the invention is illustrated in
In a number of embodiments, a geo-spatial data request is obtained (810) and bounding data is identified (812) utilizing processes similar to those described above. In many embodiments, generating (814) geo-spatial data metadata is based on the geo-spatial data request and/or the bounding data using processes similar to those described above. The geo-spatial data metadata can be stored (816) so that generating (818) geo-spatial data can be performed in response to a request for the geo-spatial data. The geo-spatial data metadata can be stored using a data mart, an interest-driven data visualization system, an interest-driven business intelligence server system, and/or a distributed computing platform as appropriate to the requirements of specific applications in accordance with embodiments of the invention.
In a variety of embodiments, geo-spatial data is generated (818) using the geo-spatial metadata and source data stored in an interest-driven business intelligence system utilizing processes similar to those described above. In many embodiments generating (818) geo-spatial data includes identifying redundant information already present within an interest-driven business intelligence server system and/or a data mart. In a variety of embodiments, redundant information is identified using reporting data schemas defining the structure of reporting data and/or source data schemas defining the structure of source data. In many embodiments, redundant information is determined using one or more files stored using the interest-driven business intelligence server system; redundancies can be determined using the data contained in the files and/or metadata associated with the files, including the creation date of the files and/or the last modified date of the files. In several embodiments, job data can be created and executed in order to generate (818) geo-spatial data from a distributed computing platform that is not present in the redundant information. Based on the identified redundant information, the job data can be targeted to minimize the amount of duplicate data retrieved from a distributed computing platform. This allows the resources of the distributed computing platform and/or the interest-driven business intelligence server system to be more efficiently used than if the redundant information not been identified.
A specific process for updating geo-spatial data is described above with respect to
As described above, trends are often identified across multiple geographic locations described within source data. In many embodiments, these trends occur not only with respect to the geographic location of the trend, but also with respect to one or more pieces of ordering data. For example, fads can be localized both to a particular geographic location and a particular point in time. By analyzing the fad across time, the spread of the fad across geographic locations can be tracked and additional insights into the fad can be determined. Geo-spatial data generated utilizing techniques similar to those described above can be grouped by identifying ordering data based on one or more dimensions across the pieces of geo-spatial data and creating ordered geo-spatial data including data taken from the plurality of pieces of geo-spatial data aligned (e.g. grouped by or ordered by) according to the ordering data. In many embodiments, the ordering data includes one or more dimensions contained within the fact tables and/or dimension tables associated with the geo-spatial data. In several embodiments, the dimensions included in the ordering data are determined based on reporting data requirements. By combining the ordering data and the bounding with facts (e.g. rows of data) stored within each of the fact tables and/or dimension tables, aligning the data according to the ordering data, and bounding the data according to the bounding data, ordered geo-spatial (reporting) data can be generated. Similarly, event-oriented data can be bounded using bounding data to generate the ordered geo-spatial data as appropriate to the requirements of specific applications in accordance with embodiments of the invention.
A process for generating ordered geo-spatial data in accordance with an embodiment of the invention is shown in
Although the present invention has been described in certain specific aspects, many additional modifications and variations would be apparent to those skilled in the art. In particular, any of the various processes described above can be performed in alternative sequences and/or in parallel (on different computing devices) in order to achieve similar results in a manner that is more appropriate to the requirements of a specific application. It is therefore to be understood that the present invention can be practiced otherwise than specifically described without departing from the scope and spirit of the present invention. Thus, embodiments of the present invention should be considered in all respects as illustrative and not restrictive. Accordingly, the scope of the invention should be determined not by the embodiments illustrated, but by the appended claims and their equivalents.
The current application is a continuation of co-pending U.S. patent application Ser. No. 14/313,191, entitled SYSTEMS AND METHODS FOR INTEREST-DRIVEN BUSINESS INTELLIGENCE SYSTEMS INCLUDING GEO-SPATIAL DATA, filed Jun. 24, 2014, which claims priority to U.S. Provisional Patent Application No. 61/894,186, entitled SYSTEMS AND METHODS FOR INTEREST-DRIVEN BUSINESS INTELLIGENCE SYSTEMS INCLUDING EVENT-ORIENTED DATA, filed Oct. 22, 2013, and to U.S. Provisional Patent Application No. 61/880,051, entitled SYSTEMS AND METHODS FOR EVENT ORIENTED LENSES IN INTEREST-DRIVEN BUSINESS INTELLIGENCE SYSTEMS, filed Sep. 19, 2013, all of which are hereby incorporated by reference in their entirety.
| Number | Date | Country | |
|---|---|---|---|
| 61894186 | Oct 2013 | US | |
| 61880051 | Sep 2013 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 14313191 | Jun 2014 | US |
| Child | 16169881 | US |
