Patent Application
20190171546
The inventive concepts described herein relate to software applications for mobile computing devices. In particular, the inventive concepts relate to mobile application analytic software that monitors, tests and/or manages software applications for mobile devices.
Mobile Application Analytics (MAA) is a class of software applications that enable application developers and others to monitor the performance of mobile applications, i.e., applications developed for mobile devices, during runtime. MAA applications can provide visibility into the operation of mobile applications and produce data that can help enhance and increase usage of the application, to respond to crash reports, and/or to identify new features for the application. MAA can enable developers to visualize and analyze the user's experience and to capture and investigate issues by providing real-world customer usage data to application development teams.
Accordingly, MAA enables application developers to understand the mobile application user experience across an application development life cycle, which can stimulate collaboration between business analysts, developers, operations and support in order to accelerate mobile application delivery, improve mobile application support and increase the value delivered via the mobile end-user experience.
Using MAA, business analysts can work together with developers to strengthen user loyalty by instrumenting mobile applications to collect critical business, performance and health data across languages, methodologies and platforms. IT operations teams can investigate where issues reside across the end-to-end mobile application infrastructure. IT operations teams and support teams can then feed that real-user data back to the application development teams to improve the next version of the application, which can enhance the user experience. MAA can also provide a better understanding of the usage of the application by geography, carrier, and device.
Mobile applications are typically distributed in an application package containing the elements needed to run the application, such as the program code, resources, certificates and a manifest.
An instrumenting application, such as an MAA application, can be attached to the mobile application by modifying the program code of the application to include the MAA code. This modification is referred to as “application wrapping.” When an application is wrapped, a layer of code is added to the application binary file to add features or modify behavior of the application without making changes to the internal application code. In addition to MAA functions, wrapping code may also be used to reduce the risk to an enterprise of improper or unauthorized use of an application. For example, wrapping can add security and management features to an application before it is deployed to the enterprise.
An MAA application may generate screenshots of the mobile application in order to document user activity for crash analysis, usage analysis, or other purposes. The screen shots may be provided by the MAA application to a developer, who can analyze the screen shots to determine what screens the user visited and what actions the user took on the screens. However, generating, storing and transmitting screen shots may consume an undesirably large amount of resources of a mobile computing device, including battery power, processing cycles and storage.
Some embodiments of the inventive concepts provide methods that include receiving application source code of a mobile application, analyzing the application source code to generate screen metadata that represents screen images that will be generated by the mobile application upon execution of binary application code generated from the application source code, storing the screen metadata in a screen metadata file, and associating the screen metadata file with the mobile application.
Some further embodiments of the inventive concepts provide methods that include analyzing operation of a mobile application during runtime execution of the mobile application, identifying a reportable runtime event during operation of the mobile application, and in response to identifying the reportable runtime event, generating screen metadata that represents a screen image generated by the mobile application and associated with the reportable runtime event, generating a runtime event report, wherein the runtime event report includes the screen metadata, and transmitting the runtime event report to an application analysis computer.
Some further embodiments of the inventive concepts provide methods that include receiving a runtime event report detailing a runtime event of a mobile application, the runtime event report including screen metadata that describes a screen image associated with the runtime event, and generating a simulated screen image of the mobile application associated with the runtime event from the screen metadata.
Some further embodiments of the inventive concepts provide methods that include receiving binary executable code of a mobile application, executing the binary executable code to generate screen images, storing the screen images in a screen image repository, and associating the screen images with the mobile application.
Some further embodiments of the inventive concepts provide methods that include receiving a runtime event report detailing a runtime event of a mobile application, the runtime event report including a screen ID associated with a screen image generated by the mobile application when the runtime event occurred, and retrieving a stored screen image associated with the screen ID from a screen image repository.
Other methods, computer program products, and/or electronic devices according to embodiments of the present disclosure will be or become apparent to one with skill in the art upon review of the following drawings and detailed description. It is intended that all such methods, computer program products, and/or electronic devices be included within this description, be within the scope of the present inventive subject matter, and be protected by the accompanying claims. Moreover, it is intended that all embodiments disclosed herein can be implemented separately or combined in any way and/or combination.
Other features of embodiments will be more readily understood from the following detailed description of specific embodiments thereof when read in conjunction with the accompanying drawings, in which:
In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of embodiments of the present disclosure. However, it will be understood by those skilled in the art that the present invention may be practiced without these specific details. In other instances, well-known methods, procedures, components and circuits have not been described in detail so as not to obscure the present invention. It is intended that all embodiments disclosed herein can be implemented separately or combined in any way and/or combination.
Referring to
The SDK 202 can include a library of development tools 204 that provide various different functionalities. The development tools 204 can include a source code editor with an integrated debugger, along with code snippets, data structures, protocols, routines, and the like. The SDK 202 can include an application programming interface (API) 206 that enables an application to interface with the development tools 204. The SDK 202 can include a compiler 208 for converting source code into machine-readable object code. It will be appreciated that the SDK 202 can include other features in addition to these, and need not include all of the illustrated features.
The development tools 204 can include tools for wrapping a mobile device software application. For example, in some embodiments, the SDK 202 includes a development tool 204 that allows an application developer to wrap an application with an MAA application that monitors the operation of the mobile application. The MAA application can perform functions such as application fault detection, application performance measurement, and the detection of reportable events, such as device crashes, hangs and freezes, during the operation of the mobile application. The MAA functionality can include the logging of detected faults, performance measurements, related events, event times, event locations, and other data within a local storage of a mobile device. The MAA application may report such data to a network resource, such as an application analysis computer 190 (
The development platform 200 may receive as inputs binary MAA application wrapper logic 72, application wrapper libraries 74, binary mobile application code 76, mobile application source code 82 and application resources 84. It will be appreciated that the binary MAA application wrapper logic 72 and the application wrapper libraries 74 may be developed and compiled on the development platform 200, or may be developed and compiled externally to the development platform 200. The binary MAA application wrapper logic 72 is the binary code of the MAA application 50. The application wrapper libraries 74 are software libraries that support the function of the MAA application code. The binary application code 76 is the executable code of the mobile application, while the application source code 82 is the source code of the mobile application.
In some embodiments, the development platform 200 may compile and link the application source code 82 to generate the binary application code 76.
In addition to the SDK 202, the development platform 200 may include a number of other functional modules, including a screen metadata generator 210, an application wrapping module 212, and a package generator 214. It will be appreciated that the screen metadata generator 210, the application wrapping module 212, and the package generator 214 may be provided as part of or separately from the SDK 202.
The application wrapping module generates 212 a wrapped mobile application by linking the binary application code 76 of the mobile application with the binary MAA application wrapper logic 72 along with the wrapper libraries 72 and any other included libraries to form binary application code 92 and binary MAA wrapper logic 96.
The package generator 214 generates an application installation package 230 for installing the mobile application on a mobile device. The application installation package 230 may include the binary application code 92 of the mobile application, the binary MAA wrapper logic 96 of the MAA application, and a screen metadata file 94, described in more detail below, along with application resources 84 and a manifest file 98 that contains a manifest of the application installation package. The manifest file 98 may be an XML file.
During the installation process, the application package 230 is provided to a mobile device 100, which extracts and installs the various components of the application package 230 according to instructions contained in the manifest 98. After installation, the mobile device 100 includes a wrapped application 80 that includes both the binary application code 92 of the mobile application along with the binary wrapper logic 96 of the MAA application. The wrapped application 80 also include the application resources 84 and the screen metadata file 94.
In some embodiments, the screen metadata generator 210 analyzes the application source code 82 of the mobile application and responsively generates data representing or describing screen images that will be displayed on the mobile device 100 by the mobile application 80 upon its execution. The screen metadata may, for example, include metadata that characterizes or describes the screen images generated by the mobile application at runtime. For example, screen metadata may describe various elements, such as images, controls, links, widgets, text boxes, labels, etc., that are displayed on the screen. The screen metadata may describe various attributes of the elements, such as the location, color, brightness, opacity, etc., of the elements. Thus, the screen metadata may include data that can be used to recreate the basic appearance of the screen, without requiring access to an actual image of the screen. The screen metadata may enable the basic appearance of the screen to be saved, transmitted and reproduced with significantly less data/bandwidth than would be required to save/transmit a screen shot or screen image.
The screen metadata generator 210 may store the screen metadata in a screen metadata file 94 and/or in a screen data repository 195. The screen metadata generator 210 may associate the screen metadata file 94 with the mobile application. In particular, the screen metadata generator 210 may associate the screen metadata file 94 with information identifying a particular version of the mobile application and/or a particular operating platform for which the application package 230 was generated.
In some embodiments, the screen metadata file 94 may be created by generating a storyboard file that provides a schematic representation of the screen images that will be generated by the mobile application upon execution of compiled code generated from the application source code. The storyboard file may be generated by parsing a NIB or XIB file and building a site map of the screen images generated by the mobile application. A NIB or XIB file may be one single file that shows all of the screens generated by the application, and showing transitions or flows between screens.
The site map may further include controls associated with the screen images generated by the mobile application. For example, the site map may include widgets, controls, switches, buttons, text boxes, check boxes, etc., that are displayed on the screen generated by the mobile application. The SDK 202 may include built-in support for parsing NIB and XIB files to obtain the various controls, widgets, etc., as well as for capturing the screen-by screen flow of the application. This information can be stored by the development platform 200 as screen metadata associated with particular screens of the application.
A model of the screen image generated by the mobile application can be recreated from data in the screen metadata file. As will be explained in more detail below, when a crash, hang, exception or other reportable event occurs during execution of the application program, the MAA application wrapper may retrieve screen metadata from the screen metadata file that is associated with the screen image that was being displayed by the mobile application when the reportable event occurred. Instead of transmitting a (relatively large) screen shot or series of screen shots to an application analysis computer in connection with the reportable event, the MAA application wrapper may transmit screen metadata describing the screen image, which may require the transmission of only a relatively small amount of data compared to the amount of image data that a screen shot comprises.
In other embodiments, the screen metadata generator 210 may generate screen metadata by loading the binary application code 76 into a virtual machine and running the binary application code. One or more screen shots of screens displayed by the mobile application can be captured showing screen layouts during various points of operation of the mobile application. The screen shots can be saved in the screen data repository 195. Each of the screen shots may be associated with a particular state of the application program 20, and may be identified by a screen ID, that may be stored in the screen metadata file 94. An application program may display a particular screen depending on the operational mode of the application program. For example, an application may display a loading screen, a landing or startup screen, a primary screen, an options screen, a login screen, a user preferences screen, etc., depending on the operational mode of the mobile application. Each screen may be saved as a screen shot in the screen data repository 195, and each screen may have an associated screen ID. For example, the loading screen, landing screen, primary screen, options screen, login screen, and user preferences screen may be associated with the screen IDs SCR000, SCR001, SCR002, SCR003, SCR004 and SCR005. These screen IDs are saved in the screen metadata file 94, along with data identifying the associated modes of the application program. The MAA application 50 can monitor the operational mode of the mobile application program 20, and when a reportable event occurs, the MAA application can send a screen ID to the application analysis computer 195 identifying the screen associated with the reportable event.
In these embodiments, when a reportable event occurs, the MAA application can send data representing the operational mode of the mobile application at the time of the reportable event to the application analysis computer 195, and the application analysis computer can retrieve a screen shot associated with the reportable event from the screen data repository 195 based on the data representing the operational mode of the mobile application provided by the MAA application.
Still referring to
Operation of the wrapped application 80 including the application code 92 and the MAA logic 96 is illustrated in
It will be appreciated that reportable events are not limited to crashes, hangs, and the like, but can include any event that occurs during the operation of the mobile application. For example, a business analyst may define a reportable event that enables the analyst to collect business, performance and health data related to the application.
When a reportable event occurs, the MAA logic 96 may send an event report 192 to the application analysis computer 190. In a typical embodiment illustrated in
The event report 192 describes the reportable event and includes screen metadata associated with the reportable event. The screen metadata may include, for example, data describing a screen or series of screens that were displayed before, during and after the reportable event. The MAA logic 96 has access to the screen identifiers from the event stream and the registry of screen metadata that matches the screen identifiers to the reference screen images/captures.
The screen metadata may enable a developer to recreate or better understand the state of the mobile application at the time of the reportable event. For example, the screen metadata may be used to reconstruct a screen or series of screens along with user interaction data so that the developer can visually “play back” the operation of the application in the time period in which the reportable event occurred.
As described above, the screen metadata may include data, such as metadata, storyboard data or other data describing the screen images displayed by the mobile application. The screen metadata may in some embodiments include a screen ID that identifies a screen shot saved in the screen shot repository 195. In other embodiments, the screen metadata may include data regarding the state of the mobile application at the time of the reportable event and from which a corresponding screen shot can be identified. The screen metadata can include data from a storyboard file, and can include controls associated with the screen images generated by the mobile application, such as widgets, switches, text boxes, buttons, check boxes, etc., that are displayed on the screen generated by the mobile application.
Notably, however, the event report 192 may not include a screen image. As noted above, image data may require a large bandwidth to transmit, which may be particularly problematic when the event report 192 is transmitted over a wireless interface, such as via the radio access network 170. By including screen metadata in the event report 192 instead of an actual screen image, the size of the event report may be greatly reduced.
The metadata describing these screen elements may be formatted in the screen metadata 230 or screen animation metadata 232 as one or more XML files. For example, the screen metadata for the screen shown in
The metadata for the screenshot shown in
It will be appreciated that the foregoing is only provided by way of example, and that screen metadata can be formatted in many different ways, any of which can be used in various embodiments. The metadata can be interpreted by the application analysis computer 190 and used to generate simulated screen shots and/or screen animations showing the screen generated by the mobile application at the time of the reportable event, along with the user's interaction with the screen.
Operations of a development platform 200 described herein according to various embodiments of the inventive concepts are illustrated in
The development platform 200 may then generate an application installation package including the mobile application that is wrapped by the MAA application, and also including the screen metadata file (block 260).
Operations of an MAA application wrapper according to further embodiments that is operating on a mobile device are illustrated in
Finally, the MAA application transmits an event report to the application analysis computer (block 286). Included in or with the event report is screen metadata describing the screen that was being displayed by the mobile application when the reportable event occurred.
Operations of an MAA application wrapper according to further embodiments that is operating on a mobile device are illustrated in
Finally, the MAA application transmits an event report to the application analysis computer (block 308). Included in or with the event report is screen metadata describing the screen that was being displayed by the mobile application when the reportable event occurred and user interaction data describing user interaction with the screen when the reportable event occurred.
Further embodiments are illustrated in
As noted above, in some embodiments, one or more screen shots of screens displayed by the mobile application can be captured showing screen layouts during various points of operation of the mobile application. The screen shots can be saved in the screen data repository 195, and can be associated with screen IDs, e.g., SCR000, SCR001, SCR002, etc. The screen identifiers can also be saved in the screen data repository 195 and can be associated with the screen shots.
Referring to
If screen metadata is stored in the screen data repository, a model of the screen image generated by the mobile application can be recreated from data in the screen metadata. In these embodiments, only a screen identifier is included in the event report. Instead of transmitting a (relatively large) screen shot or series of screen shots, or even screen metadata, to an application analysis computer in connection with the reportable event, the MAA application wrapper may transmit a screen identifier associated with the screen image, which may require the transmission of only a relatively small amount of data compared to the amount of image data that a screen shot, or even screen metadata, comprises.
Operations of the development platform 200′ illustrated in
The development platform 200 may then generate an application installation package including the mobile application that is wrapped by the MAA application, and also including the screen identifiers (block 432).
Operations of an MAA application wrapper according to further embodiments that is operating on a mobile device are illustrated in
Finally, the MAA application transmits an event report including the screen identifier to the application analysis computer (block 446).
Operations of an MAA application wrapper according to further embodiments that is operating on a mobile device are illustrated in
In the above-description of various embodiments of the present disclosure, aspects of the present disclosure may be illustrated and described herein in any of a number of patent-eligible classes or contexts including any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof. Accordingly, aspects of the present disclosure may be implemented in entirely hardware, entirely software (including firmware, resident software, micro-code, etc.) or combining software and hardware implementation that may all generally be referred to herein as a “circuit,” “module,” “component,” or “system.” Furthermore, aspects of the present disclosure may take the form of a computer program product comprising one or more computer readable media having computer readable program code embodied thereon.
Any combination of one or more computer readable media may be used. The computer readable media may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an appropriate optical fiber with a repeater, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable signal medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB.NET, Python or the like, conventional procedural programming languages, such as the “C” programming language, Visual Basic, Fortran 2003, Perl, COBOL 2002, PHP, ABAP, dynamic programming languages such as Python, Ruby and Groovy, or other programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider) or in a cloud computing environment or offered as a service such as a Software as a Service (SaaS).
Aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable instruction execution apparatus, create a mechanism for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer readable medium that when executed can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions when stored in the computer readable medium produce an article of manufacture including instructions which when executed, cause a computer to implement the function/act specified in the flowchart and/or block diagram block or blocks. The computer program instructions may also be loaded onto a computer, other programmable instruction execution apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatuses or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of this specification and the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various aspects of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The terminology used herein is for the purpose of describing particular aspects only and is not intended to be limiting of the disclosure. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. As used herein, the term “and/or” includes any and all combinations of one or more of the associated listed items. Like reference numbers signify like elements throughout the description of the figures.
The corresponding structures, materials, acts, and equivalents of any means or step plus function elements in the claims below are intended to include any disclosed structure, material, or act for performing the function in combination with other claimed elements as specifically claimed. The description of the present disclosure has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the disclosure in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the disclosure. The aspects of the disclosure herein were chosen and described in order to best explain the principles of the disclosure and the practical application, and to enable others of ordinary skill in the art to understand the disclosure with various modifications as are suited to the particular use contemplated.
This application is a divisional application of U.S. Ser. No. 14/795,996, filed on Jul. 10, 2015, the disclosure and content of which is incorporated by reference herein in its entirety.
| Number | Date | Country | |
|---|---|---|---|
| Parent | 14795996 | Jul 2015 | US |
| Child | 16264738 | US |
