Patent Application
20230281377
The described embodiments relate to adaptive computer user interfaces and, in particular, to computer-generated user interfaces for digital forensic investigation.
In a first broad aspect, there is provided a method of examining digital forensic data using a viewer computer comprising a memory and a processor, the digital forensic data extracted from at least one target device by a forensic data retrieval application, the method comprising: providing a forensic data investigation application to the viewer computer; receiving, at the viewer computer, a data collection generated by the forensic data retrieval application, the data collection comprising a plurality of data items extracted from the at least one target device; formatting the plurality of data items according to a first display type to generate a plurality of displayed data items; generating a user interface comprising the plurality of displayed data items in the first display type; and displaying the user interface via the viewer computer.
The method may further comprise receiving a context switch input associated with a selected data item of the plurality of data items; determining at least one related data artifact associated with the selected data item; formatting the at least one related data artifact according to a selected presentation format; re-generating the user interface to comprise the at least one related data artifact in the selected presentation format; and displaying the modified user interface via the viewer computer. In some cases, the first display type is one of a filesystem display type, a registry view display type and a database view display type. In some cases, the selected presentation format is selected based on the at least one related data artifact.
In some cases, the first display type is an artifact view display type and the method may further comprise: receiving a context switch input associated with a selected data item of the plurality of data items; determining at least one related data item associated with the selected data item; formatting the at least one related data item according to a selected presentation format; re-generating the user interface to comprise the at least one related data item in the selected presentation format; and displaying the modified user interface via the viewer computer.
In some cases, the selected presentation format is selected based on the at least one related data item.
The method may further comprise: receiving at least one filter input of the user interface; filtering the plurality of displayed data artifacts based on the at least one filter input; and displaying an active filter indication to indicate that the plurality of displayed data artifacts is filtered based on the at least one filter input.
In some cases, the active filter indication is a shading of the filter dialog.
In some cases, the at least one filter input is received via at least one of a filter dialog, a navigation view selection and a main view selection.
In some cases, the user interface comprises a navigation view.
In some cases, in the filesystem display type, the navigation view comprises a filesystem hierarchy selectable via the user interface to filter the displayed data items according to location within a filesystem hierarchy.
In some cases, in the artifact view display type, the navigation view comprises a plurality of artifact categories, the plurality of artifact categories selectable via the user interface to filter the displayed data items according to a selected artifact category. In some cases, in the registry view display type, the navigation view comprises a registry tree, the registry tree selectable via the user interface to filter the displayed data items according to a selected registry tree element. In some cases, in the database display type, the navigation view comprises a filesystem structure with a database data item selected, the selection of the database data item causing the viewer computer filter the displayed data items according to the selected database data item.
In some cases, the user interface comprises a main view to display the plurality of data items. In some cases, the main view displays the plurality of displayed data items in one of a column detail presentation format and a row detail presentation format. In some cases, the main view displays the plurality of displayed data items in a chat thread presentation format. In some cases, the main view displays the plurality of displayed data items in a timeline presentation format. In some cases, the main view displays the plurality of displayed data items in a map presentation format. In some cases, the main view displays the plurality of displayed data items in a thumbnail presentation format.
In some cases, the user interface comprises a detail view.
In some cases, the user interface comprises an annotation view.
In some cases, the user interface comprises a profile view.
The method may further comprise: receiving a report command via a report element of the user interface; generating a report based on at least one of the plurality of displayed data artifacts; and storing the report in a storage device of the viewer computer. In some cases, generating the report is based further on a current configuration of the user interface.
The method may further comprise, prior to receiving the report command, receiving a selection of the at least one of the plurality of displayed data artifacts via the user interface.
In another broad aspect, there is provided a non-transitory computer-readable medium storing computer-executable instructions, the instructions when executed by a computer processor for causing the computer processor to carry out methods as described herein.
In yet another broad aspect, there is provided a viewer computer comprising a memory and a processor, the processor configured to carry out methods as described herein.
A preferred embodiment of the present invention will now be described in detail with reference to the drawings, in which:
Various systems or methods will be described below to provide an example of an embodiment of the claimed subject matter. No embodiment described below limits any claimed subject matter and any claimed subject matter may cover methods or systems that differ from those described below. The claimed subject matter is not limited to systems or methods having all of the features of any one system or method described below or to features common to multiple or all of the apparatuses or methods described below. It is possible that a system or method described below is not an embodiment that is recited in any claimed subject matter. Any subject matter disclosed in a system or method described below that is not claimed in this document may be the subject matter of another protective instrument, for example, a continuing patent application, and the applicants, inventors or owners do not intend to abandon, disclaim or dedicate to the public any such subject matter by its disclosure in this document.
Furthermore, it will be appreciated that for simplicity and clarity of illustration, where considered appropriate, reference numerals may be repeated among the figures to indicate corresponding or analogous elements. In addition, numerous specific details are set forth in order to provide a thorough understanding of the embodiments described herein. However, it will be understood by those of ordinary skill in the art that the embodiments described herein may be practiced without these specific details. In other instances, well-known methods, procedures and components have not been described in detail so as not to obscure the embodiments described herein. Also, the description is not to be considered as limiting the scope of the embodiments described herein.
It should also be noted that the terms “coupled” or “coupling” as used herein can have several different meanings depending in the context in which these terms are used. For example, the terms coupled or coupling may be used to indicate that an element or device can electrically, optically, or wirelessly send data to another element or device as well as receive data from another element or device.
It should be noted that terms of degree such as “substantially”, “about” and “approximately” as used herein mean a reasonable amount of deviation of the modified term such that the end result is not significantly changed. These terms of degree may also be construed as including a deviation of the modified term if this deviation would not negate the meaning of the term it modifies.
The example embodiments of the systems and methods described herein may be implemented as a combination of hardware or software. In some cases, the example embodiments described herein may be implemented, at least in part, by using one or more computer programs, executing on one or more programmable devices comprising at least one processing element, and a data storage element (including volatile memory, non-volatile memory, storage elements, or any combination thereof). These devices may also have at least one input device (e.g. a keyboard, mouse, a touchscreen, and the like), and at least one output device (e.g. a display screen, a printer, a wireless radio, and the like) depending on the nature of the device.
It should also be noted that there may be some elements that are used to implement at least part of one of the embodiments described herein that may be implemented via software that is written in a high-level computer programming language such as object oriented programming. Accordingly, the program code may be written in C, C++, Java or any other suitable programming language and may comprise modules or classes, as is known to those skilled in computer programming.
Alternatively, or in addition thereto, some of these elements implemented via software may be written in assembly language, machine language or firmware as needed. In either case, the language may be a compiled or interpreted language.
At least some of these software programs may be stored on a storage media (e.g. a computer readable medium such as, but not limited to, ROM, magnetic disk, optical disc) or a device that is readable by a general or special purpose programmable device. The software program code, when read by the programmable device, configures the programmable device to operate in a new, specific and predefined manner in order to perform at least one of the methods described herein.
Furthermore, at least some of the programs associated with the systems and methods of the embodiments described herein may be capable of being distributed in a computer program product comprising a computer readable medium that bears computer usable instructions for one or more processors. The medium may be provided in various forms, including non-transitory forms such as, but not limited to, one or more diskettes, compact disks, tapes, chips, and magnetic and electronic storage.
Historically, forensic data investigation tools have tended to focus on features rather than usability. Often, this has required significant training of user in order to use investigation tools effectively and correctly. Even with training, users may need periodic re-training in order to maintain their knowledge current. Without training, users may quickly forget how to use the prior art tools, leading to missed investigative opportunities, difficulty with reporting findings and other difficulties.
The described embodiments provide methods and apparatus for a forensic data investigation application, which provides enhances usability through data processing and presentation, along with novel user interface techniques. In addition, the described embodiments provide graphical user interfaces that may generally enhance the teachability of the forensic data investigation application. Examples of concepts that enhance teachability include: 1) “source linking”, which is the linking through the graphical user interface of data artifacts to the location of a data item in a filesystem, thereby allowing the user to discover where evidence was originally found and identify other related data items, and to validate the underlying data; 2) “related artifacts linking”, which is the linking through the graphical user interface of data items to related data artifacts, thereby allowing the user to discover artifacts associated, for example, with a file or folder, but which may be located in a different area of the filesystem; 3) row detail presentation format, which presents the most relevant attributes for data items according to their category, thereby facilitating identification of relevant data by the user and easy reporting; and 4) implicit filters, which apply filters according to data artifact category, and thereby allow the user to learn what data artifacts have particular attributes, such as geolocation or time information.
Additionally, the described embodiments also facilitate efficient generation of reports and, in particular, may generate reports based on a “what you see is what you get” model. For example, reports may adopt a presentation format similar to that displayed in the graphical user interface. Likewise, reports may contain data items corresponding to those currently selected for display in the graphical user interface, in particular based on applied filters (whether implicit or explicit). The viewer computer may automatically generate and report relevant attributes (such as date and time, filesystem location, etc.) according to the data item category.
Referring now to
Data investigation system 100 generally comprises a viewer computer 110, which is coupled to a data storage device 130, and which optionally may be coupled to one or more target devices, such as a desktop computer 121, mobile device 122 and data storage device 123. Coupling may be achieved using a physical connection, such as a Universal Serial Bus (USB) connector or cable, an IEEE 802.3 (Ethernet) network interface, or other suitable coupling interface or adapter. Target devices may also be any type of data storage media, such as magnetic and solid state disk drives, optical media, or network file shares.
Viewer computer 110 has one or more software application as described herein. As used herein, the term “software application” or “application” refers to computer-executable instructions, particularly computer-executable instructions stored in a non-transitory medium, such as a non-volatile memory, and executed by a computer processor. The computer processor, when executing the instructions, may receive inputs and transmit outputs to any of a variety of input or output devices to which it is coupled.
Optionally, viewer computer 110 may be provided with a forensic data investigation software application, to retrieve data from one or more target device. In some other cases, a separate computer may be provided with the forensic data investigation software application to retrieve data from the target device. For example, the forensic data investigation software application may do a low-level block-based copy from a target device storage media, to retrieve all data on the device, regardless of whether attempts have been made to delete the data. In other cases, the forensic data investigation software application may simply copy files and folders using operating system-level file copy facilities. Specific techniques for forensic data retrieval will be known in the art.
The forensic data investigation software application may analyze the retrieved data to identify data items of interest. Generally, data items can represent any data that can be retrieved from target device storage media, such as files, folders, block data or byte ranges, volume information, file images, and the like.
On their own, data items generally can be viewed using a text preview, which converts the raw data into a text representation (e.g., using ASCII or UTF coding), or in a binary or hexadecimal representation. However, reviewing large amounts of data items in this format is time-consuming and difficult. Therefore, viewer computer may generate, or the data collection may contain, a plurality of data artifacts.
Data artifacts are another type of data item that represents one or more other data items in a structured way.
A simple form of data artifact is one created based on file extensions of data items. For example, the viewer computer may generate a data artifact of type “documents” for a data item with a file extension of .DOC.
However, more advanced data artifacts can also be generated through the use of one or more refining modules. For example, the viewer computer may search for data patterns indicative of particular file types, such as media files, to generate media data artifacts or text data artifacts, respectively. Such generation of data artifacts can occur regardless of whether attempts have been made to obfuscate the nature of a particular file, for example, by changing a file extension or even deleting a file (where the underlying raw data can be recovered from unused space on the target device storage media).
Refining modules can be provided or defined for a wide variety of data artifacts. Examples of data artifacts that can be generated include, but are not limited to:
Some types of data items may be used to generate more than one data artifact. For example, an e-mail database may be used to generate a large number of data artifacts corresponding to individual e-mail messages.
Data items, including data artifacts, may be stored in a data collection, which can be an image file or an electronic database file stored in a data storage device 130. The electronic database file may be a relational database, such as Microsoft SQL Server™ or a non-relational database, such as a key-value database, NoSQL database, or the like. A data collection may contain data items retrieved from more than one target device and, because data artifacts are a type of data item, the data collection may also contain data artifacts generated by the viewer computer. Each data item in the data collection may be tagged with information to identify the target device that is the source of the data item.
Data storage device 130 is a non-volatile data store coupled to viewer computer 110. For example, data storage device 130 may be an external storage device coupled to viewer computer 110 locally, an internal device such as a hard drive. In some cases, viewer computer 110 may be coupled to a networked storage device 131 via a data communication network 150. Data communication network can be a private data communication network, such as a local area network or wide area network, or may also be a public data communication network, such as the Internet. When viewer computer 110 is configured to access data storage device 130 over a public network, or even over a private network, encryption (e.g., Transport Layer Security) can be used to safeguard data.
Viewer computer 110 is provided with a forensic data investigation application. In operation, the forensic data investigation application can be used to retrieve the data collection, e.g., from data storage device 130, and to generate a user interface to facilitate forensic investigation of the data collection as described further herein.
Referring now to
Viewer computer 210 has a processor 205, which is coupled to a volatile memory 220, a non-volatile memory 225, a peripheral bus interface 230, a data communications interface 240, an output device 250. The peripheral bus interface 230 may further couple processor 205 to an external storage interface 260, a user input device 260 and a target device interface 270. It will be appreciated that
Processor 205 is a computer processor, such as a general purpose microprocessor. In some other cases, processor 205 may be a field programmable gate array, application specific integrated circuit, microcontroller, or other suitable computer processor.
Processor 205 is coupled, via a computer data bus, to volatile memory 220 and non-volatile memory 225. Non-volatile memory 225 stores computer programs consisting of computer-executable instructions, which may be loaded into volatile memory 220 for execution by processor 205 as needed. It will be understood by those skilled in the art that references herein to a viewer computer as carrying out a function or acting in a particular way imply that a processor (e.g., processor 205 of viewer computer 210) is executing instructions (e.g., a software program) stored in a memory and possibly transmitting or receiving inputs and outputs via one or more interface. Volatile memory 220 may also store data input to, or output from, processor 205 in the course of executing the computer-executable instructions. In some cases, non-volatile memory 225 may store a data collection.
Processor 205 is also coupled to an output device 250, such as a computer display, which outputs information and data as needed by various computer programs. In particular, output device 250 may display a graphical user interface (GUI) generated by viewer computer 210.
Processor 205 is coupled to data communication interface 240, which is one or more data network interface, such as an IEEE 802.3 or IEEE 802.11 interface, for communication over a network.
Processor 205 may be coupled to a peripheral bus interface 230 via a data bus. In other embodiments, peripheral bus interface 230 may be omitted and processor 205 may be coupled to devices such as external storage interface 260 directly via a data bus.
In the example embodiment, peripheral bus interface 230 is coupled to an external storage interface 260, for example, to interface with external storage device 130.
Peripheral bus interface 230 is also coupled to one or more user input device 260, such as a keyboard or pointing device.
Finally, in some embodiments, peripheral bus interface 230 may also be coupled to a target device interface 270, for interfacing with and retrieving data from one or more target devices, such as target device 121 of
In some embodiments, viewer computer 210 is a desktop or portable laptop computer 130. In other embodiments, viewer computer 210 may be a mobile device such as a smartphone or tablet computer.
Referring now to
Graphical user interface 300 may be generated by a viewer computer, such as viewer computer 110 or viewer computer 210, and displayed on a display such as output device 250 of viewer computer 210.
In particular, graphical user interface 300 may be generated and displayed to allow a user of the viewer computer to review and examine data items within a data collection, as generated by a forensic data investigation software application.
In the example embodiment, graphical user interface 300 has a navigation view area 310, a selection input 315, a main view area 320, a selection input 325, an annotation view area 330, a detail view area 340, a filter interface 350 and a search interface 355. Each of the areas or elements of graphical user interface 300 (e.g., navigation view 310, main view 320, annotation view 330, detail view 340 and preview 370) may be repositioned, resized, detached and displayed in a separate window (as shown in
Navigation view 310 may be used to display organizational data relating to data items. For example, while in an artifact view display type, navigation view 310 may be formatted to display one or more categories or subcategories of data artifacts, or both. A user of the viewer computer may select such categories or subcategories, to cause the viewer computer to search within a current data collection and generate a display of data artifacts within the selected categories or subcategories in a main view 320. Selection of a category or subcategory in navigation view 310 can be used as a type of implicit filter, in addition to explicit or contextual filters as described elsewhere herein.
Selection input 315 may be used to change the display type of navigation view 310. For example, selection input 315 may be a button or group of buttons or a drop-down dialog box, which allows the user to select one of a plurality of display types. One display type is the artifact view display type. However, examples of other display types are a filesystem display type, a database display type, a registry view display type, and generic display types.
In general, operation of the selection input 315 serves to change the display type of navigation view 310. In some cases, this change in display type may cause the presentation format of main view 320 to be changed accordingly. In such cases, viewer computer may attempt to retain the previously selected data items within main view 320 to the extent possible.
In a filesystem display type, navigation view 310 may be formatted to display a filesystem hierarchy corresponding to that of the target device or target devices used to generate the current data collection. For example, if a target device is a laptop computer, the displayed filesystem hierarchy may correspond to that of the target laptop computer's mass storage device (e.g., solid state disk). The navigation view 310 may allow the user to navigate within the filesystem hierarchy and select directories, the contents of which (i.e., data items originally found in the selected directory) can be displayed in main view 320. The navigation view 310 may allow for filesystem hierarchies to be expanded and collapsed, for example, by use of a disclosure triangle control.
In some cases, the filesystem display type may also display data items relating to filesystem components such as disk partitions, unallocated space, logical volumes, deleted files, and other objects associated with a filesystem.
In a registry view display type, navigation view 310 may be formatted to display a system registry hierarchy, such as the Microsoft Windows™ registry. For other operating systems, the registry view display type may be adapted to display system configuration files and information. For example, for the Mac OS X™ operating system, the registry view display type may display XML files and key-value data corresponding to system configuration settings. The navigation view 310 may allow the user to select certain registry parameters, and data items associated with the selected registry parameters can be displayed in main view 320. For example, the navigation view may display a registry tree, the registry tree having selectable registry tree elements that can be used to filter the displayed data items in main view 320 according to a selected registry tree element.
In a database display type, navigation view 310 may be formatted in similar fashion to filesystem display type, to display a filesystem hierarchy containing a database file or files, such as the file containing a SQL database. The navigation view 310 may allow the user to identify a database to examine, and data items associated with the selected database can be displayed in main view 320 in a database presentation format.
Main view 320 generally is used for the display of data items. Data items may be displayed in one or more presentation formats. Examples of presentation formats include, but are not limited to, column detail, row detail, chat thread, thumbnail, timeline, map, filesystem and registry. A selection input 325, such as a drop-down dialog, can be used to change between presentation formats.
In general, operation of the selection input 325 serves to change the presentation format of main view 320. Viewer computer may attempt to retain the previously selected data items within main view 320 to the extent possible for the presentation format.
Many of the described presentation formats allow for the display of data items in a heterogeneous list, that is, displaying more than one type of data item contemporaneously in main view 320. For example, a main view 320 in a row detail presentation format may display data artifacts of the media category, data artifacts of the chat category, data artifacts of the web browser category, data items of the file type, and still others in a single list (as shown in main view 420 of
When a particular data item is selected in main view 320, attributes of the data item also can be displayed in detail view 340 in a detailed summary format. Detail view 340 may be scrollable or resizable, or both, to allow a user to view all attributes relating to the selected data item. In some cases, detail view may also include a preview of the data item. In other cases, the preview may have a separate view.
Generally, detail view 340 can provide a summary of the attributes for a selected data item, where those attributes may also be displayed in columns of a column detail presentation format.
In some cases, multiple data item may be selected in main view 320, in which case detail view 340 may display aggregate information relating to, or common to, all selected data items.
A preview area 370 may also be provided in some cases. As the name implies, the preview area may display a preview of a selected data item. For example, for a media data artifact, preview area 370 may display a resized image or an image thumbnail of a video. In another example, for a document data artifact, preview area 370 may display a rendering of the document contents. In some cases, where the selected item is not a data artifact, preview area 470 may contain a text view which displays text strings extracted from the selected data item, or a hex view, which displays data in raw hexadecimal format for the selected data item. Various other types of previews for different types of data artifacts may also be displayed using a suitable renderer.
Annotation view 330 can be used to allow a user to tag data items with labels or annotations. Tags can be applied to any type of data item described herein, whether or not they are also data artifacts (e.g., files, folders, chat artifacts, etc.). Annotation view 330 may include predefined tags or labels, which can be selected in the graphical user interface 300. In some cases, annotation view 330 may allow the user to define additional tags or labels, comments and profiles, which can be applied to selected data items. Once defined, tags or labels, comments and profiles can be used as search or filter criteria.
Profile view 360 can be used to allow a user to assign a profile identifier to a data item. The profile identifier may be generated by the viewer computer when a new profile is created, and may optionally be given a friendly name by the viewer computer or the user. Generally, when the user assigns a profile identifier to a data item, viewer computer can parse the data item—which may be a data artifact—to determine whether the data item contains a unique user identifier, such as an e-mail address, chat service username, phone number, address or the like. The viewer computer may then analyze other data items within the data collection to identify instances of the unique user identifier, and assign the same profile identifier to those data items. The profile identifier can then be used to filter data items, for example using filter interface 350, allowing the user to quickly and easily identify data items that relate to a particular profile, which may itself relate to a particular person of interest. In some embodiments, profile identifiers may only be assigned to data artifacts.
Filter interface 350 can be used to filter the data items displayed in main view 320 or also navigation view 310. In general, filter interface 350 can be used to filter on any attribute of a data item, including but not limited to, type or category, dates and times, and tags. Filters can also be combined, for example by applying multiple filters successively. In some cases, Boolean operators, such as AND, OR or NOT may be applied to combine filters. In some embodiments, filter interface 350 may allow for pattern matching, e.g., with regular expressions, to be used to define filters.
When a filter is selected or applied, an active filter indication may be provided, to indicate that the filter is in effect and thereby limiting the data items displayed. In some cases, the active filter indication is a shading of the filter dialog, for example with a color. The active filter indication can be removed when all filters are deselected.
Similarly, search interface 355 can be used to enter freeform text and search for specific attributes, such as names, types, dates, and the like. An advanced search interface can also be provided, to allow a user to craft specific searches.
Referring now to
In graphical user interface 400A, main view 420 displays the column detail presentation format as indicated by selection input 425. Each row of the column detail presentation format corresponds to one data item, and each column of the column detail presentation format displays an attribute for each data item. Navigation view 410 is shown in an artifact view display type as indicated by selection input 415, with a media category selected. Accordingly, main view 420 is displaying data artifacts of the media category. The displayed columns in
More generally, in a column detail presentation format, data items may be presented in a vertical list, with attributes of each data item set out in columns. Each column may have a heading, which can be sorted by the user (e.g., by clicking the column header to select which column to sort on), or used to define a filter. Examples of attributes include those shown in
Columns may be reordered, added or hidden from view.
In some cases, a column heading may be used to define a filter and, in particular, a contextual filter. For example, a user may transmit a command to define a contextual filter by option-clicking or right-clicking on a column heading. Viewer computer may determine the type of data contained in the column, and provide a contextual control to allow the user to define filter criteria specific to the column. In one example, if the column contains date data, the contextual control may accept a start date and an end date. In another example, if the column contains numerical data, the context control may accept a lower bound and an upper bound. In general, a contextual filter can also be used to limit the display only to those data items with attributes belonging in a particular column.
In some cases, the columns presented in column detail presentation format may be context sensitive. For example, when navigation view 410 is in an artifact view display type, the selection of different artifact categories in navigation view 410 may cause the columns in main view 420 to be altered. For example, if data artifacts belonging to a chat category are displayed, one column of main view 420 may indicate attributes specific to chat artifacts, such as the type of chat (e.g., Yahoo!™, ICQ™, etc.), message sent date, etc. Similarly, when data artifacts belonging to an e-mail category are selected, the columns of main view 420 may display attributes specific to e-mail, such as the sender, recipient, subject, sent date and various other fields. When data artifacts belong to a photo media category are selected, the columns of main view 420 may display attributes specific to media, such location data (if available), camera make and model, image size, and other image metadata. However, in some cases, data artifacts of multiple categories can be displayed together in main view 420 (e.g., when the “all evidence” category is selected).
In general, the columns of main view 420 can be used to display attributes or metadata relating to each data item.
Referring now to
Referring now to
In graphical user interface 4000, main view 420 displays the row detail presentation format as indicated by selection input 425. Each cell of the row detail presentation format corresponds to one data item, with attribute information relating to the data item laid out in a table. Navigation view 410 is in an artifact view display type as indicated by selection input 415, with no category selected. This may be referred to as an “all evidence” category. Accordingly, main view 420 is displaying data artifacts of any category.
Main view 420 shows various data artifacts, including URL artifacts, social network artifacts, media artifacts, document artifacts and browser artifacts. Each cell in main view 420 contains an icon or thumbnail preview, an artifact name, and attributes specific to the data artifact type. Data artifacts of different types may have different attributes shown in the row detail presentation format. Accordingly, as with other presentation formats, the row detail presentation format can be used when the user wishes to view data artifacts of different types together in main view 420. However, the row detail presentation format provides a more compact view in some cases.
More generally, in a row detail presentation format, data artifacts may be presented in a vertical list of cells (which can be subdivided in a table-like format), with attributes of each data artifact set out within each cell. Each cell can be selected and used to perform a context switch as described elsewhere herein, to reveal a source location of the data artifact.
As noted above, the attributes presented in each cell of the row detail presentation format may be context sensitive. For example, if data artifacts belonging to a media category and an e-mail category are displayed, the e-mail artifact cell in main view 420 may display e-mail attributes, such as sender and number of attachments, whereas the media artifact cell may display a media type.
In general, the cells of main view 420 when in a row detail presentation format can be used to display attributes or metadata relating to each respective data artifact.
In some cases, the row detail presentation format may also be used for data items, more generally.
Referring now to
In graphical user interface 400D, main view 420 displays the chat thread detail presentation format as indicated by selection input 425. Each cell of the chat thread detail presentation format corresponds to one data artifact, with attribute information relating to the artifact laid out in a table, similar to row detail presentation format shown in
Each cell in main view 420 contains an icon representative of the chat service associated with the data artifact, a message sender and date and time. Each cell may also contain a link to a location associated with the message. Chat artifacts can be created by parsing a chat client database to extract the various attributes, for example.
When a cell is selected in main view 420, the viewer computer may generate a contextually-aware message thread for display in preview area 470. For example, in
Each cell can be selected and used to perform a context switch as described elsewhere herein, to reveal a source location of the data artifact.
More generally, in a chat thread detail presentation format, data artifacts may be presented in a vertical list of cells (which can be subdivided in a table-like format), with attributes of each data artifact set out within each cell. When a chat data artifact is selected, the preview area 470 can be used to show the chat data artifact in the context of the chat in which the artifact was first created.
The attributes presented in each cell of the chat thread detail presentation format may be context sensitive. For example, attributes may be unique to a particular chat service.
Referring now to
In graphical user interface 400E, main view 420 displays the thumbnail presentation format as indicated by selection input 425. Thumbnail presentation format generates and displays one or more thumbnail image 422, corresponding to data artifacts selected in navigation view 410. Navigation view 410 is in an artifact view display type as indicated by selection input 415, with a generic media category selected. Accordingly, main view 420 is displaying artifacts of all media categories.
In some cases, thumbnail presentation format may also be used for data items more generally.
Main view 420 shows thumbnail images 422 for photo and other static image artifacts and may also display thumbnail images (e.g., JPEG, GIF, PNG) for video artifacts. In some cases, the thumbnail images may contain playable video, which can be activated by clicking on or hovering a cursor over the thumbnail image in main view 420, for example. In some cases, selection of the thumbnail may cause media controls to displayed in preview area 470. Thumbnail images 422 are labeled with the artifact name, and may optionally be labeled with additional attribute information.
A thumbnail size control 426, such as a drop-down dialog or a slider, can also be provided, to allow a user to adjust the size of the thumbnail images 422.
Referring now to
In graphical user interface 400F, main view 420 displays the timeline presentation format as indicated by selection input 425. The timeline presentation is generated by viewer computer by aggregating data artifacts with date and time information, and generating an interactive graph visualization. In some cases, the timeline presentation can be used for data items more generally, where those data items have date and time information (e.g., files and folders). Individual data artifacts are represented as points along the x-axis (timeline), and data artifacts that are close in time to each other are stacked in the y-axis along the timeline. This produces a bar chart that enables easy visualization of the creation or modification dates of data artifacts. Spikes in the chart are indicative of a large number of data artifacts that are close together in time. Generally, the width and height of the timeline may be adjusted in response to changes in the displayed data artifacts. For example, as data artifacts are selected for display that require a longer timeline, the timeline may be scaled in the x-axis to display all selected data artifacts within main view 420. Similarly, as data artifacts are stacked, the scale of the timeline along the y-axis may be adjusted accordingly, to accommodate all stacks.
In operation, a user may hover over a slice of the chart to cause a pop-over tooltip to be displayed with contextual information, such as the number and type of data artifacts represented by the slice. The timeline presentation format may allow for zooming in or out on a slice. Selection of a slice may, in some cases, cause a different presentation format (e.g., column detail) to be generated containing the data artifacts belonging to the slice.
In some embodiments, multiple timelines may be shown concurrently. For example, a primary timeline 423 may be provided, which provides a visualization of all dates associated with data artifacts. One or more secondary timelines 424 or “swim lanes” may then be provided, to provide visualizations of specific types of dates (e.g., creation date, URL last visited date, cookie expiry date, etc.). A timeline control (not shown) can be used to select the types of secondary timelines 424 to be displayed. For example, in some cases, secondary timelines 424 may be defined using filter input.
Each slice can be selected and used to perform a context switch as described elsewhere herein, to reveal a source location of the data artifact or data artifacts.
Referring now to
In graphical user interface 400G, main view 420 displays the map presentation format as indicated by selection input 425. The main view displays a geographic map, with pins 491 or clusters 490 placed on the geographic map according to location data associated with data artifacts. For example, if one data artifact is associated with a street address, or latitude/longitude, a pin 491 may be placed on the corresponding location on the map. A zoom control 495 may be provided, allowing the user to zoom into or out of the map. As the user zooms out, pins may be collapsed into clusters 490. Conversely, as the user zooms in, clusters 490 may be broken out into individual pins 491. Each cluster 490 may be labeled with a number representing the number of data artifacts within the cluster.
When a user hovers over a pin 491 or cluster 490, a dialog may be displayed containing attribute data corresponding to the data artifacts represented by pin 491 or cluster 490.
Each cell of the row detail presentation format corresponds to one data artifact, with attribute information relating to the artifact laid out in a table. Navigation view 410 is in an artifact view display type as indicated by selection input 415, with a media category and pictures subcategory selected. Accordingly, main view 420 is displaying data artifacts of the picture subcategory.
Each pin or cluster can be selected and used to perform a context switch as described elsewhere herein, to reveal a source location of the data artifact or data artifacts.
Referring now to
In graphical user interface 400H, main view 420 combines elements of the row detail presentation format as shown in
As with the format of
Selection of one or more data artifacts in area 428 may cause the corresponding selection to be made in area 429, and vice versa. In general, the subdivision of main view 420 into multiple different presentation formats allows for easy visualization and selection of data artifacts by a user.
Each cell, pin or cluster can be selected and used to perform a context switch as described elsewhere herein, to reveal a source location of the data artifact or data artifacts.
In some other embodiments, main view 420 may be subdivided into other presentation formats. For example, a timeline presentation format may be shown in one subdivision and a row detail presentation format shown in another subdivision. Other combinations are also contemplated.
Referring now to
In graphical user interface 400I, main view 420 displays the filesystem presentation format as indicated by selection input 425. Each row of the filesystem presentation format generally corresponds to a data item retrieved from a target device, with attribute information presented in columns. Data items relating to filesystem objects, information and constructs can be displayed, such as files and folders as found on a target device, deleted files, volume information, and the like. Navigation view 410 is in a filesystem view display type as indicated by selection input 415, with a “local disk” selected for a target device with the identifier “Dunlop Computer”. Accordingly, main view 420 is displaying data items representing files and folders retrieved from the mass storage device of a target device. The displayed columns in
Preview area 470 may be used to display previews for certain data artifacts, such as images for media files when selected. As with preview area 370, preview area 470 may also include a text view which displays text strings extracted from a selected file, or a hex view, which displays data in raw hexadecimal format for a selected file.
In general, the columns of main view 420 can be used to display attributes or metadata relating to each data item.
Each data item can be selected and used to perform a context switch as described elsewhere herein, to reveal data artifacts related to the data item.
Referring now to
In graphical user interface 400J, main view 420 displays the registry presentation format as indicated by selection input 425. Each row of the registry presentation format corresponds to one registry setting retrieved from a target device, with attribute information and values presented in columns. Registry settings may also be considered data artifacts in some embodiments. Navigation view 410 is in registry view display type as indicated by selection input 415, displaying a hierarchical registry tree. Accordingly, main view 420 is displaying registry settings retrieved from the target device. The displayed columns in main view 420 are therefore setting name, setting type, data and modification date and time. Additional columns may be provided, if desired. Registry presentation format may be analogous in operation to column detail presentation format. For example, detailed attributes for selected settings can be displayed in detail view 440.
Preview area 470 may be used to display a hex view, which displays registry data in raw hexadecimal format for a selected key or hive.
Each row of main view 420 can be selected and used to perform a context switch as described elsewhere herein, to reveal a data artifact or data artifacts related to the selected setting.
Referring now to
In graphical user interface 400K, main view 420 displays the database presentation format as indicated by selection input 425. The database presentation format may be used to display database information generally. In particular, the database presentation format may be used to reveal information underlying data artifacts representative of database structures. For example, some chat service clients store message history and user information in a database store, which may contain multiple tables. The chat service client may construct each discrete message by querying the database, to pull information from multiple tables. In various other presentation formats, viewer computer may display data artifacts as a discrete entity, however the user may request a context switch to reveal the underlying database tables and records used to construct the selected data artifact. The context switch can be requested, for example, using a contextual dialog, a user interface element such as a button, or a link displayed in detail view 440.
When the database presentation format is enabled in main view 420, navigation view 410 may switch to the filesystem display type, revealing the relevant database files. Each row of the database presentation format can correspond to a table. Detail view 440 may display records of the table selected in main view 420. Database presentation format may be analogous in operation to column detail presentation format, for example, with columns provided for modification date, size, type and the like.
Preview area 470 may be used to display a hex view, which displays database data in raw hexadecimal format for a selected record.
Referring now to
Method 500 begins at 505, with the viewer computer receiving a data collection containing data items. The data collection may be an image file or an electronic database file stored in a data storage device 130 as described herein. A data collection may contain data items retrieved from more than one target device. In such cases, each data item in the data collection may be tagged with information to identify the target device that is the source of the data item. Likewise, the data collection may contain data artifacts generated based on one or more other data items.
At 510, viewer computer formats the plurality of data items according to a first display type and first presentation format to generate a plurality of displayed data items. The graphical user interface is generated at 515 and displayed at 520, e.g., using output device 250 of viewer computer 210.
The data artifacts can be displayed, for example, in a main view 320 or 420, with navigation data displayed in a navigation view 310 or 410 as described herein. The display type may be, for example, an artifact view display type, a filesystem display type or a registry view display type. The presentation format may be, for example, a column detail presentation format, or any other presentation format described herein.
Method 500 may enter into an idle loop and wait for further events, such as key presses or pointer input.
At 525, a display switch may be detected. A display switch may occur, for example, when the user selects selection input 315, selection input 325 or a context switch command.
Operation of selection input 315 causes the viewer computer to switch from one display type to another display type in navigation view 310. Similarly, operation of selection input 325 causes the viewer computer to switch from one presentation format to another presentation format in main view 320. In some cases, a change to display type may automatically cause a change to presentation format, or vice versa.
A context switch command instructs the viewer computer to switch from one display type or presentation format to another display type or presentation format, or vice versa, based on one or more currently selected data item. For example, if the graphical user interface is currently displaying an artifact view display type and a chat thread presentation format, with a chat artifact selected, the context switch may cause a switch to a filesystem display type and filesystem presentation format, with a corresponding file data item selected. Similarly, if the navigation view is in the filesystem display type and the main view is in the filesystem presentation format, the user may select a file or folder data item and enter a context switch command. The viewer computer identifies the selected data items (representing file or folder), then determines related data artifacts. For example, if the data item is a photo file, viewer computer may identify a data artifact corresponding to the photo for display in main view 320 in the new presentation format. In some cases, the viewer computer may also identify other data artifacts with related attributes, such as photo artifacts with a similar date or time (e.g., within a predefined time range), a similar location (e.g., within a predefined distance of the location data in the selected photo), or a similar folder (e.g., shared with the selected photo). In another example, if the data item is a chat message file, viewer computer may identify a corresponding chat artifact, and optionally other chat messages with related attributes, such as a similar date or time. Selection of the context switch command may cause the navigation view display type and main view presentation format to change accordingly. For example, when identifying photo artifacts, the navigation view may change to an artifact view display type and the main view may change to a thumbnail presentation format or other presentation format that the user may predefine.
Context switches can be performed between various display types and presentation formats. For example, a context switch can be performed from a registry display type and presentation format to filesystem display type and presentation format. Similarly, a context switch can be performed from a database display type and presentation format to filesystem display type and presentation format or to an artifact view display type and column detail presentation format (for example). Various other combinations may be performed.
A context switch can be performed bi-directionally. For example, if the navigation view is instead in an artifact view display type and main view is in, for example, a column detail presentation format, the user may select a data artifact and enter a context switch command. In this context, the viewer computer identifies the selected data artifact, determines a data item corresponding to a source file or folder of the data artifact, then changes the navigation view and main view into filesystem display type and filesystem presentation format, respectively, to display the data items representing the source file or folder of the previously selected data artifact, together with other data items in the source folder (if any).
A context switch may be input to viewer computer, for example, by the user selecting one or more data item or artifact and providing a contextual command such as, for example, right-clicking and selecting a context switch option. In other examples, a context switch input may be displayed in a graphical user interface at all times, or in detail view 340 in the form of a clickable element for example.
If the display switch is a context switch, then at 530 the viewer computer can determine at least one related data item associated with the selected data item, for display in the graphical user interface.
At 540, the viewer computer formats the selected data item and the at least one related data item according to a selected presentation format, which may be chosen based on the selected data item. In the case of a context switch, the selected presentation format can based on the first display type or the main view presentation format, or both. The display type may also change. For example, if the first display type was a filesystem display type (or the main view presentation format was a filesystem presentation format), then the second display type may be an artifact view display type, and vice versa. Generally, when the first presentation format is an artifact presentation type, such as column detail, row detail, thumbnail, chat thread or map presentation format, the second presentation format can be a filesystem presentation format (e.g., to reveal the source folder location of the selected artifact).
In the case where the display switch is due to selection of selection input 315 or selection input 325, viewer computer may simply select the new display type or presentation format without regard to the selected data items if those data items can be displayed in the new display type or presentation format. Generally, viewer computer can attempt to maintain the previous selection within the navigation view or main view after the display switch.
At 545, the graphical user interface is regenerated and then the refreshed graphical user interface is displayed again at 520.
Referring now to
Method 600 begins with display of a graphical user interface as at 520 of method 500. A user may wish to filter the display, for example, due to a large number of data items being displayed in main view 320 or 420.
At 605, the viewer computer determines that at least one filter has been applied and regenerates the main view to display only those data items that satisfy the filter criteria. The applied filter may be implicit, contextual or applied via filter interface 350.
At 610, the filter interface (e.g., filter interface 350 or 450) may be provided with an active filter indication, for example by shading the filter interface with a different color, by providing an icon, or by emphasizing text in the filter interface. In some cases, an implicit filter applied via navigation view 310 may be indicated via shading of the corresponding artifact category or subcategory.
The graphical user interface is regenerated at 620 and re-displayed at 520.
If, at 605, the viewer computer determines that a filter has been removed, the acts of method 600 may be reversed and previously filtered data items may be re-displayed. If no other filters are applied, the main view may be regenerated to display all data items once again, the active filter indication may be disabled at 630 and the graphical user interface regenerated and redisplayed.
Referring now to
Filter interface 650 is shaded to provide the active filter indication as described with respect to
Referring now to
Graphical user interface element 700 can contain an input control 705 for selecting the name of the report and the location where it is to be stored. Graphical user interface element 700 may also provide a report type selection 706 for selecting a type of report such as, for example, Portable Document Format (PDF), Microsoft Word™ or other formats.
Graphical user interface element 700 may also enable the user to specify the data items to be included in the report using a selection input 710, which may be, for example, a radio button control. For example, the user may select to include only those data items that were selected prior to receiving a report command, all data items in the current main view of the graphical user interface 300 or 400A to 400J, or all data items within a current data collection.
When the user is satisfied with the report options, the viewer computer can generate the report. The generated report can contain data relating to the data items selected for inclusion in the report, including various attributes. Depending on the type of data items (or data artifacts), the report format may be altered to display the attributes in a relevant fashion. Likewise, data items may be ordered in the report according to the selected order within main view 320 and, in some cases, using the same presentation format as in main view 320. In this manner, a user of the viewer computer can easily produce a report that replicates the data viewable within a graphical user interface such as graphical user interface 300 or 400A to 400J. This can relieve the user of the substantial burden of report preparation for the purposes of evidence submission, where data items must be meticulously documented and catalogued, and where data artifacts must be pieced together from various data items.
As described above, reports can be generated based on a “what you see is what you get” model. For example, reports may adopt a presentation format similar to that displayed in the graphical user interface. Likewise, reports may contain data items corresponding to those currently selected for display in the graphical user interface, in particular based on applied filters (whether implicit or explicit). The viewer computer may automatically generate and report relevant attributes (such as date and time, filesystem location, etc.) according to the columns or attributes displayed in main view 320 or 420.
In particular, viewer computer can generate reports to reflect the filtering, column order, sorting and visibility as selected by the user in the graphical user interface. For example, if a user has selected an artifact view display type with a media category, and a column detail presentation format, then subsequently hidden a software version column in the main view, then viewer computer can map the various user interface elements to the generated report elements, such that the generated report can display only media artifacts, with the software version column omitted.
The present invention has been described here by way of example only, while numerous specific details are set forth herein in order to provide a thorough understanding of the exemplary embodiments described herein. However, it will be understood by those of ordinary skill in the art that these embodiments may, in some cases, be practiced without these specific details. In other instances, well-known methods, procedures and components have not been described in detail so as not to obscure the description of the embodiments. Various modification and variations may be made to these exemplary embodiments without departing from the spirit and scope of the invention, which is limited only by the appended claims.
This application is a continuation of U.S. patent application Ser. No. 15/456,649, filed on Mar. 13, 2017, which claims the benefit of U.S. provisional patent No. 62/306,921 filed Mar. 11, 2016 and entitled SYSTEMS AND METHODS FOR DISPLAYING DIGITAL FORENSIC EVIDENCE, the entire contents of which are hereby incorporated by reference herein for all purposes.
| Number | Date | Country | |
|---|---|---|---|
| 62306921 | Mar 2016 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 15456649 | Mar 2017 | US |
| Child | 18184284 | US |
