Patent Grant
10698805
The present disclosure relates to design verification. More specifically, the present invention relates to method and system for profiling performance of a System on Chip.
Given the typical design complexity of today's advanced systems on chip (SoCs), many engineers validate their SoCs using production software stacks and/or employing basic directed bare-metal software tests. Such efforts, which can be time consuming and/or labor-intensive, often come up short in, for example, coverage of use cases.
There is provided, in accordance with some embodiments of the present invention, a method for debugging a system on chip (SoC) under test. The method may include automatically inserting commands in a test code for testing the SoC for invoking printing of messages of data including start time, end time of each executed action of a plurality of actions designed to be invoked by the test code when testing the SoC, the data further including identity of a processing component of a plurality of processing components of the SoC, on which that action was executed. The method may also include recording the data of the invoked printed messages during testing of the test code on the SoC. The method may further include displaying, via a graphical user interface, one or a plurality of graphical representations, each of said graphical representations relating to a period of activity of one of the plurality of processing components over time, based on the recorded data.
In some embodiments, the method may further include automatically generating the test code.
In some embodiments, the method may further include receiving from a user a selection of a graphical representation of said one or a plurality of graphical presentations, and displaying via the graphical user all actions of said plurality of actions that were performed during the period of activity of the one processing component.
In some embodiments, said all actions may be displayed using representations of time slots, each of the time slots representing a time duration during which each of said all actions was performed.
In some embodiments the time slots are represented by sub-blocks.
In some embodiments the method may further include receiving from the user a selection of an action of said all actions, and displaying via the graphical user interface only that action.
According to some embodiments, that action is displayed using one or more representations of time slots, each of the time slots representing a time duration during which that action was performed.
A system and computer readable medium are also described herein.
The subject matter regarded as the invention is particularly pointed out and distinctly claimed in the concluding portion of the specification. The invention, however, both as to organization and method of operation, together with objects, features, and advantages thereof, may best be understood by reference to the following detailed description when read with the accompanying drawings in which:
In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the invention. However, it will be understood by those skilled in the art that the present invention may be practiced without these specific details. In other instances, well-known methods, procedures, and components have not been described in detail so as not to obscure the present invention.
Although the examples disclosed and discussed herein are not limited in this regard, the terms “plurality” and “a plurality” as used herein may include, for example, “multiple” or “two or more”. The terms “plurality” or “a plurality” may be used throughout the specification to describe two or more components, devices, elements, units, parameters, or the like. Unless explicitly stated, the method examples described herein are not constrained to a particular order or sequence. Additionally, some of the described method examples or elements thereof can occur or be performed at the same point in time.
Unless specifically stated otherwise, as is apparent from the following discussions, it is appreciated that throughout the specification, discussions utilizing terms such as “adding”, “associating” “selecting,” “evaluating,” “processing,” “computing,” “calculating,” “determining,” “designating,” “allocating” or the like, refer to the actions and/or processes of a computer, computer processor or computing system, or similar electronic computing device, that manipulate, execute and/or transform data represented as physical, such as electronic, quantities within the computing system's registers and/or memories into other data similarly represented as physical quantities within the computing system's memories, registers or other such information storage, transmission or display devices.
A system on chip (SoC) can be an integrated circuit that integrates components of a computer or an electronic system into a single chip. In the development process of SoCs, electronic design automation (EDA) tools, e.g., software tools, can be used to design, and/or test (simulate and/or validate) each of the components verification separately, the entire SoC as a whole, or any combination thereof.
One approach to testing of a system on a chip (SoC) can be to exercise the system-level use-cases via the programmer-view using software (SW) driven tests (sometimes referred to as “scenarios”). These tests can be manually written (e.g., in C), e targeting bare-metal hardware without an operating system (OS) and/or on top of some form of production software stack (SS), and/or can require fast platforms such as Field Programmable Gate Array (FPGA) prototyping, emulation, and/or an actual silicon device for the testing.
Tests can be designed to exercise a SoC by, for example, causing the SoC to perform various actions it can be designed to perform. Typically, various components of a SoC may perform a large number of actions, which may only be performed in a permissible order (or “flow”).
Some actions may be performed separately (e.g., turning the power on), some may be performed concurrently with other actions (e.g., displaying video on a screen and playing audio on a loudspeaker). Some actions may only be performed after one or a plurality of other (or same) actions were previously performed and completed (e.g., microphone is turned on only after “record” is pressed). Some actions cannot be performed after one or a plurality of other (or same) actions were previously performed and completed (e.g., the power of the SoC was turned off). The testing expert can write a series of scenarios that can each including a combination of actions that the SoC is capable of performing, in, for example, a permissible order. “Permissible order” can be, for example, possible orders of actions that may be performed by the SoC, and/or that can comply with the constraints governing the operation of the SoC and in particular the performance of actions by the SoC (order of actions, data-flow, etc.).
The tedious manual approach, the quest for quality test content, and/or the lack of a coverage driven flow can be concerns for advanced multi-core SoC providers.
At the sub-system and system integration levels, hardware (HW) verification engineers may deal with more hardware and software interdependencies and may need to adjust as well. In many instances, they may wish to explore a subset of SoC use-cases that can be executed on register transfer level (RTL) simulation and acceleration platforms. A coverage driven random approach that may be useful for IP level may not serve the system-level use-cases well because it typically focuses on verifying all possible combinations, and makes the high-level directed test logic coding error prone.
Furthermore, the cost of transferring use-cases across teams and platforms (from virtual to RTL simulation to acceleration/emulation to FPGA prototype to post-silicon, etc.) is becoming untenable for chip-makers, thereby creating a commercial demand for a solution that allows complex use-case creation and reuse across teams, and that allows for the tests to be executed on all platforms. Portability across platforms not only may improve quality, and may save development time, it may also allow more efficient debugging by offering the flexibility to move a failing test found running on a fast platform with limited visibility to a different platform with more visibility.
The term ‘system-level use-cases’ may be taken, in the context of the present specification in a broad sense to include application-like flows and datapaths, stress tests of interconnect fabrics and memories, and exercising system-level protocols such as cache and input-output (IO) coherency, security, memory virtualization and low-power management. Although very different in nature, there is in fact commonality in the implementation of these.
Consider the following test implementation challenges:
A) Coming up with consistent configuration of multiple IPs and resource allocation: legal pad settings, DMA channel allocations, interrupt lines, video pipes, etc.
B) Scheduling and synchronizing concurrent activity: (a) on multiple HW devices using a single embedded core; (b) across multiple embedded cores; (c) between embedded cores, testbench transactors, and host executed logic.
C) Handling crosscutting activity on the very same cores, such as power-down or reset events during memory or cache data flow operations.
D) Covering complex state-spaces like coherency protocol state-machine or multi-domain low-power specifications.
E) Creating and coordinating multiple test artifacts (other than the embedded code) such as bus functional model (BFM) commands, expected data files, and compilation/run script.
Most if not all of the abovementioned challenges may come into play in the implementation of different kinds of system use-cases. On top of these, test implementation may be aimed at addressing other concerns, such as multiple platforms, adequate checking strategy, debug, maintenance, and functional coverage for completion measurement.
In terms of test creation, users may write a single-thread program, and even come-up with a Perl-like automation around it. Talented programmers may even write use-cases involving two or more cores, while carefully orchestrating system states, synchronization points, and latencies throughout the execution. But this is not scalable. It does not scale with complexity of the required scenarios, being longer, and involving more cores, more sophisticated IPs, more power domains. It does not scale to the amount of different parameter variations that need to be tested for each use-case. And it cannot be maintained and adapted through the changes of underlying designs and SW APIs, as well as the differences in execution platforms.
To fully address SoC-level verification, a solution may be characterized as allowing not only for vertical (IP to SoC) reuse and horizontal (cross-platform) reuse, but also for providing a way to capture and share use-cases. Reuse should preferably work both ways, as sub-system teams wish to leverage the full system context and use-cases and wish become more productive in their testing.
A system and method for system verification (hereinafter also referred to as “system verifier”) was introduced that is a use-case based verification tool tailored for software driven verification automation. The system verifier may offer a model-based approach to describe system level actions and resources that can be composed into subsystem and system level abstract use-cases. The system verifier can automatically generate SoC verification tests, rather than depend on a human testing expert to write up these tests.
The automatic generation of SoC verification tests may include receiving from a user (e.g., a testing expert) a list of all actions the SoC to be tested may perform and constraints relating to the permissible order of these actions.
The system verifier may employ a constraint solver to analyze abstract use-cases, find permissible solutions or concrete use-cases that fulfil the goals represented in the abstract use case, and ultimately generate concrete permissible (also referred to as “legal”) tests, in the form of a code in a language suitable for the target to be tested (“target language”).
Thus, the SoC may be tested using generated permissible tests.
In some embodiments of the present invention, the permissible tests may be presented in the form of graphs, and may consequently be generated in the form of executable files that may run to operate the SoC in accordance with the test scenario. A testing expert may take graphs representing the automatically generated tests and amend them. For example, the testing expert may amend the tests by adding or removing actions or changing the order of the actions (provided the new order is permissible too).
Thus, generating SoC tests became more efficient and less time-consuming.
However, the testing expert is still left with a tedious task of debugging. Since verification tests are carried out on an SoC, conventional debuggers may not be suited for the job, as actions on the SoC are carried out at a run-time speed, unlike device-under-test (DUT) code simulations. In these circumstances, it may not be possible to use conventional debugging tool.
There may be various kinds of bugs that occur when testing a SoC. A correctness bug may occur, for example, when an action causes an unexpected value to be returned. A performance bug may occur, for example, when an action fails to execute, or takes too long to be completed. Another example of a performance bug may be failure of a consecutive action to commence despite a successful completion of all action that are required to be completed before that action is performed.
There can be many reasons for the occurrence of performance bugs. Here are some examples:
a, an action included in the test (scenario) executed on the SoC failed to execute, because of a missing action—an action that was required to execute, in accordance with the design requirements of the SoC, before the failed action;
b, an action failed to execute, despite completion of execution of all the actions that were required to execute prior to execution of the failed action;
c, an action took much too long to execute than was anticipated;
d, an entire set of actions took much too longer to execute than was anticipated.
There are, of course other performance bugs that may occur during execution of tests on a SoC.
According to some embodiments of the present invention, a new method for debugging a SoC and a debugging system are introduced.
According to some embodiments, a system verifier may be used to generate a test code for testing the SoC.
The test code may include commands designed to invoke a plurality of actions, each of the actions to be performed on a processing component of a plurality of processing components of the SoC. The test code may also include commands for invoking printing of messages of data including start time, end time of each executed action of said plurality of actions and identity of a processing component of the plurality of processing components of the SoC, on which that action was executed.
In some embodiments of the invention the system verifier may automatically insert commands in a test code for testing the SoC for invoking printing of messages of data including start time, end time of each executed action of a plurality of actions designed to be invoked by the test code when testing the SoC, the data further including identity of a processing component of a plurality of processing components of the SoC, on which that action was executed. The commands invoking the printing of the messages may be automatically included in the test code, either when the test code is generated or later.
The data of the invoked printed messages during testing of the test code on the SoC may be recorded (e.g., by the system verifier, or by another device) and saved on a memory (volatile or non-transitory memory).
The recorded data may include the actions that were actually performed (distinct identification of each action, for example by a unique number) by the SoC when the test was executed, the recording including, for example, timing of these actions (e.g., start time, end time), data that was output, distinct identification of the processing component on which that action was executed, and any other output event. “Start time’ and “end time” may also refer to “resume time” and “pause time”, respectively, in the sense that the start time of the action may relate to the time when a paused action was resumed, and the end time of the action may relate to the time that action was paused. Typically, in order to send a data packet and comply with an internal-protocol, the action manager notifies the internet action controller of a packet ready to be sent and also notifies when the action has been completed, thereby the action manager initiates start-run and then waits until a notification is received that confirms that the sub-actions are completed and then resumes execution until reaching end-run. The “processing component” refers, in the context of the present invention to any component of the SoC, which is designed to process data. For example, a central processing unit (CPU), graphical processing unit (GPU), any core of a CPU or a GPU, etc. The “identification” of the processing component refers to a unique distinct identity (e.g., a unique number), so as to distinguish each processing component from the other processing components of the SoC.
According to some embodiments of the invention, the recorded data may be retrieved by logging messages for each executed action with the desired data. The messages may be generated automatically, by planting message commands in the code of the tests by the system verifier.
The recorded output may be saved in a log file or log files. The log file may be saved in any of a variety of forms (e.g., a text list, database, etc.). In some embodiments, the output of the SoC for each test is recorded and saved separately (e.g., in a separate log file). Each action is uniquely assigned (e.g., in the log) identification (ID), e.g., ID number or tag, to distinctly identify it and distinguish it from other actions (including from other occurrences of the same action). Each processing component is also uniquely assigned (e.g., in the log) identification (ID), e.g., ID number or tag, to distinctly identify it and distinguish it from other processing components of the SoC.
A system verifier (may also be referred to as “system profiler”), according to some embodiments of the present invention, may include a graphical interface which may be displayed on a display device to the testing expert.
GUI screen 100 may include one or a plurality of frames in which data is displayed to the testing user. In some embodiments, GUI screen 100 includes frame 102 in which a graph 104, which is an abstract representation of a verification test that was run on the SoC, is presented. Graph 104 may include the various actions that formed that particular test, such as, for example “read” 106 and “write” 108. These actions appear to be actions that may be performed simultaneously, for they are located at parallel nodes of graph 104. In the present example of
GUI screen 100 further includes frame 110, in which the output log 112 (the recorded output of the executed test—the same test whose abstract graph representation 104 appears in frame 102 on SoC) is presented. In the present example, the output log 112 is in the form of a list of lines of messages corresponding to the recorded actions, each message corresponding to a single action, priority status of the message (e.g., F for “full”, H for “high”, M for “Medium” and L for “Low”), the timing of that action, and the message itself. Output log 112 may include implicit output information resulting from the execution of the code of the test, and explicit output information that was added by the user.
GUI screen 100 also includes frame 116, in which a graphical representation of the occurrence of each action versus time, in a waveform, is presented. The actions 118 appearing in this waveform are shown in the form of blocks whose length is determined by their performance duration, and their position on the waveform is determined by their timing. Thus, for example it is evident that action “read” 106 of graph 104 (in frame 102) is represented by block 120 in frame 116, while action “write” 108 of graph 104 is presented by block 122 in that frame. It is also evident from this GUI that actions 106 and 108 were executed substantially concurrently, as their block representations (120 and 122 respectively) appear to commence at the same time, as suggested by their aligned (one on top of the other) start positions in the waveform, whereas the “read” action ends slightly before the “write” action.
GUI screen 100 may further include additional information (e.g., memory image, relating to the memory of the SoC, and other data), that is presentable to the testing expert on a display device.
The user may select a position on the waveform (e.g., by clicking a pointing device over that location), and a corresponding location indicator 124 or 126 may be presented with information on the selected location (e.g., the timing corresponding to that location within the test execution, or other location data).
In some embodiments of the present invention, the user may select a line of the output log 112, for example by clicking on that line 114, thereby causing that line to appear highlighted. The system verifier may be designed to react to such selection of the line of the output log, by highlighting (or otherwise indicating) the selected action in the abstract graph 104, and/or by highlighting the block on the waveform presented in frame 116 that corresponds to the selected action. This allows a testing expert to quickly and easily track and trace actions and their resulting output as appearing on the GUI screen 100.
The waveform feature of the SoC system verifier, according to some embodiments of the present invention, allows the testing expert to immediately spot irregularities or abnormalities in the execution of the SoC test. For example, the testing expert may spot on the waveform a certain action that was supposed to be performed over a short period of time but lasted longer (or too much linger). The testing expert may identify actions that where not supposed to be performed at the timing of their actual performance, or were not supposed to be performed at all. Thus, the testing expert is provided with a graphical visualization of the executed test and its results, presenting go-to-cause debug information, making it simpler for the testing expert to find bugs and identify their causes.
According to some embodiments of the present invention, the testing expert may select a line 113 in the output log 112 presented in frame 110, highlighting that line. Consequently, action 105 in graph 104, which correlates to the selected line (the action in that line and the highlighted action in graph 104 correspond—it is the same action) is also highlighted, and so is the corresponding line 206 in the script code 204, presented in frame 202.
The linking of the various representations on the same action in the code script, in the output log and on the waveform may easily be performed. One simple way of doing that is described hereinafter:
In the generation of the test code, the generator of the code plants messages (e.g., remarks) in the generated test code, assigning the start and end of each action with a unique ID (e.g., number). All starts and ends of actions are identified by a distinctly unique ID (e.g., the start of action a1 is assigned ID number 1, the end of action a1 is assigned ID number 2, the start of action a2 is assigned ID number 3, the end of action a2 is assigned ID number 4, and so on).
A reference file, in which all starts and ends of all actions in the test are saved with their corresponding IDs, is provided.
When the test is executed, the messages appear in the output log, allowing identification of the start and end of each action performed during the test, and association of these start and end of each action with the start and end of actions in the output log. The graphical waveform representation may be generated based on this identification and association, and the links between the various representations of each action may be established.
A SoC may include a plurality of processing components, such as, for example, one or more central processing units (CPUs), each including one or a plurality of cores, etc. Some actions may be performed only on a particular component or particular components, whereas other actions may be performed on different component or components. In some cases one or some of the components of the SoC may be more heavily used than other component, for various reasons.
Referring back to
Frame 500 displays the activity of processing components C1-C11 of a SoC. For example, C1-C4 may be four cores of a first CPU. C5-C8 may be four cores of a second CPU, and C9-C11 may be three cores of a GPU. The vertical axis represents time (e.g., units of milliseconds, femtoseconds, nanoseconds, etc.). The activity of the processing components is represented by blocks, whose ends correspond to the start time and end time of continued activity of that component. In the example of
This graphical presentation may be built based on knowing that while an action may be performed on one or more processing components of a system, once an instance of an action is initiated, it is executed from start to end on a single processing component, and on knowing the start time and end time for action, pause and resume times of an action (if its processing by the processing component was paused, for example, when swapping in and out with another action, or when waiting for a required condition to occur) and the identity of the processing component of which that action was performed and accumulating the activity times of each processing components of the system. This information is presented in form of a graphical presentation showing blocks of time during which each processing component was active. Furthermore, an action may be processed in several active spells on a processing component, in which case the start time and end time for each spell may be recorded, so that a full picture of the activity of processing components of the SoC may be obtained.
It is evident from the graphical representation that C1, for example, was active for a period of time during which C2 was idle at first, then active for a shorter period, idle again and active for the remaining time and even some time after C1 ended its activity and became idle. C3 was idle at first, then became active, while C2 was already active too, and ended its activity before C1 ended its activity, and while C2 was in its second period of activity, and so on. Given that the debug expert knows what is expected form the SoC, knows which actions were included in the test that was executed on the SoC, a graphical presentation of the activity of the processing components of the SoC is a useful tool that may aid the debug expert in identifying faults and bugs.
In the left hand side frame 502, the activity of processing components C1-C4 is displayed. It is evident that processing component C1 was active much longer then processing components C2-C4, whereas processing components C2-C4 where active for substantially shorter successive periods of time. This may lead the debug expert to want to further investigate the behavior of C1 and the action or actions that were executed on it.
In the central frame 504, processing components C5-C7 were concurrently active, but C5 was active much longer than the other processing components, and only when processing component C5 ended its activity did processing component C8 commence its activity. The debug expert may find it interesting to investigate what action or actions were performed by processing component C5, and also determine whether the action performed by C5 were required by the action or actions performed on C8 to be executed.
In the right hand side frame 506, processing components C9 and C10 performed actions and processing component C11 followed, but after a substantially long period of inactivity. This may lead the debug expert to investigate the reason for this pause. A reason for such a pause may be a missing action that was required by the design of the SoC to allow the next action to be executed. In order to amend this a test may be generated that includes the missing action.
According to some embodiments of the present invention, a user of the system verifier may select a specific period of activity of one of the processing components of the SoC, for example by clicking on the block presenting the desired active time of a that processing component (e.g., block 510 relating to processing component C1, see
Frame 539 may be presented, via the user interface, displaying the selected activity block of a particular processing components (in this example, block 510 of processing component C1, shown in
In some embodiments a single action may be represented by two or more sub-blocks. This is so when an action is performed in separate sessions by a processing component, for example, when performing several actions whose performance is shared on the same processing component (swapping in and out) or when an action is paused, waiting until a required condition is met.
Thus, for example, in the representation of the activity block of
According to some embodiments of the present invention, the user may select, on a presentation of an activity block that displays sub-blocks pertaining to the actions performed on the processing component during the activity time of the selected block, a specific action by clicking or otherwise selecting that action and that action—that is the sub-block or sub-blocks pertaining to the time period or periods during which that action was performed—will be presented uniquely, for example, on a separate frame.
The graphical arrangements of the various representations described hereinabove and shown in the appended figures are given as examples. Other graphical presentations and arrangements may be chosen.
The operation of SoC generates during the test is sensed, and the sensed data 608 recorded and provided to processor 610, where the received data is processed and links between the invoked actions by the test code and the actual actions performed during the test are established, and processing components of the SoC that were active and on which the actions were executed.
System 700 may include a processing unit 702 (e.g., one or a plurality of processors, on a single machine or distributed on a plurality of machines) for executing a method according to some embodiments of the present invention.
System 700 may include an input interface 701 for receiving data and instructions from a user, such as, for example, one or a plurality of keyboards, pointing devices, touch sensitive surfaces (e.g. touch sensitive screens), etc. for allowing a user to input commands and data. Processing unit 702 may be linked with memory 706 on which a program implementing a method according to some embodiments of the present invention and corresponding data may be loaded and run from and data may be saved, and storage device 708, which includes a non-transitory computer readable medium (or mediums) such as, for example, one or a plurality of hard disks, flash memory devices, etc. on which a program implementing a method according to some embodiments of the present invention and corresponding data may be stored. System 700 may further include an output device 704 (e.g., display device such as CRT, LCD, LED, etc.) on which one or a plurality user interfaces associated with a program implementing a method according to some embodiments and corresponding data may be presented.
In some embodiments of the present invention, system 600 and system 700 may be unified. In some other embodiments, these systems may be separate.
In some embodiments the method may also include recording 804 the data of the invoked printed messages during testing of the test code on the SoC. The method may also include displaying 806, via a graphical user interface, one or a plurality of graphical representations, each of said graphical representations relating to a period of activity of one of the plurality of processing components over time, based on the recorded data.
Some embodiments of the present invention may be embodied in the form of a system, a method or a computer program product. Similarly, some embodiments may be embodied as hardware, software or a combination of both. Some embodiments may be embodied as a computer program product saved on one or more non-transitory computer readable medium (or media) in the form of computer readable program code embodied thereon. Such non-transitory computer readable medium may include instructions that when executed cause a processor to execute method steps in accordance with examples. In some examples the instructions stores on the computer readable medium may be in the form of an installed application and in the form of an installation package.
Such instructions may be, for example, loaded by one or more processors and be executed.
For example, the computer readable medium may be a non-transitory computer readable storage medium. A non-transitory computer readable storage medium may be, for example, an electronic, optical, magnetic, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof.
Computer program code may be written in any suitable programming language. The program code may execute on a single computer system, or on a plurality of computer systems.
While certain features of the invention have been illustrated and described herein, many modifications, substitutions, changes, and equivalents will now occur to those of ordinary skill in the art. It is, therefore, to be understood that the appended claims are intended to cover all such modifications and changes as fall within the true spirit of the invention.
| Number | Name | Date | Kind |
|---|---|---|---|
| 5325309 | Halaviati | Jun 1994 | A |
| 5371851 | Pieper | Dec 1994 | A |
| 5920490 | Peters | Jul 1999 | A |
| 5940604 | Merryman | Aug 1999 | A |
| 5960171 | Rotman | Sep 1999 | A |
| 5991533 | Sano | Nov 1999 | A |
| 6163763 | Cox | Dec 2000 | A |
| 6230114 | Hellestrand | May 2001 | B1 |
| 6263301 | Cox | Jul 2001 | B1 |
| 6263302 | Hellestrand | Jul 2001 | B1 |
| 6336199 | Lin | Jan 2002 | B1 |
| 6457152 | Paley | Sep 2002 | B1 |
| 6539522 | Devins | Mar 2003 | B1 |
| 6571373 | Devins | May 2003 | B1 |
| 6751565 | Naroska et al. | Jun 2004 | B2 |
| 6789242 | Liu | Sep 2004 | B1 |
| 6829751 | Shen et al. | Dec 2004 | B1 |
| 6845341 | Beverly | Jan 2005 | B2 |
| 7080283 | Songer | Jul 2006 | B1 |
| 7185295 | Park et al. | Feb 2007 | B2 |
| 7437692 | Oberlaender | Oct 2008 | B2 |
| 7451070 | Devins | Nov 2008 | B2 |
| 7500046 | Puri | Mar 2009 | B1 |
| 7571400 | Park et al. | Aug 2009 | B2 |
| 7823017 | Bueti | Oct 2010 | B2 |
| 8073820 | Srinivasan et al. | Dec 2011 | B2 |
| 8176296 | Snyder | May 2012 | B2 |
| 8560893 | Hollander | Oct 2013 | B1 |
| 8639487 | Ezer | Jan 2014 | B1 |
| 8732632 | Keller | May 2014 | B1 |
| 8781808 | Yang | Jul 2014 | B2 |
| 8832615 | Hsieh et al. | Sep 2014 | B2 |
| 8869093 | Elliott | Oct 2014 | B2 |
| 8881114 | Sohm | Nov 2014 | B2 |
| 9081824 | Becker et al. | Jul 2015 | B2 |
| 9081924 | Chen et al. | Jul 2015 | B2 |
| 9280627 | Zhang | Mar 2016 | B1 |
| 9286180 | Musunuri | Mar 2016 | B1 |
| 9304880 | Olivarez | Apr 2016 | B2 |
| 9316689 | Hamid | Apr 2016 | B2 |
| 9582406 | Teplitsky et al. | Feb 2017 | B1 |
| 9606888 | Wiesner | Mar 2017 | B1 |
| 9734263 | Kohavi | Aug 2017 | B2 |
| 10042729 | Pappu | Aug 2018 | B2 |
| 20020147560 | Devins | Oct 2002 | A1 |
| 20030056163 | Rajsuman | Mar 2003 | A1 |
| 20030154063 | Lu et al. | Aug 2003 | A1 |
| 20040010401 | Davis | Jan 2004 | A1 |
| 20040017219 | Han | Jan 2004 | A1 |
| 20050102572 | Oberlaender | May 2005 | A1 |
| 20060190871 | Likovich, Jr. | Aug 2006 | A1 |
| 20060229858 | Devins | Oct 2006 | A1 |
| 20060259823 | Sohm | Nov 2006 | A1 |
| 20060259824 | Sohm | Nov 2006 | A1 |
| 20060259827 | Sohm | Nov 2006 | A1 |
| 20060259831 | Sohm | Nov 2006 | A1 |
| 20070006173 | Sohm | Jan 2007 | A1 |
| 20070011527 | Goswami | Jan 2007 | A1 |
| 20070033552 | Li | Feb 2007 | A1 |
| 20070260952 | Devanathan | Nov 2007 | A1 |
| 20080034334 | Laouamri | Feb 2008 | A1 |
| 20080127216 | Bueti | May 2008 | A1 |
| 20080147372 | Paulsen | Jun 2008 | A1 |
| 20080183457 | Klein | Jul 2008 | A1 |
| 20080184150 | Minato et al. | Jul 2008 | A1 |
| 20080215923 | Bueti | Sep 2008 | A1 |
| 20100023807 | Wu | Jan 2010 | A1 |
| 20100031209 | Luan | Feb 2010 | A1 |
| 20100095255 | Thompson et al. | Apr 2010 | A1 |
| 20100223035 | Weyh | Sep 2010 | A1 |
| 20110087813 | Clark | Apr 2011 | A1 |
| 20110238397 | Chen | Sep 2011 | A1 |
| 20110283247 | Ho | Nov 2011 | A1 |
| 20110289373 | Klein et al. | Nov 2011 | A1 |
| 20120072199 | Kawahara | Mar 2012 | A1 |
| 20120159251 | Wu | Jun 2012 | A1 |
| 20130041645 | Kawata | Feb 2013 | A1 |
| 20140157073 | Douskey | Jun 2014 | A1 |
| 20140181767 | Kohavi | Jun 2014 | A1 |
| 20140201584 | Tekumalla | Jul 2014 | A1 |
| 20140237292 | Chan | Aug 2014 | A1 |
| 20140281735 | Olivarez | Sep 2014 | A1 |
| 20150178138 | Saha | Jun 2015 | A1 |
| 20150301108 | Hamid | Oct 2015 | A1 |
| 20150331043 | Sastry | Nov 2015 | A1 |
| 20160054388 | Tamiya | Feb 2016 | A1 |
| 20160070846 | Nandini | Mar 2016 | A1 |
| 20160084906 | Tamiya | Mar 2016 | A1 |
| 20160098332 | Liu | Apr 2016 | A1 |
| 20160232264 | Oh | Aug 2016 | A1 |
| 20160282413 | Tamiya | Sep 2016 | A1 |
| 20160364308 | Shanbhogue | Dec 2016 | A1 |
| 20170082687 | Klinger | Mar 2017 | A1 |
| 20170139007 | Mueller-Schniek | May 2017 | A1 |
| 20170184662 | Varadarajan | Jun 2017 | A1 |
| 20170227603 | Hamid | Aug 2017 | A1 |
| 20170286247 | Pappu | Oct 2017 | A1 |
| Entry |
|---|
| Meiyappan Nagappan et al., Creating operational profiles of software systems by transforming their log files to directed cyclic graphs, May 23, 2011, [Retrieved on Dec. 20, 2019]. Retrieved from the internet: <URL: https://dl.acm.org/citation.cfm?id=1987869> 4 Pages (54-57) (Year: 2011). |
| Vitaly Zakharenko et al., Characterizing the performance benefits of fused CPU/GPU systems using FusionSim, Mar. 18-22, 2013, [Retrieved on Dec. 20, 2019]. Retrieved from the internet: <URL: https://dl.acm.org/citation.cfm?id=2485455> 4 Pages (685-688) (Year: 2013). |
| Non-final office action of U.S. Appl. No. 14/755,071, dated Mar. 24, 2017. |
| Notice of Allowance of U.S. Appl. No. 14/755,071, dated Jul. 25, 2017. |
