Claims
- 1. A computer-implemented method of analyzing the performance of a server, the method comprising:
monitoring a first performance metric of the server over a period of time to generate a series of values of the first performance metric, wherein the server system responds to requests from clients during said period of time; monitoring a second performance metric of the server system over said period of time to generate a series of values of the second performance metric; and measuring a degree to which the first performance metric and the second performance metric are related at least in-part by calculating a correlation coefficient between the series of values of the first performance metric and the series of values of the second performance metric.
- 2. The method of claim 1, wherein the correlation coefficient is a Pearson correlation coefficient.
- 3. The method of claim 1, wherein the correlation coefficient is a coefficient of determination.
- 4. The method of claim 1, further comprising comparing the correlation coefficient to an upper threshold to evaluate whether the first and second performance metrics represent redundant information.
- 5. The method of claim 1, further comprising comparing the correlation coefficient to a lower threshold to evaluate whether the first and second performance metrics are to be treated as unrelated.
- 6. The method of claim 1, further comprising determining whether a meaningful relationship exists between the first and second performance metrics at least in part by comparing the correlation coefficient to an upper threshold and a lower threshold.
- 7. The method of claim 1, further comprising compensating for a time offset that exists between the series of values of the first performance metric and the series of values of the second performance metric.
- 8. The method of claim 7, further comprising detecting said time offset by identifying corresponding but offset transition points in the series of values of the first performance metric and the series of values of the second performance metric.
- 9. The method of claim 1, further comprising applying a re-sampling method to the series of values of the first performance metric to facilitate a comparison with the series of values of the second performance metric.
- 10. The method of claim 1, wherein the first performance metric is a server response time.
- 11. The method of claim 10, wherein the second performance metric represents a current load on the server.
- 12. The method of claim 10, wherein the second performance metric represents server transactions executed per unit time.
- 13. The method of claim 1, further comprising applying a controlled load to the server during said period of time.
- 14. A computer system programmed to perform the method of claim 1.
- 15. A computer program that embodies the method of claim 1 represented in a computer-readable medium.
- 16. A computer-implemented method for analyzing the performance of a server, the method comprising:
monitoring a plurality of performance metrics of the server over a period of time to generate a collection of performance metric data values; performing a sampling analysis of the collection of performance metric data values to identify a significant portion thereof; and analyzing performance metric data values within the significant portion to measure, for at least one pair of said performance metrics, a degree to which the pair of performance metrics are correlated.
- 17. The method of claim 16, wherein monitoring the plurality of performance metrics comprises storing each performance metric data value in association with a time index.
- 18. The method of claim 16, wherein the plurality of performance metrics include performance metrics of at least the following types: (a) server transactions executed per unit time, (b) transaction response time.
- 19. The method of claim 16, wherein the pair of performance metrics includes a first performance metric that represents a response time of the server and a second performance metric that represents a load on the server.
- 20. The method of claim 16, wherein performing the sampling analysis comprises separately analyzing a series of data values of each performance metric.
- 21. The method of claim 16, wherein performing the sampling analysis comprises evaluating whether collected performance metric data values exhibit a statistically significant trend.
- 22. The method of claim 16, wherein performing the sampling analysis comprises determining whether a performance metric is uninformative.
- 23. The method of claim 16, wherein performing the sampling analysis comprises determining whether a sufficient number of data values have been collected for a given performance metric to perform a statistically significant analysis.
- 24. The method of claim 16, wherein performing the sampling analysis comprises identifying at least one segment of time for which the performance metric data values are sufficient for performing a statistically meaningful correlation analysis.
- 25. The method of claim 16, further comprising dividing the collection of performance metric data values into multiple significant segments based on said sampling analysis, each segment representing a different window of time.
- 26. The method of claim 16, further comprising identifying to a user a segment in which the performance metric data values are significant, and providing to the user an option to perform an automated correlation analysis on the segment.
- 27. The method of claim 16, wherein analyzing the performance metric data values comprises calculating a correlation coefficient that represents a degree to which the pair of performance metrics are correlated.
- 28. The method of claim 27, wherein the correlation coefficient is a Pearson correlation coefficient.
- 29. The method of claim 27, wherein the correlation coefficient is coefficient of determination.
- 30. The method of claim 27, further comprising determining whether a meaningful relationship exists between the pair of performance metrics at least in part by comparing the correlation coefficient to upper threshold and a lower threshold.
- 31. The method of claim 16, further comprising detecting and compensating for a time offset that exists between the pair of performance metrics prior to measuring the degree to which the pair of performance metrics are correlated.
- 32. The method of claim 16, further comprising applying a re-sampling method to a series of data values of a performance metric of said pair.
- 33. The method of claim 16, further comprising applying a controlled load to the server during said period of time.
- 34. A computer system programmed to perform the method of claim 16.
- 35. A computer program that embodies the method of claim 16 represented in a computer-readable medium.
- 36. A system for facilitating the analysis of the operation of a server, the system comprising:
a data collection component that collects data values of each of a plurality of performance metrics reflective of the performance of the server; and an analysis component that analyses the data values at least in-part by generating correlation coefficients for specific pairs of the performance metrics to identify performance metrics that are related.
- 37. The system of claim 36, wherein the analysis component evaluates whether a meaningful relationship exists between a pair of performance metrics at least in-part by comparing a correlation coefficient associated with the pair to a lower threshold value and an upper threshold value.
- 38. The system of claim 36, wherein the analysis component calculates Pearson correlation coefficients for the pairs of performance metrics.
- 39. The system of claim 36, wherein the analysis component calculates correlation coefficients of determination for the pairs of performance metrics.
- 40. The system of claim 36, wherein the analysis component uses a clustering algorithm to group together performance metrics that are deemed related.
- 41. The system of claim 36, wherein the analysis component generates a tree indicating performance metrics that, based on a clustering analysis, are deemed to demonstrate similar behavior.
- 42. The system of claim 36, wherein the analysis component detects and compensates for offsets in time between transition points in a pair of performance metrics.
- 43. The system of claim 36, wherein the analysis component performs a sampling analysis on a series of the data values to identify significant portions within said series.
- 44. The system of claim 36, wherein the data collection component associates a time value with each data value of each performance metric.
- 45. The system of claim 36, wherein the plurality of performance metrics include performance metrics indicative of at least: (a) server transactions executed per unit time, and (b) server response time.
- 46. The system of claim 36, further comprising a display component that identifies to a user segments of performance metric data values that are deemed significant, and provides an option for the user to select a segment on which to perform an automated correlation analysis.
- 47. The system of claim 36, further comprising a load generation component that applies a controlled load to the server during collection of performance metric data values by the data collection component.
- 48. A system for facilitating the analysis of the operation of a server, the system comprising:
a data collection component that collects data values of each of a plurality of performance metrics reflective of the performance of the server over a period of time to generate a collection of performance metric data values; and an analysis component that analyses the collection of performance metric data values at least in-part by a performing a sampling analysis to identify significant portions thereof, and by analyzing the significant portions to measure degrees to which specific pairs of performance metrics are correlated.
- 49. The system of claim 48, wherein the data collection component associates each performance metric data value with a time index.
- 50. The system of claim 48, wherein the analysis component performs the sampling analysis at least in-part by separately analyzing a series of data values of each performance metric.
- 51. The system of claim 48, wherein the analysis component performs the sampling analysis at least in-part by evaluating whether collected performance metric data values exhibit a statistically significant trend.
- 52. The system of claim 48, wherein the analysis component performs the sampling analysis at least in-part by determining whether a performance metric is uninformative.
- 53. The system of claim 48, wherein the analysis component performs the sampling analysis at least in-part by determining whether a sufficient number of data values have been collected for a given performance metric to perform a statistically significant analysis.
- 54. The system of claim 48, wherein the analysis component performs the sampling analysis at least in-part by identifying at least one segment of time for which the performance metric data values are sufficient for performing a statistically meaningful correlation analysis.
- 55. The system of claim 48, wherein the analysis component further re-samples a series of data values of a first performance metric to facilitate a comparison with a series of data values of a second performance metric.
- 56. The system of claim 48, wherein the analysis component further detects and compensates for a time offset between a pair of performance metrics.
- 57. The system of claim 48, wherein the analysis component analyzes the significant portions at least in-part by generating correlation coefficients for specific pairs of the performance metrics.
- 58. The system of claim 57, wherein the correlation coefficients include Pearson correlation coefficients.
- 59. The system of claim 57, wherein the analysis component evaluates whether a meaningful relationship exists between a pair of performance metrics at least in-part by comparing a correlation coefficient associated with the pair to a lower threshold value and an upper threshold value.
- 60. The system of claim 48, wherein the plurality of performance metrics include performance metrics indicative of at least: (a) server transactions executed per unit time, and (b) transaction response time.
- 61. The system of claim 48, further comprising a load generation component that applies a controlled load to the server during collection of performance metric data values by the data collection component.
RELATED APPLICATIONS
[0001] This application is a continuation of U.S. Appl. No. 10/214,421, filed Aug. 6, 2002, which claims priority benefit under 35 U.S.C. §119(e) from U.S. Provisional Application No. 60/310,724, Aug. 6, 2001, entitled “SYSTEM AND METHOD FOR AUTOMATED ANALYSIS OF LOAD TESTING RESULTS.” The technical disclosure of the aforesaid applications is hereby incorporated by reference.
Provisional Applications (1)
|
Number |
Date |
Country |
|
60310724 |
Aug 2001 |
US |
Continuations (1)
|
Number |
Date |
Country |
Parent |
10214421 |
Aug 2002 |
US |
Child |
10672445 |
Sep 2003 |
US |