1. Field of the Invention
This invention relates generally to the field of data processing systems. More particularly, the invention relates to a system and method for configuring tracing and logging functions within an enterprise network.
2. Description of the Related Art
Logging is employed within virtually all data networks. “Logging” refers generally to recording network-related and/or application-related information in response to one or more predefined network/application events. For example, when an end-user opens a TCP connection to a server, or unsuccessfully attempts to gain access to network resources (e.g., by attempting to log in to a particular server), this information is typically recorded as an entry within a log file. Similarly, if a variable within an application rises above a specified threshold value, a log entry indicating the value and the date and time that the threshold value was exceeded may be stored within a log file. Logging techniques may be employed to record any specified network/application event. Network administrators may then review the log files to identify security issues and/or troubleshoot network problems.
Logging functionality is provided within the Java™ 2 Standard Edition (“J2SE™”) platform and the Java 2 Enterprise Edition “J2EE™” platform. Referring to
Each logger 110, 112, 114 may have a threshold “Level” associated with it which reflects a minimum defined logging value (e.g., priority level) that the logger cares about. If a logger's level is set to null, then its effective level is inherited from its parent, which may in turn obtain it recursively from its parent, and so on up the tree.
In response to logging calls from applications 101, the logger objects 110, 112, 114 allocate Log Record objects which are passed to handler objects 130 for publication. For example, a first type of handler object may write log records to an output stream, a second type of handler object may write log records to a file (or to a set of rotating log files) and a third handler may write log records to remote TCP ports. Developers requiring specific functionality may develop a handler from scratch or subclass one of the handlers in J2SE.
Both loggers 110, 112, 114 and handlers 130 may use filters 120, 121 to filter out certain designated types of log records. In addition, when publishing a log record externally, a handler may optionally use a formatter 122 to localize and format the message before writing it to a particular destination. For example, J2SE includes a “simple formatter” for writing short “human-readable” summaries of log records and an XML formatter for writing detailed XML-structured information.
“Tracing” is a technique used primarily by software developers to track the execution of program code. For example, when developing an application, developers trace the execution of methods or functions within certain modules to identify problems and/or to determine if the program code may be improved. If a particular method takes an inordinate amount of time to complete, the developer may determine the reasons why and/or change the program code to operate more efficiently.
Developers use trace tools to trace the execution of program code. Trace tools are proprietary application programs which use different techniques to trace the execution flows for an executing program. One technique, referred to as event-based profiling, tracks particular sequences of instructions by recording application-generated events as they occur. By way of example, a trace tool may record each entry into, and each exit from, a module, subroutine, function, method, or system component within a trace file (e.g., a time-stamped entry may be recorded within the trace file for each such event). Trace events may also be sent to a console or other output destination.
Thus, tracing and logging techniques rely on similar event-based triggers, employ similar messaging techniques and record log/trace events to similar output destinations (e.g., trace/log files, consoles, . . . etc) in a substantially similar manner. As such, it would be beneficial to develop an integrated application programming interface which takes advantage of the similarities of tracing and logging operations, and of the synergistic effects of handling both, while not neglecting the differences.
An integrated tracing and logging system for an enterprise network is described. One embodiment of the integrated logging and tracing system has an object-oriented architecture which includes a controller class with two sub-classes: a tracing sub-class and a logging sub-class. Instances of the tracing sub-class (tracing modules) are associated with specified program code regions of network applications. The tracing modules receive method calls from the network applications and process the method calls based on defined severity levels. Instances of the logging sub-class (logging modules) are associated with specified “categories” related to the enterprise network (e.g., system, database, etc). The logging modules receive and process method calls from network components associated with the categories. The integrated logging and tracing system provides a unique architecture for configuring the logging and tracing modules using minimum/maximum severity settings and inheritance.
A better understanding of the present invention can be obtained from the following detailed description in conjunction with the following drawings, in which:
a-b illustrate severity levels associated with tracing and logging controllers according to one embodiment of the invention.
Described below is an integrated framework and method which supports both logging and tracing functions. Also described is a system and method for configuring logging and tracing functions using inheritance. Throughout the description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced without some of these specific details. In other instances, well-known structures and devices are shown in block diagram form to avoid obscuring the underlying principles of the present invention.
A system architecture according to one embodiment of the invention is illustrated in
In one embodiment, each controller 200 is an instance of a defined “controller” class (e.g., a Java class) which includes two sub-classes, a “tracing” sub-class and a “logging” sub-class (described in detail below), which provide features specific to tracing and logging operations, respectively. In an object-oriented environment such as Java, the tracing controller 202 illustrated in
The Controller Class
The controller class provides methods for associating log/trace output destinations with specific controllers 200 and for controlling the actual writing of log/trace messages. When a method is called, the writing of the log/trace message to a log/trace file 220, console 221 or other output destination 214 depends on the severity level associated with the message, the severity settings 205, 206 of the relevant controller(s) 200, and the filtering configuration of one or more optional filters 212, 216. For example, in one embodiment, messages having a severity level greater than or equal to the effective severity of the relevant controller 200 are candidates for output and are forwarded to the output destinations 214 attached to the controller 200 (i.e., assuming that the messages are not filtered by one of the filters 212, 216).
A variety of different severity levels may be defined. In one embodiment of the invention, illustrated in
Before (or after) evaluating the trace/log message based on severity, filters 212 associated with the controller 200 may filter the messages based on predefined filtering criteria. By way of example, if a particular controller 200 is capable of writing to both a file and a console, a filter 212 may be assigned to filter messages directed to the file, thereby limiting output to the console only. Thus, using filters, a finer granularity of log controller 200 output may be defined, using variables other than merely severity. As illustrated in
The Logging Sub-Class
As mentioned briefly above, in one embodiment, the logging operations performed by the logging controller 204 are associated with particular “categories” which may identify, for example, semantical topics which correspond roughly to administration tasks or problem areas. Typical areas identified by categories may include databases, networking, security and auditing.
In one embodiment, categories are named according to the hierarchical structure known from file systems on the network. For example, referring to
The Tracing Sub-Class
In one embodiment, in contrast to logging operations which are associated with categories, tracing operations performed by the tracing controllers 202 are associated with particular program code locations, identified by particular package, class, or function names. For example, in a Java environment, locations may be named according to the hierarchical structure known from Java packages.
In the example illustrated in
As an additional example, to write all the messages from monitoring classes into a single log, the location named “com.sap.tc.monitoring” may be called. All the messages from technology components (TC) may be directed to a common trace destination by simply assigning that destination to the parent location “com.sap.tc.” The trace output destination is then passed on to all child locations (e.g., to “com.sap.tc.monitoring”). Moreover, it is possible to include method signatures in location names. For example, “com.sap.tc.monitoring. Node.announce (java.lang.Object)” is a location for a method named “announce” with an argument of the class Object. In this way, overloaded methods can be identified and, by adding another suffix to such a name, even classes local to them. In a Java environment, the hierarchical components of the name should be compliant with Java identifier syntax, but illegal characters may be used, bracketing a component with single quotes (e.g., com.sap.‘great.technology’).
In one embodiment, all locations are accessed via a defined static method “Location.getLocation.” In one embodiment, the static method need not be called each time a trace message is to be generated. Instead, each class defines static fields to hold all the needed locations, and uses these fields to call logging/tracing methods.
Controlling Severity Settings Using Inheritance
In one embodiment, the various controllers 200 are arranged into logical hierarchies in which each child controller inherits properties from its parent (e.g., its severity settings and its output destinations 214). The effective severity of a child log controller is calculated from minimum and maximum defined severities, as well as the effective severity of its parent. The minimum severity setting specifies the minimum severity value for the trace/log messages to be output via the child log controller. That is, if the effective severity of the parent log controller is higher than the minimum severity of the child, then the child inherits the parent's effective severity; otherwise the effective severity is set to the minimum severity.
By contrast, the maximum severity represents the severity that trace/log messages must have for output to be guaranteed (notwithstanding intervention from filters). For example, if the effective severity of the parent log controller is lower than the maximum severity of the child then the child inherits the parent's effective severity; otherwise the child's effective severity is set to the maximum severity. This implies that if both minimum and maximum severity are set to the same value, the effective severity is set to that value regardless of the effective severity of the parent. Such a setting is therefore called dominant. For root log/trace controllers the setting must be dominant, as separate minimum and maximum values are meaningless without a parent to inherit from.
Similarly, tracing controller “com.sapmarkets.technology.monitoring” 401, is programmed with a minimum severity of INFO and a maximum severity of ERROR. Accordingly, because the effective severity of its parent controller 302 (WARNING) is higher than its minimum severity, INFO, and lower than its maximum severity, ERROR, tracing controller 401 inherits its parent's effective severity of WARNING.
By contrast, tracing controller “com.sapmarkets.application” 303 is manually configured with an effective severity of PATH, which overrides the effective severity of INFO inherited from its parent tracing controller “com.sapmarkets” 301.
It should be noted, of course, that the specific details set forth above are used merely for the purpose of illustration. The underlying principles of the invention are not limited to any particular logging/tracing controller hierarchy or any particular severity settings.
Output Formatters
As illustrated in
In one embodiment, the list formatter 230 translates the output of the log/trace controllers into a format which may be further processed by an application, e.g. a log viewer, instead of being read directly by an end-user. In one embodiment, the format generated by the list formatter comprises hash-separated fields which may be readily interpreted by the other applications. For example: #1.3#10.48.27.165:4A5AB2:E99D42D4F4:-8000#Mon Jan 01 22:00:00 PDT2001#com.sapmarkets.FooClass#com.sapmarkets.FooClass.fooMethod#main##0#0#Fatal##Plain###A sample fatal message#.
As its name suggests, the human readable formatter 231 translates the output of the log/trace controllers into a human-readable format (e.g., ASCII text). As such, this formatted may be used when users/administrators need to quickly understand what is occurring within a network or application. For example, the human readable formatter 231 may provide its formatted output to a trace/log console for immediate viewing.
The markup language formatter 232 translates the output of the log/trace controllers into a particular markup language such as the Extensible Markup Language (“XML”) format which may be interpreted by any application that includes support for XML (e.g., Microsoft Word).
Relationship Between Logging and Tracing Operations
Messages associated with a particular category may also be associated with (i.e., may be written with respect to) a source code area, or location, such as a component, a package, a class or a method. As the location may be associated with a particular tracing controller, the same method call can write a message simultaneously to, for example, the database log as well as to the location trace responsible for that part of the source code (i.e., save for the location having an appropriate severity setting). In one embodiment of the invention, both the trace message and the log message are generated with the same identification in order to facilitate cross-referencing among location and category logs. At the same time, the location provides a location description, that is a string, to the log message written to the database log. This may become tedious when a class implements a large number of methods. Therefore, as described in greater detail below, for each logging/tracing method there is a version which takes in an additional parameter, referred to herein as “subloc,” which is a string that serves as suffix to the name of the used location, thus providing the option to give a complete location name whilst avoiding clutter.
For the purpose of illustration, one particular tracing/logging example will now be described with respect to the method outlined in
The following sample code will be used for the example:
Method elements 502 and 503 are not shown at this point, but assuming the severity level is set to be Severity.ALL (accept all severity levels and output everything) and output has been formatted using a human-readable formatter (e.g., and sent to a console) the output may look like the following:
The following four sections will further explain each step illustrated in
Identifying the Source Code Area (501)
As described above, the tracing subclass and the logging subclass are subclasses of the controller class. The tracing subclass is sometimes referred to below as the “Location” subclass and the logging subclass is sometimes referred to below as the “Category” subclass. Recall that Location is the source area that generates trace messages. Typically, it corresponds to the source code structure, and can be attached to the level of component, package, class or method. Category is the source area that generates log messages, corresponding to a distinguished problem area, such as networking, system and database.
Although the naming of a Location and Category may be quite flexible, as a general rule, a valid hierarchical naming convention may be useful. A common naming practice for Location is to use the full path of Java package name (See, e.g.,
For the purpose of this example, the following static methods are provided for each class for easy access to a Location or Category:
Alternatively, instead of passing the name manually, for Location the class instance may be passed (java.lang.Object) or the class itself (java.lang.Class). In either case, the Location object is by default referring to the class level, while using the string argument (java.lang.String) provides flexibility in the definition (e.g., to also include the method name to explicitly control logging over methods individually).
Once identified, the source is ready to be configured to generate messages. In one embodiment, It is recommended that initially, the handle may be configured to be static to improve efficiency:
static final Location loc=Location.getLocation (this.getClass( ))
Assign Severity to Source (502)
Recall that the severity levels employed in one embodiment of the invention are set forth in
The severity may be assigned to the specified source using the following method: loc.setEffectiveSeverity(Severity.INFO);. Using this configuration, any messages with severity lower than INFO will be discarded. Others will be directed to the destination. The concepts associated with hierarchical severity were described above with respect to
In one embodiment, by default, the source object (assuming the ascendant has not been assigned a severity level yet) is assigned Severity.NONE. As such, a developer may freely enable the output methods in the source code, but the actual logging is not activated until it is explicitly “switched on” when ready.
Specify Output Destination (503)
An output destination such as a log/trace file, a console, an output stream, etc, is assigned to each Location sub-controller or Category sub-controller. This can be accomplished via a method such as the following:
There may be instances in which it is desirable to assign multiple output destinations to a single sub-controller. In such a case, a message generated from the sub-controller will be sent to all assigned output destinations simultaneously (e.g., to both a console and a file). This may be accomplished using the following:
In one embodiment, as a default configuration, console output destinations 221 are assigned human-readable formatters 231 and file output destinations 220 are assigned list formatters 230. Of course, the default configurations are not required. For example, to switch to an XML Formatter for a file output, the following exemplary method may be called:
After the source is defined, the severity levels are assigned and the destination is properly specified, output statements may be inserted in appropriate places in the program code. For the sake of explanation, the output methods may be logically categorized into three groups. (1) typical message output with severity; (2) output denoting the flow of a program; and (3) master gate.
(1) Typical Message Output With Severity: The following table illustrates the format for a typical output message with severity, where the designated severity level is masked with “xxxx.” The “T” designates a typical message output.
A pattern exists in method overloading which evolves around the core argument: message. The addition of subloc, args offers the flexibility for developers to log messages using the level of detail that they need. Understanding these arguments helps when selecting the heavily overloaded methods.
loc:
One difference in the API between Location and Category is an additional loc argument in Category output methods. As described above, log messages are typically written with respect to a source code area. This proves to be very helpful for logging analysis. By specifying the loc argument, a programmer may indicate that the message should be written as a trace message associated with the loc object. By properly configuring loc, logging can be just performed once and piped for both message types (i.e., logs & traces) simultaneously. This technique works for Location as well, and the API is available to specify the category argument. These techniques are explained in more detail below (section entitled “Relationship Between location and Category).
subloc:
The subloc argument is treated as the method name of the source class that the message is generated from. This is optional, but with this argument included in the trace/log, the picture when doing analysis will be much clearer, for example, because different arguments can be specified for overloaded methods).
message:
The actual message to be output is placed in argument message. A designation should be selected that meaningfully describes the situation/problem.
args:
Array of additional arguments that are informative, e.g. dynamic information to be included in the message. In one embodiment, this is achieved by using java.text.MessageFormat API to resolve arguments.
Referring again to the code example set forth above, the following is an example working from the object Location:
Potential output is as follows, assuming the human-readable formatter is used (e.g., to display on a console):
The following is another example, working from the object Category:
Note that the output will be identical to the previous example, assuming the default setting is used (using a human-readable formatter).
(2) Output Denoting the Flow of a Program: This feature is typically only used for Location sub-controllers. Tracing the flow of a program is comprised of several components: (a) Entering, (b) Exiting, (c) Throwing, and (d) Assertion.
Entering: Outputs a default message (e.g., “Entering Method” with Path severity) indicating that it is entering a source block:
(b) Exiting: Output a default message (e.g., “Exiting Method” with Path severity) indicating that it is leaving a source block:
To reiterate, refer to the sample code with method announce(Object o):
Potential output, assuming the simplest case with ConsoleLog and default TraceFormatter:
(c) Throwing: Warning message (“Throwing . . . ”), indicating that the source block is about to throw an exception:
(d) Assertion: used to test a condition and output an error message, normally with the assertion included (e.g., “Assertion failed: <assertion test>”) when the evaluation is false:
To reiterate, refer to the sample code with method announce(Object o):
The following is the potential output, again assuming a human-readable log formatter:
(3) Master Gate:
In one embodiment, all of the other output methods (with severity) are ultimately routed through a method, referred to herein as LogT, to actually perform any logging/tracing.
Location: These are similar to the first type of method, xxxxxT( ) but only with an additional severity argument at the beginning:
Category: (Similar Scenario to Location):
Often, it is useful to put several related messages together into one context. A typical example would be all trace messages stemming from one method call. In case of a database log, another example would be the messages representing the different database operations together forming one logical transaction. A formatter or log viewer can utilize this context information to visualize relations using, for example, indentation or tree controls. Groups are one mechanism to express such context information.
In one embodiment, a group is established via a call to openGroup. This call is based on the same conditions as output calls, that is, the group is opened depending on a severity and optional categories. After generating output the group is ended via a call to closeGroup. Note that there should be a balancing call of closeGroup for any call of openGroup. Even if an openGroup call did not open the group, closeGroup is matched with the call to openGroup. In case of success, in between the two calls, all output calls are assigned to the group. Messages may even be generated with the same condition as the group via the groupT and group output calls. These calls are also used to emit the opening and closing messages of a group, which are the first such messages emitted after the openGroup and closeGroup calls, respectively.
In the above method, for example, the following piece of code could be written. The message right after the openGroup call is the opening message of the group, the message after closeGroup its closing message. Note that the groupT calls do not need a severity, a location or a method name, as these are fetched from the active group.
Establishing Relationships Between Category & Location
It is a common practice to look at log messages and trace messages together when performing a diagnosis. A correlation between a problematic logical area and the source code location that generates the problem is highly desirable. For example, if an error occurs when closing the database, the actual location of the source code (from which class, which method, with what argument(s)) is reported as well.
An embodiment of the invention simplifies the generation of both log and trace messages in parallel, with the use of category and location. This will be illustrated with the following example. For simplicity, the example only shows the attachment of a location to a category (as used herein, a location associated with the category is referred to as a “relative” of the category). However, the same general principles apply to the attachment of a category to a location.
More than one category may be associated with a location at each logging. However, in one embodiment, only one location is assigned for one category. Refer to the output method APIs of each class:
In order to output all the trace and log messages highlighted in the example above, the following severity setting may be employed:
With respect to the output line from the category ‘objMgmt’, it will output two messages simultaneously: one log message and one trace message. They will have the same message id for cross-referencing each other. This greatly simplifies the analysis.
If the location has stricter severity setting (e.g. default Severity.NONE) all trace output may be suppressed, including the one from the category. In other words, that output line will NOT produce two messages simultaneously, but only the log message.
More advanced configuration may be employed regarding the correlated category and location source objects. For example, consider Category “/Objects/Management” where the only focus may be some extreme situations, that is, messages with severity FATAL. Several source code locations (‘com.sapmarkets.xxx.a’, ‘com.sapmarkets.xxx.b’, . . . ) can result in a fatal condition in this logical area, and for certain reasons, one of them may be of particular interest (e.g. ‘com.sapmarkets.xxx.a’). As such, it would beneficial to have the ability to generate additional output messages, including all with severity INFO or above, related with this location only, while maintaining FATAL for the rest.
Embodiments of the invention may include various steps as set forth above. The steps may be embodied in machine-executable instructions which cause a general-purpose or special-purpose processor to perform certain steps. Alternatively, these steps may be performed by specific hardware components that contain hardwired logic for performing the steps, or by any combination of programmed computer components and custom hardware components.
Elements of the present invention may also be provided as a machine-readable medium for storing the machine-executable instructions. The machine-readable medium may include, but is not limited to, flash memory, optical disks, CD-ROMs, DVD ROMs, RAMs, EPROMs, EEPROMs, magnetic or optical cards, propagation media or other type of machine-readable media suitable for storing electronic instructions. For example, the present invention may be downloaded as a computer program which may be transferred from a remote computer (e.g., a server) to a requesting computer (e.g., a client) by way of data signals embodied in a carrier wave or other propagation medium via a communication link (e.g., a modem or network connection).
Throughout the foregoing description, for the purposes of explanation, numerous specific details were set forth in order to provide a thorough understanding of the invention. It will be apparent, however, to one skilled in the art that the invention may be practiced without some of these specific details. For example, although the embodiments of the invention described above are implemented within a Java programming environment, various alternate environments may be employed while still complying with the underlying principles of the invention.
Accordingly, the scope and spirit of the invention should be judged in terms of the claims which follow.
Number | Name | Date | Kind |
---|---|---|---|
5201044 | Frey et al. | Apr 1993 | A |
5608720 | Biegel et al. | Mar 1997 | A |
6061721 | Ismael et al. | May 2000 | A |
6083281 | Diec et al. | Jul 2000 | A |
6144967 | Nock | Nov 2000 | A |
6308208 | Jung et al. | Oct 2001 | B1 |
6356931 | Ismael et al. | Mar 2002 | B2 |
6470388 | Niemi et al. | Oct 2002 | B1 |
6539501 | Edwards | Mar 2003 | B1 |
6553403 | Jarriel et al. | Apr 2003 | B1 |
6567809 | Santosuosso | May 2003 | B2 |
6681232 | Sistanizadeh | Jan 2004 | B1 |
6738933 | Fraenkel et al. | May 2004 | B2 |
6772178 | Mandal et al. | Aug 2004 | B2 |
6789257 | MacPhail | Sep 2004 | B1 |
6802067 | Camp et al. | Oct 2004 | B1 |
6834301 | Hanchett | Dec 2004 | B1 |
6836878 | Cuomo et al. | Dec 2004 | B1 |
6851118 | Ismael et al. | Feb 2005 | B1 |
6862711 | Bahrs et al. | Mar 2005 | B1 |
6880125 | Fry | Apr 2005 | B2 |
6925631 | Golden | Aug 2005 | B2 |
6985848 | Swoboda et al. | Jan 2006 | B2 |
6990601 | Tsuneya et al. | Jan 2006 | B1 |
7000235 | Mandal et al. | Feb 2006 | B2 |
7017051 | Patrick | Mar 2006 | B2 |
7017162 | Smith et al. | Mar 2006 | B2 |
7024474 | Clubb et al. | Apr 2006 | B2 |
7051324 | Gissel et al. | May 2006 | B2 |
7058558 | Reichenthal | Jun 2006 | B2 |
7062540 | Reddy et al. | Jun 2006 | B2 |
7069267 | Spencer, Jr. | Jun 2006 | B2 |
7086065 | Yeluripati et al. | Aug 2006 | B1 |
7120685 | Ullmann et al. | Oct 2006 | B2 |
7152104 | Musante et al. | Dec 2006 | B2 |
7174370 | Saini et al. | Feb 2007 | B1 |
7200588 | Srivastava et al. | Apr 2007 | B1 |
7206827 | Viswanath et al. | Apr 2007 | B2 |
7240334 | Fluke et al. | Jul 2007 | B1 |
7251809 | Barclay et al. | Jul 2007 | B2 |
7305671 | Davidov et al. | Dec 2007 | B2 |
20020029298 | Wilson | Mar 2002 | A1 |
20030005173 | Shah et al. | Jan 2003 | A1 |
20030110252 | Yang-Huffman | Jun 2003 | A1 |
20030120593 | Bansal et al. | Jun 2003 | A1 |
20030135556 | Holdsworth | Jul 2003 | A1 |
20030167304 | Zhu et al. | Sep 2003 | A1 |
20030225872 | Bartek et al. | Dec 2003 | A1 |
20040022237 | Elliott et al. | Feb 2004 | A1 |
20040028059 | Josyula et al. | Feb 2004 | A1 |
20040148356 | Bishop, Jr. et al. | Jul 2004 | A1 |
20040148610 | Tsun et al. | Jul 2004 | A1 |
20040158837 | Sengodan | Aug 2004 | A1 |
20040230973 | Cundiff, Jr. et al. | Nov 2004 | A1 |
20040237093 | Sluiman et al. | Nov 2004 | A1 |
20040249613 | Sprogis et al. | Dec 2004 | A1 |
20040268314 | Kollman et al. | Dec 2004 | A1 |
20050010608 | Horikawa | Jan 2005 | A1 |
20050028171 | Kougiouris et al. | Feb 2005 | A1 |
20050033777 | Moraes et al. | Feb 2005 | A1 |
20050038889 | Frietsch | Feb 2005 | A1 |
20050097110 | Nishanov et al. | May 2005 | A1 |
20050102536 | Patrick et al. | May 2005 | A1 |
20050132337 | Wedel et al. | Jun 2005 | A1 |
20070156656 | Pather et al. | Jul 2007 | A1 |
Number | Date | Country |
---|---|---|
2001195151 | Jul 2001 | JP |
2002073522 | Mar 2002 | JP |
2002082777 | Mar 2002 | JP |
WO 0205102 | Jan 2002 | WO |
WO 0241154 | May 2002 | WO |