This invention relates to breakpoint method for parallel hardware threads in a multiprocessor.
Parallel processing is an efficient form of information processing of concurrent events in a computing process. Parallel processing demands concurrent execution of many programs in a computer, in contrast to sequential processing. That is, in general all or a plurality of the stations work simultaneously and independently on the same or common elements of a problem.
Parallel processing involves multiple processors. The execution path of a microprocessor within a parallel system is highly pipelined.
In a parallel processor where many threads of execution can run simultaneously, there is a need for debugging software running on selected threads. Debugging may be used to determine a cause (or causes) of errors in the processing threads, and to correct the errors.
Debug methods implement breakpoints in software by a combination of inserting traps and single stepping. When the target program contains multiple threads of execution, a debug method that is not carefully implemented may miss breakpoints and be less than helpful to the developer.
Referring to
The hardware-based multithreaded processor 12 also includes a central processor 20 that assists in loading microcode control for other resources of the hardware-based multithreaded processor 12 and performs other general purpose computer type functions, such as handling protocols, exceptions, extra support for packet processing where the microengines pass the packets off for more detailed processing, such as in boundary conditions. In an embodiment, the processor 20 is a Strong ARM® (ARM is a trademark of ARM Limited, United Kingdom) based architecture. The processor 20 has an operating system. Through the operating system, the processor 20 can call functions to operate on microengines 22. The processor 20 can use any supported operating system, preferably a real-time operating system. For a processor 20 implemented as a Strong ARM® architecture, operating systems such as Microsoft NT Real-Time, VXWorks and μCUS, a freeware operating system available over the Internet, can be used.
As mentioned above, the hardware-based multithreaded processor 12 includes a plurality of functional microengines 22a–f. Functional microengines (microengines) 22a–f each maintain a number of program counters in hardware and states associated with the program counters. Effectively, a corresponding plurality of sets of threads can be simultaneously active on each of the microengines 22a–f while only one is actually operating at any one time.
In an embodiment, there are six microengines 22a–f, as shown. Each of the microengines 22a–f has capabilities for processing four hardware threads. The six microengines 22a–f operate with shared resources, including memory system 16 and bus interfaces 24 and 28. The memory system 16 includes a synchronous dynamic random access memory (SDRAM) controller 26a and a static random access memory (SRAM) controller 26b. SRAM memory 16a and SRAM controller 26a are typically used for processing large volumes of data, e.g., processing of network payloads from network packets. The SDRAM controller 26b and SDRAM memory 16b are used in a networking implementation for low latency fast access tasks, e.g., accessing lookup tables, memory from the core processor, and so forth.
The six microengines 22a–f access either the SDRAM 16a or SRAM 16b based on characteristics of the data. Thus, low latency, low bandwidth data is stored in and fetched from SRAM 16b, whereas higher bandwidth data for which latency is not as important, is stored in and fetched from SDRAM 16b. The microengines 22a–f can execute memory reference instructions to either the SDRAM controller 26a or SRAM controller 26b.
Advantages of hardware multithreading can be explained by SRAM or SDRAM memory accesses. As an example, an SRAM access requested by a thread_0, from a microengine will cause the SRAM controller 26b to initiate an access to the SRAM memory 16a. The SRAM controller 26b controls arbitration for the SRAM bus, accesses the SRAM 16a, fetches the data from the SRAM 16a, and returns data to a requesting microengine 22a–f. During an SRAM 26b access, if the microengine, e.g. microengine 22a, had only a single thread that could operate, that microengine would be dormant until data was returned from the SRAM 26b. By employing hardware context swapping within each of the microengines 22a–f, the hardware context swapping enables only contexts with unique program counters to execute in that same microengine. Thus, another thread, e.g., thread_1 can function while the first thread, e.g., thread_0, is awaiting the read data to return. During execution, thread_1 may access the SDRAM memory 26a. While thread_1 operates on the SDRAM unit, and thread_0 is operating on the SRAM unit, a new thread, e.g., thread_2 can now operate in the microengine 22a. Thread_2 can operate for a certain amount of time, until it needs to access memory or perform some other long latency operation, such as making an access to a bus interface. Therefore, simultaneously, the processor can have a bus operation, an SRAM operation and SDRAM operation all being completed or operated upon by one microengine 22a and have one or more threads available to process more work in the data path.
Each of the microengines 22a–f includes an arbiter that examines flags to determine the available threads to be operated upon. Any thread from any of the microengines 22a–f can access the SDRAM controller 26a, SRAM controller 26b or bus interface. The memory controllers 26a and 26b each include a number of queues to store outstanding memory reference requests. The queues either maintain order of memory references or arrange memory references to optimize memory bandwidth. For example, if a thread_0 has no dependencies or relationship to a thread_1, there is no reason that thread_1 and thread_0 cannot complete their memory references to the SRAM unit 26b out of order. The microengines 22a–f issue memory reference requests to the memory controllers 26a and 26b. The microengines 22a–f flood the memory subsystems 26a and 26b with enough memory reference operations such that the memory subsystems 26a and 26b become the bottleneck for processor 12 operation. Microengines 22a–f can also use a register set to exchange data.
The processor 20 includes a RISC core 50, implemented in a five-stage pipeline performing a single cycle shift of one operant or two operants in a single cycle, provides multiplication support and 32-bit barrel shift support. This risc core 50 is a standard Strong Arm® architecture, but is implemented with a five-stage pipeline for performance reasons. The processor 20 also includes a 16-kilobyte instruction cache 52, an 8-kilobyte data cache 54 and a prefetch stream buffer 56. The core processor performs arithmetic operations in parallel with memory writes and instruction fetches. The processor 20 interfaces with other functional units via the ARM defined ASB bus. The ASB bus is a 32-bit bi-directional bus.
Referring to
In addition to event signals that are local to an executing thread, the microengines 22 employ signaling states that are global. With signaling states, an executing thread can broadcast a signal state to all microengines 22. Receive request available signal, any and all threads in the microengines can branch on these signaling states. These signaling states can be used to determine the availability of a resource or whether a resource is due for servicing.
The context event logic 74 has arbitration for the four threads in the example. In an embodiment, the arbitration is a round robin mechanism. Other techniques could be used, including priority queuing or weighted fair queuing. The microengine 22f also includes an execution box (EBOX) datapath 76 that includes an arithmetic logic unit 76a and general purpose register set 76b. The arithmetic logic unit 76a performs arithmetic and logical functions as well as shift functions. The register set 76b has a relatively large number of general purpose registers. General purpose registers are windowed so that they are relatively and absolutely addressable.
The microengine 22f also includes a write transfer register stack 78 and a read transfer stack 80. These registers 78 and 80 are also windowed so they are relatively and absolutely addressable. The write transfer register stack 78 is where write data to a resource is located. Similarly, the read register stack 80 is for returned data from a shared resource. Subsequent to, or concurrent with data arrival, an event signal from the respective shared resource, e.g., the SRAM controller 26b, the SDRAM controller 26a, or processor 20 will be provided to context event arbiter 74 which will then alert the thread is available or has been sent. Both transfer register banks 78 and 80 are connected to the execution box 76 through a datapath. In an implementation, the read transfer register 80 has sixty-four registers and the write transfer register 78 has sixty-four registers.
Each microengine 22a–f supports multi-threaded execution of four contexts. One reason for this is to allow one thread to start executing just after another thread issues a memory reference and must wait until that reference completes before doing more work. This behavior is critical to maintaining efficient hardware execution of the microengines, because memory latency is significant. Stated differently, if only a single thread execution was supported, the microengines would sit idle for a significant number of cycles waiting for references to return and thus reduce overall computational throughput. Multithreaded execution involves all microengines to hide memory latency by performing useful, independent work across several threads.
When errors occur in software running in one or more of the threads of execution, there is a need for debugging the software running on selected threads to determine a cause (or causes) of the errors and to aid a software developer to correct the errors.
Referring to
In a normal interrupt, a hardware-supported thread, thread 502a for example, executes a write to the interrupt register 506 that carries ten bits of immediate data. A “1” in bits 6:0 is shifted left by the value of the hardware thread identification and inserted into the interrupt register.
Referring to
At any time, more than one thread can issue a normal interrupt or an interrupt indicating a breakpoint. If the interrupt indicates a breakpoint, the core processor 504 traps to an interrupt handling routine 508 in an interrupt handler 510, where the interrupt register 506 is read. If the breakpoint field is non-zero, it can be used to narrow the source of the breakpoints to three groups of threads. Bits 23:0 are used to identify which threads have raised an interrupt.
The core processor 504 is linked to a remote user interface 511. A user (not shown) inputs a source code line to be breakpointed in one of the microengines using the remote user interface 511. The source code line is sent to the central processor 504. The central processor 504 searches a breakpoint database 512 in a debug library 514 to determine whether the instruction corresponding to the source code line can be breakpointed. Since there are certain cases where breakpointing is not allowed, i.e., a trap in the code cannot be inserted at this position in the code to signal a breakpoint. For example, if two instructions must be executed in succession, i.e., a register of one is used in the very next cycle in the next, the first instruction cannot be breakpointed. This is because the software breakpoint inserts a branch to displace the breakpointed instruction, thus separating the two instructions.
After searching its breakpoint database 512 and determining that an instruction may be breakpointed, the core processor 504 invokes a remote procedure call (RPC) referred to as SetBreakpoint to the debug library 514. The SetBreakpoint RPC identifies which microengine (which instruction code) to insert the breakpoint into, which program counter (PC) to breakpoint at, which microengine threads (also referred to as contexts) to enable breakpoint for, and which microengines to stop if the breakpoint occurs.
The function definition for the set breakpoint RPC to the debug library is as follows:
The debug library 514 generates a breakpoint routine by modifying a template of several instructions, inserts the breakpointing instruction and inserts a branch to the location of the instruction after the breakpoint instruction. The breakpoint instruction is essentially a branch instruction to the breakpoint routine.
An example breakpoint template follows:
After the breakpoint instruction is inserted into the selected microengine, the microengine with this breakpoint code is resumed or restarted. If program execution within the microengine gets to the breakpoint program counter, program execution branches to the breakpoint routine and interrupts the core processor 504 with a breakpoint interrupt. If a breakpoint is set, the interrupt handler 510 calls the debug library 514. The interrupt handler 514 can quickly handle breakpoints by reading the program counters of the interrupting threads. The debug library 514 stops the selected threads (and bus ports) for the breakpoint. The debug library 514 determines which thread(s) sent the breakpoint interrupt. The debug library 514 replies to the user on the remote user interface 511, allowing the user to examine the state of the saved threads (and ports). The user selects a resume command that initiates resume remote procedure call to the debug library 514. The resume identifies which microengines (and ports) to resume. Upon receiving this resume RPC, the debug library 514 insures the thread that the sent the breakpoint interrupt will be next to start by setting a control and status register with context enable. The debug library 514 restarts the microengines (and bus ports) indicated by the resume command.
Referring to
The process 600 causes the debug library to generate 610 a breakpoint routine by modifying a template of several instructions, to insert 612 the breakpoint instruction and to insert 614 a branch to the location after the breakpoint instruction.
After the breakpoint routine is inserted 612, the process 600 causes the microengine with the inserted breakpoint routine to resume 616. If program execution in the microengine gets to the breakpoint PC, program execution branches 618 to the breakpoint routine and interrupts 620 the core processor with the breakpoint interrupt.
The process 600 causes the interrupt handler to call 622 the debug library. The debug library stops 624 the selected threads (and bus ports) for the breakpoint and determines 626 which microengine sent the interrupt. The process 600 causes the debug library to display 626 information to the user.
The process 600 receives 630 a resume command from the user and in response sets the context enable bit for the selected microengine. Setting of the context enable bit starts the microengine back into normal program execution outside of the breakpoint routine.
An embodiment of the invention has been described. Nevertheless, it will be understood that various modifications may be made without departing from the spirit and scope of the invention. For example, reset, start, pause and resume remote procedure calls can be used with the controlled stop/start of selected microengines (and bus ports). The RPC “Reset” stops the selected microengines (and bus ports) immediately, whereas the RPC “Start” starts the microengines at program counter 0 (and starts the bus ports). The RPC “Pause” stops selected microengines (and bus ports) at a safe, i.e., non-destructive stopping point. The RPC “Resume” starts the microengines at their current program counters (and starts the bus ports). Accordingly, other embodiments are within the scope of the following claims.
Number | Name | Date | Kind |
---|---|---|---|
3373408 | Ling | Mar 1968 | A |
3478322 | Evans | Nov 1969 | A |
3792441 | Wymore et al. | Feb 1974 | A |
3913074 | Homberg et al. | Oct 1975 | A |
3940745 | Sajeva | Feb 1976 | A |
4023023 | Bourrez et al. | May 1977 | A |
4130890 | Adam | Dec 1978 | A |
4392758 | Bowles et al. | Jul 1983 | A |
4400770 | Chan et al. | Aug 1983 | A |
4514807 | Nogi | Apr 1985 | A |
4523272 | Fukunaga et al. | Jun 1985 | A |
4724521 | Carron et al. | Feb 1988 | A |
4745544 | Renner et al. | May 1988 | A |
4777587 | Case et al. | Oct 1988 | A |
4866664 | Burkhardt, Jr. et al. | Sep 1989 | A |
5073864 | Methvin et al. | Dec 1991 | A |
5113516 | Johnson | May 1992 | A |
5140685 | Sipple et al. | Aug 1992 | A |
5142683 | Burkhardt, Jr. et al. | Aug 1992 | A |
5155831 | Emma et al. | Oct 1992 | A |
5155854 | Flynn et al. | Oct 1992 | A |
5168555 | Byers et al. | Dec 1992 | A |
5173897 | Schrodi et al. | Dec 1992 | A |
5255239 | Taborn et al. | Oct 1993 | A |
5263169 | Genusov et al. | Nov 1993 | A |
5347648 | Stamm et al. | Sep 1994 | A |
5357617 | Davis et al. | Oct 1994 | A |
5363448 | Koopman, Jr. et al. | Nov 1994 | A |
5367678 | Lee et al. | Nov 1994 | A |
5390329 | Gaertner et al. | Feb 1995 | A |
5392391 | Caulk, Jr. et al. | Feb 1995 | A |
5392411 | Ozaki | Feb 1995 | A |
5392412 | McKenna | Feb 1995 | A |
5404464 | Bennett | Apr 1995 | A |
5404482 | Stamm et al. | Apr 1995 | A |
5428809 | Coffin et al. | Jun 1995 | A |
5432918 | Stamm | Jul 1995 | A |
5436626 | Fujiwara | Jul 1995 | A |
5442756 | Grochowski et al. | Aug 1995 | A |
5448702 | Garcia, Jr. et al. | Sep 1995 | A |
5450351 | Heddes | Sep 1995 | A |
5452437 | Richey et al. | Sep 1995 | A |
5459842 | Begun et al. | Oct 1995 | A |
5463625 | Yasrebi | Oct 1995 | A |
5467452 | Blum et al. | Nov 1995 | A |
5481683 | Karim | Jan 1996 | A |
5487159 | Byers et al. | Jan 1996 | A |
5517628 | Morrison et al. | May 1996 | A |
5517648 | Bertone et al. | May 1996 | A |
5542070 | LeBlanc et al. | Jul 1996 | A |
5542088 | Jennings, Jr. et al. | Jul 1996 | A |
5544236 | Andruska et al. | Aug 1996 | A |
5550816 | Hardwick et al. | Aug 1996 | A |
5557766 | Takiguchi et al. | Sep 1996 | A |
5568617 | Kametani | Oct 1996 | A |
5574922 | James | Nov 1996 | A |
5574939 | Keckler et al. | Nov 1996 | A |
5592622 | Isfeld et al. | Jan 1997 | A |
5600812 | Park | Feb 1997 | A |
5606676 | Grochowski et al. | Feb 1997 | A |
5613071 | Rankin et al. | Mar 1997 | A |
5613136 | Casavant et al. | Mar 1997 | A |
5623489 | Cotton et al. | Apr 1997 | A |
5627829 | Gleeson et al. | May 1997 | A |
5630130 | Perotto et al. | May 1997 | A |
5644623 | Gulledge | Jul 1997 | A |
5649157 | Williams | Jul 1997 | A |
5652583 | Kang | Jul 1997 | A |
5659687 | Kim et al. | Aug 1997 | A |
5659722 | Blaner et al. | Aug 1997 | A |
5680641 | Sidman | Oct 1997 | A |
5689566 | Nguyen | Nov 1997 | A |
5692167 | Grochowski et al. | Nov 1997 | A |
5699537 | Sharangpani et al. | Dec 1997 | A |
5717760 | Satterfield | Feb 1998 | A |
5717898 | Kagan et al. | Feb 1998 | A |
5721870 | Matsumoto | Feb 1998 | A |
5742587 | Zornig et al. | Apr 1998 | A |
5742782 | Ito et al. | Apr 1998 | A |
5742822 | Motomura | Apr 1998 | A |
5745913 | Pattin et al. | Apr 1998 | A |
5751987 | Mahant-Shetti et al. | May 1998 | A |
5761507 | Govett | Jun 1998 | A |
5761522 | Hisanaga et al. | Jun 1998 | A |
5781774 | Krick | Jul 1998 | A |
5784649 | Begur et al. | Jul 1998 | A |
5784712 | Byers et al. | Jul 1998 | A |
5796413 | Shipp et al. | Aug 1998 | A |
5797043 | Lewis et al. | Aug 1998 | A |
5809235 | Sharma et al. | Sep 1998 | A |
5809530 | Samra et al. | Sep 1998 | A |
5812839 | Hoyt et al. | Sep 1998 | A |
5812868 | Moyer et al. | Sep 1998 | A |
5815714 | Shridhar et al. | Sep 1998 | A |
5828746 | Ardon | Oct 1998 | A |
5828863 | Barrett et al. | Oct 1998 | A |
5832215 | Kato et al. | Nov 1998 | A |
5835755 | Stellwagen, Jr. | Nov 1998 | A |
5838975 | Abramson et al. | Nov 1998 | A |
5854922 | Gravenstein et al. | Dec 1998 | A |
5860158 | Pai et al. | Jan 1999 | A |
5886992 | Raatikainen et al. | Mar 1999 | A |
5887134 | Ebrahim | Mar 1999 | A |
5890208 | Kwon | Mar 1999 | A |
5892979 | Shiraki et al. | Apr 1999 | A |
5905876 | Pawlowski et al. | May 1999 | A |
5905889 | Wilhelm, Jr. | May 1999 | A |
5915123 | Mirsky et al. | Jun 1999 | A |
5933627 | Parady | Aug 1999 | A |
5937187 | Kosche et al. | Aug 1999 | A |
5938736 | Muller et al. | Aug 1999 | A |
5940612 | Brady et al. | Aug 1999 | A |
5940866 | Chisholm et al. | Aug 1999 | A |
5946487 | Dangelo | Aug 1999 | A |
5948081 | Foster | Sep 1999 | A |
5951679 | Anderson et al. | Sep 1999 | A |
5956514 | Wen et al. | Sep 1999 | A |
5958031 | Kim | Sep 1999 | A |
5961628 | Nguyen et al. | Oct 1999 | A |
5970013 | Fischer et al. | Oct 1999 | A |
5978838 | Mohamed et al. | Nov 1999 | A |
5983274 | Hyder et al. | Nov 1999 | A |
5996068 | Dwyer, III et al. | Nov 1999 | A |
6002881 | York et al. | Dec 1999 | A |
6009515 | Steele, Jr. | Dec 1999 | A |
6012151 | Mano | Jan 2000 | A |
6014729 | Lannan et al. | Jan 2000 | A |
6023742 | Ebeling et al. | Feb 2000 | A |
6029228 | Cai et al. | Feb 2000 | A |
6058168 | Braband | May 2000 | A |
6058465 | Nguyen | May 2000 | A |
6067585 | Hoang | May 2000 | A |
6070231 | Ottinger | May 2000 | A |
6072781 | Feeney et al. | Jun 2000 | A |
6073215 | Snyder | Jun 2000 | A |
6076158 | Sites et al. | Jun 2000 | A |
6079008 | Clery, III | Jun 2000 | A |
6079014 | Papworth et al. | Jun 2000 | A |
6085215 | Ramakrishnan et al. | Jul 2000 | A |
6085294 | Van Doren et al. | Jul 2000 | A |
6092127 | Tausheck | Jul 2000 | A |
6092158 | Harriman et al. | Jul 2000 | A |
6092175 | Levy et al. | Jul 2000 | A |
6112016 | MacWilliams et al. | Aug 2000 | A |
6115811 | Steele, Jr. | Sep 2000 | A |
6134665 | Klein et al. | Oct 2000 | A |
6141689 | Yasrebi | Oct 2000 | A |
6141765 | Sherman | Oct 2000 | A |
6144669 | Williams et al. | Nov 2000 | A |
6145054 | Mehrotra et al. | Nov 2000 | A |
6145123 | Torrey et al. | Nov 2000 | A |
6157955 | Narad et al. | Dec 2000 | A |
6160562 | Chin et al. | Dec 2000 | A |
6182177 | Harriman | Jan 2001 | B1 |
6195676 | Spix et al. | Feb 2001 | B1 |
6199133 | Schnell | Mar 2001 | B1 |
6201807 | Prasanna | Mar 2001 | B1 |
6212542 | Kahle et al. | Apr 2001 | B1 |
6212611 | Nizar et al. | Apr 2001 | B1 |
6216220 | Hwang | Apr 2001 | B1 |
6223207 | Lucovsky et al. | Apr 2001 | B1 |
6223238 | Meyer et al. | Apr 2001 | B1 |
6223279 | Nishimura et al. | Apr 2001 | B1 |
6230119 | Mitchell | May 2001 | B1 |
6247025 | Bacon | Jun 2001 | B1 |
6256713 | Audityan et al. | Jul 2001 | B1 |
6272616 | Fernando et al. | Aug 2001 | B1 |
6275505 | O'Loughlin et al. | Aug 2001 | B1 |
6279113 | Vaidya | Aug 2001 | B1 |
6289011 | Seo et al. | Sep 2001 | B1 |
6298370 | Tang et al. | Oct 2001 | B1 |
6307789 | Wolrich et al. | Oct 2001 | B1 |
6324624 | Wolrich et al. | Nov 2001 | B1 |
6345334 | Nakagawa et al. | Feb 2002 | B1 |
6347344 | Baker et al. | Feb 2002 | B1 |
6351808 | Joy et al. | Feb 2002 | B1 |
6356962 | Kasper et al. | Mar 2002 | B1 |
6360262 | Guenthner et al. | Mar 2002 | B1 |
6373848 | Allison et al. | Apr 2002 | B1 |
6378124 | Bates et al. | Apr 2002 | B1 |
6378125 | Bates et al. | Apr 2002 | B1 |
6389449 | Nemirovsky et al. | May 2002 | B1 |
6393483 | Latif et al. | May 2002 | B1 |
6415338 | Habot | Jul 2002 | B1 |
6426940 | Seo et al. | Jul 2002 | B1 |
6427196 | Adiletta et al. | Jul 2002 | B1 |
6430626 | Witkowski et al. | Aug 2002 | B1 |
6434145 | Opsasnick et al. | Aug 2002 | B1 |
6463072 | Wolrich et al. | Oct 2002 | B1 |
6532509 | Wolrich et al. | Mar 2003 | B1 |
6543049 | Bates et al. | Apr 2003 | B1 |
6552826 | Adler et al. | Apr 2003 | B1 |
6560667 | Wolrich et al. | May 2003 | B1 |
6577542 | Wolrich et al. | Jun 2003 | B1 |
6584522 | Wolrich et al. | Jun 2003 | B1 |
6587906 | Wolrich et al. | Jul 2003 | B1 |
6606704 | Adiletta et al. | Aug 2003 | B1 |
6625654 | Wolrich et al. | Sep 2003 | B1 |
6629237 | Wolrich et al. | Sep 2003 | B1 |
6631430 | Wolrich et al. | Oct 2003 | B1 |
6631462 | Wolrich et al. | Oct 2003 | B1 |
6661794 | Wolrich et al. | Dec 2003 | B1 |
6667920 | Wolrich et al. | Dec 2003 | B1 |
6668317 | Bernstein et al. | Dec 2003 | B1 |
6681300 | Wolrich et al. | Jan 2004 | B1 |
6694380 | Wolrich et al. | Feb 2004 | B1 |
20020053017 | Adiletta et al. | May 2002 | A1 |
20020056037 | Wolrich et al. | May 2002 | A1 |
20030041228 | Rosenbluth et al. | Feb 2003 | A1 |
20030145159 | Adiletta et al. | Jul 2003 | A1 |
20030191866 | Wolrich et al. | Oct 2003 | A1 |
20040039895 | Wolrich et al. | Feb 2004 | A1 |
20040054880 | Bernstein et al. | Mar 2004 | A1 |
20040071152 | Wolrich et al. | Apr 2004 | A1 |
20040073728 | Wolrich et al. | Apr 2004 | A1 |
20040073778 | Adiletta et al. | Apr 2004 | A1 |
20040098496 | Wolrich et al. | May 2004 | A1 |
20040109369 | Wolrich et al. | Jun 2004 | A1 |
Number | Date | Country |
---|---|---|
0 379 709 | Aug 1990 | EP |
0 464 715 | Jan 1992 | EP |
0 633 678 | Jan 1995 | EP |
0 745 933 | Dec 1996 | EP |
0 809 180 | Nov 1997 | EP |
59111533 | Jun 1984 | JP |
WO 9415287 | Jul 1994 | WO |
WO 9738372 | Oct 1997 | WO |
WO 0115718 | Mar 2001 | WO |
WO 0116697 | Mar 2001 | WO |
WO 0116698 | Mar 2001 | WO |
WO 0116702 | Mar 2001 | WO |
WO 0116703 | Mar 2001 | WO |
WO 0116713 | Mar 2001 | WO |
WO 0116714 | Mar 2001 | WO |
WO 0116715 | Mar 2001 | WO |
WO 0116716 | Mar 2001 | WO |
WO 0116718 | Mar 2001 | WO |
WO 0116722 | Mar 2001 | WO |
WO 0116758 | Mar 2001 | WO |
WO 0116769 | Mar 2001 | WO |
WO 0116770 | Mar 2001 | WO |
WO 0116782 | Mar 2001 | WO |
WO 0118646 | Mar 2001 | WO |
WO 0141530 | Jun 2001 | WO |
WO 0148596 | Jul 2001 | WO |
WO 0148599 | Jul 2001 | WO |
WO 0148606 | Jul 2001 | WO |
WO 0148619 | Jul 2001 | WO |
WO 0150247 | Jul 2001 | WO |
WO 0150679 | Jul 2001 | WO |
Number | Date | Country | |
---|---|---|---|
20040205747 A1 | Oct 2004 | US |