System and method for logical modeling of distributed computer systems

Information

  • Patent Grant
  • 7574343
  • Patent Number
    7,574,343
  • Date Filed
    Tuesday, March 7, 2006
    18 years ago
  • Date Issued
    Tuesday, August 11, 2009
    14 years ago
Abstract
A modeling system permits developers of applications for distributed computer system, such as those used in server data centers or Internet data centers (IDCs), to architect their hardware and software in an abstract manner. The modeling system defines a set of components that represent abstract functional operations of the application that will eventually be physically implemented by one or more computers and one or more software programs executing on the computers. Associated with the model components is a schema that dictates how the functional operations are to be specified. From the model components, the developers can create logical, scale-independent models of the applications that may be implemented by the distributed computer system. The application is scale-independent in that the application is invariant in respect to the number of computers and software programs that my eventually be used to implement it. The application may subsequently be converted to a physical blueprint that specifies the number and type of hardware/software resources and the physical layout of the distributed computer system.
Description
TECHNICAL FIELD

This invention relates to distributed computer systems, such as Websites and Internet-based Services. More particularly, this invention pertains to a way to model an application for a distributed computer system in a scale-invariant manner that is abstracted from the underlying physical configuration.


BACKGROUND

It is no secret that Internet usage has exploded over the past few years and continues to grow rapidly. People have become very comfortable with many services offered on the World Wide Web (or simply “Web”), such as electronic mail, online shopping, gathering news and information, listening to music, viewing video clips, looking for jobs, and so forth. To keep pace with the growing demand for Internet-based services, there has been tremendous growth in the computer systems dedicated to hosting Websites, providing backend services for those sites, and storing data associated with the sites.


One type of distributed computer system is an Internet data center (IDC), which is a specifically designed complex that houses many computers used for hosting Internet-based services. IDCs, which also go by the names “webfarms” and “server farms”, typically house hundreds to thousands of computers in climate-controlled, physically secure buildings. These computers are interconnected to run one or more programs supporting one or more Internet services or Websites. IDCs provide reliable Internet access, reliable power supplies, and a secure operating environment.



FIG. 1 shows an Internet data center 100. It has many server computers 102 arranged in a specially constructed room. The computers are general-purpose computers, typically configured as servers. An Internet data center may be constructed to house a single site for a single entity (e.g., a data center for Yahoo! or MSN), or to accommodate multiple sites for multiple entities (e.g., an Exodus center that host sites for multiple companies).


The IDC 100 is illustrated with three entities that share the computer resources: entity A, entity B, and entity C. These entities represent various companies that want a presence on the Web. The IDC 100 has a pool of additional s computers 104 that may be used by the entities at times of heavy traffic. For example, an entity engaged in online retailing may experience significantly more demand during the Christmas season. The additional computers give the IDC flexibility to meet this demand.


While there are often many computers, an Internet service or Website may only run a few programs. For instance, one Website may have 2000-3000 computers but only 10-20 distinct software components. Computers may be added daily to provide scalability as the Website receives increasingly more visitors, but the underlying programs change less frequently. Rather, there are simply more computers running the same software in parallel to accommodate the increased volume of visitors.


Today, there is no conventional way to architect Internet Services in a way that abstracts the functionality of the Service from the underlying physical implementation. Little thought has gone into how to describe a complete Internet Service in any manner, let alone a scale-invariant manner. At best, Internet Service operators might draft a document that essentially shows each and every computer, software program, storage device, and communication link in the center as of a specific time and date. The downside with such physical schematics is, of course, that the document is always out of date and hence, it is of limited usefulness as a management tool. Furthermore, while a human may understand such a document, it holds no meaning to a computer.


SUMMARY

A modeling system permits developers to architect distributed computer applications, such as Internet Services or Websites, in an abstract manner. The modeling system defines a set of components that represent functional units of the applications that will eventually be physically implemented by one or more computers and one or more software programs executing on the computers.


In the described implementation, the modeling system defines several model components: a module, a port, and a wire; and a set of model extensions including, but not limited to: a store, an event source, an event sink, and an event wire.


The module is the basic functional unit and represents a container of behavior that may be implemented by one or more computers running one or more software programs. For instance, in the context of a Website, one module might represent a front end that renders HTML pages, another module might represent a login database, and another module might represent a mailbox program. A port is a service access point for the module. All communications into and out of the module goes through a port. A wire is the logical binding that defines an allowed communication route between two ports.


While the model consists of the three basic components described above (namely modules, ports, and wires), the model can be augmented with numerous extensions, specializations of the basic components. For example, a store is a basic unit of storage and a specialization of the module. A store represents a logical amount of storage, which may be implemented by any number of physical disks or other storage media. Like the module, the store represents behavior, in the case the ability to save and retrieve data. Also like the module, the store can communicate with other modules and stores through ports and wires. A store differs from a module in that it is labeled with additional attributes such as the amount of storage required, required access speed, or a minimum number of outstanding storage requests. The store extends the model by adding a specialized type of module with additional semantic information.


The model can be farther augmented with ports extensions. For example, an event source and an event sink are used for discrete semantic messaging between modules and module extensions, such as stores. Event sinks are specialized ports in that they are communication access points between model components, but with additional semantics, namely the specific events.


The model can also be augmented with wires extensions. For example, an event wire is a logical connection between event sources and event sinks, and carries event messages used to inform modules and implement policy. While most wire extensions allow communication at run time, it is possible for some wire extensions to transfer data only at compile or initialization time.


The model components are arranged and interconnected to form a scale-independent model of the application. Each component specifies some functionality of the application.


Associated with the model components is a schema that dictates how the functional operations represented by the components are to be specified. For example, the schema might dictate that a module specify processing requirements, software programs needed to implement the behavior, other modules with which the module should communicate, and so forth. The schema might further dictate that a port and a wire specify a set of attributes describing format, semantics, protocol, and so on. The schema might further dictate that extensions of modules, port, and wires specify further attributes.


From the model components, the developers can create logical, scale-independent models of the applications that may be implemented by the distributed computer systems. An application is scale-independent in that it is invariant with respect to the number of computers and software programs that my eventually be used to implement it. The application may subsequently be converted to a physical blueprint that specifies the number and type of hardware/software resources and the physical layout of the distributed computer system.





BRIEF DESCRIPTION OF THE DRAWINGS


FIG. 1 illustrates a conventional Internet data center (IDC).



FIG. 2 illustrates a set of model components that form the building blocks for modeling an Internet Service, along with the associated schema.



FIG. 3 illustrates a database application for an Internet Service that is modeled in terms of the components.



FIG. 4 illustrates an Internet-based email Internet Service.



FIG. 5 is a block diagram of a computer that may be used to implement the modeling software for modeling the Internet Service.



FIG. 6 is a flow diagram of a process for modeling an Internet Service.





DETAILED DESCRIPTION

A modeling system permits developers of distributed computer applications (e.g., Internet Services, Websites, and the like) to architect their hardware and software in an abstract manner. The modeling system defines a set of components used to describe the functionality of an application in a logical, scale-independent manner. An “application” within this context refers to an entire service hosted on the distributed computers. For instance, an Internet data center may host a Website for an online retailer, where the application entails the entire software and hardware configuration that implements the online retailer's Internet presence. The application might include, for example, a front end to handle client requests, an order processing system, a billing system, an inventory system, and a database system.


The model components are arranged and interconnected to form a scale-independent model of the application. Each component specifies some functionality of the application. The model can then be used to construct a scalable physical blueprint in terms of which machines run which pieces of software to form the application. The model continues to be helpful for managing ongoing operations of the application, forming the skeleton upon which operational behavior and policy mechanisms are built.


In this manner, the modeling system changes the development effort from a node-centric approach to an application-centric approach. With conventional node-centric methodology, the focus was on the computers and how they were laid out. The application was then implemented on as many nodes as needed. With the new application-centric approach, the focus is initially on the application itself. The physical nodes used to implement the application are derived once the application is created.


The modeling system is described in the context of Internet Services and Websites, such as might be deployed in Internet Data Centers, because modeling Internet Services represents one suitable use of the system. However, the modeling system may be implemented to model other large size and scalable computer systems. Accordingly, the modeling system can be implemented in a wide variety ways, including both Internet-based implementations and non-Internet-based implementations.


Model Components and Schema


The modeling system defines several model components that form the building blocks of a logical, scale-independent application: a module, a port, and a wire. It also defines a set of model extensions including, but not limited to: a store, an event source, an event sink, and an event wire. In a design tool, the components are represented pictorially as graphical elements or symbols that may be arranged and interconnected to create scale-independent models of Website applications. The graphical elements have an associated schema that dictates how the functional operations being represented by the graphical elements are to be specified.



FIG. 2 illustrates a set of model components 200 that form the building blocks of logical, scale-independent Internet Services. The components include a module, as represented by modules 202(A)-202(C), ports 206, wires 208, and extensions such as a store 204, event sources 210, event sinks 212, and event wires 214. The components 200 are arranged in a no particular manner other than to foster discussion of their individual traits.


A module 202 represents a basic unit of functionality for the Internet Service. It is a logical entity that represents some portion of the application as might be deployed at the IDC, but it does not necessarily have a physical manifestation. The module often corresponds to a software program that handles a logical set of tasks for the Service. For instance, one module might represent a front end for a Website, another module might represent a login database, and another module might represent an electronic mail program.


Each module 202 is a container of behavior. A simple module is indivisible and has associated a unique identifier. Modules can be nested into a hierarchy of modules to form more complex behaviors. In a module hierarchy, the leaf modules are simple modules, and the non-leaf modules are compound modules.


Each module 202 defines a unit of scaling. While one module logically represents a functional operation of the Service, the module may be deployed to any number of computers when actually implemented. In this way, the module is scale independent, allowing the number of underlying computers used to implement the module to change at over time. When converted to a physical implementation, “module instances” are created from the modules. The module instances are assigned a unique identifier and maintain ancestral data regarding which module created them. The module instances of simple modules are often called “engines”, which are software programs that run on individual computers.


A port 206 is a service access point (SAP) for a module 202 or store 204. All service-related communications into and out of a module go through a port 206. Each port 206 has a “type”, which is a set of attributes describing format, semantics, protocol, and so forth. At runtime, the port represents a set of physical ports associated with the instantiated engines of the modules. Note that a given module might have any number of ports representing different services or functionality provided by the module.


A wire 208 is the logical binding that defines an allowable communication route between two ports 206. Each wire 208 can be type-checked (i.e., with respect to protocols, roles) and defines protocol configuration constraints (e.g., HTTP requires TCP, TCP requires IP, etc.).


Extensions to the model are additional components that specialize the role, behavior, and possibly graphical representation of the base components. Exemplary extensions include, but are not limited to, store 204, event source 210, event sink 212, and event wire 214.


A store 204 is the most basic unit of storage. It represents a logical storage partition, which may be implemented by any number of physical disks or other storage media.


Event sources 210 and event sinks 212 are used for discrete semantic messaging between modules and module extensions, such as stores. An event wire 214 is a logical connection between sources and sinks, and carries event messages used to inform modules or module extensions and implement policy (e.g., scaling, fail-over, monitoring, application processes, etc.).


The event sources 210 and event sinks 212, together with the ports 206, collectively form interfaces for communications to and from the modules 202 and module extensions, such as stores 204. The event sources and sinks may be implemented as ports that are configured for message handling.


The model components 200 are depicted as graphical icons or symbols that may be selected and interconnected using a modeling system (described below in more detail). In the illustrated example, the modules 202 are depicted as blocks, the store 204 is depicted as a disk storage icon, and the ports 206 are depicted as spherical knobs projecting from the modules or module extensions, such as stores. Additionally, the wires 208 are depicted as bold lines, the event sources 210 are depicted as triangles pointing away from the module or module extension, the event sinks 212 are depicted as triangles pointing toward the module or module extension, and the event wire 214 is depicted as a dashed line.


The graphical icons have an associated schema that dictates how the functional operations being represented by the icons are to be specified. For instance, a module icon may have a predefined schema that specifies the hardware and software resources used to implement the functionality represented by the module. Thus, a module for a database function might have characteristics pertaining to the kind of database (e.g., relational), the data structure (e.g., tables, relationships), software (e.g., SQL), software version, and so forth.



FIG. 2 also illustrates the schema underlying the graphical elements as exemplary data structures associated with the model components. Module 202(A) has an associated structure 220 that contains various characteristics for the module, such as functionality, processing requirements, software, and so forth. Modules 202(B) and 202(C) have similar structures (not shown). Model extensions also have associated structures. The store 204 has a corresponding structure 222 that defines the requirements for storage. The store schema structure 222 might include, for example, the kind of storage (e.g., disk), the storage format, and so on.


Each port 206 has a schema structure, as represented by structure 224, which dictates the port's type. Each wire 208 is also associated with a schema structure, such as structure 226, which outlines the protocols implemented by the connection. Similar schema structures may also be provide for event sources, event sinks, and event wires.


Using the model components, a developer can logically describe and configure scale-independent Internet Service prior to physically laying them out in Internet data centers. The developer drafts a model using a user interface to select and interconnect the model components. Once constructed, the modeling software generates the Internet Service based on the depicted model and the underlying schema. The Service may subsequently be converted into a physical blueprint that details the computers and software needed to implement the Service for a specified number of clients.


The scale-invariant nature of the modeling system allows Internet Service developers to focus only on designing software for a specific functional task (e.g., front end, login database, email program, etc.). All external communications can then be expressed in terms of transmitting to and receiving from one or more associated ports. In this manner, the Service developers need not worry about how many machines will be used to run the module; or how other modules of the scale-independent Internet Service are being configured.


Exemplary Module and Application



FIG. 3 shows a fault-tolerant SQL (structure query language) database module 300 to demonstrate how the model components may be organized and connected to represent a portion of an application. In this example, the database module 300 represents a SQL database that may be used independently or as a component in a larger application. The SQL database module 300 has a module interface composed of a single port 302 that implements the TDS (Tabular Data Stream) protocol.


The SQL database module 300 is a compound module made up of three simple modules: a fail-over policy module 310, a primary SQL module 312, and a secondary SQL module 314. The primary and secondary SQL modules represent dual programs that operate in parallel so that, in the event that the primary module 312 crashes, the secondary module 314 can assume the role without loss of service. The database module 300 also has a data store 316 that represents the memory storage for the SQL database module.


The primary SQL module 312 has a module interface that includes a first port 320 for communicating with the compound module port 302 and a second port 322 for communicating with the store 316. The primary SQL module 312 also has an event source 324 and an event sink 326 for handling event messages from the fail-over policy module 310. Similarly, the secondary SQL module 314 has a module interface with a first port 330 for communicating with the compound module port 302, a second port 332 for communicating with the store 316, and an event sink 334 for receiving events from the fail-over policy module 310. A wire 336 interconnects the external compound module port 302 with the ports 320 and 330 of the primary and secondary SQL modules, respectively.


The store 316 has a port 340 to communicate with the primary and secondary SQL modules 312 and an event sink 342 to receive event messages from the fail-over policy module 310. A wire 344 interconnects the store port 340 with the ports 322 and 332 of the primary and secondary SQL modules, respectively.


The fail-over policy module 310 has a module interface that includes three event sources and one event sink. An event sink 350 receives a “fail” event from the event source 324 of the primary SQL module 312 via an event wire 352 when the primary SQL module experiences some failure. In response to receiving a “fail” event, the fail-over policy module 310 concurrently issues a first event to stop the failed primary module 312, another event to assign the secondary module 314 as the new owner of the store 316, and a third event to start the secondary module 314. The “stop” event is issued via an event source 354 over an event wire 356 to the event sink 326 of the primary SQL module 312. The “stop” event directs the primary SQL module 312 to halt operation.


The fail-over policy module 310 issues an “assign owner” (AO) event from an event source 358, over the event wire 360 to the event sink 342 of the store 316. The assign owner event directs the storage mechanisms to switch to allowing access by the secondary SQL module 314, rather than the primary SQL module 312. The fail-over policy module 310 also issues a “start” event from event source 362 over event wire 364 to the event sink 334 of the secondary module 314. The start event directs the secondary SQL module to start operation in place of the primary SQL module.


The SQL database module 300 illustrates how the base model components and exemplary model extensions—modules, ports, wires, stores, event sources, event sinks, and event wires—may be arranged and interconnected to form a complex module. The developer specifies the characteristics associated with each component according to the prescribed schema. The complex module may in turn be added to other simple or complex modules to form other complex modules. Eventually, the largest complex module becomes the Internet Service, which may then be used to form a blueprint for deploying to the data center.



FIG. 4 shows a simplified application 400 for an online retailer. The application 400 includes a front end module 402, a catalog module 404, an order processing module 406, and a fulfillment module 408. The application 400 also includes a customer database 410 and the fault-tolerant SQL database module 300. Notice that the SQL database module 300 is the same as that shown in FIG. 3 to illustrate how complex modules can be nested into even greater complex modules to form an application.


The front end module 402 handles requests from clients who wish to shop with the online retailer. The front end module 402 has a port 420 that accommodates communications with external clients using the TCP/IP protocol over the Internet. The front end module 402 also has an order port 422 to define a communication exchange with the order processing module 406 and a catalog port 424 for communication flow to the catalog module 404. The ports 422 and 424 may be configured according to any of a variety of types, which support any of a number of protocols including SOAP, TCP, or UDP. An event sink 426 is also provided to receive a “new product” message from the catalog module 404 when a new product has been added to the catalog.


The catalog module 404 provides catalog information that may be served by the front end to the requesting clients. The catalog module 404 has a front end port 430 connected via a wire 432 to the catalog port 424 of the front end module 402. The front end port 430 has a type that matches the catalog port 424. The catalog module 404 also has an event source 434 for communicating the “new product” messages over wire 436 to the event sink 426 of the front end module 402.


A SQL port 438 interfaces the catalog module 404 with the SQL database module 300. The SQL port 438 has a type that utilizes the TDS protocol for the communication exchange with the external port 302 of the SQL database 300.


The order processing module 406 has a front end port 440 to define a communication interface with the front end module 402 via a wire 442. The order processing module 406 also has a fulfillment port 444 to facilitate communication with the fulfillment module 408 over wire 446 and a database port 448 to facilitate communication with the customer database 410 via wire 450.


An event source 452 is provided at the order processing module 406 to pass “order complete” events to the fulfillment module 408 via wire 454. These events inform the fulfillment module 408 that an order is complete and ready to be filled. A second event source 456 passes “new account” events to the customer database 410 via wire 458 whenever a new customer orders a product.


The fulfillment module 408 has an order port 460 to provide access to the wire 446 to the order processing module 406 and a database port 462 to interface with the customer database 410. The fulfillment module 408 also has an event sink 464 to receive the “order complete” events from the order processing module 406.


The customer database 410 has an order port 470 to provide access to wire 450 and a fulfillment port 472 to facilitate communication with the fulfillment module 408 via wire 474. The customer database 410 further has an event sink 476 to receive the “new account” events from the order processing module 406.


The modeling approach illustrated in FIGS. 3 and 4 is tremendously beneficial because it allows developers and IDC operators to view the entire Internet Service in terms of functional pieces independent of deployment scale. The online retailer Internet Service 400, for example, requires a front end unit, a catalog unit, an order processing unit, and a fulfillment unit regardless of whether the retailer is handling 100 hits a day or 10 million hits per day.


The scale-independent nature frees the developer to focus on his/her little piece of the Service. For instance, a developer assigned the task of building the front end module 402 need only be concerned with writing software code to facilitate response/reply exchanges. Any communication to and from the module is defined in terms of order-related data being passed to the order processing module 406 via the order port 422 and product data being received from the catalog module 404 via the catalog port 424. The developer defines the data flow to and from the order port 422 and the catalog port 424 according to their respective associated protocol types.


The Internet Service 400 can then be used to construct a computer system that hosts the online retailer. Initially, the online retailer may not receive very much traffic, especially if launched away from the Christmas season. So, perhaps the front end module 402 deploys initially to only a few computers to handle the light traffic from the Internet. But, suppose that over time the site becomes more popular and the Christmas season is fast approaching. In this situation, the online retailer may authorize the IDC operator to add many more computers for the front end tasks. These computers are equipped with software and configured to accept HTTP requests for product information and to serve web pages containing the product information. The computers are added (or removed) as needed, without altering the basic description of the Internet Service 400.


Computer-Based Modeling System and Method



FIG. 5 shows an exemplary computer system 500 that implements modeling software used to design Internet Services. The modeling computer may be implemented as one of the nodes in a Internet Service, or as a separate computer not included as one of the nodes. The modeling computer has a processor 502, volatile memory 504 (e.g., RAM), and non-volatile memory 506 (e.g., ROM, Flash, hard disk, optical, RAID memory, etc.). The modeling computer 500 runs an operating system 510 and modeling system 512.


For purposes of illustration, operating system 510 and modeling system 512 are illustrated as discrete blocks stored in the non-volatile memory 506, although it is recognized that such programs and components reside at various times in different storage components of the computer 500 and are executed by the processor 502. Generally, these software components are stored in non-volatile memory 506 and from there, are loaded at least partially into the volatile main memory 504 for execution on the processor 502.


The modeling system 512 includes a user interface 514 (e.g., a graphical UI) that presents the pictorial icons of the model components 516 (e.g., modules, ports, sources, sinks, etc.), a component schema database 518, a logical-to-physical converter 520, and an instance-tracking database 522. The modeling system 512 allows a developer to design an Internet Service by defining modules, ports, wires, and event message schemes. The user interface 514 presents symbols of the components 516, such as the symbols shown in FIGS. 2-4, and permits the developer to arrange and interconnect them. The UI 514 may even support conventional UI techniques as drag-and-drop operations.


The symbols depicted on the screen represent an underlying schema 518 that is used to define the model. For instance, a block-like module symbol is associated with the characteristics of the functionality that the module is to represent in the Internet Service. Thus, the developer may define a database module that has characteristics pertaining to the kind of database (e.g., relational), the data structure (e.g., tables, relationships), software (e.g., SQL), software version, and so forth. Accordingly, by drafting the model on the UI, the developer is architecting the entire schema that will be used to design the scale-independent Internet Service.


Once the Internet Service is created, the logical-to-physical converter 520 converts the Service to a physical blueprint that details the number of computers, software components, physical ports, and so forth. The converter takes various parameters—such as how many site visitors are expected, memory requirements, bandwidth requirements, processing capabilities, and the like—and scales the Internet Service according to the schema 518 created by the developer. The converter 520 specifies the number of computers needed to implement each module, the number of disks to accommodate the stores, and the types of communications protocols among the modules and stores. The identity of every component instance is recorded in an instance-tracking database 522. Instances in the instance-tracking database 522 include those for modules, port, wires, and instances of model extensions such as stores, event ports, and event wires.


In one embodiment, the developer writes management policy, which issues commands on the schema 518 to create new instances of modules, port, and wires to deploy the Internet Service. Developers may choose to write management policy instead of using fully automatic logical-to-physical converter 520 when they want finer control over the growth and management of the Internet Service. The management code issues commands using the namespace defined by the schema 518, but the commands operate on individual module, port, and wire instances. The commands are still dispatched through the converter 520, which allocates nodes to the management policy. Whether operating automatically or driven by management policy code, the convert 520 records in the instance-tracking database 522 the individual instances of modules, port, and wires.


In this manner, the modeling system changes the development effort from a node-centric approach for architecting Internet Services to an application-centric approach. Within conventional node-centric methodology, the focus was on the computers and how they were laid out. The Internet Services was then loaded onto the nodes in an ad hoc manner. With the new application-centric approach, the focus is initially on the Internet Service itself. The physical nodes used to implement the Internet Service are derived in terms of the Service schema once it is specified. The instance-tracking database 522 gives both developers and operators information about how many instances of each module are running at any time and how the modules are connected using port instances and wires instances within the Service schema.



FIG. 6 shows a method for modeling a scale-independent Internet Service. The method 600 may be implemented, for example, by the modeling system 512 executing on the modeling computer 500. In such an implementation, the method is implemented in software that, when executed on computer 500, performs the operations illustrated as blocks in FIG. 6.


At block 602, the modeling system 512 allows the developer to define the modules and extensions, such as stores, that form the functional elements of the Internet Service. The UI 514 enables the developer to create modules and extensions, such as stores, and to define their characteristics as prescribed by a predetermined schema. This entry process begins to construct the logical building blocks of the Service.


At block 604, the modeling system 512 enables the developer to define the ports for the modules and module extensions, such as stores. The developer selects the type of ports. The modeling system ensures compatibility of ports that are connected to one another. At block 606, the developer also defines other extensions, such as events that may be passed among modules and module extensions, such as stores. For example, the developer creates event sources and event sinks to accommodate the various events. At block 608, the developer uses the modeling system 512 to interconnect the ports with wires and the port extensions, such as event sources/sinks with wire extensions, such as event wires. By joining the various modules and module extensions, such as stores, the developer effectively forms a logical representation of the Internet Service.


At block 610, the modeling system 512 generates an Internet Service using the graphical representation constructed by the developer through its associated schema. The modeling system 512 generates the logical specifications associated with the graphical model, including the characteristics of the modules and module extensions, such as stores, as well as the types of the ports and port extensions such as event sources/sinks. The Internet Service provides a complete logical representation of the Service that will eventually be implemented at the Internet data center. The Internet Service description may be stored on disk or some other form of computer-readable medium (block 612).


At block 614, the modeling system 512 converts the Internet Service description to a physical blueprint that specifies the computers, the software run by each of the computers, and the interconnections among the computers. This physical blueprint may be used by the operator to install and manage the Internet Service.


CONCLUSION

Although the description above uses language that is specific to structural features and/or methodological acts, it is to be understood that the invention defined in the appended claims is not limited to the specific features or acts described. Rather, the specific features and acts are disclosed as exemplary forms of implementing the invention.

Claims
  • 1. A computer-implemented method, wherein the method is implemented by a processor executing computer-executable instructions stored in a memory, the method comprising: generating a design of a distributed computer system at least in part by representing hardware and software resources of the distributed computer system as model components to be included in a design for the distributed computer system, wherein the model components are selected from a group comprising: a module that is representative of a behavior of an application that is implemented using the hardware and software resources and is to be implemented by the distributed computer system under design;a port that is representative of a service access point for the module; anda wire that is representative of an allowable communication connection between two or more ports; andforming, from the model components included in the design, a logical scale-independent model of the application to be implemented by the distributed computer system under design; and simulating the distributed computer system using the generated design.
  • 2. The method of claim 1, wherein at least one of the model components represent one or more similar resources.
  • 3. The method of claim 1, wherein at least one of the model components are depicted in a graphical user interface as a graphical icon.
  • 4. The method of claim 1, wherein the model components have an associated schema that specifies the hardware and software resources represented by the model components.
  • 5. The method of claim 1, wherein the group of the model components further comprises a store that is representative of persistent data storage.
  • 6. The method of claim 1, wherein the group of the model components further comprises at least one of: a store that is representative of persistent data storage;an event source that is representative of a logical connection point for the module or the store from which event messages originate;an event sink that is representative of a logical connection point for the module or the store to receive the event messages; andan event wire that is representative of an interconnection between the event source and the event sink.
  • 7. The method of claim 1, further comprising storing the scale-independent model on a computer-readable medium.
  • 8. The method of claim 1, further comprising converting the scale-independent model into a blueprint of a server data center, the blueprint specifying the hardware and software resources used to physically implement the application.
  • 9. A computer-readable medium storing computer-executable instructions that, when executed on a computer, perform the method of claim 1.
  • 10. A computer-readable medium comprising computer-executable instructions that, when executed on one or more processors, direct a computing device to: enable design of a distributed computer system at least in part by: defining individual model components as abstract functional operations that are physically implemented by one or more computers to be included in a design of the distributed computer system and one or more software programs executing on the computers, the model components having an associated schema dictating how the functional operations are specified, wherein the model components represent resources provided by the computers and the software programs, the resources being scalable from one to many, wherein the model components are selected from a group comprising: a module that is representative of a behavior of the application, a port that is representative of a communication access point for the module, and a wire that is representative of an interconnection between two or more ports;interconnecting the model components to logically connect the functional operations; andgenerating a scale-independent application from the interconnected model components and the associated schema; andconverting the scale-independent application into a blueprint that specifies the computers and the software programs used to physically implement the application.
  • 11. The computer-readable medium of claim 10, further comprising computer-executable instructions that, when executed, cause at least one of the model components to be depicted in a graphical user interface as a graphical icon.
  • 12. The computer-readable medium of claim 10, further comprising computer-executable instructions that, when executed, enable entering, via a user interface, a description of elements for implementing the functional operations.
  • 13. The computer-readable medium of claim 10, further comprising computer-executable instructions that, when executed, define an instance tracking database that provides information regarding how many instances of one more modules specified in the blueprint are running at a specified time.
  • 14. The computer-readable medium of claim 13, wherein the information further identifies how the one or more modules are connected using port instances and wire instances.
  • 15. The computer-readable medium of claim 10, wherein the group of the model components further comprises: a store that is representative of persistent data storage;an event source that is representative of a logical connection point for the module or the store from which event messages originate;an event sink that is representative of a logical connection point for the module or the store to receive the event messages; andan event wire that is representative of an interconnection between the event source and the event sink.
  • 16. A system comprising: a processor;a memory coupled to the processor;computer-executable instructions stored in the memory and executed by the processor for enabling design of a distributed computer system at least in part by representing hardware and software resources of the distributed computer system as model components to be included in a design for the distributed computer system, wherein the model components are selected from a group comprising: a module that is representative of a behavior of an application that is implemented using the hardware and software resources and is to be implemented by the distributed computer system under design;a port that is representative of a communication access point for the module; anda wire that is representative of an interconnection between two or more ports; andwherein the computer-executable instructions executed by the processor associate the model components included in the design with a schema dictating how the hardware and software resources are specified.
  • 17. The system of claim 16, wherein the group of the model components further comprises: a store that is representative of persistent data storage;an event source that is representative of a logical connection point for the module or the store from which event messages originate;an event sink that is representative of a logical connection point for the module or the store to receive the event messages; andan event wire that is representative of an interconnection between the event source and the event sink.
  • 18. The system of claim 16, wherein the computer-executable instructions executed by the processor create a scale-independent application from the model components and the associated schema.
  • 19. A system of claim 16, wherein the computer-executable instructions executed by the processor convert the scale-independent application into a blueprint that specifies the hardware and software resources used to physically implement the application on the distributed computer system.
Parent Case Info

This application is a continuation of U.S. application Ser. No. 09/695,813, filed on Oct. 24, 2000, entitled “System and Method for Logical Modeling of Distributed Computer Systems”, now U.S. Pat. No. 7,113,900. The benefit of the filing date of this parent application is hereby claimed to the fullest extent permitted by 35 U.S.C. § 120, and the contents of the parent application are incorporated by this reference as if set forth verbatim herein.

US Referenced Citations (474)
Number Name Date Kind
4200770 Hellman et al. Apr 1980 A
4218582 Hellman et al. Aug 1980 A
4405829 Rivest et al. Sep 1983 A
4424414 Hellman et al. Jan 1984 A
5031089 Liu et al. Jul 1991 A
5115505 Bishop et al. May 1992 A
5220621 Saitoh Jun 1993 A
5371852 Attanasio et al. Dec 1994 A
5430810 Saeki Jul 1995 A
5475817 Waldo et al. Dec 1995 A
5490276 Doli, Jr. et al. Feb 1996 A
5499357 Sonty et al. Mar 1996 A
5504921 Dev et al. Apr 1996 A
5539883 Allon et al. Jul 1996 A
5557774 Shimabukuro et al. Sep 1996 A
5579482 Einkauf et al. Nov 1996 A
5668995 Bhat Sep 1997 A
5686940 Kuga Nov 1997 A
5724508 Harple, Jr. et al. Mar 1998 A
5748958 Badovinatz et al. May 1998 A
5758351 Gibson et al. May 1998 A
5768271 Seid et al. Jun 1998 A
5774660 Brendel et al. Jun 1998 A
5774668 Choquier et al. Jun 1998 A
5774689 Curtis et al. Jun 1998 A
5784463 Chen et al. Jul 1998 A
5790895 Krontz et al. Aug 1998 A
5801970 Rowland et al. Sep 1998 A
5815574 Fortinsky Sep 1998 A
5818937 Watson et al. Oct 1998 A
5822531 Gorczyca et al. Oct 1998 A
5826015 Schmidt Oct 1998 A
5845124 Berman Dec 1998 A
5845277 Pfeil et al. Dec 1998 A
5850399 Ganmukhi et al. Dec 1998 A
5867706 Martin et al. Feb 1999 A
5872914 Walker, Jr. et al. Feb 1999 A
5878220 Olkin et al. Mar 1999 A
5895499 Chu Apr 1999 A
5905728 Han et al. May 1999 A
5917730 Rittie et al. Jun 1999 A
5918017 Attanasio et al. Jun 1999 A
5930798 Lawler et al. Jul 1999 A
5938732 Lim et al. Aug 1999 A
5948055 Pulsipher et al. Sep 1999 A
5951694 Choquier et al. Sep 1999 A
5958009 Friedrich et al. Sep 1999 A
5960371 Saito et al. Sep 1999 A
5968126 Ekstrom et al. Oct 1999 A
6012113 Tuckner Jan 2000 A
6035405 Gage et al. Mar 2000 A
6041054 Westberg Mar 2000 A
6047323 Krause Apr 2000 A
6047325 Jain et al. Apr 2000 A
6049528 Hendel et al. Apr 2000 A
6052469 Johnson et al. Apr 2000 A
6059842 Dumarot et al. May 2000 A
6065058 Hailpern et al. May 2000 A
6070243 See et al. May 2000 A
6073183 Slonim Jun 2000 A
6075776 Tanimoto et al. Jun 2000 A
6076108 Courts et al. Jun 2000 A
6081826 Masuoka et al. Jun 2000 A
6085238 Yuasa et al. Jul 2000 A
6086618 Al-Hilali et al. Jul 2000 A
6097818 Saito Aug 2000 A
6108699 Moiin Aug 2000 A
6108702 Wood Aug 2000 A
6111993 Matsunaga Aug 2000 A
6112243 Downs et al. Aug 2000 A
6115393 Engel et al. Sep 2000 A
6118785 Araujo et al. Sep 2000 A
6125442 Maves et al. Sep 2000 A
6125447 Gong Sep 2000 A
6141749 Coss et al. Oct 2000 A
6144959 Anderson et al. Nov 2000 A
6147995 Dobbins et al. Nov 2000 A
6151688 Wipfel et al. Nov 2000 A
6167052 McNeill et al. Dec 2000 A
6167383 Henson Dec 2000 A
6167515 Lin Dec 2000 A
6178529 Short et al. Jan 2001 B1
6182275 Beelitz et al. Jan 2001 B1
6185308 Ando et al. Feb 2001 B1
6192401 Modiri et al. Feb 2001 B1
6195091 Harple et al. Feb 2001 B1
6195355 Demizu Feb 2001 B1
6208345 Sheard et al. Mar 2001 B1
6208649 Kloth Mar 2001 B1
6209099 Saunders Mar 2001 B1
6212559 Bixler et al. Apr 2001 B1
6215877 Matsumoto Apr 2001 B1
6215878 Harkins Apr 2001 B1
6226788 Schoening et al. May 2001 B1
6230312 Hunt May 2001 B1
6233610 Hayball et al. May 2001 B1
6236365 LeBlanc et al. May 2001 B1
6236901 Goss May 2001 B1
6253230 Couland et al. Jun 2001 B1
6256773 Bowman-Amuah Jul 2001 B1
6259448 McNally et al. Jul 2001 B1
6263089 Otsuka et al. Jul 2001 B1
6266707 Boden et al. Jul 2001 B1
6269076 Shamir et al. Jul 2001 B1
6269079 Marin et al. Jul 2001 B1
6272522 Lin et al. Aug 2001 B1
6272523 Factor Aug 2001 B1
6305015 Akriche et al. Oct 2001 B1
6311144 Abu El Ata Oct 2001 B1
6311270 Challener et al. Oct 2001 B1
6317438 Trebes, Jr. Nov 2001 B1
6324571 Hacherl Nov 2001 B1
6327622 Jindal et al. Dec 2001 B1
6330602 Law et al. Dec 2001 B1
6330605 Christensen et al. Dec 2001 B1
6336138 Caswell et al. Jan 2002 B1
6336171 Coskrey, IV Jan 2002 B1
6338112 Wipfel et al. Jan 2002 B1
6341356 Johnson et al. Jan 2002 B1
6351685 Dimitri et al. Feb 2002 B1
6353861 Dolin, Jr. et al. Mar 2002 B1
6353898 Wipfel et al. Mar 2002 B1
6360265 Falck et al. Mar 2002 B1
6366578 Johnson Apr 2002 B1
6367010 Venkatram et al. Apr 2002 B1
6370573 Bowman-Amuah Apr 2002 B1
6370584 Bestavros et al. Apr 2002 B1
6377996 Lumelsky et al. Apr 2002 B1
6389464 Krishnamurthy et al. May 2002 B1
6393386 Zager et al. May 2002 B1
6393456 Ambler et al. May 2002 B1
6393474 Eichert et al. May 2002 B1
6393485 Chao et al. May 2002 B1
6408390 Saito Jun 2002 B1
6424718 Holloway Jul 2002 B1
6424992 Devarakonda et al. Jul 2002 B2
6427163 Arendt et al. Jul 2002 B1
6427171 Craft et al. Jul 2002 B1
6438100 Halpern et al. Aug 2002 B1
6442557 Buteau et al. Aug 2002 B1
6449641 Moiin et al. Sep 2002 B1
6449650 Westfall et al. Sep 2002 B1
6457048 Sondur et al. Sep 2002 B2
6463536 Saito Oct 2002 B2
6466932 Dennis et al. Oct 2002 B1
6466978 Mukherjee et al. Oct 2002 B1
6466984 Naveh et al. Oct 2002 B1
6466985 Goyal et al. Oct 2002 B1
6470025 Wilson et al. Oct 2002 B1
6470332 Weschler Oct 2002 B1
6470464 Bertram et al. Oct 2002 B2
6473791 Al-Ghosein et al. Oct 2002 B1
6480955 DeKoning et al. Nov 2002 B1
6484261 Wiegel Nov 2002 B1
6487622 Coskrey, IV et al. Nov 2002 B1
6493715 Funk et al. Dec 2002 B1
6496187 Deering et al. Dec 2002 B1
6502131 Vaid et al. Dec 2002 B1
6505244 Natarajan et al. Jan 2003 B1
6510154 Mayes et al. Jan 2003 B1
6510509 Chopra et al. Jan 2003 B1
6519615 Wollrath et al. Feb 2003 B1
6529953 Van Renesse Mar 2003 B1
6539494 Abramson et al. Mar 2003 B1
6542504 Mahler et al. Apr 2003 B1
6546423 Dutta et al. Apr 2003 B1
6546553 Hunt Apr 2003 B1
6549516 Albert et al. Apr 2003 B1
6549934 Peterson et al. Apr 2003 B1
6564261 Gudjonsson et al. May 2003 B1
6570847 Hosein May 2003 B1
6570875 Hegde May 2003 B1
6574195 Roberts Jun 2003 B2
6578144 Gennaro et al. Jun 2003 B1
6584499 Jantz et al. Jun 2003 B1
6587876 Mahon et al. Jul 2003 B1
6598077 Primak et al. Jul 2003 B2
6598173 Sheikh et al. Jul 2003 B1
6598223 Vrhel, Jr. et al. Jul 2003 B1
6601101 Lee et al. Jul 2003 B1
6606708 Devine et al. Aug 2003 B1
6609148 Salo et al. Aug 2003 B1
6609213 Nguyen et al. Aug 2003 B1
6611522 Zheng et al. Aug 2003 B1
6615256 van Ingen et al. Sep 2003 B1
6628671 Dynarski et al. Sep 2003 B1
6631141 Kumar et al. Oct 2003 B1
6640303 Vu Oct 2003 B1
6651101 Gai et al. Nov 2003 B1
6654782 O'Brien et al. Nov 2003 B1
6654796 Slater et al. Nov 2003 B1
6665714 Blumenau et al. Dec 2003 B1
6671699 Black et al. Dec 2003 B1
6675308 Thomsen Jan 2004 B1
6678821 Waugh et al. Jan 2004 B1
6678835 Shah et al. Jan 2004 B1
6681262 Rimmer Jan 2004 B1
6684335 Epstein, III et al. Jan 2004 B1
6691148 Zinky et al. Feb 2004 B1
6691168 Bal et al. Feb 2004 B1
6694436 Audebert Feb 2004 B1
6701363 Chiu et al. Mar 2004 B1
6717949 Boden et al. Apr 2004 B1
6718361 Basani et al. Apr 2004 B1
6718379 Krishna et al. Apr 2004 B1
6725253 Okano et al. Apr 2004 B1
6728885 Taylor et al. Apr 2004 B1
6735596 Corynen May 2004 B2
6738736 Bond May 2004 B1
6741266 Kamiwada et al. May 2004 B1
6742020 Dimitroff et al. May 2004 B1
6748447 Basani et al. Jun 2004 B1
6754716 Sharma et al. Jun 2004 B1
6754816 Layton et al. Jun 2004 B1
6757744 Narisi et al. Jun 2004 B1
6760765 Asai et al. Jul 2004 B1
6760775 Anerousis et al. Jul 2004 B1
6769008 Kumar et al. Jul 2004 B1
6769060 Dent et al. Jul 2004 B1
6772333 Brendel Aug 2004 B1
6779016 Aziz et al. Aug 2004 B1
6782408 Chandra et al. Aug 2004 B1
6801528 Nassar Oct 2004 B2
6801937 Novaes et al. Oct 2004 B1
6804783 Wesinger et al. Oct 2004 B1
6813778 Poli et al. Nov 2004 B1
6816897 McGuire Nov 2004 B2
6820121 Callis et al. Nov 2004 B1
6823299 Contreras et al. Nov 2004 B1
6823373 Pancha et al. Nov 2004 B1
6823382 Stone Nov 2004 B2
6829639 Lawson et al. Dec 2004 B1
6829770 Hinson et al. Dec 2004 B1
6836750 Wong et al. Dec 2004 B2
6845160 Aoki Jan 2005 B1
6853841 St. Pierre Feb 2005 B1
6854069 Kampe et al. Feb 2005 B2
6856591 Ma et al. Feb 2005 B1
6862613 Kumar et al. Mar 2005 B1
6868062 Yadav et al. Mar 2005 B1
6868454 Kubota et al. Mar 2005 B1
6880002 Hirschfeld et al. Apr 2005 B2
6886038 Tabbara et al. Apr 2005 B1
6888807 Heller et al. May 2005 B2
6895534 Wong et al. May 2005 B2
6898791 Chandy et al. May 2005 B1
6904458 Bishop et al. Jun 2005 B1
6907395 Hunt et al. Jun 2005 B1
6915338 Hunt et al. Jul 2005 B1
6922791 Mashayekhi et al. Jul 2005 B2
6928482 Ben Nun et al. Aug 2005 B1
6944759 Crisan Sep 2005 B1
6947987 Boland Sep 2005 B2
6957186 Guheen et al. Oct 2005 B1
6963981 Bailey et al. Nov 2005 B1
6968291 Desai Nov 2005 B1
6968550 Branson et al. Nov 2005 B2
6968551 Hediger et al. Nov 2005 B2
6971063 Rappaport et al. Nov 2005 B1
6971072 Stein Nov 2005 B1
6973620 Gusler et al. Dec 2005 B2
6973622 Rappaport et al. Dec 2005 B1
6976079 Ferguson et al. Dec 2005 B1
6976269 Avery, IV et al. Dec 2005 B1
6983317 Bishop et al. Jan 2006 B1
6985956 Luke et al. Jan 2006 B2
6990666 Hirschfeld et al. Jan 2006 B2
7003562 Mayer Feb 2006 B2
7003574 Bahl Feb 2006 B1
7012919 So et al. Mar 2006 B1
7013462 Zara et al. Mar 2006 B2
7016950 Tabbara et al. Mar 2006 B2
7024451 Jorgenson Apr 2006 B2
7027412 Miyamoto et al. Apr 2006 B2
7028228 Lovy et al. Apr 2006 B1
7035786 Abu El Ata et al. Apr 2006 B1
7035930 Graupner et al. Apr 2006 B2
7043407 Lynch et al. May 2006 B2
7043545 Tabbara et al. May 2006 B2
7046680 McDysan et al. May 2006 B1
7050961 Lee et al. May 2006 B1
7054943 Goldszmidt et al. May 2006 B1
7058704 Mangipudi et al. Jun 2006 B1
7058826 Fung Jun 2006 B2
7058858 Wong et al. Jun 2006 B2
7062718 Kodosky et al. Jun 2006 B2
7069480 Lovy et al. Jun 2006 B1
7069553 Narayanaswamy et al. Jun 2006 B2
7072807 Brown et al. Jul 2006 B2
7072822 Humenansky et al. Jul 2006 B2
7076633 Tormasov et al. Jul 2006 B2
7080143 Hunt et al. Jul 2006 B2
7082464 Hasan et al. Jul 2006 B2
7089281 Kazemi et al. Aug 2006 B1
7089293 Grosner et al. Aug 2006 B2
7089530 Dardinski et al. Aug 2006 B1
7093005 Patterson Aug 2006 B2
7093288 Hydrie et al. Aug 2006 B1
7096258 Hunt et al. Aug 2006 B2
7099936 Chase et al. Aug 2006 B2
7103185 Srivastava et al. Sep 2006 B1
7103874 McCollum et al. Sep 2006 B2
7113900 Hunt et al. Sep 2006 B1
7117158 Weldon et al. Oct 2006 B2
7117261 Kryskow, Jr. et al. Oct 2006 B2
7120154 Bavant et al. Oct 2006 B2
7124289 Suorsa Oct 2006 B1
7127625 Farkas et al. Oct 2006 B2
7131123 Suorsa et al. Oct 2006 B2
7134011 Fung Nov 2006 B2
7134122 Sero et al. Nov 2006 B1
7139930 Mashayekhi et al. Nov 2006 B2
7139999 Bowman-Amuah Nov 2006 B2
7143420 Radhakrishnan Nov 2006 B2
7146353 Garg et al. Dec 2006 B2
7150015 Pace et al. Dec 2006 B2
7152109 Suorsa et al. Dec 2006 B2
7152157 Murphy et al. Dec 2006 B2
7155380 Hunt et al. Dec 2006 B2
7155490 Malmer et al. Dec 2006 B1
7162427 Myrick et al. Jan 2007 B1
7162509 Brown et al. Jan 2007 B2
7174379 Agarwal et al. Feb 2007 B2
7181731 Pace et al. Feb 2007 B2
7191344 Lin et al. Mar 2007 B2
7194439 Kassan et al. Mar 2007 B2
7194616 Axnix et al. Mar 2007 B2
7197418 Fuller, III et al. Mar 2007 B2
7200530 Brown et al. Apr 2007 B2
7200655 Hunt et al. Apr 2007 B2
7203911 Williams Apr 2007 B2
7213231 Bandhole et al. May 2007 B1
7222147 Black et al. May 2007 B1
7225441 Kozuch et al. May 2007 B2
7231410 Walsh et al. Jun 2007 B1
7254634 Davis et al. Aug 2007 B1
7257584 Hirschfeld et al. Aug 2007 B2
7275156 Balfanz et al. Sep 2007 B2
7278273 Whitted et al. Oct 2007 B1
7281154 Mashayekhi et al. Oct 2007 B2
7302608 Acharya et al. Nov 2007 B1
7305549 Hunt et al. Dec 2007 B2
7305561 Hunt et al. Dec 2007 B2
7313573 Leung et al. Dec 2007 B2
7315801 Dowd et al. Jan 2008 B1
7333000 Vassallo Feb 2008 B2
7349891 Charron et al. Mar 2008 B2
7350068 Anderson et al. Mar 2008 B2
7350186 Coleman et al. Mar 2008 B2
7367028 Kodosky et al. Apr 2008 B2
7370103 Hunt et al. May 2008 B2
7379982 Tabbara May 2008 B2
7386721 Vilhuber et al. Jun 2008 B1
7395320 Hunt et al. Jul 2008 B2
7403901 Carley et al. Jul 2008 B1
7406517 Hunt et al. Jul 2008 B2
7464147 Fakhouri et al. Dec 2008 B1
20010014158 Baltzley Aug 2001 A1
20010016909 Gehrmann Aug 2001 A1
20010019554 Nomura et al. Sep 2001 A1
20010020228 Cantu et al. Sep 2001 A1
20010039586 Primak et al. Nov 2001 A1
20010047400 Coates et al. Nov 2001 A1
20010051937 Ross et al. Dec 2001 A1
20020009079 Jungck et al. Jan 2002 A1
20020022952 Zager et al. Feb 2002 A1
20020038421 Hamada Mar 2002 A1
20020040402 Levy-Abegnoli et al. Apr 2002 A1
20020049573 El Ata Apr 2002 A1
20020057684 Miyamoto et al. May 2002 A1
20020069267 Thiele Jun 2002 A1
20020069369 Tremain Jun 2002 A1
20020075844 Hagen Jun 2002 A1
20020087264 Hills et al. Jul 2002 A1
20020090089 Branigan et al. Jul 2002 A1
20020095524 Sanghvi et al. Jul 2002 A1
20020099785 Teeple Jul 2002 A1
20020120761 Berg Aug 2002 A1
20020131601 Ninomiya et al. Sep 2002 A1
20020138551 Erickson Sep 2002 A1
20020152086 Smith et al. Oct 2002 A1
20020156900 Marquette et al. Oct 2002 A1
20020171690 Fox et al. Nov 2002 A1
20020194342 Lu et al. Dec 2002 A1
20020194345 Lu et al. Dec 2002 A1
20020194369 Rawlings et al. Dec 2002 A1
20020198995 Liu et al. Dec 2002 A1
20030008712 Poulin Jan 2003 A1
20030009559 Ikeda Jan 2003 A1
20030014644 Burns et al. Jan 2003 A1
20030028642 Agarwal et al. Feb 2003 A1
20030028770 Litwin, Jr. et al. Feb 2003 A1
20030041139 Beadles et al. Feb 2003 A1
20030041142 Zhang et al. Feb 2003 A1
20030041159 Tinsley et al. Feb 2003 A1
20030046615 Stone Mar 2003 A1
20030051049 Noy et al. Mar 2003 A1
20030056063 Hochmuth et al. Mar 2003 A1
20030065743 Jenny et al. Apr 2003 A1
20030069369 Belenkaya et al. Apr 2003 A1
20030074395 Eshghi et al. Apr 2003 A1
20030101284 Cabrera et al. May 2003 A1
20030105963 Slick et al. Jun 2003 A1
20030120763 Voilpano Jun 2003 A1
20030126230 Donatelli et al. Jul 2003 A1
20030126464 McDaniel et al. Jul 2003 A1
20030130833 Brownell et al. Jul 2003 A1
20030138105 Challener et al. Jul 2003 A1
20030154404 Beadles et al. Aug 2003 A1
20030165140 Tang et al. Sep 2003 A1
20030204734 Wheeler Oct 2003 A1
20030206548 Bannai et al. Nov 2003 A1
20030214908 Kumar et al. Nov 2003 A1
20030217263 Sakai Nov 2003 A1
20030225563 Gonos Dec 2003 A1
20040002878 Maria Hinton Jan 2004 A1
20040054791 Chakraborty et al. Mar 2004 A1
20040068631 Ukeda et al. Apr 2004 A1
20040073443 Gabrick et al. Apr 2004 A1
20040073795 Jablon Apr 2004 A1
20040078787 Borek et al. Apr 2004 A1
20040111315 Sharma et al. Jun 2004 A1
20040117438 Considine et al. Jun 2004 A1
20040117476 Steele et al. Jun 2004 A1
20040160386 Michelitsch et al. Aug 2004 A1
20040161111 Sherman Aug 2004 A1
20040193388 Outhred et al. Sep 2004 A1
20040199572 Hunt et al. Oct 2004 A1
20040205179 Hunt et al. Oct 2004 A1
20040208292 Winterbottom Oct 2004 A1
20040226010 Suorsa Nov 2004 A1
20040261079 Sen Dec 2004 A1
20040264481 Darling et al. Dec 2004 A1
20040267920 Hydrie et al. Dec 2004 A1
20040268357 Joy et al. Dec 2004 A1
20040268358 Darling et al. Dec 2004 A1
20050008001 Williams et al. Jan 2005 A1
20050021742 Yemini et al. Jan 2005 A1
20050055435 Gbadegesin et al. Mar 2005 A1
20050080811 Speeter et al. Apr 2005 A1
20050086502 Rayes et al. Apr 2005 A1
20050091078 Hunt et al. Apr 2005 A1
20050091227 McCollum et al. Apr 2005 A1
20050097097 Hunt et al. May 2005 A1
20050097146 Konstantinou et al. May 2005 A1
20050102388 Tabbara et al. May 2005 A1
20050125212 Hunt et al. Jun 2005 A1
20050138416 Qian et al. Jun 2005 A1
20050152270 Gomez Paredes et al. Jul 2005 A1
20050192971 Tabbara et al. Sep 2005 A1
20050193103 Drabik Sep 2005 A1
20050246529 Hunt et al. Nov 2005 A1
20050246771 Hunt et al. Nov 2005 A1
20050251783 Torone et al. Nov 2005 A1
20050257244 Joly et al. Nov 2005 A1
20060025984 Papaefstathiou et al. Feb 2006 A1
20060025985 Vinberg et al. Feb 2006 A1
20060031248 Vinberg et al. Feb 2006 A1
20060034263 Outhred et al. Feb 2006 A1
20060037002 Vinberg et al. Feb 2006 A1
20060048017 Anerousis et al. Mar 2006 A1
20060123040 McCarthy et al. Jun 2006 A1
20060149838 Hunt et al. Jul 2006 A1
20060155708 Brown et al. Jul 2006 A1
20060161879 Lubrecht et al. Jul 2006 A1
20060161884 Lubrecht et al. Jul 2006 A1
20060235664 Vinberg et al. Oct 2006 A1
20060259609 Hunt et al. Nov 2006 A1
20060259610 Hunt et al. Nov 2006 A1
20060271341 Brown et al. Nov 2006 A1
20070006177 Aiber et al. Jan 2007 A1
20070112847 Dublish et al. May 2007 A1
20070192769 Mimura et al. Aug 2007 A1
20080059214 Vinberg et al. Mar 2008 A1
Foreign Referenced Citations (23)
Number Date Country
1368694 Sep 2002 CN
1375685 Oct 2002 CN
0962861 Dec 1999 EP
1063815 Dec 2000 EP
1180886 Feb 2002 EP
1307018 May 2003 EP
8297567 Nov 1996 JP
11007407 Jan 1999 JP
2001526814 Dec 2001 JP
2002084302 Mar 2002 JP
2003532784 Nov 2003 JP
2005155729 Jun 2005 JP
10-2002-0026751 Apr 2002 KR
10-2004-0008275 Jan 2004 KR
WO9853410 Nov 1998 WO
WO9963439 Dec 1999 WO
WO0022526 Apr 2000 WO
WO0031945 Jun 2000 WO
WO0073929 Dec 2000 WO
WO0237748 May 2002 WO
WO02085051 Oct 2002 WO
WO03017615 Feb 2003 WO
WO03039104 May 2003 WO
Related Publications (1)
Number Date Country
20060149838 A1 Jul 2006 US
Continuations (1)
Number Date Country
Parent 09695813 Oct 2000 US
Child 11276608 US