Patent Application
20240370243
The disclosed embodiments generally relate to desired state configuration. In particular, the disclosed embodiments describe schemas for defining and using packages in a language-neutral way to perform desired state configuration management.
A desired state configuration system facilitates causing a target environment, such as a cloud-based environment, to have a particular desired state. For example, a particular cloud-based target environment could be made to have a specified number of virtual servers, with specified communication paths between them, and specified data in storage buckets. However, it can be extremely cumbersome to specify all the instructions and other data required to specify a desired system state, particularly across the various different programming languages that may be used within a state configuration project.
The figures depict various embodiments of the present technology for purposes of illustration only. One skilled in the art will readily recognize from the following description that other alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles of the technology described herein.
A desired state configuration system allows users to specify the desired state of their systems and manages the systems to make them conform to that desired state. This simplifies the system administration required to achieve the desired state, which would otherwise be a very manual and error-prone process.
In some embodiments, the wrapper generation module 140 generates wrappers 145 using the following transformations: 1. Type tokens are mapped into nested namespaced resources and types (nested by module name in the type token). 2. Functions are mapped into module-level functions 3. Resources are mapped into types with constructors (e.g., classes in languages which have such a combined notion). 4. Each resource and function has a bag of strongly typed inputs and outputs. 5. These inputs are derived from the schema but deeply wrapped in type “Input” (which can be either a raw value (e.g., string, integer, Boolean, list map, etc.), an asynchronously computed value (e.g., a Promise or Task), or an output read from another resource's values) to allow accepting either known value or values dependent on other resources. 6. Outputs for resources are represented as properties on a class/type. 7. Outputs are represented as the schema type wrapped in type “Output,” making clear that their values are not necessarily known until they are created/updated.
In some embodiments, the wrapper generation module 140 generates a general (static) “get” function for a package that allows obtaining an instance of a resource of the type represented by that package, where the instance is not yet managed by the desired state configuration system 340. This effectively permits the code of one package to import and use an instance not created through the desired state configuration system 340. As one example, the Python code for a first package could contain the line group=aws.ec2.SecurityGroup.get(‘group’, ‘sg-0dfd33cdac25bec9’), which obtains a reference to an existing instance of an Amazon™ Elastic Compute Cloud™ security group with AWS™ ID “sg-0dfd33cdac25b1ec9′” using the static “get( )” function. In one embodiment, the get function takes an argument specifying the ID within the source cloud system of the instance to be obtained, and another argument specifying a name for that object (e.g., “group”) when accessed by packages managed by the desired state configuration system 340.
The schemas 120 may also be input to a documentation generation module 130, which produces documentation for human perusal for each of various programming languages supported by the desired state configuration system 340. The documentation generation module 130 presents the interface in terms of the desired state configuration model using resources and functions, including examples and descriptions of those resources/functions and of properties thereof. In some embodiments, documentation for a package is provided on a web page or other document with separate sections for different examples and different syntax, with choices of language-specific instances for each. For example, the multi-language documentation for a package named “aws.ec2.Instance” might indicate that for the Go programming language the syntax for creating a new instance is “func Newlnstance (ctx *Context, name string, args *InstanceArgs, opts . . . ResourceOption) (*Instance, error)”; clicking on a neighboring tab for “C#” would in turn indicate that the syntax for creating a new instance in C# is “public Instance (string name, InstanceArgs ?args=null, CustomResourceOptions? opts=null).” Thus, documentation for all the supported languages is provided within a single web page or other document, with the various concepts each providing syntax or other examples for each of the supported languages.
Software engineers of the organization 310 also author application code 150 that uses the packages 110 to create and/or modify the state of the user environment 275. Specifically, the application code 150 uses the wrappers 145 generated by the wrapper generation module 140—and more specifically, the wrapper for the programming language in which the application code 150 is written—to call the code in the corresponding package 110.
Appendix A, below, provides a simple example of a schema (“schema.yaml”) for a particular resource component (defined by “staticPage.go”), where a wrapper (“staticPage.ts”) is generated based on the schema and called from application code (“index.ts”).
Appendix B, below, lists formats for schemas, according to some embodiments. In these embodiments, schemas may describe a number of units, such as packages 110, resources 122, functions 124, types, properties, and the like. For example, the schema of Appendix A describes a resource named “StaticPage,” using a map data structure defined in superset of JSON to specify attributes as a set of keys and values. The schema has a number of attributes that provide information about the units described in the schema, such as the “name” attribute with the atomic value “xyz” (indicating that the name of the described package is “xyz”), the “resources” attribute (which in turn has a map data structure as its value), and the like. A package 110 is said to implement the schema corresponding to it if its data matches the structure specified by the schema as a whole, and a unit of a package (e.g., a resource) is said to implement the schema if its data matches the structure specified by the corresponding portion of the schema (e.g., the portion of the schema defining a resource). Packages match their respective schemas (and units of packages match their respective schema portions) if they satisfy the various implicit or explicit assertions of the schema, such as that all input to the package or unit is of the types (or compatible subtypes thereof) specified for the input in the schema; all output form the package is of the types (or compatible subtypes) specified for the output in the schema; all required properties of a package's resources are marked in the schema as being required inputs; and all resources and function specified in the schema have an implementation in the package.
Schemas for packages 110 define information such as (but not limited to) package name and version; the license under which the package is made available for use by others; the URL at which the source code for the package is made available; object types that the package defines; resources 122 that the package defines; functions 124 that the package defines; and language-specific data about the package.
Schemas for object types define information about new types that have been created and are being made available for use by other code. Such information can include (but is not limited to): any required input properties or output properties of the object type; metadata describing the different properties (including types and informational data such as human-readable descriptions); and data about the type that is specific to a particular programming language.
Schemas for resources 122 define information about a particular resource in the target environment (e.g., a storage bucket in a cloud environment). The information can include (but is not limited to): any required input properties for creating the resource; whether the resource represents a single resource maintained by the provider of the target environment, or is a composite of many such resources; any functions 124 of the resource 122; and data about the resource that is specific to a particular programming language.
Schemas for functions 124 define information about functions that may be executed in a programming language. The information can include (but is not limited to): a list of any input properties for calling the function; a list of any output properties resulting from the function call; and data about the function that is specific to a particular programming language.
Schemas for properties define information about particular input or output variable properties, such as whether a particular property has a constant (fixed) value, or a default value; whether the property is secret; and data about the property that is specific to a particular programming language.
The “language” property describing attributes that are specific to a particular programming may specify languages such as Node.js, Python, Go, C#, or Java, and may relate to aspects such as any dependencies; compatibility; paths for package imports; namespaces in which the package or other item should be included.
In some embodiments, the output properties of units such as resources 122 and functions 124 are evaluated asynchronously, given that the cloud system 270 may perform non-trivial amounts of operations to create or modify components within the user environment 275. Further, dependencies between output data and the input data that led to it can be tracked. In some embodiments, the output properties are of a special data type, such as “Output[T]” (where “T” represents a templated data type), where that data type contains references to the input sources resulting in the output, as well as the value of the output itself. In this way, input and output operations result in a dependency graph through which the flow of the data may be traced from the original inputs, through intermediate outputs, to the ultimate outputs. In some embodiments, the handling of special output data types, and the formation of the dependency graph, is handled by the code of the wrappers 145.
The existence of the dependency graph enables a number of different useful operations. As a first example, the dependency graph supports the ability to run a Preview within the desired state configuration system 340. A proposed change to the desired state can be “previewed” by evaluating the application code 150 without actually making any changes, and instead treating all ‘Output’ values as “unknown.” These unknown values can propagate into dependencies so that the preview can indicate which resource may impact the input to another resource, without having to yet know what the actual values will be (since that won't be known until the change is actually made). This allows users of the system 340 to carefully validate the impact of changes before they are made. As a second example, the dependency graph enables the ability to deterministically delete resources managed by the desired state configuration system 3 based on the dependency graph. Resources can be deleted in reverse-dependency-order to ensure that a resource is deleted before anything that depends on it, thereby ensuring safe and correct deletion. As a third example, in order to effect replacements of resources, in some cases, a replacement of the resource will be created, then resources dependent on it will be updated to point to the new resource, then the old instance will be deleted. This uses the dependency graph directly, but must also compose with the potential replacement of any of the dependent resources recursively, leading to a non-trivial subgraph of replacements that must be processed in a nested form before the deletion of the “old” instances in reverse order.
In some embodiments, a schema 120 has properties with values that reference separate schemas as a way to more succinctly define a data type. For example, in the schema of Appendix A, the excerpted lines
The storage device 408 is any non-transitory computer-readable storage medium, such as a hard drive, compact disk read-only memory (CD-ROM), DVD, or a solid-state memory device. The memory 406 holds instructions and data used by the processor 402. The graphics adapter 412 displays images and other information on the display 418. The network adapter 416 couples the computer 400 to a local or wide area network. The keyboard 410 and pointing device 414 allow a user to manually provide input. The audio input (e.g., microphone) 424 and output (e.g., internal or external speaker) 426 provide the ability obtain sound input (e.g., for speech recognition) and produce sound output.
As is known in the art, a computer 400 can have different and/or other components than those shown in
As is known in the art, the computer 400 is adapted to execute computer program modules for providing functionality described herein. As used herein, the term “module” refers to computer program logic utilized to provide the specified functionality. Thus, a module can be implemented in hardware, firmware, and/or software. In one embodiment, program modules are stored on the storage device 408, loaded into the memory 406, and executed by the processor 402.
Embodiments of the entities described herein can include other and/or different modules than the ones described here. In addition, the functionality attributed to the modules can be performed by other or different modules in other embodiments. Moreover, this description occasionally omits the term “module” for purposes of clarity and convenience.
One possible embodiment has been described herein. Those of skill in the art will appreciate that other embodiments may likewise be practiced. First, the particular naming of the components and variables, capitalization of terms, the attributes, data structures, or any other programming or structural aspect is not mandatory or significant, and the mechanisms described may have different names, formats, or protocols. Also, the particular division of functionality between the various system components described herein is merely for purposes of example, and is not mandatory; functions performed by a single system component may instead be performed by multiple components, and functions performed by multiple components may instead be performed by a single component.
Some portions of the above description present the inventive features in terms of algorithms and symbolic representations of operations on information. These algorithmic descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. These operations, while described functionally or logically, are understood to be implemented by computer programs. Furthermore, it has also proven convenient at times, to refer to these arrangements of operations as modules or by functional names, without loss of generality.
Unless specifically stated otherwise as apparent from the above discussion, it is appreciated that throughout the description, discussions utilizing terms such as “determining” or “displaying” or the like, refer to the action and processes of a computer system, or similar electronic computing device, that manipulates and transforms data represented as physical (electronic) quantities within the computer system memories or registers or other such information storage, transmission or display devices.
Certain aspects described herein include process steps and instructions in the form of an algorithm. It should be noted that the process steps and instructions could be embodied in software, firmware or hardware, and when embodied in software, could be downloaded to reside on and be operated from different platforms used by real time network operating systems.
The concepts described herein also relate to an apparatus for performing the operations herein. This apparatus may be specially constructed for the required purposes, or it may comprise a general-purpose computer selectively activated or reconfigured by a computer program stored on a computer readable medium that can be accessed by the computer. Such a computer program may be stored in a non-transitory computer readable storage medium, such as, but is not limited to, any type of disk including floppy disks, optical disks, CD-ROMs, magnetic-optical disks, read-only memories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, magnetic or optical cards, application specific integrated circuits (ASICs), or any type of computer-readable storage medium suitable for storing electronic instructions, and each coupled to a computer system bus. Furthermore, the computers referred to in the specification may include a single processor or may be architectures employing multiple processor designs for increased computing capability.
The algorithms and operations presented herein are not inherently related to any particular computer or other apparatus. Various general-purpose systems may also be used with programs in accordance with the teachings herein, or it may prove convenient to construct more specialized apparatus to perform the required method steps. The required structure for a variety of these systems will be apparent to those of skill in the art, along with equivalent variations. In addition, the concepts described herein are not described with reference to any particular programming language. It is appreciated that a variety of programming languages may be used to implement the teachings as described herein, and any references to specific languages are provided for purposes of enablement and best mode.
The concepts described herein are well suited to a wide variety of computer network systems over numerous topologies. Within this field, the configuration and management of large networks comprise storage devices and computers that are communicatively coupled to dissimilar computers and storage devices over a network, such as the Internet.
Finally, it should be noted that the language used in the specification has been principally selected for readability and instructional purposes, and may not have been selected to delineate or circumscribe the inventive subject matter. Accordingly, the disclosure is intended to be illustrative, but not limiting, of the scope of the concepts described herein, which are set forth in the below claims.
Appendix A contains four related code listings for a simple example: displaying the “Hello, world!” message in a static page hosted on the Amazon™ S3™ cloud storage system.
The document “schema.yaml” includes markup language defining the schema for a package named “xyz” that contains a simple component resource named “StaticPage” that contains HTML content.
The document “staticPage.go” includes procedural code in the “Go” programming language that provides the implementation of the “StaticPage” type declared in “schema.yaml”. The implementation (among other things) creates a storage “bucket” in the Amazon S3 cloud storage system for a static index web page, defining the resources “StaticPageArgs” and “StaticPage” (data types) and “NewStaticPage” (a function).
The document “staticPage.ts” is a wrapper generated based on the “schema.yaml” schema and allowing the “StaticPage” component to be called from within the TypeScript procedural programming language.
Finally, the document “index.ts” contains example application code written in the TypeScript programming language that constructs an instance of the StaticPage component that displays the message “Hello, world!”, using the SDK provided by “staticPage.ts”.
schema.yaml
staticPage.go
staticPage.ts
index.ts
Metadata for the importer about this package.
A package's configuration variables.
An object type.
An object or enum type.
The Values Metadata Associated with an Enum Type.
Extra Information about the Default Value for a Property.
Informs the consumer of an alternative schema based on the value associated with it.
Several schema types contain a language field which is a map from a supported language to a language-specific schema type. The schema for this type is language dependent, and offers extensibility for language-specific SDK code generators to encode language-specific information about the schema entry.
For csharp:
