Patent Application
20230326088
Data compression plays an increasingly important role in daily life. Recently, there has been a surge in digital data such as images and videos, with users creating and sharing content at a rapid pace. Video conferencing has also seen a significant rise as companies and individuals come to see it as a normal means of communication. Due to poor network and bandwidth issues in various parts of the world, large file sizes become a severe bottleneck leading to a poor consumer experience. This creates a need for optimal storage and transport of visual data. Thus, end-to-end image compression techniques that maintain strong image quality have become extremely important. Additionally, as online collaboration has become more commonplace, it is necessary that data is stored optimally in a cloud computing environment to provide a good user experience.
Various compression techniques are used to achieve optimal data storage. One feature of any compression technique is the flexibility to achieve a user-preferred bitrate as users often have strict storage budget requirements. Traditional codecs (JPEG, BPG) allow users to control the bitrate through quality factors. However, they are hardcoded methods with no information of the image context. Deep learning-based approaches forgo hardcoded values by incorporating context and image semantics, but require costly hyperparameter search and retraining to work for multiple desired bitrates. Very recently, variable-rate compression methods have been introduced but provide no theoretical or empirical guarantees of the bitrate in production time. Users are also burdened with the task of having to second-guess parameters that map to an exact bitrate, making the interfaces tedious to use.
Additionally, some users, such as content creators, have specific needs to preserve the details of important parts of their data. For example, in an image capturing the historical monument ‘Taj Mahal’, preserving the details of the monument is far more important than the background.
Introduced here are techniques/technologies that enable user-guided variable-rate image compression. The user provides an image to be compressed and a segmentation map that segments the image into regions. For example, for an image of a person's face, the image may be segmented into face, hair, and background segments. For each segment, the user specifies an importance value. The importance value is used by the image compression system to allocate bits to the most important parts of the image, improving the appearance of the important portions of the image upon reconstruction. The user also specifies a target bitrate for the image compression.
The image compression system includes a compression network, which includes an encoder and an importance map network, and a reconstruction network, which includes a decoder. The image, the segmentation map with importance values, and the target bitrate are provided to the compression network which then generates a compressed representation of the image at or near the target bitrate. The compressed representation is generated based on a latent space representation of the image and a learned importance map generated by the importance map network. Subsequently, a reconstruction network is used to reconstruct an image from the compressed representation. The compression and reconstruction networks are trained using a discriminator, to train the reconstruction network to generate photorealistic reconstructed images, and various loss functions to ensure bits are allocated optimally in the importance map while staying within the limits of user-provided bit budget.
Additional features and advantages of exemplary embodiments of the present disclosure will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of such exemplary embodiments.
The detailed description is described with reference to the accompanying drawings in which:
Embodiments provide a user-guided approach to learned image compression method that enables the user to have a direct control over the bitrate and the quality of the reconstructed image. Specifically, a Generative Adversarial Network (GAN)-based image compression allows users to explicitly input a desired bitrate. This also allows for the desired bitrate to be achieved with a single model by leveraging the user input region-wise importance values as a guidance for a learned importance map, which in turn determines the attained output bitrate. Embodiments work on multiple datasets over a wide range of bitrates and achieves similar or better performance as compared to previous learned image compression methods that are trained only for particular bitrates.
In some embodiments, the user provides an image, a corresponding importance map, and a target bitrate to a compression network. The importance map may include a segmentation mask that includes multiple regions, with each region assigned an importance value by the user. Based on the target bitrate and the importance map, the compression network generates a compressed representation of the image. As discussed further below, the compression network includes an encoder, which generates a latent space representation of the input image, and an importance map network, which learns an importance map. These are combined to create the compressed representation. The compressed representation can then be stored, locally or remotely, shared with other users, etc., just like any other compressed file. When the image is later requested, the compressed representation is provided to a reconstruction network which includes a decoder trained to reconstruct the image.
The overall compression-decompression pipeline provides user-guided machine learning-based variable bitrate compression. The importance network learns optimal bit allocations via user preferences while staying within the limits of available bit budget. This gives the user control over the bitrate and the allocation of bits to more important regions of the image. Further, a single network can provide variable bitrate compression. This saves significant time and resources that would ordinarily be required to be used to train completely separate models for each bitrate that is to be supported.
The compression network and reconstruction network are trained using a generative adversarial network approach. In particular, a discriminator is used to train the compression network and reconstruction network along with an equivalence-distortion loss function that takes in the user importance map and the desired bitrate as input to constrain the learning of output importance weights, which in turn help in achieving the desired bitrate.
Traditional image compression standards, like JPEG, JPEG2000, and HEVC rely on hand-crafted modules involving discrete cosine transforms or wavelet transforms, quantization, and entropy coding. Traditional codecs like JPEG2000 compress on a per-instance level with no learning involved, thereby having poor performance at low bitrates with artifacts such as blurring, ringing, and smudging. To overcome the limitations of hand-crafted modules, several compression approaches based on RNNs, and auto-encoders have been proposed. However, these algorithms construct visually pleasing images at considerably high bitrates but do not perform as well at extremely low bitrates. Additionally, these techniques typically work on only a single bitrate target, and new networks have to be trained for each bitrate that is to be supported. To the extent that prior techniques provide variable bitrate support, they require extensive knowledge on the part of the user to be able to tune parameters to attain the target bitrate.
Accordingly, embodiments provide a single model for a wide range of bitrates that ensures photo-realism and maintains visual quality as per the bit budget allocated while facilitating the user an explicit control on the desired target bitrate. As such, no network-specific knowledge is required by the user to obtain the target bitrate.
When a request to compress an image is received, the user can provide input data 102 to image compression system 100, as shown at numeral 1. In particular, the user can provide an input image 104, importance data 106, and a bitrate target 108. In some embodiments, the input image 104 can be provided by selecting a locally stored image or providing an identifier corresponding to a storage location of the image (e.g., a URL, URI, etc.). The importance data 106 may include an importance map which is a mask of the image that has been segmented into a plurality of regions, where each region has been annotated with an importance value that represents the relative importance of each region to the user. During compression, the importance values are used to allocate more bits to more important regions and fewer bits to less important regions. This way, the appearance of more important regions, following compression, is better preserved as compared to less important regions. In some embodiments, the segmentation mask is provided by the user. Alternatively, the input image 104 may first be provided to a semantic segmentation model which generates the segmentation mask. Once generated, the user annotates the segmented regions with importance values.
Once the input data 102 is received by image compression system 100, it is provided to compression network 110. Compression network 110 is a machine learning model, or models, such as a neural network, that has been trained to compress input images at variable bitrates based on the importance data. A neural network may include a machine-learning model that can be tuned (e.g., trained) based on training input to approximate unknown functions. In particular, a neural network can include a model of interconnected digital neurons that communicate and learn to approximate complex functions and generate outputs based on a plurality of inputs provided to the model. For instance, the neural network includes one or more machine learning algorithms. In other words, a neural network is an algorithm that implements deep learning techniques, i.e., machine learning that utilizes a set of algorithms to attempt to model high-level abstractions in data.
At numeral 2, the compression network 110 generates a compressed representation of input image 104 based on the importance data 106, where the compressed representation achieves the bitrate target 108. As discussed further below, the compression network 110 may include multiple networks, including an encoder and an importance map network. The encoder generates a latent space representation of the input image and the importance map network learns an importance map for the image based on the user's importance data 106 and the input image 104. The latent space representation and the learned importance map are then combined to generate a compressed representation 116 of the input image.
At numeral 3, the compressed representation can be stored or shared by storage manager 112. For example, the compressed representation 116 can be stored to a remote data store 120, as shown at numeral 4. This may include a storage location such as a cloud storage location provided by a storage service, an FTP or other storage site accessible to the user, etc. In some embodiments, the remote data store may be associated with a different user, such as a different user's cloud storage account, computing device, etc. and the compressed representation may be shared with the different user. Optionally, the compressed representation 116 may additionally, or alternatively, be stored locally in local data store 114 (e.g., on the same device as image compression system 100 or a local network connected device accessible to image compression system 100), as shown at numeral 5.
In some embodiments, encoder 200 generates a latent representation of shape (C, H/n, W/n) from a concatenation of the user-provided importance map and the input image. As discussed further below, the encoder may include a series of convolution layers and residual blocks. The importance map network 202 takes an intermediate input representation from the encoder and generates a single channel learned importance map that is close to the user input importance map. Each pixel of this learned map includes values between 0 and 1, dictating the number of latent representation channels to use for storing the information of that pixel.
The output of the encoder 200 is a latent representation of the importance map and input image concatenation. This latent representation is then provided to a quantizer 204. In particular, a binary quantizer may be used which binarizes the output of the encoder by converting each value to either −1 or 1. This quantized output is then combined with the learned importance map to generate the compressed representation 206. In some embodiment, the quantized output and the learned importance map are combined by taking a Hadamard Product (e.g., an element-wise product). Since there is a loss of information at this step, the compression of the input image is lossy.
In some embodiments, an image is requested by providing an image identifier 300. For example, a URL, URI, or other identifier. associated with the image, a storage location, etc. The storage manager 112 can locate a compressed representation associated with the requested image using the requested image identifier 300. Where the requested image identifier is a storage location, the storage manager retrieves the compressed representation 302 from the storage location associated with the identifier. Alternatively, the storage manager 112 may maintain a mapping of image identifiers to compressed representations, where the image identifier serves as a key and the storage location as a corresponding value in the mapping. The storage manager identifies the storage location of the compressed representation 302 using the identifier and retrieves it. The example of
The compressed representation is then provided to reconstruction network 304 to generate reconstructed image 306. In some embodiments, the reconstruction network is a decoder neural network. In some embodiments, the decoder minors the encoder with a series of deconvolution layers and residual blocks. It reconstructs the image from the compressed latent representation of input image. The output of the reconstruction network 304 is the reconstructed image 306 corresponding to the compressed representation 302. The reconstructed image 306 is then returned to the requestor. For example, the reconstructed image may then be rendered on a display for viewing by a user.
The encoder receives an input image having three channels (e.g., RGB) and a height and width of H and W, respectively, and a user input importance map having one channel (e.g., importance values) and the same height and width as the input image. The encoder generates a latent representation of shape (C, H/n, W/n) from the concatenation of user input importance map (mu=Σpi*si where si are segmentation masks of various regions) and input image of shape (3+1, H, W). As shown, the encoder 200 is a neural network including a plurality of layers including convolutional layers and residual block layers. For example, in the example of
The specific architecture depicted in
The importance map network 202 receives an intermediate input representation from the encoder (e.g., following the first pair of residual blocks) and generates a single channel learned importance map that is close to the user input importance map. Each pixel of this learned map contains values between 0 and 1, which determines the number of latent representation channels to use for storing the information of that pixel. For example, in the example of
In some embodiments, each residual layer includes:
In the example of
As noted, the binary quantizer (BQ) binarizes the output of the encoder by converting each value to either −1 or 1. The output of the importance map is provided to a mask generator (MG) which creates an 8-dimensional mask representation to match the dimensions of the output of the encoder 200. The importance mask representation and the quantized output are then combined (e.g., via elementwise multiplication) to generate the compressed representation 206.
As discussed, the decoder mirrors the encoder with a series of deconvolution layers and residual blocks. It reconstructs the image from the compressed latent representation of the input image. In some embodiments, the decoder includes the following layers:
During training, training inputs 502 are provided to the compression network 110. The training inputs may include data much like that which would be introduced at inference time, specifically input images and corresponding importance maps, as well as target bitrates. For each training input (e.g., image, importance map, target bitrate triplet), the compression network 110 generates a compressed representation of the input training image using the techniques described above. The compressed representation is then provided to reconstruction network 304 which generates a reconstructed image.
The reconstructed image and the training input are then provided to a discriminator 504 which attempts to distinguish between the original image and the reconstructed image. In some embodiments, the discriminator receives the reconstructed image concatenated with the learned importance map and the input training image concatenated with the training importance map. The discriminator then predicts which of the images is real or fake. This prediction is used by one or more loss functions 506 to generate an error that is propagated back to the networks for training (as indicated by dotted lines in
In some embodiments, various loss functions are used during training. In the following section, the input image is denoted by ‘x’ and the reconstructed image is denoted by ‘y’. The loss function 506 may include an Equivalence Distortion (ED) Loss. This loss function ensures bits are allocated optimally in the importance map while staying within the limits of user-provided bit budget (e.g., to achieve the target bitrate). Let Lmse and Lssim denote the mean-squared error and MS-SSIM between input and reconstructed images. Then, the distortion loss LD is given by,
L
mse
=m
u
′⊙∥x−y∥
2
L
ssim=1−MS−SSIM(x,y)
LD=λ1Lmse+λ2Lssim
where λ1=1.25 and λ2=0.1. Let LE denote the equivalence loss obtained when importance values of different regions between the user input importance map and learned importance map are compared. Then,
where λ3=0.9, n is the number of regions in an image, α is a smoothing constant with value 10−8, mu denotes the nearest-neighbor down-sampled version of user input importance mapm
Lwhole penalizes the model when the sum of values in learned map exceed the user input map, thereby staying within the limits of available bit budget. L1region and L2region enforce both the maps to contain similar importance values. Since the maps are compared region-wise and not as one single entity, the model adaptively distributes importance values within a region in the learned map by giving higher weight to areas with high-texture or edges and lower weight to flat areas. L2region is given a smaller coefficient of λ3=0.9 since mui−mji>0 remains within the bit budget consideration. The total loss, LED then becomes LED=LE+LD.
In some embodiments, a VGG Feature Loss is also utilized. For example, the reconstructed image and the input image are both fed into a pre-trained VGG16 network and the features computed by the network at various intermediate layers for the two inputs are compared using an L1 Loss. In one implementation, features can be extracted from four positions in the VGG network and the corresponding L1 losses are aggregated to get the VGG loss value, as follows:
VGG−Loss(x,y)=L1(VGG(x),VGG(y))
In some embodiments, a GAN Feature Loss is also utilized. The features learned by the discriminator network for both of its inputs are compared with each other using L1 Loss. This enforces the reconstructed image to be visually close to the input image so that the discriminator gets fooled and makes mistakes in its task of classifying the original image. Let D denote the discriminator and x, y denote the input and reconstructed image respectively. Then,
GAN−Feature−Loss(x,y)=L1(D(x),D(y))
Additionally, training is performed at a variety of target bitrates. This ensures that the resulting model can perform variable bitrate compression. As discussed, this is an improvement over past attempts at machine learning-based compression which require a distinct model to be trained for each bitrate that is to be supported at inference time.
Given an image of size (H, W), then the latent image size is (H/n, W/n) with C channels to hold the bits. For simplicity, assume that the image includes three areas: A1, A2, A3 in the original image, and A1r, A2r, A3r in the latent image. Let the proportion of their areas in the image be k1, k2, k3. In order to train for multiple target bitrates (t) and importance value combinations [p1, p2, p3], the target bitrate and importance value combinations are sampled from a Uniform distribution and Dirichlet distribution respectively. In this example, a maximum bitrate is 0.5, however in various embodiments the maximum bitrate that may be supported can be larger or smaller.
t˜U(0,0.5)
p˜Dir([1.0,1.0,1.0])
Once the networks have converged (e.g., the loss is below a threshold value), then the networks are trained and may be deployed for use in compressing and reconstructing arbitrary input images.
When the user requests that the image be compressed (e.g., by interacting with the user interface 700), then compression is performed on the image, as discussed above. In some embodiments, the user can be presented with the original image, a copy of the reconstructed image 710, and a visual representation of the learned importance map 712. The user can then iterate over the compression settings as needed. For example, if the reconstructed image is not of high enough quality the user may increase the bitrate, change the relative importance values, etc.
As can be seen in
As illustrated in
Additionally, the user interface manager 1102 allows users to request the image compression system 1100 to reconstruct an image from a compressed representation. For example, the user can provide or identify a compressed representation of an image and the user interface manager can send a request to reconstruct an image from the compressed representation to the reconstruction network. In some embodiments, the user interface manager 1102 enables the user to view the resulting reconstructed image.
As illustrated in
As illustrated in
As illustrated in
As further illustrated in
Each of the components 1102-1108 of the image compression system 1100 and their corresponding elements (as shown in
The components 1102-1108 and their corresponding elements can comprise software, hardware, or both. For example, the components 1102-1108 and their corresponding elements can comprise one or more instructions stored on a computer-readable storage medium and executable by processors of one or more computing devices. When executed by the one or more processors, the computer-executable instructions of the image compression system 1100 can cause a client device and/or a server device to perform the methods described herein. Alternatively, the components 1102-1108 and their corresponding elements can comprise hardware, such as a special purpose processing device to perform a certain function or group of functions. Additionally, the components 1102-1108 and their corresponding elements can comprise a combination of computer-executable instructions and hardware.
Furthermore, the components 1102-1108 of the image compression system 1100 may, for example, be implemented as one or more stand-alone applications, as one or more modules of an application, as one or more plug-ins, as one or more library functions or functions that may be called by other applications, and/or as a cloud-computing model. Thus, the components 1102-1108 of the image compression system 1100 may be implemented as a stand-alone application, such as a desktop or mobile application. Furthermore, the components 1102-1108 of the image compression system 1100 may be implemented as one or more web-based applications hosted on a remote server. Alternatively, or additionally, the components of the image compression system 1100 may be implemented in a suit of mobile device applications or “apps.”
As illustrated in
As illustrated in
As illustrated in
As illustrated in
In some embodiments, receiving a request to compress an image, the request including the image, a corresponding importance data, and a target bitrate, further comprises receiving, via a user interface, an importance value for each region of a segmentation mask corresponding to the image.
In some embodiments, the method further comprises receiving a request for the image, the request including at least a reference to the compressed representation of the image, providing the compressed representation of the image to a reconstruction network, and returning a reconstructed image generated by the reconstruction network. As discussed, the compressed representation can be stored remotely and retrieved to be reconstructed. Given the low bitrates that the compression pipeline enables, the storage space and bandwidth required to maintain the compressed representation and transmit them is greatly reduced.
As discussed, GAN-style training may be performed on the compression network and reconstruction network. For example, a discriminator network and one or more loss functions are used to train the compression network to generate compressed representations at variable bitrates based on user-provided importance values and a reconstruction network to generate photorealistic images from the compressed representations. In some embodiments, the one or more loss function includes an equivalence distortion loss function that trains the importance map network to learn to allocate bits optimally in the learned importance map while staying within the target bitrate.
Although
Similarly, although the environment 1300 of
As illustrated in
Moreover, as illustrated in
In addition, the environment 1300 may also include one or more servers 1304. The one or more servers 1304 may generate, store, receive, and transmit any type of data, including input images 1118, compressed representations 1120, reconstructed images 1122, training data 1124, or other information. For example, a server 1304 may receive data from a client device, such as the client device 1306A, and send the data to another client device, such as the client device 1302B and/or 1302N. The server 1304 can also transmit electronic messages between one or more users of the environment 1300. In one example embodiment, the server 1304 is a data server. The server 1304 can also comprise a communication server or a web-hosting server. Additional details regarding the server 1304 will be discussed below with respect to
As mentioned, in one or more embodiments, the one or more servers 1304 can include or implement at least a portion of the image compression system 1100. In particular, the image compression system 1100 can comprise an application running on the one or more servers 1304 or a portion of the image compression system 1100 can be downloaded from the one or more servers 1304. For example, the image compression system 1100 can include a web hosting application that allows the client devices 1306A-1306N to interact with content hosted at the one or more servers 1304. To illustrate, in one or more embodiments of the environment 1300, one or more client devices 1306A-1306N can access a webpage supported by the one or more servers 1304. In particular, the client device 1306A can run a web application (e.g., a web browser) to allow a user to access, view, and/or interact with a webpage or website hosted at the one or more servers 1304.
Upon the client device 1306A accessing a webpage or other web application hosted at the one or more servers 1304, in one or more embodiments, the one or more servers 1304 can provide access to one or more digital images (e.g., the input image data 1118, such as camera roll or an individual's personal photos and corresponding compressed representations 1120) stored at the one or more servers 1304. Moreover, the client device 1306A can receive a request (i.e., via user input) to retrieve an image and provide the request to the one or more servers 1304. Upon receiving the request, the one or more servers 1304 can return a corresponding compressed representation of the requested image. The client device can then automatically perform the methods and processes described above to reconstruct the requested image using the compressed representation.
As just described, the image compression system 1100 may be implemented in whole, or in part, by the individual elements 1302-1308 of the environment 1300. It will be appreciated that although certain components of the image compression system 1100 are described in the previous examples with regard to particular elements of the environment 1300, various alternative implementations are possible. For instance, in one or more embodiments, the image compression system 1100 is implemented on any of the client devices 1306A-N. Similarly, in one or more embodiments, the image compression system 1100 may be implemented on the one or more servers 1304. Moreover, different components and functions of the image compression system 1100 may be implemented separately among client devices 1306A-1306N, the one or more servers 1304, and the network 1308.
Embodiments of the present disclosure may comprise or utilize a special purpose or general-purpose computer including computer hardware, such as, for example, one or more processors and system memory, as discussed in greater detail below. Embodiments within the scope of the present disclosure also include physical and other computer-readable media for carrying or storing computer-executable instructions and/or data structures. In particular, one or more of the processes described herein may be implemented at least in part as instructions embodied in a non-transitory computer-readable medium and executable by one or more computing devices (e.g., any of the media content access devices described herein). In general, a processor (e.g., a microprocessor) receives instructions, from a non-transitory computer-readable medium, (e.g., a memory, etc.), and executes those instructions, thereby performing one or more processes, including one or more of the processes described herein.
Computer-readable media can be any available media that can be accessed by a general purpose or special purpose computer system. Computer-readable media that store computer-executable instructions are non-transitory computer-readable storage media (devices). Computer-readable media that carry computer-executable instructions are transmission media. Thus, by way of example, and not limitation, embodiments of the disclosure can comprise at least two distinctly different kinds of computer-readable media: non-transitory computer-readable storage media (devices) and transmission media.
Non-transitory computer-readable storage media (devices) includes RAM, ROM, EEPROM, CD-ROM, solid state drives (“SSDs”) (e.g., based on RAM), Flash memory, phase-change memory (“PCM”), other types of memory, other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store desired program code means in the form of computer-executable instructions or data structures and which can be accessed by a general purpose or special purpose computer.
A “network” is defined as one or more data links that enable the transport of electronic data between computer systems and/or modules and/or other electronic devices. When information is transferred or provided over a network or another communications connection (either hardwired, wireless, or a combination of hardwired or wireless) to a computer, the computer properly views the connection as a transmission medium. Transmissions media can include a network and/or data links which can be used to carry desired program code means in the form of computer-executable instructions or data structures and which can be accessed by a general purpose or special purpose computer. Combinations of the above should also be included within the scope of computer-readable media.
Further, upon reaching various computer system components, program code means in the form of computer-executable instructions or data structures can be transferred automatically from transmission media to non-transitory computer-readable storage media (devices) (or vice versa). For example, computer-executable instructions or data structures received over a network or data link can be buffered in RAM within a network interface module (e.g., a “NIC”), and then eventually transferred to computer system RAM and/or to less volatile computer storage media (devices) at a computer system. Thus, it should be understood that non-transitory computer-readable storage media (devices) can be included in computer system components that also (or even primarily) utilize transmission media.
Computer-executable instructions comprise, for example, instructions and data which, when executed at a processor, cause a general-purpose computer, special purpose computer, or special purpose processing device to perform a certain function or group of functions. In some embodiments, computer-executable instructions are executed on a general-purpose computer to turn the general-purpose computer into a special purpose computer implementing elements of the disclosure. The computer executable instructions may be, for example, binaries, intermediate format instructions such as assembly language, or even source code. Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the described features or acts described above. Rather, the described features and acts are disclosed as example forms of implementing the claims.
Those skilled in the art will appreciate that the disclosure may be practiced in network computing environments with many types of computer system configurations, including, personal computers, desktop computers, laptop computers, message processors, hand-held devices, multi-processor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, mobile telephones, PDAs, tablets, pagers, routers, switches, and the like. The disclosure may also be practiced in distributed system environments where local and remote computer systems, which are linked (either by hardwired data links, wireless data links, or by a combination of hardwired and wireless data links) through a network, both perform tasks. In a distributed system environment, program modules may be located in both local and remote memory storage devices.
Embodiments of the present disclosure can also be implemented in cloud computing environments. In this description, “cloud computing” is defined as a model for enabling on-demand network access to a shared pool of configurable computing resources. For example, cloud computing can be employed in the marketplace to offer ubiquitous and convenient on-demand access to the shared pool of configurable computing resources. The shared pool of configurable computing resources can be rapidly provisioned via virtualization and released with low management effort or service provider interaction, and then scaled accordingly.
A cloud-computing model can be composed of various characteristics such as, for example, on-demand self-service, broad network access, resource pooling, rapid elasticity, measured service, and so forth. A cloud-computing model can also expose various service models, such as, for example, Software as a Service (“SaaS”), Platform as a Service (“PaaS”), and Infrastructure as a Service (“IaaS”). A cloud-computing model can also be deployed using different deployment models such as private cloud, community cloud, public cloud, hybrid cloud, and so forth. In this description and in the claims, a “cloud-computing environment” is an environment in which cloud computing is employed.
In particular embodiments, processor(s) 1402 includes hardware for executing instructions, such as those making up a computer program. As an example, and not by way of limitation, to execute instructions, processor(s) 1402 may retrieve (or fetch) the instructions from an internal register, an internal cache, memory 1404, or a storage device 1408 and decode and execute them. In various embodiments, the processor(s) 1402 may include one or more central processing units (CPUs), graphics processing units (GPUs), field programmable gate arrays (FPGAs), systems on chip (SoC), or other processor(s) or combinations of processors.
The computing device 1400 includes memory 1404, which is coupled to the processor(s) 1402. The memory 1404 may be used for storing data, metadata, and programs for execution by the processor(s). The memory 1404 may include one or more of volatile and non-volatile memories, such as Random Access Memory (“RAM”), Read Only Memory (“ROM”), a solid state disk (“SSD”), Flash, Phase Change Memory (“PCM”), or other types of data storage. The memory 1404 may be internal or distributed memory.
The computing device 1400 can further include one or more communication interfaces 1406. A communication interface 1406 can include hardware, software, or both. The communication interface 1406 can provide one or more interfaces for communication (such as, for example, packet-based communication) between the computing device and one or more other computing devices 1400 or one or more networks. As an example and not by way of limitation, communication interface 1406 may include a network interface controller (NIC) or network adapter for communicating with an Ethernet or other wire-based network or a wireless NIC (WNIC) or wireless adapter for communicating with a wireless network, such as a WI-FI. The computing device 1400 can further include a bus 1412. The bus 1412 can comprise hardware, software, or both that couples components of computing device 1400 to each other.
The computing device 1400 includes a storage device 1408 includes storage for storing data or instructions. As an example, and not by way of limitation, storage device 1408 can comprise a non-transitory storage medium described above. The storage device 1408 may include a hard disk drive (HDD), flash memory, a Universal Serial Bus (USB) drive or a combination these or other storage devices. The computing device 1400 also includes one or more input or output (“I/O”) devices/interfaces 1410, which are provided to allow a user to provide input to (such as user strokes), receive output from, and otherwise transfer data to and from the computing device 1400. These I/O devices/interfaces 1410 may include a mouse, keypad or a keyboard, a touch screen, camera, optical scanner, network interface, modem, other known I/O devices or a combination of such I/O devices/interfaces 1410. The touch screen may be activated with a stylus or a finger.
The I/O devices/interfaces 1410 may include one or more devices for presenting output to a user, including, but not limited to, a graphics engine, a display (e.g., a display screen), one or more output drivers (e.g., display drivers), one or more audio speakers, and one or more audio drivers. In certain embodiments, I/O devices/interfaces 1410 is configured to provide graphical data to a display for presentation to a user. The graphical data may be representative of one or more graphical user interfaces and/or any other graphical content as may serve a particular implementation.
In the foregoing specification, embodiments have been described with reference to specific exemplary embodiments thereof. Various embodiments are described with reference to details discussed herein, and the accompanying drawings illustrate the various embodiments. The description above and drawings are illustrative of one or more embodiments and are not to be construed as limiting. Numerous specific details are described to provide a thorough understanding of various embodiments.
Embodiments may include other specific forms without departing from its spirit or essential characteristics. The described embodiments are to be considered in all respects only as illustrative and not restrictive. For example, the methods described herein may be performed with less or more steps/acts or the steps/acts may be performed in differing orders. Additionally, the steps/acts described herein may be repeated or performed in parallel with one another or in parallel with different instances of the same or similar steps/acts. The scope of the invention is, therefore, indicated by the appended claims rather than by the foregoing description. All changes that come within the meaning and range of equivalency of the claims are to be embraced within their scope.
In the various embodiments described above, unless specifically noted otherwise, disjunctive language such as the phrase “at least one of A, B, or C,” is intended to be understood to mean either A, B, or C, or any combination thereof (e.g., A, B, and/or C). As such, disjunctive language is not intended to, nor should it be understood to, imply that a given embodiment requires at least one of A, at least one of B, or at least one of C to each be present.
