Patent Grant
10315110
Computer graphics content items such as video games are becoming increasingly popular and widespread. When content is eventually presented to end users, scenes of the content may be used to generate resulting images for display. The resulting images may be generated by way of a process commonly referred to as rendering, which may, for example, incorporate concepts such as projection, reflection, lighting, shading and others. For example, some graphics content may be three-dimensional content, which is content having associated three-dimensional scenes that may be used to render resulting two-dimensional images. Some other graphics content may be two-dimensional content, which is content having associated two-dimensional scenes that may be used to render resulting two-dimensional images. The term scene, as used herein, refers to a representation that may be used in association with generation of an image. Rendering and other procedures associated with graphics content often involve highly complex and computationally intensive operations. For these and other reasons, rendering and other related procedures are typically performed by components, such as graphics processing units, that are specialized for graphics processing.
The inclusion of highly detailed and complex graphical objects may often be used as selling points for graphics content items that may distinguish them favorably from other products. For example, content users may desire to view images with detailed displays of nature related objects such as trees, water and fire. Each of these example objects may be displayed with highly detailed textures and color arrangements, for example, to represent features such as leaves on a tree, waves in an ocean and flames in a fire. As another example, it may also be desirable to display clothing and cloth objects with highly decorative and intricate patterns and designs. The need to quickly render these and other highly detailed objects in multiple images may place a heavy operational burden on graphics processing units and associated components. In some cases, this heavy operational burden may cause images to be displayed at a lower quality and/or slower speed than desired.
The following detailed description may be better understood when read in conjunction with the appended drawings. For the purposes of illustration, there are shown in the drawings example embodiments of various aspects of the disclosure; however, the invention is not limited to the specific methods and instrumentalities disclosed.
In general, this disclosure describes techniques for operation of a graphics object service that provides graphics object data for graphics content such as video games. In accordance with the disclosed techniques, a graphics object service may be marketed or publicized to graphics content developers for generating data associated with graphics objects such as trees, fire, water, clouds, cloths, clothing humans, animals and others. In some cases, information for communicating with the graphics object service may also be provided. For example, information associated with an application programming interface (API) may be exposed that enables graphics object data to be requested and described in a format that is understandable by the graphics objects service. The API may have associated parameters for describing features of objects such as type, allowable delay, time, depth, shading, latency, size, color, texture and others. Content developers may use the API information to, for example, insert, into a content item, various requests to obtain graphics object data from the graphics objects service.
When a content item is executed at a client device, the content item may generate various scenes or portions of scenes for display. When a scene is generated, the content item may identify any graphics object service requests associated with the scene. Each scene may have any number of associated graphics object service requests that may be sent to any number of different graphics object services. For example, a particular scene may include a tree, fire, water and a house. The content item may, for example, request rendering of the tree from a tree service, rendering of fire from a fire service, rendering of water from a water service and rendering of the house locally by the client device. The graphics object services may be accessible over a network such as the Internet and may be distributed across any number of computing devices. The graphics object services may, in some cases, employ state of the art graphics processing units and other associated graphics components. By requesting object data from graphics object services, a content item may, for example, reduce at least part of the computational burden on a graphics processing unit of the client device, while also making use of the additional, and sometimes more advanced, graphics processing components operated by the graphics object services.
When a graphics object service request is identified, the request may be sent over a network to a graphics object service associated with the request. The request may, in some cases, describe the requested object in association with a two-dimensional or three-dimensional scene. The request and the associated description may, for example, be in a format that is understandable by the associated graphics object service, such as a format corresponding to API information associated with the graphics object service. In some cases, the request may instruct the graphics object service to provide either data associated with a rendered object or data associated with graphics commands for rendering the object. Upon receiving the request, the graphics object service may generate and return the requested data to the client.
Upon receiving the requested data from the graphics object service, the client device may compile an image of the scene by combining the requested graphics object data with other graphics object data that may be generated locally at the client and/or at other graphics object services. Various techniques may be employed to combine the object data from the client device and the different graphics object services. In particular, one example combination technique may employ an image representation having multiple layers. Each layer of the representation may correspond to a respective portion of the image. For example, a first layer may include a first portion of the image rendered by a first graphics object service, while a second layer may include a second portion of the image rendered by a second graphics object service. Another example object combination technique may involve inserting rendered objects into identified areas within the image. For example, a first object rendered by a first graphics object service may be inserted at a first identified image area, while a second object rendered by a second graphics object service may be inserted at a second identified image area. To assist with blending of objects rendered based on data from different sources, various filtering techniques and visual effects, such as blurring effects, may be applied, for example, at border areas between the different objects.
As set forth above, a graphics object service may, receive requests for graphics object data and provide graphics object data to requesting devices. An example computing environment in which a graphics object service may operate will now be described in detail. In particular,
Each type or configuration of computing resource may be available in different sizes, such as large resources—consisting of many processors, large amounts of memory and/or large storage capacity—and small resources—consisting of fewer processors, smaller amounts of memory and/or smaller storage capacity. Customers may choose to allocate a number of small processing resources as web servers and/or one large processing resource as a database server, for example.
Data center 210 may include servers 216a-b (which may be referred herein singularly as server 216 or in the plural as servers 216) that provide computing resources. These resources may be available as bare metal resources, or as virtual machine instances 218a-d and (which may be referred herein singularly as virtual machine instance 218 or in the plural as virtual machine instances 218). Virtual machine instances 218c and 218d are graphics object service (“GOS”) virtual machine instances. The GOS virtual machine instances 218c and 218d may be configured to perform all or any portion of the graphics object service techniques in accordance with the present disclosure and described in detail below. As should be appreciated, while the particular example illustrated in
The availability of virtualization technologies for computing hardware has provided benefits for providing large scale computing resources for customers and allowing computing resources to be efficiently and securely shared between multiple customers. For example, virtualization technologies may allow a physical computing device to be shared among multiple users by providing each user with one or more virtual machine instances hosted by the physical computing device. A virtual machine instance may be a software emulation of a particular physical computing system that acts as a distinct logical computing system. Such a virtual machine instance provides isolation among multiple operating systems sharing a given physical computing resource. Furthermore, some virtualization technologies may provide virtual resources that span one or more physical resources, such as a single virtual machine instance with multiple virtual processors that spans multiple distinct physical computing systems.
Referring to
Communication network 230 may provide access to computers 202. User computers 202 may be computers utilized by users 200 or other customers of data center 210. For instance, user computer 202a or 202b may be a server, a desktop or laptop personal computer, a tablet computer, a wireless telephone, a personal digital assistant (PDA), an e-book reader, a game console, a set-top box or any other computing device capable of accessing data center 210. User computer 202a or 202b may connect directly to the Internet (e.g., via a cable modem or a Digital Subscriber Line (DSL)). Although only two user computers 202a and 202b are depicted, it should be appreciated that there may be multiple user computers.
User computers 202 may also be utilized to configure aspects of the computing resources provided by data center 210. In this regard, data center 210 might provide a gateway or web interface through which aspects of its operation may be configured through the use of a web browser application program executing on user computer 202. Alternately, a stand-alone application program executing on user computer 202 might access an application programming interface (API) exposed by data center 210 for performing the configuration operations. Other mechanisms for configuring the operation of various web services available at data center 210 might also be utilized.
Servers 216 shown in
It should be appreciated that although the embodiments disclosed above discuss the context of virtual machine instances, other types of implementations can be utilized with the concepts and technologies disclosed herein. For example, the embodiments disclosed herein might also be utilized with computing systems that do not utilize virtual machine instances.
In the example data center 210 shown in
In the example data center 210 shown in
It should be appreciated that the network topology illustrated in
It should also be appreciated that data center 210 described in
In at least some embodiments, a server that implements a portion or all of one or more of the technologies described herein may include a general-purpose computer system that includes or is configured to access one or more computer-accessible media.
In various embodiments, computing device 100 may be a uniprocessor system including one processor 10 or a multiprocessor system including several processors 10 (e.g., two, four, eight or another suitable number). Processors 10 may be any suitable processors capable of executing instructions. For example, in various embodiments, processors 10 may be general-purpose or embedded processors implementing any of a variety of instruction set architectures (ISAs), such as the ×86, PowerPC, SPARC or MIPS ISAs or any other suitable ISA. In multiprocessor systems, each of processors 10 may commonly, but not necessarily, implement the same ISA.
System memory 20 may be configured to store instructions and data accessible by processor(s) 10. In various embodiments, system memory 20 may be implemented using any suitable memory technology, such as static random access memory (SRAM), synchronous dynamic RAM (SDRAM), nonvolatile/Flash®-type memory or any other type of memory. In the illustrated embodiment, program instructions and data implementing one or more desired functions, such as those methods, techniques and data described above, are shown stored within system memory 20 as code 25 and data 26.
In one embodiment, I/O interface 30 may be configured to coordinate I/O traffic between processor 10, system memory 20 and any peripherals in the device, including network interface 40 or other peripheral interfaces. In some embodiments, I/O interface 30 may perform any necessary protocol, timing or other data transformations to convert data signals from one component (e.g., system memory 20) into a format suitable for use by another component (e.g., processor 10). In some embodiments, I/O interface 30 may include support for devices attached through various types of peripheral buses, such as a variant of the Peripheral Component Interconnect (PCI) bus standard or the Universal Serial Bus (USB) standard, for example. In some embodiments, the function of I/O interface 30 may be split into two or more separate components, such as a north bridge and a south bridge, for example. Also, in some embodiments some or all of the functionality of I/O interface 30, such as an interface to system memory 20, may be incorporated directly into processor 10.
Network interface 40 may be configured to allow data to be exchanged between computing device 100 and other device or devices 60 attached to a network or networks 50, such as other computer systems or devices, for example. In various embodiments, network interface 40 may support communication via any suitable wired or wireless general data networks, such as types of Ethernet networks, for example. Additionally, network interface 40 may support communication via telecommunications/telephony networks such as analog voice networks or digital fiber communications networks, via storage area networks such as Fibre Channel SANs (storage area networks) or via any other suitable type of network and/or protocol.
In some embodiments, system memory 20 may be one embodiment of a computer-accessible medium configured to store program instructions and data as described above for implementing embodiments of the corresponding methods and apparatus. However, in other embodiments, program instructions and/or data may be received, sent or stored upon different types of computer-accessible media. Generally speaking, a computer-accessible medium may include non-transitory storage media or memory media such as magnetic or optical media, e.g., disk or DVD/CD coupled to computing device 100 via I/O interface 30. A non-transitory computer-accessible storage medium may also include any volatile or non-volatile media such as RAM (e.g. SDRAM, DDR SDRAM, RDRAM, SRAM, etc.), ROM (read only memory) etc., that may be included in some embodiments of computing device 100 as system memory 20 or another type of memory. Further, a computer-accessible medium may include transmission media or signals such as electrical, electromagnetic or digital signals conveyed via a communication medium such as a network and/or a wireless link, such as those that may be implemented via network interface 40. Portions or all of multiple computing devices such as those illustrated in
A compute node, which may be referred to also as a computing node, may be implemented on a wide variety of computing environments, such as commodity-hardware computers, virtual machines, web services, computing clusters and computing appliances. Any of these computing devices or environments may, for convenience, be described as compute nodes.
A network set up by an entity such as a company or a public sector organization to provide one or more web services (such as various types of cloud-based computing or storage) accessible via the Internet and/or other networks to a distributed set of clients may be termed a provider network. Such a provider network may include numerous data centers hosting various resource pools, such as collections of physical and/or virtualized computer servers, storage devices, networking equipment, and the like, needed to implement and distribute the infrastructure and web services offered by the provider network. The resources may in some embodiments be offered to clients in various units related to the web service, such as an amount of storage for storage, processing capability for processing, as instances, as sets of related services and the like. A virtual computing instance may, for example, comprise one or more servers with a specified computational capacity (which may be specified by indicating the type and number of CPUs, the main memory size and so on) and a specified software stack (e.g., a particular version of an operating system, which may in turn run on top of a hypervisor).
A number of different types of computing devices may be used singly or in combination to implement the resources of the provider network in different embodiments, including general purpose or special purpose computer servers, storage devices, network devices and the like. In some embodiments a client or user may be provided direct access to a resource instance, e.g., by giving a user an administrator login and password. In other embodiments the provider network operator may allow clients to specify execution requirements for specified client applications and schedule execution of the applications on behalf of the client on execution platforms (such as application server instances, Java™ virtual machines (JVMs), general purpose or special-purpose operating systems, platforms that support various interpreted or compiled programming languages such as Ruby, Perl, Python, C, C++ and the like or high-performance computing platforms) suitable for the applications, without, for example, requiring the client to access an instance or an execution platform directly. A given execution platform may utilize one or more resource instances in some implementations; in other implementations multiple execution platforms may be mapped to a single resource instance.
In many environments, operators of provider networks that implement different types of virtualized computing, storage and/or other network-accessible functionality may allow customers to reserve or purchase access to resources in various resource acquisition modes. The computing resource provider may provide facilities for customers to select and launch the desired computing resources, deploy application components to the computing resources and maintain an application executing in the environment. In addition, the computing resource provider may provide further facilities for the customer to quickly and easily scale up or scale down the numbers and types of resources allocated to the application, either manually or through automatic scaling, as demand for or capacity requirements of the application change. The computing resources provided by the computing resource provider may be made available in discrete units, which may be referred to as instances. An instance may represent a physical server hardware platform, a virtual machine instance executing on a server or some combination of the two. Various types and configurations of instances may be made available, including different sizes of resources executing different operating systems (OS) and/or hypervisors, and with various installed software applications, runtimes and the like. Instances may further be available in specific availability zones, representing a logical region, a fault tolerant region, a data center or other geographic location of the underlying computing hardware, for example. Instances may be copied within an availability zone or across availability zones to improve the redundancy of the instance, and instances may be migrated within a particular availability zone or across availability zones. As one example, the latency for client communications with a particular server in an availability zone may be less than the latency for client communications with a different server. As such, an instance may be migrated from the higher latency server to the lower latency server to improve the overall client experience.
In some embodiments the provider network may be organized into a plurality of geographical regions, and each region may include one or more availability zones. An availability zone (which may also be referred to as an availability container) in turn may comprise one or more distinct locations or data centers, configured in such a way that the resources in a given availability zone may be isolated or insulated from failures in other availability zones. That is, a failure in one availability zone may not be expected to result in a failure in any other availability zone. Thus, the availability profile of a resource instance is intended to be independent of the availability profile of a resource instance in a different availability zone. Clients may be able to protect their applications from failures at a single location by launching multiple application instances in respective availability zones. At the same time, in some implementations inexpensive and low latency network connectivity may be provided between resource instances that reside within the same geographical region (and network transmissions between resources of the same availability zone may be even faster).
As set forth above, in accordance with the disclosed techniques, a graphics object service may be employed to generate data associated with one or more objects. The term object, as used herein, refers to any portion of a scene, image or other collection of information. An object may be, for example, a particular pixel or collection of pixels. An object may be, for example, all or any portion of a particular asset. An object may also be, for example, all or any portion of a collection of assets. An object may also be, for example, all or any portion of an entity such as a tree, fire, water, a cloud, a cloth, clothing, a human, an animal and others. For example, an object may be a portion of a tree. An object may also, for example, include all or any portion of a collection of objects, entities and/or assets. For example, an object may be a group of multiple trees or clouds that may be located, for example, at any location with respect to one another.
A graphics object service may be marketed or publicized to graphics content developers. In some cases, a graphics objects service may specialize in certain types of objects. For example, one graphics objects service may specialize in rendering fire, while another graphics objects service may specialize in rendering water. In some cases, a specialized location such as a specialized website may be provided for various different graphics object services to advertise their services. A graphics object service may reside on one or more servers. The service may be scalable to meet the demands of one or more customers and may increase or decrease in capability based on, for example, the number and type of incoming client requests. For certain objects, graphics object services may process graphics objects in parallel across multiple nodes of the graphics objects services. This may be done, in one embodiment, to reduce the latency for rendering the graphics object. Portions of the graphics object service may also be migrated to be placed in a position of reduced latency with a requesting client.
A graphics object service may provide information for communicating with the graphics object service to request and obtain graphics object data. For example, information associated with an application programming interface (API) may be exposed that enables graphics object data to be requested and described in a format that is understandable by the graphics object service. Some example API information that provides instructions for describing a requested object will now be described in detail. In particular,
Type parameter 310A provides information that describes a type of object that is requested for rendering. As set forth above, some example object types include trees, fire, water, clouds, cloths, clothing, humans, animals and any portion of one or more of these or others. In some cases, a particular graphics object service may only render a single, specialized object type, and it may not be necessary to specify a requested object type.
In some cases, depending upon the values of certain parameters or sub-parameters, the logical configuration shown in
Referring back to
In some cases, an object's depth in association with a viewpoint of a scene may be calculated based on distance sub-parameter 311A and direction sub-parameter 311B. Also, in some cases, an object's depth in association with a viewpoint of a scene may, for example, be indicated based on a separate depth parameter or depth sub-parameter.
As should be appreciated, because the viewpoint parameter 310B and its associated sub-parameters 311A and 311B are dependent upon the scene viewpoint, these parameters may change when the scene viewpoint changes. In some cases, changes in scene viewpoint may occur rapidly, sometimes even occurring with each newly generated scene. This is in contrast with other parameters such as type parameter 310A, size parameter 310B and other parameters that may, in some cases, change less frequently.
Size parameter 310C indicates the size of the object. In some cases, size parameter 310C may correspond to the object's abstract size as opposed to the object's perceived size when represented at a particular distance. As should be appreciated, the object's perceived size may, in some cases, be calculated based upon size parameter 310C and viewpoint distance sub-parameter 311A. In some cases, if it is desired to have a particular content item calculate the object's perceived size, then the perceived size may also be provided as an additional or alternative parameter. In other cases, however, size parameter 310C may be employed to correspond to the object's abstract size. Depending upon the type of object selected, different sizes may be specified for different portions or areas of the object. Additionally, depending upon the particular object or area being described, any combination of different size measurement attributes such as length, width, height, radius, diameter, volume and others may be employed. Some examples of multiple size parameters for different object areas are described below with respect to
Color parameter 310D provides information about the color of an object. A single color or a range of different colors may be specified for color parameter 310D. Additionally, depending upon the type of object being requested, different color parameters may be specified for different portions or areas of the object. Some examples of multiple color parameters for different object areas are described below with respect to
Texture parameter 310E provides information about the texture of an object. A single texture or a range of different texture may be specified for texture parameter 310E. Additionally, depending upon the type of object being requested, different texture parameters may be specified for different portions or areas of the object. Some examples of multiple texture parameters for different object areas are described below with respect to
Light source parameter 310F provides information about a light source associated with the scene. Light source parameter 310F includes light source intensity sub-parameter 312A, light source distance sub-parameter 312B and light source direction sub-parameter 312C. Light source intensity sub-parameter 312A indicates an intensity of the light source. Light source distance sub-parameter 312B indicates a straight line distance of the light source from a reference point associated with the object. Light source direction sub-parameter 311B may indicate the direction of the straight line distance. The reference point of the object associated with light source parameter 310F may be the same or different from the reference point of the object associated with viewpoint parameter 310B. As should be appreciated, in some cases, a scene may include multiple light sources, and values for light source parameter 310F and its sub-parameters 312A-C may be provided for any or all of these multiple light sources.
Rotation parameter 310G provides information about the rotation of an object. In some cases, rotation parameter 310G may be specified using measurements such as degrees. For example, in some cases, a human character standing directly in front of a scene viewpoint and directly facing the scene viewpoint may be expressed as standing at 0 degrees or 360 degrees. By contrast, a human character standing directly in front of a viewpoint while facing directly away from the scene viewpoint (i.e., with his back to the viewer) may be expressed as standing at 180 degrees. In some cases, rotation parameter 310G may include three sub-parameters, for example, to indicate rotation with respect to axes of a three-dimensional scene, which may commonly be referred to as the X, Y and Z axes. In other cases, greater or fewer than three sub-parameters may be included. Additionally, depending upon the type of object being requested, different rotation parameters 310G may be specified for different portions or areas of the object.
Reflectivity parameter 310H may indicate a degree of reflectivity of an object. For example, objects such as glass and water may tend to have higher reflectivity than objects such as trees or cloth. In some cases, reflectivity may be determined based on the type of object selected and need not necessarily be specified. Also, in some cases, reflections may be added to the object by, for example, the client device and need not be handled by the graphics object service. When an object is reflective and it is desired to have at least a portion of the reflection rendered or otherwise processed by the graphics object service, additional information about the scene may sometimes be provided to the graphics object service in order to assist the graphics object service in rendering or processing the reflection.
Time parameter 310I may indicate a time associated a request. Time parameter 310I may indicate, for example, a desired time at which the client device would like to receive all or any portion of data requested from the graphics object service. Time parameter 310I may also indicate, for example, an allowable delay time associated with generating, transmitting and/or receiving requested data. Time parameter 310I may also indicate, for example, an allowable latency associated with generating, transmitting and/or receiving requested data. Time parameter 310I may, for example, include any number of sub-parameters corresponding to attributes such as allowable delay and/or latency. Time parameter 310I may, for example, help the client device to better organize or schedule when it receives results associated with a number of different requests. In some cases, time parameter 310I may allow the client to create a schedule according to a logical progression, whereby results that are needed earlier are scheduled to be received at an earlier time, while results that are not needed until later may be scheduled to be received at a later time. Also, in some cases, time parameter 310I may allow the client to create a schedule such that related results are received within a desired temporal proximity of one another.
Shading parameter 310J may include any information associated with shading of an object and/or one or more portions of an object. Shading parameter 310J may, for example, include any number of sub-parameters associated with different portions of an object. Shading parameter 310J may also include, for example, information associated with varying levels of light and/or color. Shading parameter 310J may also include, for example, information that may indicate various angles of portions of an object with respect to one or more light sources. Shading parameter 310J may also include, for example, information that may indicate various types of shading such as flat shading or smooth shading.
As set forth above, an association of a number of the parameters and sub-parameters of render call 300 may be dependent upon the values of other parameters and sub-parameters of render call 300. To illustrate these concepts,
As also shown in
As should be appreciated,
Viewpoint parameter 410B of
Size parameter 410C of
Color parameter 410D of
Texture parameter 410E of
The remaining parameters of
As set forth above, in addition to rendering objects, a graphics object service may also be employed to generate graphics commands that may be executed, for example, in order to render an object. These commands may include, for example, instructions for operations such as projection, shading, reflection, lighting and others.
It is noted that the logical configurations depicted in
Thus, as described above, graphics object data may be requested from a graphics object service based on, for example, information provided by the graphics object service for communicating with the graphics object service. Such information may be employed to embed or insert graphics object service requests into a particular content item such as a video game. When a content item is executed at a client device, the content item may generate various scenes or portions of scenes for display. When a scene is generated, the content item may identify any graphics object service requests associated with the scene. Each scene may have any number of associated graphics object service requests that may be sent to any number of different graphics object services. For example, a particular scene may include a tree, fire, water and a house. The content item may, for example, request rendering of the tree from a tree service, rendering of fire from a fire service, rendering of water from a water service and rendering of the house locally by the client device.
A graphics object service request may include, for example, a call or other command for requesting graphics object data such as the example API calls 300, 400 and 500 described above or any other appropriate call or command. A graphics object service request may include, for example, information for identifying and establishing communications with a graphics object service such as an Internet Protocol (IP) or other communications address. A graphics object service request may also include, for example, information identifying an account associated with the request and identifying payment for the requested graphics object data.
Accounts may be opened with a graphics object service using a number of techniques. For example, when a content item is purchased, a seller of the content item may prompt a purchaser to open an account with one or more graphics object services. In some cases, if authorized by the purchaser, the same purchaser information and payment method used to purchase the content item may be provided to one or more graphics object services. For example, if authorized by the purchaser, the purchaser's name and credit card information may be provided to one or more graphics object services. As another example, code may be included within the content item to determine when the content item is being used for the first time and, on such occasions, may prompt the user to open an account with one or more graphics object services. In some cases, there may be a permanent or temporary option on a navigation menu associated with the content item for opening an account with one or more graphics object services. The content item may, for example, include code for communicating with the graphics object services and for obtaining any necessary information from the user in order to open an account.
In some cases, an account may be opened using a telephone or using the Internet by, for example, navigating to a particular web page associated with a graphics object service and opening an account. In some cases, payment information such as a credit card number may be required to open an account, while, in other cases, such information may not be required or may be provided at a later time. An account may, for example, be associated with a particular user or group of users, a particular device or group of devices, a particular content item or group of content items or any combination of these or other entities.
A number of different techniques may be employed to identify a particular account. For example, in some cases, an account may be identified by a particular identifier such as a user name. In some cases, if an account is associated with a particular content item, then the account may be identified by an identifier assigned to the content item. Additionally, in some cases, if an account is associated with a particular device, then the account may be identified by a hardware identifier or software identifier assigned to the particular device.
In some cases, it may be desirable to validate the use of the account to ensure that the account is being used in an authorized manner. A number of different techniques may be employed to validate a use of an account. In some cases, if a request specifies an authorized device or address to which to return the requested graphics object data, then it may be assumed that the account is being used by an authorized entity. Additionally, in some cases, a validation identifier such as a password may be employed to validate the use of the account. For example, prior to playing a video game that includes one or more graphics object service requests, a user may be prompted to enter a password associated with his account. The password may then be provided to the graphics object services, for example, either at the beginning of a game session and/or along with each individual request. In some cases, graphics object service requests may only be provided to graphics object services for which a password is received and validated, while graphics object service requests to services with non-received or non-validated passwords may be ignored.
Any number of different approaches may be employed to determine costs for graphics service requests. For example, in some cases, an account may be billed individually for each request. Additionally, in some cases, there may be a single flat fee for each individual request. In other cases, the cost per request may vary depending on factors such as a number of requests over a particular time period, the complexity of the request, the size of the object associated with the request, the type of the object associated with the request, the content item for which the request is issued, the account issuing the request and any combination of these or other factors. The complexity of the request may be determined based on factors such as, for example, the number of different colors, textures and areas of the object. For example, a tree in the summer with leaves and birds on its branches may be more complex than a tree in winter with no leaves and no birds.
In some cases, an account holder may purchase or otherwise obtain a subscription associated with an account. The subscription may, for example, charge the account a single flat fee in exchange for an unlimited number of graphics object service requests for a particular time period. For example, an account may be charged a monthly fee for a monthly subscription that entitles the account to an unlimited number of graphics object service requests for a month. In some cases, the subscription may entitle an account to, for example, a limited number of requests or a limited amount of total estimated complexity across any number of requests. After the limit is exceeded, the account may, for example, be charged individually for each request in excess of the specified limit.
Thus, as set forth above, a content item such as a video game may include any number of associated graphics object service requests directed to any number of different graphics object services. Content items including graphics object service requests may, for example, be acquired by client devices and executed at the client devices. When a content item is executed at a client device, the content item may generate various scenes or portions of scenes for display. When a scene is generated, the content item may identify any graphics object service requests associated with the scene. Some scenes may not have any associated graphics object service requests. When a graphics object service request is identified, the request may be sent over a network to a graphics object service associated with the request.
For each scene, graphics object service request processing component 612 may identify any associated graphics object service requests within content information 611. Graphics object service request processing component 612 may then perform operations such as selecting one or more of the identified graphics object service requests for execution, providing each selected request to a communications component 620 for transmission to a corresponding graphics object service 650, monitoring the progress of graphics object service requests, substituting local data for requests that are not satisfactorily fulfilled and managing a combination of received request results with local data in order to compile an image in accordance with the scene.
Upon identifying one or more graphics object service requests associated with a particular scene, graphics object service request processing component 612 may select none, some or all of the identified requests for execution. Graphics object service requests may be selected for execution based on any number of different factors. In some cases, a user may provide input to indicate whether the user desires to issue some or all of the graphics object service requests identified within content information 611. For example, a user may specify certain graphics object services with which the user may or may not want to interact at any given time. In some cases, graphics object service request processing component 612 may be limited to interact only with graphics object services 650 for which a user has provided authorization information such as a validated password.
Additionally, a user may provide input regarding, for example, a desired graphics quality for content item 610. In some cases, in order to save costs associated with graphics object service requests or to possibly improve the display speed of the content item, a user may select to display images with a lower quality. Graphics object service request processing component 612 may select to ignore some or all graphics object service requests based on such graphics quality input.
As another example, graphics object service request processing component 612 may, in some cases, interact with a network monitoring component 630, which may provide information regarding, for example, current or recent network quality of service conditions. In some cases, if network quality of service is unfavorable, graphics object service request processing component 612 may determine that network conditions may prevent some or all graphics object service requests from being received within a desired time period. Graphics object service request processing component 612 may select to ignore some or all graphics object service requests based on this determination.
In some cases, content information 611 may include multiple requests to different graphics object services 650 for providing the same or similar data. This may, for example, allow graphics object service request processing component 612 to make a runtime determination regarding a preferred graphics object service 650 for satisfying the request and providing the data. In some cases, a user may provide input that ranks certain graphics object services 650 by order of user preference. Graphics object service request processing component 612 may use this ranking, or any other user input or other information, to choose between multiple graphics object services 650 for providing the same or similar data.
Upon selecting one or more graphics object service requests for execution, graphics object service request processing component 612 may provide each selected request to communications component 620 for transmission to a graphics object service 650. Some example features of a graphics object service request are described in detail above. Generally, a graphics object service request may include, for example, a request to render an object or provide graphics commands associated with rendering of an object, a description of an object, a description of aspects of a two-dimensional or three-dimensional scene, an address or other information for identifying and establishing communications with a graphics object service, account information payment information such as credit card information and any other appropriate information. As also set forth above, any portion of information in a graphics object service request may be provided in accordance with information associated with an API or any other format understandable by a particular graphics object service. For example, a request may include a call and parameters such as, for example, any one or more of the parameters depicted in
While a graphics object service request may, in some cases, include a description of a requested object, a graphics object service request is not required to include an object description. For example, in some cases, as an alternative or in addition to an object description, a graphics object service request may include an identifier associated with an object and/or a pointer to a collection of information associated with a requested object that may describe the requested object. For example, a graphics object service request may include a uniform resource locator (URL) corresponding to a picture or other file associated with a requested object that may describe the requested object.
Service requests to a graphics object service (also referred to herein as “graphics object service requests”) may have a variety of different characteristics. For example, service requests may include both web service requests and non-web service requests. Service requests may also be issued using any number of different protocols. For example, service requests may be issued using both protocols that are considered reliable and protocols that are considered non-reliable. Also, for example, service requests may be issued using either hypertext transfer protocol (HTTP) or non-HTTP protocols. In some cases, service requests may be issued using different transport layer protocols such as transmission control protocol (TCP), user datagram protocol (UDP) and others. Service requests and graphics object service requests in accordance with the disclosure herein may include any number of these or other different characteristics.
Communications component 620 may transmit each selected graphics object service request to a corresponding graphics object service 650 using a network 640 such as the Internet. In particular,
In some cases, when multiple graphics object service requests are being sent simultaneously or within a certain time period with respect to one another, client 600 may perform various operations to help ensure that client 600 will not be overburdened by attempting to receive too much data when the requests are being returned and received by client 600. For example, in some cases, client 600 may determine a maximum amount of incoming data that it can receive within a particular time period. Client 600 may then allocate a certain proportion of this incoming data to various requests. In some cases, client 600 may use these proportions or other calculations to determine certain encoding bitrates or other transmission parameters that may be used in accordance with returned request results. In some cases, client 600 may perform these operations and provide the resulting determinations to graphic object services 650. In other cases, graphic object services 650 or other components may perform these operations based on information provided by the client 600 and/or other information.
Each graphics object service 650, and any or all of the components operated in association therewith, may be distributed across any number of different computers in any number of different locations. For example, each graphics object service 650 and its components may be distributed across a number of different server computers in one or more data centers. Each graphics object service 650 and its components may, in some cases, be executed in accordance with one or more virtual machine instances. An example system of distributed computers that may be used to execute a graphics object service is described above with reference to
Additionally,
Account and payment processing components 651 may be employed to perform operations such as identifying an account associated with a request, validating account authorization information, validating payment information and other operations. As set forth above, a graphics object service request may, in some cases, include information for identifying an account associated with the request. This account identification information may include, for example, a user name, an account number, a credit card number, hardware and/or software identifiers associated with the client 600 and/or content item 610, an IP address or other communications address associated with the client 600 or any combination of this or other information.
Additionally, in some cases, a graphics object service request may include account authorization information for validating that the account is being used in an authorized manner. The account authorization information may include, for example, a password, hardware and/or software identifiers, an IP address or other communications address or any combination of this or other information. Account and payment processing components 651 may, for example, validate received account authorization information against information for the identified account within stored account and payment information 661. In some cases, use of account may be presumed to be valid in some or all circumstances, and validation of account authorization information may not be desired.
Furthermore, in some cases, a graphics object service request may include payment information associated with the request. The payment information may include, for example, credit card information, Internet payment account information or any other information. Account and payment processing components 651 may validate received payment information by, for example, confirming that credit card information satisfies valid date and number formats. If the payment information is valid, account and payment processing components 651 may keep the information in stored account and payment information 661. Additionally, in some cases, account and payment processing components 651 revalidate previously received payment type information by, for example, confirming that a credit card has not expired since a previous use.
If, for example, the account authorization information and payment information are sufficient and valid, then account and payment processing components 651 may indicate this to other components of graphics object service 650. By contrast, if, for example, the account authorization information and payment information are insufficient or invalid, then account and payment processing components 651 may provide instructions to send an error message for transmission to client 600 indicating a problem with the request and possibly requesting additional information.
Graphics processing components 652 may include one or more components for performing graphics operations such as rendering an object or generating graphics commands associated with rendering of an object. Rendering of an object may encompass operations such as, for example, projection, reflection, lighting, shading and others. Graphics processing components 652 may include, for example, one or more graphics processing units. In some cases, an indication of sufficient and valid account authorization and payment information may be required prior to beginning graphics processing operations. In some other cases, graphics operations may be initiated without first receiving an indication of sufficient and valid account authorization and payment information, but graphics operations may be stopped if such an indication is not received within a certain time period.
Graphics processing components 652 may access stored graphics information 662, which may include, for example, information associated with one or more different object types. In some cases, graphics processing components 652 may obtain a portion of stored graphics information 662 for a particular object type identified by a request and then combine the stored graphics information 662 with information received in the request. For example, if a request is received for a tree, then graphics processing components 652 may obtain stored graphics information 662 associated with a tree, which may include certain instructions for generating areas of a tree such a trunk, branches and leaves. In some cases, graphics information 662 could include, for example, information that describes the shape of a leaf and some patterns on the surface of the leaf. The leaf could then be colored and textured based on, for example, color and texture information in the received request.
Upon performing graphics processing operations, the processed data may, in some cases, be provided to encoding and compression components 653. Encoding and compression components 653 may generally use any appropriate technique to encode and/or compress graphics data for transmission in response to a graphics object service request. In some cases, it may not be necessary to encode and/or compress the requested graphics data. After any necessary encoding and/or compressing operations are performed, communications and routing components 654 may transmit the requested data over network 640 back to the requesting client 600.
Referring back to
In some cases, results of a graphics object service request may not be returned to client 600 quickly enough to be incorporated into a resulting image. This may occur, for example, due to technical problems at a graphics object service 650 or network communications problems. In order to deal with these instances, graphics object service request processing component 612 or another component may, in some cases, monitor the progress of results of graphics object service requests. When results are not received within a desired time period, a determination may be made to substitute local data at client 600 for non-received results of a graphics object service request. For example, in some cases, a rendered version of an object from a previous image may be substituted for the object associated with the request. As another example, in some cases, the object may be rendered locally by client 600 as a substitute for the non-received graphics object service request data.
When graphics object data is successfully received from one or more graphics object services 650, client 600 may compile an image of the scene by, for example, combining graphics object data received from one or more graphics object services 650 with data generated based on graphics operations performed locally at client 600. It is unnecessary for client 600 to wait to receive results for all graphics object service requests in order to begin the image compilation process. Rather, in some cases, while one or more graphics objects are being processed by corresponding graphics object services, the image compilation process may be started at client 600 by, for example, providing to graphics processing components 614 data corresponding to portions of an image that are rendered locally. In these and other cases, greater efficiency may be sometimes achieved by processing data for various portions of an image in parallel at both client 600 and one or more graphics object services 650. An image may include, for example, a collection of information associated with a displayable output.
Client 600 may employ various techniques for combining graphics object data received from one or more graphics object services 650 with scene data generated locally at client 600. One example combination technique, which is referred to herein as a layering technique, may employ an image representation having multiple layers. Each layer of the representation may correspond to a respective portion of the image. For example, a first layer may include a first portion of the image rendered by a first graphics object service, while a second layer may include a second portion of the image rendered by a second graphics object service. In particular,
An example depiction of the layering technique is illustrated in
Another example combination technique, which is referred to herein as a stitching technique, may involve inserting rendered objects into identified areas within an image. For example, a first object rendered by a first graphics object service may be inserted at a first identified image area, while a second object rendered by a second graphics object service may be inserted at a second identified image area. Each image area may be identified using, for example, coordinate values identified based on content information 611 of
Regardless of which of the above described techniques or other techniques are employed to combine graphics data from graphics object services 650 and client 600, there may be some cases in which combined portions of an image are determined to not blend into one another as smoothly or seamlessly as desired. In these and other cases, a variety of visual effects may be employed to help improve the appearance of combined portions of an image. In some cases, blurring effects and image filtering effects may be applied to image areas such as borders or transition points between objects or combined portions of an image. For example, a tree rendered by a graphics object service 650 may be compiled into an image, but the tree's trunk may not blend into the ground as smoothly as desired. In this example scenario, a blurring effect may be applied to the base of the tree trunk in order to make the trunk appear to blend more smoothly into the ground.
Thus, as described above, one or more graphics object services may be employed to provide data associated with various objects for compilation into an image.
At operation 1114, any graphics object service requests associated with the scene are identified. Some scenes may have no associated graphics object service requests, while other scenes may have one or more associated graphics object service requests for data associated with one or more objects in the scenes. The graphics object service requests may be identified based on, for example, content information 611 of
At operation 1116, none, some or all of the graphics object service requests identified at operation 1114 are selected for execution. The selection of graphics object service requests for execution may be performed, for example, by graphics object service request processing component 612 of
At operation 1118, each selected graphics object service request is transmitted to its associated graphics object service for execution. As set forth above, in some cases, information may also be provided to one or more graphics object services to coordinate the size and/or quality of request result data in order to prevent network communications to client 600 from overloading client 600 with too much data in a particular time period when the request results are returned.
At operation 1120, results of the graphics objects request are received by client 600. As set forth above, in some cases, it may be necessary to decompress and/or decode some or all of the received results.
As set forth above, in some cases, if any results are not received within a desired time period, then local data may be substituted for the non-received results. At operation 1122, it may be determined whether results for any of the graphics object service requests transmitted at operation 1118 have not been received within such a desired time period. If it is determined that any of the results have not been received within the desired time period, then, at act 1124 locally available data may be substituted for the non-received graphics object service request results. As set forth above, for example, a rendered version of an object from a previous image may be substituted for an object associated with the request. As another example, the object may be rendered locally by client 600 as a substitute for the non-received graphics object service request data.
At operation 1126, local graphics operations associated with the scene may be performed by client 600. The local graphics operations may include, for example, rendering or initiating a rendering process for portions of the scene that are not being rendered by graphics object services.
At operation 1128, data received from graphics object services is combined with local graphics data to compile a resulting image. In some cases, data may be combined such that rendered objects received from graphics object service requests may be at least partially included in the resulting image. Additionally, in some cases, data may be combined such that at least some commands received from graphics object service requests may be used to, at least in part, render one or more objects that are at least partially included in the resulting image. Various techniques may be employed to combine graphics data from the client device and the different graphics object services. As set forth above, one example combination technique may employ an image representation having multiple layers. Each layer of the representation may correspond to a respective portion of the image. For example, a first layer may include a first portion of the image rendered by a first graphics object service, while a second layer may include a second portion of the image rendered by a second graphics object service. Another example object combination technique may involve inserting rendered objects into identified areas within the image. For example, a first object rendered by a first graphics object service may be inserted at a first identified image area, while a second object rendered by a second graphics object service may be inserted at a second identified image area.
At operation 1130, one or more visual effects may, if desired, be applied to the resulting compiled image. As set forth above, there may be some cases in which combined portions of an image do not appear to blend into one another as smoothly or seamlessly as desired. In these and other cases, a variety of visual effects may be employed to help improve the appearance of combined portions of an image. In some cases, blurring effects and image filtering effects may be applied to image areas such as borders or transition points between objects or combined portions of an image. At operation 1132, the resulting image is displayed.
At operation 1134, it is determined whether there are any remaining scenes for display associated with execution of the content item. If so, the process returns to operation 1112 to identify data associated with a next remaining scene. If not, then, at operation 1136, execution of the content item is completed.
Thus, as set forth above, a client may interact with one or more graphics object services to obtain graphics object data.
At operation 1210, a graphics object service request is received by a graphics object service. Some example contents of a graphics object service request are described in detail above and are not repeated here.
At operation 1211, it is determined whether the request is part of a pre-established session with a particular requesting client. In some cases, when a particular client issues a graphics object service request, the graphics object service may establish a communications session with the particular client such that the particular client may issue subsequent requests as part of the same session (provided that the session is not terminated prior to receiving the subsequent requests). The establishment of such a session may provide a number of advantages. For example, in some cases, establishment of the session may allow account information, authorization information and/or payment information to be identified and associated with the pre-established session. This may result in higher efficiency by, for example, allowing the graphics object service to process subsequent requests within the session without having to repeatedly identify the account information, authorization information and/or payment information for each subsequent request (see operations 1212, 1214 and 1216 of
At operation 1212, an account associated with the graphics object service request is identified. As set forth above, the account may, for example, correspond to a particular user or group of users, a particular client device from which the request was sent, a particular content item or any combination of these or other entities. The received graphics object service request may include information identifying the account. This account identification may include, for example, a user name, an account number, a credit card number, hardware and/or software identifiers, an IP address or any combination of this or other information.
At operation 1214, account authorization information is identified. In some cases, a graphics object service request may include authorization information such as a password, hardware and/or software identifiers, an IP address or any combination of this or other information. In other cases, use of an account may be presumed to be authorized and no authorization information may be provided.
At operation 1216, payment information is identified. As described above, in some cases, a subscription may be associated with an account. The subscription may, for example, charge the account a single flat fee in exchange for an unlimited number of graphics object service requests for a particular time period. In some other cases, an account may be billed individually for each request. As set forth above, in some cases, a graphics object service request may include payment information such as credit card information. Additionally, in some cases, previously received payment information may be stored in association with a particular account. Furthermore, in some cases, an account may have an associated balance or credit.
At operation 1217 a cost associated with the request is determined. As set forth above, any number of different approaches may be employed to determine costs for graphics service requests. In some cases, there may be a single flat fee for each individual request. In other cases, the cost per request may vary depending on factors such as a number of requests over a particular time period, the complexity of the request, the size of the object associated with the request, the type of the object associated with the request, the content item for which the request is issued, the account issuing the request and any combination of these or other factors. In some cases, it may not be necessary to determine cost such as when a subscription has been paid for unlimited operations within a particular time period.
At operation 1218, it is determined whether the identified authorization information and/or the identified payment information for the account are satisfactory. This determination may include, for example, determining whether sufficient information has been received to enable payment and authorization. This determination may also include, for example, validating the identified information against stored information and requirements. For example, authorization information may be validated by comparing authorization information in the received graphics object service request against stored authorization information associated with the identified account.
In some cases, received payment information may be validated by, for example, confirming that received credit card information satisfies valid date and number formats and is not expired. Previously received payment information may also sometimes be revalidated. Payment information may also be validated by, for example, confirming that a subscription is still valid or confirming that a balance or credit associated with an account exceeds the cost determined at operation 1217.
If either or both of the account authorization information or the payment information are unsatisfactory, then, at operation 1220, an error message may be transmitted back to the requesting client to indicate a problem with the request and possibly request additional information or payment. If the account authorization information and the payment information are satisfactory, then, at operation 1222, one or more graphics operations may be performed in accordance with the received graphics object service request. It is noted that, in some cases, it may not be necessary or desired to determine that both payment and authorization information or satisfactory. In some cases, a determination of satisfaction with only payment information or only authorization information or neither type of information may be performed. Additionally, there is no requirement that the determinations with respect to payment and authorization be made simultaneously or in combination with one another.
The graphics operations performed at operation 1222 may be performed by components such as graphics processing components 652 of
After performance of the graphics operations at operation 1222, the resulting graphics object data may be encoded and compressed for transmission back to the requesting client device at operation 1224. In some cases, it may not be necessary to encode and/or compress the requested graphics data. Additionally, in some cases, encoding and/or compression may be performed in accordance with information provided by the requesting client or other components to help ensure that the requesting client is not overburdened with receiving too much data resulting from graphics object service requests within a particular time period. After any necessary encoding and/or compressing operations are performed, the requested graphics object data may be transmitted back to the requesting client 600 at operation 1226.
In addition or as an alternative to the embodiments described above, there are some cases in which a graphics object service request may be delivered to a different device or node then the device or node that issued the request. For example, in some cases, a server computer may execute a content item and transmit images or portions of images to a client device for display at the client device. The server may also submit various graphics object requests to one or more graphics object services, the results of which are also delivered to the client device. The client device may be provided with instructions for combining different portion of each image using, for example, any of the combination techniques described above. As another example, in some cases, graphics commands generated by a graphics object service may be executed by a third party or other device or node to render an object before being provided to the client device. As yet another example, results of graphics object requests may be combined by a third party or other device or node using, for example, any of the combination techniques described above before being provided to a client device.
Each of the processes, methods, and algorithms described in the preceding sections may be embodied in, and fully or partially automated by, code modules executed by one or more computers or computer processors. The code modules may be stored on any type of non-transitory computer-readable medium or computer storage device, such as hard drives, solid state memory, optical disc and/or the like. The processes and algorithms may be implemented partially or wholly in application-specific circuitry. The results of the disclosed processes and process steps may be stored, persistently or otherwise, in any type of non-transitory computer storage such as, e.g., volatile or non-volatile storage.
The various features and processes described above may be used independently of one another, or may be combined in various ways. All possible combinations and subcombinations are intended to fall within the scope of this disclosure. In addition, certain methods or process blocks may be omitted in some implementations. The methods and processes described herein are also not limited to any particular sequence, and the blocks or states relating thereto can be performed in other sequences that are appropriate. For example, described blocks or states may be performed in an order other than that specifically disclosed, or multiple blocks or states may be combined in a single block or state. The example blocks or states may be performed in serial, in parallel or in some other manner. Blocks or states may be added to or removed from the disclosed example embodiments. The example systems and components described herein may be configured differently than described. For example, elements may be added to, removed from or rearranged compared to the disclosed example embodiments.
It will also be appreciated that various items are illustrated as being stored in memory or on storage while being used, and that these items or portions thereof may be transferred between memory and other storage devices for purposes of memory management and data integrity. Alternatively, in other embodiments some or all of the software modules and/or systems may execute in memory on another device and communicate with the illustrated computing systems via inter-computer communication. Furthermore, in some embodiments some or all of the systems and/or modules may be implemented or provided in other ways, such as at least partially in firmware and/or hardware, including, but not limited to, one or more application-specific integrated circuits (ASICs), standard integrated circuits, controllers (e.g., by executing appropriate instructions, and including microcontrollers and/or embedded controllers), field-programmable gate arrays (FPGAs), complex programmable logic devices (CPLDs), etc. Some or all of the modules, systems and data structures may also be stored (e.g., as software instructions or structured data) on a computer-readable medium, such as a hard disk, a memory, a network or a portable media article to be read by an appropriate drive or via an appropriate connection. The systems, modules and data structures may also be transmitted as generated data signals (e.g., as part of a carrier wave or other analog or digital propagated signal) on a variety of computer-readable transmission media, including wireless-based and wired/cable-based media, and may take a variety of forms (e.g., as part of a single or multiplexed analog signal, or as multiple discrete digital packets or frames). Such computer program products may also take other forms in other embodiments. Accordingly, the present invention may be practiced with other computer system configurations.
Conditional language used herein, such as, among others, “can,” “could,” “might,” “may,” “e.g.,” and the like, unless specifically stated otherwise, or otherwise understood within the context as used, is generally intended to convey that certain embodiments include, while other embodiments do not include, certain features, elements, and/or steps. Thus, such conditional language is not generally intended to imply that features, elements and/or steps are in any way required for one or more embodiments or that one or more embodiments necessarily include logic for deciding, with or without author input or prompting, whether these features, elements and/or steps are included or are to be performed in any particular embodiment. The terms “comprising,” “including,” “having,” and the like are synonymous and are used inclusively, in an open-ended fashion, and do not exclude additional elements, features, acts, operations and so forth. Also, the term “or” is used in its inclusive sense (and not in its exclusive sense) so that when used, for example, to connect a list of elements, the term “or” means one, some or all of the elements in the list.
While certain example embodiments have been described, these embodiments have been presented by way of example only and are not intended to limit the scope of the inventions disclosed herein. Thus, nothing in the foregoing description is intended to imply that any particular feature, characteristic, step, module or block is necessary or indispensable. Indeed, the novel methods and systems described herein may be embodied in a variety of other forms; furthermore, various omissions, substitutions and changes in the form of the methods and systems described herein may be made without departing from the spirit of the inventions disclosed herein. The accompanying claims and their equivalents are intended to cover such forms or modifications as would fall within the scope and spirit of certain of the inventions disclosed herein.
This application is a continuation of and claims priority to application Ser. No. 14/077,127 filed Nov. 11, 2013, the entirety of which is hereby incorporated by reference. This application is also related to the following applications, each of which is hereby incorporated by reference in its entirety: U.S. patent application Ser. No. 14/076,718 filed Nov. 11, 2013, entitled “VIDEO ENCODING BASED ON AREAS OF INTEREST”; U.S. patent application Ser. No. 14/076,821 filed Nov. 11, 2013, entitled “ADAPTIVE SCENE COMPLEXITY BASED ON SERVICE QUALITY”; U.S. patent application Ser. No. 14/077,136 filed Nov. 11, 2013, entitled “IMAGE COMPOSITION BASED ON REMOTE OBJECT DATA”; U.S. patent application Ser. No. 14/077,165 filed Nov. 11, 2013, entitled “MULTIPLE PARALLEL GRAPHICS PROCESSING UNITS”; U.S. patent application Ser. No. 14/077,084 filed Nov. 11, 2013, entitled “ADAPTIVE CONTENT TRANSMISSION”; U.S. patent application Ser. No. 14/077,180 filed Nov. 11, 2013, entitled “VIEW GENERATION BASED ON SHARED STATE”; U.S. patent application Ser. No. 14/077,186 filed Nov. 11, 2013, entitled “MULTIPLE STREAM CONTENT PRESENTATION”; U.S. patent application Ser. No. 14/077,149 filed Nov. 11, 2013, entitled “DATA COLLECTION FOR MULTIPLE VIEW GENERATION”; U.S. patent application Ser. No. 14/077,142 filed Nov. 11, 2013, entitled “STREAMING GAME SERVER VIDEO RECORDER”; U.S. patent application Ser. No. 14/076,815 filed Nov. 11, 2013, entitled “LOCATION OF ACTOR RESOURCES”; U.S. patent application Ser. No. 14/077,146 filed Nov. 11, 2013, entitled “SESSION IDLE OPTIMIZATION FOR STREAMING SERVER”; U.S. patent application Ser. No. 14/077,023 filed Nov. 11, 2013, entitled “APPLICATION STREAMING SERVICE”; U.S. Patent Application No. 61/902,740 filed Nov. 11, 2013, entitled “EFFICIENT BANDWIDTH ESTIMATION”.
| Number | Name | Date | Kind |
|---|---|---|---|
| 4090221 | Connor | May 1978 | A |
| 5390188 | Dawson | Feb 1995 | A |
| 5475817 | Waldo et al. | Dec 1995 | A |
| 5774668 | Choquier et al. | Jun 1998 | A |
| 5956489 | San Andreas et al. | Sep 1999 | A |
| 6343313 | Salesky et al. | Jan 2002 | B1 |
| 6351467 | Dillon | Feb 2002 | B1 |
| 6525731 | Suits et al. | Feb 2003 | B1 |
| 6556206 | Benson et al. | Apr 2003 | B1 |
| 6694346 | Aman et al. | Feb 2004 | B1 |
| 6941078 | Onaka | Sep 2005 | B1 |
| 7068729 | Shokrollahi et al. | Jun 2006 | B2 |
| 7177448 | Sah | Feb 2007 | B1 |
| 7249291 | Rasmussen et al. | Jul 2007 | B2 |
| 7447235 | Luby et al. | Nov 2008 | B2 |
| 7522167 | Diard et al. | Apr 2009 | B1 |
| 7548242 | Hughes et al. | Jun 2009 | B1 |
| 7616206 | Danilak | Nov 2009 | B1 |
| 7616207 | Diard et al. | Nov 2009 | B1 |
| 7623131 | Johnson | Nov 2009 | B1 |
| 7660245 | Luby | Feb 2010 | B1 |
| 7711068 | Shokrollahi et al. | May 2010 | B2 |
| 7720174 | Shokrollahi et al. | May 2010 | B2 |
| 7721184 | Luby et al. | May 2010 | B2 |
| 7956772 | Shokrollahi et al. | Jun 2011 | B2 |
| 7969444 | Biermann et al. | Jun 2011 | B1 |
| 7984179 | Huang | Jul 2011 | B1 |
| 8065676 | Sahai et al. | Nov 2011 | B1 |
| 8185809 | Luby et al. | May 2012 | B2 |
| 8190760 | Hurst et al. | May 2012 | B2 |
| 8279755 | Luby | Oct 2012 | B2 |
| 8458567 | Luby et al. | Jun 2013 | B2 |
| 8473557 | Ramakrishnan et al. | Jun 2013 | B2 |
| 8478816 | Parks et al. | Jul 2013 | B2 |
| 8484284 | Elliott et al. | Jul 2013 | B2 |
| 8527646 | Khatib et al. | Sep 2013 | B2 |
| 8572251 | Srinivas | Oct 2013 | B2 |
| 8574074 | van Datta et al. | Nov 2013 | B2 |
| 8582299 | Phillips et al. | Nov 2013 | B1 |
| 8671163 | Luby et al. | Mar 2014 | B2 |
| 8795076 | van Datta et al. | Aug 2014 | B2 |
| 8838722 | Ridges et al. | Sep 2014 | B2 |
| 8928659 | Bar-Zeev et al. | Jan 2015 | B2 |
| 8996864 | Maigne et al. | Mar 2015 | B2 |
| 9170961 | Das et al. | Oct 2015 | B2 |
| 9192859 | Perlman et al. | Nov 2015 | B2 |
| 9338226 | Korangy et al. | May 2016 | B2 |
| 9466074 | van Data et al. | Oct 2016 | B2 |
| 9940647 | Perry et al. | Apr 2018 | B2 |
| 20020039371 | Hedayat et al. | Apr 2002 | A1 |
| 20020114404 | Aizawa et al. | Aug 2002 | A1 |
| 20020129159 | Luby et al. | Sep 2002 | A1 |
| 20020147611 | Greene et al. | Oct 2002 | A1 |
| 20020173984 | Robertson et al. | Nov 2002 | A1 |
| 20020176367 | Gross | Nov 2002 | A1 |
| 20030091000 | Chu et al. | May 2003 | A1 |
| 20030105925 | Yoshimura et al. | Jun 2003 | A1 |
| 20030212742 | Hochmuth et al. | Nov 2003 | A1 |
| 20040057379 | Chen et al. | Mar 2004 | A1 |
| 20040073903 | Melchione et al. | Apr 2004 | A1 |
| 20040093206 | Hardwick | May 2004 | A1 |
| 20040098748 | Bo et al. | May 2004 | A1 |
| 20040101274 | Foisy et al. | May 2004 | A1 |
| 20040153316 | Hardwick | Aug 2004 | A1 |
| 20040236547 | Rappaport et al. | Nov 2004 | A1 |
| 20050135305 | Wentink | Jun 2005 | A1 |
| 20060036756 | Driemeyer et al. | Feb 2006 | A1 |
| 20060061651 | Tetterington | Mar 2006 | A1 |
| 20060072831 | Pallister | Apr 2006 | A1 |
| 20060088093 | Lakaniemi et al. | Apr 2006 | A1 |
| 20060092847 | Mohan | May 2006 | A1 |
| 20060168147 | Inoue et al. | Jul 2006 | A1 |
| 20060285489 | Francisco et al. | Dec 2006 | A1 |
| 20070009043 | Craig et al. | Jan 2007 | A1 |
| 20070053692 | Hoshida et al. | Mar 2007 | A1 |
| 20070094094 | Yaron et al. | Apr 2007 | A1 |
| 20070140359 | Ehret et al. | Jun 2007 | A1 |
| 20070156725 | Ehret et al. | Jul 2007 | A1 |
| 20070183493 | Kimpe | Aug 2007 | A1 |
| 20070216776 | Woolfe | Sep 2007 | A1 |
| 20070226364 | Landspurg | Sep 2007 | A1 |
| 20070240160 | Paterson-Jones et al. | Oct 2007 | A1 |
| 20070265073 | Novi et al. | Nov 2007 | A1 |
| 20080008093 | Wang et al. | Jan 2008 | A1 |
| 20080104608 | Hyser et al. | May 2008 | A1 |
| 20080172140 | Kim et al. | Jul 2008 | A1 |
| 20080281793 | Mathur | Nov 2008 | A1 |
| 20090081964 | Buznach | Mar 2009 | A1 |
| 20090124387 | Perlman et al. | May 2009 | A1 |
| 20090131177 | Pearce | May 2009 | A1 |
| 20090150750 | Liu et al. | Jun 2009 | A1 |
| 20090195537 | Qiu | Aug 2009 | A1 |
| 20090216364 | Grissom | Aug 2009 | A1 |
| 20090245426 | Ratnakar et al. | Oct 2009 | A1 |
| 20090249440 | Platt et al. | Oct 2009 | A1 |
| 20090251488 | Clavel | Oct 2009 | A1 |
| 20090307565 | Luby et al. | Dec 2009 | A1 |
| 20100017686 | Luby et al. | Jan 2010 | A1 |
| 20100063992 | Ma et al. | Mar 2010 | A1 |
| 20100064102 | Brown et al. | Mar 2010 | A1 |
| 20100088453 | Solki et al. | Apr 2010 | A1 |
| 20100091669 | Liu et al. | Apr 2010 | A1 |
| 20100110634 | Woodbury, II et al. | May 2010 | A1 |
| 20100156892 | Chan et al. | Jun 2010 | A1 |
| 20100166058 | Perlman et al. | Jul 2010 | A1 |
| 20100166063 | Perlman et al. | Jul 2010 | A1 |
| 20100166065 | Perlman et al. | Jul 2010 | A1 |
| 20100166068 | Perlman et al. | Jul 2010 | A1 |
| 20100214940 | Macauley | Aug 2010 | A1 |
| 20100218014 | Bozek et al. | Aug 2010 | A1 |
| 20100289803 | Klosowski | Nov 2010 | A1 |
| 20100312891 | Pairault et al. | Dec 2010 | A1 |
| 20110002377 | Raveendran | Jan 2011 | A1 |
| 20110002378 | Raveendran | Jan 2011 | A1 |
| 20110002379 | Raveendran | Jan 2011 | A1 |
| 20110002399 | Raveendran | Jan 2011 | A1 |
| 20110002405 | Raveendran | Jan 2011 | A1 |
| 20110040396 | Kraemer et al. | Feb 2011 | A1 |
| 20110040894 | Shrum et al. | Feb 2011 | A1 |
| 20110047263 | Martins et al. | Feb 2011 | A1 |
| 20110055372 | Elyashev et al. | Mar 2011 | A1 |
| 20110055602 | Kamay et al. | Mar 2011 | A1 |
| 20110067072 | Parekh et al. | Mar 2011 | A1 |
| 20110125746 | Leary et al. | May 2011 | A1 |
| 20110126110 | Vilke et al. | May 2011 | A1 |
| 20110134111 | Stone | Jun 2011 | A1 |
| 20110149751 | Li et al. | Jun 2011 | A1 |
| 20110157193 | Boucher et al. | Jun 2011 | A1 |
| 20110252181 | Ouye et al. | Oct 2011 | A1 |
| 20110252356 | Morris | Oct 2011 | A1 |
| 20110276689 | Rosen | Nov 2011 | A1 |
| 20110283045 | Krishnan et al. | Nov 2011 | A1 |
| 20110304634 | Urbach | Dec 2011 | A1 |
| 20110317568 | Flinta et al. | Dec 2011 | A1 |
| 20120004040 | Pereira et al. | Jan 2012 | A1 |
| 20120004041 | Pereira et al. | Jan 2012 | A1 |
| 20120004042 | Perry et al. | Jan 2012 | A1 |
| 20120005316 | Perry et al. | Jan 2012 | A1 |
| 20120069036 | Dharmapurikar | Mar 2012 | A1 |
| 20120084774 | Post et al. | Apr 2012 | A1 |
| 20120089980 | Sharp et al. | Apr 2012 | A1 |
| 20120093252 | Strait | Apr 2012 | A1 |
| 20120117145 | Clift et al. | May 2012 | A1 |
| 20120141089 | Hunt | Jun 2012 | A1 |
| 20120154389 | Bohan | Jun 2012 | A1 |
| 20120173715 | Selitser et al. | Jul 2012 | A1 |
| 20120182387 | Enenkl et al. | Jul 2012 | A1 |
| 20120188341 | Klien et al. | Jul 2012 | A1 |
| 20120192031 | Liu et al. | Jul 2012 | A1 |
| 20120206572 | Russell | Aug 2012 | A1 |
| 20120209933 | Ridges et al. | Aug 2012 | A1 |
| 20120213272 | Liu et al. | Aug 2012 | A1 |
| 20120224490 | Ikada | Sep 2012 | A1 |
| 20120227058 | Hunt | Sep 2012 | A1 |
| 20120268553 | Talukder | Oct 2012 | A1 |
| 20120281715 | Shojania et al. | Nov 2012 | A1 |
| 20120281962 | Hunt et al. | Nov 2012 | A1 |
| 20120311448 | Achour et al. | Dec 2012 | A1 |
| 20120331147 | Dutta et al. | Dec 2012 | A1 |
| 20130007499 | Moy | Jan 2013 | A1 |
| 20130016107 | Dharmapurikar | Jan 2013 | A1 |
| 20130024812 | Reeves et al. | Jan 2013 | A1 |
| 20130031161 | Yang | Jan 2013 | A1 |
| 20130036476 | Roever et al. | Feb 2013 | A1 |
| 20130038618 | Urbach | Feb 2013 | A1 |
| 20130067469 | Das et al. | Mar 2013 | A1 |
| 20130069946 | Venon et al. | Mar 2013 | A1 |
| 20130106855 | Urbach | May 2013 | A1 |
| 20130151683 | Jain et al. | Jun 2013 | A1 |
| 20130151803 | Tofano | Jun 2013 | A1 |
| 20130210522 | Dharmapurikar | Aug 2013 | A1 |
| 20130212477 | Averbuch | Aug 2013 | A1 |
| 20130227710 | Barak et al. | Aug 2013 | A1 |
| 20130263131 | Beda, III et al. | Oct 2013 | A1 |
| 20130281791 | Aferzon | Oct 2013 | A1 |
| 20130282791 | Kruglick | Oct 2013 | A1 |
| 20130306269 | Helbig et al. | Nov 2013 | A1 |
| 20130307847 | Dey | Nov 2013 | A1 |
| 20130322517 | Zurpal et al. | Dec 2013 | A1 |
| 20130326024 | Chen et al. | Dec 2013 | A1 |
| 20130344960 | Perry et al. | Dec 2013 | A1 |
| 20130344961 | Iannetta | Dec 2013 | A1 |
| 20130344966 | Mustafa | Dec 2013 | A1 |
| 20130346899 | Cole et al. | Dec 2013 | A1 |
| 20140025710 | Sarto | Jan 2014 | A1 |
| 20140108967 | Markham et al. | Apr 2014 | A1 |
| 20140143301 | Watson et al. | May 2014 | A1 |
| 20140171186 | Arnone et al. | Jun 2014 | A1 |
| 20140173060 | Jubran et al. | Jun 2014 | A1 |
| 20140173674 | Wolman et al. | Jun 2014 | A1 |
| 20140176536 | Kern et al. | Jun 2014 | A1 |
| 20140176583 | Abiezzi et al. | Jun 2014 | A1 |
| 20140213191 | Courtice | Jul 2014 | A1 |
| 20140216688 | Shelnutt et al. | Aug 2014 | A1 |
| 20140267283 | Nystad et al. | Sep 2014 | A1 |
| 20140279581 | Devereaux | Sep 2014 | A1 |
| 20140297798 | Bakalash et al. | Oct 2014 | A1 |
| 20140337835 | Johnson | Nov 2014 | A1 |
| 20140364228 | Rimon | Dec 2014 | A1 |
| 20140379775 | Korangy et al. | Dec 2014 | A1 |
| 20150019965 | Roberts et al. | Jan 2015 | A1 |
| 20150084981 | Clarberg | Mar 2015 | A1 |
| 20150091903 | Costello et al. | Apr 2015 | A1 |
| 20150130789 | Heinz et al. | May 2015 | A1 |
| 20150130813 | Taraki et al. | May 2015 | A1 |
| 20150130814 | Taraki et al. | May 2015 | A1 |
| 20150130815 | Taraki et al. | May 2015 | A1 |
| 20150131969 | Taraki et al. | May 2015 | A1 |
| 20150133214 | Heath et al. | May 2015 | A1 |
| 20150133216 | Heinz et al. | May 2015 | A1 |
| 20150134770 | Heinz et al. | May 2015 | A1 |
| 20150134771 | Kalman et al. | May 2015 | A1 |
| 20150134772 | Heinz et al. | May 2015 | A1 |
| 20150134779 | Thompson et al. | May 2015 | A1 |
| 20150134840 | Thompson et al. | May 2015 | A1 |
| 20150142925 | Hix et al. | May 2015 | A1 |
| 20150187333 | Loeffler et al. | Jul 2015 | A1 |
| 20150200983 | Pearce et al. | Jul 2015 | A1 |
| 20150249623 | Phillips et al. | Sep 2015 | A1 |
| 20150281258 | Watkins et al. | Oct 2015 | A1 |
| 20150331813 | Perrin et al. | Nov 2015 | A1 |
| 20160012465 | Sharp et al. | Jan 2016 | A1 |
| 20160044125 | Hardin et al. | Feb 2016 | A1 |
| 20160073286 | Wang et al. | Mar 2016 | A1 |
| 20170300368 | Jain et al. | Oct 2017 | A1 |
| Number | Date | Country |
|---|---|---|
| 1154169 | Jul 1997 | CN |
| 101025901 | Aug 2007 | CN |
| 102867073 | Jan 2013 | CN |
| 2073486 | Jun 2009 | EP |
| 1290642 | Sep 2010 | EP |
| 2000-132705 | May 2000 | JP |
| 2004-266575 | Sep 2004 | JP |
| 2006-061716 | Mar 2006 | JP |
| 2006-279544 | Oct 2006 | JP |
| 2007-011494 | Jan 2007 | JP |
| 2009-003584 | Jan 2009 | JP |
| 2010-176662 | Aug 2010 | JP |
| 2010-224914 | Oct 2010 | JP |
| 2011-511367 | Apr 2011 | JP |
| 2012-069011 | Apr 2012 | JP |
| 2012-143012 | Jul 2012 | JP |
| 2012-245347 | Dec 2012 | JP |
| WO 2012079825 | Jun 2012 | WO |
| WO 2012107739 | Aug 2012 | WO |
| WO 2013069654 | May 2013 | WO |
| Entry |
|---|
| U.S. Appl. No. 14/317,441, filed Jun. 27, 2014, Kalman et al. |
| http://commons.wikimedia.org/wiki/File:Stereoscopic_3D_render_from_Blender.jpg;_File:Stereoscopic_3D_render_from_Blender.jpg; Wikimedia; Mar. 4, 2012; accessed Nov. 12, 2013; 2 pages. |
| http://en.wikipedia.org/wiki/Mipmap; Mipmap; Wikipedia; Oct. 29, 2013; accessed Nov. 10, 2013; 3 pages. |
| http://en.wikipeida.org/wiki/Texture__atlas; Texture atlas; Wikipedia; Sep. 28, 2013; accessed Nov. 10, 2013; 2 pages. |
| http://msdn.microsoft.com/en-us/library/windows/hardware/ff569022(v=vs.85).aspx; Pipelines for Direct3D Version 11; Microsoft; Oct. 14, 2013; accessed Nov. 9, 2013; 3 pages. |
| Hasselgren et al.; “An Efficient Multi-View Rasterization Architecture”; Proceedings of the 17th Eurographics Conf. on Rendering Techniques; Jun. 2006; 12 pages. |
| Sorbier et al.; “GPU-based multi-view rendering”; 3rd Annual Int'l Conf. on Computer Games, Multimedia and Allied Technology (CGAT); 2010; 8 pages. |
| Hubner et al.; “Single-Pass Multi-View Rendering”; IADIS Int'l Journal on Computer Science and Information Systems; vol. 2 No. 2; 2007; p. 122-140. |
| European Patent Application No. 14860984.5; Extended Search Report; dated May 26, 2017; 14 pages. |
| European Patent Application No. 14860565.2; Extended Search Report; dated May 10, 2017; 9 pages. |
| “Remote Session Environment”;https://technet.microsoft.com/de-de/library/ee791847(v=ws.10).aspx; Microsoft; ® 2017; accessed Jul. 18, 2017; 8 pages. |
| European Patent Application No. 14859905.3; Extended Search Report; dated Jun. 27, 2017; 10 pages. |
| Ekelin et al.; “Real-Time Measurement of End-to-End Available Bandwidth using Kalman Filtering”; Proc. 10th IEEE/IFIP Network Operations and Management Symposium; 2006; 12 pages. |
| Number | Date | Country | |
|---|---|---|---|
| 20170151496 A1 | Jun 2017 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 14077127 | Nov 2013 | US |
| Child | 15430704 | US |
