Patent Application
20230381665
The gaming industry makes up a large portion of the technology sector. As the technology looks towards a future that is more and more connected, gaming will be a central component of the connected future. Artificial intelligence has always been a major component of the gaming industry in general, however, as AI developments continue to advance, it becomes more difficult for the gaming industries to incorporate the advances into games. For example, it is difficult for game development studios to incubate AI as the majority of advanced AI systems require a lot of custom domain expertise.
It is with respect to these and other general considerations that the aspects disclosed herein have been made. In addition, although relatively specific problems may be discussed, it should be understood that the examples should not be limited to solving the specific problems identified in the background or elsewhere in this disclosure.
Aspects of the present disclosure relate to a personalized agent service that generates and evolves customized agents that can be instantiated in-game to play with users. Machine learning models are trained to control the agent's interactions with the game environment and the user during gameplay. As the user continues to play with the agent, the one or more machine learning models develop gameplay styles for the agent that complement the user's preferred playstyle, incorporate the user's preferred strategies, and is generally customized for interaction with the user. The agent personalization data generated during gameplay is stored by the service, thereby allowing the user to import the agent in different games, thereby creating a constant gameplay companion that the user can play with across a variety of different games without requiring the different games to include technology capable of developing AI controlled agents.
In further aspects, an application programming interface is provided by the personalized agent service. Using the API, games can import agent personalization data in order to customize in-game non-player characters (NPCs), thereby customizing the in-game NPCs in accordance with the user's preferences. The imported agent personalization data can be modified in order to comply with game or NPC constraints, thereby allowing for the customization of the NPC in a way that does not break the game or the NPCs role in the game's story.
This Summary is provided to introduce a selection of concepts in a simplified form, which is further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Additional aspects, features, and/or advantages of examples will be set forth in part in the following description and, in part, will be apparent from the description, or may be learned by practice of the disclosure.
Non-limiting and non-exhaustive examples are described with reference to the following figures.
Various aspects of the disclosure are described more fully below with reference to the accompanying drawings, which from a part hereof, and which show specific example aspects. However, different aspects of the disclosure may be implemented in many different ways and should not be construed as limited to the aspects set forth herein; rather, these aspects are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the aspects to those skilled in the art. Practicing aspects may be as methods, systems, or devices. Accordingly, aspects may take the form of a hardware implementation, an entirely software implementation or an implementation combining software and hardware aspects. The following detailed description is, therefore, not to be taken in a limiting sense.
Aspects of the present disclosure provide systems and methods which utilizes machine learning techniques to provide a personalized agent or bot that can be used in gaming or other types of environments. Among other examples, aspects disclosed herein relate to: using reinforcement learning agents that can be trained via computer vision without relying upon deep game hooks; training agents using models (e.g., foundation models, language models, computer vision models, speech models, video models, audio models, multimodal machine learning models, etc.) such that the agents are operable to understand written and spoken text, have a defined body of knowledge, and may write its own code; providing agents that interpret state information using computer vision and audio cues, providing agents that receive user instructions based upon computer vision and audio cues, and provide a system in which user feedback is used to improve the future interactions with agents across different games and applications.
In examples, a generative multimodal machine learning model processes user input and generates multimodal output. For example, a conversational agent according to aspects described herein may receive user input, such that the user input may be processed using the generative multimodal machine learning model to generate multimodal output. The multimodal output may comprise natural language output and/or programmatic output, among other examples. The multimodal output may be processed and used to affect the state of an associated application. For example, at least a part of the multimodal output may be executed or may be used to call an application programming interface (API) of the application. A generative multimodal machine learning model (also generally referred to herein as a multimodal machine learning model) used according to aspects described herein may be a generative transformer model, in some examples. In some instances, explicit and/or implicit feedback may be processed to improve the performance of multimodal machine learning model.
In examples, user input and/or model output is multimodal, which, as used herein, may comprise one or more types of content. Example content includes, but is not limited to, spoken or written language (which may also be referred to herein as “natural language output”), code (which may also be referred to herein as “programmatic output”), images, video, audio, gestures, visual features, intonation, contour features, poses, styles, fonts, and/or transitions, among other examples. Thus, as compared to a machine learning model that processes natural language input and generates natural language output, aspects of the present disclosure may process input and generate output having any of a variety of content types.
In doing so, the systems and methods disclosed herein support personalized agents that learn who the user (also referred to herein interchangeably as the “player”) is, how the user speaks, what the user strategies are, and how the user plays. It retains a “memory” of past interactions with the user and can act as a constant companion to the user as the user engages in different experiences, such as playing different games, playing different game modes, etc. In order to do so, aspects disclosed herein are operable to store meta classifications (e.g., sentiment analysis, intent analysis, etc.) associated with the personal data about the user, provided the user has given permission to do so.
In yet another aspect, a system is provided in which an application programming interface (API) can be used to access the personalized agents. Such an API allows game studios to create non-player character (NPC) companions that are designed to optionally be “possessed” by a player's personalized agent. As used herein, an NPC “possessed” by the player's personalized agent means the NPC takes on characteristics of the personalized agent. Said characteristics may comprise: how the agent communicates with the player (e.g., preferred communication style, how the agent refers to the player), how the agent interacts with the player (e.g., the agent employs a playstyle that matches the player's preferred strategies based upon the player's past experience in games), personality characteristics of the agent, agent playstyle characteristics, and the like. The game may provide the personalized agent specific permissions to access game state, bodies of knowledge and other game information via a game API. The game may also be able to limit the personalized agent from performing specific tasks or to tune down the “strength” of the agent based on game design.
In example, one or more agents from the personalized agent library 107 interact with the game via an instantiated agent(s) 108 based upon text communications, voice commands, and/or player actions received from the user device 102. That is, system 100 supports interactions with agents as if they were other human players, or alternatively, as a player would normally interact with a conventional in-game NPC. In doing so, one or more agents hosted by the agent library 108 are operable to interact with different games that the user plays without requiring changes to the game. That is, system 100 is operable to work with games without requiring the games to be specifically developed to support the agents (e.g., the agents do not require API access, games do not have to be developed with specific code to interact with the agents, etc.). In doing so, system 100 provides a scalable solution which allows users to play with customized agents across a variety of different games. That is, game state is communicated between one or more instantiated agents 108 a game via game service 106 using audio and visual interactions and/or an exposed API. In doing so, the instantiated agent(s) 108 are able to interact with the game in the same manner as a user (e.g., by interpreting video, audio, and/or haptic feedback from the game) and/or in the same manner as an in-game NPC (e.g., via the exposed API). Similarly, the one or more agents are capable of interacting with the user playing the game using gaming device 102 as another player would or in a manner similar to an NPC interacting with the user. That is, the one or more agents are capable of receiving visual, textual, or voice input from the user playing the game via game service 106 and/or game information (e.g., current game state, NPC inventory, NPC abilities, etc.) via an exposed API. As such, the system 100 allows the user playing the game via the user device 102 to interact with the one or more agents as they would interact with any other player on NPC, however, the instantiated agent(s) 108 may be personalized to the user based upon past interactions with the user that occurred both in the current game and in other games. In order to facilitate this type of user interaction, the one or more instantiated agents 108 may employ computer vision, speech recognition, or other known techniques when processing interactions with the user in order to interpret commands received from the user and/or generate a response action based upon the user's actions.
For example, consider a user playing a first-person shooter with a personalized agent. The user may say “cover the right side.” A speech recognition model that may be one of the models 118 may be utilized to interpret the audio received from the user to a modality understood by the agent. In response, the agent 108 may take a position on the right side of the user or otherwise perform an action to cover the right side of the user in response to the user's audio instruction. As yet another example, consider an action role playing game in which two objectives must simultaneously be defended. An agent playing in conjunction with the user may employ computer vision to analyze a view current view of the game to determine that two objectives exist (e.g., identifying two objectives on a map, interpreting a displayed quest log indicating that two objectives must be defended, etc.). Similarly, computer vision may be used to identify the user's player character and determine the player character is heading to the first objective. Based upon the feedback from the computer vision model, the agent can instruct its representative character to move towards and defend the second objective.
From these examples, one of skill in the art will appreciate that, by employing speech recognition, computer vision techniques, and the like, the one or more agents from the agent library will be able to interact with the user in a manner similar to how other users would interact during cooperative gameplay. In doing so, one or more agents from the agent library can be generated separate from a specific game as the instantiated agent(s) 108 may not have to have API or programmatic access to the game in order to interact with the user. Alternatively, the instantiated agent(s) 108 may have API access or programmatic access to the game, such as, when the instantiated agent is “possessing” and in-game NPC. In said circumstances, the agent may interact with the game state and user based via the API access. Because system 100 provides a solution in which agents can be implemented separate from the individual games, the one or more agents in the agent library can be personalized to interact with a specific user. This personalization can be carried across different games. That is, over time, the agent learns details about the user, such as the user's likes and dislikes, the user's playstyles, the user's communication patterns, user's preferred strategies, etc., and be able to accommodate the user accordingly across different games.
The agent personalization may be generated, and updated over time, via the feedback collection engine 110. Feedback collection engine 110 receives feedback from the user and/or from the instantiated agent(s) 108 that are performed in-game. The feedback collected can include information related to the user's playstyle, user communication, user interaction with the game, user interaction with other players, user interaction with other agents, outcomes of the instantiated agent(s) 108 actions performed in game, interactions between the player and the instantiated agent(s) 108 actions in game or any type of information generated by user device 102 as a user plays a game. In order to comply with user privacy considerations, information may only be collected by the feedback collection engine 110 upon receiving permission from the user to do so. The user may opt in or out of said collection at any time. The data collected may be implicit data, e.g., data based upon the user's normal interactions with the game, or explicit data, such as specific commands provided by the user to the system. An example of a specific command may be the user instructing an agent to address the user by a specific character name. Data collected by the feedback collection engine 110 may be provided to a prompt generator 112.
The prompt generator 112 may use data collected by the feedback collection engine 110 to generate prompts used to personalize the one or more agents of the agent library 108. That is, the prompt generator 112 interprets the collected feedback data to generate instructions that can be executed by the one or more agents to perform actions by the agent. The prompt generator is operable to generate new prompts or instructions based upon the collected feedback or alter existing prompts based upon newly collected feedback. For example, if a user initially plays a game as a front-line attacker, prompt generator 112 may generate instructions that cause the instantiated agent(s) 108 to implement a supporting playstyle, such as being a ranged attacker or a support character. If the user transitions playstyle to one of a ranged damage dealer, the prompt generator 112 can identify this change via the feedback data and adjust the instantiated agent(s) to adjust to the player's new style (e.g., switch to a front-line attacker to pull aggro or enmity from the player character). Instructions generated by the prompt generator are provided to the cloud service 104 to be stored by the cloud service 104 as part of the agent library 108, thereby storing meta classifications (e.g., sentiment analysis, intent analysis, etc.) associated with a specific user. In doing so, the instructions generated based upon user playstyle or preference in a first game can be incorporated by the agent in not only the first game, but other games that the user plays. That is, using instructions generated by the prompt generator 112, the cloud service 104 can instantiate agents that across a variety of different games that are already personalized for a specific user based upon the user's prior interactions with an instantiated agent(s) 108, regardless of whether the user is playing the same game or a different game. While aspects described herein describe a separate prompt generator 112 as generating commands to control the instantiated agent(s) 108, in alternate aspects, the commands may be generated directly by the one or more machine learning models employed by the agent library 107, or via a combination of various different components disclosed herein.
In some scenarios, however, there may be issues encountered during the training process for the various machine learning models that may be leveraged to generate a personalized agent. For example, there could be a failed training session due to data or processing errors. In yet another example, the training process may fail due to user error. For example, a user picking up a new game may play the game incorrectly at first. The user's incorrect actions or experiences might train the personalized agent to play in a manner that negatively affects gameplay. As such, system 100 may include processes to rollback or reset the one or more machine learning models (or any of the other components disclosed herein) in order to correct errors that may occur while training the one or more machine learning models or any other errors that may occur in general as the personalized agent is developed. For example, the system 100 may periodically maintain a snapshot of the different machine learning models (or other components) that save the state of the component at the time of the snapshot. This allows the system 100 to rollback all the components, a subset of components, or specific components in response to detecting training errors in the future. A plurality of different snapshots can be store representing states of the personalized agent as it develops overtime, thereby providing the system 100 (or the user) options to determine an appropriate state for rollback upon encountering an error.
The personalized agent library 107 may also include a fine-tuning engine 114 and one or more models 116. The fine-tuning engine 114 and models 116 are operable to interact with the user actions via the user device 102, the instantiated agent(s) 108, and game training model(s) 124 in order to process the feedback data received form the various sources. Any number of different models may be employed individually or in combination for use with the examples disclosed herein. For example, foundation models, language models, computer vision models, speech models, video models, and/or audio models may be employed by the system 100. As used herein, a foundation model is a model trained on broad data that can be adapted to a wide range of tasks (e.g., models capable of processing various different tasks or modalities).
The one or more models 116 may process video, audio, and/or textual data received from the user or generated by the game during gameplay in order to interpret user commands and/or derive user intent based upon the user communications or in-game actions. The output from the one or more models 116 is provided to the fine-tuning model 114 which can use the output to modify the prompts generated by prompt generator 112 in order to further personalize the instructions generated for the user based upon the agent's past interactions with the user. The Personalized agent library 107 may also include a game data (lore) 122 component which is operable to store information about various different games. The game data (lore) 122 component stores information about a game, the game's lore, story elements, available abilities and/or items, etc. that the agent can be utilized by the other components (e.g., models 118, fine-tuning engine 114, prompt generator 112, etc.) to generate instructions to control the instantiated agent(s) 108 in accordance with the game's themes, story, requirements, etc.
The various components described herein may be leveraged to interpret current game states based upon data received from the game (e.g., visual data, audio data, haptic feedback data, data exposed by the game through APIs, etc.). Further, the components disclosed herein are operable to generate instructions to control the personalized agent's actions in game based upon the current game state. For example, the components disclosed herein may be operable to generate instructions to control the personalized agent's interaction with the game in a similar manner as a human would interact with the game (e.g., by providing specific controller instructions, keyboard instructions, or any other type of interfaces with supported gameplay controls). Alternatively, or additionally, the various components may generate code which controls how the personalized agent interacts in the game. For example, the code may be executed by the personalized agent which causes the personalized agent to perform specific actions within the game.
Personalized agent library 107 may also include an agent memory component 120. The agent memory component can be used to store personalized data generated by the various other components described herein as well as playstyles, techniques, and interactions learned by the agent via past interactions with the user. The agent memory 120 may provide additional inputs to the prompt generator 112 that can be used to determine the instantiated agent(s) actions during gameplay.
To this point, the described components of system 100 have focused on the creation of personalized agents, the continued evolution of personalized agents via continued interaction with a user, and the instantiation of personalized agents in a user's gaming session. While personalization of the agents for specific user provides many gameplay benefits, the instantiated agents also require and understanding of how to interact with and play the game. In one aspect, the components of the personalized agent library 107 are operable to learn and refine the agent's gameplay based upon sessions with the user. However, when the user plays a new game, the time required for the agent to learn gameplay mechanics in order to be a useful companion may not be feasible through gameplay with the user alone. In order to address this issue, system 100 also includes a gameplay training service 124 which includes a game library 126 and gameplay machine learning models 128. In aspects, the game library includes any number of games that are supported by the cloud service 104 and/or the user device 102. The gameplay training service 124 is operable to execute sessions for the various games stored in game library 126 and instantiate agents within the executed game. The gameplay machine learning models 128 are operable to receive data from the executed game and the agent's actions performed in the game as input and, in response, and generate control signals to direct the gameplay of the agent within the game. Through use of reinforcement learning, the gameplay machine learning models are operable to develop an understanding of gameplay mechanics for both specific games and genres of games. In doing so, the gameplay training service 124 provides a mechanism in which agents can be trained to play specific games, or specific types of games, without requiring user interaction. The personalized agent library 107 is operable to receive trained models (or interact with the trained models) from the gameplay training service 124, which may be stored as part of the agent memory, and employ those models with the other personalization components to control the instantiated agent(s) 108. In doing so, the user experience interacting with the agent in-game is greatly enhanced, as the user would not be required to invest the time to train the agent in the game's specific mechanics. Additionally, by importing or interacting with the trained gameplay machine learning models 128 provided by the gameplay training service 124, personalized agent library 107 is able to employ trained instantiated agent(s) 108 to play with the user the first time a user boots up a new game.
One of skill in the art will appreciate that the system 100 provides a personalized agent, or artificial intelligence, that is operable to learn a player's identity, learn a player communication style or tendencies, learn the strategies that are employed and used by a player in various different games and scenarios, learn gameplay mechanics for specific games and game genre's etc. Further, the one or more agents generated by system 100 can be stored as part of a cloud service which allows the system to retain a “memory” of a user's past interactions, thereby allowing the system to generate agents that act as a consistent user companion across different games without requiring games to be designed specifically to support such agents.
For example, agent persona engine 160 may modify or adjust a prompt (or action determined by a prompt) in accordance with the personalization information associated with the agent. For example, the user may employ an agent with a preferred personality. Agent persona engine 160 may modify the prompt or response to generated by the prompt in accordance with the agent's personality. User intent or goals engine may modify the prompt (or action determined by the prompt) based upon the user's current goal or an intent behind the user's action or request. The user's goals or intent may change over time, may be based upon a specified user goal, or may be determined based upon the user's actions. Game lore or constraints engine 164 may modify or adjust a prompt (or action determined by a prompt) in accordance with characteristics of the game. For example, the agent may be “possessing” an in-game non-player character (as discussed in further detail below). The game lore or constraints engine 164 may modify the prompt based upon the NPCs personality or limitations. The various engines of the helper service may be employed individually or in combination when modifying or adjusting the prompts. The adjusted prompt is then provided to the one or more associated agents 156 for execution.
At operation 206, the current game state is interpreted through audio and visual data and/or through API access granted to the agent by the game. As previously noted, certain aspects of the disclosure provide for the generation of agents that can interact with a game without requiring API or programmatic access to the game. As such, the instantiated agent interacts with the game in the same way a user would, that is, through audio and visual data associated with the game. Alternatively, the agent may be granted API access to the game, for example, when the agent is possessing an NPC, in order to interact with the game. At operation 206, various speech recognition, computer vision, object detection, OCR processes, or the like, may be employed to process communications received from the player (e.g., spoken commands, text-based commands) and game state through the current displayed view (e.g., using computer vision) or via an API to interpret game state. The current game state is then used to generate agent actions at operation 208. For example, an agent action may be performed based upon a spoken command received from the user. Alternatively, an agent command may be generated based upon the current view. For example, if an enemy appears on screen, computer vision and/or object detection may be used to identify the enemy, and a command for the agent to attack the enemy may be generated at operation 208. Although not shown, operations 206 and 208 may be continually performed while the gaming session is active.
Flow continues to operation 210 where user feedback is received. The user feedback received may be explicit. For example, the user may issue a specific command to an agent to perform an action or to change the action they are currently performing. Alternatively, or additionally, user feedback may be implicit. Implicit user feedback may be feedback data that is generated based upon user interactions with the game. For example, the user may not explicitly provide a command to an agent, rather, the user may adjust their actions or playstyle based upon the current game state and/or in response to an action performed by the agent. In examples, user feedback may be collected continually during the gaming session. The collected feedback may be associated with concurrent game states or agent actions.
Upon collecting the user feedback, flow continues to operation 212 where prompts are generated for the one or more agents based upon the user feedback. In examples, the generated prompts are instructions to perform agent actions in response to the state of the game or specific user interactions. The prompts may be generated using one or more machine learning models which receives the user feedback, and/or actions performed by the one or more agents, and/or existing prompts, and/or state data. The output of the machine learning model may be used to generate one or more prompts. In examples, the machine learning model may be trained using information related to the user such that the output from the machine learning model is personalized for the user. Alternatively, or additionally, the machine learning model may be trained for a specific game or application, for a specific group of users (e.g., an e-sports team), or the like. Multiple machine learning models may be employed at operation 212 to generate the prompts. In still other examples, other processes, such as a rule-based process, may be employed in addition to or instead of the use of machine learning models at operation 212. Further, new prompts may be generated at operation 212 or existing prompts may be modified.
Once the one or more prompts are generated at operation 212, flow continues to operation 214 where the one or more prompts are stored for future use by the one or more agents. For example, the one or more prompts may be stored in an agent library. By storing the prompts generated at 212 with the agent library, the agent will be able to utilize the prompts to interact with the user across different games, thereby providing a personalized agent that a user can play with across different games.
While aspects of the disclosure described so far have related to importing a personalized agent to play with a user, game developers may leverage the agent data to control an in-game non-player character (NPC). In doing so, the NPC may act in accordance with the user's preferred playstyle, may employ techniques to help a user with accessibility requirements, or the like, while still maintaining qualities of the NPC based upon story requirements or NPC in-game abilities. For example, an agent system, such as the system described in
As depicted in
The personalized agent API 306 is operable to receive requests from one or more games, such as game 304, or personalized agent data. In one aspect, the personalized agent API 306 is operable to receive a request for a specific agent. In said scenario, the personalized agent API 306 is operable to instantiate the requested agent as part of the game session. However, in other aspects, the request received from game 304 may not be for an agent, rather, it may be for personalization data from one or more agents stored by the personalized agent service 302. In said scenario, the game 304 may request personalization data to personalize an in-game NPC according to the user preferences. For example, the game may include one or more NPC companions that accompany the player character as part of the game story. In said instances, the game may not allow important a player created agent, however, the game would allow the in-game NPCs to be personalized according to the user's preferences, playstyles, etc. As such, one or more in-game NPCs may be “possessed” by one or more personalized user agents. That is, the in-game NPCs may take on certain characteristics of the personalized agent (e.g., how the agent interacts with the user, agent playstyles, agent roles, etc.), while maintaining some characteristics of the in-game NPC (e.g., NPC's story role, NPC abilities, etc.).
When receiving a request for agent characteristics, the personalized agent API 306 parses the request to determine the parameters of the request. For example, the request may be for specific personalization characteristics (e.g., playstyle, communication behaviors, personality traits, etc.). Alternatively, or additionally, the request may include information about the game, the NPCs abilities, game genre, etc. The request parameters may be provided to the personalization selector 308. The personalization selector 308 analyzes the request parameters and identifies requested agent characteristics based upon the request parameters. For example, the personalization selector may identify request for specific characteristics from the request parameters. Additionally, the personalization selector 306 may infer characteristics based upon request parameters. For example, information about the game, such as the game type or game genre, may be used to identify agent characteristics related to the game or NPC. For example, if the game is a roleplaying game, the personalization selector 306 may identify agent characteristics that are relevant or useful to roleplaying games. Similarly, if the game is a first-person shooter, the personalization selector 306 may identify agent characteristics related to first person shooter games. Similarly, characteristics of the NPC that is to be “possessed” by the agent may be used to infer relevant agent characteristics. For example, if the NPC is a healer, the personalization selector may identify agent characteristics that are relevant to a healer class. Upon identifying the relevant agent characteristics, agent data related to the relevant agent characteristics is retrieved from the personalized agent datastore 310. In examples, the personalized agent data store 306 may store any type of personalization information (e.g., prompts, machine learning models, agent personality traits, etc.) that were previously generated for the agent. The retrieved personalization information may be exposed to the game 304 via the personalized agent API 306.
Upon receiving the personalization information, game 304 analyzed the received personalization information using game constraint analyzer 314. In certain scenarios, application of the agent personalization information to the in-game NPC without modification may lead to game breaking scenarios. For example, the unmodified application of the agent personalization information may make the NPC either overpowered or underpowered, may alter the NPC's personality in a manner that negatively affects the NPC's role in the story, or the like. As such, the game constraint analyzer 314 analyzes the agent personalization information and modifies the agent personalization data in accordance with the game and/or NPC constraints. The modified agent personalization data may then be stored in an NPC profile 316 for the possessed NPC. Actions performed by the NPC in-game may then be directed by the modified agent personalization data stored in the NPC profile. As such, the in-game NPC may be possessed by the personalized agent's traits, thereby customizing the user's interaction with the NPC during gameplay. Game 304 may further include a game API 318 that allows the possessed API to interact with the user and the game. For example, the possessed NPC actions may be generated based upon personalized machine learning models or other personalization data received from the personalized agent service. The game API 318 allows the personalized machine learning models and/or personalization data to receive game state data, user interaction data, user actions, etc. performed in-game and to control interaction of the in-game NPC.
Upon accessing and/or importing the agent data, flow continues to operation 404 where the accessed data is associated with one or more NPCs. For example, the game may include a number of NPCs that interact with the player character. These NPCs may be incorporated as part of the player character's party, for example, as members of a player's party in a role-playing game. Generally, NPCs are developed to have a certain personality based upon the NPCs role, relationship to the player character, and/or story driven characteristics. However, said NPCs are developed in a generic matter, that is, a matter that is developed for the player base as a whole, rather than in a manner that is specific to a player. This is a limitation based upon the lack of access game developers have with respect to individual users. Aspects of the present disclosure, however, do not lack access to user data. As such, the disclosed aspects can be leveraged by the game developers to modify generic NPCs to personalize the generic NPCs to individual players by associating the accessed agent data with the generic NPC at operation 404.
As discussed, the personalized agents are developed through user playstyle and user interactions with the one or more agents across a number of different games. As such, some of the agent actions may not be compatible with specific NPCs. These may be due to NPC limitations, such as the NPCs available skillset, or other considerations, such as story requirements. As such, at operation 406, the imported agent data, may be modified in accordance with the NPC limitations set by the game. For example, if the NPC is set to play a specific role (e.g., a healer, a tank, or a damage dealer in a role-playing game), the imported agent data may be modified to remove actions or characteristics that do not fit the NPC's role. Other considerations may be used to determine how to modify the imported data. For example, as discussed, the agent data is personalized for the user. However, if the NPC is initially introduced as a stranger to the player character, it would be immersion braking if the NPC incorporated knowledge specific to the user. As such, some of the personalization data from the imported agent may be removed or modified based upon the story requirements. In further examples, the modifications made at operation 406 may change as the player progresses through the game. For example, as the NPC and the player character spend more time together, the more of the personalization information from the agent may be incorporated into the NPC behavior. In doing so, a natural progress of the relationship between the NPC and player character may be emulated. Upon making any modifications, flow continues to operation 408 where the NPC is operated in accordance with the modified agent characteristics. In doing so, the generalized NPC is improved to operate, for example, in accordance with a playstyle that fits with the user, thereby providing an improved player experience.
By allowing access to player history, games can further be tailored to provide specific information to a user when the user begins a new game. That is, rather than providing generic tutorials, which are common in many games today, a game with access to the player's history and skills, the game can tailor a tutorial to individual users based upon their knowledge (e.g., omit portions of the tutorial that the user is familiar with based upon their history, focus on information that is new to the user in the tutorial, etc.). That is, tutorials can be customized based upon user experience from other games, a feature which was previously unavailable to individual games because, as noted above, individual games lack access to user history and experience gained from other games.
At operation 504, the request is analyzed to determine agent personalization characteristics relevant to the request. For example, the request parameters may be parsed to identify specific agent characteristics requested by the game. Additionally, or alternatively, game information, or NPC information, included in the request or associated with the game making the request may be analyzed to determine agent personalization characteristics that are relevant to the game or the NPC as discussed above. Upon determining the requested personalization characteristics, relevant agent personalization data is identified. The relevant agent personalization data is retrieved at operation 506 from an agent personalization data store. For example, data stored in the personalization agent library 107 may be retrieved at operation 506, including agent memory, prompts, machine learning models utilized by the agent to interact with the game and/or user, relevant game data or lore, etc. In one example, rather than retrieving the data at operation 506, references may be generated which allow the game to retrieve or communicate with the personalization components, for example, via an API exposed by the agent personalization service. At operation 508, the retrieved agent personalization data, or references to the agent personalization data, is provided to the requesting game.
In response to sending the request, agent personalization data is received at operation 606. In one aspect, the agent personalization data may be received and stored in the game's files. Alternatively, rather than receiving the agent personalization data, references or instructions to access the agent personalization data may be received. The game may use the references or instructions to receive agent personalization data on demand by the game or, alternatively, the game may provide game state information to the agent personalization components of the personalized agent service, such as the one or more machine learning models used to control the agent and receive instructions or data in response to providing the data during gameplay.
After receiving the agent personalization data, the received data is analyzed at operation 608. In examples, the received agent personalization data is analyzed against game and/or NPC constraints. As previously discussed, importing the agent personalization information to the in-game NPC without modification may lead to game breaking scenarios. For example, the unmodified application of the agent personalization information may make the NPC either overpowered or underpowered, may alter the NPC's personality in a manner that negatively affects the NPC's role in the story, or the like. As such, the agent personalization data is analyzed against the NPC's constraints (e.g., the NPC's available abilities, the NPC's power level, the NPC's personality constraints relative to the NPC's role in the story, the NPC's party role, game constraints such as level or power limitation, or the like). Based upon the analysis performed at operation 608, the agent personalization data is modified at operation 610. For example, agent personalization data may be modified in accordance with game or NPC limitations. The modification may include setting limitations on the abilities or power levels provided by the agent personalization data, to modify or remove certain agent personality techniques that conflict with the NPC's defined role, or the like.
Flow continues to operation 612 where the NPC profile in the game is updated with the modified agent personalization data. For example, the NPC game files may be updated to include the modified agent personalization data. In doing so, the game data used to control the NPC (e.g., control the NPCs in-game actions, controls the NPC's dialog responses, controls the NPC's role during gameplay, etc. In doing so, the NPC is “possessed” by the user's agent. As such, the NPC is customized to interact with the user according to the user's preferences, to adopt a playstyle that compliments the user's preferred playstyle, adopt gameplay strategies preferred by the user, or the like. At operation 614, the possessed NPC is instantiated in the game to interact with the user. In one example, the instantiated NPC may be controlled in accordance with the component of the personalized agent library discussed herein, albeit modified in accordance with the game and NPC constraints as discussed above. Alternatively, the NPC may be controlled in accordance with updated game files that include the modified agent personalization data.
While specific examples described herein relate to utilizing agents in a game environment, one of skill in the art will appreciate that said techniques can be applied to generate and use agents in other types of environments, such as an enterprise environment. For example, personalized agents may be generated to help users perform tasks in an enterprise environment, or using any other type of application.
As stated above, a number of program tools and data files may be stored in the system memory 704. While executing on the at least one processing unit 702, the program tools 706 (e.g., an application 720) may perform processes including, but not limited to, the aspects, as described herein. The application 720 includes a personalized agent generator 730, NPC customization instructions 732, the various APIs 734 disclosed herein, as well as instructions to perform the various processes disclosed herein. Other program tools that may be used in accordance with aspects of the present disclosure may include electronic mail and contacts applications, word processing applications, spreadsheet applications, database applications, slide presentation applications, drawing or computer-aided application programs, etc.
Furthermore, aspects of the disclosure may be practiced in an electrical circuit comprising discrete electronic elements, packaged or integrated electronic chips containing logic gates, a circuit utilizing a microprocessor, or on a single chip containing electronic elements or microprocessors. For example, aspects of the disclosure may be practiced via a system-on-a-chip (SOC) where each or many of the components illustrated in
The computing device 700 may also have one or more input device(s) 712, such as a keyboard, a mouse, a pen, a sound or voice input device, a touch or swipe input device, etc. The output device(s) 714 such as a display, speakers, a printer, etc. may also be included. The aforementioned devices are examples and others may be used. The computing device 700 may include one or more communication connections 716 allowing communications with other computing devices 750. Examples of the communication connections 716 include, but are not limited to, radio frequency (RF) transmitter, receiver, and/or transceiver circuitry; universal serial bus (USB), parallel, and/or serial ports.
The term computer readable media as used herein may include computer storage media. Computer storage media may include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, or program tools. The system memory 704, the removable storage device 709, and the non-removable storage device 710 are all computer storage media examples (e.g., memory storage). Computer storage media may include RAM, ROM, electrically erasable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other article of manufacture which can be used to store information and which can be accessed by the computing device 700. Any such computer storage media may be part of the computing device 700. Computer storage media does not include a carrier wave or other propagated or modulated data signal.
Communication media may be embodied by computer readable instructions, data structures, program tools, or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and includes any information delivery media. The term “modulated data signal” may describe a signal that has one or more characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media may include wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, radio frequency (RF), infrared, and other wireless media.
One or more application programs 866 may be loaded into the memory 862 and run on or in association with the operating system 864. Examples of the application programs include phone dialer programs, e-mail programs, information management (PIM) programs, word processing programs, spreadsheet programs, Internet browser programs, messaging programs, and so forth. The system 802 also includes a non-volatile storage area 868 within the memory 862. The non-volatile storage area 868 may be used to store persistent information that should not be lost if the system 802 is powered down. The application programs 866 may use and store information in the non-volatile storage area 868, such as e-mail or other messages used by an e-mail application, and the like. A synchronization application (not shown) also resides on the system 802 and is programmed to interact with a corresponding synchronization application resident on a host computer to keep the information stored in the non-volatile storage area 868 synchronized with corresponding information stored at the host computer. As should be appreciated, other applications may be loaded into the memory 862 and run on the mobile computing device 800 described herein.
The system 802 has a power supply 870, which may be implemented as one or more batteries. The power supply 870 might further include an external power source, such as an AC adapter or a powered docking cradle that supplements or recharges the batteries.
The system 802 may also include a radio interface layer 872 that performs the function of transmitting and receiving radio frequency communications. The radio interface layer 872 facilitates wireless connectivity between the system 802 and the “outside world” via a communications carrier or service provider. Transmissions to and from the radio interface layer 872 are conducted under control of the operating system 864. In other words, communications received by the radio interface layer 872 may be disseminated to the application programs 866 via the operating system 864, and vice versa.
The visual indicator 820 (e.g., LED) may be used to provide visual notifications, and/or an audio interface 874 may be used for producing audible notifications via the audio transducer 825. In the illustrated configuration, the visual indicator 820 is a light emitting diode (LED) and the audio transducer 825 is a speaker. These devices may be directly coupled to the power supply 870 so that when activated, they remain on for a duration dictated by the notification mechanism even though the processor 860 and other components might shut down for conserving battery power. The LED may be programmed to remain on indefinitely until the user takes action to indicate the powered-on status of the device. The audio interface 874 is used to provide audible signals to and receive audible signals from the user. For example, in addition to being coupled to the audio transducer 825, the audio interface 874 may also be coupled to a microphone to receive audible input, such as to facilitate a telephone conversation. In accordance with aspects of the present disclosure, the microphone may also serve as an audio sensor to facilitate control of notifications, as will be described below. The system 802 may further include a video interface 876 that enables an operation of devices connected to a peripheral device port 830 to record still images, video stream, and the like.
A mobile computing device 800 implementing the system 802 may have additional features or functionality. For example, the mobile computing device 800 may also include additional data storage devices (removable and/or non-removable) such as, magnetic disks, optical disks, or tape. Such additional storage is illustrated in
Data/information generated or captured by the mobile computing device 800 and stored via the system 802 may be stored locally on the mobile computing device 800, as described above, or the data may be stored on any number of storage media that may be accessed by the device via the radio interface layer 872 or via a wired connection between the mobile computing device 800 and a separate computing device associated with the mobile computing device 800, for example, a server computer in a distributed computing network, such as the Internet. As should be appreciated such data/information may be accessed via the mobile computing device 800 via the radio interface layer 872 or via a distributed computing network. Similarly, such data/information may be readily transferred between computing devices for storage and use according to well-known data/information transfer and storage means, including electronic mail and collaborative data/information sharing systems.
In some examples, a system for generating a personalized agent is provided. The system comprises at least one processor and memory encoding computer-executable instructions that, when executed by the at least one processor, cause the at least one processor to perform operations comprising: generate agent personalization data, wherein generating personalized agent data comprises: instantiating an agent in a gameplay session with a user for a first game, wherein the instantiated agent is operable to interact with the user playing the first game based upon one or more machine learning models trained to play the game; receiving, by the agent, a user interaction during gameplay; generating, via the one or more machine learning models, an agent response to the user interaction; instructing the agent to perform the agent response; receiving feedback to the agent response from the user; and generating agent personalization data based upon the agent response and the user feedback; store the agent personalization data; receive a request from a second game for agent personalization data; based upon the request, identify a subset of agent personalization data; and send at least a subset of the agent personalization data.
In some examples, at least one of the one or more machine learning models is trained using reinforcement learning to control agent gameplay in the first game.
In some examples, the one or more machine learning models comprise: a foundation model; a language model; a computer vision models; or a speech model.
In some examples, the requests comprises one or more of: information about the second game; information about a non-player character (NPC) of the second game associated with the request; identification of a specific agent; or identification of specific agent characteristics.
In some examples, when the request comprises information about the second game, identifying the subset of agent personalization data comprises: determining a genre for the second game; and identifying agent personalization data related to the genre.
In some examples, when the request comprises information about the NPC, identifying the subset of agent personalization data comprises: determine one or more NPC characteristics, wherein NPC characteristic comprise one or more of: NPC abilities; NPC party role; or NPC characteristics; and identify agent personalization data related to the one or more NPC characteristics.
In some examples, when the request comprises the identification of the specific agent, identifying the subset of agent personalization data comprises identifying personalization data associated with the specific agent.
In some examples, a method for providing agent personalization data is provided, the method comprising: generating agent personalization data, wherein generating personalized agent data comprises: instantiating an agent in a gameplay session with a user for a first game, wherein the instantiated agent is operable to interact with the user playing the first game based upon one or more machine learning models trained to play the game; receiving, by the agent, a user interaction during gameplay; generating, via the one or more machine learning models, an agent response to the user interaction; instructing the agent to perform the agent response; receiving feedback to the agent response from the user; and generating agent personalization data based upon the agent response and the user feedback; storing the agent personalization data; receiving a request from a second game for agent personalization data; based upon the request, identifying a subset of agent personalization data; and sending at least a subset of the agent personalization data.
In some examples, at least one of the one or more machine learning models is trained using reinforcement learning to control agent gameplay in the first game.
In some examples, the one or more machine learning models comprise: a foundation model; a language model; a computer vision models; or a speech model.
In some examples, the request comprises one or more of: information about the second game; information about a non-player character (NPC) of the second game associated with the request; identification of a specific agent; or identification of specific agent characteristics.
In some examples, the request comprises information about the second game, identifying the subset of agent personalization data comprises: determining a genre for the second game; and identifying agent personalization data related to the genre.
In some examples, when the request comprises information about the NPC, identifying the subset of agent personalization data comprises: determine one or more NPC characteristics, wherein NPC characteristic comprise one or more of: NPC abilities; NPC party role; or NPC characteristics; and identify agent personalization data related to the one or more NPC characteristics.
In some examples, when the request comprises the identification of the specific agent, identifying the subset of agent personalization data comprises identifying personalization data associated with the specific agent.
In some examples, when the request comprises the identification of the specific agent characteristics, identifying the subset of agent personalization data comprises identifying the specific agent characteristics from a plurality of agent characteristics.
In some examples, a method to personalize an NPC is provide, the method comprising: receiving a request to personalize a non-player character (NPC); sending a request for agent personalization data to an agent library; in response to sending the request, receiving the agent personalization data; analyzing the agent personalization data against one or more constraints; modifying the agent personalization data based upon the analysis of the one or more constraints; update an NPC profile with the modified agent personalization data; and instantiate the NPC based upon the updated NPC profile.
In some examples, the request to personalize the NPC is received upon creation of a new game or in response to meeting the NPC during gameplay.
In some examples, the one or more constraints relate NPC characteristics, wherein the one or more NPC characteristics comprise one or more of: NPC abilities; NPC power levels; or NPC role in the game story.
In some examples, modifying the agent personalization data comprises at least one of: deleting agent personalization data that does not comply with the NPC characteristics; or modify the agent personalization data to comply with the NPC characteristics.
The description and illustration of one or more aspects provided in this application are not intended to limit or restrict the scope of the disclosure as claimed in any way. The claimed disclosure should not be construed as being limited to any aspect, for example, or detail provided in this application. Regardless of whether shown and described in combination or separately, the various features (agent structural and methodological) are intended to be selectively included or omitted to produce an aspect with a particular set of features. Having been provided with the description and illustration of the present application, one skilled in the art may envision variations, modifications, and alternate aspects falling within the spirit of the broader aspects of the general inventive concept embodied in this application that do not depart from the broader scope of the claimed disclosure.
This application claims priority to U.S. Provisional Application Ser. No. 63/345,216, filed May 24, 2022, entitled “Creation and Personalization of Virtual Gamers,” the entire disclosure of which is hereby incorporated herein by reference.
| Number | Date | Country | |
|---|---|---|---|
| 63345216 | May 2022 | US |
