Patent Application
20240374994
The present invention relates to systems and methods for providing a user interface on a game controller.
The user interface (UI) for playing a video game typically consists of various on-screen elements and controls that allow the player to interact with the game environment and control the game character. The specific UI elements and controls can vary depending on the type of games and platforms.
One common example of controls is a game controller that has a variety of input means such as buttons, joysticks and touchscreens that enable the user to manipulate the game character such as performing an action, and interact with the game elements in the game world. Game controllers may take various forms, for example, a console controller with directional pads, functional buttons, and joysticks. Other examples includes specialized controllers designed for a specific type of game, such as racing or flight simulators, and motion sensing controllers for detecting the player's body movement.
Feedback to the players is usually provided through a display device showing the game characters and game environment. Other on-screen UI elements are also used to provide important game information to the player, such as a heads-up display (HUD) which is a graphical overlay of the game player's view of the game environment displaying information such as health status, weapon status, score, and countdown timer. Known game controllers also include some type of feedback mechanism, such as vibration or force feedback, which provides tactile sensations that help to enhance the player's immersion in the game. Nevertheless, the conventional arrangements tend to suffer from one or more of a multiplicity of drawbacks:
Firstly, the conventional game controller feedback mechanism cannot provide information specifically for the operation of input devices on the game controller. Although such information may be provided on the game screen shown on the display device, the game screen usually contains overwhelming amount of information and it may be difficult for the player to locate such information.
Secondly, the conventional game controller feedback mechanism also cannot feedback information specifically for a game element related to the input devices on the game controller, such as a button or a joystick, so as to allow effective operation and control of the game controller.
The present invention seeks to mitigate or alleviate some or all of the above-mentioned problems.
Various aspects and features of the present invention are defined in the appended claims and within the text of the accompanying description.
In a first aspect, a method for providing a user interface on a controller having a plurality of input devices is provided in accordance with claim 1.
In another aspect, a controller for interfacing with a game apparatus is provided in accordance with claim 18.
In a further aspect, an information processing apparatus for providing a user interface on a controller having a plurality of input devices is provided in accordance with claim 31.
A more complete appreciation of the disclosure and many of the attendant advantages thereof will be readily obtained as the same becomes better understood by reference to the following detailed description when considered in connection with the accompanying drawings, wherein:
A video game system and method are disclosed. In the following description, a number of specific details are presented in order to provide a thorough understanding of the embodiments of the present invention. It will be apparent, however, to a person skilled in the art that these specific details need not be employed to practice the present invention. Conversely, specific details known to the person skilled in the art are omitted for the purposes of clarity where appropriate.
Embodiments of the present description are applicable to a video game system involving a video game console, a development kit for such a system, or a video game system using dedicated hardware or a computer and suitable controllers. In the present application terms such as ‘user’ and ‘player’, ‘dialogue’ and ‘narrative’ may be used interchangeably except where indicated otherwise.
For the purposes of explanation and referring now to the drawings, wherein like reference numerals designate identical or corresponding parts throughout the several views,
According to embodiments of the present application, the video game system 100 provides a user interface on a game controller 180 which allows the user to receive information from the game controller regarding a specific input device on the game controller. In this case, the game console 105, such as a PlayStation®, accesses the game title database 120 and loads game data to initiate interactive gameplay. According to embodiments of the present application, the game controller 180 contains multiple input devices such as buttons and joysticks. In the case of buttons, the game controller 180 may further contain multiple light emitting elements, such as light emitting diodes, that are associated with the respective buttons. In the case of joysticks, multiple light emitting elements, such as light emitting diodes, may be arranged on the game controller 180 to associate with different movement directions of the respective joysticks. The light emitting elements may produce different lighting effects based on different colours, brightness and patterns, in order to represent different game information corresponding to the associated input devices on the game controller 180. Although
Referring to the video game system 100 in
The game logic 110 executes game code loaded from the game content storage 124 of the game title database 120, in order to generate the game environment which interacts with the player character. The game logic 110 may also load user save data and user settings stored in local storage 160. The user may control the player character by operating the game controller 180, and the game logic 110 processes the input signals received from the game controller via the game controller interface 150.
The image processing unit 130 renders 2D or 3D computer graphics for the game environment and game feedbacks generated by the game logic 110. The image processing unit 130 then generates video signals for the game graphics, such as the game environment based on the player's perspective, and transmits the video signals to the display device 140.
Additionally, the audio processing unit 135 retrieves sound files and music files from the game title database 120 corresponding to the game environment and feedbacks generated by the game logic 110, then decompresses and decodes the files into audio signals for playback at the speaker system 145 to produce background music, character speech, audio tracks, sound effects of the game environment and the like.
Further, the controller interface 150 generates haptic feedback effects such as vibration on the game controllers 180 based on the game environment and game feedbacks generated by the game logic 110.
According to embodiments of the present application, the controller interface 150 also generates lighting feedback effects on the game controller 180 to provide game information such as directional indication to the user. The lighting feedback is generated by displaying lighting patterns on the light emitting elements associated with the respective input devices of the game controller 180. As such, the user can receive information that corresponding to the input devices of the game controller 180 in a way that feels more natural and intuitive. Accordingly, a more engaging and immersive gaming experience can be provided.
Further details of the video game console 105 will be described with reference to
The entertainment device also comprises RAM 240, and may either have separate RAM for each of the CPU and GPU, or shared RAM as in the PS5. The or each RAM can be physically separate, or integrated as part of an SoC as in the PS5. Further storage is provided by a disk 250, either as an external or internal hard drive, or as an external solid state drive, or an internal solid state drive as in the PS5.
The entertainment device may transmit or receive data via one or more data ports 260, such as a USB port, Ethernet® port, WiFi® port, Bluetooth® port or similar, as appropriate. It may also optionally receive data via an optical drive 270.
Interaction with the system is typically provided using one or more handheld controllers 280, such as the DualSense® controller in the case of the PS5.
Audio/visual outputs from the entertainment device are typically provided through one or more A/V ports 290, or through one or more of the wired or wireless data ports 260.
Where components are not integrated, they may be connected as appropriate either by a dedicated data link or via a bus 210.
Such an entertainment device may be used as a game console 105 in a video game system 100 to generate signals for the game controller 180 based on the interactive gameplay to provide a lighting effect that corresponds to the associated input devices on the game controller 180. It will be appreciated that this is a non-limiting example and that as noted previously herein other examples of a game console may include a phone or smart television.
The controller (typically in the central portion of the device) may also comprise one or more system buttons 306, which typically cause interaction with an operating system of the entertainment device rather than with a game or other application currently running on it; such buttons may summon a system menu, or allow for recording or sharing of displayed content. Furthermore, the controller may comprise one or more other elements such as a touchpad 308, a light for optical tracking (not shown), a screen (not shown), haptic feedback elements (not shown), and the like.
The controller may also comprise light emitting elements 311-314, each associated with one of the controls on the controller 180. The light emitting elements 311-314 may be light emitting diodes embedded in the controls. In the case of button control, the light emitting elements 311, 312 may be placed under the respective buttons that are transparent or partially transparent. In the case of joystick control, the light emitting elements 313, 314 may be arranged around the joystick corresponding to different movement directions of the respective joysticks, and placed under the controller housing that is transparent or partially transparent. The light emitting elements 311-314 may produce different lighting effects based on different colours, brightness and patterns, in order to represent different game information corresponding to the associated controls on the controller.
Optionally or additionally, the lighting effect produced by the light emitting elements may be lighting up, blinking, or glowing. In one example, the lighting effect may be based on a status of the game character corresponding to an input, such as a weapon button. The weapon button may flash in red colour when the weapon is out of ammunition. Optionally or additionally, the colour of the lighting effect may be based on the colours in the part of the game environment within view of the in-game virtual camera, such as the colour of the sky in the game world, thereby adding user immersion and realism to the game.
Optionally or additionally, the lighting effect of the light emitting element associated with an input devices for providing directional indication is based on the distance between the game character and a target object in the game environment. For example, the brightness and/or colour of the light emitting elements associated with a directional button or a direction of a directional stick corresponding to the guiding direction may be based on the distance of a quest location from the game character.
Optionally or additionally, the lighting effect based on the directional indications is produced on the light emitting elements 411, 412 only if an operation following the directional indication is not performed by the user within a predetermined length of time, such as 3 seconds, after the information of the directional indication has been displayed on the display device 140. As such, unnecessary distraction caused by the lighting effect can be avoided.
In the meantime, the game console 105 generates a visual feedback on the game controller 180 by lighting up the light emitting elements associated with the functional buttons 511, 512, that correspond the choices of response 504, 505. As such, the user can receive hints of performing an operation coming directly from a corresponding input device of the game controller 180 in a way that feels more natural and intuitive. Accordingly, a more engaging and immersive gaming experience can be provided.
Optionally or additionally, the lighting effect produced by the light emitting elements may be lighting up, blinking, or glowing. The colour of the lighting effect may be based on the colours in the part of the game environment within view of the in-game virtual camera, such as the colour of the room where the game characters conduct the conversation, thereby adding user immersion and realism to the game.
Optionally or additionally, the lighting effect based on a predetermined operation required to be performed is produced on the light emitting elements associated with the functional buttons 511, 512, only if the predetermined operation is not performed by the user within a predetermined length of time, such as 3 seconds, after the information of the predetermined operation has been displayed on the display device 140, for example, the dialogue window 503. As such, unnecessary distraction caused by the lighting effect can be avoided.
In the meantime, the game console 105 generates a visual feedback on the game controller 180 by lighting up the light emitting elements 611, 613, 615 associated with the buttons for performing the super move, to indicate that the energy bar is fully charged and the super move is ready to perform. Additionally, the sequence of button presses required to perform the super move can be demonstrated to the player by blinking the light emitting elements 611, 613, 615 in sequential order. Optionally or additionally, if the predetermined sequence of operations involves movements of a joystick, the light effect demonstration is provided by one or more of the light emitting elements around the associated joystick, in such a way that the light emitting elements 612, 614 corresponding to the movement directions are turned on in the order of the predetermined sequence. As such, the user can see a demonstration of the predetermined operation directly from the corresponding input devices of the game controller 180 in a way that feels more natural and intuitive. Accordingly, a more engaging and immersive gaming experience can be provided.
Optionally, in addition to showing, reminding, or training the user to become proficient with the input sequence per se, the lighting may alternatively or in addition be used to indicate the timing of an input sequence where this is relevant. Further optionally, the lighting may be used to indicate a period in which to use the associated input device, so that the user must follow the within a sequence of lights to correctly input the sequence on the controller.
Optionally or additionally, an input device of the game controller is an analogue input device, such as a directional stick, a steering wheel or an analogue button, the lighting effect of the light emitting element associated with the analogue input devices is based on a degree of movement applied or needed to be applied to the analogue input devices. For example, if an input device is a directional stick such as an analogue joystick or a flight stick having multiple movement directions, the game controller may have multiple light emitting elements, each associated with a different movement directions of the directional stick. The brightness and/or colour of the light emitting elements around the directional stick may also be based on a degree of movement applied to the directional stick in the associated movement direction. In another example, if an input device is a steering wheel, the game controller may have multiple light emitting elements, each associated with a different rotation directions of the steering wheel. The brightness and/or colour of the light emitting elements on the steering wheel may also be based on a degree of movement applied to the steering wheel in the associated rotation direction. In a further example, if an input device is an analogue button or a pedal, the game controller may have a light emitting element associated with the analogue button or the pedal. The brightness and/or colour of the light emitting elements associated with the analogue button or the pedal may also be based on a degree of push movement applied to the analogue button or the pedal.
Optionally or additionally, the lighting effect for demonstrating the predetermined operation is produced on the light emitting elements 611-615 only if the predetermined operation is not performed by the user within a predetermined length of time, such as 3 seconds, after the relevant information has been displayed on the display device 140, for example, the fully charged energy bar 603. As such, unnecessary distraction caused by the lighting effect can be avoided.
Optionally or additionally, the lighting effect produced by the light emitting elements may be lighting up, blinking, or glowing. In one example, the lighting effect may be based on a status of the game character corresponding to an input device, such as a weapon button. In the event the weapon becomes out of ammunition, the weapon button may flash in red colour to alert the player. In another example, a button may be associated with an ultimate skill of the game character. The light emitting element associated with the ultimate skill button may display blue colour when the ultimate skill is on cool down, and flash in green colour when the ultimate skill is ready.
Optionally or additionally, the colour of the lighting effect may be based on the colours in the part of the game environment within view of the in-game virtual camera, such as the theme colour of the fighting arena in the game world, thereby adding user immersion and realism to the game.
Optionally or additionally, the lighting effect produced by a light emitting element may be a visual feedback in response to an operation performed on the associated input device of the game controller. For example, the embedded LED of a push button will light up when the push button is pressed.
The process then moves to step 710 where the game console waits until a predetermined operation of input devices on the game controller 180 is required by the interactive gameplay. The process then proceeds to step 715 where the game console 105 extracts from the interactive gameplay the predetermined operation of input devices required to be performed by a user.
At step 720, the game console 105 sends control signals to the game controller 180 so as to demonstrate the predetermined operation through a lighting effect on a light emitting element corresponding to the input devices. Optionally or additionally, the game console 105 sends control signals to the game controller 180 to produce the lighting effect based on the part of game environment at a predetermined delay, such as 3 seconds, after the information of the part of game environment is displayed on the display device 140. As such, unnecessary distraction caused by the lighting effect due to rapid changes of the game environment can be avoided.
The process then moves to step 810 where the game console waits until highlighting a part of game environment (or game state) to the player is required by the interactive gameplay. The process then proceeds to step 815 where the game console 105 extracts from the interactive gameplay the part of game environment corresponding to an input device that is required to be highlighted to the user.
At step 820, the game console 105 sends control signals to the game controller 180 so as to display a lighting effect corresponding to the input device, on a light emitting element associated with the input device, based on the part of game environment extracted from the interactive gameplay.
The process then moves to step 910 where the game console waits until providing directional indications to the player is required by the interactive gameplay. The process then proceeds to step 915 where the game console 105 extracts from the interactive gameplay the directional indications that is required to be provided to a user.
At step 920, the game console 105 sends control signals to the game controller 180 so as to provide the directional indications through a lighting effect on a light emitting element corresponding to the input devices. Optionally or additionally, the game console 105 sends control signals to the game controller 180 to produce the lighting effect for directional indications only if an operation following the directional indication is not performed by the user within a predetermined length of time, such as 3 seconds, after the information of the directional indication has been displayed on the display device 140. As such, unnecessary distraction caused by the lighting effect can be avoided.
Examples of the directional indications that may reflect the game environment or game state include indicating available directions (e.g. exits), indicating a direction relevant to a quest or other in-game goal (thus providing path guidance potentially without the need for such a path to be visible on-screen, which can break immersion or occlude other important elements of the environment), and indicating other player or non-player characters, which may for example be outside the in-game field of view. Where the lighting effect can be differentiated by colour, brightness or pattern, prioritisation or categorisation of the indicated game environment or state elements can also be provided, for example using green lights for allies and red lights for enemies, or indicating all exits with static lights but one relevant to a quest with a pulsing light.
Variations in the systems, methods and techniques herein may also be contemplated.
For example, rather than turning on the light emitting elements, various lighting effect can be produced by the light emitting elements, such as a dynamic display pattern by blinking, flashing or glowing a single light emitting element, or a group of related light emitting elements, or even generating a dynamic running pattern by sequentially turning on the relevant light emitting elements. Further visual effect can be provided by changing the colour of the light emitting elements, and a combination of colour change and dynamic display pattern.
While the analogue movement of an input device, such as a steering wheel, may be represented by a lighting effect including multiple levels of brightness of an associated light emitting element, an array of light emitting elements may also be used to reflect the degree of movement of the input device. For example, the number of light emitting elements in the array being switched on is determined by the degree of rotation of the steering wheel.
It is relatively common for users to change the user settings and customize the mapping of various input devices of the controller to game functions. For example, a player may prefer to use the left button group 302L to control the movement directions of the game character, whereas another player may prefer to use the right button group 302R for the same functions. In such case, the lighting effects produced by a light emitting element are therefore associated with the function performed by the button instead of a physical position of the button.
Different users may have different preferences of the lighting effects displayed on the game controller. Hence optionally, the light effects produced by a light emitting element may be customized by the user by changing the user settings stored in local storage 160. For example, the user may assign a specific colour or dynamic display pattern in response to an operation of an associated input device, a change of game element in the game environment, a change of player character status, or the like.
It will be appreciated that the above methods may be carried out on conventional hardware suitably adapted as applicable by software instruction (e.g. game console 105) or by the inclusion or substitution of dedicated hardware.
Thus the required adaptation to existing parts of a conventional equivalent device may be implemented in the form of a computer program product comprising processor implementable instructions stored on a non-transitory machine-readable medium such as a floppy disk, optical disk, hard disk, solid state disk, PROM, RAM, flash memory or any combination of these or other storage media, or realised in hardware as an ASIC (application specific integrated circuit) or an FPGA (field programmable gate array) or other configurable circuit suitable to use in adapting the conventional equivalent device. Separately, such a computer program may be transmitted via data signals on a network such as an Ethernet, a wireless network, the Internet, or any combination of these or other networks.
Hence in a summary embodiment of the present description, an information processing apparatus comprises the following.
Firstly, a game execution module (e.g.: game logic 110) is configured (for example by suitable software instruction) to load game data to initiate interactive gameplay, as described elsewhere herein.
Secondly, an interface module (e.g.: game controller interface 150) is configured (for example by suitable software instruction) to send control signals to a controller based on the interactive gameplay to provide a lighting effect on a light emitting element associated with one of a plurality of input devices on the controller, as described elsewhere herein. In particular, the lighting effect corresponds to the associated input device.
It will be apparent to a person skilled in the art that variations in the above system corresponding to the various embodiments of the method as described and claimed herein are considered within the scope of the present invention, including but not limited to that:
In an instance of the summary embodiment, the light emitting element is embedded in the associated input device, as described elsewhere herein;
In an instance of the summary embodiment, the plurality of input devices comprises an digital input device, and the lighting effect of the light emitting element associated with the digital input device is based on the state of the digital input device, as described elsewhere herein;
In an instance of the summary embodiment, the digital input device is a push button, as described elsewhere herein;
In an instance of the summary embodiment, the plurality of input devices comprises an analogue input device, and the lighting effect of the light emitting element associated with the analogue input device is based on a degree of movement of the analogue input device, as described elsewhere herein;
In an instance of the summary embodiment, the analogue input device is a directional stick having a plurality of movement directions, and the controller further comprising a plurality of light emitting elements, each associated with the plurality of movement directions of the directional stick, as described elsewhere herein;
In an instance of the summary embodiment, the lighting effect of the light emitting element is based on a degree of movement in the associated movement direction of the directional stick, as described elsewhere herein;
In an instance of the summary embodiment, the analogue input device is a rotational wheel, and further comprising a plurality of light emitting elements, each associated with one of the rotational directions of the rotational wheel, as described elsewhere herein;
In an instance of the summary embodiment, the lighting effect of the light emitting element is based on a degree of rotation in the associated rotational direction of the rotational wheel, as described elsewhere herein;
In an instance of the summary embodiment, the method comprises extracting from the interactive gameplay a predetermined operation of at least one of the plurality of the input devices to be performed by a user, and demonstrating the predetermined operation through a lighting effect on a light emitting element corresponding to the input device, as described elsewhere herein;
In an instance of the summary embodiment, the predetermined operation of at least one of the plurality of the input devices comprises simultaneous operations or a sequence of operations, as described elsewhere herein;
In an instance of the summary embodiment, the method comprises displaying information of the predetermined operation on a display device, wherein the predetermined operation is demonstrated through the lighting effect on the light emitting element corresponding to the input device at a predetermined delay after the information of the predetermined operation is displayed on the display device, as described elsewhere herein;
In an instance of the summary embodiment, the method comprises extracting from the interactive gameplay a part of game environment corresponding to at least one of the input devices, displaying a lighting effect corresponding to the input device, on a light emitting element associated with the input device, based on the part of game environment, as described elsewhere herein;
In an instance of the summary embodiment, the method comprises displaying information of the part of game environment on a display device, wherein the lighting effect based on the part of game environment is displayed on the light emitting element at a predetermined delay after the information of the part of game environment is displayed on the display device, as described elsewhere herein;
In an instance of the summary embodiment, the method comprises extracting from the interactive gameplay a plurality of directional indications provided to a user, displaying a lighting effect corresponding to the input device, on a light emitting element associated with the input device, based on the plurality of directional indications, as described elsewhere herein; and
In an instance of the summary embodiment, the method comprises displaying information of the directional indications on a display device, wherein the lighting effect based on the directional indications is displayed on the light emitting element at a predetermined delay after the information of the directional indications is displayed on the display device, as described elsewhere herein.
The foregoing discussion discloses and describes merely exemplary embodiments of the present invention. As will be understood by those skilled in the art, the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. Accordingly, the disclosure of the present invention is intended to be illustrative, but not limiting of the scope of the invention, as well as other claims. The disclosure, including any readily discernible variants of the teachings herein, defines, in part, the scope of the foregoing claim terminology such that no inventive subject matter is dedicated to the public.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2307066.7 | May 2023 | GB | national |
