Patent Application
20240024769
The present technique relates to the field of entertainment devices and more particularly to interpretation of input actuation information received from a controller.
Even with the recent advances in performance capture (i.e., tracking of a system user's motion to induce a corresponding effect on a displayed graphic), a handheld controller remains the most popular input device for the electronic game industry.
The handheld controller 80 may also comprise, inside the controller's body, motion detection circuitry (e.g. accelerometer and/or gyroscope circuitry) to detect motion of the controller.
In typical use, a handheld controller such as the DualSense® controller 80 is typically held by a user in two hands, as shown in
However, this arrangement can be problematic for users who are unable to operate the handheld controller 80 with both hands simultaneously, since it can be difficult to reach all of the controls on the controller 80 when holding the controller in one hand or when one of two hands is not fully functional. For example, some users may have mobility issues which affect one hand more than the other, or may only have one hand.
The present invention seeks to mitigate or alleviate this problem.
In a first example of the present technique, there is provided an entertainment device as defined in claim 1.
In another example of the present technique, there is provided a method as defined in claim 13.
In another example of the present technique, there is provided a computer program as defined in claim 14.
In another example of the present technique, there is provided a computer-readable storage medium as defined in claim 15. The computer-readable storage medium can be transitory or non-transitory.
Further respective aspects and features of the invention are defined in the appended claims.
A more complete appreciation of the disclosure and many of the attendant advantages thereof will be readily obtained as the same becomes better understood by reference to the following detailed description when considered in connection with the accompanying drawings, wherein:
For clarity of illustration, elements illustrated in the figures have not necessarily been drawn to scale. Further, where considered appropriate, reference numerals have been repeated among the figures to indicate corresponding or analogous elements.
The terms “coupled” and “connected,” along with their derivatives, may be used herein to describe structural relationships between components of the apparatus for performing the operations herein. It should be understood that these terms are not intended as synonyms for each other. Rather, in particular embodiments, “connected” is used to indicate that two or more elements are in direct physical or electrical contact with each other while “coupled” is used to indicate two or more elements are in either direct or indirect (with other intervening elements between them) physical or electrical contact with each other, and/or that the two or more elements co-operate or communicate with each other (e.g., as in a cause an effect relationship).
Methods and systems are disclosed for controlling an interpretation of input actuation information received from a controller. In the following description, a number of specific details are presented in order to provide a thorough understanding of the embodiments of the present invention. It will be apparent, however, to a person skilled in the art that these specific details need not be employed to practice the present invention. Conversely, specific details known to the person skilled in the art are omitted for the purposes of clarity where appropriate.
Referring now to the drawings, wherein like reference numerals designate identical or corresponding parts throughout the several views,
The entertainment device 200 comprises a central processor 220. This may be a single or multi core processor, for example comprising eight cores as in the PS5. The entertainment system also comprises a graphical processing unit or GPU 30. The GPU can be physically separate to the CPU, or integrated with the CPU as a system on a chip (SoC) as in the PS5. The central processor 220 is an example of processing circuitry to process a game state of a game and, in at least a first mode of operation, select, from amongst a plurality of interpretations, an interpretation of control actuation information in dependence on the associated motion detection information, and update the processing of the game state in dependence on the selected interpretation of the control actuation information.
The entertainment device also comprises RAM 240, and may either have separate RAM for each of the CPU and GPU, or shared RAM as in the PS5. The or each RAM can be physically separate, or integrated as part of an SoC as in the PS5. Further storage is provided by a disk 250, either as an external or internal hard drive, or as an external solid state drive, or an internal solid state drive as in the PS5.
The entertainment device may transmit or receive data (e.g. including input actuation information and motion detection information transmitted by the one or more handheld controllers 80) via one or more data ports 260 (e.g. receiver circuitry), such as a USB port, Ethernet® port, WiFi® port, Bluetooth® port or similar, as appropriate. It may also optionally receive data via an optical drive 270. The data ports 260 and the optical drive 270 are examples of receiver circuitry to receive, from a controller, control actuation information and associated motion detection information, wherein the control actuation information is indicative of detected actuation of at least one control of the controller and the associated motion detection information is indicative of detected motion of the controller.
Interaction with the system is typically provided using one or more handheld controllers 80, such as the DualSense® controller in the case of the PS5 (as illustrated in
Audio/visual outputs from the entertainment device are typically provided through one or more A/V ports 290, or through one or more of the wired or wireless data ports 260. An example of a device for displaying images output by the entertainment system is a head mounted display ‘HMD’ 294, worn by a user 296.
Where components are not integrated, they may be connected as appropriate either by a dedicated data link or via a bus 292.
In examples of the present technique, detected motion of the controller is used, by the entertainment device, to interpret the user's actuation (e.g. operation) of the controls. This allows the user to actuate controls which are within their reach (e.g. the controls in the right and central regions, in the example of
In a particular example, the user may apply a particular pattern of motion to the controller to indicate that detected actuation of one of the controls on the right-hand side of the controller should be interpreted as if a corresponding one of the controls on the left-hand side was actuated. This interpretation is an example of a control-remap interpretation in which actuation of the at least one control is interpreted as actuation of at least one different control. The user may then apply a different pattern of motion (or may hold the controller still) to indicate that detected actuation of one of the controls on the right-hand side of the controller should be interpreted as normal (e.g. interpreting actuation of the controls on the right-hand side as actuation of those same inputs).
The controller 80 may be configured to detect motion along or around any of three axes—translational motion along the “X”, “Y” and “Z” axes (left and right; forward and back; up and down) and rotational motion around these same axes (e.g. rotation about the X-axis (“pitch”), rotation about the Y-axis (“roll”) and rotation about the Z-axis (“yaw”)). The patterns of motion applied by the user may, therefore, involve motion along or about any of these axes either singly or in combination. Motion about or around any of the X, Y and Z-axes is an example of the detected motion comprising at least one of a change in orientation (e.g. rotation about one of the axes) of the controller and a change in position (e.g. movement along one of the axes) of the controller.
In a particular example, the user may rotate the controller 80 about the pitch axis to indicate that actuation of any of the controls on the right-hand side should be interpreted as if a corresponding control on the left-hand side was actuated. The corresponding control may be any input on the left-hand side. For example, if the right joystick 84R, right bumper 90R or right trigger 92R is actuated while the controller is rotated about the X-axis, this may be interpreted as actuation of the left joystick 84L, left bumper 90L or left trigger 92L respectively. Similarly, actuation of any of the buttons in the right button group 82R could be interpreted as actuation of a corresponding button in the left button group 82L. For example, the corresponding button could be a button in the same position in the left-button group relative to the position of the button in the right-hand group 82R (e.g. the right-most button in the right button group may be interpreted as the right-most button in the left button group); this is an example of a transpose interpretation, in which a position of the control relative in the second group of controls substantially aligns with a position of the control in the first group of controls. In another example, the corresponding button could be a button in a reflected position in the left-button group (e.g. the right-most button in the right button group may be interpreted as the left-most button in the left button group); this is an example of a mirror interpretation, in which a position of the control in the second group of controls substantially aligns with a minor image of a position of the control in the first group of controls. Note that, in some examples, the entertainment device may be arranged to support both the mirror interpretation and the transpose interpretation, with each interpretation being selectable in dependence on the motion detection information.
It will be appreciated that the exact pattern of motion to be applied by the user is not particularly limited, and the user may even be able to set some control value in a menu displayed by the entertainment system to determine which pattern or patterns of motion trigger which interpretation(s). Moreover, it will be appreciated that although
The method also comprises processing 704 a game state of a game and selecting 706, from amongst a plurality of interpretations, in dependence on the motion detection information, a selected interpretation of the control actuation information. Once an interpretation has been selected, the processing of the game state is updated 708 in dependence on the selected interpretation.
Hence, by applying the method of
On the other hand, if the predetermined pattern of motion is detected (Y), the method comprises interpreting 806 actuation of any controls in a first group of controls (e.g. this could include all controls on the left/right-hand side of the controller, or all controls in one of the button groups on the controller) as actuation of controls in a second group of controls (comprising different controls than the first group of controls)—for example, actuation of any button “Y” in a first group of buttons may be interpreted as if different button “Z” in a second group of buttons was actuated.
This approach allows actuation of any controls in a given group of controls to be mapped, when a given pattern of motion is detected, onto a different group of controls. This improves accessibility by allowing a user to control the game state to be updated based on functions associated with the second group of controls, even when these controls are not within the user's reach.
The method then involves continuing to operate in the default mode until it is determined 906 that the predetermined value has been de-selected and/or the in-game condition no longer exists, at which point the method comprises returning 908 to the first mode of operation.
In the method of
When it is determined that the (at least one) predetermined criterion has been met (Y), an interpretation of the input actuation information is then selected based on the filtered detected motion. On the other hand, when it is determined that the (at least one) predetermined criterion has not been met (N), the method comprises 804 interpreting the actuation of any of the manual actuation inputs of the controller as normal (e.g. as actuation of those same manual actuation inputs).
Once an interpretation has been selected 8041008, the processing of the game state is updated 708.
Hysteresis may also be applied at a threshold used as the trigger point for selecting how a group of controls is interpreted; for example, the threshold may be when the controller is held at an angle □ relative to the horizontal, so that when held more horizontally (below □), inputs are interpreted as normal, but when held more vertically (above □), inputs in the first group are interpreted as being from the second group as described elsewhere herein. In this case, there may be a hysteresis of, for example, 5 degrees so that when transitioning through angle □, the new interpretation is maintained even if the controller transitions back through angle □ by up to the hysteresis angle of 5 degrees (and vice-versa). A benefit of this may be for example that the user can set the angle □ to be comfortable to use, potentially with a relatively small motion required to transition, but the hysteresis can allow the small motion to trigger the transition without needing to be too precise, or necessarily maintain the position for long. It will be appreciated that this approach may apply to changes in position as well as rotation.
When (Y) the locking pattern of motion is detected, the method comprises locking 1104 the interpretation of any further control actuation information received from the controller to a previously-selected interpretation. The previously-selected interpretation is an example of a locked interpretation, comprising an interpretation selected in dependence on previous motion detection information received before entering the locked mode of operation.
For example, where the locking pattern of motion is a predetermined pattern of motion that is performed twice, the previously-selected interpretation could be whichever interpretation corresponds with the predetermined pattern of motion, and this could be applied to subsequently-received control actuation information independent of any motion detection information received with the subsequently-received control actuation information.
Steps 1102 and 1104 are an example of processing circuitry of the entertainment device or computer being responsive to the associated motion detection information indicating that a locking pattern of motion of the controller has been detected to begin operating in a locked mode of operation; and in response to further control actuation information received by the receiver circuitry while in the locked mode of operation, updating the processing of the game state in dependence on a locked interpretation of the further control actuation information, independent of further motion detection information received in association with the further control actuation information.
The selected interpretation then remains locked until it is determined 1106 that an unlocking pattern of motion has been detected. Once this unlocking pattern of motion has been detected (Y), the method comprises returning 908 to the first (e.g. unlocked) mode of operation.
The entertainment device and methods of the present technique enable use of a handheld controller by a user who is unable to access all of the controls on the controller, by allowing them to—for example—actuate a control that they can reach while applying a pattern of motion that indicates, to the entertainment device, that the actuation of that control should be interpreted as if a different control was actuated.
To give a particular example, purely for illustrative purposes, a user who is only able to use their right hand to hold the controller may turn the controller upside down (e.g. by rotating the controller approximately 180° about one of the X, Y and Z axes) to indicate that actuation of, for example, the right-joystick should be interpreted as if it were actually the left-joystick being actuated. This allows the function associated with the left joystick—which the user may not be able to reach—to be useable by the user. For example, in some games the left joystick controls motion of an in-game avatar while the right joystick controls a camera angle (e.g. allowing the user to look around the avatar's environment). In this example arrangement, the user could hold the controller face-up when operating the right joystick to look around, and hold the controller face-down when operating the right joystick to move.
It will, of course, be appreciated that this is just one specific example of how the present technique may be implemented. Any other interpretation of any of the controls of the controller could be applied in addition or instead of that discussed above, and the pattern of movement can be any pattern of movement.
It will be appreciated that the methods and techniques herein refer to detected motion and to patterns of movement; whilst this may relate for example to moving the controller in a circle as a pattern of movement, or turning it upside down, or moving it horizontally to the right by at least 20 cm as a detected motion, more generally the detected motion indicates a change in physical position and/or orientation of the controller that transitions through some triggering criterion; for example changing an angle in one or more axes beyond a threshold angle, changing position along one or more axes beyond a threshold displacement, or describing a path (such as a circle) to a threshold extent and/or with a threshold accuracy.
It will be appreciated that methods and techniques described herein may be carried out on conventional hardware (such as entertainment device 200) suitably adapted as applicable by software instruction or by the inclusion or substitution of dedicated hardware.
Thus the required adaptation to existing parts of a conventional equivalent device may be implemented in the form of a computer program product comprising processor implementable instructions stored on a non-transitory machine-readable medium such as a floppy disk, optical disk, hard disk, solid state disk, ROM, PROM, RAM, flash memory or any combination of these or other storage media, or realised in hardware as an ASIC (application specific integrated circuit) or an FPGA (field programmable gate array) or other configurable circuit suitable to use in adapting the conventional equivalent device. Separately, such a computer program may be transmitted via data signals on a network such as an Ethernet, a wireless network, the Internet, or any combination of these or other networks.
For example, the methods and techniques described herein may be implemented by an operating system (or helper app thereof) of the entertainment device, which presents the interpreted inputs to a game or other application so that this game or other application can operate transparently and without specific adaptation of its own. Optionally however a specific game or application itself may implement the methods and techniques described herein in the event that these are not supported at the system level.
A particular benefit of the approaches described herein is that the change of interpretation of controls is instigated by a detected movement of the controller; this means that it is not necessary to assign a button of the controller to this function; typically games use all the available buttons of a controller, and so it is not straightforward to assign a new function to one (particularly in the case of legacy games whose functionality is already fixed). This is then exacerbated in the case of a user that already has restricted physical access to the buttons. Meanwhile, motion control input is used less frequently or for less critical functions, and even where it is used, it will generally be possible to identify and select a trigger motion for the purposes of the present techniques that is not required or used by the current game or application, allowing the present techniques to coexist transparently with the games and applications being controlled in this manner. It will also be appreciated the such motion is typically caused by the user's wrist or arm rather than by their fingers; hence again this may be easier to actuate for a user with finger mobility issues, for example due to arthritis.
The foregoing discussion discloses and describes merely exemplary embodiments of the present invention. As will be understood by those skilled in the art, the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. Accordingly, the disclosure of the present invention is intended to be illustrative, but not limiting of the scope of the invention, as well as other claims. The disclosure, including any readily discernible variants of the teachings herein, defines, in part, the scope of the foregoing claim terminology such that no inventive subject matter is dedicated to the public.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2210671.0 | Jul 2022 | GB | national |
