Patent Application
20200269130
Generally, this disclosure relates to controllers used with teleoperation, virtual reality (VR), augmented reality (AR), toy, gaming, or other systems. More specifically, the present disclosure relates to mating a controller to “mate-points” or objects in the environment, mating to accessories, or mating multiple controllers together in different configurations by joining them at one of several possible mating locations that are part of the controllers, wherein mating in one of several possible configurations can be used to transform both: (1) the way that input from the controllers is interpreted in each respective use case (e.g., mating the controllers in a machine gun configuration could make a machine gun appear onscreen in a video game), and (2) how feedback is sent back to the controller (e.g., audio or haptic feedback sent to the controller from a video game). The mating interfaces between the controllers can be tailored to prevent or allow relative motion as desired for a specific use case and to accommodate human ergonomics.
A powerful enhancement when interacting with teleoperated, virtual, toy, or gaming interfaces is to allow the user to more closely mimic the configuration (e.g., relative hand position) of the activity the user is remotely or virtually controlling, or mimic the configuration of the activity the user is pretending to do (i.e. in the case of children playing with toys). Typically, toys, video games, and telerobots have been controlled with a bi-manual game pad. Embodiments of the present approach utilize multiple attachment points on the controller that can be reconfigured and mated by the user either at the beginning of an interaction session or on-the-fly during the session. This creates the ability for the user to experience enhanced interaction in multiple configurations with the same controller without the need to purchase additional accessories. Furthermore, when the controllers are capable of providing haptic feedback, the haptic feedback can be tailored to reflect each device configuration, whether a controller is mated with an attachment point in the environment or an accessory, or multiple controllers are mated to each other.
The subject matter claimed herein is not limited to embodiments that solve any disadvantages or that operate only in environments such as those described above. Rather, this background is only provided to illustrate one exemplary technology area where some embodiments described herein may be practiced.
Embodiments disclosed herein address one or more of the foregoing or other problems in the art with apparatuses, systems, and methods for connecting multiple controllers together for the purpose of transforming the input to and feedback from a telerobotic system, virtual or augmented reality system, toy system, or video game system (System). That is, input from the controllers may be interpreted differently when they are mated in a particular configuration, mated to an accessory, or to a particular mate-point in the environment, and the interpretation can also depend on the specific use case. For example, mating the controllers in a configuration with one hand behind the other (like holding a “Tommy Gun” machine gun) could make a machine gun appear onscreen in a video game, whereas in a virtual surgery simulation, this same configuration may have a 2-handed endoscope appear onscreen in the simulation (This configuration is referred to as a machine gun configuration).
Changing the mating configuration of the controllers also correspondingly changes how feedback is sent back to the controller. Audio or haptic feedback sent to the controller could be changed based on the hand configuration and use case, where haptic feedback is broadly defined meaning vibration feedback, force feedback, shear or skin stretch feedback, contact feedback, or any other type of touch feedback. For example, in a shooter-based video game, machine gun audio and haptic feedback could be portrayed on the controller when the controllers are mated in the machine gun configuration, whereas patient interaction forces in an endoscopic procedure in a surgery simulator could be portrayed on the controllers when the controllers are held in a pose that resembles a pose that a surgeon would hold an endoscope (or held in some other pre-defined pose).
This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.
Additional features and advantages of exemplary implementations of the disclosure will be set forth in the description which follows and in part will be obvious from the description or may be learned by the practice of such exemplary implementations. The features and advantages of such implementations may be realized and obtained by means of the instruments and combinations particularly pointed out in the appended claims. These and other features will become more fully apparent from the following description and appended claims, or may be learned by the practice of such exemplary implementations as set forth hereinafter.
In order to describe the manner in which the above-recited and other advantages and features of the invention can be obtained, a more particular description of the invention briefly described above will be rendered by reference to specific embodiments thereof which are illustrated in the appended drawings. For better understanding, like elements have been designated with reference numbers throughout the various accompanying figures. Understanding that these drawings depict only typical embodiments of the invention and are not therefore to be considered to be limiting of its scope, the invention will be described and explained with additional specificity and detail through the use of the accompanying drawings in which:
One or more embodiments disclosed herein relate to mating controllers together, to accessories, or to mate-points in the environment. These controllers can be used with teleoperation, virtual reality (VR), augmented reality (AR), toy, gaming, or other systems. The present disclosure relates to mating a controller to “mate-points” or objects in the environment, mating to accessories, or to mating multiple controllers together in different configurations by joining them at one of several possible mating locations that are part of the controllers, wherein mating in one of several possible configurations can be used to transform both: (1) the way that input from the controllers is interpreted in each respective use case (e.g., mating the controllers in a machine gun configuration could make a machine gun appear onscreen in a video game), and (2) how feedback is sent back to the controller (e.g., audio or haptic feedback sent to the controller from a video game). The mating interfaces between the controllers can be tailored to prevent or allow relative motion as per desired for a specific use case and to accommodate human ergonomics.
Examples of accessories that could be mated with the controller include, but are not limited to, a tracker, a passive or actuated accessory (e.g., spring, gimbal, or motorized joint), a cosmetic item that resembles the simulated interaction (e.g., a sword blade), or other accessory items.
As an example of an environment attach point, if the user's left controller was mated with a rotating revolute attachment point in front of the user and the right controller was mated to a rotating ball joint to the right of the user, the left controller could be used as a steering wheel input and the right controller could be used as a gear shifter input. Another example of an environment attach point might be a mate-point on a virtual control panel in a standing AR or VR experience. Mating the user's controller to the mate-point on the panel could provide both inputs as if controlling that panel and feedback that specifically represents that interaction. For example, a spring-loaded 2-axis gimbal as an attach point could transform the user's controller into a very natural feeling 2-axis joystick, such that the user could pilot a ship, control a crane, etc. While these control actions could also be input just using a position-tracked controller in a VR, AR, robotic, toy, or gaming system, plugging in and attaching the controllers at a specific location in the environment can enhance the experience by assigning a specific purpose to mating with that location in the environment and help the user cognitively parse and map the interaction experience in a more intuitive manner.
Likewise, the specific axis that haptic feedback is calculated on and fed back to the user can be tailored to correspond to how the haptic feedback would be experienced when the user's hand configuration changes, e.g., as the user holds a tool/object with two hands in a pose that corresponds to a gun, steering wheel, shovel, sword, bow staff, etc. Note that the interaction forces for the user's two hands naturally change to reflect different hand poses and interactions they would have with real objects, thus the haptic feedback (e.g., haptic feedback in the form of force feedback, vibration feedback, or shear/skin stretch feedback that would be fed back to a user as a function of the user's hand configuration and interaction/activity type would also change). For example, holding a long fighting staff with one hand versus two hands provides a simple example of the change in interaction forces experienced by a user when interacting with real objects, hence the provided haptic feedback could also reflect this difference. In the case where the staff is hit (e.g., by an opponent) transversely to its length and near the end of the staff, holding the staff in one hand requires the user to oppose both the applied lateral forces and torques (or moment) that are generated when the end of the staff is hit. In contrast, when the staff is held in two hands, the user still must resist the lateral force, but a force couple generated by the user's two hands can resist the generated torque. Hence the net forces and torques experienced when holding the staff with one hand versus two hands are different, but so are the friction and shear forces (which is relevant when the controller can convey shear or skin-stretch feedback). Likewise, the vibration feedback will also be affected when the staff is hit depending on whether one or two hands grasp the staff. For example, the hand closest to the point of impact in a two-handed interaction could be fed back a larger vibration magnitude than the hand that is farther away from the collision point on the staff due to attenuation of the vibration as it travels through the staff In addition to relative hand configuration of controllers when interacting, selectively constraining relative motion of or between controllers can also be advantageous.
Designing the mechanical connection between the controllers to have specific degrees of freedom (DOF) can be used to tailor the input to the System (VR, AR, telerobot, toy, or gaming systems). Allowed degrees-of-freedom of motion at the mating interfaces can be used as an input to the use case. For example, a rotation about an axis of the controllers could be used to simulate the throttle on the handle bars of a motor cycle. Allowed degrees-of-freedom of motion at the mating interfaces can also be used to accommodate ergonomic variation among people as well as variations in relative hand pose between the controllers. The user moves the controllers while the controllers are mated without the need to re-grip or shift the user's grip on the controllers.
The orientation and location of the mate-points on the controller can be placed in an ergonomic mating location where the hands naturally come together in a particular body/arm pose so that the controllers are comfortable to hold in this position and so that it is advantageous for finding the mate-points without directly looking at one's hands (“blind mating”). For example, in at least one embodiment, a user may be wearing a head-mounted display that completely occludes the user's view. In such a case, the user is still able to mate controllers without looking at his/her hands. The use of magnets and guiding geometric features, such as a peg and mating conical hole (or the use of other passive or active features) can further aid the user in mating the controllers without directly looking at their hands or the controllers. Furthermore, magnets can help the user maintain a connection between the controllers when the controllers are mated, which is especially helpful when the user is moving his/her arms. To avoid magnets from making contact with other objects in the environment, especially ferrous magnetic materials, steel or other ferrous material may be used on one side of the magnetic guiding interface. Placing the magnet in the most isolated or female side of the magnetically guided mating features may be desirable for this reason, with the mating half of the interface utilizing a ferrous material component for making contact with the mating magnet.
In addition, the mate-points should not interfere with the person's arms or with hand/arm motions. At the same time, the attachment points can be placed as close as possible to the middle of their hands so as to promote better mating accuracy (especially for “blind mating”) since people's proprioception for bringing their hands together (i.e. mating their hands) is much better when making contact between their hands (or a point on their body) as opposed to mating a point that is remote and outside of the actual hand. Furthermore, the attachment points should place the user's hands in a comfortable ergonomic pose once mated while keeping the user's hands space far enough apart to not interfere or collide with the body of the controller on the opposite hand or items in the environment (in the case of a mate-point in the environment). As a compromise between the requirements of not interfering with the user's hand or natural hand motions, being in an ergonomic pose once mated, and keeping attachment points as close to the center of the hand as possible, placing attachment points in-line with a point slightly above the user's hand provides clearance for the user's forearm (e.g., in a machine gun pose) while still being along a natural trajectory for the hands to approach each other for mating the controllers together. The mate-points can be placed above the user's hands, but their hands still approach each other as if they are making contact between their hands with the mate-points in line with the approach trajectory of their hands.
The general locations for mate-points on controllers of main interest for interaction with VR, AR, robotic, toy, and gaming systems include, but are not limited to, the top, bottom, front, back and medial side of the controller (medial meaning facing the centerline of a person, i.e. the left side of the right controller and right side of the left controller. The medial side connections form a side-by-side configuration similar to a gamepad). Note that because of human ergonomics, limb configuration, kinematics, and motion, the exact location and orientation of the mate-points should be tailored as mentioned above and as illustrated in the examples that follow. For example, when mating two controllers using the medial mate-points (i.e. side-by-side configuration) to form a configuration and pose similar to a modern 2-handed game pad, the mate-points should allow the users hands and wrists to be angled toward the centerline similar to the angle of their forearms, just as modern game controllers do. Such a mating configuration could be used to mimic a steering wheel, flight yoke, deck machine gun, game pad, etc.
Attaching the front mate-point of the right controller to the back mate-point of the left controller (i.e. front-to-back) requires similar ergonomic considerations. That is, aligning the forward axis of the individual controllers to each other would be uncomfortable for the user's wrists, so having a mate-point on the front of the right controller attach to a mate-point projecting out of the left controller that is oriented toward the back and right provides a more comfortable pose when the controllers are mated and a more natural location for the user to reach out toward when mating the controllers. The front-to-back mating configuration could be used to mimic a machine gun, flame thrower, endoscope, saw, etc.
Mating controllers top-to-top forms an interface that resembles a user's hand pose on a bicycle or motor cycle handlebar, bow staff, chin-up bar, weight-lifting bar, lawn mower handle, shopping cart, baby stroller, bob sled, snow mobile, 4-wheeler motor vehicle, etc. These mate interfaces can stick directly out of the controller top but may result in a more comfortable pose if the mounts allow the user's hands to angle downward as they extend away from the center.
Mating controllers in a top-to-bottom configuration resembles the hand pose when using a shovel, 2-handed sword, fighting staff, spear, pole-arm, flag, etc. Note that the comfort of this mated configuration can be improved by rotating (clocking) this mate interface about a rotation axis along the length of the controller to allow the user's wrists to more closely follow the angle of their forearms, i.e. angle outward toward their shoulders.
The controller configurations described above are purely for example and do not limiting the scope of this invention. They are meant to be purely illustrative and provide tangible examples of the invention in practice
Furthermore, the releasing (or allowing) of a degree of freedom between the controllers in the above device configurations or when attached to the environment provides an opportunity for additional input to the System and can provide a very natural input for the user (e.g., people are used to the notion of turning the grip on a motor cycle to go faster). While tracked, tetherless controllers are versatile and can be used to represent most interactions, the presence of some mechanical structure to confine the motions of an interaction can aid in adding realism, immersion, and acceptance of the user.
The allowed degrees of freedom could be sensed either by tracking the controllers relative to each other, tracking the controllers relative to the environment mate-point (e.g., through a potentiometer on a gimbal), or by tracking the controllers individually and calculating the relative motion with respect to each other or the environment. The allowed degrees of freedom between controllers or between the controller and the environment or accessory can be tailored as desired through the use of appropriate mechanical designs at the mate-points or as an accessory added between the mate-points, e.g., revolute joints, universal joints, ball joints, prismatic joints, combinations thereof, etc.
An example of a revolute degree of freedom input would be for use as a motor cycle throttle input when controllers are mated in a top-to-top configuration and rotation is allowed along the length of the handle (i.e. the handlebar axis). A revolute degree of freedom could also be isolated and used between controllers in the side-by-side (medial mate-point) game-pad configuration. This degree of freedom could be used again as a throttle input or in place of tank-style steering (which is typically accomplished with side-by-side levers). An example of a revolute degree of freedom used as an input in the front-back mating between controllers could be as a control valve for a flame thrower, where the relative rotation about the rotational degree of freedom is approximately parallel with the axis of the barrel of the flamethrower. Rotating one direction could increase the flame thrower output and rotating the opposite direction could reduce the output.
In addition to allowed degrees of freedom being useful as an additional input, they can also be useful for accommodating ergonomic differences between people based on their limb size, etc. For example, the angle between users' hands when they bring them together may be different based on the size of the person.
Furthermore, allowed degrees of freedom between the controllers, accessory, or the environment can also accommodate the change in arm/hand angle when the user's hands grasp the joined controllers and the user moves through the range of motion of their arms. For example, when joining one's hands as if praying and then moving their hands toward or away from their chest the angle between their forearms becomes smaller and more parallel (when viewed from an overhead view). The same angle change occurs when a user grabs a single handle, e.g., a two-handed sword handle, and moves through a range of motion. In the case of the 2-handed sword, the user's hands must slip around/about the handle as the move through a range of motion to avert uncomfortable wrist strain. When controllers are grasped by the user and mated together, the same relative angle change of their arms occurs. While some of this relative rotation can be accommodated by rotation at the wrists, having a rotational degree of freedom at the attach points can provide a more comfortable experience for the user and not require them to regrip the device (regripping may not be advantageous as it can add an offset to the tracked position of the user's controller or could impact some types of haptic feedback).
While allowing specific degrees of freedom between the controllers or controller and accessory/environment can be advantageous for ergonomic or input reasons, it can lead to a loss of how “solidly” the users perceive the controllers are connected, which can reduce their belief that the joined controllers feel like the object they are controlling or seeing in the virtual, augmented, game, robotic, or teleoperated experience. Travel stops can be implemented to confine the total rotation or translational motion allowed at these mate-points. These travel stops can provide an intuitive sense of range and allow users to quickly map the range of “fully on” and “fully off” when controlling programmed tasks. By placing the travel stops near the end of range that a user naturally holds their hands, the user can bias their hand pose to push or rest against these stops (e.g., resting against a rotational stop with top-to-bottom mated controllers while holding a virtual shovel), which can restore the sense that the user is holding a rigid object, despite rotation being allowed between the controllers. Another example would be to place rotational stops for the front-to-back machine gun mate-points such that one rational stop near the natural pose when the user's hands are resting near their body, and place the rotational stop at the other end of the rotation range to correspond with their relative wrist rotation when they have raise their “virtual gun” and are looking down its barrel.
The “feel” of the allowed degrees-of-freedom at the mate interfaces can also be tailored through the addition of mechanical features or one or more components such as, but not limited to, magnets, dampers, springs, and/or actuators within the mate interface or within an accessory placed in or between the mate interfaces. These additional features can be used to create the sense of viscosity or springiness when moving the controllers relative to each other or the accessory/environment on the allowed degree of freedom, or may give an indication of the center position or preferred position between travel stops on the mating interface. As an example, this center or preferred position on the allowed degree of freedom can be indicated to the user by a bi-stable mechanism such as the spring-cam interface found within house-hold light switches. A spring-loaded ball detent sliding on the surface of the mating interface could also be used to indicate a preferred position. Pairs of magnets on the mating interface could also indicate preferred position along the allowed degree(s) of freedom of the mate-point. Pairs of magnets can also be added at the travel stops (or other pre-determined locations) of the allowed degree(s) of freedom to make the ends-of-travel (or other pre-determined locations) feel sticky and/or make the mated controllers feel more stable or rigid when held in this orientation. The feel of the motion at the mate interface can also be tailored by controlling the surface friction or texture at the mate-point interface. All of the above effects can also be portrayed by placing a controlled actuation mechanism at or in between the mating interfaces to provide centering, springy, damped, sticky, rigid, or other simulated behavior of the allowed degree of freedom of the mate-point.
Being able to directly see the controllers during mating, or providing guidance in VR environments can further aid in the mating of controllers. However, if the tracking system used in the VR system has tracking inaccuracies in it, only providing the coarse features is more advantageous than providing detailed representation and location of mate-points. If a more coarse (lower resolution) representation is used, magnetic or geometric features can be effective in aiding the user with mating the controllers to each other or the environment. However, if the exact mate-point is represented and tracking accuracy is poor, the user's increased confidence in exactly where to move the controllers can place their actual position outside of the range in which the magnetic or geometric guiding features can aid them, and/or they may fight against these guiding features since they seem to be guiding them away from where they need to move to mate the controller(s).
In some cases, it can be advantageous to mate the controller before an interaction session. In this case, the use of more heavy-duty mate-point interconnects is possible, as opposed to lighter-duty interfaces that are easily separable on-the-fly. A stronger, heavy-duty mount can be made to feel like a permanent connection and replace the need to purchase separate peripherals for specific use cases (e.g., gun peripheral, steering wheel peripheral, etc.). The heavy-duty interfaces could utilize traditional screws, bolts, or other fasteners and more conveniently would employ designs that can still be quickly released, such as a twist- or slide-and-lock interface with a release lever, quarter-turn lug design, etc.
A heavy-duty interface can also make it convenient to provide electrical connections between the controllers, attachments, and/or environmental mate-points. These electrical signals could include power, communication lines (e.g., wired serial, nearfield serial communication, RFID tags), etc. Standard electrical interfaces such as pogo pins, spring pins, slip rings, etc. could be utilized to connect electrical signals across the mated interfaces. Electrical signals could also be connected across light-duty on-the-fly mating interfaces (e.g., power and ground, which could be used to recharge the controller device while mated with an environment mate-point such as a dashboard, steering wheel, shifter, or joystick).
The mated configuration of the controllers can be detected by on-controller sensors (e.g., continuity, magnet/hall effect pairs, or optical emitter/detector pairs) or by tracking the locations of each of the controllers (e.g., through optical, inertial, magnetic tracking, or a hybrid of these tracking methods) and deducing that a mated connection has been made between the controllers based on their relative pose and proximity. In such a case, it is not necessary for any sensors that are internal to the controllers to directly detect the physical mating. Instead, an external tracking system and/or positional sensors within the controllers can detect that the controllers are being held relative to each other in such a pose that it is inferred that they are mated. A magnet on the controllers at the mating interfaces can help ensure a stable connection between the controllers. In the case where a connection is inferred between the controllers based on tracking the controller individually, placing a magnet at the mating interfaces can improve the prediction that controllers are mated.
A large variety of mate-points can be utilized, including but not limited to the interfaces shown in
A simple mate-point (e.g., 100a, 100b, 100h, 100i) resembles a peg-in-hole (or peg and hole) interface, as shown in
In addition to simply providing a connection, mate-points can be used to specifically control the allowed motions (degrees of freedom) once mated. In particular, the light-duty mate-point 100a in
In general, the relative motion input can be sensed by sensors embedded in the controller, accessory, or environment using a potentiometer, encoder, hall-effect sensor or other means known in the art. The relative motion of the mate-point can also be sensed by sensing/tracking the position/orientation of the individual controllers, accessories, or environment mate-points and calculating the relative motion. This tracking can be accomplished via optical tracking, inertial tracking, inductive tracking, capacitive tracking, magnetic tracking, hybrids of these tracking methods, or other methods.
In addition to using the mate-point allowed degrees of freedom as a system input, the allowed degrees of freedom can also be used for accommodating ergonomic variation (e.g., size) across users. The allowed degrees of freedom can also allow more comfortable ergonomics when mating the controllers and moving them within the user's range of motion with their arms. Allowing relative motions between the controllers permits a user to maintain his/her grip on the controllers without the need to regrip or slide their grip (regripping may not be advantageous when haptic feedback is provided through the controller).
The mate-point design 100c shown in
The mate-point design 100d shown in
As seen in
Whereas all of the preceding mate-point designs have utilized asymmetric interface designs (e.g., male peg, female hole), it can be advantageous to have a mate-point which utilizes the same geometry on both sides of the interface (i.e. a hermaphroditic interface design). This is advantageous as this allows these hermaphroditic interfaces to be placed without regard to foreknowledge of how they will be combined and mated.
Another example of a light-weight hermaphroditic mate-point 100g is shown in
In contrast, the mate-point design shown in
The interface design 100i shown in
In additional to light-duty interfaces, which a user can easily mate on-the-fly, heavy-duty interfaces can also be useful for attaching multiple controllers together or attaching controllers to accessories or the environment in a more permanent and/or rigid manner than possible with interfaces that can be easily mated on-the-fly or even blind-mated. An example of a heavy-duty hermaphroditic interface 100j is shown in
Another example of a heavy-duty mate-point 100k is shown in
In contrast to the prior heavy-duty mate-points,
Several other light-duty and heavy-duty designs could be utilized.
The peg-in-hole mate-point designs (see
Furthermore, these two accessories utilize male mate-points on the accessory for one controller (the right controller as shown in
Game controllers (e.g., those shown in
In various embodiments, the sensor group may take different forms. For example, the sensor group may comprise a pose tracking system 2230 that is external to the first controller 2000 and the second controller 2010. The pose tracking system 2230 may also be internal to the first controller and second controller. The pose tracking system 2230 may also have components to it that are internal and external to the first controller 2000 and second controller 2010. The pose tracking system 2230 may also track the accessories and environment mate-points used with this system. The position and orientation of these accessories and environment mate-points may also be fixed or anchored in a known location, such that actively tracking their position and orientation is not necessary to recognize when a controller is in a mating configuration with the accessory or environment mate-point. The device position and orientation tracking system can include optical or image tracking equipment (e.g., cameras), depth sensors, LIDAR sensors, or any other conventional system used for tracking the pose of a user and/or user controllers. In additional or alternative embodiments, the sensor group may comprise a first set of sensors integrated within the first controller and a second set of sensors integrated within the second controller. The first set of sensors and the second set of sensors may both comprise IMUs embedded within the respective controllers. Each IMU may comprise various motion detecting sensors. Additionally, the first set of sensors and the second set of sensors may comprise GPS, optical sensors, sonars, magnetometers, and other similar intra-device sensors that are useable for tracking relative position and movement of a controller.
In at least one embodiment, an I/O module 2244 within the computer system receives the sensor data from the sensor group and processes it with a processing unit 2242. The sensor data may be received through wired communication, wireless communication, or through direct observation by the computer system. Further, in at least one embodiment, the computer system 2240 is at least partially integrated within both the first controller 2000 and the second controller 2010. In such an embodiment, processing units within the first controller and the second controller perform at least some of the processing described herein.
Once the sensor data is received, the computer system 2240 determines that the first pose and the second pose map to a first pose profile selected from a set of pose profiles that are within storage 2246. Each profile within the set of profiles maps to a particular configuration of mated controllers. For example, the first pose of the first controller may indicate that it is being held vertically aligned with a longitude plane of the user. The second pose of the second controller may indicate that it is also being vertically aligned with a longitude plane of the user directly, but above the first controller. Based upon this received sensor data from the sensor group, the computer system maps the first pose and the second pose to a pose profile that indicates that user is treating the two separate controllers as if they were a single controller mated together in a vertically aligned fashion (e.g., like a bow staff or two-handed sword). Examples of other possible pose profiles are provided herein and include poses such as a machine gun pose and a handle bar pose.
Once the first pose profile is identified, the computer system activates the first pose profile. The first pose profile comprises an input configuration file that is unique to the particular configuration of mated controller. For example, the IMU data from each controller will be interpreted differently when the two controllers are treated as being joined then when the two controllers are separate. Additionally or alternatively, in at least one embodiment, the first pose profile also comprises a haptic feedback configuration file (as shown in
Accordingly, embodiments disclosed herein are capable of using sensor data about the position of a single controller or about two controllers (e.g., handheld controllers) to determine if the controllers are being held in relative poses that indicate the use of joining (or connecting or mating) the two controllers. As explained above, in at least one embodiment, the controllers are physically mated together using a connector. However, in at least one embodiment, the physical connection is not required for the computer system to determine that the user is treating the two controllers as if they are mated. For example, the computer system can determine that the pose of each controller relative to the user is such that the user is holding the controllers in accordance with a particular pose profile. The computer system can then activate the corresponding pose profile. Such a system allows a user to easily transform two separate controllers into a single device within a teleoperation environment, virtual reality (VR) environment, augmented reality (AR) environment, toy environment, gaming environment, or other environment.
More generally, a connection sensor 2202, 2222 can be integrated into the device housing, wherein the connection sensor could include at least one of an inertial measurement unit (IMU), a contact switch, electrical serial communication, electrical parallel communication, a magnetic sensor, a capacitive sensor, an inductive sensor, an optical sensor, or an RFID tag. Placing a connection sensor 2202, 2222 at each mate-point can be used to sense which of a plurality of connection locations on a controller is connected to another controller or mate-point in the environment. This approach can be advantageous as it doesn't require explicit position tracking of the device to determine if the devices are mated. Connection sensors 2202, 2222 at the mate-points, near the mate-points, or elsewhere in/on the device can also be used to measure relative position and orientation between mated controllers or a controller and an environment mate-point.
The terms “approximately,” “about,” and “substantially” as used herein represent an amount close to the stated amount that still performs a desired function or achieves a desired result. For example, the terms “approximately,” “about,” and “substantially” may refer to an amount that is within less than 10% of, within less than 5% of, within less than 1% of, within less than 0.1% of, and within less than 0.01% of a stated amount.
The present invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described embodiments are to be considered in all respects only as illustrative and not restrictive. The scope of the invention is, therefore, indicated by the appended claims rather than by the foregoing description. All changes that come within the meaning and range of equivalency of the claims are to be embraced within their scope.
This application claims priority to and the benefit of U.S. Provisional Application Ser. No. 62/564,096 entitled “RECONFIGURABLE CONTROLLER DEVICES, SYSTEMS, AND METHODS”, filed on Sep. 27, 2017, the entire content of which is incorporated by reference herein in its entirety.
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/US18/52750 | 9/25/2018 | WO | 00 |
| Number | Date | Country | |
|---|---|---|---|
| 62564096 | Sep 2017 | US |
