Patent Grant
9939902
The embodiments are generally directed to electronic devices, and more particularly, to electronic devices that produce haptic effects.
Video games and video game systems have become extremely popular. Video game devices or controllers typically use visual and auditory cues to provide feedback to a user. In some interface devices, kinesthetic feedback (e.g., active and resistive force feedback) and/or tactile feedback (e.g., vibration, texture, temperature variation, and the like) may be provided to the user. In general, such feedback is collectively known as “haptic feedback” or “haptic effects.” Haptic feedback provides cues that enhance and simplify a user's interaction with a video game controller, or other electronic device. For example, haptic effects may provide cues to users of video game controllers or other electronic devices to alert the user to specific events, or provide realistic feedback to create greater sensory immersion within a simulated or virtual environment.
Other devices in which a user interacts with a user input element to cause an action also may benefit from haptic feedback or haptic effects. For example, such devices may include medical devices, automotive controls, remote controls, and other similar devices.
Embodiments of the present invention are directed toward electronic devices configured to produce haptic effects that substantially improve upon the related art.
Features and advantages of the embodiments are set forth in the description which follows, or will be apparent from the description, or may be learned by practice of the invention.
In one example, a controller having a plurality of surfaces, and a haptic output device located within its housing and coupled to an isolated deformable region disposed at one of the plurality of surfaces is provided. The example isolated deformable region may expand and/or contract in response to the haptic output device.
In another example, a controller having a plurality of isolated surface regions, and a plurality of haptic output devices located within its housing and coupled to respective isolated surface regions is provided. Each of the example isolated surface regions may be configured to provide localized haptic effects.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not intended to limit the invention to the described examples.
Further embodiments, details, advantages, and modifications will become apparent from the following detailed description of the preferred embodiments, which is to be taken in conjunction with the accompanying drawings.
Reference will now be made in detail to embodiments, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it will be apparent to one of ordinary skill in the art that the present invention may be practiced without these specific details. In other instances, well-known methods, procedures, components, and circuits have not been described in detail so as not to unnecessarily obscure aspects of the embodiments. Wherever possible, like reference numbers will be used for like elements.
In the various embodiments, a variety of user interfaces and methods for using a device are described. In some embodiments, the device is a portable electronic device (e.g., a game controller, console, mobile phone, smartphone, tablet, etc.). It should be understood, however, that the user interfaces and associated methods may be applied to numerous other devices, such as personal computers, medical devices, laptops, and the like that may include one or more other physical user-interface devices, such as a keyboard, mouse, trackball, joystick, motion and orientation sensing devices, and the like.
System 100 may include a communication device 110 configured to transmit and/or receive data from remote sources. Communication device 110 may enable connectivity between processor 120 and other devices by encoding data to be sent from processor 120 to another device over a network (not shown) and decoding data received from another system over the network for processor 120.
For example, communication device 110 may include a network interface card that is configured to provide wireless network communications. A variety of wireless communication techniques may be used including infrared, radio, Bluetooth, Wi-Fi, and/or cellular communications. Alternatively, communication device 110 may be configured to provide wired network connection(s), such as an Ethernet connection.
Processor 120 may comprise one or more general or specific purpose processors to perform computation and control functions of system 100. Processor 120 may include a single integrated circuit, such as a micro-processing device, or may include multiple integrated circuit devices and/or circuit boards working in cooperation to accomplish the functions of processor 120. In addition, processor 120 may execute computer programs, such as operating system 141, isolated haptic module 142, and other applications 143, stored within memory 140.
System 100 may include memory 140 for storing information and instructions for execution by processor 120. Memory 140 may contain various components for retrieving, presenting, modifying, and storing data. For example, memory 140 may store software modules that provide functionality when executed by processor 120. The modules may include an operating system 141 that provides operating system functionality for system 100. The modules may further include isolated haptic module 142 that modifies a haptic effect experienced at isolated surface regions of controller 150. In certain embodiments, isolated haptic module 142 may comprise a plurality of modules, where each module provides specific individual functionality for modifying a haptic effect experienced at an isolated surface region. System 100 also may include one or more additional application modules 143 that include additional functionality, such as peripheral firmware configured to provide control functionality for a peripheral device, such as controller 150 (e.g., a gamepad).
Non-transitory memory 140 may include a variety of computer-readable medium that may be accessed by processor 120. In the various embodiments, memory 140 may include volatile and nonvolatile medium, removable and non-removable medium. For example, memory 140 may include any combination of random access memory (“RAM”), dynamic RAM (DRAM), static RAM (SRAM), read only memory (“ROM”), flash memory, cache memory, and/or any other type of non-transitory computer-readable medium.
Although shown as a single system, the functionality of system 100 may be implemented as a distributed system. For example, memory 140 and processor 120 may be distributed across multiple different computers that collectively comprise system 100. In one embodiment, system 100 may be part of a device (e.g., personal computer, console, video game console, etc.), and system 100 provides isolated haptic effect functionality for the device. In another embodiment, system 100 may be separate from the device, and may remotely provide the aforementioned functionality for the device.
System 100 may be operably connected to controller 150. Controller 150 may be a peripheral device configured to provide input to the system 100. Controller 150 may be operably connected to system 100 using either a wireless connection or a wired connection. Controller 150 also may include a local processor configured to communicate with system 100 using either a wireless connection or a wired connection. Alternatively, controller 150 may be configured to not include a local processor, and all input signals and/or output signals associated with controller 150 may be processed by the components of system 100.
Controller 150 may further include one or more digital buttons, one or more analog buttons, one or more bumpers, one or more directional pads, one or more analog or digital sticks, one or more driving wheels, and/or one or more user input elements that can be interacted with by a user, and that can provide input to system 100. Controller 150 may also include one or more analog or digital trigger buttons (or “triggers”) that can further be interacted with by the user, and that can further provide input to system 100. As is described below in greater detail, controller 150 can further include a motor, or another type of actuator or haptic output device, configured to exert a bi-directional push/pull force on at least one trigger of controller 150.
Controller 150 can also include one or more actuators, or other types of haptic output devices. The local processor of controller 150, or processor 120 in embodiments where controller 150 does not include a local processor, may transmit a haptic signal associated with a haptic effect to at least one actuator of controller 150. The actuator, in turn, outputs haptic effects such as vibrotactile haptic effects, kinesthetic haptic effects, or deformation haptic effects, in response to the haptic signal. The haptic effects can be experienced at a user input element (e.g., a digital button, analog button, bumper, directional pad, analog or digital stick, driving wheel, or trigger) of controller 150. Alternatively, the haptic effects can be experienced at an outer surface of controller 150.
The actuator may be, for example, an electric motor, an electro-magnetic actuator, a voice coil, a shape memory alloy, an electro-active polymer, a solenoid, an eccentric rotating mass motor (“ERM”), a harmonic ERM motor (“HERM”), a linear resonant actuator (“LRA”), a piezoelectric actuator, a high bandwidth actuator, an electroactive polymer (“EAP”) actuator, an electrostatic friction display, or an ultrasonic vibration generator. In some instances, the actuator may include an actuator drive circuit.
In some embodiments, the actuator design may be configured to create directional inertial haptic effects, such as haptic effects produced along an axis. Here, multiple magnetic components may be suspended in a gel such that two sets of magnets may cause the magnetic components to move along two independent directions.
An actuator is an example of a haptic output device, where a haptic output device is a device configured to output haptic effects, such as vibrotactile haptic effects, electrostatic friction haptic effects, temperature variation, and/or deformation haptic effects, in response to a drive signal. In alternate embodiments, the one or more actuators within controller 150 can be replaced by some other type of haptic output device. In some embodiments, a speaker located within controller 150 and the like may be configured as a haptic output device. In some alternative embodiments, haptic output devices may be configured to provide both audio and vibration outputs.
Controller 150 may further include one or more speakers. The local processor of controller 150, or processor 120 in embodiments where controller 150 does not include a local processor, may transmit an audio signal to at least one speaker of controller 150, which in turn outputs audio effects. The speaker may be, for example, a dynamic loudspeaker, an electrodynamic loudspeaker, a piezoelectric loudspeaker, a magnetostrictive loudspeaker, an electrostatic loudspeaker, a ribbon and planar magnetic loudspeaker, a bending wave loudspeaker, a flat panel loudspeaker, a heil air motion transducer, a plasma arc speaker, and a digital loudspeaker.
Controller 150 can further include one or more sensors. A sensor may be configured to detect a form of energy, or other physical property, such as, but not limited to, sound, movement, acceleration, bio signals, distance, flow, force/pressure/strain/bend, humidity, linear position, orientation/inclination, radio frequency, rotary position, rotary velocity, manipulation of a switch, temperature, vibration, or visible light intensity. The sensor may further be configured to convert the detected energy, or other physical property, into an electrical signal, or any signal that represents virtual sensor information, and controller 150 can send the converted signal to the local processor of controller 150, or processor 120 in embodiments where controller 150 does not include a local processor.
As illustrated in
Controller 200 may include local process 208. Local processor 208 may exchange commands and data with host computer 204 via connection 205. Connection 205 may be a wired or wireless connection using one or more communication protocols known to those skilled in the art. In some instances, controller 200 may be alternatively configured to not include local processor 208. Here, input/output signals from controller 200 may be handled and processed directly by host computer 204. Host computer 204 may be a gaming device console and display device 206 may be screen which is operably coupled to the gaming device console. In some instances, host computer 204 and display device 206 may be combined into a single device.
Controller 200 may include targeted actuators 212, 216, 220 (e.g., motors) to directly drive each of the user input elements thereof as well as one or more general or rumble actuators 222, 224 operably coupled to housing 202 in a location where a hand of the user is generally located. More particularly, analog or digital stick 210 includes a targeted actuator or motor 212 operably coupled thereto, button 214 includes a targeted actuator or motor 216 operably coupled thereto, and trigger 218 includes a targeted actuator or motor 220 operably coupled thereto. In addition to a plurality of targeted actuators, controller 200 includes a position sensor operably coupled to each of the user input elements thereof. More particularly, analog or digital stick 210 includes a position sensor 211 operably coupled thereto, button 214 includes a position sensor 215 operably coupled thereto, and trigger 218 includes a position sensor 219 operably coupled thereto. Local processor 208 is operably coupled to targeted actuators 212, 216, 220 as well as position sensors 211, 215, 219 of analog or digital stick 210, button 214, and trigger 218, respectively. In response to signals received from position sensors 211, 215, 219, local processor 208 instructs targeted actuators 212, 216, 220 to provide directed or targeted kinesthetic effects directly to analog or digital stick 210, button 214, and trigger 218, respectively. Such targeted kinesthetic effects are discernible or distinguishable from general or rumble haptic effects produced by general actuators 222, 224 along the entire body of the controller. The collective haptic effects provide the user with a greater sense of immersion to the game as multiple modalities are being simultaneously engaged (e.g., video, audio, and haptics).
Device modules 310 may include a variety of modules such as input management code 311, peripheral input application programming interface (“API”) 312, rumble API 313, isolated haptic effect API 314, direct playback/crossover 315, isolated surface engine 316, spatialization engine 317, and encoder 318.
Input management code 311 may include a set of computer-readable instructions that manage input provided by controller 330 in the context of a game application, or other type of application, executed within a device.
Peripheral input API 312 may include a set of computer-readable functions or routines that enable game input management code 311 to interact with peripheral firmware 320 in order to receive and manage input provided by controller 330.
Rumble API 313 may include a set of computer-readable functions or routines that enable input management code 311 to interact with peripheral firmware 320 in order to transmit rumble instructions to one or more rumble motors or rumble actuators of controller 330 (e.g., rumble motors L and R of
Isolated haptic effect API 314 (identified in
Isolated haptic effect API 314 also may store one or more haptic effect definitions. A haptic effect definition is a data structure that includes haptic data, such as a haptic signal, that is pre-defined and that can be stored within a storage, such as a haptic file or haptic stream, and that can be sent to one or more rumble motors, rumble actuators, targeted motors, or targeted actuators, to produce a haptic effect at a component, or user input element, of controller 330. The haptic data can include one or more attributes of the corresponding haptic effect, where the attributes can be stored as parameters. Example parameters of a haptic effect definition may include an amplitude parameter, a frequency parameter, a waveform parameter, an envelope parameter, a magnitude (or strength) parameter, and a duration parameter.
Isolated haptic effect API 314 may enable game input management code 311 to interact with direct playback/crossover 315, trigger engine 316, and spatialization engine 317, and may further manage direct playback/crossover 315, trigger engine 316, and spatialization engine 317 according to requests invoked by game input management code 311. Further, isolated haptic effect API 314 may store data used for communication with peripheral firmware 320, and used for generation of one or more isolated haptic effects.
Direct playback/crossover 315 may receive haptic data as input, produce haptic data as output, and transmit haptic data to one or more targeted motors, or targeted actuators, of controller 330 (e.g., motors L and R of
The format of the haptic data may be a haptic elementary stream (“HES”) format. A HES format is a file or data format for representing haptic data that may be streamed to a device. The haptic data can be represented in a manner that is identical or similar to how uncompressed sound is represented, although the haptic data can be encrypted within the HES format.
Isolated surface engine 316 may receive haptic data, such as an isolated surface haptic effect definition, and may modify the haptic data based on user input data, such as trigger data 323. Trigger data is data that includes one or more parameters that indicate a position and/or range of one or more triggers of controller 330 (e.g., triggers L and R of
Spatialization engine 317 may receive haptic data, such as an isolated surface haptic effect definition, and may modify the haptic data based on spatialization data. Spatialization data may include data that indicates a desired direction and/or flow of a haptic effect, such as an ordering of haptic effects on respective user input elements and/or isolated surfaces. In certain embodiments, spatialization engine 317 may receive spatialization data that includes a direction and/or flow from input management code 311.
Spatialization engine 317 may modify the haptic data so that a haptic effect, such as a trigger haptic effect, is scaled for one or more rumble motors, or rumble actuators, of controller 330 (e.g., rumble motors L and R of
Encoder 318 encodes haptic data received from direct playback/crossover 315, isolated surface engine 316, and/or spatialization engine 317 into a format. In one embodiment, the format may be an HES format. Encoder 318 may transmit the encoded haptic data to peripheral firmware 320.
Peripheral firmware 320 is firmware for one or more peripheral devices (e.g., controllers). Peripheral firmware 320 may include a variety of modules such as decoder and crossover 321, trigger control 322, trigger data 323, other functions 324, and rumble control 325.
Decoder and crossover 321 may receive the encoded haptic data from encoder 318 and decodes the encoded haptic data. In some embodiments, decoder and crossover 321 computes a programmable crossover in order to decode the encoded haptic data. Decoder and crossover 321 may compute the programmable crossover in real-time.
Trigger control 322 is a low-level control API for one or more targeted motors or targeted actuators of controller 330 (e.g., motors L and R of
Trigger data 323, as previously described, is data that includes one or more parameters that indicate a position and/or range of one or more triggers of controller 330 (e.g., triggers L and R of
Other gamepad functions 324 may be functions of controller 330 managed by peripheral firmware 320. Such functions may include such functions as wired/wireless communications, input reporting, protocol implementation, power management, etc.
Rumble control 325 is a low-level control API for one or more rumble motors or rumble actuators of controller 330 (e.g., rumble motors L and R of
Controller 330 may include triggers L and R. Controller 330 may further include gear boxes L and R and motors L and R. Motor L and gearbox L are operably coupled to trigger L within controller 330. Likewise, motor R and gearbox R are operably coupled to trigger R within controller 330. When motor L receives a trigger instruction, motor L and gearbox L may collectively cause a trigger haptic effect to be experienced at trigger L. Likewise, when motor R receives a trigger instruction, motor R and gearbox R may collectively cause a trigger haptic effect to be experienced at trigger R. Peripheral firmware 320 may send trigger instructions to motors L and R of controller 330 using drive electronics 340.
Controller 330 may further include potentiometers L and R. Potentiometer L may detect a position and/or range of trigger L, and may further send the detected position and/or range of trigger L to peripheral firmware 320 as trigger data. Likewise, potentiometer R may detect a position and/or range of trigger R, and may further send the detected position and/or range of trigger R to peripheral firmware 320 as trigger data.
Controller 330 may further include rumble motors L and R. When rumble motor L receives a rumble instruction, rumble motor L causes a haptic effect to be experienced along a left component of controller 330. Likewise, when rumble motor R receives a rumble instruction, rumble motor R causes a haptic effect to be experienced along a right component of controller 330. Peripheral firmware 320 may send rumble instructions to rumble motors L and R using rumble drive electronics 350.
Housing 402 is shaped to easily accommodate user gripping of controller 400. Controller 400 is an example embodiment of a controller, and the embodiments of the invention may be readily applied to other controller shapes.
Isolated surface 532 may enable isolated portions of controller 500 to be deformed by expanding and/or contracting. Although deformation haptic effects are being described, the embodiments are not so limited. Numerous other haptic effects may alternatively or additionally be applied to isolated surface 532. Example haptic effects include expansion, contraction, vibration, texture, temperature variation, rumble, and the like.
Use of isolated surface 532 provides advantages as compared to prior controllers. Here, the haptic effect may be isolated to a specific portion of controller 500 (e.g., left or right side) such that the user perceives the haptic effect only at the isolated surface 532 (i.e., not the whole controller, or not the entire controller housing 502). For example, use of isolated surface 532 may provide a wider range of haptic effects. In another example, use of isolated surface 532 provides improved spatialization. Here, both localized haptic effects as well as haptic effects that convey a sense of direction may be provided to a user (e.g., a more realistic effect of contacting a ball). In yet another example, use of isolated surface 532 may provide kinesthetic programmable haptics at surface(s) where the isolated surface 532 may expand and contract.
The various components of isolated surface structure 600 may be configured to isolate the haptic effects (e.g., vibration and/or deformations applied to floating section 620 from housing 602. Accordingly, floating section 620 is indirectly coupled to housing 602 by suspension flexures 611, 612. When a haptic effect is applied to floating section 620, suspension flexures 611, 612 substantially dampen the haptic effects such that they are not applied to the housing 602. Additionally, or alternatively, a rubber ring or gasket (not shown) may surround floating section 620 to dampen the haptic effects.
Motor 630 may drive a plurality of haptic components 631, 632 in order to apply haptic effects to floating section 620. In some embodiments, motor 630 may be configured to drive a plurality of pins (not shown) positioned along floating section 630. Here, haptic components 631, 632 or pins may be individually driven to provide a wide range of haptic effects.
In addition, motor 630 may comprise a variety of motor types. For use in controller 600, motor 630 is preferably small, low-power, and lightweight. Typically, isolated surface structure 600, including motor 630, is lightweight (e.g., approximately 35 grams or less).
Example electric motors may include an electro-magnetic actuator, a voice coil, a shape memory alloy, an electro-active polymer, a solenoid, an eccentric rotating mass motor (“ERM”), a harmonic ERM motor (“HERM”), a linear resonant actuator (“LRA”), a piezoelectric actuator, a high bandwidth actuator, an electroactive polymer (“EAP”) actuator, an electrostatic friction display, an ultrasonic vibration generator, and the like.
In some instances, motor 630 and/or actuator 640 may be grounded to isolated floating section 620. Isolated floating section 620 may be connected to the housing via the suspension mechanisms described herein or other mechanisms that dampen the haptic effects such that they are not applied to housing 602. Motor 630 and/or actuator 640 generate haptic effects and may be coupled directly to isolated floating section 620.
Although
By using discrete localized areas of the controller 700 that may be individually manipulated to provide one or more haptic effects, enhanced and full spatialization may be provided. Controller 700 may produce spatialized haptic effects in which localized haptic feedback may be experienced by the user. For example, a user may perceive localized haptic effects that are provided at one or more localized regions 711-716 the controller 700. Such localized haptic effects may be distinct from general haptic effects applied directly to the housing of the controller. In another example, haptic effects may be applied to a subset of localized regions 711-716 to convey a sense of direction to a user (e.g., left or right sensation).
In addition, haptic effects may be applied to a subset of localized regions 711-716 by mapping what is happening to an avatar (e.g., game character) to respective localized regions 711-716 of controller 700. For example, if the avatar is hit on the top left, then the controller 700 may have a corresponding haptic effect on the top left localized region. In another example, a user hit on the left side may feel haptic effects at the localized regions on the left side, such as localized regions 711, 714, and 716. In yet another example, a user's ship hit on the upper right side may feel haptic effects at localized region 713.
In some embodiments, isolated regions 711-716 may be mechanically arranged so that each portion of controller 700 may be vibrated separately. Here, multiple or all isolated regions 711-716 of controller 700 may be configured to apply a haptic effect, not just particular region(s). Isolated regions 711-716 also may be locked together such that a haptic effect may be applied to a subset of regions as one unit. In addition, the housing of the controller may be co-molded with various materials of varying stiffness properties in order to isolate the vibration to each section. In some instances, individual user elements such as joysticks, buttons, and the like may also be isolated from the different sections of the gamepad housing.
In some embodiments, controller 700 may be mechanically divided into isolated portions. For example, the controller 700 may be divided into 8 vibration isolated portions or octants. In this example, a haptic effect may be applied to an upper left octant and may be substantially isolated from the other octants. Here, isolated portions or octants may be interconnected by an active suspension having adjustable stiffness to interconnect the isolated portions. In other words, isolated sections may have variable dampening means between them in order to isolate haptic effects to particular portions(s) of controller 700. The stiffness may be increased in order to effectively couple two or more adjacent isolated portions of the controller into a single portion.
Use of isolated track pads 821, 822 dynamically changes the feel and purpose of track pads. In addition, use of isolated track pads 821, 822 enhances ergonomics and better enables pressure-sensitive applications.
Accordingly, enhanced haptic effects may be provided by adopting the various embodiments of the present invention. For example, isolated deformable regions that expand and contract in response to a haptic output device are provided. In another example, isolated surface regions may be configured to provide localized haptic effects are also described. Accordingly, improved spatialization via multiple discrete areas of a controller (e.g., gamepad) that can be separately vibrated is provided.
One having ordinary skill in the art will readily understand that the invention as discussed above may be practiced with steps in a different order, and/or with elements in configurations which are different than those which are disclosed. Therefore, although the invention has been described based upon these preferred embodiments, it would be apparent to those of skill in the art that certain modifications, variations, and alternative constructions would be apparent, while remaining within the spirit and scope of the invention. In order to determine the metes and bounds of the invention, therefore, reference should be made to the appended claims.
This application claims the benefits of U.S. Provisional Patent Application No. 62/078,502, filed on Nov. 12, 2014, which is incorporated herein by reference in its entirety.
| Number | Name | Date | Kind |
|---|---|---|---|
| 20040207542 | Chang et al. | Oct 2004 | A1 |
| 20050017947 | Shahoian et al. | Jan 2005 | A1 |
| 20080163051 | Olien | Jul 2008 | A1 |
| 20090085882 | Grant et al. | Apr 2009 | A1 |
| 20100283731 | Grant | Nov 2010 | A1 |
| 20110018696 | Ryu et al. | Jan 2011 | A1 |
| Number | Date | Country |
|---|---|---|
| 2508137 | May 2014 | GB |
| Number | Date | Country | |
|---|---|---|---|
| 20160132116 A1 | May 2016 | US |
| Number | Date | Country | |
|---|---|---|---|
| 62078502 | Nov 2014 | US |
