Patent Application
20150070129
The present specification generally relates to methods and systems for providing navigation assistance to a user and, more specifically, methods and systems for providing navigation assistance to a user based on tactile feedback or audible feedback indicative of a virtual wall.
Blind and visually impaired individuals often use physical structures, such as walls, as navigation aids. However, in an open space (a space in which a user cannot touch any surrounding structures with his or her person or tools), where these structures are not present (e.g., a field), navigation may become very difficult using traditional methods. Thus, it is desirable to provide navigation assistance to users in such open spaces in which physical navigation aids may not exist.
Accordingly, a need exists for alternative methods and systems for providing navigation assistance to a blind or visually impaired user.
In one embodiment, a method for providing navigation assistance to a user includes defining at least one virtual wall along at least a portion of a path for navigation through an environment. The at least one virtual wall is offset from the path by an offset distance. The method further includes determining a location of the user and providing tactile feedback to the user, automatically by a processor, when the location of the user is within a threshold distance of the at least one virtual wall.
In another embodiment, a system for providing navigation assistance to a user includes an electronic control unit. The electronic control unit includes a non-transitory electronic memory that stores a set of machine readable instructions and a processor for executing the machine readable instructions. The system further includes a tactile feedback device communicatively coupled with the electronic control unit. When executed by the processor, the machine readable instructions cause the system to define at least one virtual wall along at least a portion of a path for navigation through an environment. The at least one virtual wall is offset from the path by an offset distance. When executed by the processor, the machine readable instructions further cause the system to determine a location of the user and provide tactile feedback to the user with the tactile feedback device when the location of the user is within a threshold distance of the at least one virtual wall.
In yet another embodiment, a method for providing navigation assistance to a user includes defining at least one virtual wall along at least a portion of a path for navigation through an environment. The at least one virtual wall is offset from the path by an offset distance. The method further includes determining a location of the user and providing audible feedback to the user, automatically by a processor, when the location of the user is within a threshold distance of the at least one virtual wall.
These and additional features provided by the embodiments described herein will be more fully understood in view of the following detailed description, in conjunction with the drawings.
The embodiments set forth in the drawings are illustrative and exemplary in nature and not intended to limit the subject matter defined by the claims. The following detailed description of the illustrative embodiments can be understood when read in conjunction with the following drawings, where like structure is indicated with like reference numerals and in which:
The embodiments disclosed herein generally include feedback guidance methods and systems for providing navigation assistance to visually impaired users through tactile feedback or audible feedback. Referring generally to
Referring now to the drawings,
The feedback guidance system 100 comprises an electronic control unit 110 to which various components are communicatively coupled, as will be described in further detail below. In some embodiments, the electronic control unit 110 and/or the other components are included within a single device. In other embodiments, the electronic control unit 110 and/or the other components may be distributed among multiple devices that are communicatively coupled, an example of which is shown in
The electronic control unit 110 includes a non-transitory electronic memory 130 that stores a set of machine readable instructions and a processor 120 for executing the machine readable instructions. The non-transitory electronic memory 130 may comprise RAM, ROM, flash memories, hard drives, or any device capable of storing machine readable instructions such that the machine readable instructions can be accessed by the processor 120. The machine readable instructions comprise logic or algorithm(s) written in any programming language of any generation (e.g., 1GL, 2GL, 3GL, 4GL, or 5GL) such as, for example, machine language that may be directly executed by the processor 120, or assembly language, object-oriented programming (OOP), scripting languages, microcode, etc., that may be compiled or assembled into machine readable instructions and stored in the non-transitory electronic memory 130. Alternatively, the machine readable instructions may be written in a hardware description language (HDL), such as logic implemented via either a field-programmable gate array (FPGA) configuration or an application-specific integrated circuit (ASIC), or their equivalents. Accordingly, the methods described herein may be implemented in any conventional computer programming language, as pre-programmed hardware elements, or as a combination of hardware and software components. The non-transitory electronic memory 130 may be implemented as one memory module or a plurality of memory modules.
In some embodiments, the non-transitory electronic memory 130 implements one or more logic operations to execute the functions of the feedback guidance system 100. These operations may include path planning logic 131, virtual wall generation logic 133, location determination logic 135, feedback determination logic 137, and operating logic 139. The path planning logic 131 may be configured to plan a path based on information in the non-transitory electronic memory 130 (e.g., information pertaining to the surrounding environment compiled from user inputs, electronic communication with other devices, or information gathered by the electronic control unit 110). The virtual wall generation logic 133 may be configured to define at least one virtual wall that is offset from the path, which may be utilized to assist a user with navigation, as will be described in further detail below. The location determination logic 135 may be configured to determine a user's location from information gathered by one or more location sensors or from information stored in the non-transitory electronic memory 130 (e.g., when the user's location is input with a user input device and stored in the non-transitory electronic memory 130). The feedback determination logic 137 may be configured to determine when and how to provide tactile feedback and/or audible feedback to the user. The feedback determination logic 137 will signal the tactile feedback device 150 and optionally the audible feedback device 170 to send feedback to the user. The operating logic 139 may include an operating system and/or other software for managing components of the feedback guidance system 100. The functionality of each of these logic modules will be described in further detail below.
The processor 120 may be any device capable of executing machine readable instructions. For example, the processor 120 may be an integrated circuit, a microchip, a computer, or any other computing device. The non-transitory electronic memory 130 and the processor 120 are coupled to a communication path 160 that provides signal interconnectivity between various components and/or modules of the feedback guidance system 100. Accordingly, the communication path 160 may communicatively couple any number of processors with one another, and allow the modules coupled to the communication path 160 to operate in a distributed computing environment. Specifically, each of the modules may operate as a node that may send and/or receive data. As used herein, the term “communicatively coupled” means that coupled components are capable of exchanging data signals with one another such as, for example, electrical signals via conductive medium, electromagnetic signals via air, optical signals via optical waveguides, and the like.
Accordingly, the communication path 160 may be formed from any medium that is capable of transmitting a signal such as, for example, conductive wires, conductive traces, optical waveguides, or the like. Moreover, the communication path 160 may be formed from a combination of mediums capable of transmitting signals. In some embodiments, the communication path 160 comprises a combination of conductive traces, conductive wires, connectors, and buses that cooperate to permit the transmission of electrical data signals to components such as processors, memories, sensors, input devices, output devices, and communication devices. Additionally, it is noted that the term “signal” means a waveform (e.g., electrical, optical, magnetic, mechanical or electromagnetic), such as DC, AC, sinusoidal-wave, triangular-wave, square-wave, vibration, and the like, capable of traveling through a medium.
In the embodiments described herein, the non-transitory electronic memory 130 and the processor 120 are integral with the electronic control unit 110. However, it is noted that the electronic control unit 110, the non-transitory electronic memory 130, and the processor 120 may be discrete components communicatively coupled to one another without departing from the scope of the present disclosure.
As schematically depicted in
The location sensor 140 may be any device capable of generating an output indicative of a location. In some embodiments, the location sensor 140 includes a global positioning system (GPS) sensor, a camera, a radio frequency identification (RFID) sensor, a time of flight (TOF) sensor, a laser, or an ultrasonic sensor. It should be understood that the location sensor 140 is not limited to the specific sensors identified above because the location sensor 140 includes any sensor capable of generating an output indicative of a location.
The tactile feedback device 150 may be any device capable of providing tactile feedback to a user. The tactile feedback device 150 may include a vibration device (such as in embodiments in which tactile feedback is delivered through vibration), an air blowing device (such as in embodiments in which tactile feedback is delivered through a puff of air), or a pressure generating device (such as in embodiments in which the tactile feedback is delivered through generated pressure). In some embodiments, the tactile feedback device 150 comprises an array of feedback devices that provide the user with more detailed feedback. For example, an array (e.g., a 2×2 array or 3×3 array) of tactile feedback devices can provide directional feedback to the user by only providing feedback on one side of the array, e.g., tactile feedback received on a left side of a user directs the user to move right. In some embodiments, the tactile feedback device 150 is wearable on the user, for example as a belt, a wristband, a waist-pack, an adhesive, or a button. In some embodiments, the tactile feedback device 150 is located in a device separate from some or all of the other components of the feedback guidance system 100 and communicatively coupled with the feedback guidance system 100.
The audible feedback device 170 may be any device capable of providing audible feedback to a user. The audible feedback device 170 may include a speaker, headphones, or the like. In some embodiments, the audible feedback may be delivered to the user with the speaker or headphones in a 3-dimensional (3D) audio placement format. In some embodiments, the audible feedback device 170 is integral with the feedback guidance system 100, as depicted in
While
For example, referring now to
Referring now to the operation of the feedback guidance system 100 depicted in
The machine readable instructions of the virtual wall generation logic 133, when executed by the processor 120, cause the feedback guidance system 100 to define at least one virtual wall along a path for navigation. In some embodiments, the feedback guidance system 100 defines the path for navigation by a user with the path planning logic 131. The path planning logic 131 may plan the path in a number of ways, such as using a path planning algorithm to plan the path from a current location or starting location to a destination location. The path planning logic 131 may include a path planning algorithm that plans the path using optimization methods, for example a “shortest distance” path planning optimization method or an “avoiding crowded locations” path planning algorithm.
Referring once again to the virtual wall generation logic 133, in some embodiments, the machine readable instructions of the virtual wall generation logic 133, when executed by the processor 120, cause the feedback guidance system 100 to define a single virtual wall that is offset from the path by an offset distance. For example, a virtual wall offset to the right of the path is configured to provide feedback to the user when the user comes within a threshold distance from virtual wall, i.e. the user strays too far to the right of the path and is alerted to correct course. In some embodiments, the threshold distance is zero. In some embodiments, the threshold distance is greater than zero. In some embodiments, the feedback guidance system 100 defines a first virtual wall and a second virtual wall, each offset from a centerline of the path. In embodiments that include a first virtual wall and a second virtual wall, the first virtual wall and the second virtual wall are each offset the same distance from the centerline of the path, creating a virtual corridor for the user to traverse. For example, the feedback guidance system can define virtual walls offset along each side of a path from a starting point to a destination point creating a virtual corridor along the entire path. Such virtual walls provide the user with the navigation benefits of physical walls as well as additional navigational benefits not provided by physical walls, as will be described in further detail below.
Still referring to the operation of the feedback guidance system 100, the machine readable instructions of the location determination logic 135, when executed by the processor 120, cause the feedback guidance system 100 to determine the location of the user. The location determination logic 135 determines the location of the user based on the output of the location sensor 140. The location determined by the location determination logic 135 allows the feedback guidance system 100 to locate the user and compare the user's location to the location of the virtual wall. In some embodiments, the location sensor comprises a GPS sensor that determines the location of the user and provides the determined location to the feedback guidance system 100 for processing with the location determination logic 135. As noted above, in some embodiments, the location sensor 140 includes a camera operable to sense the location and movement of a user. A camera, or other embodiments of the location sensor, can locate the user as a whole or can locate more specific segments of a user, such as a user's hand or a cane.
As also noted above, the location sensor 140 may include a time of flight (TOF) camera which uses a range imaging system to resolve distance and location by measuring the TOF of a light signal between the camera and the user. In some embodiments, the location sensor determines the location of a user with a laser. In this embodiment, a laser is reflected off the user and user location is determined by analyzing the reflected light. In other embodiments, the location sensor comprises ultrasonic location sensing technology. In some embodiments, sensing and locating the user is accomplished using traditional techniques for localization (when the map of the environment is known) such as Extended Kalman filter, Particle filters or any other such filtering mechanism.
Still referring to the operation of the feedback guidance system 100, the machine readable instructions of the feedback determination logic 137, when executed by the processor 120, cause the feedback guidance system 100 to provide tactile feedback or audible feedback to the user. When the feedback guidance system 100 determines that the user is a threshold distance from a defined virtual wall, the feedback determination logic 137 transmits a signal to the tactile feedback device 150 and/or the audible feedback device 170 to provide feedback to a user. In some embodiments, the tactile feedback is provided as vibrations, air puffs, pressure, or the like. In other embodiments, the audible feedback is provided as speech or abstract sounds. In further embodiments, feedback is provided in multiple forms, each associated with a particular function or location. For example, the feedback guidance system 100 may be configured to recognize a discrete location and transmit unique feedback related to that location. For example, a coffee shop may be associated with a unique audible or tactile feedback such that that particular feedback is provided to the user each time the user is near that coffee shop. Such unique feedback can be implemented for multiple locations, providing the user information about their surroundings without visual cues.
As noted above, in some embodiments, the tactile feedback device 150 comprises an array of tactile feedback devices, such as vibrating devices. In embodiments that include an array of tactile feedback devices, the feedback determination logic 137 may provide the user more detailed information about his or her location with respect to the virtual wall and the surrounding environment. For example, a tactile feedback device 150 that includes an array of vibrators may be incorporated into a wearable device, such as a belt. The feedback determination logic 137 may signal feedback using the array to provide feedback depending on the location of the user with respect to the virtual wall or the surrounding environment. For example, when the user contacts (or comes within a threshold distance of) a virtual wall on his or her left side, the feedback determination logic may activate the vibrators on the left side of the tactile feedback array. Such feedback may inform the user that the user has contacted (or has come within a threshold distance of) a virtual wall on his or her left. More complex arrays may be provided in order to communicate more precise feedback to the user. It should be understood that the tactile feedback device 150 may be housed in a plurality of devices, both wearable and non-wearable.
In some embodiments, the optional audible feedback device 170 provides audible feedback to mimic natural sounds of the physical world. Visually impaired individuals often use echolocation to navigate. In some embodiments, the audible feedback device 170 provides audible feedback indicative of the virtual wall in a manner that mimics the sound that may be reflected from a physical wall for the purposes of echolocation. For example, an individual may tap a cane on the ground and determine his or her distance to a physical wall by the sound of the reverberation. In a similar manner, some embodiments of the feedback guidance system 100 use the audible feedback device 170 to provide audible feedback to a user mimicking the sound of a cane tap reverberated off a physical wall in response to the feedback guidance system 100 sensing a tap of a cane. Such audible feedback may allow a user to determine his or her distance from a virtual wall.
In some embodiments, the feedback determination logic 137 signals the tactile feedback device 150 and/or the audible feedback device 170 to provide feedback when the distance between the user and a virtual wall reaches a threshold distance. In some embodiments, the feedback is binary such that there is either no feedback or feedback at a consistent intensity. In some embodiments, the feedback is provided in a gradient format such that the intensity of the feedback varies based on the location of the user with respect to the virtual wall, the environment or other conditions. In some embodiments, for example, as the user approaches the threshold distance between the user and the virtual wall, feedback is provided with increasing intensity until the threshold distance is reached. Once the threshold distance is reached, full feedback intensity is provided. Gradient feedback may be incorporated into audible feedback and/or tactile feedback. In some embodiments, as the user approaches the threshold distance, audible feedback is provided by the audible feedback device at an increasing volume until the threshold distance is reached and full feedback volume is provided. In some embodiments, as the user approaches a threshold distance, tactile feedback is provided by the tactile feedback device at an increasing intensity, for example, increasing vibrations, until the threshold distance is reached and full intensity tactile feedback is provided.
In some embodiments, the feedback guidance system 100 may be toggled into alternative modes, for example a normal operating mode, a minimal feedback mode, and a learning mode. The normal operating mode may function as described above. In the minimal feedback mode, the tactile feedback and/or audible feedback provided to the user may be reduced and certain components, such as the virtual wall, may be disabled. In the learning mode, the user, often under the guidance of a friend, may follow a path that the feedback guidance system 100 “learns” by storing information indicative of the path in the non-transitory electronic memory 130. Once the path information is stored, the feedback guidance system 100 may use this information to plan the path along which the virtual wall is defined. In the learning mode, the feedback guidance system 100 may continue to sense the user's location within the environment and stores the location information in the non-transitory electronic memory 130 for future use. Upon future reference of this learned path, the feedback guidance system 100 is able to provide navigation assistance to the user as the user traverses the learned path by defining at least one virtual wall along the path and providing the user the various forms of feedback available in the feedback guidance system 100. In some embodiments, learning mode may operate simultaneous with other modes, for example, the feedback guidance system can learn new information about paths and the surrounding environment while in minimal feedback mode or in normal operating mode.
In some embodiments, the feedback guidance system 100 determines the location of hazards along the path and in the surrounding environment. When the user approaches a hazard, feedback is provided to the user to alert them of potential danger. Hazard feedback can be provided through the tactile feedback device 150 and/or the audible feedback device 170. In some embodiments, hazard feedback may be provided to the user when the virtual walls are enabled.
Referring now to
Referring still to
As shown in
Referring now to
It should now be understood that the embodiments described herein relate to feedback guidance systems for providing tactile and audible navigation assistance to a visually impaired user, in a manner they are familiar with in an open environment. The embodiments provide a flexible and adaptive feedback guidance system that defines virtual walls, provides a user with the navigation benefits of a physical wall and is tailored to provide straightforward and precise navigation assistance. The embodiments provide a user with tactile or audible feedback to provide intuitive navigation assistance and allow a visually impaired user to traverse open environments with ease.
It is noted that the terms “substantially” and “about” may be utilized herein to represent the inherent degree of uncertainty that may be attributed to any quantitative comparison, value, measurement, or other representation. These terms are also utilized herein to represent the degree by which a quantitative representation may vary from a stated reference without resulting in a change in the basic function of the subject matter at issue.
While particular embodiments have been illustrated and described herein, it should be understood that various other changes and modifications may be made without departing from the spirit and scope of the claimed subject matter. Moreover, although various aspects of the claimed subject matter have been described herein, such aspects need not be utilized in combination. It is therefore intended that the appended claims cover all such changes and modifications that are within the scope of the claimed subject matter.
