Patent Grant
11153673
The present invention relates to: an information processing device to be worn at an ear of a user; a sound emission control method; a sound emission control program; and an acoustic system.
In recent years, information processing devices (hearable devices) of an earphone type, a headphone type, or the like, which have advanced processing capability and which, for example, store content data such as music and wirelessly or otherwise receive such content data from an external device so that the content data can be played, have been proposed. A user wearing such an information processing device can casually listen to music and the like anywhere, or can further use various kinds of services using voices and the like. However, while a vehicle is being driven, when an information processing device covering an ear of a user plays music or the like, it becomes hard to hear ambient sound, and thus such playback is not preferable. In order to solve this problem, patent document 1 (Japanese Laid-Open Patent Publication No. 2008-100598) discloses a vehicular acoustic system which, when predetermined communication is established between a headphone-type information processing device and an in-vehicle device, makes a notification for prompting the user to take off the information processing device, or stops the playback of music or the like being performed by the information processing device.
In the acoustic system of patent document 1, while a user is driving a vehicle, the information processing device covering the ear has to be taken off in order to allow ambient sound to be heard well, and thus convenience for the user is low.
The present invention has been made in view of the aforementioned problem, and an object of the present invention is to provide an information processing device having improved convenience in a case where a vehicle is driven while the information processing device is worn.
In order to attain the above-described object, one aspect of the present invention is an information processing device having a to-be-worn portion capable of being worn at one ear of a user, the information processing device including: a first sound pickup section configured to pick up external sound; a sound emission section configured to emit sound to the one ear of the user; and a control section configured to determine whether or not a predetermined condition is satisfied in a state where the information processing device is in a vehicle, and, when determining that the predetermined condition is satisfied, cause the sound emission section to emit the external sound picked up by the first sound pickup section.
Accordingly, since switching to a state where external (ambient) sound is emitted is made when the predetermined condition is satisfied, ambient sound can be made easily heard without taking off the information processing device from the ear, in a case where, for example, the vehicle is highly likely to be driven.
In addition, when causing the sound emission section to emit the external sound picked up by the first sound pickup section, the control section may amplify a sound pressure of the external sound before causing the external sound to be emitted.
Accordingly, ambient sound can be made easily heard more clearly.
In addition, the information processing device may further include another to-be-worn portion capable of being worn at another ear of the user, wherein the control section further makes a notification for prompting the user to take off either the to-be-worn portion or the other to-be-worn portion if the predetermined condition is satisfied when the to-be-worn portion and the other to-be-worn portion are worn at the one ear and the other ear of the user.
Accordingly, in a case where a law is set forth that the user must not wear the information processing device at both ears at the same time during driving, the user can be prompted to drive in compliance with the law.
In addition, the predetermined condition may be that the user is seated on a driver seat of the vehicle.
Accordingly, it can be suitably determined that the user is highly likely to drive the vehicle.
In addition, the sound emission section may emit sound for authentication having a predetermined frequency and a predetermined sound pressure into an ear hole of the one ear of the user, the information processing device may further include a second sound pickup section configured to pick up an echo, from the ear hole, caused by the sound for authentication, and the control section may perform authentication of the user on the basis of the echo.
Accordingly, control of access to the information processing device and the vehicle can be performed with use of the result of the authentication.
Another aspect of the present invention is a sound emission control method to be executed by a computer of an information processing device which has a to-be-worn portion capable of being worn at one ear of a user, and which includes a sound pickup section configured to pick up external sound and a sound emission section configured to emit sound to the one ear of the user, the sound emission control method including: a step of determining whether or not a predetermined condition is satisfied in a state where the information processing device is in a vehicle; and a step of causing, when it is determined that the predetermined condition is satisfied in the step of performing the determination, the sound emission section to emit the external sound picked up by the sound pickup section.
Another aspect of the present invention is a computer-readable non-transitory storage medium storing a sound emission control program to be executed by a computer of an information processing device which has a to-be-worn portion capable of being worn at one ear of a user, and which includes a sound pickup section configured to pick up external sound and a sound emission section configured to emit sound to the one ear of the user, the sound emission control program including: a step of determining whether or not a predetermined condition is satisfied in a state where the information processing device is in a vehicle; and a step of causing, when it is determined that the predetermined condition is satisfied in the step of performing the determination, the sound emission section to emit the external sound picked up by the sound pickup section.
Another aspect of the present invention is an acoustic system including an in-vehicle device and an information processing device which is to be worn at at least one ear of a user, wherein the in-vehicle device includes a vehicle-side communication section configured to communicate with the information processing device, and the information processing device includes: a sound pickup section configured to pick up external sound; a sound emission section configured to emit sound to the at least one ear of the user; a communication section configured to communicate with the in-vehicle device; and a control section configured to determine, on the basis of a result of communication with the in-vehicle device by the communication section, whether or not a predetermined condition is satisfied in a state where the information processing device is in a vehicle, and, when determining that the predetermined condition is satisfied, cause the sound emission section to emit the external sound picked up by the sound pickup section.
Owing to these features, since switching to the state where external (ambient) sound is emitted is made when the predetermined condition is satisfied, ambient sound can be made easily heard without taking off the information processing device from the ear, while, for example, a vehicle is being driven.
As described above, in the present invention, switching to a state where external (ambient) sound is emitted is made when the predetermined condition is satisfied in the vehicle, and thus ambient sound can be made easily heard without taking off the information processing device from the ear while, for example, the vehicle is being driven, whereby an information processing device having improved convenience can be provided.
(Outline)
An example of an information processing device according to the present invention is an earphone-type information processing device to be worn in an ear of a user, and is a so-called hearable device that can play music and the like and provide services using voices and the like. In the information processing device according to the present invention, switching to an ambient mode for picking up external sound and emitting the external sound to the ear of the user, is made when a predetermined condition indicating that the user is highly likely to drive is satisfied in a vehicle, whereby ambient sound can be made easily heard while the vehicle is being driven, without taking off the information processing device from the ear.
Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.
<Configuration>
The in-vehicle device 20 includes: a vehicle-side communication section 21 which communicates with the information processing device 10 and the like; and a vehicle-side control section 22 which controls the vehicle-side communication section 21 and exchanges information with another in-vehicle device.
<Process>
(Step S101): The control section 15 of the information processing device 10 causes the sound emission section 13 to emit sound for authentication having a predetermined frequency and a predetermined sound pressure into the ear hole of the user. The control section 15 causes the second sound pickup section 12 to pick up an echo, in the ear hole of the user, caused by the sound for authentication. The pattern of the sound pressure characteristic, with respect to frequency, of the echo varies among ear holes. The control section 15 calculates a degree of matching between the characteristic of the picked-up echo and the characteristic, of an echo in an ear hole of an authorized user, registered in the information processing device 10 in advance. In a case where the degree of matching is higher than a predetermined value, the control section 15 authenticates the user as the authorized user for the information processing device 10 (first authentication). This first authentication is executed every time the user takes off the information processing device 10 from the ear and wears it again.
(Step S102): The control section 15 of the information processing device 10 causes the sound emission section 13 to play music or the like, on the basis of an operation by the user authenticated in the first authentication. In addition to or instead of the above, the control section 15 may perform an operation of a noise cancelling mode in which the control section 15 causes the first sound pickup section 11 to pick up external sound, generates a sound having a phase reverse to that of the picked-up external sound, and causes the sound emission section 13 to emit the generated sound, thereby cancelling the external sound in the ear of the user.
(Step S103): When the user wearing the information processing device 10 approaches the vehicle to get on the vehicle and comes within the above-described range 500, the information processing device 10 and the in-vehicle device 20 become able to communicate with each other. When the communication section 14 receives radio waves from the in-vehicle device 20, the control section 15 of the information processing device 10 detects that communication therewith can be performed. Meanwhile, when the vehicle-side communication section 21 receives radio waves from the information processing device 10, the vehicle-side control section 22 of the in-vehicle device 20 detects that communication therewith can be performed.
(Step S104): When the user performs a touch operation on a door handle of the vehicle and a sensor of the vehicle detects the touch operation, the vehicle-side control section 22 of the in-vehicle device 20 acquires a result of the detection.
(Step S105): The vehicle-side control section 22 of the in-vehicle device 20 performs second authentication. The second authentication is performed through the following method. The vehicle-side control section 22 communicates with the information processing device 10 via the vehicle-side communication section 21, and performs, on the basis of a result of the communication, detection as to whether the information processing device 10 is a device registered in advance. In addition, the vehicle-side control section 22 receives the result of the first authentication from the information processing device 10. If the information processing device 10 is the device registered in advance and the result of the first authentication is successful, the vehicle-side control section 22 authenticates the user as an authorized user for the vehicle. Alternatively, from the information processing device 10, the vehicle-side control section 22 may receive, instead of the result of the first authentication, information indicating an echo of the user picked up by the second sound pickup section 12. In this case, the vehicle-side control section 22 calculates a degree of matching with the characteristic, of an echo in an ear hole of the authorized user for the vehicle, registered in advance in the in-vehicle device 20 or a server that is a predetermined enquiry destination. In a case where the degree of matching is higher than a predetermined value, the vehicle-side control section 22 authenticates the user as the authorized user for the vehicle.
(Step S106): If the second authentication is successful, the vehicle-side control section 22 of the in-vehicle device 20 unlocks the door.
(Step S107): The user opens the door, gets on the vehicle, and starts the engine of the vehicle (turns on a power switch). At a time when the ignition switch (power switch) is operated to be ON, the vehicle-side control section 22 of the in-vehicle device 20 may perform the second authentication again.
(Step S108): The vehicle-side control section 22 of the in-vehicle device 20 causes the vehicle-side communication section 21 to transmit a notification in response to the starting, by the user, of the engine of the vehicle.
(Step S109): When the communication section 14 receives the notification, the control section 15 of the information processing device 10 determines, on the basis of the notification, that a predetermined condition described later is satisfied in the vehicle. Accordingly, the control section 15 performs an operation of an ambient mode in which the control section 15 stops playback of the music or the like and the operation of the noise cancelling mode, causes the first sound pickup section 11 to pick up external sound, and causes the sound emission section 13 to emit the picked-up external sound, thereby allowing the external sound to reach the ear of the user. At this time, the sound to be emitted by the sound emission section 13 may be a sound obtained by amplifying the external sound picked up by the first sound pickup section 11, so as to allow the external sound to be more clearly heard by the user. In the ambient mode, the music or the like being played may be kept played so as to be overlaid on the external sound as long as the user can hear the external sound. After the above-described steps, the process is ended. The above-described steps may be changed in terms of details and order, or omitted, as appropriate.
When the user gets out of the above-described range 500 and communication with the in-vehicle device 20 becomes unable to be performed, the information processing device 10 may restart the playback of music or the like and the operation of the noise cancelling mode. Alternatively, the information processing device 10 may be configured to be able to detect, through communication with the in-vehicle device 20, that the driving has been ended, by detecting, for example: stoppage of the engine; user's getting out of the vehicle in addition to the stoppage of the engine; or a door-locking operation in addition to the stoppage of the engine and the user's getting out of the vehicle. By being so configured, the information processing device 10 may restart the playback of music or the like and the operation of the noise cancelling mode in response to the detection of any of these conditions.
As described above, in a case where the information processing device 10 is worn by an authorized user for a vehicle and a predetermined condition that, for example, the user starts the engine or turns on the power switch as described above is satisfied in the vehicle, the information processing device 10 applies the ambient mode in which ambient sound is easily heard. The predetermined condition is not limited thereto, and may be to detect that releasing of the parking brake is also performed after the engine is started. Alternatively, the predetermined condition may be to detect that the user is seated on the driver seat. This detection can be performed as follows, for example: the vehicle-side communication section 21 of the in-vehicle device 20 is provided with a speaker for emitting, in the vehicle, an audible sound having a predetermined pattern, or an inaudible sound, specifically an ultrasound or an extremely-low-frequency sound, having the predetermined pattern, and the information processing device 10 performs, by means of sound-based communication in which the first sound pickup section 11 thereof picks up a sound having the predetermined pattern, position measurement based on the sound pressure or the reaching time period of the sound. Alternatively, communication may be used that is based on, instead of such a sound, a very faint radio wave with the range of the driver seat being a reaching range for the radio wave. Owing to the above, in a case where the user is a passenger who is not authenticated in the second authentication, or in a case where, even if being authenticated in the second authentication, the user simply performs unloading or the like but does not start the engine or is not seated on the driver seat, the operation of the information processing device 10 is unchanged, and the user can suitably listen to music and the like.
(Modification 1)
If the information processing device 10 described in the above-described embodiment is used as each of two devices forming one pair and the devices are operated as appropriate so as to be linked to each other by means of, for example, wireless communication, a single information processing apparatus is obtained that has the to-be-worn portion capable of being worn in the one ear and another to-be-worn portion capable of being worn in the other ear so that the information processing apparatus can be worn in both ears. In this case, an echo in either one of the ears, or echoes in both ears, may be used for the above-described authentication. By performing the same process as that described above, the ambient mode can be applied for both ears when it is determined that the predetermined condition is satisfied in the vehicle. In addition, for compliance with a law described later, a notification may be made for prompting the user to take off at least one of the to-be-worn portion and the other to-be-worn portion from the ear. The notification may be made with use of sounds, or may be made with use of vibrations, with a vibrator being included. In addition, the control section 15 of the information processing apparatus may be configured to determine whether or not the to-be-worn portion and the other to-be-worn portion are worn in the respective ears, and make such a notification if these portions are worn in the respective ears. Whether or not these portions are worn in the respective ears may be determined as follows, for example: the respective to-be-worn portions are provided with contact sensors at locations where the to-be-worn portions come into contact with the respective ears, and the determination is performed on the basis of detection results from the contact sensors; or the determination is performed on the basis of whether or not echoes in both ears can be acquired after the above-described sound for authentication is emitted.
Such an information processing apparatus capable of being worn in both ears can be configured by the single information processing device 10 described in the above-described embodiment being further provided with another to-be-worn portion to be worn in the other ear of the user, and being provided, as appropriate, with a sound emission section, a sound pickup section, a contact sensor, a communication section capable of communicating with the control section 15, and the like which are for the other ear and which can be controlled from the control section 15.
Depending on the country, a law is set forth that earphones and the like must not be worn in both ears at the same time during driving. Therefore, in a country in which such a law is set forth, a state where at least one ear is uncovered needs to be obtained regardless of whether or not to apply the ambient mode. By giving the above-described notification to the user, the user can be prompted to drive in compliance with the law. If the user ignores the notification and continues wearing the information processing device 10 in both ears for a predetermined time period, the following operation may be performed: the control section 15 of the information processing device 10 detects this continued wearing and causes the communication section 14 to give a notification to the in-vehicle device 20, then the vehicle-side control section 22 of the in-vehicle device 20 receives the notification via the vehicle-side communication section 21 and instructs another in-vehicle device to stop the engine (or motor). Accordingly, driving that does not comply with the law, can be forcedly prohibited.
(Modification 2)
The information processing device 10 described in the above-described embodiment and the information processing apparatus according to modification 1, may not include the second sound pickup section 12, and may not perform such echo-based user authentication as that described above. In this case, control of access such as unlocking a door of a vehicle can be performed with use of another smart key system or the like. Also in this case, ambient sound can be made easily heard while the user is driving, by applying the ambient mode when it is determined, by means such as communication with the in-vehicle device 20 or sound waves from the in-vehicle device 20, that the above-described predetermined condition that, for example, the engine is started or the user is seated on the driver seat is satisfied in a state where the information processing device 10 is in the vehicle.
(Effect)
As described above, the information processing device according to the present invention performs the process of applying the ambient mode when, for example, the predetermined condition indicating that the user is highly likely to drive is satisfied in the vehicle. Accordingly, the situation in which the user drives is determined, and, in a case where the user drives, ambient sound is easily heard without taking off the information processing device, whereby convenience is improved. In a case where the user does not drive, the operation of the information processing device is unchanged, and the user can suitably listen to music and the like. In addition, a notification can be made for compliance with the law. In addition, such situation-dependent switching of operations is automatically performed. Convenience can be improved also in terms of these features. In addition, in the case where the information processing device does not perform the user authentication but the in-vehicle device independently performs user authentication as in modification 2, the process of associating (pairing) the information processing device and the in-vehicle device with each other in advance is not necessary, whereby this associating process can be omitted.
Although the present invention has been described as above based on the embodiment thereof, the present invention can be taken as an information processing device, a sound emission control method and a sound emission control program to be executed by a control section included in the information processing device, a computer-readable non-transitory storage medium storing the sound emission control program, and an acoustic system including the information processing device and an in-vehicle device.
The present invention is useful for an information processing device and an acoustic system to be mounted to a vehicle or the like.
| Number | Date | Country | Kind |
|---|---|---|---|
| JP2018-101581 | May 2018 | JP | national |
| Number | Name | Date | Kind |
|---|---|---|---|
| 20160212522 | Lee | Jul 2016 | A1 |
| 20180040093 | Boesen | Feb 2018 | A1 |
| 20200092628 | Osugi | Mar 2020 | A1 |
| Number | Date | Country |
|---|---|---|
| 2686262 | Mar 2005 | CN |
| 105812974 | Jul 2016 | CN |
| 2008-100598 | May 2008 | JP |
| 2010-86328 | Apr 2010 | JP |
| Number | Date | Country | |
|---|---|---|---|
| 20190364353 A1 | Nov 2019 | US |
