Patent Grant
10142752
The present application is a U.S. National Stage filing under 35 U.S.C. § 371 of international patent cooperation treaty (PCT) application No. PCT/CN2014/090439, filed Nov. 6, 2014, and entitled “INTERACTION WITH DEVICES”, which claims the benefit of priority to Chinese Patent Application No. 201310752762.2, filed on Dec. 31, 2013, which applications are hereby incorporated into the present application by reference herein in their respective entireties.
The present application relates to the field of interaction technologies, and in particular to interaction with devices, such as displays, speakers, etc.
During use of a loudspeaker, if an object completely or partially blocks a sound-making surface of the loudspeaker, for example, when an obstacle is close to the loudspeaker, a sound-making effect of the loudspeaker is influenced negatively, causing an undesirable experience for a user. Therefore, conventionally, in arrangement of a position of a loudspeaker, it blocking the sound-making surface of the loudspeaker is avoided as much as possible. However, in some scenarios, a sound-making surface of a loudspeaker may be unintentionally or unexpectedly blocked, or it may be further unavoidable to block a sound-making surface of a loudspeaker. For example, a transparent film loudspeaker appears, and may be applied on a surface of a screen to replace an existing loudspeaker and generate touch feedback, and the like; however, when the screen is a touchscreen, an unavoidable touch operation of a user on the surface of the screen negatively influences the sound making capability of the loudspeaker.
An example, non-limiting objective of the present application is to provide an interaction technology, so as to alleviate influence on a sound-making effect of a loudspeaker from an obstacle on a surface of or near the loudspeaker during use as much as possible, and improving user experience.
According to a first aspect, an example embodiment of the present application provides an interaction apparatus, which comprises:
According to a second aspect, an example embodiment of the present application provides an interaction method, which comprises:
In at least one embodiment of the present application, a loudspeaker array comprising a plurality of speaker units is disposed, an obstacle that influences a sound-making effect of the loudspeaker array is detected, and an input signal of a corresponding speaker unit is adjusted when the obstacle exists, thereby alleviating influence from the obstacle on the sound-making effect of the loudspeaker array during use of the loudspeaker array, and improving user experience.
The disclosure will become more fully understood from the detailed description given herein below for illustration only, and thus are not limitative of the disclosure, and wherein:
Specific embodiments of the present application are further described in detail in the following with reference to the accompanying drawings (same numerals represent same elements in several accompanying drawings) and embodiments. The following embodiments are used to describe the present application rather than to limit the scope of the present application.
During use of a loudspeaker, when an obstacle exists inside a certain range on a side of a sound-making surface of the loudspeaker, for example, when the obstacle is very close to the loudspeaker, a sound-making effect of the loudspeaker is severely influenced, thereby influencing auditory experience of a user. Therefore, as shown in
The obstacle detection module 120 may detect, in real time, whether an obstacle exists on the side of the sound-making surface of the loudspeaker array as well as some relevant attributes of the obstacle; therefore, during use of the interaction apparatus 100, when the obstacle exists, the compensation module 130 may perform proper compensation in real time.
In an example embodiment, the compensation module 130 may directly and correspondingly generate a completely new input signal according to the at least one obstacle parameter and a current input signal. In another example embodiment, the compensation module 130 superimposes, according to the at least one obstacle parameter, a compensation signal component on the input signal corresponding to each speaker unit; this example embodiment is further described hereinafter, and is not elaborated here. Here, the compensating for a sound-making effect of the loudspeaker array influenced by the at least one obstacle is: reducing a difference of a waveform of a sound wave, which is produced by the loudspeaker array under the influence of the at least one obstacle, at at least one target position (for example, the position of an ear of a listener) from a waveform when the obstacle does not exist.
Definitely, when no obstacle exists inside the space, the compensation module 130 makes no adjustment to the input signal. In an example embodiment, the compensation module 130 may compare and analyze the at least one obstacle parameter acquired by the obstacle detection module 120 and a preset reference parameter, to determine whether an obstacle exists as well as a position parameter and the like of the obstacle (for example, when the difference between the at least one obstacle parameter and the reference parameter is inside a set range, it corresponds to that no obstacle exists inside the foregoing space; when the difference is beyond the set range, it corresponds to that the obstacle exists inside the space, and the difference is analyzed to obtain position information and the like of the obstacle).
In conclusion, in this embodiment, a loudspeaker array comprising a plurality of speaker units is disposed, an obstacle that influences a sound-making effect of the loudspeaker array is detected, and an input signal of a corresponding speaker unit is adjusted when the obstacle exists, thereby alleviating influence from the obstacle on the sound-making effect of the loudspeaker array during use of the loudspeaker array, and improving user experience.
As shown in
The input signal adjustment parameter comprises at least one of an amplitude adjustment parameter, a phase adjustment parameter, and a frequency adjustment parameter for the input signal.
As shown in
Definitely, in another example embodiment of the embodiment of the present application, the first determination subunit 1311 may further be configured to determine at least one of following relevant parameters of the obstacle: a shape parameter of the obstacle, a posture parameter of the obstacle, and an acoustic characteristic parameter of the obstacle. The different relevant parameters of the obstacle in the foregoing cause different influence to a sound-making effect of a corresponding speaker unit. The posture parameter of the obstacle is a current posture parameter (for example, an included angle between a certain direction of the obstacle and the sound-making surface of the loudspeaker array) of the obstacle inside the certain space at the sound-making surface of the loudspeaker array; the acoustic characteristic parameter of the obstacle is a parameter such as an absorption rate and a reflection rate for a sound wave of the obstacle. The second determination subunit 1312 determines, according to the relevant parameters of the obstacle, a speaker unit that needs to be adjusted and an input signal adjustment parameter corresponding to the speaker unit that needs to be adjusted.
In this example embodiment, the determination unit 131 determines, according to the at least one obstacle parameter acquired by the obstacle detection module 120, which speaker units in the loudspeaker array need to be adjusted, and obtains, through calculation and processing, an input signal adjustment parameter corresponding to a speaker unit that needs to be adjusted, for example, obtains the input signal adjustment parameter by using a sound synthesis method such as sound ray tracing (Sound Ray Tracing) and a head related transfer function (HRTF).
Here, at least one speaker unit that needs to be adjusted may comprise an impeded speaker unit of which a sound-making direction is blocked by the obstacle; in this case, an input signal adjustment parameter corresponding to the impeded speaker unit may be, for example, an amplitude adjustment parameter for decreasing or increasing the amplitude of a current input signal of the impeded speaker unit. In addition, the at least one speaker unit that needs to be adjusted may comprise an unimpeded speaker unit. To compensate for an impeded sound-making effect of the foregoing impeded speaker unit, one compensation component relevant to the impeded speaker unit usually needs to be superimposed on an input signal corresponding to the unimpeded speaker unit; in this way, the speaker unit additionally produces a sound-making effect corresponding to the compensation component. Definitely, to ensure an overall sound-making effect of the loudspeaker array, compensation components corresponding to some speaker units may also be negative.
The appearance of a transparent loudspeaker, for example, a transparent film loudspeaker (for example, a transparent gel loudspeaker, in which another layer of transparent salt water gel is added on a surface of a transparent ionic gel film, and when being powered, the entire film deforms rapidly, so as to produce sound in full range) cause a loudspeaker to be applicable on a display screen. For example, a layer of transparent film loudspeaker is disposed on an outer surface of a display layer of an electronic device having a display function, and therefore it is no longer necessary to dispose a loudspeaker or another external loudspeaker near a display region of the electronic device. The applicant of the present application finds that, when the display screen is a touch display screen, a user needs to touch the surface of the screen to implement interaction with the electronic device, and in this case, a touch with a finger significantly influences a sound-making effect of a loudspeaker at the position of the touch.
Therefore, as shown in
The interaction apparatus 300 comprises the loudspeaker array, the obstacle detection module 320, and the compensation module 330 that are recorded in the embodiment in
As shown in
In this example embodiment, the display module 340 is a panel display module; however, persons skilled in the art may learn that the display module 340 may also have a shape such as a curved surface.
As shown in
As shown in
Definitely, in addition to the foregoing structures, in another example embodiment of the embodiment of the present application, the display layer 341, the positions of the touch input unit 342 and the speaker layer 310 in the layered structure may also be in other forms; for example, the speaker layer 310 is located between the display layer 341 and the touch input unit 342.
In this example embodiment, a transparent protection layer 350 is further provided on the side of the light outgoing direction of the speaker layer 310 to produce an effect for protecting the speaker layer 310.
In this example embodiment, the function of the compensation module 330 may be implemented by a processor (not shown in
In this example embodiment, in addition to effects of input and interaction by a user, the touch input unit 342 may further serve as a touch sensing unit of the obstacle detection module 320. In this case, the part, for example, a finger, with which the user touches the touch input unit 342, is the obstacle that influences the sound-making effect of the loudspeaker array. In this example embodiment, the touch input unit 342 may be used to sense the position touched by the obstacle and the shape of the obstacle, so that the compensation module 330 determines influence from the obstacle on the sound-making effect of the loudspeaker array.
Because different acoustic characteristics of the obstacle also have different influence on the sound-making effect of the corresponding speaker unit, in another example embodiment of the embodiment of the present application, for example, the touch input unit may be further used to sense the type of the obstacle and an acoustic characteristic corresponding to the known type, so that the compensation module 330 performs compensation more effectively.
In addition to the foregoing touch, proximity without contact to the sound-making surface of the loudspeaker array may also influence the sound-making effect of the loudspeaker array; therefore, an embodiment of the present application further provides an interaction apparatus 400 in another example embodiment shown in
In an example embodiment, the proximity sensing unit 421 may replace the touch input unit 342 in the embodiment shown in
In another example embodiment, the interaction apparatus 400 comprises both the touch input unit and the proximity sensing unit 421; in this case, the touch input unit only has the effect of a touch input unit in an existing touchscreen, and the proximity sensing unit 421 implements the function of the obstacle detection module 420. In this case, for example, the proximity sensing unit 421 may comprise a depth sensor, configured to acquire in real time a depth map of a facing direction of the sound-making surface of the loudspeaker array, and obtain the at least one obstacle parameter according to the acquired depth map.
Definitely, in addition to the several forms of the obstacle detection module in the foregoing, other structures that can detect at least one obstacle parameter inside the space may also be applied to an example embodiment of an embodiment of the present application.
The communications interface 520 is configured to perform communication with a network element such as a client.
The processor 510 is configured to execute a program 532, and may specifically execute relevant steps in the following method embodiment.
Specifically, the program 532 may comprise a program code, where the program code comprises a computer operation instruction.
The processor 510 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits that are arranged to implement the embodiments of the present application.
The memory 530 is configured to store the program 532. The memory 530 may comprise a high-speed random access memory (RAM), or may also further comprise a non-volatile memory, for example, at least one disk memory. The program 532 may be specifically configured to cause the interaction apparatus 500 to execute the following steps:
For the specific implementation of each step in the program 532, reference may be made to the corresponding description of corresponding modules and units in the embodiments in
In conclusion, by means of the several example embodiments of the embodiments of the present application in the foregoing, influence on a sound-making effect of a loudspeaker from an obstacle inside certain space of a sound-making surface of the loudspeaker may be alleviated, and in a case where the obstacle exists, desirable auditory experience may still be provide to a user.
As shown in
In this embodiment, at least one obstacle parameter corresponding to an obstacle that influences a sound-making effect of a loudspeaker array comprising a plurality of speaker units is detected, and when the at least one obstacle parameter shows that the obstacle exists, an input signal of a corresponding speaker unit is adjusted, thereby alleviating influence from the obstacle on the sound-making effect of the loudspeaker array during use of the loudspeaker array, and improving user experience.
In the following, each step in the embodiment of the present application is further described.
Here the “certain space” on the side of the sound-making surface of the loudspeaker array may be preset, or may also be obtained according to the performance of a detection module. For example, it may be preset that the certain space is space having the thickness of 1 centimeter that extends from the sound-making surface of the loudspeaker array along a sound-making direction. Alternatively, when the detection module comprises the touch input unit in the foregoing apparatus embodiment, the certain space may be space in which the touch input unit can sense signal input, for example, a sensing surface of the touch input unit.
In an example embodiment, the input signals corresponding to at least two speaker units in the plurality of speaker units are different. For example, the plurality of speaker units in the loudspeaker array may correspond to a plurality of different input signals, respectively; that is, the plurality of different input signals may be separately controlled. The plurality of speaker units may further be grouped into a plurality of speaker unit groups, and each speaker unit group comprises at least two speaker units, and the speaker units in each speaker unit group correspond to a same input signal (for example, one group of speaker units share one same conductor for signal input); such a structure may enable that when a sound-making effect of one or more of speaker units is impeded, a corresponding input signal is adjusted to cause other speaker units to compensate for an overall sound-making effect of the loudspeaker array. Here, for example, the input signal may be a vibration sequence signal, which comprises amplitude, frequency, phase information, and the like, so as to decide the amplitude, frequency, and phase of the vibration of the corresponding speaker unit. Definitely, in an optional example embodiment, the plurality of speaker units has a same input signal, and when the at least one obstacle is detected, the amplitude of the input signal may also be increased to compensate for the sound-making effect of the loudspeaker array.
In an example embodiment, in the compensation step, a completely new input signal may be generated directly and correspondingly according to the at least one obstacle parameter and a current input signal. In another example embodiment, in the compensation step, according to the at least one obstacle parameter, a compensation signal component is superimposed on the input signal corresponding to each speaker unit; this example embodiment is further described hereinafter, and is not elaborated here. Here, the compensating for a sound-making effect of the loudspeaker array influenced by the at least one obstacle is: reducing a difference of a waveform of a sound wave, which is produced by the loudspeaker array under the influence of the at least one obstacle, at at least one target position (for example, the position of an ear of a listener) from a waveform when the obstacle does not exist.
Definitely, when no obstacle exists inside the space, in the compensation step S120, the input signal is not adjusted. In an example embodiment, in the compensation step, the at least one obstacle parameter acquired in the obstacle detection step and a preset reference parameter may be compared and analyzed, to determine whether an obstacle exists as well as parameters such as the position and shape of the obstacle.
In an example embodiment, the compensation step S120 comprises:
The input signal adjustment parameter comprises at least one of an amplitude adjustment parameter, a phase adjustment parameter, and a frequency adjustment parameter for the input signal.
In this example embodiment, the determination step comprises:
When the speaker unit is a transparent speaker unit, for example, a transparent film speaker unit, the loudspeaker array may be applied to a display screen of an electronic device, and in this scenario, the interaction method further comprises:
In an example embodiment, the display region is a plane display region, and the sound-making surface of the loudspeaker array and the plane display region are superposed along a normal direction of the plane display region.
In this example embodiment, the obstacle detection step comprises:
In an example embodiment, the touch sensing region covers the display region, the at least one obstacle parameter, such as the position of the at least one obstacle in the touch sensing region, acquired in the obstacle detection step may be further used for implementing interaction output corresponding to touch input of the user. For example, corresponding processing is performed according to the position of the obstacle in the touch sensing region, and a corresponding display input signal or an input signal corresponding to each speaker unit of the loudspeaker array is output.
In addition to the foregoing touch, proximity without contact to the sound-making surface of the loudspeaker array may also influence the sound-making effect of the loudspeaker array; therefore, in another example embodiment, the obstacle detection step comprises:
In this example embodiment, as long as an obstacle gets close to the sound-making surface of the loudspeaker array and enters the proximity sensing range, a corresponding obstacle parameter may be obtained in the obstacle detection step, and an input signal of a corresponding speaker unit is adjusted in the compensation step.
Reference may be made to corresponding description of the apparatus embodiments in
Persons skilled in the art may understand that in the foregoing methods in the specific embodiments of the present application, the sequence numbers of the steps do not mean an execution order, an execution order of the steps should be determined by the functions and internal logic of the steps, and the sequence numbers should not constitute any limitation on the implementation process of the specific embodiments of the present application.
It can be appreciated by those skilled in the art that each exemplary unit and method step described with reference to the embodiments disclosed in this text can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware mode or a software mode depends on the specific applications and design constraint conditions of the technical solution. The professional technicians can use different methods to realize the functions described with respect to each specific application, but this realization should not be considered to go beyond the scope of the present application.
If said function is realized in the form of a software functional unit and is sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present application essentially or the part which contributes to the prior art or a part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, and comprises several instructions for enabling a computer device (which can be a personal computer, a server, or a network apparatus, etc.) to execute all or some steps of the method described in each embodiment of the present application. The preceding storage medium comprises various media which can store a program code, such as a USB disk, a mobile hard disk, a read-only memory (ROM), a RAM, a magnetic disk or an optical disk, etc.
The above embodiments are only used to describe the present application, without limiting the present application; various alterations and variations can be made by those skilled in the art without departing from the spirit and scope of the present application, so all equivalent technical solutions also belong to the scope of the present application, and the scope of patent protection of the present application should be defined by claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2013 1 0752762 | Dec 2013 | CN | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/CN2014/090439 | 11/6/2014 | WO | 00 |
| Publishing Document | Publishing Date | Country | Kind |
|---|---|---|---|
| WO2015/101105 | 7/9/2015 | WO | A |
| Number | Name | Date | Kind |
|---|---|---|---|
| 4359606 | Shoichi | Nov 1982 | A |
| 4369412 | Sakano | Jan 1983 | A |
| 4379209 | Sakano | Apr 1983 | A |
| 4990286 | Gordon | Feb 1991 | A |
| 5361305 | Easley et al. | Nov 1994 | A |
| 6005957 | Meeks | Dec 1999 | A |
| 6760451 | Craven et al. | Jul 2004 | B1 |
| 6798889 | Dicker et al. | Sep 2004 | B1 |
| 7158643 | Lavoie et al. | Jan 2007 | B2 |
| 7630501 | Blank et al. | Dec 2009 | B2 |
| 8880058 | Helm et al. | Nov 2014 | B2 |
| 9661432 | Silzle et al. | May 2017 | B2 |
| 20020071569 | Wood | Jun 2002 | A1 |
| 20030008689 | Uda | Jan 2003 | A1 |
| 20030048911 | Furst et al. | Mar 2003 | A1 |
| 20030059069 | Bank et al. | Mar 2003 | A1 |
| 20030142833 | Roy et al. | Jul 2003 | A1 |
| 20030220705 | Ibey | Nov 2003 | A1 |
| 20040202333 | Csermak et al. | Oct 2004 | A1 |
| 20050008165 | Sack et al. | Jan 2005 | A1 |
| 20050152557 | Sasaki et al. | Jul 2005 | A1 |
| 20060083391 | Nishida et al. | Apr 2006 | A1 |
| 20060251265 | Asada | Nov 2006 | A1 |
| 20070202917 | Phelps et al. | Aug 2007 | A1 |
| 20080063211 | Kusunoki | Mar 2008 | A1 |
| 20080085019 | Wagenaars et al. | Apr 2008 | A1 |
| 20080273714 | Hartung | Nov 2008 | A1 |
| 20090066499 | Bai et al. | Mar 2009 | A1 |
| 20100008512 | Packer et al. | Jan 2010 | A1 |
| 20100074451 | Usher et al. | Mar 2010 | A1 |
| 20100272273 | Chua et al. | Oct 2010 | A1 |
| 20100296658 | Ohashi | Nov 2010 | A1 |
| 20120263309 | Hiraki | Oct 2012 | A1 |
| 20120294450 | Ozcan | Nov 2012 | A1 |
| 20130028430 | Bares et al. | Jan 2013 | A1 |
| 20130070932 | Nyu et al. | Mar 2013 | A1 |
| 20130073748 | Masuda et al. | Mar 2013 | A1 |
| 20130089210 | Martin et al. | Apr 2013 | A1 |
| 20130179163 | Herbig et al. | Jul 2013 | A1 |
| 20130315419 | Chien | Nov 2013 | A1 |
| 20140307881 | Fuertes, III | Oct 2014 | A1 |
| 20150139427 | Sakai | May 2015 | A1 |
| Number | Date | Country |
|---|---|---|
| 1554208 | Dec 2004 | CN |
| 101170843 | Apr 2008 | CN |
| 101326075 | Dec 2008 | CN |
| 102006541 | Apr 2011 | CN |
| 102375536 | Mar 2012 | CN |
| 202276442 | Jun 2012 | CN |
| 103425335 | Dec 2013 | CN |
| 103702259 | Apr 2014 | CN |
| 103747409 | Apr 2014 | CN |
| 2429155 | Mar 2012 | EP |
| 2007053610 | Mar 2007 | JP |
| 20110127930 | Nov 2011 | KR |
| 2012064285 | May 2012 | WO |
| 2012090032 | Jul 2012 | WO |
| Entry |
|---|
| Naef et al., “Spatialized audio rendering for immersive virtual environments”, Proceeding VRST '02 Proceedings of the ACM symposium on Virtual reality software and technology, pp. 65-72, ACM New York, NY, USA, Nov. 2002. Retrieved on Jun. 8, 2016, 8 pages. |
| International Search Report for International Application No. PCT/CN2014/090439, dated Feb. 3, 2015, 2 pages. |
| Office Action dated May 3, 2017 for U.S. Appl. No. 15/109,121, 133 pages. |
| International Search Report for International Application No. PCT/CN2014/090440, dated Feb. 10, 2015, 2 pages. |
| Office Action dated Nov. 15, 2017 for U.S. Appl. No. 15/109,121, 53 pages. |
| Notice of Allowance dated Jul. 17, 2018 for U.S. Appl. No. 15/109,121, 33 pages. |
| Number | Date | Country | |
|---|---|---|---|
| 20160330559 A1 | Nov 2016 | US |
