The disclosure relates to multi-media reproduction systems and methods.
In the fields of video gaming, television and video entertainment it is often the case that numerous persons wish to view separate video contents in physical proximity to each other. To this end, many televisions offer features known as picture-in-picture or split screen viewing, by which video images from different sources are shown on the television at the same time. It is also common for more than one video monitor to be placed in a room at the same time for viewing different contents. In most cases audio content corresponding to the video content is presented simultaneously. However, there is a recurring drawback involving viewers of one video image being distracted by the acoustics corresponding to another video content. Headphones are commonly used to provide the recipients of a video content with the respective audio content, but headphones are considered unpleasant and annoying in the situations described above. Thus, there is a general need for a system or method which facilitates split screen viewing in connection with the reproduction of corresponding audio content.
An exemplary multi-media system includes a display array comprising at least one electronic visual display and a video control module configured to operate the display array in a split screen mode to provide different video content at least at two different recipient positions. The multi-media system further includes a loudspeaker arrangement comprising at least two identical or similar loudspeakers so that the loudspeaker arrangement has adjustable, controllable or steerable directivity characteristics. The multi-media system further includes an audio control module configured to drive, adjust, control and/or steer the loudspeaker arrangement so that at least one acoustically isolated acoustic wave field is generated at each of the at least two recipient positions to provide different audio content at the at least two different recipient positions.
An exemplary multi-media reproduction method includes reproducing different video content with a display array that comprises at least one electronic visual display at least at two different recipient positions, and reproducing different audio content with a loudspeaker arrangement comprising at least two identical or similar loudspeakers so that the loudspeaker arrangement has adjustable, controllable or steerable directivity characteristics. The method further includes driving, adjusting, controlling and/or steering the loudspeaker arrangement so that at least one acoustically isolated acoustic wave field is generated at each of the at least two recipient positions to provide different audio content at the at least two different recipient positions.
Other systems, methods, features and advantages will be, or will become, apparent to one with skill in the art upon examination of the following figures and detailed description. It is intended that all such additional systems, methods, features and advantages be included within this description, be within the scope of the invention, and be protected by the following claims.
The systems, arrangements, assemblies and methods may be better understood with reference to the following drawings and description. The components in the figures are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the invention. Moreover, in the figures, like referenced numerals designate corresponding parts throughout the different views.
Two-dimensional or three-dimensional audio may be realized using a sound field description by a technique called Higher-Order Ambisonics. Ambisonics is a full-sphere surround sound technique which may cover, in addition to the horizontal plane, sound sources above and below the listener. Unlike other multichannel surround formats, its transmission channels do not carry loudspeaker signals. Instead, they contain a loudspeaker-independent representation of a sound field, which is then decoded to the listener's loudspeaker setup. This extra step allows a music producer to think in terms of source directions rather than loudspeaker positions, and offers the listener a considerable degree of flexibility as to the layout and number of loudspeakers used for playback. Ambisonics can be understood as a three-dimensional extension of mid/side (M/S) stereo, adding additional difference channels for height and depth. In terms of First-Order Ambisonics, the resulting signal set is called B-format. The spatial resolution of First-Order Ambisonics is quite low. In practice, that translates to slightly blurry sources, and also to a comparably small usable listening area or sweet spot.
The resolution can be increased and the sweet spot enlarged by adding groups of more selective directional components to the B-format. In terms of Second-Order Ambisonics, these no longer correspond to conventional microphone polar patterns, but look like, e.g., clover leaves. The resulting signal set is then called Second-, Third-, or collectively, Higher-Order Ambisonics (HOA). However, common applications of the HOA technique require, dependent on whether a two-dimensional (2D) and three-dimensional (3D) wave field is processed, specific spatial configurations notwithstanding whether the wave field is measured (decoded) or reproduced (encoded): Processing of 2D wave fields requires cylindrical configurations and processing of 3D wave fields requires spherical configurations, each with a regular distribution of the microphones or loudspeakers.
Alternative use of the multiple-input multiple-output technology instead of the Ambisonics technology allows for creating a two-dimensional higher-order loudspeaker of the first order even with only two lower-order loudspeakers. Other options include the creation of three-dimensional higher-order loudspeakers with four lower-order loudspeakers that are regularly distributed on a sphere using the Ambisonics technology and with four lower-order loudspeakers that are regularly distributed on a sphere using the multiple-input multiple-output technology. Furthermore, the higher-order loudspeaker assemblies may be arranged other than in a straight line, e.g., on an arbitrary curve in a logarithmically changing distance from each other or in a completely arbitrary, three-dimensional arrangement in a room.
The four lower-order loudspeakers 111 to 114, 121 to 124, 131 to 134 may be substantially the same size and have a peripheral front surface, and an enclosure having a hollow, cylindrical body and end closures. The cylindrical body and end closures may be made of material that is impervious to air. The cylindrical body may include openings therein. The openings may be sized and shaped to correspond with the peripheral front surfaces of the lower-order loudspeakers 111 to 114, 121 to 124, 131 to 134, and have central axes. The central axes of the openings may be contained in one radial plane, and the angles between adjacent axes may be identical. The lower-order loudspeakers 111 to 114, 121 to 124, and 131 to 134 may be disposed in the openings and hermetically secured to the cylindrical body. However, additional loudspeakers may be disposed in more than one such radial plane, e.g., in one or more additional planes above and/or below the radial plane described above. Optionally, the lower-order loudspeakers 111 to 114, 121 to 124, 131 to 134 may each be operated in a separate, acoustically closed volume 115 to 118, 125 to 128, 135 to 138 in order to reduce or even prevent any acoustic interactions between the lower-order loudspeakers of a particular higher-order loudspeaker assembly. Furthermore, the lower-order loudspeakers 111 to 114, 121 to 124, 131 to 134 may each be arranged in a dent, hole, recess or the like. Additionally or alternatively, a wave guiding structure such as but not limited to a horn, an inverse horn, an acoustic lens etc. may be arranged in front of the lower-order loudspeakers 111 to 114, 121 to 124, 131 to 134.
A control module 140 receives, e.g., three Ambisonic signals 144, 145, 146 to process the Ambisonic signals 144, 145, 146 in accordance with steering information 147, and to drive and steer the higher-order loudspeakers 101, 102, 103 based on the Ambisonic signals 144, 145, 146 so that at least one acoustic wave field is generated at least at one position that is dependent on the steering information. The control module 140 comprises beamformer modules 141, 142, 143 that drive the lower-order loudspeakers 111 to 114, 121 to 124, 131 to 134. Examples of beamformer modules are described further below.
Furthermore, a center channel, e.g., the C channel, may be reproduced at the sweet area by way of two high-order loudspeakers. Alternatively, a third high-order loudspeaker, disposed between the two high-order loudspeakers, may be used to separately direct the Lf and Rf channels and the C channel to the sweet area. Since with three high-order loudspeakers each channel is reproduced by a separate unit, the spatial sound impression of a listener at the sweet area can be further improved. Furthermore, with each additional high-order loudspeaker added to the high-order soundbar a more diffuse sound impression can be realized and further channels such as, e.g., effect channels may be radiated from the rear side of the high-order soundbar, which is in the present example from the rear side of the TV set to, e.g., the rear wall where the sound provided by the effect channels is diffused.
In contrast to common soundbars in which the lower-order loudspeakers are arranged in line, higher-order soundbars provide more options for the positioning of the directional sound sources, e.g., on the side and rear, so that in a common listening environment such as a living room, a directivity characteristic that is almost independent from the spatial direction can be achieved with higher-order soundbars. For example, a common side bar having 14 lower-order loudspeaker equidistantly distributed inline over a distance of 70 cm can only generate virtual sound sources in an area of maximum ±90° (degree) from the front direction, while higher-order soundbars allow for virtual sound sources in an area of ±180°.
A television set 316 is arranged at the front wall (e.g., above the higher order soundbar) and in line of sight of the sofa 321. The front left (Lf) channel higher-order loudspeaker 310 and the front right (Rf) channel higher-order loudspeaker 311 are arranged under the left and right corners of the television set 316 and the center (C) higher-order loudspeaker 322 is arranged below the middle of television set 316. The low frequency effects (Sub) channel loudspeaker 323 is disposed in the corner between the front wall and the right wall. The loudspeaker arrangement on the rear wall, including the rear left (Ls) channel higher-order loudspeaker 324 and the rear right (Rs) channel under loudspeaker 312, do not share the same center line as the loudspeaker arrangement on the front wall including the front left (Lf) channel loudspeaker 310, the front right (Rs) channel loudspeaker 311, and low frequency effects (Sub) channel loudspeaker 323. An exemplary sweet area 314 may be on the sofa 321 with the table 320 and the television set 316 in front. As can be seen, the loudspeaker setup shown in
In the set-up shown in
If effect channels or surround channels (e.g., the Ls and Rs channels) are to be disposed between the sweet area and the rear wall, where not sufficient room may be available, higher-order loudspeaker may be implemented as “bulbs” in the same sockets as light bulbs. Such bulb-type higher-order loudspeakers may provide not only sound, but also light in connection with space-saving light emitting diodes. The power required for the bulb-type higher-order loudspeakers (including signal processing and amplifying circuitry) can be supplied via the mains as with common light bulbs. Signals to be reproduced (and others if required) may be provided via a wired (e.g., power-line) or wireless connection such as Bluetooth or WLAN.
By way of a set-up similar to that shown in
For each of the higher-order loudspeakers of the soundbar (and the other higher-order loudspeakers) a beamformer module 500 or 600 as depicted in
The N modified and weighted Ambisonic signals 506 are then input into the regularization sub-module 509, which includes the necessary radial filter Wn,mσ(ω) for considering the susceptibility of the playback device Higher-Order-Loudspeaker (HOL) preventing e.g. a given White-Noise-Gain (WNG) threshold from being undercut. Output signals 510 [Wn,mσ(ω) Cn,mσYn,mσ(θDes, φDes)] of the regularization sub-module 509 are then transformed, e.g. by pseudo-inverse Y+=(YTY)−1YT, which simplifies to
if the Q lower-order loudspeakers are arranged at the body of the higher-order loudspeakers in a regular fashion, into Q loudspeaker signals 508 [y1(n), . . . , yQ(n)] by the matrixing sub-module 507 using a N×Q weighting matrix as shown in
An example of a simple Ambisonic panner (or encoder) takes an input signal, e.g., a source signal S and two parameters, the horizontal angle θ and the elevation angle φ. It positions the source at the desired angle by distributing the signal over the Ambisonic components with different gains for the corresponding Ambisonic signals W (Y0,0+1(θ, φ), X (Y1,1+1(θ, φ), Y (Y1,1−1(θ, φ) and Z (Y1,0+1(θ, φ):
Being omnidirectional, the W channel always delivers the same signal, regardless of the listening angle. In order that it has more-or-less the same average energy as the other channels, W is attenuated by w, i.e., by about 3 dB (precisely, divided by the square root of two). The terms for X, Y, Z may produce the polar patterns of figure-of-eight. Taking their desired weighting values at angles θ and φ(x, y, z), and multiplying the result with the corresponding Ambisonic signals (X, Y, Z), the output sums end up in a figure-of-eight radiation pattern pointing now to the desired direction, given by the azimuth θ and elevation φ, utilized in the calculation of the weighting values x, y and z, having an energy content that can cope with the W component, weighted by w. The B-format components can be combined to derive virtual radiation patterns that can cope with any first-order polar pattern (omnidirectional, cardioid, hypercardioid, figure-of-eight or anything in between) and point in any three-dimensional direction. Several such beam patterns with different parameters can be derived at the same time to create coincident stereo pairs or surround arrays.
Referring now to
For example, when superimposing the five basic functions depicted in
The matrixing module 601 may be implemented as a multiple-input multiple-output system that provides an adjustment of the output signals of the higher-order loudspeakers so that the radiation patterns approximate as closely as possible the desired spherical harmonics, as shown e.g. in
To adjust or (singularly or permanently) adapt the sound reproduced by the soundbar to the specific room conditions and the specific requirements of the sweet area of the loudspeaker set-up, which includes the high-order soundbar and, possibly, other (high-order) loudspeakers, the wave field needs to be measured and quantified. This may be accomplished by way of an array of microphones (microphone array) and a signal processing module able to decode the given wave-field, that, e.g., form a higher-order Ambisonic system to determine the wave field in three dimensions or, which may be sufficient in many cases, in two dimensions, which requires fewer microphones. For the measurement of a two-dimensional wave field, S microphones are required to measure sound fields up to the Mth order, wherein S=2M+1. In contrast, for a three-dimensional wave field, S=(2M+1)2 microphones are required. Furthermore, in many cases it is sufficient to dispose the microphones (equidistantly) on a circle line. The microphones may be disposed on a rigid or open sphere or cylinder, and may be operated, if needed, in connection with an Ambisonic decoder. In an alternative example, the microphone array 314 may be integrated in one of the higher-order loudspeakers (not shown).
Furthermore, a master-slave loudspeaker set-up may be employed. The master unit may include a higher-order soundbar, a microphone array, and a signal processing and steering module. The slave unit(s) may include (a) further higher-order loudspeaker(s) electrically connected (wired or wireless) to the master unit. The microphone array may be detachable, so that it can be used standing alone to conduct the measurements, e.g., in connection with a battery driven power supply and a wireless connection to the master unit. When the microphone array is attached to the master unit again it can be used for other tasks such as speech control of the audio system (e.g., volume control, content selection), or hands-free operation of a telephone interface (e.g., a teleconference system) including adapting (steering) the speaker. The sound reproduction system may also include a DOA module for determining the direction of arrival (DOA) of a sound wave, which, in this application, would suffice to be purely triggered by speech signals, i.e., no optical DOA detection is required.
The DOA module may include one or more optical detectors such as one or more cameras to detect the position of a listener and to reposition the sweet area by steering the direction of the higher-order loudspeakers. In this case an optical DOA detector, optionally in combination with the previously mentioned purely speech triggered DOA detection, is necessary since now the sound-field should be adjusted in respect to the current position of the listener, which by no means implies that the person has to be speaking. An exemplary optical detector is shown in
Referring to
In the arrangement shown in
Alternatively, as shown in
In the arrangement shown in
Paired with the film are viewing glasses 1305 and 1306. These viewing glasses 1305 and 1306 are polarized to correspond with the matching polarized field 1303 or 1304. There is at least one pair of viewing glasses corresponding with each polarized field 1303 and 1304 of the screen 1301. Thus, in the exemplary arrangement where the screen 1301 is split into two polarized fields 1303 and 1304 having polarizations perpendicular to each other, there will also be two pairs of viewing glasses 1305 and 1306, one having horizontal polarization and the other having vertical polarization. With regard to the structure of the glasses 1305 and 1306, any type of removable glasses, add-ons or clip-on eyewear which effectively allows the users 1307 and 1308, i.e., recipients at respective recipient positions, to wear the glasses 1305 and 1306 etc. may be used. The display 1302 may be disposed on and electrically coupled through an audio-video control module (not shown) to a loudspeaker arrangement 1309 such as the sound reproduction system 100 described above in connection with
Particularly in an automotive environment such as the interior of a car, polarizing glasses can be impractical or even disturbing.
The display 1401 and/or the video controller 1408 may include one or more luminance modulation units (not shown) to visualize respective sequences of images being provided by means of multiple image video sources. In the case of a single luminance modulation unit, temporal or spatial multiplexing is applied to render the images of the respective sequences. Video (and audio) sources may be DVD players, receivers for receiving broadcast video, set-top boxes, satellite-tuners, VCR players or any types of computers or processors arranged to render graphical images. The luminance modulation units can be based on known display technologies like CRT (Cathode Ray Tube), LCD (Liquid Crystal Display) or PDP (Plasma display panel).
The display 1401 further comprises optical means (not shown) to direct a first sequence of images in the first direction 1405, resulting in the first view 1404 (video content A), and to direct a second sequence of images in the second direction 1407, resulting in the second view 1406 (video content B). The first view 1404 can only be seen by the first user 1402 and the second view can only be seen by the second user 1406. The audio controller 1409 generates at least two sound fields that spatially correspond to the positions of the users 1402 and 1403, and whose audio contents correspond to video contents A and B. By using, e.g., an array of higher-order loudspeakers (e.g., in form of a higher-order soundbar), each of them having a versatile directivity, arbitrary wave fields can be approximated, even in reflective venues such as living rooms where home audio systems are typically installed. This is possible because, due to the use of higher-order loudspeakers, versatile directivities can be created, radiating the sound only in directions where no reflective surfaces exists, or deliberately making use of certain reflections if those turn out to positively contribute to the creation of a desired wave field to be approximated. Thereby, the approximation of the desired wave field at a desired position within the target room (e.g. a certain region at the couch in the living room) can be achieved by using adaptive methods, such as an adaptive multiple-input multiple-output (MIMO) system, given e.g. by the multiple-FXLMS filtered input least mean squared (multiple-FXLMS) algorithm, which could also operate not just in the time or spectral domain, but also in the so-called wave-domain.
Utilizing wave domain adaptive filters (WDAF) is of special interest, since this promises very good results in the approximation of the desired wave field. WDAF can be used if the recording device fulfills certain requirements. For example, circular (for 2D) or spherical microphone arrays (3D), equipped with regularly distributed microphones at the surface, may be used to record the wave field, having, depending on the desired order in which the wave field has to be recorded, respectively reproduced a number of microphones that have to be chosen accordingly. However, if beamforming filters are calculated using e.g. a MIMO system, arbitrary microphone arrays having different shapes and microphone distributions can be used as well to measure the wave field, leading to high flexibility in the recording device. The recording device can be integrated in a main unit of the complete new acoustic system. Thereby it can be used not only for the already mentioned recording task, but also for other needed purposes, such as enabling a speech control of the acoustic system to verbally control e.g. the volume, switching titles, and so on. Further, the main unit to which the microphone array is attached could also be used as a stand-alone device e.g. as a teleconferencing hub or as a portable music device with the ability to adjust the acoustic in dependence of the relative position of the listener to the device, which is only possible if a video camera is integrated in the main unit as well
Loudspeaker arrangements with adjustable, controllable or steerable directivity characteristics include at least two identical or similar loudspeakers which may be arranged in one, two or more loudspeaker assemblies, e.g. one loudspeaker assembly with two loudspeakers or two loudspeaker assemblies with one loudspeaker each. The loudspeaker assemblies may be distributed somewhere around the display(s), e.g., in a room. With the help of arrays of higher-order loudspeakers, it is possible to create wave fields of the same quality, but with fewer devices as compared with ordinary loudspeakers. An array of higher-order loudspeakers can be used to create an arbitrary wave field in real, e.g., reflective environments. The necessary recording device (microphone array) can be of arbitrary shape and microphone distribution if special beamforming concepts are used, which can be achieved e.g. by using a suitable adaptive MIMO system, such as the multiple-FXLMS algorithm. This new concept is able to create a much more realistic acoustic impression, even in reflective environments such as those given in living rooms.
The description of embodiments has been presented for purposes of illustration and description. Suitable modifications and variations to the embodiments may be performed in light of the above description. The described assemblies, systems and methods are exemplary in nature, and may include additional elements or steps and/or omit elements or steps. As used in this application, an element or step recited in the singular and proceeded with the word “a” or “an” should be understood as not excluding plural of said elements or steps, unless such exclusion is stated. Furthermore, references to “one embodiment” or “one example” of the present disclosure are not intended to be interpreted as excluding the existence of additional embodiments that also incorporate the recited features. The terms “first,” “second,” and “third,” etc. are used merely as labels, and are not intended to impose numerical requirements or a particular positional order on their objects. A signal flow chart may describe a system, method or software implementing the method dependent on the type of realization. e.g., as hardware, software or a combination thereof. A module may be implemented as hardware, software or a combination thereof.
Number | Date | Country | Kind |
---|---|---|---|
16150043.4 | Jan 2016 | EP | regional |
16174534.4 | Jun 2016 | EP | regional |
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/EP2016/081010 | 12/14/2016 | WO | 00 |