Imaging Panels Including Arrays Of Audio And Video Input And Output Elements

Abstract

Arrays of audio and video input and output elements are combined in various combinations in an imaging panel. The audio input and output elements may take the form of miniature speakers and microphones manufactured utilizing CMOS-MEMS technology. The video imaging input and output elements may take the form of display elements such as LCD's and elements such as CMOS light detectors or CCD's capable of sensing an image. The arrays of audio and video elements may be interspersed or partially segregated, and be distributed homogeneously or non-homogeneously.

Description

DESCRIPTION OF THE DRAWINGS.

FIG. 1 is diagrammatic representation of an imaging panel having arrays of audio and video elements thereon according to the invention.

FIG. 2 is a diagrammatic representation of an imaging panel having arrays of audio input elements and audio output elements combined with arrays of video imaging input elements and video imaging output elements together with a block diagram of associated components.

FIG. 3 is a diagrammatic representation of an imaging panel having arrays of audio output elements and video imaging output elements and a block diagram of associated components.

FIG. 4 is a diagrammatic representation of an imaging panel having arrays of audio input elements and video imaging input elements together with a block diagram of associated components.

FIG. 5 is a diagrammatic representation of an imaging panel having arrays of audio input elements and video imaging output elements with a block diagram of the associated components.

FIG. 6 is a diagrammatic representation of an imaging panel having arrays of audio output elements and video imaging input elements with a block diagram of the associated components.

FIG. 7 is a diagrammatic representation of an imaging panel having arrays of audio input elements, audio output elements, and video imaging output elements together with a block diagram of associated components.

FIG. 8 is a diagrammatic representation of audio input and output elements combined on an imaging panel with video imaging input elements and a block diagram of associated components.

FIG. 9 is a diagrammatic representation of an imaging panel having arrays of audio output elements combined with video imaging input and output elements and a block diagram representation of associated components.

FIG. 10 is a diagrammatic representation of an imaging panel having arrays of audio input elements and arrays of video imaging input and output elements together with a block diagram representation of associated components.

FIG. 11 is a plan view of a cell phone with an imaging panel having video and audio elements mounted thereon.

FIG. 12 is a front view of a television display formed by video imaging elements on an imaging panel also including audio signal input speaker elements.

DETAILED DESCRIPTION

In the following detailed description, certain specific terminology will be employed for the sake of clarity and a particular embodiment described in accordance with the requirements of 35 USC 112, but it is to be understood that the same is not intended to be limiting and should not be so construed inasmuch as the invention is capable of taking many forms and variations within the scope of the appended claims.

Referring to FIG. 1, according to the present invention, audio elements 10 and video elements 12 are combined in a generally planar imaging panel 14 to provide related functions in one device. The elements are shown diagrammatically as being relatively large for clarity, but would typically be much smaller i.e., the size of pixels in computer monitor displays or smaller.

These audio elements may comprise one or more arrays of audio signal input elements (i.e., sound reproducing elements such as CMOS-MEMS speakers) and audio signal output elements (i.e., sound sensing elements such as CMOS-MEMS microphones) such as described in U.S. Pat. No. 6,936,524 B2 referred to above.

An array of video elements may comprise video imaging signal input elements (i.e., image producing elements such as LCD's) collectively forming a displayed image, and video imaging signal output elements (i.e., light sensing elements such as CMOS light sensors, CCD's, or photosensors) collectively providing a camera function as described in U.S. Pat. No. 7,034,866 B1, published application U.S. 2006/0007222A1, and published application WO 2004/107301 A1.

Each audio or video array may be comprised of groupings of constituent elements.

These video imaging signal input elements may be arranged in an array on the imaging panel 14 comprised of vertical and horizontal crossing rows as shown in FIG. 1, or various other patterns with various differing proportions of other arrays of audio and/or video elements; or, with equal numbers of each as required for a particular application. The distribution of each type of audio and video element may be homogeneous throughout the imaging panel or non-homogeneous. As such, certain areas of the imaging panel may be devoid of particular audio or video elements. The various audio and video elements 10 and 12 may also be partially or completely segregated in the imaging panel 14. Such flexible layouts/arrangements of the various audio and video elements are also applicable to the various imaging panels exemplified by FIGS. 1-12. Typically, the imaging elements 12 which form an image would be greater in number than the audio elements 10 to achieve greater resolution of the images sensed or displayed.

Each video imaging signal input element V_i provides one pixel of the image displayed.

The audio elements are sized to be sufficiently small so as to not substantially interfere with the video image display produced by the array of video image display elements (i.e., LCD's). The audio elements may be sized on the order of an individual display pixel or smaller in order to achieve this, such as by use of the CMOS-MEMS audio elements referenced above.

Thus, audio and video imaging functions are enabled by the single imaging panel 14 to save manufacturing costs, save space, and provide for a thin profile one-piece design.

The specifications and functionality of each individual audio and video element in an imaging panel may differ from the specifications and functionality of other audio and video elements in the same imaging panel. This enables varying operating characteristics for individual elements or groups of elements in order to fulfill the requirements of each particular application. For example, speakers (or microphones) may have different frequency responses from other speakers (or microphones) in the same imaging panel. Similarly, video imaging display elements (i.e., LCD's) and video imaging sensing elements (i.e., CMOS MEMS camera elements) may have varying spectral responses compared to other video imaging display or sensing elements in the same imaging panel.

Referring to FIG. 2, both audio signal input elements A_i, i.e. CMOS-MEMS speakers, and audio signal output elements A_o, i.e. CMOS-MEMS microphones as well as video imaging signal output elements V_o (such as CMOS light sensors, or CCD's) and video imaging signal input elements V_i (such as LCD's) may all be combined in a single imaging panel 16.

Audio signal output elements A_o such as microphones are interspersed with audio signal input elements A_i such as speakers. Video imaging signal output elements V_o such as camera elements, i.e., CMOS light sensors, CCD's or similar devices are also included on the same imaging panel 16. Lenses (not shown) may be utilized in conjunction with the video imaging elements V_o in order to generate acceptable images for a particular application, such as described in U.S. 2006/0007222A1 and U.S. Pat. No. 7,034,866 B1 referenced above.

Also included are video imaging signal input elements V_i such as LCD's. The various elements A_o, A_i, V_o and V_i are depicted as being evenly interspersed on the imaging panel 16 but other distributions may be employed to achieve any desired particular purpose. The audio signal output elements A_o each generate audio output signals which are read by an audio output signal reader 18. These signals would typically be processed in an audio output signal processor 20 used to operate a utilization (such as an external speaker) or memory device 22 for storing the audio signals. Such audio output signal processor 20 may use technology and algorithms well known in the art to mix or otherwise produce signals with desired acoustical characteristics from signals obtained from audio signal output elements A_o.

The audio signal input elements A_i are connected to an audio input signal source 24 to drive the audio signal input elements such as the speakers described above to produce sound.

Similarly, the video imaging signal output elements V_o, such as CMOS light sensors or CCDs, transmit their output signals to a video imaging output signal reader 26. A video imaging output signal processor 28 then prepares the signals prior to utilizing the signals in a utilization/memory device 30. Such video imaging output signal processor 28 may use technology and algorithms well known in the art to assemble a final image from data obtained from video imaging signal output elements V₀.

A video imaging signal source 32 is connected to the video imaging signal input elements V_i (such as LCD's) to produce an image by the array of elements V_i.

Thus, the single imaging panel 16 is capable of recording and displaying video imaging signals and broadcasting and recording audio signals.

It should be understood that elements A_o, A_i, V_o, and V_i may be integrated into imaging panel 16 by using CMOS techniques, but the specific connections which are apparent to one skilled in the art are not shown in order to simplify and improve the clarity of the drawings. Of course, other fabrication processes and techniques may also be employed.

An example of the use of this imaging panel 16 is the bi-directional audio-visual conferencing monitor described above.

The diagrammatic representations of various functional groups and their interrelationships, e.g., audio output signal reader 18, audio output signal processor 20, and utilization/memory device 22 are examples of some of the possible configurations of various panel embodiments. Numerous other examples, variants, and configurations are possible and are apparent to those skilled in the art.

FIG. 3 shows an imaging panel 34 comprised of audio signal output elements A_o and video imaging signal output elements V_o. The audio signal output elements A_o are connected as before to an audio output signal reader 18. These signals may then be processed in an audio output signal processor 20 and then sent to a utilization/memory device 22.

Similarly, the video imaging signal output elements V_o, such as CMOS light sensors, CCDs, or photosensors, transmit their output signals to a video imaging output signal reader 26. A video imaging output signal processor 28 then prepares the signals prior to utilizing the signals in a utilization/memory device 30. This imaging panel 34 could be used in a video-audio recorder or other applications.

FIG. 4 depicts an imaging panel 44 wherein an array of audio signal input elements A_i is combined with an array of video imaging signal input elements V_i.

In this imaging panel 44, an audio input signal source 24 is connected to the audio signal input elements A_i, such as the CMOS MEMS speakers referred to above, and a video imaging input signal source 32 connected to the video imaging signal input elements V_i, such as LCD's. Thus a single device 44 can provide both sound and video playback as in a television or monitor, control panel, entertainment/game display, communication (i.e. mobile) devices, and a multitude of other applications.

FIG. 5 illustrates imaging panel 46 which incorporates audio signal input elements A_i, such as the CMOS MEMS speakers described above. The audio signal input elements A_i are connected to an audio input signal source 24. The imaging panel 46 also includes interspersed video imaging signal output elements V_o connected to a video imaging output signal reader 26; video imaging output signal processor 28 and then to a utilization/memory device 30.

This device allows an audio message to be broadcast, as for example audio instructions or prompts, to aid in camera use while video images are being recorded, and may also be employed in other applications.

FIG. 6 shows another embodiment in the form of imaging panel 48 which includes audio signal output elements A_o interspersed with video imaging signal input elements V_i.

An audio output signal reader 18 is connected to the audio signal output elements A_o whereby the audio output signal reader 18 is then connected to audio output signal processor 20 which in turn is connected to a memory/utilization device 22.

A video imaging input signal source 32 is connected to the video imaging signal input elements V_i to generate a display by the array 48. This device would be useful to provide visual prompting while recording an audio signal in utilization/memory device 22, and may also be used in other applications.

FIGS. 7-10 illustrate some various other possible combinations of audio and video elements that may be combined in an imaging panel.

FIG. 7 shows an imaging panel 50 in which audio signal input and output elements A_i, A_o are combined with video imaging signal output elements V_o. The audio signal input elements A_i, such as CMOS MEMS speakers are connected to an audio input signal source 24, while the audio signal output elements A_o are connected to an audio output signal reader 18, then connected to an audio output signal processor 20, and then connected to a utilization/memory device 22.

The video imaging signal output elements V_o are connected to a video imaging output signal reader 26, in turn connected to a video imaging output signal processor 28 and then utilization/memory device 30. This device would be useful in video monitoring applications such as where audible 2-way communications between medical personnel and a patient can take place while the medical personnel can concurrently view the patient. In this application, there is no need for the patient to view the medical staff. Other applications for this device are also applicable.

The FIG. 8 depicted imaging panel combines audio signal input and output elements A_i, A_o, with video imaging signal input V_i elements on an imaging panel 52. The audio signal input elements A_i are connected to an audio input signal source 24, and the audio signal output elements A_o are connected to an audio output signal reader 18, in turn connected to an audio output signal processor 20 and then utilization or memory device 22.

The video imaging signal input elements V_i are connected to a video imaging input signal source 32. Such a device could combine a visual display with an audio recorder with a playback capability, such as in a display monitor with bi-directional audio capability. This device may also be employed in other applications.

FIG. 9 shows an imaging panel 54 combining video imaging signal input and output elements V_i and V_o with audio signal output elements A_o. The video image signal input elements V_i, such as LCD's are connected to a video imaging input signal source 32.

The video imaging signal output elements V_o, are connected to a video imaging output signal reader 26 in turn connected to a video imaging output signal processor 28 and then utilization/memory device 30. Audio signal output elements A_o are connected to an audio output signal reader 18, in turn connected to an audio output signal processor 20 and then utilization or memory device 22. Such a device 54 could be used in a video recorder/display device without audio playback. An example of an application of this device 54 is in an instructor's conferencing monitor used in remote teaching environments where an instructor's image is captured for presentation to a room of remotely located students, while a video image of the students is presented to the instructor, as the instructor provides verbal lessons to said students as the instructor's voice is captured by audio signal output elements A_o.

FIG. 10 shows an imaging panel 56 which includes video imaging signal input and output elements V_i, V_o combined with audio signal input elements A_i.

The video imaging signal input elements V_i, are connected to a video imaging input signal source 32. The video imaging signal output elements V_o are connected to a video imaging output signal reader 26, in turn connected to a video imaging output signal processor 28, which in turn is connected to a utilization or memory device 30. The audio signal input elements A_i are connected to an audio input signal source 24. Such imaging panel 56 could be used with a video recorder and display device having audio promptings during video recording, as well as in other applications.

FIG. 11 illustrates a cell phone 58 combined with an imaging panel 60 which has an upper region including an array of audio signal input elements A_i in the form of speakers such as CMOS MEMs speakers The entire imaging panel 60 includes video imaging signal input elements V_i to provide a visual display, and also may include video imaging signal output elements V_o to provide a cell phone camera.

The bottom region of imaging panel 60 has audio signal output elements A₀ arrayed therein in the form of microphones such as CMOS-MEMS microphones. Thus, the imaging panel 60 may provide both audio and video functions. The imaging panel 60 may also have some or all of the above described audio and video elements dispersed throughout the entire array 60 area.

It should be appreciated that the imaging panel 60 according to the invention doesn't necessarily require that the various audio and video elements be interspersed throughout the entire array area, but some regions may have some elements segregated or dispersed only therein.

Note that the above described imaging panel 60 of audio and video elements replaces the dedicated display component and microphone and speaker assemblies commonly found in mobile communication devices, thus freeing up the valuable limited space often dedicated both above and below the display screen to accommodate separate speaker and microphone assemblies. By replacing these three independent component assemblies (display, microphone, and speaker) with the one imaging panel 60 of this invention, the size/area of the display screen may thus be enlarged by utilizing the space previously utilized by the microphone and speaker assemblies. This enlarged display area is thus accomplished without enlarging the physical size of the cell phone. Other advantages of the use of imaging panel 60 include higher reliability due to a lower component count, less macro interconnections thus allowing faster assembly time, and reduced inventories of individual repair service parts.

The above described imaging panel may similarly be used in PDAs (personal digital assistants), various mobile and non-mobile communication devices, telephones, and numerous other devices such as control panels, game/entertainment interfaces/displays, advertising kiosks and displays, consoles, etc.

FIG. 12 shows a TV/Computer Monitor/Conferencing Monitor 62 in combination with an imaging panel 64 which is comprised of arrays of elements according to the present invention. Audio signal input elements A_i here may take the form of arrays of speakers occupying regions on either side of the imaging panel 64, while video imaging signal input elements V_i may occupy the entire area of the imaging panel 64. Thus, both audio and video are provided by a single device. For video conferencing, video imaging signal output elements V_o may also be included distributed across the entire imaging panel 64, or areas thereof to add image capture (camera) functionality. An array of audio output elements A_o such as MEMs microphones may also be distributed within the array of video elements to enable the capture of sounds directed towards monitor 62 to enable video conferencing. The audio/visual elements A_o. A_i, V_o and V_i may be distributed in a variety of positions, proportions, and distributions in order to fulfill the requirements of each particular application.

Note that the just described imaging panel based conferencing monitor 62 eliminates the traditional use of external peripherals such as cameras, microphones, and speakers along with their associated space occupying support fixtures and associated cables. The imaging panel implementation thus creates a robust, low weight, compact, reliable, self-contained audio/visual system.

The use of CMOS-MEMS speaker and microphone elements may advantageously be combined with CMOS light sensors as video imaging elements, as the attractive economics of CMOS techniques in manufacturing can be realized, but other fabrication processes and elements may also be employed.

The positions of the audio signal input and output element arrays may be located in a common plane of the imaging panel along with the video imaging signal input and output element arrays.

Each audio signal output or input element may occupy its own discrete position, or each position may contain various combinations of audio signal input elements, audio signal output elements, video image signal input elements, and/or video image signal output elements.

The text in this specification often refers to an “imaging panel”. This term is not meant to limit the scope of the medium which contains or supports the various discussed input and output elements (e.g., microphones, speakers, video imaging signal input and output elements) to a planar medium or structure that functions only as a display. Such input and output elements may occupy other two or three dimensional areas, regions, or spaces.

Another example of the use of the technology of the present invention is its implementation with reflective mirrors or transparent (or non-transparent) windows or other surfaces, media, or spaces. In such applications, audio signal input element arrays such as described above may be incorporated with such media so that a person may hear audio emanating from the media. Similarly, video imaging signal input element arrays such as described above may also be incorporated with the media to display information. Applications that incorporate this technology enable a transparent window to act as a television display and audio system whereby a person in a room obtains visual images and sound from a device, structure, or area that otherwise would simply function as i.e., a window. Such “smart windows” save table space that may not be available (to accommodate a traditional television) and also eliminates the need for a television stand. This audio-visual system would be connected to electronics that normally control television operation.

Due to the small size and visual non-obtrusiveness (or near unobtrusiveness) of the audio/visual element arrays, media such as a window still serves its original function. An on-off switch can simply disable the visual display (e.g., LCD) elements so that a person can see through the window without distraction. Such “smart windows” can effectively serve as an advertisement media in outdoors bus stops or advertising kiosks. Without taking up additional space, media that is merely used to provide a static or otherwise non audio/visual advertisement to a consumer can now be used to display various i.e. computer controlled images from various advertisers as well as video and sound. As previously described, audio signal output element arrays and video imaging signal output element arrays may also be incorporated into such implementations.

Claims

1. An imaging panel comprising: a panel;an array of video display elements extending over on said panel, each video element forming a pixel of images formed by activating said video display elements by video input signals received from a source of said video input signals so as to form said images;one or more arrays of audio elements extending over said imaging panel at least partially interspersed with said array of said video display elements, and said audio elements sized to be sufficiently small so as to not substantially interfere with images displayed by said video display elements and said audio elements capable of reproducing sound when receiving audio signals from an audio signal source and/or sensing sound and producing output signals corresponding to said sensed sound, whereby an image display and one or more audio functions are provided by said imaging panel.

2. The imaging panel according to claim 1, wherein said one or more arrays of audio elements includes an array of said audio elements capable of reproducing sound.

3. The imaging panel according to claim 1 wherein said one or more arrays of audio elements includes an array of said audio elements capable of sensing sound.

4. The imaging panel according to claim 2 wherein said one or more arrays of audio elements also include an array of said audio elements capable of sensing sound.

5. The imaging panel according to claim 1 further including an array of image sensing video elements collectively producing output signals corresponding to images sensed collectively by said array of image sensing video elements, said image sensing video elements small enough to not substantially interfere with images produced by said array of video display elements whereby said imaging panel also may function as a camera.

6. The imaging panel according to claim 1 wherein said audio elements include an array of audio elements capable of reproducing sound and comprise CMOS MEMS speakers.

7. The imaging panel according to claim 4 in combination with a cellular telephone.

8. The imaging panel according to claim 2 in combination with a television.

9. The imaging panel according to claim 2 in combination with a computer monitor.

10. The imaging panel according to claim 1 wherein said audio elements comprise CMOS-MEMS speakers or CMOS-MEMS microphones.

11. The imaging panel according to claim 1 wherein said video display elements comprise LCD's.

12. The imaging panel according to claim 5 wherein said image sensing video elements comprise CCD (charged coupled device) light sensors.

13. The imaging panel according to claim 5 wherein said image sensing video elements comprise CMOS light sensors.

14. The imaging panel according to claim 1 wherein one of said one or more arrays of audio elements are substantially completely interspersed with said video display elements.

15. The imaging panel according to claim 1 wherein at least some of said audio elements are segregated together on said panel.

16. The imaging panel according to claim 1 wherein said video display elements and said audio elements are distributed non-homogeneously over said imaging panel.

17. The imaging panel according to claim 1 wherein said video display elements and said audio elements are distributed homogeneously over said imaging panel.

18. An imaging panel comprising: a panel;an array of video elements extending over said panel, each video element comprising a pixel of images generated by sensing or displaying images by said video elements;one or more arrays of audio elements extending over said imaging panel at least partially interspersed with said array of said video elements and sized to be sufficiently small so as to not substantially interfere with images generated by said video elements, said audio elements capable of reproducing sound when receiving audio signals from an audio signal source and/or of sensing sound and producing output signals corresponding to said sensed sound, whereby one or more imaging functions are provided by said imaging panel.

19. The imaging panel according to claim 18, wherein said one or more arrays of audio elements includes an array of said audio elements capable of reproducing sound.

20. The imaging panel according to claim 18 wherein said one or more arrays of audio elements includes an array of said audio elements capable of sensing sound.

21. The imaging panel according to claim 19 wherein said one or more arrays of audio elements also include an array of said audio elements capable of sensing sounds.

22. The imaging panel according to claim 18 wherein said video elements include video display elements.

23. The imaging panel according to claim 22 wherein said video elements also including video sensing elements.

24. The imaging panel according to claim 18 wherein said video elements comprise video sensing elements.

25. The imaging panel according to claim 18 in combination with a cellular telephone.

26. The imaging panel according to claim 18 in combination with a computer monitor.

27. The imaging panel according to claim 18 in combination with a conferencing monitor.

28. The imaging panel according to claim 18 wherein said video elements and said audio elements are distributed non-homogeneously over said imaging panel.

29. The imagining panel according to claim 18 wherein said video elements and said audio elements are distributed homogeneously over said imaging panel.