Patent Application
20080049830
The details of the present invention, both as to its structure and operation, may be gleaned in part by study of the accompanying drawings, in which like reference numerals refer to like parts, and in which:
Certain embodiments as disclosed herein provide for a system and method for processing separate streams of image frames from a single source or from more than one source to provide a single stream of combined image frames. After reading this description it will become apparent to one skilled in the art how to implement the invention in various alternative embodiments and alternative applications. However, although various embodiments of the present invention will be described herein, it is understood that these embodiments are presented by way of example only, and not limitation. As such, this detailed description of various alternative embodiments should not be construed to limit the scope or breadth of the present invention as set forth in the appended claims.
As illustrated in
In the embodiment illustrated in
The receiver station may be a remote station to which the encoded image stream is transmitted for further processing, or may be a local station where the encoded image stream is simply stored until needed. The compressed single image stream will take up less storage space than the separate, uncompressed video or other image streams 1, 2, 3.
In
The image frame from the first stream and the image frame from the second stream which are combined in module 35 may be synchronized in time, but this may not be essential and the image frames which are combined may be unsynchronized in other embodiments.
Although
MPEG encoder module 36 will receive the single output stream of successive combined image frames from image combiner module 35 and will use the MPEG standard video compression technique to produce the encoded output stream 21. The output data stream 21 may be provided to a local data processing unit or stored in a local data storage unit for processing or viewing at a later time, or may be transmitted over a network 22 to a receiving station for further processing, as illustrated in
As illustrated in
It will be understood that image combiner and codec modules of
Although multi-stream MPEG-4 codecs are available, they are cost prohibitive in cameras. The above arrangement allows a less expensive, single stream MPEG-4 encoder to be used for encoding image streams from multiple sources. MPEG-encoding of video uses key frames and difference frames. The video from each source is potentially very different, making it inefficient to send frames from different sources to a codec that supports only one input stream in an interleaved or sequential fashion. Sending single streams from separate sources sequentially through a codec will take more time. Instead, as described above, frames from different sources are located in separate portions of a single combined image frame, which can then be sent to the codec module 36 as if it was a single source of video.
In the illustrated embodiment, wireless communication device 650 comprises an antenna 652, a multiplexor 654, a low noise amplifier (“LNA”) 656, a power amplifier (“PA”) 658, a modulation circuit 660, and a baseband processor 662. A central processing unit (“CPU”) 668 with a data storage area 670 is connected to the baseband processor 662, and a hardware interface 672 is connected to the baseband processor.
In the wireless communication device 650, radio frequency (“RF”) signals are transmitted and received by antenna 652. Multiplexor 654 acts as a switch, coupling antenna 652 between the transmit and receive signal paths. In the receive path, received RF signals are coupled from a multiplexor 654 to LNA 656. LNA 656 amplifies the received RF signal and couples the amplified signal to a demodulation portion of the modulation circuit 660.
Typically modulation circuit 660 will combine a demodulator and modulator in one integrated circuit (“IC”). The demodulator and modulator can also be separate components. The demodulator strips away the RF carrier signal leaving a base-band receive signal, which is sent from the demodulator output to the base-band processor 662.
The baseband processor 662 also codes digital signals for transmission and generates a baseband transmit signal that is routed to the modulator portion of modulation circuit 660. The modulator mixes the baseband transmit signal with an RF carrier signal generating an RF transmit signal that is routed to the power amplifier 658. The power amplifier 658 amplifies the RF transmit signal and routes it to the multiplexor 654 where the signal is switched to the antenna port for transmission by antenna 652.
At the transmitting end of the system illustrated in
The baseband processor 662 is also communicatively coupled with the central processing unit 668. The central processing unit 668 has access to data storage area 670. The central processing unit 668 is preferably configured to execute instructions (i.e., computer programs or software) that can be stored in the data storage area 670. Computer programs can also be received from the baseband processor 662 and stored in the data storage area 670 or executed upon receipt.
The central processing unit 668 is also preferably configured to receive notifications from the hardware interface 672 when new devices are detected by the hardware interface. Hardware interface 672 can be a combination electromechanical detector with controlling software that communicates with the CPU 668 and interacts with new devices. The hardware interface 672 may be a firewire port, a USB port, a Bluetooth or infrared wireless unit, or any of a variety of wired or wireless access mechanisms. Examples of hardware that may be linked with the device 650 include data storage devices, computing devices, headphones, microphones, and the like.
The computer system 750 preferably includes one or more processors, such as processor 752. Additional processors may be provided, such as an auxiliary processor to manage input/output, an auxiliary processor to perform floating point mathematical operations, a special-purpose microprocessor having an architecture suitable for fast execution of signal processing algorithms (e.g., digital signal processor), a slave processor subordinate to the main processing system (e.g., back-end processor), an additional microprocessor or controller for dual or multiple processor systems, or a coprocessor. Such auxiliary processors may be discrete processors or may be integrated with the processor 752.
The processor 752 is preferably connected to a communication bus 754. The communication bus 754 may include a data channel for facilitating information transfer between storage and other peripheral components of the computer system 750. The communication bus 754 further may provide a set of signals used for communication with the processor 752, including a data bus, address bus, and control bus (not shown). The communication bus 754 may comprise any standard or non-standard bus architecture such as, for example, bus architectures compliant with industry standard architecture (“ISA”), extended industry standard architecture (“EISA”), Micro Channel Architecture (“MCA”), peripheral component interconnect (“PCI”) local bus, or standards promulgated by the Institute of Electrical and Electronics Engineers (“IEEE”) including IEEE 488 general-purpose interface bus (“GPIB”), IEEE 696/S-100, and the like.
Computer system 750 preferably includes a main memory 756 and may also include a secondary memory 758. The main memory 756 provides storage of instructions and data for programs executing on the processor 752. The main memory 756 is typically semiconductor-based memory such as dynamic random access memory (“DRAM”) and/or static random access memory (“SRAM”). Other semiconductor-based memory types include, for example, synchronous dynamic random access memory (“SDRAM”), Rambus dynamic random access memory (“RDRAM”), ferroelectric random access memory (“FRAM”), and the like, including read only memory (“ROM”).
The secondary memory 758 may optionally include a hard disk drive 760 and/or a removable storage drive 762, for example a floppy disk drive, a magnetic tape drive, a compact disc (“CD”) drive, a digital versatile disc (“DVD”) drive, etc. The removable storage drive 762 reads from and/or writes to a removable storage medium 764 in a well-known manner. Removable storage medium 764 may be, for example, a floppy disk, magnetic tape, CD, DVD, etc.
The removable storage medium 764 is preferably a computer readable medium having stored thereon computer executable code (i.e., software) and/or data. The computer software or data stored on the removable storage medium 764 is read into the computer system 750 as electrical communication signals 778.
In alternative embodiments, secondary memory 758 may include other similar means for allowing computer programs or other data or instructions to be loaded into the computer system 750. Such means may include, for example, an external storage medium 772 and an interface 770. Examples of external storage medium 772 may include an external hard disk drive or an external optical drive, or and external magneto-optical drive.
Other examples of secondary memory 758 may include semiconductor-based memory such as programmable read-only memory (“PROM”), erasable programmable read-only memory (“EPROM”), electrically erasable read-only memory (“EEPROM”), or flash memory (block oriented memory similar to EEPROM). Also included are any other removable storage units 772 and interfaces 770, which allow software and data to be transferred from the removable storage unit 772 to the computer system 750.
Computer system 750 may also include a communication interface 774. The communication interface 774 allows software and data to be transferred between computer system 750 and external devices (e.g. printers), networks, or information sources. For example, computer software or executable code may be transferred to computer system 750 from a network server via communication interface 774 which may be wired or wireless. Examples of communication interface 774 include a modem, a network interface card (“NIC”), a communications port, a Personal Computer Memory Card International Association (“PCMCIA”) slot and card, an infrared interface, and an IEEE 1394 fire-wire, just to name a few.
Communication interface 774 preferably implements industry promulgated protocol standards, such as Ethernet IEEE 802 standards, Fiber Channel, digital subscriber line (“DSL”), asynchronous digital subscriber line (“ADSL”), frame relay, asynchronous transfer mode (“ATM”), integrated digital services network (“ISDN”), personal communications services (“PCS”), transmission control protocol/Internet protocol (“TCP/IP”), serial line Internet protocol/point to point protocol (“SLIP/PPP”), and so on, but may also implement customized or non-standard interface protocols as well.
Software and data transferred via communication interface 774 are generally in the form of electrical communication signals 778. These signals 778 are preferably provided to communication interface 774 via a communication channel 776. Communication channel 776 carries signals 778 and can be implemented using a variety of wired or wireless communication means including wire or cable, fiber optics, conventional phone line, cellular phone link, wireless data communication link, radio frequency (RF) link, or infrared link, just to name a few.
Computer executable code (i.e., computer programs or software) is stored in the main memory 756 and/or the secondary memory 758. Computer programs can also be received via communication interface 774 and stored in the main memory 756 and/or the secondary memory 758. Such computer programs, when executed, enable the computer system 750 to perform the various functions of the present invention as previously described.
In this description, the term “computer readable medium” is used to refer to any media used to provide computer executable code (e.g., software and computer programs) to the computer system 750. Examples of these media include main memory 756, secondary memory 758 (including hard disk drive 760, removable storage medium 764, and external storage medium 772), and any peripheral device communicatively coupled with communication interface 774 (including a network information server or other network device). These computer readable mediums are means for providing executable code, programming instructions, and software to the computer system 750.
In an embodiment that is implemented using software, the software may be stored on a computer readable medium and loaded into computer system 750 by way of removable storage drive 762, interface 770, or communication interface 774. In such an embodiment, the software is loaded into the computer system 750 in the form of electrical communication signals 778. The software, when executed by the processor 752, preferably causes the processor 752 to perform the inventive features and functions previously described herein.
Various embodiments may also be implemented primarily in hardware using, for example, components such as application specific integrated circuits (“ASICs”), or field programmable gate arrays (“FPGAs”). Implementation of a hardware state machine capable of performing the functions described herein will also be apparent to those skilled in the relevant art. Various embodiments may also be implemented using a combination of both hardware and software.
Those of skill in the art will appreciate that the various illustrative units, modules and method steps described in connection with the above described figures and the embodiments disclosed herein can often be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative units, modules and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled persons can implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the invention. In addition, the grouping of functions within a unit, module or step is for ease of description. Specific functions or steps can be moved from one module or unit to another without departing from the invention.
Moreover, the various illustrative units, modules and methods described in connection with the embodiments disclosed herein can be implemented or performed with a general purpose processor, a digital signal processor (“DSP”), an application specific integrated circuit (“ASIC”), a field programmable gate array (“FPGA”) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general-purpose processor can be a microprocessor, but in the alternative, the processor can be any processor, controller, microcontroller, or state machine. A processor can also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.
Additionally, the steps of a method described in connection with the embodiments disclosed herein can be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module can reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium including a network storage medium. An exemplary storage medium can be coupled to the processor such the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium can be integral to the processor. The processor and the storage medium can also reside in an ASIC.
The above description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles described herein can be applied to other embodiments without departing from the spirit or scope of the invention. Thus, it is to be understood that the description and drawings presented herein represent a presently preferred embodiment of the invention and are therefore representative of the subject matter which is broadly contemplated by the present invention. It is further understood that the scope of the present invention fully encompasses other embodiments that may become obvious to those skilled in the art and that the scope of the present invention is accordingly limited by nothing other than the appended claims.
