The details of the present invention, both as to its structure and operation, may be gleaned in part by study of the accompanying drawings, in which like reference numerals refer to like parts, and in which:
Certain embodiments as disclosed herein provide for an image capture and display system and method. For example, one method as disclosed herein allows for different images to be displayed simultaneously in different portions of a display screen, or one image to be displayed alongside a reflected image in a portion of the display screen acting as a mirror.
After reading this description it will become apparent to one skilled in the art how to implement the invention in various alternative embodiments and alternative applications. However, although various embodiments of the present invention will be described herein, it is understood that these embodiments are presented by way of example only, and not limitation. As such, this detailed description of various alternative embodiments should not be construed to limit the scope or breadth of the present invention as set forth in the appended claims.
The digital camera module 22 may be built into a portion of the display module 12, as indicated in
The mirrored display panel 20 is a combination mirror and image display panel which may operate in a mirror mode, a display mode, or a combination of mirror and display modes in different regions of the panel.
An enhanced image is then created (32). In one embodiment of the invention, the enhanced image is created by applying enhancements directly to the subject of the original image, and then capturing another image of the subject. This may be done, for example, in a store or business by applying make-up, jewelry, eyeglasses, changing hairstyles or wigs, different clothing, or the like. One or more enhanced images of the subject are then captured, for example to show different hairstyles, clothing, eyeglasses, or the like.
In another embodiment of the invention, the enhanced image is created by computer software in computer module 14 which applies virtual enhancements to the original image as directed by an operator via the control module 15. The virtual enhancements may be of any desired nature, such as make-up, hair color and style, eyeglasses, clothing, jewelry, body size adjustments, and the like. The enhancements may be stored in local data storage of the computer module 14. Alternatively, such enhancements may be obtained from a remote server 55 over the Internet 54, as indicated in
Another method of enhancing an image is to scan in an image having a desired enhancement, such as a hairdo, and then move the hairdo onto the original image to create the enhanced image.
The enhanced image is then stored (34). The stored images may be real or virtually enhanced images, or both types of enhanced image may be created and stored. The screen may then be switched to a display mode in order to display selected images (35).
When no more images are to be captured, created, or displayed (36), the system may be switched back to a full mirror mode (38). The camera may be activated to capture images while the panel is in a mirror mode.
As noted above, the mirrored display panel 20 is a combination mirror and image display panel or screen. As illustrated in
In the alternative of
The display layer in any of the above embodiments may be configured to display only one image or more than one image simultaneously. The display panel may be of any desired shape such as square, rectangular, oval, round or the like, and the display layer may be of similar shape or of different shape. In the embodiment of
There are many possible applications for the image capture and display system of
Instead of actually applying make-up to a customer, computer software techniques may be used to apply make-up to the original or “before” image in order to create an enhanced image which can then be viewed by the customer. Many different make-up techniques may be applied quickly and easily in this manner, without having to physically apply the make-up to the customer. The customer can easily compare different make-up effects to decide which one is best, simply by comparing different enhanced images simultaneously on the display panel 20. If desired, a follow-up physical application of “make-up” could be applied for a finished look without going through numerous trials of application of a variety of “make-ups”. This will save a lot of time and money. Again, the images can be captured and archived by the store for future reference as well as providing a hard copy and/or CD to the customer.
Various make-up procedures for virtual application to a stored image may be stored in the local data storage module 18 for use by an operator of the system. Additionally, make-up enhancements may be available over the Internet 54 from a remote server 55, so that an operator can download them to computer 14 for selectively enhancing an image.
The same procedure may also be used by a person in their home without needing to visit a store or beauty shop, using a similar system or a home computer with a monitor having a built-in digital camera. The user can first capture an image of his or her face without make-up, and can then apply virtual make-up to the image on the home system/computer prior to physically applying the selected version of the make-up which was tested. Computer generated “make-up” items, as well as descriptive techniques of application could be obtained at a store or over the Internet. Advantageously, new make-up items may become available over the Internet to download to a home system/computer. This allows an individual the ability to have instant access to the latest make-up items on the market, without leaving their home. The individual may then choose to purchase selected items over the Internet after viewing the computer-generated make up application on their digital image. The same basic method can be used to shop for other consumer items such as jewelry, clothing, eyewear, hairstyles, surgical enhancements, cosmetic dentistry, and the like.
One method of shopping over the Internet using the image display system of
Although the method and system is described above in connection with selecting make-up, exactly the same method and system may be used to test and select jewelry, either in a store or at home over the Internet. The hardware in this case can be identical to
Alternatively, rather than having the customer actually try on jewelry to create enhanced images, the method of
The system and method described above may also be used in shopping for clothing. The display panel 20 in this case may be full body length to provide a complete view of the selected clothing. An original digital image of the customer can be captured, followed by images of the same customer after trying on different clothing. The system can then be used to allow the customer to compare their image wearing different clothing side-by-side, as indicated in
Instead of the customer actual trying on different clothing, the same basic software may be used for applying the “digital” clothing to the digital customer image. The same technique may be used by the store in showing a customer a virtually created enhanced image of their appearance when wearing different clothing, as well as by the customer in their own home by retrieving selected clothing enhancements or overlays over the Internet. Again, this gives the customer the ability to apply (overlay) the latest clothing images from the Internet onto the customer's image in the privacy of their own home. These items can then be purchased via the Internet.
The same technology may also be used to assist in buying any item which is worn, such as eyeglasses, for example. In the simplest option, a digital image of a customer will be taken in a store and subsequent images will be taken while the customer tries various eyeglass styles. These images can be archived and/or given hard copy and/or CD to the customer. The customer can view different eyeglass styles side-by-side for better comparison than when they simply view different eyeglasses successively in a mirror. They can also take the images home for showing to others before deciding which style suits them best. As in the previous examples, computer software may alternatively be used to apply the “digital” eyewear to the digital customer image. This option may be provided in a store or the customer may be able to compare eyeglasses while shopping over the Internet, as in
Another optional use of the image capture system and method of
Although the system is described above in connection with shopping for various accessories or services, it may also be designed to replace a conventional home vanity or bedroom mirror. In this case, the panel 20 may be a full length mirror to display full dress appearance. The user can take an initial image for reference purposes using the digital camera 22. For every special event they can take and store an image of their appearance for that event, showing what outfit was worn as well as other enhancements such as hair, nail, jewelry, and make-up. This will allow the user to archive their appearance for each function or party, for later reference when attending another function. The home vanity system will allow a user to ensure that the same outfit is not worn to similar events, or to compare a new outfit to previous outfits. The system in this option may also have the capability to apply various virtual enhancements to the user's image, retrieving enhancements either from local storage or over the Internet as described above for the previous embodiments. The home vanity system may also allow the user to purchase items over the Internet if desired.
Other possible applications of the same technology are in the fitness, dental, health, and cosmetic surgery fields. In each of these cases, actual before and after digital images may be taken, or an original image may be digitally enhanced to illustrate to the customer or individual the potential effect of various dental options, weight loss programs, fitness programs, cosmetic surgery options, and the like, using the same methods as described above in connection with
Although the system is described above in connection with enhancement of human images, it will be understood that the same system may also be used in other areas such as pet grooming, room decor, home re-modeling, landscaping, and the like. In the pet grooming field, an initial image of a pet will be obtained and stored, followed by images of the pet with different grooming styles, which may be taken after actual grooming or using computer applied virtual grooming enhancements to the pet's original image.
A hand-held or portable version of the image capture and display system of
The image capture and display system described above can also allow landscape images to be archived and digitally enhanced to overlay new landscape effects over a current landscape for viewing by a user. The various digital plant, tree, grass, fence, brick, water, or other landscaping features may be obtained by a user over the Internet or on a CD provided by various vendors, allowing them to view the potential end result of any landscaping decision in advance.
The system may also be used to create enhanced art. An image may be made of an original drawing or painting, and the image may then be enhanced digitally to create a more realistic or aesthetically pleasing end result.
In this embodiment, the television with the mirrored flat panel monitor will add decoration as a mirror when it is not in use. The built-in camera module 74 will provide a means for capturing images of family events and the like, and may be used to take family group photos, individual photos, and the like. The camera may be a still camera, a video camera, or may be capable of recording both still and video images. Various stored images can be compared with later images or actual reflections of the same people. An attached video phone module 85, if used, can allow video phone calls to be made using the system. The same technology as described above may also be used to allow individuals to compare different images, such as a current and previous image, a current and digitally enhanced image, as well as images with different clothing, jewelry, make-up, hair styles, and the like. The camera module 74 may also be used to take images of the room and the resultant images may be enhanced by the user to add different room decor, paint or wall coverings, furniture items and the like in order to help them select new room decor. These effects can be accessed by the user over the Internet or on a CD or other storage medium obtained by a vendor of the various items of interest. This would allow the home user the opportunity to provide different digital images for helping them in remodeling.
Although a home system is described above in connection with
By adding a motion sensor 84 to the system as indicated in
Once the user has completed capturing, storing, and/or enhancing images, they may choose to switch to a different mode (99), or exit the system (100). If the user chooses to compare stored images (step 102) rather than create new images, they can select images for viewing from storage (step 104), and can display one or more images on the display screen (step 105). They can choose to select and compare more images (106), or to switch to taking photographs (107). When they are done, they can choose to switch mode 99 or exit the system (100 or 108).
The user may also select conventional TV mode 110 in step 91, in which case the display layer behind the mirror of the TV monitor will be operative and light will be transmitted through the mirror layer in normal TV operation (112). When the user has finished watching television, they may switch to a different mode (114), or exit the system (115).
If security/monitoring mode 116 is selected, the motion sensor module 84 will be activated to monitor the area within its field of view for movement, as illustrated in
In another embodiment, the system may be used in a store or the like and the security/monitoring mode may be used when customers are browsing at a store counter or the like. The motion sensor module 84 in this case activates the camera module to capture images of a potential customer, which can then be suitably enhanced and displayed to the same customer for advertising purposes.
The system with a built-in motion sensor and associated security mode can also be used for commercial and military high secure areas. The mirrored display with a built in digital camera and computer could quickly archive visitors. In addition, a quick comparison of threat images could be compared to the current image. Captured security images could be transmitted via a wired or wireless network to a security monitor at a remote location for further action.
Although the system is described above in connection with relatively large scale image capture and display systems for use in the home, businesses, stores and the like, the same basic system may be scaled down for use in smaller, portable devices such as cameras, cell phones, palm pilots, portable digital assistants, game displays, and the like.
One field unit 130 is shown in an enlarged, flipped open view in
The security application of
In the above embodiments where enhanced images are created, one or more images may be taken of each subject prior to enhancement. Multiple images from different angles may be taken if desired, and software may be provided for creating images which can be rotated to check appearance from all directions, using software fill-in techniques, for example.
In the above embodiment, the built-in computer of the image capture and display system may communicate with a remote system over a network in any known manner, in order to download image enhancement features or software for applying such features, or to transmit images to a remote location. Some options for such communications are described below.
In the illustrated embodiment, wireless communication device 650 comprises an antenna 652, a multiplexor 654, a low noise amplifier (“LNA”) 656, a power amplifier (“PA”) 658, a modulation circuit 660, a baseband processor 662, and a speaker 664 and microphone 666. A central processing unit (“CPU”) 668 with a data storage area 670 is connected to the baseband processor, and a hardware interface 672 is connected to the CPU. In the wireless communication device 650, radio frequency (“RF”) signals are transmitted and received over the air by the antenna 652. In one embodiment, the antenna 652 may comprise one or more antennae. Multiplexor 654 performs a switching function to provide the antenna system with transmit and receive signal paths. In the receive path, received RF signals can be coupled from the multiplexor to the LNA 656 that amplifies the received RF signal and sends the amplified signal to the demodulator part of the modulation circuit 660.
Typical modulation circuit 660 will combine a demodulator and modulator in one integrated circuit (“IC”). The demodulator and modulator can also be separate components. In the incoming path, the demodulator strips away the RF carrier signal leaving a baseband receive audio signal, which is sent from the demodulator to the baseband processor 652.
If the received signal contains audio information, then baseband processor 652 decodes the signal and converts it to an analog signal. Then the signal is amplified and sent to the speaker 664. The baseband processor also receives analog audio signals from the microphone 666. These analog audio signals are converted to digital signals and encoded by the baseband processor 662. The baseband processor 662 also codes the digital signals for transmission and generates a baseband transmit audio signal that is routed to the modulator portion of the modulation circuit 660. The modulator mixes the baseband transmit audio signal with an RF carrier signal generating an RF transmit signal that is routed to the antenna system through power amplifier 658. The power amplifier amplifies the RF transmit signal and routes it to the multiplexor 654 where the signal is switched to the antenna port for transmission.
The baseband processor 662 is also communicatively coupled with the central processing unit 668. The central processing unit 668 has access to data storage area 670. The central processing unit 668 is preferably configured to execute instructions (i.e., computer programs or software) that can be stored in the data storage area 670. Computer programs can also be received from the baseband processor 662 and stored in the data storage area 670 or executed upon receipt. Such computer programs, when executed, enable the wireless communication device 650 to perform the various functions of the present invention as previously described. For example, data storage area 670 may include various software modules (not shown) such as image enhancement software that was previously described with respect to
In this description, the term “computer readable medium” is used to refer to any media used to provide executable instructions (e.g., software and computer programs) to the wireless communication device 650 for execution by the central processing unit 668. Examples of these media include the data storage area 670, microphone 666 (via the baseband processor 662), antenna 652 (also via the baseband processor 662), and hardware interface 672. These computer readable media are means for providing executable code, programming instructions, and software to the wireless communication device 650. The executable code, programming instructions, and software, when executed by the central processing unit 668, preferably cause the central processing unit 668 to perform the inventive features and functions previously described herein.
The central processing unit 668 is also preferably configured to receive notifications from the hardware interface 672 when new devices are detected by the hardware interface. Hardware interface 672 can be a combination electromechanical detector with controlling software that communicates with the CPU 668 and interacts with new devices. The hardware interface 672 may be a firewire port, a USB port, a Bluetooth or infrared wireless unit, or any of a variety of wired or wireless access mechanisms. Examples of hardware that may be linked with the device 650 include data storage devices, computing devices, headphones, microphones, and the like.
The computer system 750 preferably includes one or more processors, such as processor 752. Additional processors may be provided, such as an auxiliary processor to manage input/output, an auxiliary processor to perform floating point mathematical operations, a special-purpose microprocessor having an architecture suitable for fast execution of signal processing algorithms (e.g., digital signal processor), a slave processor subordinate to the main processing system (e.g., back-end processor), an additional microprocessor or controller for dual or multiple processor systems, or a coprocessor. Such auxiliary processors may be discrete processors or may be integrated with the processor 752.
The processor 752 is preferably connected to a communication bus 754. The communication bus 754 may include a data channel for facilitating information transfer between storage and other peripheral components of the computer system 750. The communication bus 754 further may provide a set of signals used for communication with the processor 752, including a data bus, address bus, and control bus (not shown). The communication bus 754 may comprise any standard or non-standard bus architecture such as, for example, bus architectures compliant with industry standard architecture (“ISA”), extended industry standard architecture (“EISA”), Micro Channel Architecture (“MCA”), peripheral component interconnect (“PCI”) local bus, or standards promulgated by the Institute of Electrical and Electronics Engineers (“IEEE”) including IEEE 488 general-purpose interface bus (“GPIB”), IEEE 696/S-100, and the like.
Computer system 750 preferably includes a main memory 756 and may also include a secondary memory 758. The main memory 756 provides storage of instructions and data for programs executing on the processor 752. The main memory 756 is typically semiconductor-based memory such as dynamic random access memory (“DRAM”) and/or static random access memory (“SRAM”). Other semiconductor-based memory types include, for example, synchronous dynamic random access memory (“SDRAM”), Rambus dynamic random access memory (“RDRAM”), ferroelectric random access memory (“FRAM”), and the like, including read only memory (“ROM”).
The secondary memory 758 may optionally include a hard disk drive 760 and/or a removable storage drive 762, for example a floppy disk drive, a magnetic tape drive, a compact disc (“CD”) drive, a digital versatile disc (“DVD”) drive, etc. The removable storage drive 762 reads from and/or writes to a removable storage medium 764 in a well-known manner. Removable storage medium 764 may be, for example, a floppy disk, magnetic tape, CD, DVD, etc.
The removable storage medium 764 is preferably a computer readable medium having stored thereon computer executable code (i.e., software) and/or data. The computer software or data stored on the removable storage medium 764 is read into the computer system 750 as electrical communication signals 778.
In alternative embodiments, secondary memory 758 may include other similar means for allowing computer programs or other data or instructions to be loaded into the computer system 750. Such means may include, for example, an external storage medium 772 and an interface 770. Examples of external storage medium 772 may include an external hard disk drive or an external optical drive, or and external magneto-optical drive.
Other examples of secondary memory 758 may include semiconductor-based memory such as programmable read-only memory (“PROM”), erasable programmable read-only memory (“EPROM”), electrically erasable read-only memory (“EEPROM”), or flash memory (block oriented memory similar to EEPROM). Also included are any other removable storage units 772 and interfaces 770, which allow software and data to be transferred from the removable storage unit 772 to the computer system 750.
Computer system 750 may also include a communication interface 774. The communication interface 774 allows software and data to be transferred between computer system 750 and external devices (e.g. printers), networks, or information sources. In the above embodiments, the built-in computer of the image capture and display system may communicate with a server over a network in any known manner, in order to download image enhancement features or software for applying such features, or to transmit security images to a central station. For example, computer software or executable code may be transferred to computer system 750 from a network server via communication interface 774. Examples of communication interface 774 include a modem, a network interface card (“NIC”), a communications port, a Personal Computer Memory Card International Association (“PCMCIA”) slot and card, an infrared interface, and an IEEE 1394 fire-wire, just to name a few.
Communication interface 774 preferably implements industry promulgated protocol standards, such as Ethernet IEEE 802 standards, Fiber Channel, digital subscriber line (“DSL”), asynchronous digital subscriber line (“ADSL”), frame relay, asynchronous transfer mode (“ATM”), integrated digital services network (“ISDN”), personal communications services (“PCS”), transmission control protocol/Internet protocol (“TCP/IP”), serial line Internet protocol/point to point protocol (“SLIP/PPP”), and so on, but may also implement customized or non-standard interface protocols as well.
Software and data transferred via communication interface 774 are generally in the form of electrical communication signals 778. These signals 778 are preferably provided to communication interface 774 via a communication channel 776. Communication channel 776 carries signals 778 and can be implemented using a variety of wired or wireless communication means including wire or cable, fiber optics, conventional phone line, cellular phone link, wireless data communication link, radio frequency (RF) link, or infrared link, just to name a few.
Computer executable code (i.e., computer programs or software) is stored in the main memory 756 and/or the secondary memory 758. Computer programs can also be received via communication interface 774 and stored in the main memory 756 and/or the secondary memory 758. Such computer programs, when executed, enable the computer system 750 to perform the various functions of the present invention as previously described.
In this description, the term “computer readable medium” is used to refer to any media used to provide computer executable code (e.g., software and computer programs) to the computer system 750. Examples of these media include main memory 756, secondary memory 758 (including hard disk drive 760, removable storage medium 764, and external storage medium 772), and any peripheral device communicatively coupled with communication interface 774 (including a network information server or other network device). These computer readable mediums are means for providing executable code, programming instructions, and software to the computer system 750.
In an embodiment that is implemented using software, the software may be stored on a computer readable medium and loaded into computer system 750 by way of removable storage drive 762, interface 770, or communication interface 774. In such an embodiment, the software is loaded into the computer system 750 in the form of electrical communication signals 778. The software, when executed by the processor 752, preferably causes the processor 752 to perform the inventive features and functions previously described herein.
Those of skill in the art will appreciate that the various illustrative modules and method steps described in connection with the above described figures and the embodiments disclosed herein can often be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled persons can implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the invention. In addition, the grouping of functions within a module or step is for ease of description. Specific functions or steps can be moved from one module or step to another without departing from the invention.
Moreover, the various illustrative modules and methods described in connection with the embodiments disclosed herein can be implemented or performed with a general purpose processor, a digital signal processor (“DSP”), an ASIC, FPGA or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general-purpose processor can be a microprocessor, but in the alternative, the processor can be any processor, controller, microcontroller, or state machine. A processor can also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.
Additionally, the steps of a method or algorithm described in connection with the embodiments disclosed herein can be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module can reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium including a network storage medium. An exemplary storage medium can be coupled to the processor such the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium can be integral to the processor. The processor and the storage medium can also reside in an ASIC.
The above description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles described herein can be applied to other embodiments without departing from the spirit or scope of the invention. Thus, it is to be understood that the description and drawings presented herein represent a presently preferred embodiment of the invention and are therefore representative of the subject matter which is broadly contemplated by the present invention. It is further understood that the scope of the present invention fully encompasses other embodiments that may become obvious to those skilled in the art and that the scope of the present invention is accordingly limited by nothing other than the appended claims.