The present invention relates to a communication apparatus for pictorially communicating with a plurality of users of communication terminals over a network and, in particular, to a communication apparatus with the emotional expression indicating capability for properly indicating a method for establishing communication to initiate communication and emotional information about a sender during communication and the capability of modifying images.
The growth of Internet technologies has propelled the widespread use of electronic chats and a variety of communication tools such as internet-enabled phones and mobile phones.
In order for a person to initiate communication with another person on the Internet, a communication tool of that person must be active and connected to the Internet. Existing facilities for indicating whether or not a communication tool of the person of interest is active include those described in ICQ (http://web.icq.com/). Various communication tools are available such as text-based communication tools, voice conversation tools, and video conversation tools. Some communication tools identify an active communication tool of a person of interest and notify the availability of the tool to the initiator of conversation.
These technologies enables a user to know whether or not a person with whom he or she wants to converse is connected to the Internet, or what kind of media, such as text, voice, and video, can be used to communicate with the person of interest, by specifying the e-mail address of the person of interest and initiate a conversation by using an appropriate medium.
However, such a conversation candidate that a user wants to communicate with is often offline and therefore it is difficult to call up a number of members of interest to immediately hold a conversation or conference at any time. There is a problem that a user must specify the start time of a conversation or a conference and notify it to the members, and all the members establish connection to the Internet by that time before the conversation or conference starts. There is another problem that what a sender wants to convey to the other parties is not always accurately understood by them because the display size of transmittable images are small and they are compressed at high compression ratios due to low data transfer rates of communication links on the Internet.
Moreover, in these days, mail messages with image data are being exchanged commonly to perform communication between users by mobile phones. In some cases, however, an image cannot properly be displayed on a receiving mobile phone due to difference between the performance capabilities of a sending mobile phone and those of the receiving mobile phones. To avoid this, before a sender sends image data from his or her mobile phone to other mobile phones, the sender stores the image data supported by the capabilities of each of the receiving mobile phones beforehand in a server on the Internet, the receivers provide information about their mobile phone models to the server by e-mail through their mobile phones, and then the server selects image data supported by each of the models to send it to each receiving mobile phones.
However, these conventional communication apparatuses require that a sender selecting image data registered with a server to send should provide image data that has the same content as that of the registered image data but image properties different from those of the registered data in order to allow the image data to be displayed properly on receiving mobile phones. As the amount of image data registered with the server or the number of supported models of receiving mobile phone is increased, more storage areas in the server is required for storing the increased amount of image data. Moreover, if unregistered image data, such as image data captured by using a camera function included in a sender's mobile phone, is sent or a composite image data of registered image data and unregistered image data is sent to mobile phones, the unregistered image data is possibly not supported by some of the receiving mobile phones and the image may not properly be displayed on them.
The present invention solves these problems of the prior art, and an object of the present invention is to provide a communication apparatus that allows a user who wants to perform communication over the Internet to request a conversation candidate/candidates to converse to connect a server on the network.
Another object of the present invention is to provide a communication apparatus having the capability of indicating feelings of a user during communication on the Internet.
Yet another object of the present invention is to provide a communication apparatus that allows any image contained in data exchanged between communication terminals such as mobile phones to be properly displayed in accordance with the capabilities of the receiving communication terminals.
In order to achieve these objects, a communication apparatus according to a first aspect of the present invention is an apparatus for performing conversation over the Internet, comprising: a plurality of first communication terminals each having a module for shooting an image of a speaker and a module for displaying the images of all participants in a conversation, the first communication terminal enabling conversation over the Internet while displaying the image of the speaker; second communication terminals provided correspondingly to the first communication terminals; a server for performing internet communication; a first communication link for connecting the server with the first communication terminals; and a second communication link for connecting the server with the second communication terminals. When a conversation over the Internet is initiated from one of the first communication terminals, a request for participating in the conversation is sent to the second communication terminals owned by intended one or more conversation partners.
According to a second aspect of the invention, there is provided the communication apparatus according to claim 1, wherein if any of the requested conversation partners cannot participant in the conversation, information indicating the existence of an unavailable conversation partner and any information provided by the unavailable conversation partner are sent from the second communication terminal to the server and the server sends the information indicating the existence of the unavailable conversation partner and the information provided by the unavailable conversation partner to the first communication terminals of the other intended conversation partners.
According to a third aspect of the invention, there is provided the communication apparatus according to the first or second aspect, wherein the second communication terminal is an e-mail-enabled mobile phone.
According to a fourth aspect of the invention, there is provided the communication apparatus according to the first or second aspect, wherein the second communication terminal is a pager.
According to a fifth aspect of the invention, there is provided the communication apparatus according to the first or second aspect, wherein an e-mail-enabled mobile phone is used as both of the first and second communication terminals.
According to a sixth aspect of the invention, there is provided the communication apparatus according to the fifth aspect, wherein one communication link is used as both of the first and second communication links to connect to the server.
According to a seventh aspect of the invention, there is provided the communication apparatus according to any of the first to sixth aspects, wherein a real name or a handle is used as information for identifying each of the intended conversation partners.
According to an eighth aspect of the invention, there is provided the communication apparatus according to any of the first to sixth aspect, wherein an e-mail address is used as information for identifying each of the intended conversation partners.
According to a ninth aspect of the invention, there is provided a communication apparatus for performing conversation over the Internet, comprising: a plurality of communication terminals each having a module for capturing an image of a person performing communication, a module for displaying images of all participants in the communication, a switch for adding image information, and a module for selecting emotion-indicating image information indicating emotion and performing communication over the Internet; a server for performing internet communication; a communication link for connecting the server with the communication terminals; and a storage for storing the emotion-indicating image information. The emotion-indicating image information selected by the module for selecting emotion-indicating image information is combined with the image captured by the image capturing module in response to an instruction provided through the image information adding switch, and the resulting composite image is displayed on the image displaying module.
According to a tenth aspect of the invention, there is provided the communication apparatus according to the ninth aspect, wherein the emotion-indicating image information selected by the module for selecting emotion-indicating image information and the image captured by the image capturing module are combined at a communication terminal having selected the image information and the resulting composite image is sent to a communication terminal of an intended recipient of the communication through the server and displayed on the image displaying module of the communication terminal of the intended recipient of the communication.
According to an eleventh aspect of the invention, there is provided the communication apparatus according to the ninth aspect, wherein the emotion-indicating image information selected by the module for selecting emotion-indicating image information and the image captured by the image capturing module are combined at the server, sent to the communication terminal of the intended recipient of the communication, and displayed on the image displaying module of the communication terminal of the intended recipient of the communication.
According to a twelfth aspect of the invention, there is provided the communication apparatus according to the ninth aspect, wherein the emotion-indicating image information selected by the module for selecting emotion-indicating image information and the image captured by the image capturing module are sent to the communication terminal of the intended recipient of the communication, combined at the communication terminal of the intended recipient of the communication, and displayed on the image displaying module.
According to a thirteenth aspect of the invention, there is provided the communication apparatus according to the ninth aspect, wherein the position and shape of the emotion-indicating image information selected by the module for selecting emotion-indicating image information are changed according to a change in the image captured by the image capturing module.
According to a fourteenth aspect of the invention, there is provided the communication apparatus according the thirteenth aspect, wherein the change in the image captured by the image capturing module is the change of the position of the face of a person in the image.
According to a fifteenth aspect of the invention, there is provided the communication apparatus according to the ninth aspect, wherein the instruction provided through the information adding switch is mechanically and directly provided by a sender.
According to a sixteenth aspect of the invention, there is provided the communication apparatus according to the ninth aspect, wherein the image information adding switch has a storage containing an audio keyword and the instruction for adding image information is determined on the basis of whether or not a voice in the communication matches the audio keywords.
According to a seventeenth aspect of the invention, there is provided the communication apparatus according to the ninth aspect, wherein the information adding switch has a storage containing a facial expression image and the instruction for adding image information is determined on the basis of whether or not an image matches the facial expression image.
According to an eighteenth aspect of the invention, there is provided the communication apparatus according to the ninth aspect, wherein any of the emotion-indicating image information stored in the server is obtained and combined with the image captured.
According to a nineteenth aspect of the invention, there is provided the communication apparatus according to the ninth aspect, wherein emotion-indicating image information created in a device connected to the server is obtained and combined with the image captured.
According to a twentieth aspect of the invention, there is provided the communication apparatus according to the ninth aspect, wherein the emotion-indicating image information is graphics.
According to a twenty-first aspect of the invention, there is provided the communication apparatus according to the ninth aspect, wherein the emotion-indicating image information is a natural image.
According to a twenty-second aspect of the invention, there is provided the communication apparatus according to the ninth aspect, wherein the emotion-indicating image information is a character or symbol.
According to a twenty-three aspect of the invention, there is provided the communication apparatus according to any of the first, second and ninth aspects, wherein, when the communication is performed with voice conversation, the volume of the voice in the conversation is used to identify a speaker by the server and a zoomed image of the speaker is displayed on the image displaying module.
According to a twenty-fourth aspect of the invention, there is provided the communication apparatus according to any of the first, second and ninth aspects, wherein, when the communication is performed with voice conversation, an audio waveform of voice of each speaker in the conversation is rectified and rectified direct-current components are integrated for a certain unit of time to give an integral, in the server, and if the integral associated with any participant in the conversation exceeds a predetermined threshold, the image of the participant is zoomed, and if there is no participant whose associated integral exceeds the threshold, evenly sized images of the participants in the conversation are displayed on the image displaying module.
According to a twenty-fifth aspect of the invention, there is provided the communication apparatus according to any of the first, second and ninth aspects, wherein, when the communication is performed with voice conversation, the audio waveform of voice of each speaker in the conversation is rectified and rectified direct-current components are integrated for a certain unit of time, and an average of integrals calculated for a plurality of units of time is given, in the server, and if the average integral associated with any participant in the conversation exceeds a predetermined threshold, the image of the participant is zoomed, and if there is no participant whose associated integral exceeds the threshold, evenly sized images of the participants in the conversation are displayed on the image displaying module.
According to a twenty-sixth aspect of the invention, there is provided the communication apparatus according to the first or second aspect, wherein a vibration device is provided in the second communication terminals and a request for participating in communication is made by activating the vibration device in the second communication terminals of intended recipients of the communication.
According to a twenty-seventh aspect of the invention, there is provided the communication apparatus according to the ninth aspect, wherein a vibration device is provided in the communication terminals and a request for participating in communication is made by activating the vibration device in the communication terminals of intended recipients of the communication.
According to a twenty-eighth aspect of the invention, there is provided the communication apparatus according to the ninth aspect, wherein a vibration device for indicating emotion by vibration and a switch for activating the vibration of the vibration device of intended recipient of communication are provided in the communication terminals and the vibration device is activated by an instruction provided through the switch to indicate an emotion to the intended recipient of the communication.
According to a twenty-ninth aspect of the invention, there is provided a communication apparatus comprising a server connected to a network and a plurality of communication terminals capable of communicating with one another through the server, the communication apparatus being capable of combining a plurality of images and sending the resulting composite image from any of the communication terminal to any other communication terminals, the server comprising: an image storage for receiving and storing an image sent from a sending communication terminal; an additional information storage for storing a plurality of pieces of additional information to be combined with the sent image; an additional information selector for selecting a piece of additional information to be combined with the sent image according to an instruction from the sending communication terminal; a terminal information requesting module for requesting an intended receiving communication terminal to send terminal information about the receiving communication terminal; a terminal information acquisition module for receiving the terminal information sent form the receiving communication terminal; an additional information combining module for converting the image properties of the sent image and the additional information according to the terminal information and combining converted the image with the converted additional information; and a composite image transmitter for sending the composite image to the receiving communication terminal. The image properties of the sent image and the additional information are converted so as to be optimally displayed on the intended receiving communication terminal, then the sent image and the additional information are combined and the resulting composite image is sent and received.
According to a thirtieth aspect of the invention, there is provided a communication apparatus comprising a server connected to a network and a plurality of communication terminals each being capable of communicating with one another through the server, the communication apparatus being capable of combining a plurality of images and sending the resulting composite image from any of the communication terminal to any other communication terminals, the server comprising: an image storage for receiving and storing an image sent from a sending communication terminal; an additional information storage for storing a plurality of pieces of additional information to be combined with the sent image and different versions of each piece of additional information, the versions having different image properties; an additional information selector for selecting a piece of additional information to be combined with the sent image according to an instruction from the sending communication terminal; a terminal information requesting module for requesting a receiving communication terminal to send terminal information about the receiving communication terminal; a terminal information acquisition module for receiving the terminal information sent form the receiving communication terminal; an additional information selector for selecting the most suitable version of additional information for the receiving communication terminal according to the terminal information from among versions of the additional information selected in response to an instruction provided from the sending communication terminal; an additional information combining module for converting the image properties of the sent image according to the terminal information and combining the converted image with the selected additional information; and a composite image transmitter for sending the composite image to the receiving communication terminal. The image properties of the sent image is converted so as to be optimally displayed on the intended receiving communication terminal, the additional information is selected and combined with the sent image, and the resulting composite image is sent and received.
According to a thirty-first aspect of the invention, there is provided a communication apparatus comprising a server connected to a network and a plurality of communication terminals each being capable of communicating with one another through the server, the communication apparatus being capable of combining a plurality of images and sending the resulting composite image from any of the communication terminal to any other communication terminals, the server comprising: an image storage for receiving and storing an image sent from a sending communication terminal; an additional information storage for storing a plurality of pieces of additional information to be combined with the sent image; an additional information selector for selecting a piece of additional information to be combined with the sent image according to an instruction from the sending communication terminal; a terminal information requesting module for requesting an intended receiving communication terminal to send terminal information about the receiving communication terminal; a terminal information acquisition module for receiving the terminal information sent from the receiving communication terminal; an image feature detector for detecting a feature of the sent image; an additional information combining module for converting the image properties of the sent image and the additional information according to the terminal information and then superimposing the additional information on a position in the sent image, the position being determined according to the feature of the sent image; and a composite image transmitter for sending the composite image to the receiving communication terminal. The image properties of the sent image and the additional information are converted so as to be optimally displayed on the intended receiving communication terminal, then the sent image and the additional information are combined and the resulting composite image is sent and received.
According to a thirty-second aspect of the invention, there is provided a communication apparatus comprising a server connected to a network and a plurality of communication terminals each being capable of communicating with one another through the server, the communication apparatus being capable of combining a plurality of images and sending the resulting composite image from any of the communication terminal to any other communication terminals, the server comprising: an image storage for receiving and storing an image sent from a sending communication terminal; an additional information storage for storing a plurality of pieces of additional information to be combined with the sent image and different versions of each piece of additional information, the versions having different image properties; an additional information selector for selecting a piece of additional information to be combined with the sent image according to an instruction from the sending communication terminal; a terminal information requesting module for requesting an intended receiving communication terminal to send terminal information about the receiving communication terminal; a terminal information acquisition module for receiving the terminal information sent form the receiving communication terminal; an additional information selector for selecting the most suitable version of additional information for the receiving communication terminal according to the terminal information from among versions of the additional information selected in response to an instruction provided from the sending communication terminal; an image feature detector for detecting a feature of the sent image; an additional information combining module for converting the image properties of the sent image according to the terminal information and then superimposing the selected additional information on a position in the sent image, the position being determined according to the feature of the sent image; and a composite image transmitter for sending the composite image to the receiving communication terminal. The image properties of the sent image is converted so as to be optimally displayed on the intended receiving communication terminal, the additional information is selected and combined with the sent image, and the resulting composite image is sent and received.
According to a thirty-third aspect of the invention, there is provided the communication apparatus according to the thirty-first or thirty-second aspect, wherein the image feature detector detects the size, position and/or motion of a person in the sent image.
According to a thirty-fourth aspect of the invention, there is provided the communication apparatus according to the thirty-first or thirty-second aspect, wherein the image feature detector detects the size and position of the face or other parts of the person in the sent image.
According to a thirty-fifth aspect of the invention, there is provided the communication apparatus according to the thirty-third or thirty-fourth aspect, wherein the image feature detector detects the skin color of the person in the sent image so that the skin color of the person is not impaired by the conversion of the image properties.
According to a thirty-sixth aspect of the invention, there is provided a communication apparatus comprising a server connected to a network and a plurality of communication terminals each being capable of communicating with one another through the server, the communication apparatus being capable of adding a special effect to an image and sending the resulting image from any of the communication terminal to any other communication terminals, the server comprising: an image storage for receiving and storing an image sent from a sending communication terminal; a special effect storage unit for storing a program for adding a special effect to the sent image; a special effect selector for selecting a special effect to be added to the sent image according to an instruction provided from the sending communication terminal; a terminal information requesting module for requesting an intended receiving communication terminal to send terminal information about the receiving communication terminal; a terminal information acquisition module for receiving the terminal information sent form the receiving communication terminal; a special effect adding module for adding the selected special effect after converting the image properties of the sent image according to the terminal information; and a composite image transmitter for sending the composite image to which the special effect is added to the receiving communication terminal. The image properties of the sent image is converted so as to be optimally displayed on the intended receiving communication terminal and then the image to which the special effect is added is sent and received.
According to a thirty-seventh aspect of the invention, there is provided the communication apparatus according to the thirty-sixth aspect, wherein the server comprises an image feature detector for detecting a feature of the sent image and the special effect is added to the detected feature.
According to a thirty-eighth aspect of the invention, there is provided the communication apparatus according to the thirty-seventh aspect, wherein the image feature detector detects the size, position and/or motion of a person in the sent image.
According to a thirty-ninth aspect of the invention, there is provided the communication apparatus according to the thirty-seventh aspect, wherein the image feature detector detects the size and position of the face or other parts of the person in the sent image.
According to a fortieth aspect of the invention, there is provided a communication apparatus comprising a server connected to a network and a plurality of communication terminals each being capable of communicating with one another through the server, the communication apparatus being capable of adding a special effect to an image and sending the resulting composite image from any of the communication terminal to any other communication terminals, the server comprising: an image storage for receiving and storing an image sent from a sending communication terminal; an additional information storage for storing a plurality of pieces of additional information to be combined with the sent image; an additional information selector for selecting a piece of additional information to be combined with the sent image according to an instruction from the sending communication terminal; a composite image confirming module for combining the sent image with the additional information and sending the resulting composite image to the sending communication terminal for confirmation; an image transmitter for sending the image to an intended receiving communication terminal in response to confirmation provided from the sending communication terminal; and an additional information transmitter for sending the additional information to the receiving communication terminal in response to confirmation provided from the sending communication terminal; the communication terminal comprising: an image viewing module for displaying the composite image received from the composite image confirming module; an image transmitter for selecting an image and sending the image to the server; an additional information specifying module for specifying additional information to be added to the image to be sent while viewing the additional information on the image viewing module; a transmission commanding module for issuing a command for transmitting the image and the additional information to the receiving communication terminal; an image receiver for receiving the sent image from the server; an additional information receiver for receiving the additional information from the server; a terminal information acquisition module for acquiring terminal information about the intended receiving communication terminal; an additional information combining unit for converting the sent image and the additional information according to the terminal information and combining the converted image with the converted additional information; and a display for displaying the composite image generated by the additional information combining module. The image properties of the sent image and the additional information are converted so as to be optimally displayed on the intended receiving communication terminal, then the sent image and the additional information are combined and the resulting composite image is sent and received.
According to a forty-first aspect of the invention, there is provided the communication apparatus according to the fortieth aspect, wherein the communication terminal comprises a program downloading module connected to the network for obtaining over the network a program for converting the image properties of the image and additional information according to the terminal information and combining the converted image with the converted additional information in the additional information combining module.
According to a forty-second aspect of the invention, there is provided a communication apparatus comprising a server connected to a network and a plurality of communication terminals each being capable of communicating with one another through the server, the communication apparatus being capable of combining a plurality of images and sending the resulting composite image from any of the communication terminal to any other communication terminals, the server comprising: an image storage for receiving and storing an image sent from a sending communication terminal; an additional information storage for storing a plurality of pieces of additional information to be combined with the sent image; an additional information selector for selecting a piece of additional information to be combined with the sent image according to an instruction from the sending communication terminal; a composite image confirming module for combining the sent image with the additional information and sending the resulting composite image to the sending communication terminal for confirmation; an image transmitter for sending the sent image to an intended receiving communication terminal in response to confirmation provided from the sending communication terminal; and an additional information transmitter for sending the additional information to the receiving communication terminal in response to confirmation provided from the sending communication terminal; the communication terminal comprising: an image viewing module for displaying the composite image received from the composite image confirming module; an image transmitter for selecting an image and sending the image to the server; an additional information specifying module for specifying additional information to be added to the image to be sent while viewing the additional information on the image viewing module; a transmission commanding module for issuing a command for transmitting the image and the additional information to the receiving communication terminal; an image receiver for receiving the sent image from the server; an additional information receiver for receiving the additional information from the server; a terminal information acquisition module for acquiring terminal information about the intended receiving communication terminal; an image feature detector for detecting a feature of the sent image; an additional information combining module for converting the image properties of the sent image and the additional information according to the terminal information and then superimposing the additional information on a position in the sent image, the position being determined according to the feature of the sent image; and a display for displaying the composite image generated by the additional information combining module. The image properties of the sent image and the additional information are converted so as to be optimally displayed on the intended receiving communication terminal, then the sent image and the additional information are combined and the resulting composite image is sent and received.
According to a forty-third aspect of the invention, there is provided the communication apparatus according to the forty-second aspect, wherein the image feature detector detects the size, position and/or motion of a person in the sent image.
According to a forty-fourth aspect of the invention, there is provided the communication apparatus according to the forty-second aspect, wherein the image feature detector detects the size and position of the face or other parts of the person in the sent image.
According to a forty-fifty aspect of the invention, there is provided the communication apparatus according to the forty-third or forty-fourth aspect, wherein the image feature detector detects the skin color of the person in the sent image so that the skin color of the person is not impaired by the conversion of the image properties.
According to a forty-sixth aspect of the invention, there is provided a communication apparatus comprising a server connected to a network and a plurality of communication terminals each being capable of communicating with one another through the server, the communication apparatus being capable of adding a special effect to an image and sending the resulting composite image from any of the communication terminal to any other communication terminals, the server comprising: an image storage for receiving and storing an image sent from a sending communication terminal; a special effect storage unit for storing a program for adding a special effect to the sent image; a special effect selector for selecting a special effect to be added to the sent image according to an instruction provided from the sending communication terminal; a composite image confirming module for adding the special effect to the image and sending the resulting composite image to the sending communication terminal for confirmation; an image transmitter for sending the sent image to an intended receiving communication terminal in response to confirmation provided from the sending communication terminal; and an additional information transmitter for sending the program for adding a special effect to the sent image to the receiving communication terminal; the communication terminal comprising: an image viewing module for displaying the composite image received from the composite image confirming module; an image transmitter for selecting an image and sending the image to the server; a special effect specifying module for specifying a special effect to be added to the image to be sent while viewing the special effect on the image viewing module; a transmission commanding module for issuing a command for transmitting the image and the program for adding a special effect to the image to the receiving communication terminal; an image receiver for receiving the sent image from the server; an additional information receiver for receiving from the server the program for adding a special effect to the image; a terminal information acquisition module for acquiring terminal information about the intended receiving communication terminal; a special effect adding module for converting the image properties of the sent image according to the terminal information and then adding the selected special effect to the image; and a display for displaying the composite image to which the special effect is added by the special effect adding module. The image properties of the sent image are converted so as to be optimally displayed on the intended receiving communication terminal, then the image to which the special effect is added is sent and received.
According to a forty-seventh aspect of the invention, there is provided the communication apparatus according to any of the forty-second to forty-fifth aspects, wherein the communication terminal comprises a program downloading module connected to the network for obtaining over the network a program for converting the image properties of the image and additional information according to the terminal information and then superimposing the additional information on a position in the image, the position being determined according to the feature of the image.
According to a forty-eighth aspect of the invention, there is provided the communication apparatus according to the forty-sixth aspect, wherein the communication terminal comprises a program downloading module connected to the network for obtaining over the network a program for converting the image properties of the image according to the terminal information and adding the selected special effect to the image in the special effect adding module.
According to a forty-ninth aspect of the invention, there is provided a communication apparatus comprising a server connected to a network and a plurality of communication terminals each being capable of communicating with one another through the server, the communication apparatus being capable of combining a plurality of images and sending the resulting composite image from any of the communication terminal to any other communication terminals, the server comprising: an image storage for receiving and storing an image sent from a sending communication terminal; an additional information storage for storing a plurality of pieces of additional information to be combined with the sent image; an additional information selector for selecting a piece of additional information to be combined with the sent image according to an instruction from the sending communication terminal; a composite image confirming module for combining the sent image with the additional information and sending the resulting composite image to the sending communication terminal for confirmation; and an additional information combining program transmitter for sending the image, the additional information, and a program for converting image properties of the send image and the additional image according to the terminal information and combining the converted image with the converted additional information; the communication terminal comprising: an image viewing module for displaying the composite image received from the composite image confirming module; an image transmitter for selecting an image and sending the image to the server; an additional information specifying module for specifying additional information to be added to the image to be sent while viewing the additional information on the image viewing module; a transmission commanding module for issuing a command for transmitting the image and a program for converting the image and the additional information according to the capabilities of a receiving communication terminal and combining the image with the additional information to the receiving communication terminal; a terminal information acquisition module for acquiring terminal information about the intended receiving communication terminal; an additional information combining program execution module for receiving the program for converting the image and the additional information according to the capabilities of a receiving communication terminal and combining the image with the additional information and executing the program to convert the image and the additional information according to the image properties of the terminal information and combine the image with the additional information; and a display for displaying the composite image generated by the additional information combining program execution module. The image properties of the sent image and the additional information are converted so as to be optimally displayed on the intended receiving communication terminal, then the sent image and the additional information are combined and the resulting composite image is sent and received.
According to a fiftieth aspect of the invention, there is provided a communication apparatus comprising a server connected to a network and a plurality of communication terminals each being capable of communicating with one another through the server, the communication apparatus being capable of combining a plurality of images and sending the resulting composite image from any of the communication terminal to any other communication terminals, the server comprising: an image storage for receiving and storing an image sent from a sending communication terminal; an additional information storage for storing a plurality of pieces of additional information to be combined with the sent image; an additional information selector for selecting a piece of additional information to be combined with the sent image according to an instruction from the sending communication terminal; a composite image confirming module for combining the sent image with the additional information and sending the resulting composite image to the sending communication terminal for confirmation; and an additional information combining program transmitter for sending the image, the additional information, and a program for converting the image properties of the send image and the additional image according to the terminal information and combining the converted image with the converted additional information; the communication terminal comprising: an image viewing module for displaying the composite image received from the composite image confirming module; an image transmitter for selecting an image and sending the image to the server; an additional information specifying module for specifying additional information to be added to the image to be sent while viewing the additional information on the image viewing module; a transmission commanding module for issuing a command for transmitting the image and a program for converting the image and the additional information according to the capabilities of a receiving communication terminal and combining the image with the additional information to the receiving communication terminal; a terminal information acquisition module for acquiring terminal information about the intended receiving communication terminal; an image feature detector for detecting a feature of the sent image; an additional information combining program execution module for receiving the program for converting the image and the additional information according to the capabilities of a receiving communication terminal and combining the image with the additional information and executing the program to convert the image and the additional information and superimpose the additional information on a position in the sent image, according to the terminal information, the position being determined according to the feature of the image; and a display for displaying the composite image generated by the additional information combining program execution module. The image properties of the sent image and the additional information are converted so as to be optimally displayed on the intended receiving communication terminal, then the sent image and the additional information are combined and the resulting composite image is sent and received.
According to a fifty-first aspect of the invention, there is provided the communication apparatus according to the fiftieth aspect, wherein the image feature detector detects the size, position, and/or motion of a person in the sent image.
According to a fifty-second aspect of the invention, there is provided the communication apparatus according to the fiftieth aspect, wherein the image feature detector detects the size and position of the face or other parts of the person in the sent image.
According to a fifty-third aspect of the invention, there is provided the communication apparatus according to the fifty-first or fifty-second aspect, wherein the image feature detector detects the skin color of the person in the sent image so that the skin color of the person is not impaired by the conversion of the image properties.
According to a fifty-forth aspect of the invention, there is provided a communication apparatus comprising a server connected to a network and a plurality of communication terminals each being capable of communicating with one another through the server, the communication apparatus being capable of adding a special effect to an image and sending the resulting composite image from any of the communication terminal to any other communication terminals, the server comprising: an image storage for receiving and storing an image sent from a sending communication terminal; a special effect storage for storing a program for adding a special effect to the image; a special effect selector for selecting a special effect to be added to the image according to an instruction provided from the sending communication terminal; a composite image confirming module for adding a special effect to the image and sending the resulting composite image to the sending communication terminal for confirmation; and an additional information combining program transmitter for sending the image and a program for converting the image according to the capabilities of a receiving communication terminal and generating a composite image to the receiving communication terminal; the communication terminal comprising: an image viewing module for displaying the composite image received from the composite image confirming module; an image transmitter for selecting an image and sending the image to the server; an additional information specifying module for specifying a special effect to be added to the image to be sent while viewing the special effect on the image viewing module; a transmission commanding module for issuing a command for transmitting the image and the program for converting the image according to the capabilities of a receiving communication terminal and adding the specified special effect to the converted image to the receiving communication terminal; a terminal information acquisition module for acquiring terminal information about the intended receiving communication terminal; an additional information combining program execution module for receiving the program for converting the image according to the capabilities of a receiving communication terminal and adding the specified special effect to the image and executing the program to convert the image according to the terminal information and add the selected special effect to the image; and a display for displaying the composite image to which the special effect is added by the additional information combining program execution module. The image properties of the image are converted so as to be optimally displayed on the intended receiving communication terminal and then the image to which the special effect is added is sent and received.
Embodiments of the present invention will be described below with reference to the accompanying drawings.
(First Embodiment)
A first embodiment will be described with reference to
In
In
Thus, a communication apparatus can be provided that can request users of interest to connect a server on a network in order to join a conversation even if they are not connected to the Internet.
On the monitor screen 10 shown in
The low-pass filter 22 may integrate outputs from the detector 21 for a predetermined period of time and directly output the result or calculate and output the average of the results of m consecutive integrations. The integration period and the number m of integrations for taking an average may be adjusted by taking the relationship between the state of actual conversation and the frequency of image enlargement into consideration.
While the embodiment has been described with respect to a chat as a communication tool on the Internet, the embodiment can also be used with other communication tools on the Internet such as internet-enabled telephones.
In this way, a clearer image of a speaker can be provided and therefore feelings of the speaker can be indicated to the receiver(s) during communication on the Internet.
(Second Embodiment)
A communication apparatus according to a second embodiment of the present invention will be described with reference to
In
In
While a facial information compositor 49 for combining facial expression information is provided on senders' personal computers 6 in this example, it may be provided on the server 1 or receivers' personal computer 6.
Information for expressing emotions is managed in the facial expression information circuit 44. However, the information may be generated by users on their personal computers 6 or other devices connected to the network 4. In addition, the information may be updated by downloading required information from the information library 7 over the network 4 and storing it in the expression information circuit 44.
The procedure for inviting participants in a conversation in the second embodiment is the same as that described with respect to the first embodiment and shown in
The screen shown in
If user b is annoyed by such a state during conversation and wants to communicate the emotion with a facial expression to the other users, user b presses a switch 41 corresponding to anger from among the emotional expression switches assigned to the keyboard of his or her personal computer 6. As a result, facial expression information 12 outputted from the facial expression information circuit 44 that is associated with the switch 41 is superimposed on the image 11 of user b captured by the camera and the resulting image is sent and displayed on the monitor screen 10 of each user's personal computer 6. While a case where anger is expressed has been described herein, other expressions such as delight, anger, sadness and pleasure may be expressed as well.
The positions in which facial expression information 12 is superimposed are fixed. For example, the facial expression information 12, which is horns and fangs in this example, is superimposed on the right positions with respect to the position and size of the face that are detected in the photograph 11. The positions of the facial expression information 12 therefore change according to changes in the photograph 11, that is, the movement of the face.
While graphics are used as an example of the facial expression information 12 in the description of the second embodiment, various other images such as a photograph of a celebrity or, more straightforwardly, letters can be superimposed. Furthermore, instead of the keyboard 6 of the personal computer 6, voice or facial expressions may be assigned as switches for emotional expressions and speech or image recognition may be used to match them with preset ones during conversation to turn on one of the switches.
While the embodiment has been described with respect to a chat as a communication tool on the Internet, the embodiment can also be used with other communication tools on the Internet such as internet-enabled telephones.
Thus, according to the configurations described above, a communication apparatus can be provided that has an emotional expression indicating capability that can add user's emotional expression to a message to send to the receiving user during communication on the Internet.
(Third Embodiment)
A communication apparatus according to a third embodiment of the present invention will be described below with reference to
In
If, after a video conversation among a number of users starts, user a wants to indicate a feeling such as irritation or wants to attract the other user's attention, user a turns on the switch associated with the vibration device 13. This switching information is provided to the other users' communication terminals 8 through the server 1 and the network 4 to drive the vibration devices provided in the communication terminals 8 and the vibration is transmitted to the other users. Thus, user a can indicate the irritation or the intention to attract the other users' attraction to them.
Thus, according to the configurations described above, a communication apparatus can be provided that has an emotional expression indicating capability that can add user's emotional expression to a message to send to the receiving user during communication on the Internet.
Furthermore, the vibration device 13 may be used for requesting the users to connect to the server.
While the embodiment has been described with respect to a chat as a communication tool on the Internet, the embodiment can also be used with other communication tools on the Internet such as internet-enabled telephones.
Thus, a communication apparatus can be provided that allows a user who wants to perform communication on the Internet to request users of interest to connect to a server on the Internet if they are not connected to the Internet.
(Fourth Embodiment)
A communication according to a fourth embodiment of the present invention will be described with reference to
In
An operation of the apparatus will be described with reference to
The server 1101 sends a synthesized image with additional information added to an image by a sender with the sending terminal 1109 to a receiver.
The most commonly used sending and receiving terminals 1109 and 1110 are mobile phones. With the explosion in the use of mobile phones with cameras in recent years, it has become commonplace to send a photograph taken by a user to his/her friend's mobile phone. The communication apparatus of the present invention allows a user to add additional information such as animation, for example, to a picture taken by the user before sending so that an elaborated image or content is sent.
While the present embodiment will be described with respect to an image as an example of information to be added, other data such as text or music can be added.
Although both moving and still images can be sent, an example in which a moving image is sent will be described below.
First, a sender stores an image to be sent to a receiver's receiving terminal 1110 in the image storage 1102 of the server 1101 from a sending terminal 1109 (S1). This process is commonly known as upload of the image. The image to be sent is typically a picture taken with a camera contained in a mobile phone. However, images obtained with any other means may be used.
Then, the sender uses the additional information selector 1105 to select additional information to be added to the image to be sent from the additional image storage 1103 containing additional information to be added to images to be sent (S2). The sender previews additional information as shown in
The additional information compositor 1104 combines the image stored in the image storage 1102 with the additional information selected by the sender from the additional information storage 1103. The composite image data is sent to the sending terminal so that the sender can confirm and select it (S3). The composite image is recomposed according to a format that can be displayed on the receiving terminal 1110 and then sent to the terminal 1110, as will be described later.
If the sender's sending terminal 1109 and the receiver's receiving terminal 1110 are of the same model and use the same communication company, then the receiver can see exactly the same image as the one viewed on the sending terminal 1109. However, in many cases the sending terminals 1109 and the receiving terminal 1110 are of different models and use different communication companies. As a result, the receiving terminal 1110 possibly cannot display or cannot properly display the composite image. For example, if the sending terminal 1109 supports MPEG4, which is the international standard for motion pictures, generates the composite image based on MPEG4, and sends it to the receiving terminal 1110 but the receiving terminal 1110 does not support MPEG4, the composite image received at the receiving terminal 1110 cannot be displayed on the receiving terminal 1110.
Therefore, the communication apparatus according to the fourth embodiment obtains information about the receiving terminal 1110, uses the information to convert the image in the image storage 1102 and the additional information in the additional information storage 1103 into a format that can be displayed on the receiving terminal 1110, then combines them with each other in the additional information compositor 1104 and sends the resulting image to the receiving terminal 1110.
For that purpose, the apparatus obtains the format supported by the receiving terminal 1110 through the terminal information acquisition module 1107. This is done because most senders do not hold information about the models of receiving terminals owned by receivers and the information must automatically be obtained.
Therefore, when the sender sends the image combined with the additional information to the receiver, the server 1101 first sends mail to the receiver through the e-mail transmitter 1108 (S4), rather than immediately sending the composite image. The mail message prompts the receiver to access the server 1101 as shown in
The receiver clicks the URL contained in the e-mail to access the sever 1101 (S5). On accessing the server 1101, the receiving terminal 1110 sends information about the terminal such as hardware capabilities, including the size of its display screen, display colors supported, and the CPU throughput, and the type of its browser and image formats supported. The terminal information acquisition module 1107 receives the information (S6) and sends information required for adding images to the additional information compositor 1104.
The additional information compositor 1104 converts the format of the image stored in the image storage 1102 and the additional information in the additional information storage 1103 according to the display capabilities and an image format supported by the receiving terminal 1110 (S7). It then combines the image and the additional information (S8). Finally, the composite image is sent from the server 1101 to the receiving terminal 1110 (S9). Thus, the receiver can see the composite image on the receiving terminal 1110.
The image conversion and composition will be described in detail with reference to
A case will be described in which a sequence of MPEG4 images as shown in
First, if the terminal information acquisition module 1107 determines that the receiving terminal 1110 supports animation GIF, rather than MPEG4, as the format of moving images and can display only three image due to its display capabilities, then the additional information compositor 1104 selects three images from among the MPEG4 images stored in the image storage 1102, as shown in
The additional information compositor 1104 then converts the selected image and animation according to information provided from the terminal information acquisition module 1107 such as the display size, display capacity, display colors of the receiving terminal 1110 and combines them as shown in
The composite image is sent through the composite image transmitter 1106 to the receiving terminal 1110 and the receiver can view the composite image optimized for the receiving terminal 1110.
The series of operations are automatically performed in the server 1101. All that required to be performed by the sender is to send out the image combined with the additional image to the receiver. The server 1101 adapts the image to the capabilities of the receiving terminal 1110 and transfers it.
In this way, when a communication terminal such as a mobile phone sends and receives data including an image, the image and additional information added to it are converted to suit the capabilities of that receiving communication terminal. Therefore, any image can be properly displayed according to the capabilities of the receiving terminal.
Furthermore, when the image properties of an image stored in the image storage unit 1102 and additional information in the additional information storage 1103 are converted in accordance with the display capabilities and an image format supported by the receiving terminal 1110 and the images are combined, additional information image is resized and the number of colors are reduced. A number of versions of additional information having different image sizes and colors may be provided so that the most appropriate one can be selected.
In particular, after an image is selected from the additional information stored in the additional information storage 1103, the image should be scaled up or down or the number of colors in the image should be reduced to suit a receiving terminal 1110. The receiving terminal typically is a mobile phone, which has a display screen of small size. When the size of additional information is reduced, it can be difficult to identify what information is displayed. For example, if the additional information is letters, reduction in size of the letters can make them unreadable. Furthermore, when the number of colors is reduced to automatically optimize the additional information, it can be difficult to determine what colors will ultimately be used. As a result, the appearance of the additional information can be altered unexpectedly.
Therefore, a number of versions of additional information that would not become unreadable when their sizes are reduced are provided beforehand. In addition, a number of versions of each additional information that would not take on a strange appearance with a smaller number of colors are provided. Then, the most appropriate combination is selected from the additional information stored in the additional storage 1103 and superimposed on an image by the additional information compositor 1104. For letters, a number of font sizes are provided so that the most suitable font size for the receiving terminal 1110 is selected. Thus, the most suitable image can be sent to the receiving terminal.
While the image to be sent and the additional information are separately undergo the image property conversion and are combined in the sever and then sent to the receiving terminal in the foregoing description, the image and the additional information may be sent from the server to the receiving terminal, where they may undergo the image property conversion and combination.
(Fifth Embodiment)
A communication apparatus according to a fifth embodiment of the present invention will be described with reference to
The communication apparatus shown in
The person recognition module 1111 identifies the position of body parts such as the face, eyes, and mouth of a person in an image stored in the image storage unit 1102. The person recognition is accomplished by using technologies such as skin color detection and eye and mouth pattern matching.
The additional information compositor 1104 converts the image properties of the image stored in the image storage 1102 and additional information in the additional information storage 1103 according to the display capabilities and an image format supported by the receiving terminal 1110 and combines them with each other. The way they are combined is varied in accordance with the size, position, and motion of the person in the image in the image storage 1102.
For example, if the sender selects an eye animation image as shown in
Moreover, the way of superimposition may be varied in accordance with the motion of the person.
It has been described in the fourth embodiment that if the terminal information acquisition module 1107 determines that the receiving terminal supports animation GIF, instead of MPEG4, as the format of moving images and only three pictures can be selected from MPEG4 pictures due to the display capabilities of the animation GIF, the three image may be the first image, an intermediate image, and the last image or may be the three most significantly varying pictures.
In the fifth embodiment, three pictures which considerably differ in the motion of the person can be selected and combined. The pictures that best characterize an image can be selected in this way. Consequently, the most appropriate image combined with additional information can be generated and sent to the receiving terminal 1110.
If the display capability of the receiving terminal 1110 necessitates the use of animation GIF and a base image is in full color, color reduction must be performed. The color reduction may result in an inappropriate color of the face because the number of available colors is limited and an appropriate skin color is not assigned to the face. Therefore, if the presence of a person is recognized by the person recognition module 1111, color reduction is performed in such a way that the skin color is simply not impaired by the color reduction. Consequently, the image can be sent without impairing the skin color of the person.
In this way, before data including a picture of a person is sent and transmitted by a communication terminal such as a mobile phone, the image and additional information are properly converted to suit the capabilities of the receiving terminal. Thus, any image can be properly displayed according to the capabilities of the receiving terminal.
While the image to be sent and the additional information are separately undergo the image property conversion and are combined in the sever and then sent to the receiving terminal in the foregoing description, the image and the additional information may be sent from the server to the receiving terminal, where they may undergo the image property conversion and combination.
(Sixth Embodiment)
A communication apparatus according to a sixth embodiment of the present invention will be described with reference to
The communication apparatus shown in
According to the sixth embodiment, a special effect is applied to an image stored in an image storage 1102. A special effect herein referred to effects that alters the shape of facial features as shown in
To apply a special effect, a person in an image in the image storage 1102 is recognized and the position of the face and parts of the face such as the eyes and mouth are recognized by a person recognition module 1111. The person recognition is accomplished by using technologies such as skin color detection and eye and mouth pattern matching.
The special effect compositor 2104 converts image properties of an image to which a special effect is applied to suit the display capabilities and the image format supported by the receiving terminal 1110. For example, if the display size of the receiving terminal 1110 is small and the degree of a change to an image is small, the special effect cannot be distinct to the eye. Therefore, before the special effect is applied to the image, image properties of the image are converted to increase the degree of the change so that the special effect becomes distinctive on the receiving terminal 1110.
In another example, if the presence of a person is recognized by the person recognition module 1111, color reduction is performed in such a way that the skin color of the person is simply not impaired. In this way, loss of the skin color can be prevented.
In this way, before data including a special-effect image is sent and transmitted by a communication terminal such as a mobile phone, the image is properly converted to suit the capabilities of the receiving terminal. Thus, any image can be properly displayed according to the capabilities of the receiving terminal.
While the image to be sent and the additional information are separately undergo the image property conversion and are combined in the sever and then sent to the receiving terminal in the foregoing description, the image and the additional information may be sent from the server to the receiving terminal, where they may undergo the image property conversion and combination.
(Seventh Embodiment)
A communication apparatus according to a seventh embodiment will be described below with reference to
In the embodiments described above, the server converts the image properties of an image to be sent and additional information to suit a receiving terminal, combines the resulting images into a composite image, and then sends it to the receiving terminal. However, the receiving terminal may convert the image properties of an image and additional information that it received to appropriate image properties and combines the images.
In
A process performed by these components will be described below.
First, when a sender wants to send a composite image produced from an image and a selected piece of additional information from the sending terminal 1109 as in the fourth embodiment, the server 2301 in the seventh embodiment does not perform conversion nor combination but separately sends the image to be sent and additional information through the image transmitter 2305 and the additional information transmitter 2307 to the receiving terminal 2309.
Omitted from
Next, in the receiving terminal 2309, the additional information compositor 2311 uses the terminal information about the receiving terminal 2309 to convert the image received through the image receiver 2310 and the additional information received through the additional information receiver 2312 into an image and additional information having image properties that can properly be displayed on the receiving terminal and then combines them with each other.
Finally, the composite image is displayed on the display 2314 of the receiving terminal 2309.
The program for the additional information compositor 2311 to convert the image and additional information can be obtained at the program download module 2315 over the network 2316. Consequently, up-to-data or optimum conversion can be performed.
A program for combining the image with the additional information after their image properties are converted to suit to the terminal information may be sent from the server 2301 to the receiving terminal 2309 along with the image and additional information, thereby allowing the receiving terminal 2309 to combine the image with the additional information after image property conversion.
In
The additional information composition program contains a program for obtaining terminal information from the receiving terminal 2403.
In the configuration described above, instead of the additional information composition module 2311 converting the image properties of an image and additional information and combining them, the additional information composition program execution module 2404 uses terminal information, a received image and additional information, and the program for converting the image properties of the received image and additional information according to the terminal information and combining the converted image with additional information to convert the image properties of the image and additional information according to the terminal information and then combine them so that the composite image can be properly displayed on the receiving terminal.
The additional information composition program may be written in a platform-independent program language such as JAVA® so as to support a wide range of machine models.
According to the embodiment as described above, before data including a special-effect image is sent and transmitted by a communication terminal such as a mobile phone, the image is properly converted in the receiving terminal to suit the capabilities of the receiving terminal. Thus, any image can be readily and properly displayed according to the capabilities of the receiving terminal. Furthermore, even if an image is sent to a plurality of receiving terminals, the image can be properly displayed according to the capabilities of each receiving terminals in a single process because the conversion and combination process is performed in each receiving terminal without the need for the server to perform conversion and combination for each receiving communication terminal.
In summary, the communication apparatus of the present invention allows a user to send a request for participating in a conversation to a communication terminal of a user of interest in order to initiate communication over the Internet so that the sender can request the user of interest to connect to the Internet if the user of interest is not connected to the Internet.
Moreover, when users perform communication over the Internet while displaying images provided from each other, images indicating feelings that are stored in a library can be combined with the images to be transmitted. Thus, the users can indicate their feelings during the communication over the Internet.
Furthermore, when an image is transmitted between communication terminals such as mobile phones, the image can properly be displayed on the receiving communication terminal because the image properties of the image are converted according to the capabilities of the receiving terminal.
Moreover, when a plurality of images are combined or an modification is made to an image in communication terminals such as mobile phones before transmission, the image properties of each image are converted to suit the capabilities of a receiving communication terminal before the combination or modification. In this way, any image can be properly displayed in accordance with the capabilities of the receiving terminal.
Number | Date | Country | Kind |
---|---|---|---|
2002-009314 | Jan 2002 | JP | national |
Number | Date | Country | |
---|---|---|---|
Parent | PCT/JP02/13038 | Dec 2002 | US |
Child | 10873507 | Jun 2004 | US |