Patent Application
20050278749
The present invention relates to radio telecommunication systems. More particularly, and not by way of limitation, the invention relates to a system and method for automatically modifying multimedia messages sent from a mobile phone with multimedia capabilities.
The MMC Relay/Server 34 sends an M-Notification.ind message 37 to the MT 32 via a HTTP/PUSH through a WAP Gateway 38. The M-Notification.ind message informs the MT that a multimedia message is available for retrieval. Included as a data component is a Universal Resource Locator (URL) of the multimedia message that is being stored by the MMC Relay/Server. The URL functions as a message ID for the specific message to be delivered to this specific MT. The MT sends an M-Notify Response.ind 39 to the MMC Relay/Server indicating that the M-Notification.ind message was received. At some time later, when the MT desires to retrieve the multimedia message, the MT sends a WSP GET.req message 40 to the MMC Relay/Server. The retrieval activity is performed via the WSP/HTTP on the URL provided. In response, the MMC Relay/Server sends to the MT an M-Retrieve.conf message 41, which contains the actual multimedia message to be presented to the receiving subscriber.
The MMC Relay/Server 34 may also request information from the MT 32 to confirm that the MT actually received the multimedia message. In one approach, a distinct M-Acknowledge.ind message 42 is sent from the MT to the MMC Relay/Server when the multimedia message is successfully received. The MMC Relay/Server may optionally send a delivery report back to the MO 31. This may be done utilizing HTTP/PUSH technology to send an M-Delivery.ind message 43 to the MO.
Despite the features described above, the number and frequency with which subscribers utilize MMS is below the expectations of network operators. In many cases, subscribers might utilize MMS, but conditions are not good for taking photographs. Alternatively, the subscribers may not think that their photograph is interesting enough to send. The present invention provides a solution to help overcome these problems and increase subscriber utilization of MMS.
It is therefore an object of the present invention to overcome the above mentioned problems and to provide a system and method for automatically modifying multimedia messages sent from a mobile phone with multimedia capabilities.
Thus, in one aspect, the present invention is directed to a method of modifying a photograph contained in a multimedia message sent from a camera-enabled mobile phone. The method includes the steps of receiving in a service center, an original photograph in the multimedia message; extracting an image from an image database; and creating a modified photograph by pasting the image into the original photograph. The multimedia message may also include location information for the camera-enabled mobile phone, and the extracting step may include utilizing the location information to extract from the database, an image associated with the location of the mobile phone. The method may also include sending the modified photograph to the camera-enabled mobile phone; receiving from the camera-enabled mobile phone, an indication that the modified photograph is selected for sending to a recipient; and sending the modified photograph to the recipient.
In yet another aspect, the present invention is directed to a method of modifying a photograph contained in a multimedia message sent from a camera-enabled mobile phone. The method includes the steps of receiving in a service center, an original photograph in the multimedia message; and creating a modified photograph by enhancing the brightness, color, or clarity of the original photograph.
In still yet another aspect, the present invention is directed to a Multimedia Messaging Service Center for modifying a video stream contained in a multimedia message sent from a video-enabled mobile device. The Center includes means for receiving an original video stream from the video-enabled mobile device; a database for storing a plurality of video images; and a video manipulator that creates a modified video stream by pasting a plurality of video images extracted from the database into the original video stream. The Center may also include means for sending the modified video stream to the video-enabled mobile device; means for receiving from the video-enabled mobile device, an indication that the modified video stream is selected for sending to a recipient; and means for sending the modified video stream to the recipient.
In the following section, the invention will be described with reference to exemplary embodiments illustrated in the figures, in which:
In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular embodiments, circuits, signal formats, and the like in order to provide a thorough understanding of the present invention. It will be apparent to one skilled in the art that the present invention may be practiced in other embodiments that depart from these specific details.
The present invention provides an enhanced MMS+ service, which may be particularly useful when environmental conditions are not optimal for taking photographs. For example, there may not be enough light, it may be raining, or a particular tourist attraction that the subscriber wishes to photograph may be under construction or renovation. The present invention enables the subscriber to take a photograph under the less than optimal conditions, and send the photograph to an MMS+ service provider. The MMS+ service provider automatically manipulates the photograph to improve its quality, and sends one or more versions of the photograph back to the subscriber for review and approval. The MMS+ service provider may manipulate the photograph itself (for example, zooming in on items of interest or adjusting the brightness, color, or clarity), or may paste images from an image database into the photograph. The subscriber can accept a manipulated photograph and send it to another subscriber, reject the manipulated photograph and discard it in favor of the original photograph, or cancel the transaction altogether.
In one embodiment, the subscriber's mobile phone requests its position via location-based services, and sends the location information together with a photograph to the MMS+ service provider. The location information may be pasted into the photograph in a suitable manner, for example by a java application, before the photograph is sent from the mobile phone. When the MMS+ service provider receives the photograph and the location information, the MMS+ service provider extracts the location information. The location information may then be converted to an ID associated with the approved MMS+ location closest to the location received from the mobile phone. The location information, or optionally the location ID, is then used to access a database of images to determine whether there is an image stored in the database that is applicable to the photograph received from the subscriber. If so, automatic manipulation is performed.
In another embodiment, the mobile phone sends an MMS message to the MMS+ service provider with a photograph and an indication that enhanced MMS+ services are requested. The MMS+ service provider then obtains the location of the mobile phone through location-based services. Alternatively, deferred location reporting is used, and the MMS+ service provider is notified whenever the user enters an identified area where enhanced MMS+ services are provided. The MMS+ services provider stores this information so that when the MMS+ services provider receives an MMS message from the mobile phone, the MMS+ services provider can associate the location with the received MMS message.
In one embodiment, if the photograph contains people in front of a poor quality background, the MMS+ service provider may enhance the photograph by keeping the people in the foreground and replacing the background with a high quality background image appropriate for the reported location. For example, if the location information indicates that the subscriber is near the Eiffel Tower, a high quality background image of the Eiffel Tower is selected and pasted into the photograph. Since the photograph has been taken from a known location in a known direction, the perspective of the original photograph and the image in the database should be quite similar.
Additionally or alternatively, the MMS+ service provider may manipulate the photograph to enhance the colors, brighten the image, and/or improve the clarity of the image. The MMS+ service provider compares the two pictures and adjusts them to minimize differences in brightness and color per pixel. Additionally, a specific area in the original photograph such as the area containing the people in the foreground, or an area containing a point of interest, may be marked with a stencil. Within this area, the original photograph is compared with the image in the database. Each pixel between the original photograph and the image in the database is compared regarding color and brightness. If the differences exceed a predefined threshold, the pixel color and brightness from the original photograph are taken; otherwise, the pixel color and brightness from the database image are taken. This algorithm may be enhanced with more sophisticated, known, edge-detection mechanisms to minimize the border between the stenciled area and the rest of the photograph.
In another embodiment, the MMS+ service provider may add elements into the photograph that were not in the original photograph. For example, images of celebrity persons of public interest, such as politicians, entertainers, sports stars, and the like may be inserted into the photograph in such a way that they appear to be standing with the people in the original photograph. Images of other objects may also be inserted into the photograph to add interest or humor. This mechanism may simply copy silhouettes into the original photograph. The position where the silhouette is inserted may be fixed, or provisions can be made to enable the subscriber to designate the desired position using his mobile phone. For example, after a photograph is taken with the camera-enabled mobile phone, or when a preview image is received, a small application in the mobile phone is started. The subscriber marks the desired position of the silhouette in the picture with a small cursor. This information may be transferred to the MMS+ service provider via a separate data message such as a Short Message Service (SMS) message, or the data may be included in the picture itself. The MMS+ service provider retrieves this information and positions the silhouette accordingly. This process may alternatively be used to reposition a stenciled area within the photograph.
The image database may contain pictures for different camera types or lens focal lengths. Therefore, the camera-enabled mobile phone may also send to the MMS+ service provider, other information such as the camera type, lens focal length, resolution, and distance to the person in the photograph. This information enables the MMS+ service provider to utilize stored perspective information to select and paste an image with an appropriate scale into the photograph.
The camera-enabled mobile phone may also download or otherwise contain an editing application that provides the subscriber with greater editing capabilities. The editing may be performed in the phone, or the application may enable the subscriber to indicate desired editing changes to the MMS+ service provider, which then performs the actual editing and returns the edited photograph to the subscriber for review. For example, a building of particular interest may appear in the background of the photograph, but the building is too small. The editing application enables the subscriber to crop the photograph and enlarge the building so that people in the foreground appear to be standing closer to the building. If the MMS+ service provider performs the editing, the subscriber may place a cursor on the building and click a command to zoom. The photograph and editing commands are then sent to the MMS+ service provider, which performs the manipulation and returns the photograph to the subscriber for review. Other types of editing and special effects such as wide angle or fisheye lens effect can also be performed.
At step 54, instead of immediately sending an M-Notification.ind message to the MT, the MMS+Center accesses the database and may manipulate the photograph with images from the database to create one or more preview versions. Many methods that are used by today's picture or film-editing programs for PCs can be applied by the MMS+Center to automatically manipulate the photograph. At step 55, the MMS+Center sends one or more of the previews back to the MO for review. The previews may be sent one at a time or, optionally, several previews may be sent in one image. Alternatively, descriptive text may be sent, enabling the MO to select previews that he wants to review. At step 56, the MO selects one of the previews and requests it from the MMS+Center. The request may be made by sending an SMS message to the MMS+Center. At step 57, the MMS+Center receives the request and sends the requested manipulated photograph to the MO. At step 58, the MO receives the manipulated photograph and at step 59 sends it via the MMS+Center or a regular MMC to the MT.
There are several alternative points during the process at which the MO may be billed for the service. For example, he may be billed when he sends the photograph to the MMS+Center requesting manipulation. Alternatively, the MO may be billed later, only if he likes and selects one of the previews.
The MMS+Center 63 then sends an M-Notification.ind message 68 to the MT 32 via a HTTP/PUSH through the WAP Gateway 38. The M-Notification.ind message informs the MT that a multimedia message is available for retrieval. The MT sends an M-Notify Response.ind 69 to the MMS+Center indicating that the M-Notification.ind message was received. At some time later, when the MT desires to retrieve the multimedia message, the MT sends a WSP GET.req message 70 to the MMS+Center. In response, the MMS+Center sends to the MT an M-Retrieve.conf message 71, which contains the manipulated photograph sent by the MO. The MT then sends an M-Acknowledge.ind message 72 to the MMS+Center when the multimedia message is successfully received. The MMS+Center may optionally send an M-Delivery.ind message 73 back to the MO indicating that the delivery was made.
The photograph, and optionally the camera type, are sent to a photograph manipulator 85. The photograph manipulator may include a background image replacer 86; a color, brightness, and sharpness enhancer 87; and an image inserter 88 as previously described. Images from the image database 84 are supplied to the photograph manipulator, if available. If images are available and/or other manipulation can be performed, the photograph manipulator informs a preview controller 89 that one or more previews are available. The preview controller notifies the MO 61 that previews are available, and if the MO requests them, they are sent in the M-Retrieve.conf message 66 to the MO. The MO then reviews the previews and sends a preview selection to the MMS+Center in the M-Send.req message 67. The preview controller 89 then sends the requested photograph to the normal MMS forwarding function 82 for forwarding to the MT 32, as shown in steps 68-73 of
The process described above for still photographs can also be utilized to modify video and/or audio sequences. Thus, streaming video taken by a mobile device and sent to an MMS+Center can be modified to manipulate or enhance the quality of the video. For example, if a short film is made of a person standing in front of a sight seeing attraction, the MMS+Center can add images to the video sequence in such a way that a famous celebrity appears to walk through the background of the film. If the video has associated audio, the audio can also be modified.
As will be recognized by those skilled in the art, the innovative concepts described in the present application can be modified and varied over a wide range of applications. Accordingly, the scope of patented subject matter should not be limited to any of the specific exemplary teachings discussed above, but is instead defined by the following claims.
