System and method for integrating audio and visual messaging

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates in general to communication systems such as voice paging systems, cellular telephone systems, text paging systems, voice mail systems, and conventional land line telephone systems and, more specifically, to methods, systems and apparatus for non-real-time audio and visual messaging between two messaging devices wherein a communication device receives visual messages from a visual message originator device and transmits audio messages (e.g., voice messages) to the visual message originator device for playback.

2. Discussion of the Background

A conventional mobile communication device (MCD), such as a pager or a mobile telephone, can receive various types of messages. There are numeric pagers which receive only numbers such as a telephone numbers (i.e. 818-555-1212), and there are alpha/numeric pagers which can receive alpha/numeric messages (e.g., Please call me at 929-555-1212). Additionally, there are voice pagers which receive voice messages. In addition to receiving messages, pagers are often configured to transmit messages. An alpha/numeric pager may be configured to receive alpha/numeric messages from an Internet-based computer and to transmit to the Internet-based computer originator alpha/numeric reply messages. Similarly, a voice pager that receives voice messages from a telephone-based originator can be configured to transmit a voice message to a voice mail system for retrieval by the originator via a telephone. In short, alpha/numeric pagers are configured to transmit alpha/numeric reply messages and voice pagers are configured to transmit voice reply messages.

In the field of wireless messaging generally, numeric and alpha/numeric messaging is, by far, more bandwidth efficient than voice messaging. Radio spectrum is a dwindling resource and wireless messaging companies are increasingly sensitive to their bandwidth efficiencies. Better bandwidth utilization means more capacity on the system. More capacity equates to more customers. Hence, the wireless messaging industry has migrated from voice paging to alpha/numeric paging in order to provide a bandwidth efficient, robust and high information content messaging service for their so called “road warrior” customers.

However, to date, there is no simple method for sending a message containing more than a small amount of information from an alpha/numeric pager. Generally, alpha/numeric pagers that have the ability to send messages are configured to transmit small, preprogrammed (“canned”) text messages, such as “I will call you tonight” or “yes” or “no” or “I will reply later.” Other alpha/numeric pagers have been configured with a miniature, QWERTY type, text keyboard. The miniature keyboards are difficult to type on because of their size. This makes input very slow, inaccurate and very annoying to operate. Additionally, the miniature size of the keyboard is still too large for a pocket-sized pager.

Audio pagers are generally larger than keyboardless alpha/numeric pagers, in part because of the large speaker required in order to reproduce a high quality, audible audio message. An audio pager's battery life is lower than an alpha/numeric pager due to the amount of time it takes to receive a message and the amount of power required to process and play the message.

One solution to the above problem is to route a mobile telephone originated voice reply message to a computer. However, it's simply not convenient to send a voice message via a mobile telephone, configure a computer to receive a voice reply message and then mentally correlate the sent and received messages together (i.e. Was the reply “sounds good to me” meant to be an answer to the message, “I will pay $140,000.00 for your home” or “lets have lunch tomorrow at the fish place”).

A need, therefore, exists to blend audio (e.g., voice) and visual (e.g., text and/or graphics) messaging in a manner that conserves valuable bandwidth and simplifies user input of messages in a mobile communication device. An attempt to that end, is the so-called “smart phone.” Smart Phones are wireless mobile telephones that have added features, implying that they are smarter than the average telephone. These features may include a numeric and alpha/numeric messaging feature, a personal digital assistant (PDA), computer functions, Internet access, and a miniature keyboard. Similar to an alpha/numeric pager, many mobile telephones today are capable of receiving alpha/numeric messages and have the capability of connecting to the Internet for sending alpha/numeric reply messages via a miniature keyboard. There is a need to simplify the input of reply messages into wireless devices for delivery of the reply messages to the originating device, wherein both messages may be correlated.

In an alpha/numeric pager messaging environment, text messages are routinely exchanged between a computer and a text pager (i.e. text in/text out). In a voice pager messaging environment, voice messages are routinely exchanged between a telephone configured voice mail system and a voice pager (i.e. voice in/voice out). In a mobile telephone messaging environment, as with an alpha/numeric pager, text messages routinely are exchanged in non real-time (i.e. text in/text out). However, mobile telephones also have the ability to make and receive real-time voice calls. More often than not, when a mobile telephone user receives a text message requiring some type of response, he or she will simply use the mobile telephone to place a real-time telephone call to the message originator. In many cases, the line is busy because the originator is either on the phone or connected to the Internet. In either case, the mobile caller is diverted to a voice mail system or answering device and is instructed to leave a message, which is then retrieved by the called person at a later time.

Many people prefer to communicate by messaging as opposed to real-time conversation in order to manage their time. Thus, there is a need for a device that can not only send non real-time text messages, but also receive non real-time voice messages (i.e. text out/voice in). At the wireless side of the messaging loop, there is a need to send non real-time voice messages from the same mobile device that receives non real-time text messages (i.e. text in/voice out).

It is widely accepted in the field of two way paging, that only a fraction of received messages generate a reply message response. On the other hand, when text message reception capability is combined with a mobile telephone, the mobile telephone user will attempt to reply much more often via a real-time voice call. Mobile telephone companies call this process “call completion” and it is highly favored among mobile telephone companies because additional calling generates more revenue. There is a need to increase reply traffic in a wireless environment without decreasing the efficiency of text message delivery to a mobile communication device. There is also a need to simplify the input of messages in a mobile communication device, such as a pager or wireless mobile telephone. Finally, there is a need for a wireless messaging system that provides end to end audio and visual messaging, wherein only one device is required at each end of the messaging loop (e.g., computer on the one end and mobile communication device on the other).

SUMMARY OF THE INVENTION

The present invention provides a communication system for integrating audio and visual messaging. The communication system includes a communication device for receiving visual messages and for transmitting voice messages to a recipient, and an integrated mail gateway for receiving from the communication device a voice message and addressing information. The integrated mail gateway is programmed to create an electronic mail (hereinafter “e-mail”) message comprising the voice message. The integrated mail gateway is also programmed to use the addressing information to address the e-mail message, and to send the addressed e-mail message to the recipient.

Preferably, the communication device is a wireless mobile communication device. However, this is not a requirement. The present invention is contemplated to be used with wireless as well as non-wireless communication devices.

In one embodiment, the communication system is used by a user of a communication device to send a voice message in reply to a received visual message. For example, consider the situation where a first person uses a messaging device with Internet e-mail capability to transmit a visual message to a second person. The communication system of the present invention enables the second person to easily transmit a voice reply message to the first user's messaging device. In one embodiment, the second person uses a communication device (such as a conventional telephone or mobile telephone having visual message reception capability) to establish a telephone call with an integrated mail gateway (IMG) that preferably has access to the visual message sent to the second person. After the telephone call is established, the second person uses the communication device to transmit, or the communication device automatically transmits, to the IMG addressing information associated with the visual message received from the first person. For example, a keypad on the communication device could be used to transmit the addressing information, or a processor in the communication device can be programmed to automatically transmit the addressing information. In one embodiment, after the IMG receives the addressing information, it prompts the second user to begin speaking a voice reply message for the first person after hearing a tone. The IMG records and stores the voice reply message. Optionally, the IMG will format the voice reply message into a conventional audio file format. The IMG then creates an e-mail message and includes the voice reply message in the e-mail. Optionally, the e-mail message includes at least part of the received visual message to which the voice message is a reply. The IMG uses the addressing information provided by the mobile communication device to address the e-mail message. After the e-mail message has been addressed, the e-mail message can be sent to the first user. Upon receiving the e-mail message, the first user's messaging device can play the voice reply message associated with the original visual message so that the voice reply message is heard by the first user.

Further features and advantages of the present invention, as well as the structure and operation of various embodiments of the present invention, are described in detail below with reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated herein and form part of the specification, illustrate various embodiments of the present invention and, together with the description, further serve to explain the principles of the invention and to enable a person skilled in the pertinent art to make and use the invention. In the drawings, like reference numbers indicate identical or functionally similar elements. Additionally, the left-most digit(s) of a reference number identifies the drawing in which the reference number first appears.

FIG. 1 is a schematic illustration of a communication system according to the present invention.

FIGS. 2A-2D are diagrams of a routing map illustrating a voice message path from a mobile communication device to a messaging device.

FIG. 3 illustrates in block form a mobile communication device according to the present invention.

FIG. 4 is a flow diagram of a process for sending a voice reply message according to the present invention.

FIG. 5 is a schematic illustration of one embodiment of an integrated mail gateway and electronic mail gateway.

FIGS. 6A-6C are flow diagrams of a process for receiving voice messages from a mobile communication device.

FIG. 7 illustrates an example of a visual message received and displayed by a messaging device.

FIG. 8 illustrates an example of a reply message received and displayed by a messaging device.

FIG. 9 illustrates an exemplary address list message.

DESCRIPTION OF THE PREFERED EMBODIMENTS

A first embodiment of the present invention is hereinafter described, with reference to the diagram of FIG. 1. FIG. 1 illustrates two mobile communication devices, a paging transceiver 140 and mobile telephone 145, for communicating with messaging devices 105(1), 105(2), and 105(3), collectively referred to as messaging device 105. More specifically, the paging transceiver 140 and mobile telephone 145 are configured for receiving visual messages from and transmitting audio messages to the messaging device 105. Visual messages include such messages as text messages (i.e. alpha/numeric messages) and graphic messages, which can include text and graphic images. A messaging device is a device having the ability to send and receive visual messages. An example of a messaging device is a personal computer configured with e-mail software and a connection to the Internet. Messaging devices are well known to those skilled in the art of computers and communications.

Preferably, messaging device 105 is directly connected to a data network 120 (such as the Internet or other like network) or connected to an internet service provider (ISP) 170(1) or 170(2) (such as America Online) that has a connection to data network 120. ISP 170(1) and 170(2) each has a point of presence including data storage and retrieval equipment for enabling e-mail services and file transfer as is well know to those skilled in the art of internet communications. Messaging device 105(3) is shown connecting to ISP 170(2) through PSTN 110. Although this is the most common way today for messaging devices to connect to an ISP, other connectivity options are available, such as satellite links and cable modems. Messaging device 105 is enabled for two-way e-mail and file transfer communications. Data network 120 is a communications network for transporting data. There are no limits to the type of data carried by data network 120. For example, the data can be simple text data or it can be voice or video.

An integrated message gateway (IMG) 150, is connected to a public switched telephone network (PSTN) 110 for automated storage and forwarding of audio messages. Operation of the IMG 150 will be described in detail hereinafter.

A Mobile Switching Office (MSO) 125 (also referred to as a base station) comprises all of the necessary equipment and software for enabling communications between mobile telephone 145 and PSTN 110 as well as data network 120. The MSO 125 and mobile telephone 145 are configured for voice communications and visual messaging. Other than the improvements to be described in detail hereinafter, the MSO 125 and mobile telephone 145 components are readily available and are well known to those skilled in the art of mobile telephone communications. The mobile telephone system described herein may be configured to operate using various format technologies well known in the art of mobile telephone systems such as for example, CDMA, TDMA, GSM, N-AMPS, etc.

A Pager Switching Office (PSO) 130 (also referred to as a base station), comprises all of the necessary equipment and software for enabling communications between a paging transceiver 140 and the PSTN 110 or the data network 120. The PSO 130 and paging transceiver 140 are configured for voice messaging from the paging transceiver 140 to the PSO 130 and for visual messaging from the PSO 130 to the paging transceiver 140. Visual messaging is well known to those skilled in the art of paging systems. Paging systems capable of transmitting visual messages may be purchased from Motorola, a U.S. manufacturer of paging equipment. Voice paging systems are also well known to those skilled in the art of paging systems. Motorola manufacturers paging systems for transmitting voice messages to voice pagers. Readycom of Chapel Hill, N.C. produces a system for transmitting voice messages to cellular voice pagers and for transmitting voice messages from cellular voice pagers.

Other than the improvements to be described in detail hereinafter, the PSO 130 and paging transceiver 140 components are readily available and are well known to those skilled in the art of paging communications. The paging system described herein may be configured to operate using various format technologies well known to those skilled in the art of mobile telephone systems and paging systems, such as for example Inflection, pACT, TDMA, etc.

In operation, a user (not shown) enters an e-mail message via a keyboard (not shown) attached to messaging device 105. The e-mail message is addressed to one or more communication devices, such as paging transceiver 140 and/or a mobile telephone 145. The e-mail message is routed through the data network 120 to MSO 125 or PSO 130 for transmission to the designated communication device(s).

The mobile communication device receives/retrieves and stores the e-mail message and, optionally, alerts the user that a message has been received. The user may view the message from a display screen on the communication device. In order to reply to the e-mail message, the user simply activates a reply command (e.g., the user can press a “reply” button or select a “reply” menu option) and then speaks a voice message into the mobile communication device microphone (not shown). The voice message and addressing data are transmitted to the wireless system operator (i.e., MSO 125 or PSO 130) and forwarded to the IMG 150. The IMG 150 formats the voice message into an acceptable messaging file format. It then creates an e-mail message with the formatted voice message attached thereto. Optionally, part or all of the received visual message is included in the e-mail message, thus providing the e-mail originator with a means to easily identify the message to which the voice message is a reply. The e-mail message is then sent to the ISP 170(1) for retrieval by messaging device 105(2) or is sent directly to messaging device 105(1). The user of messaging device 105 then listens to the formatted voice message using an audio system.

The process of sending an e-mail message from a messaging device 105 to a mobile communication device is well known to those skilled in the art of wireless visual messaging systems. However, the integration of audio and visual messaging presents novel features never before available to a wireless service company or end user. With the present invention, a cellular telephone company may now offer integrated (i.e., audio/visual) messaging services which are transported over the Internet, thereby achieving a substantial reduction in cost. Cellular telephone companies are offering e-mail type text messaging to mobile telephone subscribers today (through the Internet). In short, the cellular telephone company is already connected to the Internet. The present invention provides new opportunities for transporting non real-time voice messages over a network connection that would not be possible in real-time.

Referring to FIG. 2A, there is shown a routing map illustrating a voice message path from a mobile telephone 145 to messaging device 105(2), wherein a remote IMG 150 is configured for dial up service and located between the data network 120 and the PSTN 110. A mobile user 202 speaks a voice message into the mobile telephone 145. The voice message is transmitted to the MSO 125 and routed through the PSTN 110 to the IMG 150. The IMG 150 stores and formats the voice message and sends it as an e-mail through the data network 120 to ISP 170(1). Messaging device 105(2) retrieves the voice message from service provider 170(1), and the voice message is played back on the messaging device 105(2) to a user 204.

Referring to FIG. 2B, there is shown a routing map illustrating a voice message path from a mobile telephone 145 to messaging device 105(2) wherein IMG 150 is privately connected to the MSO 125. The mobile user 202 speaks a voice message into the mobile telephone 145. The voice message is transmitted to the MSO 125 and routed to the IMG 150. The IMG 150 stores and formats the voice message and sends it as an e-mail through the data network 120 to ISP 170(1). Preferably, IMG 150 uses data network 120 to send the voice message to service provider 170(1). However, one skilled in the art will appreciate that IMG 150 can have a direct connection to service provider 170(1), thereby bypassing data network 120. Messaging device 105(2) retrieves the voice message from service provider 170(1), and the voice message is played back on the messaging device 105(2) to the user 204.

Referring now to FIG. 2C, there is shown a routing map illustrating a voice message path from paging transceiver 140 to messaging device 105(2), wherein an IMG 150(2) is directly coupled with the PSO 130 and interfaced to the PSO 130 e-mail network system. A pager user 206 speaks a voice message into the paging transceiver 140. The voice message is then stored in the paging transceiver. The stored voice message is transmitted to the PSO 130 using a paging protocol and routed to the IMG 150(2). The IMG 150(2) stores and formats the voice message and sends it as an e-mail through the data network 120 to ISP 170(1). The message is then retrieved and played by the messaging device 105(2).

Referring to FIG. 2D, there is shown a routing map illustrating a voice message path from paging transceiver 140 to messaging device 105(2), wherein an IMG 150 is privately networked to ISP 170(1). A pager user 206 speaks a voice message into the paging transceiver 140. The voice message is transmitted to the PSO 130 and routed to a IMG 150 where the voice message is formatted and sent via the ISP 170(1) and continues as previously described.

As illustrated above, a voice message may be routed from a mobile communication device 140, 145 to a messaging device 105 utilizing a variety of message routing designs. It is important to note that the voice message may be routed through one particular path while a visual message may be routed through a different path. For example, a text e-mail message may be routed from the messaging device 105(1) through the data network 120 to an e-mail server and short messaging service at the MSO 125. From the MSO 125, the visual e-mail text message is transmitted to the mobile telephone 145. The e-mail message is viewed by the user and the user speaks a voice reply message to be delivered back to the message originator at the messaging device 105(1). The voice message is then routed to the messaging device 105(1) through one of, or a variation of, the routes previously described. The system operator is given the option to choose a two-way messaging system for voice and visual messaging that utilizes the most efficient delivery path for routing messages depending on the type of message to be delivered or received (i.e. audio or visual). An MSO 125 or PSO 130 may now utilize the data network 120 for transporting voice messages.

Prior art systems currently exist for sending e-mail text messages from a computer over the Internet to a mobile communication device. The present invention enables a mobile communication device to send a voice reply message over the Internet (or other data network) to the user that originated the e-mail text message. This is a highly desirable feature. For example, consider a mobile device user who is driving a car and receives an e-mail message to which a reply is urgently required. Such a user is unable to safely use a keyboard to enter a text reply message, but can easily create a voice reply message while keeping his or her hands on the wheel.

Referring to FIG. 3, a mobile communication device, such as a mobile telephone 145 or a paging transceiver 140, according to one embodiment is shown in block form. An antenna 351 connects to an antenna interface 352. The antenna interface 352 may be a duplexer or equivalent device depending on the type of mobile communication device desired. An antenna switch, for example, may be utilized for configuration as a paging transceiver 140 or two antennas might be used, thereby eliminating the requirement for a duplexer. A receiver 353 and a transmitter 354 are connected to the antenna interface 352 and processing module 355. The processing module 355 includes a microprocessor, RAM and ROM memory, switching, and I/O ports.

The necessary software instructions and operating system for enabling mobile telephone 145 or paging transceiver 140 to receive visual messages are included and well known to those skilled in the art of mobile telephone and paging systems. Mobile telephones for two-way voice communications are commonly available today. Many of these mobile telephones receive and display visual messages such as text messages. Generally, this service is called Short Messaging Service or SMS.

One format for receiving SMS is known as Cellular Digital Packet Data or CDPD. There are many variations for text messaging in mobile telephones and there are many formats in which the text messages may be transmitted. A few mobile telephone types that receive text messaging are CDMA (Code Division Multiple Access), TDMA (Time Division Multiple Access), and GSM (Global System for Mobile Communications). There are also a plurality of languages and protocols for enabling a wireless mobile telephone to communicate over the Internet. A few of them are HDML (Handheld Device Mark up Language), HDTP (Handheld Device Transport Protocol), TTML (Tagged Text Mark up Language) and WAP (Wireless Application Protocol).

There are many prior art configurations for receiving visual messages by mobile telephone 145 or paging transceiver 140, and all such configurations are herein contemplated for use in combination with the novel features of the present invention. In short, it is widely known in the art of wireless paging systems and mobile telephone systems that paging transceivers and mobile telephones receive visual messages.

The process by which a visual message, such as text, is received is not critical. However, the novel system and method for processing the received message information in connection with transmitting a voice reply message will be described in detail hereinafter.

Referring again to FIG. 3, a user interface 356 connects to the processing module 355. The user interface 356 comprises all of the input and output devices for interaction with the mobile communication device such as speaker, alerting transducer, microphone, liquid crystal display (LCD) or light emitting diode (LED) display, switches, etc. These devices are normally included with a conventional mobile telephone having short messaging service features. In the case of the paging transceiver 140, a microphone is included in the user interface 356 for enabling speech input because a conventional paging transceiver for visual messaging does not have a microphone.

The antenna 351, antenna interface 352, receiver 353, transmitter 354, processing module 355 and user interface 356 are connected in a typical mobile telephone or paging transceiver configuration. A controller 357 and memory 358 have been included for processing of received visual messages, storage of visual messages, and processing of transmitted voice messages. The controller 357 comprises a conventional microprocessor of the type commonly used in mobile telephones and paging transceivers. The controller 357 also includes a memory manager, I/O ports, RAM and ROM memory and all necessary software instructions required to communicate with the processing module 355, user interface 356, and memory 358. The controller 357 connects to the processing module 355 for receiving and sending data, messages, and commands. The controller 357 is connected to memory 358 for storage and retrieval of messages and message data and to the user interface 356 for input and output interfacing with a user.

Continuing with FIG. 3, when a visual message is received by receiver 353, it is stored for reproduction by the processing module 355 in a conventional manner. The visual message may contain addressing data, identification data or other data corresponding to the visual message which is also stored. After the visual message is received, an alert may be generated for notifying a user that a message has been received. The user may then select a display message option provided by the user interface 356 to cause the stored visual message to be displayed on the user interface 356 LCD or LED display.

When a user wishes to send a voice message in reply to a received visual message, the user enables a voice reply mode via user interface 356. This causes the controller 357 to read the necessary addressing information from the processing module 355 and memory 358 for addressing and sending the voice reply message to the originator of the received message. Optionally, the user may enter addressing information associated with the received visual message manually using a keypad, as previously described. After enabling the voice reply mode, the user speaks into the microphone of the user interface 356, and a real-time voice message is transmitted to the IMG 150 for forwarding, in non real-time, to the visual message originator (e.g., messaging device 105). In an alternative embodiment of the present invention, a pre-recorded voice message is transmitted to the IMG 150 as will be described in detail hereinafter.

When a mobile communication device receives a visual message, the mobile communication device also receives and stores messaging data associated with the visual message. This messaging data may include, but is not limited to: a reply name, a reply address, a reply code, a reply type code, and reply subject matter.

The reply name is generally the name of the person who sent the visual message. The reply address may be an e-mail address such as “jsmith@hotmail.com” or an abbreviated address such as a sequence of letters and/or numbers that is associated with an e-mail address stored at the IMG 150. The reply address may also include an IMG 150 system identifier or e-mail system identifier or a telephone number to a particular IMG 150. The reply code is a code that corresponds to the original visual message stored at the MSO's 125 or PSO's 130 e-mail system or at IMG 150. The reply type code indicates the type of voice message that is allowable by the system (i.e. pre-recorded, real-time, analog, digital, format, etc.). The type code may also indicate the allowable length or size of a voice message. The reply subject matter may contain the original text subject matter of the received visual message.

A complete system according to the preferred embodiment of the present invention will now be described in connection with the mobile telephone 145 and the MSO 125 system of FIG. 2B. The mobile telephone 145 is enabled to receive visual messages from the MSO 125 using any number of methods previously described. The system may operate in any number of formats including CDMA, TDMA, GSM, etc. Short messaging service (SMS) for sending text messages to a mobile telephone is well known to those skilled in the art of mobile telephone systems. Other methods for delivering visual messages are also very common. For example, a modem could be added to an analog mobile telephone or tone signaling could be used for transmitting text from the MSO 125 to the mobile telephone 145. The use of CDPD is also quite common in the art of mobile telephone systems.

In order to send a visual message from the messaging device 105 to the mobile telephone 145, a user of messaging device 105 creates an e-mail message using an e-mail program, such as one provided from AOL, Lotus, Netscape or Microsoft. The e-mail message is sent from the messaging device 105 to the MSO 125 via data network 120. An e-mail gateway (EMG) 115 is configured at the MSO 125 for receiving the e-mail message for delivery to the mobile telephone 145. Such a configuration is well known to those skilled in the art of mobile telephone systems.

After the e-mail message is received by the EMG 115, the MSO 125 transmits a signal to the mobile telephone 145. The signal includes the e-mail message and messaging data associated with the e-mail message. For this example, the signal consists of a reply name of “John Doe,” a reply code, and the e-mail message, as shown below:

- From: John Doe
- To: Mary Jane
- Subject: Airport Delay
- Reply code: 12
- Mary,
  - My Flight is Delayed. I now arrive at 10:00 p.m., Flt. 2216.
    
    The reply code is a code that is associated with the e-mail message stored in EMG 115. The reply code may be any sequence of numbers or characters, such as: 12 or 123ABC. Each message stored in the EMG 115 has a unique reply code associated with it.

The antenna 351 of mobile telephone 145 receives the signal sent from the MSO 125. The receiver 353 demodulates the signal to recover the e-mail message and message data contained in the signal. The processing module 355 stores the e-mail message and message data in memory and causes the user interface 356 to alert the user that a message has arrived.

Upon being alerted, the user may activate the user interface 356 to cause the processor 355 to read the stored visual message and associated messaging data from the memory and display it on a display screen for viewing by the user. The time at which the message was sent or delivered may also be displayed on the display screen. Additionally, a reply indicator, which is triggered by the reply code and gives an indication to the user that the visual message selected is one in which a voice reply may be sent, is displayed. After viewing the displayed information, the user makes a decision to send a voice reply message to the originator by selecting a reply option provided by the user interface.

Referring now to FIG. 4, there is illustrated a process performed by the controller 357 for sending a voice reply message. The process begins with step 402, where the controller 357 receives an indication that the user has selected a reply option from the user interface 356 and, thus, enters a reply mode. Flow proceeds to step 404, where the controller 357 reads the memory from processing module 355 for messaging data corresponding to the visual message to which the user desires to reply. A pre-programmed telephone number to the MSO 125 for connection to the IMG 150 is also read from memory 358 or obtained from the messaging data. The messaging data may or may not include a reply code. If the messaging data does not include a reply code, the process proceeds to step 406, otherwise the process proceeds to step 412.

In step 406, the controller 357 searches an electronic address book (hereafter “address book”) stored in memory 358 for an address associated with the reply name that was included in the messaging data. If the controller 357 does not find such an address, then flow proceeds to step 410, otherwise flow proceeds to step 412. In step 410, the user is informed via a visual or audible indicator that there is no return address and the process ends.

It should be noted that the reply code search at step 404 was for a code that corresponds to the visual message stored at the MSO 125. If the reply code does not exist, the reply name included in the messaging data is used to search for a corresponding address. As previously described, the reply code might simply be the sequence “12”. Reply codes reduce air time because all of the necessary addressing information is stored at the MSO 125 and the reply code points to that data.

In step 412, the controller 357 of mobile telephone 145 attempts to establish communications with the IMG 150 at the MSO 125 by sending the pre-programmed telephone number read from the memory 358 or obtained from the messaging data. This attempt is automatically accomplished by the controller 357. After step 412, flow proceeds to step 414, where a determination is made to see if communication is established. If communication is not established, flow proceeds to step 416, otherwise flow proceeds to step 418.

In step 416, the user is informed that a connection could not be established and the process ends. If this occurs, the user can simply attempt to send the reply message at a later time. In step 418, the mobile telephone transmits addressing data to the IMG 150, and the IMG 150 transmits acceptance or error codes back to the mobile telephone 145. The addressing data either includes the reply code or the address determined in step 406.

After step 418, flow proceeds to step 420, where the controller 357 checks for “invalid data” error messages sent from the IMG 150. If an invalid data error is detected, the controller 357 tags the invalid addressing data stored in the memory 358 as invalid and reads the memory 358 for valid messaging data in order to correct the problem. For example, if the mobile telephone 145 sends a reply code associated with a message that is no longer available to the IMG 150, the IMG 150 will send an error code to the mobile telephone 145 indicating that the message can not be replied to using the current reply code.

When the controller receives this error code it reads the memory 358, to see if there is a name and address for the recipient stored in memory 358 (step 422). If yes, at step 428, flow proceeds to step 418 and the process continues. If the controller 357 does not find valid addressing data, flow proceeds to step 430 where the user is informed of the problem via an indicator and the process ends.

If at step 420 it is determined that the addressing data is valid, flow proceeds to the message record process at step 424. At step 424, the mobile telephone 145 user is informed by the IMG 150 to begin speaking a message at the tone for the designated recipient. The mobile telephone user may also be informed via an LCD or any other means of indication to the user to begin speaking. The user may also be informed as to how much time the user may speak a voice message. For example, “begin speaking a 30 second message for Mary Jane at the tone.” However, a preferred embodiment provides for a more efficient means for notifying a user of the amount of available record time. As previously discussed, a reply type code can be included in the messaging data transmitted with the visual message. The reply type code can include a length code indicating the amount of time allowable for a reply message. This feature can be very useful for a mobile telephone 145 user, in that the user is informed at the time the visual message is viewed, of the amount of time given to reply. This advanced notice gives the user time to compose a reply message in advance as opposed to finding out two seconds before recording starts.

At step 424, the voice message is recorded by the IMG 150 in a manner consistent with conventional voice mail systems well known to those skilled in the art of voice mail systems. For example, the user may be given the option to review the message, delete the message, re-record the message, etc. When the user is finished recording, flow proceeds to step 426, where the IMG 150 notifies the mobile telephone 145 user that the voice message has been accepted and or sent and the process ends.

As illustrated by the above process, a voice message may be sent as a reply message to a visual message. A number of schemes may be used to send the voice reply message to a messaging device 105 so that the messaging device 105 user knows that the voice message is a voice reply to a particular visual message. In one scheme, the mobile telephone 145 simply transmits a reply code to the IMG 150. The reply code corresponds to a particular visual message available to the IMG 150. For example, the visual message could be stored within the IMG 150, MSO 125 or EMG 115. When the IMG 150 formats the voice reply message, the IMG 150 can include in the reply message the particular visual message associated with the reply code.

In another scheme, if the visual message is not available to the IMG 150, the mobile telephone 145 searches its memory 358 for a name or address. If found, the name or address, together with the received visual message or part of the received visual message, is transmitted along with the voice message to the IMG 150, which will then create an e-mail message containing the voice message and the received visual message or part thereof. The name or address sent to the IMG 150 is used by the IMG 150 to address the e-mail message.

In short, if the visual message is available to the IMG 150, and the messaging data transmitted to the mobile telephone 145 includes a reply code, then there is no need for the mobile telephone 145 to transmit to the IMG 150 anything other than the voice message and the reply code, thereby saving valuable bandwidth.

Voice messages that are not reply messages may also be initiated by the mobile telephone 145. The mobile telephone user may simply select a pre-stored name or address from a name and address book (also referred to as, “send message list”) stored in memory 358. When the name appears on the display screen, the user simply uses the user interface to select a send message function and the process starts at step 412, FIG. 4. The stored name or address is utilized for addressing the voice message and the process proceeds as previously described.

Voice messages transmitted from the mobile telephone 145 may be analog or digital. If analog transmission is utilized, the IMG 150 converts the analog voice message to digital data representative of the voice message for storage and transmission to the messaging device 105. If the voice message is in digital format, the digital data representative of the voice message may be stored directly by the IMG 150.

Additionally, a voice message may be pre-recorded and stored in memory 358 for transmission to the IMG 150. In such a configuration, a digital signal processor and associated analog to digital converter may also be configured with the controller 357 in a conventional manner for recording voice messages and storing the voice messages as data in memory 358. If the stored voice message is to be transmitted in analog format, then a digital to analog converter may also be utilized for converting the voice message data stored in memory 358 back to analog. For a pre-recorded voice message configuration, step 424 of FIG. 4 would be changed to: Recall the stored message from memory and transmit with associated addressing data. Optionally, the stored voice message my be compressed by conventional compression algorithms prior to being transmitted.

In connection with the paging transceiver 140, it should be noted that a telephone call is not normally initiated between a paging transceiver and the pager switching office (PSO) 300. In the case of paging transceiver 140 sending voice messages to the PSO 130, a pre-recorded voice message, as opposed to a real-time voice message, may be sent as previously described. However, at step 412, a telephone connection is not established. Step 412 would be changed to: Page PSO 130 utilizing a conventional voice paging protocol such as Inflection, pACT, etc. Step 414 would be changed to: Did the PSO 130 respond to the selective call (i.e. page and handshake). Finally, step 424 would be changed to: Transmit the pre-recorded voice message using a paging protocol. Optionally, prior to transmission, the pre-recorded voice message can be compressed using conventional compression algorithms to decrease air time.

Referring now to FIG. 5, one embodiment of the IMG 150 and EMG 115 are shown. The IMG 150 comprises a voice mail system (VMS) 510 networked to a voice mail gateway (VMG) 520. The IMG 150 may also include a mail server 530(1). The EMG 115 includes a short messaging service (SMS) 532 and an e-mail conversion system (ECS) 531. The EMG 115 may also include a mail server 530(2). Optionally, the EMG 115 and IMG 150 can share the same mail server 530.

The mail server 530(2) is coupled to data network 120 and receives visual messages sent from messaging device 105. The ECS 531 retrieves visual messages from the mail server 530 and reformats the messages for wireless transmission. The ECS 531 sends the reformatted messages to the SMS 532 where it is queued and sent to the MSO 125 for transmission to a targeted mobile telephone 145. There are many formats and systems available for delivering e-mail messages to the MSO 125 or PSO 130 for transmission to a mobile telephone 145 or paging transceiver 140, respectively. These systems are well known to those skilled in the art of wireless messaging. Other than the improvement described hereinafter, these systems are readily available.

The VMS 510 is connected to the MSO 125 so that voice messages sent from mobile telephone 145 can be received and stored by the VMS 510. The VMS can be directly connected to the MSO 125 (as shown in FIG. 2B) or the VMS 510 can be connected to the MS0125 through the PSTN 110.

The VMS 510 is an enhanced voice mail system. Voice mail systems are readily available from manufacturers such as Centigram Communications Corporation in San Jose, Calif., Nortel Networks in Santa Clara, Calif. and AVT in Kirkland, Wash., to name a few. The VMG 520 is an enhanced voice mail gateway. Voice mail gateway systems are also readily available from the above manufacturers. Voice mail systems and voice mail gateways are generally software driven and adaptable to various messaging environments. These systems are easily networked for communication between different manufacturer's platforms. Additionally, these systems have evolved to the point that many have the capability of sharing messages and data between platforms. One such system manufacturer is Data Connection Limited in Enfield, England. Data Connection Limited manufactures voice mail systems, voice mail gateways and networking systems which will communicate utilizing a protocol called “Voice Profile for Internet Mail” (VPIM). The VPIM protocol is a common messaging language for the transport of voice messages between platforms. VPIM additionally allows for the sending of voice messages from a computer or voice mail system to a voice mail system or computer via the Internet. Other than the improvements described hereinafter, voice mail systems, voice mail gateways, voice mail networks, mail servers, e-mail gateways, short messaging service systems, MSO systems, protocols for transmitting voice messages over the Internet and protocols for transmitting e-mail messages to a wireless transceiver are well known to those skilled in the art of these systems and are readily available.

FIGS. 6A-6C illustrate a process for receiving voice messages from a mobile telephone 145 or paging transceiver 140 performed by the IMG 150. Referring to FIG. 6A, the process begins in step 600, where the IMG 150 is placed in call stand-by mode where it awaits a call. When a call is received from a mobile telephone 145, the VMS 510 receives from the MSO 125 subscriber identification data corresponding to the mobile telephone 145 which placed the call (step 602). The subscriber identification data may include the phone number or identification of the mobile telephone 145. The name of the subscriber and the subscriber's messaging address, such as an e-mail address, etc., can be found by the MSO 125 by cross indexing the subscriber identification data.

After step 602, flow proceeds to step 604, where the VMS 510 accepts or rejects the call based on the subscriber identification data. If the call is rejected, flow proceeds to step 606, otherwise flow proceeds to step 608. In step 606 the call is terminated, an error message is transmitted to the mobile telephone 145, and flow returns to step 600.

In step 608, the VMS 510 receives data from the mobile telephone 145. The data may include a request code. The request code indicates the type of action requested. For example, a request code of “01” indicates that the current request is for sending a general message, and a request code “02” indicates that the current request is for sending a reply message. Request codes may also be used for forwarding, cc, bcc, etc. The data may also include addressing information such as an e-mail address, an abbreviated e-mail address, a name, subject matter, type, a reply code, a coded address, etc.

In a preferred embodiment, the present invention utilizes messaging data codes whenever possible in order to conserve transmission time. When a mobile telephone 145 user wishes to originate a message (as opposed to reply to a message), the user selects the name of the person to whom the message is to be sent from an address book stored in the telephone 145. When the user activates the send message command after selecting a recipient, only the message to be forwarded to the recipient, a request code, and a coded address normally needs to be transmitted to the IMG 150.

The request code directs the VMS 510 to perform an action, in this case, the example is to send a message. The coded address corresponds to all other information required to send the message to the recipient, such as the intended recipient's name, e-mail address, message type code, etc. To send a reply message, only the request code and a reply code need be sent together with the reply message.

It should be noted that there are many methods contemplated that may be used for finding information stored at the MSO 125 or IMG 150. A mobile telephone 145 may, for example, transmit the name of the message recipient. The VMS 510, may utilize the received name for looking up the associated address stored in a database associated with the IMG 150 in order to reduce the amount of transmission time required by the mobile telephone 145. The mobile telephone 145 according to the preferred embodiment of the present invention does not transmit the actual addressing information if that information is otherwise available to the IMG 150 or associated interconnected systems via a coded address or the like.

Referring back to FIG. 6A, after step 608, flow proceeds to step 610, where the VMS 510 determines the request based on the received request code. The request may be to forward a message, reply to a message, send a general message, send a cc, send a bcc, etc. For the purpose of illustration, the decision step at step 610 has been limited to send a general message or send a reply message. The process of sending a copy or forwarding a message, etc. may be readily implemented in a manner consistent with the process described in detail hereinafter. If at step 610, it is determined that the request is for sending a general voice message, flow proceeds to step 616, otherwise a reply message is assumed and flow proceeds to step 640.

Referring to FIG. 6B, at step 616, the VMS 510 determines if the message to be sent is completely addressed or if the VMS 510 received a coded address, which needs to be decoded or cross referenced. If the message to be sent is completely addressed, flow proceeds to the record voice message process, step 626. A completely addressed message would include information pertaining to whom the message is from. This information was acquired at step 602 from the MSO 125, as previously described. A completely addressed message also includes the name and e-mail address of the recipient such, as: Bill Jones, “Bjones@hotmail.com”.

It is beneficial for the mobile telephone to transmit a complete recipient name and address when the mobile telephone 145 may be roaming in another system area where the IMG 150 does not have a corresponding address book. If at step 616 the VMS 510 determines that coded data representing a recipient was received (i.e. coded address), flow proceeds to step 618. At step 618, the VMS 510 searches an address book stored in its database for a corresponding match. The VMS 510 may alternatively utilize an algorithm for converting the code to an address or name.

After step 618, flow proceeds to step 620, where the VMS 510 determines if the recipient address can be produced from the coded information or found in the VMS 510 address book. If at step 620 an address cannot be obtained, flow proceeds to step 622, where an error is transmitted to the mobile telephone 145, the connection is terminated and the process is returned. If an address is obtained at step 620 flow proceeds to step 626.

Step 626 is a voice recording process performed by the VMS 510. If a real-time voice message is to be received by the VMS 510, then voice prompts are returned to the mobile telephone 145 (e.g., “start recording at the tone”). The VMS 510 allows a caller to review, re-record, append, erase, etc., messages in a manner consistent with typical voice mail systems. If, on the other hand, a non-real-time message is to be received, then voice prompts are not returned. A record type indicator is transmitted from the mobile telephone 145 at the beginning of step 626 indicating the desired recording format (i.e., real-time vs. pre-recorded). A paging transceiver 140 may, for example, utilize the pre-recorded format while a mobile telephone 145 may utilize a real-time recording format.

At step 626, the voice message is received from the mobile telephone 145 and recorded by the VMS 510. Recording stops after an assigned amount of time or after the VMS 510 receives a stop record command from the mobile telephone 145. Recording may also be terminated by the VMS 510 responsive to a noisy communications connection. When the recording is complete, the VMS 510 sends an acknowledgment to the mobile telephone 145 and the communication is terminated (step 628). Flow then proceeds to step 630, where the VMS 510 transfers the voice message and the necessary addressing information to the VMG 520.

The VMG 520 converts the voice message to a conventional audio file format suitable for transmission over the Internet 120 and reproduction by the destination messaging device 105. An example of a conventional audio file format is the “.wav” format developed by Microsoft. The VMG 520 also creates an e-mail message comprising the converted voice message and uses the addressing information to address the e-mail. The converted voice message can be stored in a file and attached to the e-mail message.

The addressed e-mail message is then forwarded to a mail server such as the mail server 530 (step 632). The VMG 520 preferably inserts the words “voice message” in the subject line of the e-mail and inserts instructions for playing the message in the body of the e-mail. It is preferable that the e-mail message be given a priority level equal to primary mail so that it will not be inadvertently filtered by a recipient's e-mail system that limits attachments or treats attachments as secondary mail. After the addressed e-mail message is delivered to the mail server 530, the process returns to step 600. The mail server 530 is responsible for sending the addressed e-mail message to the intended recipient.

Referring back to FIG. 6A, at step 610, if the request is for a reply, flow proceeds to step 640. Referring you to FIG. 6C, at step 640, the VMS 510 searches for the original message and addressing information files for sending a reply message. This information is located by correlating a received reply code with an archived original message file. The original visual message data may be archived on the mail server 530, the VMG 520, the VMS 510 or any other associated device. For this description, the archived e-mail visual message file is stored in an archive file on the mail server 530 for a 5 day period.

After step 640, flow proceeds to step 642, where the VMS 510 determines if the reply information corresponding to the received reply code is present in an archive file. If the reply information is not present, then flow proceeds to step 644, otherwise flow proceeds to step 658.

In step 644, the VMS 510 requests reply addressing information from the mobile telephone 145. Flow then proceeds to step 646, where the VMS 510 analyzes the information received from the mobile telephone 145. If the reply addressing information is complete, flow proceeds to the record process (step 658). If the reply information is coded, flow proceeds to step 648 where the address and or name is calculated or found, as previously described. Flow then proceeds to step 650 where the VMS 510 makes a determination as to the validity of received data. If no addressing data was received or if the data received was not valid, flow proceeds to step 652, where an error code is returned to the mobile telephone 145, communication is terminated, and the process returns to step 600. If at step 650 it is determined that the data is valid, flow proceeds to step 658.

At step 658, if the process flow is from step 642, the “reply to” name, address, subject matter and text are obtained from the e-mail archive at the mail server 530 via the VMG 520 and appended to the voice message by the VMS 510. The “mailed from” information, as previously described, is obtained by the VMS 510 when communication is established by cross indexing the subscriber identification data with the subscriber data base associated with the VMS 510. At step 658, if the process flow is from step 650 or step 646, a “reply to” name, address and subject matter including “mail from” information is appended or attached to the voice message by the VMS 510.

It should be noted that the preferred method for receiving reply information from a mobile telephone 145 is to receive a reply code in order to conserve transmission time. The original message file stored in the archive at the mail server 530 provides all of the necessary information required to send a reply. The next preferred method for receiving reply information from the mobile telephone 145 is to send a coded address and subject matter. The coded address may then be correlated with the subscribers mailing list stored at the VMS 510 in order to produce the name and/or address. The least favorable method is to receive from the mobile telephone 145 the complete name, complete address and subject matter.

During step 658, the voice message is recorded by the VMS 510 in the same manner as described with reference to step 626. After step 658, flow proceeds to step 660 where the call is terminated. The VMS 510 then sends to the VMG 520 the recorded voice message, addressing data, and a pointer to or the actual visual message to which the voice message is a reply (step 662). The VMG 520 converts the recorded voice message to an acceptable Internet and messaging device 105 format and packages the reply voice message with the original visual message for transmission. Flow proceeds to step 664, where the VMG 520 forwards the packaged addressed message (i.e. audio and visual) to the mail server 530 for transmission to the messaging device 105, as previously described, and the process returns to step 600.

FIG. 7 illustrates an example of a visual message 700 received and displayed by a messaging device 105. The visual message 700 includes a voice message in the “.wav” format attached thereto. Because the visual message 700 includes a voice message as an attachment the visual message 700 is referred to as an audio/visual message. FIG. 8 illustrates an example of a reply audio/visual message 800 received and displayed by a messaging device 105 including the original visual message sent to the mobile telephone 145. If the original message is no longer archived (as previously described) or available to the IMG 150, the original message would not be included. However, the subject: Re: Airport Delay, would still be included.

Referring once again to FIG. 3, the mobile telephone 145 or paging transceiver 140 can store names and addresses or abbreviated addresses and coded addresses in an address book stored in memory 358. When the mobile telephone 145 receives a visual message, the user may select an “add to list” feature from the user interface 356, causing the controller 357 to add to the address book the name and actual address of the individual who sent the message, along with an abbreviated or coded address corresponding to the actual address.

As previously described, it is preferable that the mobile telephone 145 transmit coded address information representative of an actual address stored in the IMG 150. However, there may be times, such as when the mobile telephone is roaming in another IMG 150 system area, when it is necessary to transmit an un-coded name and address. In order to solve this problem, the mobile telephone 145, according to one embodiment, may store both the complete name and address and a coded address which corresponds to the complete address information stored at the IMG 150.

When the IMG 150 sends a visual message to the mobile telephone 145, complete addressing information is also sent if the coded address is not already stored in memory 358 of the mobile telephone 145. In other words, the IMG 150 need not transmit information to the mobile telephone 145, if the information is already stored there. When a visual message is transmitted to the mobile telephone 145, the IMG 150 first sends a coded address to the mobile telephone 145. The controller 357 then searches for corresponding complete addressing information stored in memory 358. If a match is found, the mobile telephone 145 indicates to the IMG 150 that a match was found. The IMG 150 then does not need to transmit the actual data. It is a simple matter for the controller 357 to insert the corresponding name and address in the proper place within the message indicating from whom the message was sent. If the complete address information including the name is not stored in the memory 358, the mobile telephone 145 indicates to the IMG 150 that a match was not found. The IMG 150 then transmits the complete information to the mobile telephone 145 for storage.

When the mobile telephone 145 receives a visual message from the IMG 150, the message is stored as previously described for viewing on an LCD type display. When a mobile telephone 145 user views the message, the user may elect to save the name and address. The name and address may be saved in memory 358 by selecting the “add to list” option on the user interface which causes the controller 357 to store the address information in the address book in memory 358 for addressing out-going voice messages.

The first time that the mobile telephone 145 transmits the saved address information to the IMG 150 (i.e. when sending a message), the IMG 150 will issue an associated coded address to the mobile telephone 145 for storage in memory 358. The mobile telephone 145 then adds the coded address to the entry in its address book associated with the stored address. The IMG 150 adds the address information and coded address to its address book. In this manner, the mobile telephone 145 need only transmit the complete address information one time. Thereafter, only the coded address need be transmitted.

A mobile telephone subscriber may also add an address to the mobile telephone 145 address book using a messaging device 105. The subscriber may simply send an “address list message” to their own mobile telephone 145. An “address list message” is a visual message having a predetermined format and including a list of names and corresponding addresses. One example of an address list message is an e-mail message wherein the body of the e-mail includes a list of address book entries, wherein each entry includes a name and at least one corresponding address. FIG. 9 illustrates one embodiment of an address list message. When an address list message is received by the IMG 150, the IMG 150 detects that the message is an address list message based on its format and reads the names and address, and transmits the names, addresses and corresponding coded address information to the mobile telephone 145 for addition to the mobile telephone 145 address book. The IMG 150 also adds the same names, addresses and corresponding coded addresses to its address book. In an alternative embodiment, the address list message is sent to the mobile telephone 145. Upon the address list message being received, the controller 357 parses the message to obtain the names and corresponding addresses to be added to the address book. After obtaining the names and corresponding addresses, the controller adds them to the address book.

The integrated voice and visual messaging system described and illustrated herein is readily adaptable to a plurality of messaging formats, protocols, modulation schemes and system configurations. Voice messages may be transmitted from a mobile communication device such as a paging transceiver 140 or a mobile telephone 145 to an integrated message gateway for forwarding over the Internet or other type of network to a personal computer or other visual messaging device. Voice messages may be transmitted to the integrated mail gateway in analog or digital format. Additionally, voice messages may be pre-recorded at the mobile communication device for non real-time transmission or real-time voice messages may be transmitted to the integrated mail gateway for forwarding in non real-time. The system is adaptable to various wireless telephone systems and paging systems. The mobile communication device may be configured in a mobile telephone, pager, wireless PDA, or other wireless device which provides visual messaging and includes means for voice communications.

While various embodiments of the present invention have been described above, it should be understood that they have been presented by way of example only, and not limitation. Thus, the breadth and scope of the present invention should not be limited by any of the above-described exemplary embodiments, but should be defined only in accordance with the following claims and their equivalents.

	Number	Date	Country
	60126939	Mar 1999	US
	60155055	Sep 1999	US

	Number	Date	Country
Parent	09408841	Sep 1999	US
Child	10958731	Oct 2004	US

System and method for integrating audio and visual messaging

Information

Publication Number

Date Filed

Date Published

Inventors

Original Assignees

CPC

US Classifications

International Classifications

Abstract

Description

Claims

CROSS REFERENCE TO RELATED APPLICATIONS

Provisional Applications (2)

Continuations (1)