Patent Grant
9100200
The present invention relates to near real-time communication in one medium, which is augmented with a real-time or near real-time communication in another medium. More particularly but not exclusively, the present invention relates to text messaging, also known as text chat and/or instant messaging, which is augmented by video streaming calling.
Text messaging, instant messaging and text chat are known in the art and is a very popular communication medium for both desktop computers and mobile terminals. Video calling and similar communication technologies are also well known in the art. The combination of text and video as well as multimedia messaging are also known and used.
Methods for creating messages of various media types are known, including automatic or semiautomatic triggering of message creation. Such methods usually invoke a message of the type associated with the triggering event. For example, keystrokes may trigger a text message while sound may trigger a voice message. U.S. patent application 20090286515 is believed to represent the state of the art in this respect.
The limitation on all types of combination of text and video, or another media combination, is that the combined media communication should be predetermined and prearranged to take place.
There is thus a widely recognized need for, and it would be highly advantageous to have, a method and a system for devoid of the above limitations.
According to one aspect of the present invention there is provided a method for a video augmented text chat session including the initiation of a video call session between at least two parties while the parties are engaged in a text chat session. The method includes the steps of: reaching consent between the parties to engage in a video session while in a text chat session; identifying user activity associated with the text chat session; automatically activating a video call session with the other party while the user is actively associated with the text chat session.
According to another aspect of the present invention there is provided a method for a video augmented text chat session where the step of activating the video conference session is executed when both the parties are actively associated with the text chat session.
According to another aspect of the present invention there is provided a method for a video augmented text chat session while the parties are engaged in a text chat session, the method including:
According to yet another aspect of the present invention there is provided a wireless communication device for a video augmented text chat session including: a display, a front camera, and means for entering text; a wireless communication unit operative to communicate text transmissions and video transmissions; a processor for executing a text communication module and a video communication module, where the text communication module being operative to communicate the text transmissions with a user via a first part of the display and the video communication module being operative to communicate the video transmissions with a user via a second part of the display; where the video communication module being operative to automatically display video transmission received from a second terminal when the user is engaged in text chat communication with the second terminal.
According to still another aspect of the present invention there is provided a wireless communication device for a video augmented text chat session where the processor is operative to activate the video communication module automatically when the user is engaged in text communication with a second user with whom video over text messaging was initiated in the past.
Further according to another aspect of the present invention there is provided a wireless communication device for a video augmented text chat session where the engagement in text communication includes at least one of: detecting the user moving the terminal; detecting the user using the terminal to enter text; detecting the user's face in a picture taken by the camera; and detecting a terminal wakeup.
Still further according to another aspect of the present invention there is provided a wireless communication device for a video augmented text chat session additionally including storage operative to store video and where the video communication module is operative to: store the video transmission received from the second terminal; and retrieve and display the stored video when the user is engaged in text communication with the second terminal.
Even further according to another aspect of the present invention there is provided a wireless communication device for a video augmented text chat session additionally including an icon, displayable on the display, representing at least one of: a device being capable of video augmented text session; and a device being engaged in a video augmented text session.
Additionally according to another aspect of the present invention there is provided a wireless communication device for a video augmented text chat session where the icon represents a status associated with the device and another device with which the device is currently communicating.
Also according to another aspect of the present invention there is provided a server operative to store and forward video transmissions, the server communicating with two the wireless communication device of claims and operative to receive the video transmission for one of the wireless communication device and to forward the video transmission to the other wireless communication device upon demand.
Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The materials, methods, and examples provided herein are illustrative only and not intended to be limiting.
Implementation of the method and system of the present invention involves performing or completing certain selected tasks or steps manually, automatically, or a combination thereof. Moreover, according to actual instrumentation and equipment of preferred embodiments of the method and system of the present invention, several selected steps could be implemented by hardware or by software on any operating system of any firmware or a combination thereof. For example, as hardware, selected steps of the invention could be implemented as a chip or a circuit. As software, selected steps of the invention could be implemented as a plurality of software instructions being executed by a computer using any suitable operating system. In any case, selected steps of the method and system of the invention could be described as being performed by a data processor, such as a computing platform for executing a plurality of instructions.
The invention is herein described, by way of example only, with reference to the accompanying drawings. With specific reference now to the drawings in detail, it is stressed that the particulars shown are by way of example and for purposes of illustrative discussion of the preferred embodiments of the present invention only, and are presented in order to provide what is believed to be the most useful and readily understood description of the principles and conceptual aspects of the invention. In this regard, no attempt is made to show structural details of the invention in more detail than is necessary for a fundamental understanding of the invention, the description taken with the drawings making apparent to those skilled in the art how the several forms of the invention may be embodied in practice.
In the drawings:
The principles and operation of video augmented text messaging according to the present invention may be better understood with reference to the drawings and accompanying description.
Before explaining at least one embodiment of the invention in detail, it is to be understood that the invention is not limited in its application to the details of construction and the arrangement of the components set forth in the following description or illustrated in the drawings. The invention is capable of other embodiments or of being practiced or carried out in various ways. In addition, it is to be understood that the phraseology and terminology employed herein is for the purpose of description and should not be regarded as limiting.
In this document, an element of a drawing that is not described within the scope of the drawing and is labeled with a numeral that has been described in a previous drawing has the same use and description as in the previous drawings. Similarly, an element that is identified in the text by a numeral that does not appear in the drawing described by the text has the same use and description as in the previous drawings where it was described.
The present invention relates to near real-time communication in one medium, which is augmented with a real-time or near real-time communication in another medium. More particularly but not exclusively, the present invention relates to text messaging, also known as text chat and/or instant messaging, which is augmented by video streaming or video messaging. However, other combinations of media are also contemplated. Particularly, the initiation of the augmenting medium, which is the video media in this case, is triggered by the augmented medium, which in this case is the text messaging medium. Further, the exchange of the augmenting medium, which is the video media in this case, is conditioned on prior agreement between the communicating parties. Specifically, if the communicating parties have previously agreed to augment their communication, the system automatically invokes the augmenting medium when the augmented medium is triggered.
Text messaging, text chat and instant messaging may be used interchangeably unless the difference is described for a particular issue. Similarly, video call, video streaming, video messaging or video session may be used interchangeably unless the difference is described for a particular issue.
Reference is now made to
As shown in
Communication networks 11 can be telephone networks (e.g. PSTN) cellular networks (e.g. PLMN, GPRS, 3G, 4G, etc.), WiMAX networks, satellite communication networks, WLAN (e.g. WiFi), PAN (e.g. Bluetooth), MAN, WAN, computer communication network (e.g. Ethernet, IP, etc.) the Internet, etc.
Terminals 12 are preferably capable of communicating in at least two media. Particularly, the terminals 12 are capable of instant messaging and video. Preferably, the terminals 12 are equipped with video cameras 19, preferably mounted at the front of the terminals 12 to capture video images of their respective users 13.
As shown in
It is appreciated that the text transmission 20 and the video transmission 21 are sent as independently media, but are associated with each other, and also associated with and/or triggered by the user's activity and/or engagement in one of the media, and particularly the creation of the text transmission 20.
Preferably, the terminals 12 include displays 22 displaying screens 23 for communicating information such as text and video. Preferably, the screens 23 include text-messaging area for incoming messages 24 and outgoing messages 25, video exchange area 26 and information area 27, preferably presenting information about the communicating users 13.
As shown in
Preferably, a video session is initiated between the parties only while the parties are engaged in a text chat session. Preferably, the parties agree to engage in the video session before the video session begins. Preferably, if two (or more) users have agreed to use video augmented text chat, this agreement holds whenever these two (or more) users engage in a text chat.
Preferably, a video recording process starts automatically when a terminal 12 identifies a user's 13 activity associated with the text chat session. The video recording process may start automatically only in the terminal that identifies the user's activity or in all the terminals involved in the video augmented text chat session. Alternatively, a video session is activated when both parties are actively associated with the text chat session.
The media-augmenting communication network 10, preferably, the terminals 12, initiate the augmenting video session between parties while the parties are engaged in the augmented text chat session. The augmenting medium initiation process includes:
Reference is now made to
As seen in
The software program 34 preferably includes a text communication module 39, a video communication module 40 and a video augmentation module 41. The text communication module 39 preferably communicates the text transmissions with a user, preferably by operating the text-entering device 32 and a part of the display 22. The video communication module 40 preferably communicates video transmissions with the user, preferably by operating the camera 19 and another part of the display 22. The video augmentation module 41 preferably performs the following steps:
The augmented content 36 preferably contains received content 42 and sent content 43 (including content in preparation to be sent). The augmented content 36 preferably contains text content (text transmissions, text chat, text messages, etc., as relevant)
The augmenting content 37 preferably contains received content 44 and sent content 45 (including content in preparation to be sent). The augmented content 36 preferably contains video content (video transmissions, video chat, video messages, etc., as relevant)
The video communication module 40 preferably automatically creates augmenting content 37, such as video transmissions, when the user is engaged in text chat communication with the second terminal. The video communication module 40 may send (stream) the video content immediately as it is created. Alternatively, the video communication module 40 may store the video content in the memory 35 and send it later. For example, the video communication module 40 may send the video content as a video message.
The video communication module 40 preferably automatically displays augmenting content 37, such as video transmissions received from a second terminal, when the user of the first terminal is engaged in text chat communication with the second terminal. Preferably, the video communication module 40 stores the received video transmission in the memory 35 and retrieves it for display when the user is engaged in text chat communication with the second terminal. Alternatively, the video communication module 40 displays the received video transmission as it is received. Hence, the augmenting content, that is the video content, is created, transmitted and/or presented in real-time or near real-time.
Preferably, the processor 33 is operative to activate the video communication module 40 automatically when the user is engaged in text communication with a second user with whom video over text messaging was initiated in the past.
Optionally but preferably, if two or more users agreed to use media augmented communication such as video augmented text messaging, this consent holds for these users until it is revoked by a consenting user. This means that, for example, if users Joan and John agreed once to enter into video augmented text messaging, the next time John and Joan enters text messaging the system (that is the media-augmenting communication network 10, and/or the media-augmenting communication terminals 12, and/or the media-augmenting software program 34) automatically initiates media augmentation. For example, augmenting text chat with video content as described above.
Initiating content augmentation preferably include, for example: creating and/or capturing and/or recording video content and/or displaying video content. Detecting user activity for initiating content augmentation includes at least one of:
Preferably, the terminals 12 exchange the augmenting content (such as the video content) directly. Alternatively, the sending terminal 12 sends the augmenting content to the server 14 of
Preferably, the terminal 12 is capable of displaying on the display 22 an icon representing video augmented text session. The icon preferably represents a device being capable of video augmented text session (capability icon). A modification of the icon, or a different icon, represents a device being engaged in a video augmented text session (engagement icon). Particularly, the icon represents a status associated with the present terminal 12 and another terminal 12 with which said present 12 is currently communicating. Specifically, when the user is engaged in text chat with a particular terminal (or user) the terminals display the icon representing the video augmented capability and/or activity (engagement) for these two (or more) terminals.
Reference is now made to
More particularly but not exclusively,
It is appreciated that video augmentation module 41 is not restricted to managing video augmentation of text chat session, and can manage media other then video augmenting media other then text chat.
As seen in
If video augmentation is possible the video augmentation module 41 preferably proceeds to step 49 to display the video capability icon, and then to step 50 to check if consent is available. Consent is available, for example, if both parties have agreed in the past to engage in a video augmented text chat and this consent holds until revoked by one of the parties. If consent is not available, the video augmentation module 41 preferably proceeds to step 51 to send a request for consent to the other party. After exchanging requests and consent messages, and if consent is reached (step 52) video augmentation module 41 preferably proceeds to step 53 to save the consent for future occasions and to initiate the video augmenting call (step 54). Otherwise, if consent is not reached, the procedure ends (step 55).
The video augmentation module 41 preferably proceeds to step 56 to check if a server is used to exchange the video content. If a server should be used the server is contacted in step 57. There are the following typical situations or mechanisms for exchanging video transmission:
It is appreciated that video is either sent substantially concurrently with the text chat content, or is received substantially concurrently with (or in parallel to) the text chat content, or both.
As seen in
Various mechanisms and procedures are preferably available to determine and/or identify, and/or recognize and/or detect user activity, and particularly that the user is actively associated with the relevant text chat session. Such mechanisms and procedures include, but are not limited to:
A special case of activating video augmentation is when both parties are actively associated with the text chat session.
If the user is actively chatting the video augmentation module 41 preferably proceeds to step 59 to display the engagement icon and then to step 60.
In step 60 video augmentation module 41 preferably checks if video streaming is possible (by either terminals) and available (streaming from the other terminal). If video streaming is possible and available the video augmentation module 41 preferably proceeds to step 61 to display the incoming video stream. Optionally, the received video stream is also recorded, in whole or in part (step 62). For example, the last 20 seconds of the received video stream is stored.
If streaming is not available, video augmentation module 41 preferably retrieves and displays recorded incoming video (steps 63 and 64). Such recorded video content can be obtained as streaming video that was internally recorded earlier. Alternatively, the video content was received as a video message or a similar near real-time video. Alternatively, the video content is retrieved from a server (such as server 14 of
The video augmentation module 41 preferably proceeds to step 65 to capture video content, preferably video content including the user. The video augmentation module 41 then preferably proceeds to step 66 to either stream the video content in real-time to the other terminal(s) or to the server or to store it internally for later delivery. When the user sends the text chat content or otherwise leave the text chat the video augmentation module 41 preferably proceeds to step 67 to send the recorded video content (if video content was recorded internally in step 66.
In step 58, if the user is no longer actively chatting, the video augmentation module 41 preferably proceeds to step 68 to turn off the engagement icon. If the chat session has ended (step 69) for example by discontinuing the video session, the video augmentation module 41 preferably proceeds to step 70 to turn off the capability icon and then ends.
Thus, the software program 34, and preferably the video augmentation module 41 preferably perform a method for initiating a video call between parties while the parties are engaged in a text chat session. Preferably, the program executes the steps of:
It is expected that during the life of this patent many relevant methods and systems will be developed and the scope of the terms herein, particularly of the terms “wireless communication technology”, “text chat” and “video conferencing” are intended to include all such new technologies a priori.
It is appreciated that certain features of the invention, which are, for clarity, described in the context of separate embodiments, may also be provided in combination in a single embodiment. Conversely, various features of the invention, which are, for brevity, described in the context of a single embodiment, may also be provided separately or in any suitable sub-combination.
Although the invention has been described in conjunction with specific embodiments thereof, it is evident that many alternatives, modifications and variations will be apparent to those skilled in the art. Accordingly, it is intended to embrace all such alternatives, modifications and variations that fall within the spirit and broad scope of the appended claims. All publications, patents and patent applications mentioned in this specification are herein incorporated in their entirety by reference into the specification, to the same extent as if each individual publication, patent or patent application was specifically and individually indicated to be incorporated herein by reference. In addition, citation or identification of any reference in this application shall not be construed as an admission that such reference is available as prior art to the present invention.
This application is a U.S. National Phase Application under 35 U.S.C. 371 of PCT International Application No. PCT/IB2011/052121, which has an international filing date of May 15, 2011, and which claims the priority benefit of U.S. Provisional Patent Application No. 61/350,026, filed on Jun. 1, 2010, which is incorporated herein by reference.
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/IB2011/052121 | 5/15/2011 | WO | 00 | 3/10/2013 |
| Publishing Document | Publishing Date | Country | Kind |
|---|---|---|---|
| WO2011/151751 | 12/8/2011 | WO | A |
| Number | Name | Date | Kind |
|---|---|---|---|
| 20020073150 | Wilcock | Jun 2002 | A1 |
| 20030074451 | Parker et al. | Apr 2003 | A1 |
| 20050143053 | Virtanen et al. | Jun 2005 | A1 |
| 20070216762 | Toiyama | Sep 2007 | A1 |
| 20070268312 | Marks et al. | Nov 2007 | A1 |
| 20080022297 | Walter et al. | Jan 2008 | A1 |
| 20090005089 | Paik et al. | Jan 2009 | A1 |
| 20090119709 | Kim et al. | May 2009 | A1 |
| 20090271705 | Sheng et al. | Oct 2009 | A1 |
| 20090286515 | Othmer | Nov 2009 | A1 |
| 20090298548 | Kim et al. | Dec 2009 | A1 |
| 20090316688 | Meenavalli | Dec 2009 | A1 |
| 20100058445 | Zhu | Mar 2010 | A1 |
| 20100208030 | Kim et al. | Aug 2010 | A1 |
| 20100267369 | Lim et al. | Oct 2010 | A1 |
| 20110126258 | Emerson et al. | May 2011 | A1 |
| Number | Date | Country | |
|---|---|---|---|
| 20130162750 A1 | Jun 2013 | US |
| Number | Date | Country | |
|---|---|---|---|
| 61350026 | Jun 2010 | US |
