Patent Application
20090070688
The present disclosure relates generally to the field of shared virtual environments, and more particularly to managing interactions between participants of the shared virtual environment.
Various virtual environments are known in the art. Such environments typically serve to permit a group of participants who share a similar interest, goal, or task to interact with one another. Because of the shared similarities, such an environment is generally referred to as a shared virtual environment. Participants in a shared virtual environment are generally represented by an avatar. A participant viewing the shared virtual environment will typically see, within the shared virtual environment, one or more avatars that represent the other participants that are present in the shared virtual environment. The participants interact with each other in the shared virtual environment.
The shared virtual environment allows participants to have audio interactions, to have visual interactions, to share documents, and so forth. In the shared virtual environment, a situation may arise where all the participants interact with each other at the same time. In such a case, the shared virtual environment may then look chaotic, sound noisy, and/or be unpleasant. Where two participants are having audio interactions with each other, the audio interactions may disturb the other participants having other interactions (audio or otherwise) in the shared virtual environment.
Accordingly, there is a need for a method and apparatus for managing interactions.
The accompanying figures, where like reference numerals refer to identical or functionally similar elements throughout the separate views, together with the detailed description below, are incorporated in and form part of the specification, and serve to further illustrate embodiments of concepts that include the claimed invention, and explain various principles and advantages of those embodiments.
a illustrates a display unit, in accordance with an embodiment of the present invention.
b illustrates a display unit, in accordance with another embodiment of the present invention.
Skilled artisans will appreciate that elements in the figures are illustrated for simplicity and clarity and have not necessarily been drawn to scale. For example, the dimensions of some of the elements in the figures may be exaggerated relative to other elements to help to improve understanding of embodiments of the present invention.
The apparatus and method components have been represented where appropriate by conventional symbols in the drawings, showing only those specific details that are pertinent to understanding the embodiments of the present invention so as not to obscure the disclosure with details that will be readily apparent to those of ordinary skill in the art having the benefit of the description herein.
Various embodiments of the invention provide a method and an apparatus for managing interactions between participants of a shared virtual environment. A communication session is established by a first participant of the shared virtual environment with a second participant of the shared virtual environment. In the shared virtual environment, the first participant is represented as a first avatar and the second participant is represented as a second avatar. A data stream is received by the first participant located at a first location from the second participant located at a second location. Using the received data stream, a view of the shared virtual environment is generated for the first participant so that the shared virtual environment comprises the second avatar where the second avatar represents the second participant as seen from a perspective of the first participant. The audio of the data stream of the second participant is controlled by the first participant. A text of the controlled audio is generated and displayed in the view of the shared virtual environment of the first participant.
Before describing in detail the method and apparatus for managing interactions between participants of a shared virtual environment, it should be observed that the present invention resides primarily in combinations of method steps and system components related to a method and apparatus for managing interactions. Accordingly, the apparatus components and method steps have been represented where appropriate by conventional symbols in the drawings, showing only those specific details that are pertinent to understanding the present invention so as not to obscure the disclosure with details that will be readily apparent to those of ordinary skill in the art having the benefit of the description herein.
The shared virtual environment server may reside on a network such as an Internet, a Public Switched Telephone Network (PSTN), a mobile network, a broadband network, and so forth. In accordance with various embodiments of the invention, the shared virtual environment 110 can also reside on a combination of different types of networks. Thus, the use of the term “network” is meant to encompass all such variants.
Once the communication session is established, the participants (102-108) communicate by transmitting and receiving data streams across the network. Each of the data streams can be an audio stream, an audio data stream, a video stream or an audio-visual data stream, as is generally known.
The display unit 204 receives processed data from the processing unit and displays an avatar of each of the participants (102-108) of the shared virtual environment 110. The avatars correspond to virtual representations of participants of the shared virtual environment 110. Examples of virtual representation are an image, animation, a video, audio, or any combination of these examples. As such, an avatar of the second participant 104 may be a video received from the second participant 104 or an avatar of the second participant may be an image (e.g., a drawing, picture, shape, etc.) in combination with audio received from the second participant.
In an embodiment, the audio controller 306 includes a switch 308. The switch 308 operates to enable the decoded audio of the second participant to be controlled. As an example, the decoded audio will be controlled by the audio controller 306 when the switch 308 is in an active state. When the switch 308 is in an inactive state, which is normally the case, the decoded audio will be sent to the speaker 206 of the apparatus 200, e.g., associated with second participant 104.
The speech to text converter 310 coupled to the audio controller 306 generates text of the audio being controlled. The text is generated by converting the controlled audio of the second participant 104 into text, e.g., using any well known speech recognition algorithm or transcription service.
The rendering unit 312 generates a view of the shared virtual environment 502, e.g., by a process called rendering. As is known, rendering is a process of generating an image from a description of three dimensional objects. Typically, description includes geometry, viewpoint, texture, lighting, and shading information for the three dimensional objects. In an embodiment, the rendering unit 312 generates the view of the shared virtual environment 502 for the first participant 102 by generating images of the other participants' (namely 104-108) avatars and objects in the shared virtual environment. Specifically, the view of the shared virtual environment 502 has a surface upon which the avatar of the second participant is rendered. In one embodiment, the view of the shared virtual environment 502 is generated using the data stream received from the second participant 104. In any case, the rendering unit 312 coupled to the speech to text converter 310 then displays the text received from the speech to text converter 310 in the view of the shared virtual environment 502.
At step 404, a data stream is received by the first participant 102 from the second participant 104. As mentioned above, the data stream can include audio, video, still images, visualizations, slide shows, and/or any combination. In an embodiment, the first participant 102 is located at a first location and the second participant 104 is located at a second location. In such an embodiment, the second location may be remote to the first location.
At step 406, a view of the shared virtual environment 502 for the first participant 102 is generated using the data stream received from the second participant 104. The view of the shared virtual environment 502 includes the second avatar 504 as seen from a perspective of the first participant 102. In an embodiment, generating the view of the shared virtual environment 502 is implemented by selecting a surface 501 within the view of the shared virtual environment 502 and rendering the second avatar 504 on the selected surface 501.
The surface 501 (also referred to as a “texture”) is defined as any surface that can be drawn upon in a virtual environment or an application user interface of the apparatus 200. As such, the texture may be, e.g., a Java Mobile 3D Graphics (“M3G”) texture, a Java Abstract Window Toolkit (“AWT”) image, a 3D virtual environment drawable surface, a 2D drawable surface, or a user interface element. Once the view of the shared virtual environment 502 is generated, the participants (102-108) of the shared virtual environment 110 begin interacting with each other. The interactions are facilitated by the sending and receiving of data streams among the participants (102-108).
At step 408, the audio of the data stream of the second participant 104 is controlled by the first participant 102. In one example, if the first participant 102 has sufficient authority, the first participant 102 will be able to control the audio of the second participant 104. In any case, in an embodiment, controlling the decoded audio of the second participant 104 is done in a number of ways. One way is by muting the audio (namely the decoded audio) where muting the audio of the second participant 104 means to silence the audio of the second participant 104. In another way, controlling the decoded audio of the second participant 104 is done by varying the volume of the audio. In an example, the volume of the audio of the second participant 104 is changed, e.g., lowered. In yet another way, controlling the decoded audio of the second participant 104 means to make the audio of the second participant 104 audible.
Upon controlling the audio of the second participant 104, a text of the audio of the second participant 104 is generated at step 410. At step 412, the generated text is displayed in the view of the shared virtual environment 502 of the first participant 102. In an embodiment, the method of displaying the generated text is performed by locating a region in the view of the shared virtual environment 502 and then displaying the generated text as an overlay in the located region. Displaying the generated text as an overlay is done by superimposing the generated text on the located region. In an example, locating a region in the view of the shared virtual environment 502 means to find a region in the view of the shared virtual environment that pertains to the second avatar 504.
In an alternate embodiment, as shown in
In an example embodiment, the shared virtual environment includes the third participant 106. Accordingly, a communication session is established by the first participant 102 with the third participant 106 of the shared virtual environment 110. The first participant 102 is located at a first location and the third participant 106 is located at a third location. A data stream is received by the first participant 102 from the third participant 106. A view of the shared virtual environment 502 for the first participant 102 is generated using the data stream received from the third participant 106. The view of the shared virtual environment 502 includes the third avatar 506 as seen from a perspective of the first participant 102. The third avatar 506 represents the third participant 106. The audio of the data stream of the third participant 106 is controlled by the first participant 102. The text of the controlled audio of the third participant 106 is displayed in the view of the shared virtual environment 502 of the first participant 102.
In another embodiment, one of the participants of the shared virtual environment 110 behaves as a controller while the rest of the participants of the shared virtual environment 110 behave as controllees. The controller may be the participant having sufficient authority to control the interactions of the controllees. The controller is also given authority to authenticate the controllees trying to establish communication sessions. In this embodiment, the controller is located at a first location and the controllee is located at a second location. The second location is remote to the first location. The controller and the controllee are represented as avatars. The controller receives data streams from the controllee in real-time. Receiving data in real-time means to acquire data as and when it is being generated and transmitted as opposed to receiving recorded data for later playback. In real-time, delay is limited to the actual time required to transmit the data. Using the received real-time data stream, a view of the shared virtual environment is generated for the controller. The view of the shared virtual environment includes an avatar of the controllee as seen from a perspective of the controller. On controlling the audio of the controllee by the controller, a text of the controlled audio is generated. The generated text is then displayed in the view of the shared virtual environment of the controller.
Regardless of the display of the generated text, examples of embodiments of the present invention provide for managing interactions. In an example, the shared virtual environment represents a public safety environment in which members of police department and Federal Bureau of Investigation (FBI) may be present as participants of the shared virtual environment. In use, a marshal (e.g., a first participant) may enter the shared virtual environment to see avatars of the local chief of police (e.g., a second participant) and a regional head of the FBI (e.g., a third participant). The chief of police and the regional head of the FBI are engaged in a loud argument on who has jurisdiction on a case. This acrimonious exchange is so overwhelming that the marshal mutes the audio of the chief of police and the audio of the regional head of the FBI. As the avatars of the chief of police and the regional head of the FBI go silent, a text of the argument between the chief of police and the regional head of the FBI appears overlaid on their respective avatars. A participant of the shared virtual environment will thus be able to follow and keep up with the silenced interactions as well as the interactions between other participants in the shared virtual environment. The participant then has the advantage of being able to manage interactions.
In this document, relational terms such as first and second, top and bottom, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. The above description and the diagrams do not necessarily require the order illustrated.
The terms “comprises,” “comprising,” or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by “comprises . . . a” does not, without more constraints, preclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.
It will be appreciated that embodiments of the invention described herein may be comprised of one or more conventional processors and unique stored program instructions that control the one or more processors to implement, in conjunction with certain non-processor circuits, some, most, or all of the functions described herein. The non-processor circuits may include, but are not limited to, a radio receiver, a radio transmitter, signal drivers, clock circuits, power source circuits, and user input devices. As such, these functions may be interpreted as steps of a method. Alternatively, some or all functions could be implemented by a state machine that has no stored program instructions, or in one or more application specific integrated circuits (ASICs), in which each function or some combinations of certain of the functions are implemented as custom logic. Of course, a combination of the two approaches could be used. Thus, methods and means for these functions have been described herein. Further, it is expected that one of ordinary skill, notwithstanding possibly significant effort and many design choices motivated by, for example, available time, current technology, and economic considerations, when guided by the concepts and principles disclosed herein will be readily capable of generating such software instructions and programs and ICs with minimal experimentation.
In the foregoing specification, specific embodiments of the present invention have been described. However, one of ordinary skill in the art appreciates that various modifications and changes can be made without departing from the scope of the present invention as set forth in the claims below. Accordingly, the specification and figures are to be regarded in an illustrative rather than a restrictive sense, and all such modifications are intended to be included within the scope of present invention. The benefits, advantages, solutions to problems, and any element(s) that may cause any benefit, advantage, or solution to occur or become more pronounced are not to be construed as a critical, required, or essential features or elements of any or all the claims. The invention is defined solely by the appended claims including any amendments made during the pendency of this application and all equivalents of those claims as issued.
The Abstract of the Disclosure is provided to allow the reader to quickly ascertain the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In addition, in the foregoing Detailed Description, it can be seen that various features are grouped together in various embodiments for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed embodiment. Thus the following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as a separately claimed subject matter.
