Patent Application
20150020003
This invention relates to computer software enabling interactions between users in a shared virtual space. One of the many uses of computer networks, such as the Internet, is to enable people to make new acquaintances “online.” There are a variety of different forums that are used for this purpose, such as chat rooms, message boards, or shared three-dimensional virtual spaces. The designers of such online or virtual environments often go to great lengths to simulate environments that are as close to real life environments as possible. One aspect that is among the most difficult ones to simulate in an online environment is a sense of a common purpose, both in the immediate sense in and in the larger sense, coupled with an atmosphere of serendipity, synchrony and fun.
One type of online environment which many users find fascinating and intriguing is known as Google Earth and provided by Google Inc. of Mountain View, Calif. Several versions of Google Earth exist, but the main purpose and functionality is the same in all versions. On a conceptual level, Google Earth can be described as a virtual globe that sits inside a user's personal computer. A user can point and zoom to any place on Earth that she would like to explore. When doing so, satellite images and local facts zoom into view on the user's display screen. Users can also look up specific addresses, get driving directions, and “fly” along a route. Generally speaking, all of these tasks are achieved by running client applications on the user's computer, which communicates with a remote server over a network, such as the Internet. The client applications send data to the remote server about the user's position in space (for example, in the form of an (x,y,z) coordinate triplet). In response to receiving this information, the remote server presents the user with satellite images that are displayed to the user by the client application in such a manner that they appear to the user as if the point of view were the user's coordinate position. Doing this dynamically in real-time gives the user the sensation of “flying” above the surface of the Earth.
While most users find it fascinating to explore different parts of planet Earth, it can be a solitary type of activity.
In general, in one aspect, the invention provides methods and apparatus, including computer program products, implementing and using techniques for establishing interaction between two or more users simultaneously viewing a virtual representation of a physical object on two or more computers connected to a network. A view of a portion of the physical object is displayed to a user. The view is based on a set of coordinates that identifies a current position of the user. One or more figurines are displayed in the view. The figurines represent other users that are simultaneously looking at a similar portion of the virtual representation of the physical object. Each figurine is shown in the current view at a coordinate position corresponding to a current position of a respective other user. A request is received from the user to interact with one or more of the other users whose figurines are displayed within the current view. The request is transmitted to the one or more other users. Interactions between the requesting user and the one or more other users are established based on responses received from the one or more other users to the request.
Advantageous implementations can include one or more of the following features. Displaying figurines can include displaying figurines only of other users that would like to be visible to the user. Displaying figurines can include displaying customized figurines of at least some other users, according to preferences selected by the other users represented by the customized figurines. The customized figurines can include one or more of: a geometrical shape, a color, an image, and a video. The customized figurines can include a status indicator showing the other users' availability for interactions with the user.
Receiving a request from the user to interact with one or more of the other users can include receiving a request to broadcast a message to all other users whose current position is within a region of influence centered on the current position of the user. The region of influence can be a sphere of a specified radius, centered on the current position of the user. Receiving a request to broadcast a message to all users within a region of influence can further include determining whether the user has permission to broadcast a message to all other users within the region of influence, and broadcasting the message if the user has permission.
Receiving a request from the user to interact with one or more of the other users can include receiving a request to send a private message to a single user. Receiving a request from the user to interact with one or more of the other users can include receiving a request to broadcast a message to a selected group of other users independently of the current positions of the other users in the selected group of other users. The interactions between the requesting user and the one or more other users can include one or more of communicating by instant text messages, communicating by audio, and communicating by video. Receiving a request from the user to interact with one or more of the other users can include receiving a request to join one of the other users at the other user's current position and to look at a same portion of the virtual representation of the physical object. The other user can be joined when the other user moves from the current position of the other user to a different position.
A request can be received from the user to add a place mark to the current position before moving to a new position. The place mark is visible to the other users and provides a means for the other users to request interaction with the user after the user has moved away from the current position. A user can move from the current position to a new position and leave a trail visible to the other users, such that anyone of the other users can follow the path taken by the user when moving from the current position to the new position. Contact information can be saved in a list about the one or more other users with whom interactions were established, such that the one or more other users can be contacted again. The one or more other users can be asked for permission to save their contact information in the list, and contact information can be saved only for the one or more other users who give permission to save their contact information. The virtual representation of the physical object can be a virtual representation of planet Earth that is generated from satellite imagery of planet Earth, and the portion of the physical object can be a geographical region of planet Earth.
In general, in another aspect, the invention provides methods and apparatus, including computer program products, implementing and using techniques for interacting with one or more users in a shared virtual reality space representing planet Earth. The shared virtual reality space is generated from satellite imagery of planet Earth. A set of coordinates is sent to a remote server. The set of coordinates represents a current position of a user in the shared virtual reality space. Information representing a view of a geographical portion of planet Earth from the current position of the user is received from the remote server. A set of figurines is received from the remote server. The figurines represent one or more other users that are simultaneously located proximate to the current position of the user in the shared virtual reality space. The received information and set of figurines is displayed to a user and each figurine is shown at a position corresponding to each other user's current position. A user input requesting an interaction with one or more of the other users is received. The request is transmitted to the remote server and interactions are established between the requesting user and the one or more other users based on response information to the request received from the server.
The invention can be implemented to include one or more of the following advantages. Users can toggle in and out of a mode where they become visible to and can be contacted by other users. Users can personalize their flying figurines and duplicate their presence by placing sentinels. Users can engage each other synchronously through textual, audio and video communication. Users can employ these types of media to communicate through public broadcasts to several other users, and/or to select users through private communication channels. Users can co-browse planet Earth by merging points of view and sharing navigation controls determining what portions of planet Earth to look at. Users can ask other users for permission to access their current location, in order to more easily find them again at a later point in time. Users can indicate their availability status for interactions with other users, such as “I'm just here to observe,” “Please feel free to engage me in conversation,” or “Please do not disturb,” and so on.
These tools and features enable a variety of scenarios and use cases. For example, the tools allow users to meet by chance and connect with new people who may have similar geographical interests, since the users are exploring similar portions of planet Earth. By interacting with other users, it is also possible to find and discuss information that is not accessible through regular use of Google Earth, since other users may provide further information and contribute their own personal knowledge about specific geographic regions. Users can lead one or more users through real-time guided tours while answering their questions. Virtual gatherings can be staged, for example, friends can decide to gather in one place, or public demonstrations can be staged for the world to see.
In a larger sense, Google Earth provides a shared virtual space that has the potential of creating a sense of common purpose, coupled with an atmosphere of serendipity, synchrony and fun in a way that has not been possible before. For example:
The details of one or more embodiments of the invention are set forth in the accompanying drawings and the description below. Other features and advantages of the invention will be apparent from the description and drawings, and from the claims.
Like reference symbols in the various drawings indicate like elements.
As was described above, on a conceptual level, Google Earth can be described as a virtual globe that sits inside a user's personal computer. A user can point and zoom to any place on Earth that she would like to explore. When doing so, satellite images and local facts zoom into view on the user's display screen. Users can also look up specific addresses, get driving directions, and “fly” along a route. Various embodiments of this invention allow users to become represented in the three-dimensional virtual Google Earth space and to become visible to others as a figurine spatially located at the coordinate position of their point of view. Whenever the person represented by the figurine changes points of view (i.e., positions) using the keyboard or mouse controls (or other input device) on his computer, the figurine moves accordingly in space. Expressed differently, the point of view of a user and the point in space where that same user appears to other users are one and the same.
In some embodiments, which will be described in further detail below, in order to be visible to other users and to be able to see other users, a user must switch on a visibility toggle. This is indicated by a “V” for each user shown in
As was described above, and can be seen in
As can be seen in
In one implementation, the flying figurines occupy a specific volume of the virtual space that remains to scale relative to the representation of the Earth (e.g. a cube of dimensions 20 m×20 m×20 m). As a figurine representing a user gets further away from the viewpoint of another user, the figurine's two-dimensional representation will shrink, similar to what it would appear in real life. When the figurine is far away enough that the representation of the figurine reaches a certain arbitrary threshold of smallness, (e.g. a couple pixels), the figurine never quite disappears, but is instead replaced by a tiny dot-like icon, which would require minimal computing power to continue to display. As will be discussed below, in one implementation, similar principles apply for sound, where past a certain point, the sound might be turned off and instead be replaced by a visual indicator. As the skilled person realizes, most figurines will be invisible anyway when a user is zoomed in close to the surface of planet Earth. In situations when a user is not zoomed in close to the surface of planet Earth, the figurines representing other users will typically be at a far distance, and thus have a small enough dot-like representation as to not obscure the view of the user.
In some embodiments of the invention, all the other users' orientations are in the direction of the gaze of the user looking at his display screen. That is, returning to
In some embodiments, the users can personalize their flying figurines using their client applications. Some examples of personalized figurines can be seen in
In some embodiments, the users can select the color of their figurine from a set of recommended colors that will stand out well against the satellite imagery background. In other embodiments, the users can choose their own colors. Colors can also be used to indicate an “availability status” for the user on the figurine to mimic the visual cues that are perceived in the real world by observing other people's body language. For example, red can indicate “Do not disturb,” yellow can indicate “I'm just here to observe,” and green can indicate “I'm available to be engaged in conversations or other interactions.” In some embodiments, these “availability status” indicators can be represented as text, such as the “Do not disturb” message in
In some embodiments, the users can customize their figurines with a text message, such as “Fredrik,” “Joe's tour group,” and “Maria” in
In some embodiments, such as the one shown in
Another feature that is used in some embodiments to facilitate finding other users is to place sentinels (504) at specific coordinate positions on the representation of Earth. A sentinel (504) can be represented as a semi-transparent “ghost image” of the user's figurine. In some embodiments, this is accomplished by the server (204) keeping records of a specific set of positions, at the user's request, and supplying those positions to other users in the vicinity. When another user selects the sentinel (504), the user who placed the sentinel will be contacted, regardless of his current position. In some embodiments, selecting a sentinel (504) also allows the other user to move from the sentinel's position to the current position of the user represented by the sentinel (504).
As was noted above, in the various embodiments of the invention the users can communicate with each other using one or more of three channels: text, audio, and video. These channels are not mutually exclusive and can be used concurrently. As has also been noted above, each of these channels can be used either publicly (also referred to herein as “broadcasting”) or privately (also referred to herein as “whispering”).
In some embodiments, when a user publicly broadcasts a text message, the text message propagates spherically around the user and reaches all users whose position falls within a particular radius that the user has chosen. This is schematically illustrated in
Broadcasting, as described here, can be implemented by the broadcasting user's client application sending to the server (204) the broadcasting user's text message together with his current position. The server (204) then identifies what other users are in the vicinity of the broadcasting user and transmits the message to these other users, together with the position coordinates of the broadcasting user. The receiving users' client applications then adjust the text font size according to the distance between the broadcasting user and the respective receiving users.
When a user privately whispers a text message to another user, the user's text will appear to the other user in large clear fonts, regardless of the distance separating the two users. In some embodiments, the private whisper will not appear in a bubble but instead in a distinct chat module not pointing at anyone. In some embodiments private whispers can be sent simultaneously to more than a single user. Regardless of whether the private whisper is sent to a single user or to a group of users, only the selected recipients will be able to see the text in the private whisper.
The whispering, as described here, can be implemented by the sender's client application sending to the server (204) the sender's text message together with his name and picture or video. The server (204) then transmits the message only to the other users that have been specified by the sender (i.e., here the location of the sender is insignificant). The receiving users' client applications then display the name and picture or video, and the message from the sender, for example, in the bottom left corner of the display screen. No adjustments are made of the text font size based on the distance between the sender and the recipients.
If a user decides to publicly broadcast an audio message, the audio message will propagate spherically around the user and reach all other users within the chosen broadcast radius, similar to the way text messages behave as described above. In some embodiments, the recipients of the audio message will hear the user's voice in stereo, from the direction the message is coming from in the three-dimensional space, whether the broadcasting user is in the line of sight of the receiving user or not. The further away the broadcasting user is from the receiving users, the fainter the volume of the broadcasting user's voice will be. Similar to the text messaging above, if the receiving users are intrigued, they will have to fly closer to the broadcasting user.
In some embodiments, when a user privately sends an audio message to another user, the user's voice will appear to the other user loud and clear, regardless of the distance separating the two users. The sound will not be directional, but will simply fill up the other user's auditory space. No one other than the receiving user will hear the sending user's voice, but as in text messaging there can be more than one receiving user.
In some embodiments, when a user publicly broadcasts a picture or a live streaming video of himself, the picture or video is visible on the user's figurine to any other users that are sufficiently close to see the figurine. The further away the user is from other users, the smaller the figurine will appear, and as a result, the smaller the picture or video will also appear. Similar to the text and audio messaging above, if the receiving users are intrigued, they will have to fly closer to the broadcasting user.
When a user privately sends a picture or live streaming video of himself to someone, the picture or video will appear large and crisp, regardless of the distance separating the two users. The picture or video will not appear on the user's figurine, but in a distinct image module. No one other than the receiving user will se the sending user's picture or video, although there can be several recipients, just as in the above text and audio communication scenarios.
The selection of an audience for broadcast or private transmissions of messages can be done in a variety of ways. In some embodiments of the invention a communication panel is used for this purpose.
The user has also decided that whatever he says in his microphone, represented by an audio field (604), will always reach his contacts Stefanie (610) and Yan (612) directly, provided they are logged-on, but not any other people. That way, the user, Stefanie (610) and Yan (612) can be looking at different parts of the globe, yet still be communicating with each other.
As for his picture, shown in the video field (606), the user has employed the white arrows to browse through his album and settled on the one displaying his face, which he chose to display to everyone flying near his figurine, as indicated by the globe (608) and also set it so that Stefanie (610) and Yan (612) would always see it in a corner of their display screen whenever the user speaks up. If the user decides, he may at some later point replace this photo with a streaming video, as described above.
In some embodiments, there can be limitations of how many broadcast messages a user can send out to everyone around him. For example, the communication panel (600) can have an associated meter that contains a number of credits that are awarded to the user on a regular basis. Whenever a user broadcasts a public message a certain amount of these credits are used up. In some embodiments, more credits are used if the user selects a large broadcast radius compared to if the user selects a small broadcast radius. In some embodiments, more credits are used if there are many users around him, compared to if there are few users around him. This would, for example, limit the number of broadcasts over a densely populated or visited area, such as Paris, while allowing a user to broadcast to his hearts delight over a sparsely populated or visited area, such as the Kalahari Desert. In other embodiments, a user can only broadcast a message every few minutes or hours. Many variations of such broadcast meters will be apparent to those of ordinary skill in the art.
Some embodiments of the invention enable functionality referred to as “vision sharing.” Vision sharing allows users to co-browse the globe and fly with other users.
It should be noted that the method of inviting other users to join you that is shown in
In some embodiments, the user in control of the group can specify a particular flight formation that the following users must assume. These formations can be selected from a set of geometrical shapes, or, if the user has a sufficient number of followers, the user can choose to spell out letters to form words with his formation. This feature will be discussed in further detail below with reference to
Returning now to
At some point, Laurent may decide to hand over the controls to the user or any other guest that is concurrently flying with him. If the user accepts, the user will become the host and start controlling the course of the flight for everyone else onboard. In some embodiments it is also possible to yield control to someone other than the guests that are currently part of the formation. For example, if Laurent decides to join someone else's flight, Laurent and all his guests will be added to the list of guests of that new host. This can be, for example, a useful technique to quickly round up a group of friends.
In some embodiments, the host has a “passenger list” of the guests flying with the host. The guests can see the guest list, which has a special marker indicating who the host is. Whenever a host inputs changes in position through his client application, these position changes are transmitted to the server (204). The server (204) then transmits the same position updates and relevant imagery data to all the guests, including the host. The host can select any guest (or any guest can request control from the host), for example, by right-clicking on the guest's name and then selecting an option to pass control. The guest who has been asked to take control, or a host that has been asked to give control, can either accept or refuse the request. When control is successfully passed, the server (204) stops receiving input from the former host and begins parsing input coming from the newly appointed host, and then transmitting the same position updates and relevant imagery data to all guests, including the new host.
In some embodiments, the formation is selected by the host when the invitation is sent out to other users to fly with her. The formation can be selected from various shapes, such as a wedge, circle, triangle, square, and so on, or as a set of letters, as described above. When the other users become guests, they fly towards the host, but instead of merging points of view, they fly in close proximity to the host at a place computed by a ‘best fit’ algorithm (e.g., if the host has selected a circle formation and there are only three users, they would be distributed as the nodes in an equilateral triangle). In some embodiments, the guests' places are simply marked and the guests are responsible for placing themselves at the appropriate mark. In other embodiments the guests' figurines are ‘snapped in’ to the computed places when they get close, similar to the actions of a black hole.
As is clear from the above description, this three-dimensional shared virtual representation of Planet Earth encourages people to meet and interact with each other. As a result of these interactions, often people would like to save each others' contact information so that they can find each other again at a later point in time. Typically, there are two kinds of people that a user would like to keep handily for future contact: those who were randomly encountered and who the user would like to talk to again, and those that the user already knows and would like to be able to contact whenever they are also using Google Earth.
In various embodiments there are different ways of storing these people. For the random or serendipitous encounters, the user can ask for permission from the person to store them as a place mark. The place mark will take the user straight to the other person when the other person is online, similar to how the sentinels described above work, and will appear grayed out when the other person is offline. For the people that the user already knows, the user can send invitations to join the user in this three-dimensional shared virtual space. If the user's invitation is accepted, the other person and the user will appear as place marks on each other's display screens.
The invention can be implemented in digital electronic circuitry, or in computer hardware, firmware, software, or in combinations of them. Apparatus of the invention can be implemented in a computer program product tangibly embodied in a machine-readable storage device for execution by a programmable processor; and method steps of the invention can be performed by a programmable processor executing a program of instructions to perform functions of the invention by operating on input data and generating output. The invention can be implemented advantageously in one or more computer programs that are executable on a programmable system including at least one programmable processor coupled to receive data and instructions from, and to transmit data and instructions to, a data storage system, at least one input device, and at least one output device. Each computer program can be implemented in a high-level procedural or object-oriented programming language, or in assembly or machine language if desired; and in any case, the language can be a compiled or interpreted language. Suitable processors include, by way of example, both general and special purpose microprocessors. Generally, a processor will receive instructions and data from a read-only memory and/or a random access memory. Generally, a computer will include one or more mass storage devices for storing data files; such devices include magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and optical disks. Storage devices suitable for tangibly embodying computer program instructions and data include all forms of non-volatile memory, including by way of example semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM disks. Any of the foregoing can be supplemented by, or incorporated in, ASICs (application-specific integrated circuits).
To provide for interaction with a user, the invention can be implemented on a computer system having a display device such as a monitor or LCD screen for displaying information to the user. The user can provide input to the computer system through various input devices such as a keyboard and a pointing device, such as a mouse, a trackball, a microphone, a touch-sensitive display, a transducer card reader, a magnetic or paper tape reader, a tablet, a stylus, a voice or handwriting recognizer, or any other well-known input device such as, of course, other computers. The computer system can be programmed to provide a graphical user interface through which computer programs interact with users.
Finally, the processor optionally can be coupled to a computer or telecommunications network, for example, an Internet network, or an intranet network, using a network connection, through which the processor can receive information from the network, or might output information to the network in the course of performing the above-described method steps. Such information, which is often represented as a sequence of instructions to be executed using the processor, may be received from and outputted to the network, for example, in the form of a computer data signal embodied in a carrier wave. The above-described devices and materials will be familiar to those of skill in the computer hardware and software arts.
A number of implementations of the invention have been described. Nevertheless, it will be understood that various modifications may be made without departing from the spirit and scope of the invention. For example, all of the interactions described above have been described in the context of Google Earth. However, it should be clear that any continuous three-dimensional virtual space can be used as a platform for the types of interactions described above. For example, imagery showing the interior of the human body can be used and physicians may “fly” through blood vessels, and so on, similar to what was described above, and discuss treatment options for various abnormalities that are found, and so on. Text messages were described as getting smaller as the broadcasting user gets further away from the receiving user. However, other mechanisms can also be used, such as the text message having the same size but fading away or changing colors. Audio messages were described as being directional and in stereo, but they can of course also be non-directional and in mono. The broadcasting was described as being spherical, but of course any geometrical shape is possible, such as cubes, or “horizontal slices” so that only users at the same height above Earth as the broadcasting user can receive the broadcast message, and so on. In some embodiments, users may allow other users to eavesdrop on their private conversations, for example, by broadcasting their private conversations over a very small radius, and let the eavesdropping user decide whether to join the conversation or not.
The above interactions have been described by way of example in terms of exploring planet Earth, flying with other users, and so on. It should however be realized that these basic functionalities can be used to stage various types of games and contests. For example, various types of scavenger hunts can be organized in which teams of users compete against each others in finding and solving virtual clues that are hidden at various places on the representation of planet Earth, and so on. Other examples include guided tours, high-school or college reunions, political rallies, capture-the-flag and other spatial team games or quests, and so on. The various modes of interactions that have been described above can also be used in other settings than Google Earth. For example, in various mapping applications, which also provide a continuous space, but which today offer no capabilities of users to interact with each other. Accordingly, other embodiments are within the scope of the following claims.
