Patent Application
20180124374
The invention relates to the system and method of displaying an abbreviated virtual reality 360-degree image without reducing the user experience.
The present invention is a useful and novel method for providing a human user with the experience of being in a 360-degree virtual reality (VR360) experience while reducing systems resources by not providing a full 360-degree video feed.
Virtual reality (VR) is a misused term that encompasses a range of distinctive technologies. Virtual reality is simply the display of what appears to be a three-dimensional (3D) image. Today, we assume the image is electronically produced and displayed. However, 3D images have been displayed since the early 1800s in what was called stereoscope. Humans perceive depth because of the space between our eyes. Like VR, stereoscope presents a different image for each eye (L/R images). When viewed with both eyes, the two images produce the appearance of a single 3D image, or anaglyph image. This can be reproduced by alternately closing the left and right eyes while reading this document. The image jumps left to right and centers again when both eyes are opened. VR simply recreates this exercise. The L/R images are shifted only slightly apart along the horizontal vision.
In the mid-1800s, David Brewster, invented the lenticular stereoscope with the introduction of lenses to combine the L/R images. This allowed the stereoscope to be portable and led to the familiar View-Master. The View-Master, still sold today, produces a 3D VR still image. Newer models of the View-Master blur the line of lenticular stereoscope by offering a VR360 still image using a smartphone (even including sound).
In motion picture theaters, 3D images are produced by projecting the L/R images from two projectors onto the same screen. The user wears polarized eyeglasses that allow each eye to discern a different image.
As such, what we think of as VR360 is not virtual reality as it was first conceived. Also, 360-degree video, by itself, is not necessarily virtual reality. For the purposes of this application VR360 will refer to a stream of L/R images presented to a user, using special electronic equipment, expecting a 360-degree immersion experience. This includes ‘room scale’ solutions that allows the user to walk around in the video or interact in other physical ways. The user is actually only able to observer at any given moment, through peripheral vision, 180 to 220 degrees on their visual horizontal plane (temporarily), 50 degrees upward (superiorly) and 60 to 70 degrees downward (inferiorly).
The L/R images are traditionally created with cameras with stereoscopic lenses: one camera lens for the left eye and one camera lens for the right eye. The two images are stored separately and presented simultaneously to the user employing a viewing device.
However, digital software also exists that can take a 2D video and create L/R images by adding the eye separation distance (disparity cue) to simulate depth. This is becoming more and more common because it provides cost and post-production advantages. Today, most non-CGI 3D films are converted from a 2D video. While today this is done on a server, it is not hard to imagine the viewing devices of the future would be able to do the conversion from a 2D video feed.
No matter the method used to create the L/R image, the systems in the prior art require a production software that takes the adjacent sensors, pre-stitches them into a single file. The production file is then adjusted for visual clutter created by combining the individual sensors into a single file. The clutter can include adjusting distance, size, opacity, and clarity. After the adjustment are made, either manually or with a software sweep, the single file is stored with the entire VR 360 images created by the sensors.
All the VR360 systems heretofore known suffer from a number of disadvantages:
An invention, which meets the needs stated above, is a system and method to provide a VR360 experience without transmitting a full 360-degree view to the viewing device.
Accordingly, besides the objects and advantages of the System and Method for Reducing System Requirements for a Virtual Reality 360 Display, described above, several objects and advantages of the present invention are:
Further objects and advantages of this invention will become apparent from a consideration of the drawings and the ensuing description of the drawings.
According to an embodiment of the disclosure, a system for displaying streamed video from a distance comprises one or more capturing devices and one or more servers. Each of the one more capturing device have a plurality of sensors configured to capture light used in forming image frames for a video stream. The plurality of sensors is arranged around a shape to capture the light at different focal points and at different angles. The one or more servers are configured to receive light data from the one or more capturing devices, and to provide a dynamically user selected subset of the light data captured by the plurality of sensors to a remote end user as a stream of image frames for a video stream. The subset of the light data provided by the one more servers at a particular instance depend on selections from the end user.
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the present invention and together with the description, serve to explain the principles of this invention. In the figures:
3D: displaying an image that has the appearance of depth with some objects closer and some objects further away.
360: displaying an image that provides the user the perception they have a 360-horizontal view surrounding their person.
Anaglyph image: The result when L/R images are combined by the mind with the use of tool.
L/R images: two separate images presented to each eye to produce a 3D effect.
VR360: a system of cameras, servers and viewers that provide human users with the experience of being in a manufactured video experience that closely resembles being present at the actual event. The user is able to turn all 360 degrees and see objects at each degree. Using L/R images, the video presents a 3D virtual reality effect.
Not everyone has the opportunity of a much envied 50-yard line tickets at a college or professional football game. And, not everyone has the time in their schedule to attend the wedding of a friend of loved ones or a concert from their favorite band. Moreover, the videos of such events don't substitute for actually being at the actual event. The viewer of the videos must watch what the cameraman or producer viewed important.
Given concerns such as these, embodiments of the disclosure provide a system that emulates the switching of information one chooses to see, for example, based on movement of their head and eyes, but at a distance from the actual event. According to particular embodiments of the disclosure, the switched information provided to the user may be the next best thing to actual being at the event (or perhaps even better because of rewind capability). According to particular embodiments, the information can be played back in real time, later played back, and even rewound for a selection of a different view than selected the first time.
Referring to the drawings, in which like numerals represent like elements:
Turning to
Examples of an endpoint(s) include, but are not necessarily limited to, a computer or computers (including servers, applications servers, enterprise servers, desktop computers, laptops, netbooks, tablet computers (e.g., IP AD), a switch, mobile phones (e.g., including iPHONE and Android-based phones), networked televisions, networked watches, networked viewing devices 250, networked disc players, components in a cloud-computing network, or any other device or component of such device suitable for communicating information to and from the communication network 130. Endpoints may support Internet Protocol (IP) or other suitable communication protocols. In particular configurations, endpoints may additionally include a medium access control (MAC) and a physical layer (PHY) interface that conforms to IEEE 801.11. If the endpoint is a device, the device may have a device identifier such as the MAC address and may have a device profile that describes the device. In certain configurations, where the endpoint represents a device, such device may have a variety of applications or “apps” that can selectively communicate with certain other endpoints upon being activated.
The communication network 130 and links 115, 125 connected to the communication network 130 may include, but are not limited to, a public or private data network, a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a wireline or wireless network (e.g., WIFI, GSM, CDMA, LTE, WIMAX, BLUETOOTH or the like), a local, regional, or global communication network, portions of a cloud-computing network, a communication bus for components in a system, an optical network, a satellite network, an enterprise intranet, other suitable communication links, or any combination of the preceding. Yet additional methods of communications will become apparent to one of ordinary skill in the art after having read this specification. In particular configuration, information communicated between one endpoint and another may be communicated through a heterogeneous path using different types of communications. Additionally, certain information may travel from one endpoint to one or more intermediate endpoint before being relayed to a final endpoint. During such routing, select portions of the information may not be further routed. Additionally, an intermediate endpoint may add additional information.
Although an endpoint generally appears as being in a single location, the endpoint(s) may be geographically dispersed, for example, in cloud computing scenarios. In such cloud computing scenarios, an endpoint may shift hardware during back up. As used in this document, “each” may refer to each member of a set or each member of a subset of a set.
When the endpoints(s) 110, 120 communicate with one another, any of a variety of security schemes may be utilized. As an example, in particular embodiments, endpoint(s) 110 may represent a client and endpoint(s) 120 may represent a server in client-server architecture. The server and/or servers may host a website. And, the website may have a registration process whereby the user establishes a username and password to authenticate or log in to the website. The website may additionally utilize a web application for any particular application or feature that may need to be served up to website for use by the user.
According to particular embodiments, the imaging system 140 and controller 150 are configured to capture and process multiple video and/or audio data streams and/or still images. In particular configurations as will be described below, imaging system 140 comprises a plurality of low latency, high-resolution cameras, each of which is capable of capturing still images or video images and transmitting the captured images to controller 150. By way of example, in one embodiment, imaging system 140 may include eight (8) cameras, arranged in a ring, where each camera covers 45 degrees of arc, to thereby provide a complete 360-degree panoramic view. In another embodiment, imaging system 140 may include sixteen (16) cameras in a ring, where each camera covers 22.5 degrees of arc, to provide a 360-degree panoramic view.
In an example embodiment, one or more of the cameras in imaging system 140 may comprise a modification of an advanced digital camera, such as a LYTRO ILLUM™ camera (which captures multiple focal lengths at the same time), and may include control application that enable zooming and changing the focus, depth of field, and perspective, after a picture has already been captured. Additional information about the LYTRO ILLUM™ camera may be found at www.iytro.com. Yet other light field cameras may also be used. In particular embodiments, such light field cameras are used to capture successive images (as frames in a video) as opposed to one image at a time.
Either separate from or in conjunction with such cameras, a variety of microphones may capture audio emanating towards the sensors from different locations.
In certain embodiments, controller 150 is operable, in response to commands from endpoint 110, to capture video streams and/or still images from some or all of the cameras in imaging system 140. Controller 150 is further configured to join the separate images into a continuous panoramic image that may be selectively sent to endpoint 110 and subsequently relayed to endpoint 120 via communication network 130. In certain embodiments, capture from each of the cameras and microphones is continuous, with the controller sending select information commanded by the endpoint. As a non-limiting example, that will be described in more detail below, the endpoint may specify viewing from a focal point at a particular angle. Accordingly, the controller will stream and/or provide the information corresponding to that particular focal point and angle, which may include stitching of information from more than one particular camera and audio gathered from microphones capturing incoming audio.
In an advantageous embodiment, a user of endpoint 120 may enter mouse, keyboard, and/or joystick commands that endpoint 120 relays to endpoint 110 and controller 150. Controller 150 is operable to receive and to process the user inputs (i.e., mouse, keyboard, and/or joystick commands) and select portions of the continuous panoramic image to be transmitted back to endpoint 120 via endpoint 110 and communication network 130. Thus, the user of endpoint 110 is capable of rotating through the full 360-degree continuous panoramic image and can further examine portions of the continuous panoramic image in greater detail. For example, the user of endpoint 110 can selectively zoom one or more of the cameras in imaging system 140 and may change the focus, depth of field, and perspective, as noted above. Yet other more advanced methods of control will be described in greater detail below with reference to other figures.
Referring now to
The system of
In particular embodiments, multiple cameras may be pointed at the same location to enhance the focal point gathering at a particular angle. For example, a first light field camera may gather focal points for a first optimum range, a second light field camera may gather focal points at a second optimal range, and a third light field camera may gather focal points for a third optimal range. Thus, a user who chooses to change the focal point may receive information from different cameras. The same multiple cameras for multiple focal points may also be used in scenarios where non-light field camera are used, for example, instead using cameras with relatively fixed focal points and switching between cameras as a different focal point is used. In the switching between cameras of different focal points (using light field cameras or not), stitching may be used to allow a relatively seamless transition. In particular embodiments, such stitching may involve digitally zooming on frames of images (the video) and then switching to a different camera. To enhance such seamless stitching, a variety of image matching technologies may be utilized to determine optimal points at which to switch cameras.
In particular configurations, the capturing device 200 may be stationary. In other configurations, the capturing device may be mobile. As a non-limiting example, the capturing device 200 may be mounted on an air-borne drone or other air-borne device. As another example, the capturing device may be mounted on a remotely controlled vehicle to survey an area. As yet another example, the capturing device may be mounted on a suspended wire system that are typically used in sporting events such as football.
In some configurations, the surveillance—either airborne or not—may be of a dangerous area. As non-limiting examples, one or more capturing devices may be placed on a robot to monitor a hostage situation. One or more capturing devices may also be placed at crime scenes to capture the details that may later be played back and reviewed over and over for details.
Although one capturing device 200 has been shown, more than one capturing device 200 may exist with switching (and stitching) between such capturing devices 200. For example, as will be described below with reference to
The sensors 210 may be any suitable sensors configured to capture reflected light which, when combined, forms images or video. As a non-limiting example, as described above, modified LYTRA cameras may be utilized to capture light at multiple focal points over successive frames for video. In other embodiments, other types of cameras, including light field cameras, may also be utilized with cameras capturing different foci. In yet other embodiments, non-multiple-focus-at-the-same-time gathering cameras may also be used. That is, in other embodiments, cameras that have a particular focal point (as opposed to more than one) may be utilized.
Although the sensors 210 are generally shown as a single box, the box for the sensor 210 may represent a plurality of sensors that can capture multiple images. As a non-limiting example, a single LYTRA camera may be considered multiple sensors because of gathering light from multiple focal points.
In addition to light, the sensors 210 may capture audio from different angles. Any suitable audio sensor may be utilized. In particular embodiments, the audio—in similar fashion to the light sensors—may be directed to capture audio at different distances using different sensors.
The information captured by the capturing device 200 is sent to one or more servers 230 on a network 110. The one or more servers 230 can process the information for real-time relay for select portions to a viewing device 250. In alternative configurations, the one or more servers 230 can store the information for selective playback and/or rewind of information. As a non-limiting example, a viewer of a sports event may select particular view in a live stream and then rewind to watch a certain event multiple times to view such an event from different angles and/or focus.
In one particular configuration, the server 230 pieces together the various streams of information that have been sent from the capturing device 200 (or multiple capturing devices 200) that the viewing device 250 has requested. As a non-limiting example, the viewing device 250 may wish to view images or video (and audio) from a particular angle with a particular pitch at a particular focal point. The server 230 pulls the information the sensors 210 capturing such information and sends it to the viewing device 250. In some configurations, the relay of information may be real-time (or near real-time with a slight delay). In other configurations, the playback may be of information previously recorded. In addition to information switching from a particular capturing device 200 in particular configurations, the one or more servers 230 may also switch between different capturing devices 200 as will be describe with reference to
In particular configurations, the information may be stitched—meaning information from more than one sensor is sent. As a simple example, an angle between two or more cameras may be viewed. The information from such two or more cameras can be stitched to display a single view from such multiple sensors. In particular configurations, stitching may occur at the one or more servers 230. In other configurations, stitching may occur at the viewing device 250.
In particular configurations, the stream of information stitching and relaying may be analogous to a function performed by a human eye when incoming light is switched to focus on a particular light stream. When audio is combined to this light switching, the viewed information may take on appearance as though one were actually present at the same location as the capturing device 200. Other switching of information may be analogous to eye and/or head movement of a user. The applications of the viewing of information captured by capturing devices 200 are nearly unlimited. As non-limiting examples, the capturing devices 200 can be placed at select locations for events—whether they be sporting events, concerts, or lectures in a classroom. Doctors and physicians may also use mobile versions of capturing devices 200 to virtually visit a patient remotely. Police enforcement may also use mobile versions of the capturing devices 200 (or multiple ones) to survey dangerous areas. Yet additional non-limiting examples will be provided below.
Any of the above-referenced scenarios may be viewed in a real-time (or near real-time) or recorded playback scenario (or both). For example, in watching a sport event (real-time or not), a user may pause and rewind to watch the event from a different angle (or from a different capturing device 200) altogether. Police may view the scene—again—looking at clues from a different angle or focus than previously.
The one or more servers 240 represent additional information that may be displayed to a user. In one configuration, the one or more servers 240 may display an augmented reality. In yet other configurations, only information from the one or more servers 240 may be displayed.
The viewing device 250 may be any suitable device for displaying the information. Non-limiting examples include glasses, projected displays, holograms, mobile devices, televisions, and computer monitors. In yet other configurations, the viewing device 250 may be a contact lens placed in one eyes with micro-display information. The request (generally indicated by arrow 232) for return information 234 may be initiated in a variety of different manners—some of which are described below.
As a first non-limiting example, the viewing device 250 may be glasses that are opaque or not. The glasses may be mounted with accelerometers, gyroscopes, and a compass, or any other suitable device such an inertial measurement units (IMUs), to detect the direction one's head (or in some scenarios, eyes) is facing. Such detected information can switch toward the collection of information in a particular direction. To obtain a particular focus of information, one may use hand gestures (haptics) that are detected by the glasses. Alternatively, the glasses can include a sensor to detect whether the eye is searching for a different focus and switch to that particular focus. Other devices for switching the input to the glasses may also be utilized.
In other configurations, yet other detection mechanisms may be included using input devices or hand gestures. As a non-limiting example, Meta (www.getmeta.com) has developed glasses 300 with sensors to detect hand movement with respect to such glasses. Such glasses can be augmented to switch streams being captured (or previously captured) from one or more capturing devices. Other technologies using reflected waves, image analysis of hands with pre-sets for a particular skeletal make-up of a user, may also be utilized according to embodiments of the disclosure.
For other types of viewing devices 250 any suitable mechanism to switch the information stream may be utilized—including those mentioned above. For example, a standard tablet or smartphone can be moved around to view different views as though one were actually at the event. Accelerometers, gyroscopes, compasses and other tools on the smart phone may be used to detect orientation. Yet other components will be described below with reference to
In one particular configuration, the viewing device may be a band worn on the arm that projects a display onto one's arm.
In particular configurations, in addition to the information captured by the capturing device 200 being displayed, information from the one more servers 240 may be displayed to augment the remotely-captured real-time (or near real-time) or previously recorded reality. As a non-limiting example, one watching a sporting event may identify a particular player and inquire as to such a player's statistical history. One or a combination of the viewing device 250, the one or more servers 230, and/or the one or more servers 240 may utilize any suitable technology to determine what a particular user is viewing and also to detect the switch 260 requested by the viewing device. The requested information 242 returns the information 244. A verbal request 242 may be recognized by one or a combination of the viewing device 250, the one or more servers 230, and/or the one or more servers 240.
In other configurations, information may be automatically displayed—in an appropriate manner. For example, in a football game, a first down maker may be displayed at the appropriate location.
In yet other configurations, standard production overlays may be displayed over a virtual (e.g., score of the game, etc.). These can be toggled on or off.
As another example of use of information from both the one or more servers 230 and one or more servers 240, a professor may give a lecture on an engine with the professor, himself, viewing the engine as an augmented reality. The wearer of the glasses may view the same engine as an augmented remote reality—again recorded or real-time (or near real-time) with a choice of what to view.
In particular configurations, only information from the one or more servers 240 is utilized forming an “Internet Wall” of sorts to allow a viewer to look at information. In such a configuration, where the viewing device 250 is glasses 300, a user can view information over the internet through various windows. However, the initiation of such applications can effectively be a typing or gesturing in the air. Further details of this configuration will be described below with reference to
In such configurations as the preceding paragraph, there is little fear of one viewing over your shoulder. The user is the only one able to see the screen. Thus, for example, when in a restaurant or on a plane, there is little fear that one will see private conversations or correspondence.
As yet another example, a user may be wearing the glasses 300 while driving down the road and order ahead using a virtual menu displayed in front of him or her. The user may also authorize payment through the glasses. Non-limiting examples of payment authorization may be a password provided through the glasses, the glasses already recognizing the retina of the eye, or a pattern of the hand through the air. Thus, once the user arrives at a particular location, the food will be ready and the transaction will already have occurred.
The glasses 300 of this embodiment is shown as including the following components: display 310, head movement tracker 320, speakers 330, communication 340, geolocation 350, camera 360, focus detection 370, and other 380. Although particular components are shown in this embodiments, other embodiments may have more, fewer, or different amounts of components.
The display 310 component of the glasses 300 provides opaque and/or transparent display of information to a user. In particular configurations, the degree of transparency is configurable and changeable based on the desired use in a particular moment. For example, where the user is watching a sporting event or movie, the glasses can transform to an opaque or near-opaque configuration. In other configurations such as augmented reality scenarios, the glasses can transform to a partially transparent configuration to show the portion of the reality that needs to be seen and the amount of augmentation of that reality.
The speakers 330 component provide an audio output to a user. The audio may or may not correspond to the display 310 component.
In
The propagated signal detector 329 may use any technique used, for example, by mobile phones in detecting position, but on a more local and more precise scale in particular configurations. For example, the glasses 300 may be positioned in a room with a signal transmission that is detected by multiple propagated signal detectors 329. For example, knowing the position of three propagated signal detectors on the glasses and the relative time difference of their receipt of the signal, the three-dimensional relative position of the glasses 300 can be detected. Although three propagated signal detectors are referenced in the preceding sentence, more than three propagated signal detectors may be utilized to enhance confidence of location. Although the term “relative” is utilized, a configuration of glasses upon set-up will determine the relative location for setup.
The other 380 component include any standard components that are typical of smartphones today such as, but not limited to, processors, memory, and the like.
Referring now to
The camera 371 (which may be more than one camera) may either be the same or separate from the camera 360 discussed above. In particular embodiments, the camera 371 may be configured to detect movement of one's hand. As an example, the focus may change based on a particular hand gesture to show the focus is to change (e.g., pinching). Yet other hand gestures may also be used to change focus. The camera 371 may also be used to manipulate or change augmented objects placed in front of the glasses. For example, one may have a virtual rotating engine providing different view points. In particular embodiments, such different viewpoints may be from various cameras, for example, in sporting events or in a reconnaissance type scenario as described herein.
The eye detection 375 component may be used to detect where a user is looking for information—using a sensor such a camera or an autorefractor. In particular embodiments, a focus can change based on changing parameters of the eye as measured by a miniaturized autorefractor. Additionally, when an eye looks in a different direction, a camera can detect the “whites” of one's eye veering in a different direction. Although the eye detection 375 component is used in particular configurations, in other configurations, the other components maybe utilized.
The light emission and detection 373 component emits a light for detection of the reflection by the camera or other suitable light detection. A user may place a hand in front of these detectors with gestures such as moving in or moving out to indicate a change of focus. The light emission and detection 373 component and any associated detectors can also be used to determine the direction of one's focus or changing of camera.
As a first non-limiting example, capturing devices 600b and 600e may be positioned on the 50-yard line of a football field. Depending on where game play is occurring, a user may desire to switch capturing devices 600. Any suitable mechanism may be used. For example, a user may place both hands up in front of the glasses 300 and move them in one direction—left or right—to indicate movement between capturing devices. Such a movement may allow a pan switching between capturing device 600b to 600a or 600c. Another non-limiting example, a user may place both hands with a rotational movement to switch to the opposite side of the field, namely from capturing device 600b to 600e. A variety of other hand gestures should become apparent to one reviewing this disclosure.
In switching between one capturing device 600 to another, stitching may also be utilized to allow for relatively seamless transitions.
Next, referring to
The particular virtual screen 700 shown in
The virtual screen 700 may take on any of a variety of configurations such as, but not limited to, those provided by a smart phone or computer. Additionally, the virtual screen in particular embodiments may provide any content that a smart phone or computer can provide—in addition the other features described herein. For example, as referenced above, virtual augmented reality models can be provided in certain configurations. Additionally, the remote viewing of information gathered by, for example, one or more capturing devices 200 may also be displayed.
The following provides some non-limiting example configurations for use of the glasses 300 described with reference to
The glasses 300 may be provided to visitors of a movie studio; however, rather than the viewers of the movie studio viewing a movie on the big screen, they will be viewing the content they choose to view by interacting with the glasses 300. The content may be any event (such a sporting event, concert, or play). In addition to information from the event, the viewer may choose supplemental content (e.g., statistics for a player, songs for a musician, or other theatrical events for an actor). Alternatively, the content may be a movie shot from multiple perspectives to provide the viewer a completely new movie viewing experience.
The particular configuration in the preceding paragraph may assist with scenarios where a user does not have the particular bandwidth capacity needed, for example, at home to stream content (which in particular configurations can become bandwidth intensive). Additionally, in particular embodiments, all the data for a particular event may be delivered to the movie theater for local as opposed to remote streaming. And, portions of the content are locally streamed to each respective viewing device 250, such as glasses 300, (using wired or wireless configurations) based on a user's selection. Moreover, in the streaming process, intensive processing may take place to stitch as appropriate information gathered from different sources.
In scenarios where bandwidth is adequate, in particular scenarios, a user may be allowed to view the content from home—in an on-demand type scenario for any of the content discussed herein. As referenced above, in such scenarios, stitching (across focus, across cameras, and across capturing devices) may either occur locally or remotely. And, in some configurations, certain levels of pre-stitching may occur.
As another non-limiting example, a user may have received content from a personal drone that allows view from different elevated perspective.
For example, a golfer may place the glasses 300 on to view an overhead of a layout of the course for strategies in determining how best to proceed. In reconnaissance type scenarios, a single drone may provide a plurality of personnel “visuals” on a mission—with each person choosing perhaps different things they want to look at.
As another example, a user may place a capturing device 200 on his or her self in GO-PRO-style fashion to allow someone else to view a plurality of viewpoints that the user himself would not necessarily view. This information may either be stored locally or communicated in a wireless fashion.
As yet another example, students in a classroom may be allowed to take virtual notes on a subject with a pen that specifically interoperates with the glasses 300. In such a scenario, the cameras 360 and/or other components of the glasses can detect a particular plane in front of the glasses (e.g., a desk). Thus, a virtual keyboard can be displayed on the desk for typing. Alternatively, a virtual scratch pad can also be placed on the desk for creating notes with a pen. In such scenarios, a professor can also have a virtual object and/or notes appear on the desk. For example, where the professor is describing an engine, a virtual representation of the engine may show up on the desktop with the professor controlling what is being seen. The user may be allowed to create his or her own notes on the engine with limited control provided by the professor.
As yet another example, deaf people can have a real-time speech-to-text input of interpreted spoken content displayed. Blind people can have an audio representation of an object in front of the glasses 300—with certain frequencies and/or pitches being displayed for certain distances of the object.
As yet another example, a K-9 robot device can be created with capturing devices mounted to a patrol unit used for security—with audio and visual views much greater than any human or animal. If any suspicious activity is detected in any direction, an alert can be created with enhanced viewing as to the particular location of the particular activity. For example, the K-9 device can be programmed to move toward the suspicious activity.
As yet another example, one giving a speech can be allowed access to his or her notes to operate in a virtual teleprompter type manner.
As yet another example, the glasses 300 may have image recognition type capabilities to allow recognition of a person—followed by a pulling up of information about the person in an augmented display. Such image recognition may tap into any algorithms for example, used by Facebook, in the tagging of different types of people. As a non-limiting example, such algorithms use characteristics such as space between facial features (such as eyes) to detect a unique signature for a person.
As yet another example, the glasses 300 may display a user's social profile page, which may be connected to more than one social profile like Google+, Facebook, Instagram, and Twitter.
Embodiments of the present disclosure may include programs that may be stored in the RAM 914, the ROM 916 or the disk drives 922 and may be executed by the processor 912 in order to carry out functions described herein. The communications link 928 may be connected to a computer network or a variety of other communicative platforms including, but not limited to, a public or private data network; a local area network (LAN); a metropolitan area network (MAN); a wide area network (WAN); a wireline or wireless network; a local, regional, or global communication network; an optical network; a satellite network; an enterprise intranet; other suitable communication links; or any combination of the preceding. Disk drives 922 may include a variety of types of storage media such as, for example, floppy disk drives, hard disk drives, CD ROM drives, DVD ROM drives, magnetic tape drives or other suitable storage media. Although this embodiment employs a plurality of disk drives 922, a single disk drive 922 may be used without departing from the scope of the disclosure.
Although
The logic may also be embedded within any other suitable medium without departing from the scope of the disclosure.
The rounded, double-border rectangles represent the L/R images where the left image 1010 and right image 1020 use different sensors to create the stereopsis effect of a 3D environment. In humans, and many other animals, the separation of two eyes generates a 3D depth to an image, also referred to as binocular vision. In this embodiment, the separate collection of a left image 1010 and right image 1020 using different sensor 1000 clusters creates two streams of data. For the purposes of illustration:
This slight variation in sensor data recreates the eye separation of human users and presents two slightly different concurrent images 1030 to the user. When viewed together, the user's mind generates a 3D anaglyph.
If the user holds the viewing device 250 in place, and no other switches 260 are signaled, the L/R images will continue to collect recursively from the same sensors 1000.
In one embodiment, the cluster of sensors 1000 for the R/L images 1010,1020 are stitched from the entire image generated by the sensors employed. This eliminates processor time required to trim the images 1030 before the transfer of the two images 1030 to the viewing device 250. The result is a reduction in the display server's 1290 memory 914, processor 912, and communication link 928 requirements.
The processor on the display server 1290 records the address of each sensor 1050 held in memory 914 where the address points to sensor files stored on the ROM 916 and available for rendering on the viewing device 250. As the display server 1290 flushes the memory 1270, the address is deleted and the sensor feed 210 is removed from memory. Concurrently, the server 1290 establishes a new memory address for the added adjacent sensors 1050 and supplies that stored image 1030 to the RAM 914. The memory 914 may represent multiple memory units and may be located remotely. Additional memory hardware may preload adjacent sensors 1050 predictively for rapid transfer to the central rendering memory.
As in
The system allows live production by pulling the original sensors 1000 based on the viewing device's 250 switch 1230, stitching them in memory 914, pitching the image to the viewing device 1260, and then discarding 1270 sensor feeds 210 when they are no longer needed to render the image 1030. In parallel, a structure file may improve the render by advising on object distances, opacity, size, clarity, etc.
With reference to
Finally, turning to
The flowchart in
Although the present disclosure and its advantages have been described in detail, it should be understood that various changes, substitutions and alterations can be made herein without departing from the spirit and scope of the disclosure as defined by the appended claims. Moreover, the scope of the present application is not intended to be limited to the particular embodiments of the process, machine, manufacture, composition of matter, means, methods and steps described in the specification. As one of ordinary skill in the art will readily appreciate from the disclosure, processes, machines, manufacture, compositions of matter, means, methods, or steps, presently existing or later to be developed that perform substantially the same function or achieve substantially the same result as the corresponding embodiments described herein may be utilized according to the present disclosure. Accordingly, the appended claims are intended to include within their scope such processes, machines, manufacture, compositions of matter, means, methods, or steps.
In the foregoing description, and the following claims, method steps and/or actions are described in a particular order for the purposes of illustration. It should be appreciated that in alternate embodiments, the method steps and/or actions may be performed in a different order than that described. Additionally, the methods described above may be embodied in machine-executable instructions stored on one or more machine-readable mediums, such as disk drives, thumb drives or CD-ROMs. The instructions may be used to cause the machine (e.g., computer processor) programmed with the instructions to perform the method. Alternatively, the methods may be performed by a combination of hardware and software. While illustrative and presently preferred embodiments of the invention have been described in detail herein, it is to be understood that the inventive concepts may be otherwise variously embodied and employed, and that the appended claims are intended to be construed to include such variations, except as limited by the prior art.
Benefits, other advantages, and solutions to problems have been described herein with regard to specific embodiments. However, the advantages, associated benefits, specific solutions to problems, and any element(s) that may cause any benefit, advantage, or solution to occur or become more pronounced are not to be construed as critical, required, or essential features or elements of any or all the claims of the invention. As used herein, the terms “comprises”, “comprising”, or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus composed of a list of elements that may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
From the description above, a number of advantages become evident for the “System and Method for Reducing System Requirements for a Virtual Reality 360 Display.” The present invention provides all new benefits for participating parties including the user and provider:
This application is a Continuation-In-Part of pending U.S. patent application Ser. No. 14/812,880 filed Apr. 24, 2015, for which priority is claimed and is incorporated herein by reference in its entirety; and claims the benefit of U.S. Provisional Patent Applications Ser. No. 62/156,266 filed May 3, 2015 and 62/031,437 filed Jul. 31, 2014.
| Number | Date | Country | |
|---|---|---|---|
| 62156266 | May 2015 | US | |
| 62031437 | Jul 2014 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 14812880 | Jul 2015 | US |
| Child | 15680067 | US |
