Patent Application
20180365897
Augmented reality or “mixed” reality technologies have been incorporated into many diverse applications across a variety of industries, including gaming, enterprise, design, and gaming, amongst others. These technologies have been incorporated into various product offerings to enhance user experience and productivity, and to provide users with capabilities not realized with previous technology offerings. One particular area of enhancement facilitated by these technologies relates to environment visualization and collaboration within those environments.
Augmented reality is a technology that essentially superimposes a computer-generated image onto a user's real-world view, such that a user of the technology can perceive a composite view including their perceived real-world view overlaid by the computer-generated image. With respect to environment visualization, augmented reality has enabled users to view virtual three-dimensional textured objects within a real-world environment. Further, augmented reality devices have enabled the sending and receiving of these three-dimensional objects between devices at different scales. Conventional systems, however, do not provide intuitive methods for capturing three-dimensional objects and segmenting them from their environment for storage or communication.
This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used in isolation as an aid in determining the scope of the claimed subject matter.
Embodiments of the technology described herein are directed towards capturing, storing, communicating, and/or presenting virtual representations of real-world spaces and objects in various augmented reality (AR) environments. Further, the virtual representations of the real-world spaces and objects can maintain any number of physical properties as presented in an AR environment. As such, embodiments of the technology provide techniques for capturing, and at a later time regenerating, real-world spaces and/or objects in an AR environment. Further, other embodiments of the technology described herein can facilitate collaboration in such AR environments, for example by sending and receiving virtual representations of real-world spaces and objects between devices. While the present disclosure primarily references AR as the embodied technology, other technologies such as mixed reality, holography, and virtual reality, are technologies that remain within the purview of the present disclosure.
According to some embodiments, a head-mounted display (HMD) device worn by a user, can scan a physical environment (e.g., a room) at a first location, also described herein as a source environment, and further generate a first collection of three-dimensional datasets that corresponds to the physical environment at the first location. The collection of three-dimensional datasets can be generated, for example, based on optical information obtained by one or more optical input devices coupled to the HMD. Further, the three-dimensional datasets can include detected physical properties associated with the physical environment at the first location, for example geometric, dimensional, spatial-relation, or chromatic properties. The HMD can store the datasets either locally or at a remote device (e.g., the cloud). At a second location, the HMD can re-create the source environment, by rendering for display a virtual representation of the source environment based on the collection of three-dimensional datasets, where the virtual representation of the source environment is rendered to maintain any combination of the detected physical properties associated with the physical environment. Further, the HMD can enable a user to view and scan one or more physical objects at the second location, within the virtual representation of the source environment, such that the physical object(s) are perceived to be within the source environment. To this end, a user can view a true to life comparison of the physical objects at the second location relative to the source environment, by viewing the physical objects within the virtual representation of the source environment.
In some further embodiments, the HMD can scan one or more physical objects at the second location to generate at least a second collection of three-dimensional datasets that corresponds to at least a first physical object at the second location. The collection of three-dimensional datasets can be generated, for example, based on optical information obtained by one or more optical input devices coupled to the HMD. The three-dimensional datasets corresponding to the physical object(s) can include detected physical properties associated with the physical object(s) at the second location. The HMD can then store the datasets corresponding to the physical object(s) either locally or at the remote device. Based on the collection of three-dimensional datasets, the HMD can re-create the physical object(s), and render for display a virtual representation of the physical object(s), where the virtual representation of the physical object(s) is rendered to maintain one or more detected physical properties associated with the physical object(s). In this regard, the HMD can enable a user to view one or more virtual representations of physical objects within the virtual representation of the source environment. Moreover, a user can view a true to life comparison of the physical objects at the second location relative to the source environment, by viewing the virtual representations of physical objects within the virtual representation of the source environment.
In some further embodiments, the stored three-dimensional datasets can be received and/or retrieved by the HMD, such that any number or combination of virtual representations of physical objects can be viewed at another location, such as the first location (i.e. source environment). Accordingly, the HMD can re-create a physical object, by rendering for display a virtual representation of the physical object while maintaining one or more detected physical properties associated with the physical object. As such, the HMD can enable a user to view one or more virtual representations of physical objects, from a second location, within the source environment of the first location. To this end, a user can view a true to life comparison of the physical objects at the second location relative to the source environment, by viewing the virtual representation(s) of physical object(s) within the source environment.
Additional objects, advantages, and novel features of the invention will be set forth in part in the description which follows, and in part will become apparent to those skilled in the art upon examination of the following, or can be learned by practice of the invention.
Aspects of the technology presented herein are described in detail below with reference to the attached drawing figures, wherein:
The subject matter of aspects of the present disclosure is described with specificity herein to meet statutory requirements. However, the description itself is not intended to limit the scope of this patent. Rather, the inventors have contemplated that the claimed subject matter might also be embodied in other ways, to include different steps or combinations of steps similar to the ones described in this document, in conjunction with other present or future technologies. Moreover, although the terms “step” and/or “block” can be used herein to connote different elements of methods employed, the terms should not be interpreted as implying any particular order among or between various steps herein disclosed unless and except when the order of individual steps is explicitly described.
As used herein, the term “virtual object” (VO) can refer to any computer-generated object or any computer-generated visualization of a physical or a non-physical object. Such computer-generated objects (or computer-generated visualizations thereof) can be one-dimensional (1D) objects, two-dimensional (2D), or three-dimensional (3D) objects (or visualizations thereof). An “environment” can refer to any physical (e.g., “real-world”) or non-physical (e.g., “computer-generated” or “virtual”) environment, or any combination of such environments. Such environments include, but are not limited to, virtual-reality environments, augmented-reality environments, and/or mixed-reality environments. While the present disclosure primarily references AR as the embodied technology, other technologies such as mixed reality, holography, and virtual reality, are technologies that remain within the purview of the present disclosure.
According to some aspects of the technology described herein, wearable display devices, for example head-mounted display (HMD) devices or HMDs, can be equipped with one or more optical input devices (e.g. a camera) to enable accumulation of three-dimensional data (e.g. 3D maps, depth maps) representing the geometry of a user's environment and/or objects therein. The three-dimensional data can represent any number of environments (e.g. real-world spaces) at a particular location, including any number of objects therein. The environments and/or objects can then be presented to a user of a display device as one or more virtual representations of the real-world spaces and objects. Accordingly, such display devices can further present the virtual representations accurately with respect to their geometry and/or appearance. In circumstances where a real-world space or object is obstructed by another object (e.g., as a device generates the three-dimensional data), the space or object can be extrapolated according to the visible portions, in some embodiments. Moreover, the virtual representations can be normalized according to spatial illumination and relationships. The three-dimensional data can then be stored on a device or on a remote server, by way of example.
According to some aspects, the display devices can send and receive virtual objects or representations to and from other display devices (e.g. HMDs), respectively, to facilitate collaboration on visual data in AR environments. In other words, a HMD in accordance with some embodiments described herein can share virtual representations of real-world spaces and objects to HMDs of collaborating AR users. In some aspects, the sharing of real-world spaces and objects between HMD users located at different locations is contemplated as an advantage of the present disclosure. In some embodiments, the virtual representations can be displayed at full size (e.g., a 1:1 aspect ratio) or, in other embodiments, miniaturized to a consistent scale (e.g. a 20:1 aspect ratio) to fit into the real-world environment of a user. In this way, virtual representations of spaces and objects can be viewed via a HMD such that the virtual representations maintain true to their real-world counterparts, and to some degree, maintain a sense of realism by way of its rendered physical properties (e.g. size, geometry, appearance). In some aspects, a user can specify one or more virtual representations (e.g. real-world space and/or objects) to send, via a first HMD, to a second HMD of another user. A receiving HMD can receive the virtual representations at its respective location, such that the virtual representations can be viewed, via the receiving HMD, having similar or same properties of their real-world counterparts.
According to some embodiments, implementations of scanning, generating, rendering, viewing, sending, and receiving three-dimensional virtual spaces and objects (including both geometry and appearance) via a HMD is described. In some embodiments, a user operating a head-mounted display (HMD) device can scan an environment with the HMD device (e.g., via its camera), which in some embodiments, can further segment the scanned environment to generate one or more virtual objects that together represent a portion or entirety of the scanned environment. A user can then specify, via a first HMD, one or more virtual objects from the scanned environment to share with a second HMD (e.g., of another user). It will be appreciated that more than one object can be selected by the HMD from one or more environments to store or share with another HMD. As a user scans their environment, the HMD device can recognize the appearance and physical properties of both the physical environment and the physical objects therein. In some embodiments, objects or parts of objects that cannot be directly scanned (e.g., are at least partially obstructed) can be automatically completed by the HMD device according to the visible areas of the scanned environment and relationships between objects detected therein. Any one object's appearance can be normalized based on a scanned environment's detected illumination and its spatial relationships to other objects within the environment. An object from the scanned environment can be stored by a first HMD, recreated at a second location by the first HMD, or sent to a second HMD at a second location to be recreated by the second HMD. The receiving HMD, within a receiving environment (e.g., the physical environment of a second location) can then display the received object(s) at full-size or with reduced dimensions (e.g., miniaturized). Further the user at the second location can employ the second HMD to view the object at various locations within the receiving environment, and in some aspects, can be adaptively rendered to change its appearance based on a detected illumination of the receiving environment. In some instances, a HMD device can automatically position a received virtual object so that when rendered for display, will not intersect, overlap, or obstruct any physical object(s) physically present in the receiving environment. In this regard, in some embodiments virtual objects can be automatically positioned or aligned according to any detected physical properties of a receiving environment during a scan thereof by the receiving HMD.
Referring now to the figures, with reference to
Among other components not shown, example operating environment 100 includes at least two user devices, such as HMD devices 110 and 120; a network 130; at least one server 140; and at least one data source 150. It should be understood that environment 100 shown in
It should be understood that any number of user devices, servers, and data sources can be employed within operating environment 100 within the scope of the present disclosure. Each can comprise a single device or multiple devices cooperating in a distributed environment. For instance, server 140 can be provided via multiple devices arranged in a distributed environment that collectively provide the functionality described herein. Additionally, other components not shown can also be included within the distributed environment.
User devices 110 and 120 can be client devices on the client-side of operating environment 100, while server 140 can be on the server-side of operating environment 100. Server 140 can comprise server-side software designed to work in conjunction with client-side software on user devices 110 and 120 so as to implement any combination of the features and functionalities discussed in the present disclosure. This division of operating environment 100 is provided to illustrate one example of a suitable environment, and there is no requirement for each implementation that any combination of server 140 and user devices 110 and 120 remain as separate entities.
User devices 110 and 120 can comprise any type of computing device capable of use by a user. For example, in one embodiment, user devices 110 and 120 can be the type of computing device described in relation to
Data storage 150 can comprise data sources and/or data systems, which are configured to make data available to any of the various constituents of operating environment 100, or systems 200, 300, 400 described in connection to
Operating environment 100 can be utilized to implement one or more of the components of system 200, described in
In some embodiments, the functions performed by components of system 200 can be associated with one or more HMD applications, services, or routines. In particular, such applications, services, or routines can operate on one or more user devices (such as user device 102), servers (such as server 140), and/or components thereof, and/or can be distributed across one or more user devices and servers, or be implemented in the cloud. Moreover, in some embodiments, the components of system 200 can be distributed across a network, including one or more servers (such as server 140) and client devices (such as user device 110, 120), in the cloud, or can reside on a user device, such as user device 110, 120. Moreover, these components, functions performed by these components, or services carried out by these components can be implemented at appropriate abstraction layer(s), such as the operating system layer, application layer, hardware layer, of the computing system(s). Alternatively, or in addition, the functionality of these components and/or the embodiments described herein can be performed, at least in part, by one or more hardware logic components. For example, and without limitation, illustrative types of hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Application-specific Integrated Circuits (ASICs), Application-specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), and Complex Programmable Logic Devices (CPLDs). Additionally, although functionality is described herein with regards to specific components shown in example system 200, it is contemplated that in some embodiments functionality of these components can be shared or distributed across other components.
Continuing with
In some embodiments, the HMD 210 includes at least one processor included in computing component 230. The processor can include any type of computing processor, including a graphics processor. The processor can be employed by the HMD to perform any of the functions or operations described herein, and can further render for display, holographic data obtained by the HMD 210. The HMD 210 can employ a display component, such as display component 232, that can display and/or project computer-generated objects to be perceived as an augmented object positioned within the real-world, by a user of the HMD. In accordance with embodiments described herein, the display component 220 can comprise a see-through display (e.g., a waveguide), a virtual retinal display, prisms, or any other photonic technologies operable to facilitate a mixed-reality experience for a user.
In some embodiments, the HMD 210 includes a communication component 234 that can facilitate communication with a remote server device, such as server 140 of
According to some aspects of the present technology, the HMD device 210 includes one or more scanning components, such as 212, 214 of
The HMD device 210 can include a source environment re-creation component 216, to render for display a virtual representation of a physical environment based on one or more generated collections of three-dimensional datasets, such as the three-dimensional datasets generated by the source environment scanning component 212. In some aspects, the virtual representation of the physical environment maintains the various physical properties detected by the source environment scanning component 212, which can include any number of physical properties, such as dimensional properties, relative properties (e.g., relative positions), color, and/or texture. The source environment re-creation component 216 is generally responsible for creating a virtual representation of a scanned physical environment at a first location for display at a second location that is different than the first location. In some instances, the virtual representation can be rendered at a 1:1 aspect ratio.
Once a virtual representation of an environment is generated, a segmentation component 228 can automatically segment the virtual environment into one or more virtual objects. The segmented virtual objects can in some instances correspond to the physical (real-world) objects in the source environment. In some aspects the HMD device 210 can present for display one or more segmentation options for selection by a user via one or more input commands. The segmentation of a virtual environment into individual virtual objects can then facilitates the transfer and manipulation of those objects in the virtual environments.
The HMD device 210 can further include a destination object scanning component 214. The destination object scanning component 214 can be configured to generate a collection of three-dimensional datasets that corresponds to a physical object (e.g., a real-world object) within a physical environment. The three-dimensional datasets can be generated based on optical information obtained via an optical input device coupled to the HMD device 210. The generated three-dimensional datasets can include detected physical properties associated with the physical object, such as dimension, color, texture, and the like. The destination object scanning component can operate in conjunction with a destination object selection component 218, which can enable the selection of a physical object within a physical environment to be viewed on the display of the HMD device 210, based on a received input. The physical object can be selected and then rendered for display by the HMD device 210 to be perceived as being located within a virtual representation of an environment, for example, the virtual representation of a physical environment rendered for display by the source environment re-creation component 216.
The HMD device 210 can further include an object re-creation component, for example a destination object re-creation component 220 or a physical object re-creation component. The object re-creation component can render for display on the HMD device 210 a virtual representation of a physical object based on a generated collection of three-dimensional datasets, for example, datasets acquired by the destination object scanning component 214. The virtual representation of the physical object can be rendered to maintain any number of physical properties associated with the physical object, in accordance with some embodiments described herein. The destination object re-creation component can retrieve any portion of any of the generated collections of three-dimensional datasets to render for display. For example, a second generated collection of three-dimensional datasets may be stored that corresponds to the physical object scanned by the destination object scanning component 214. The destination object re-creation component can retrieve the stored collection of three-dimensional datasets and render for display, at the first location and via the HMD, the second virtual representation of the selected physical object.
In various aspects of the technology described herein, the HMD device 210 can further include any number of virtual representation components, for example, a virtual representation positioning component 222, a virtual representation miniaturization component 224, and an occlusion adjustment component 226. In an embodiment, a virtual representation positioning component 222 can determine a position to render a virtual object either within a physical environment or a virtual representation of that physical environment.
In some embodiments, a virtual representation miniaturization component 224 can be implemented to render a virtual environment or virtual object with reduced dimensions, to facilitate a top down view of the virtual environment, by way of a non-limiting example. In this way a user can select one or more virtual objects to place within a miniature virtual environment at the same reduced scale as the selected virtual object(s) to view the virtual object(s) within the virtual environment at a high level, and in some further embodiments, manipulate (e.g., move, reposition, rotate, remote, swap) the virtual objects within the virtual environment.
In some further embodiments, the HMD device 210 can also include an occlusion adjustment component 226 configured to modify a displayed virtual representation (e.g. a virtual environment or virtual object). The occlusion adjustment component 226 can modify or change the illumination (e.g. ambient occlusion) of a virtual object or environment or modify a visible portion of a virtual object. For example, the illumination of a virtual object generated at a second location can be modified to match the illumination (e.g., lighting characteristics) of a virtual environment generated at a first location. Further, portions of a virtual object can be modified as it is manipulated within a virtual environment. For example, a virtual representation of a chair can be made to look as if it were at least partially underneath a virtual representation of a table.
Turning now to
Looking now at
In some aspects, a user 310 can select which physical object 340, 342 to be viewed in the virtual environment 305′. For example, the HMD device 320 can include a physical object selection component so that based on some received input by the HMD device 320 the selected physical object 340, 342 can be perceived in the virtual environment 305′. In some aspects the user 310 can select one or more physical objects 340, 342 to be included in the augmented reality (AR) environment. In some aspects, a user 310 can hide one or more physical objects 340, 342 such that they are not perceived in the virtual environment 305′. In this way, while located in the second physical environment 300B, a user 310 can view or otherwise incorporate only those physical objects 340, 342 they wish to see and obstruct or otherwise hide any object they do not wish to view. In some instances an HMD 320 may view or hide one or more physical objects based on a user's 310 input, for example via the occlusion adjustment component 226 of
Looking back at
As a user 310 is viewing the physical object 342 at the second location 300B, the HMD device 320 can scan the physical object 342 via an object scanning component (e.g. a destination object scanning component 214 of
Referring now to
Turning now to
Turning to
Other arrangements and elements (e.g., machines, interfaces, functions, orders, and groupings of functions, etc.) can be used in addition to or instead of those shown, and some elements can be omitted altogether. Further, many of the elements described herein are functional entities that can be implemented as discrete or distributed components or in conjunction with other components, and in any suitable combination and location. Various functions described herein as being performed by one or more entities can be carried out by hardware, firmware, and/or software. For instance, various functions can be carried out by a processor executing instructions stored in memory.
Any one or more augmented-reality images 440 (e.g. virtual images) are reflected by the display component 414 toward a first user's eye, as exemplified by a light ray 416, so that the first user sees an image 418. In the image 418, a portion of the real-world scene 450 is visible along with any number of augmented-reality images 440.
Turning now to
At step 510, a HMD device at a first location can generate a first collection of three dimensional datasets based on optical input data obtained by an optical input device that is in operable communication with the HMD device. The HMD device can obtain the optical input data through, for example, dynamic scanning. The first collection of three-dimensional datasets corresponds to a first physical environment at the first location and can include detected physical properties associated with the first physical environment. The detected physical properties can include, for example, color and shading, dimensional properties, geometric relationships, or any other spatial or optical and/or chromatic properties.
At step 520, the HMD device can segment one or more identified physical objects of the first physical environment from the generated first collection of three-dimensional datasets. The segmenting can be based on any number of known techniques, including determined correlations to known objects from a dataset of known objects, determined indications of strong geometry and/or features, and a completion of smooth geometry to the generated first collections of three-dimensional datasets.
At step 530, the HMD device at a second location can provide for display a virtual representation of the first physical environment based on the first generated collection of three-dimensional datasets. The virtual representation can be displayed to maintain any of the detected physical properties associated with the first physical environment. In this way, while at a second location, a user can view, via the HMD device, a virtual representation of the environment at the first location. In some embodiments, the virtual representation has the same dimensional and spatial properties as the first physical environment (i.e. source environment). In some further embodiments, the virtual representation has the same illumination properties as the first physical environment.
At step 540, the HMD device at the second location can generate a second collection of three dimensional datasets based on optical input data obtained by an optical input device that is in operable communication with the HMD device. The HMD device can obtain the optical input data through, for example, dynamic scanning. The second collection of three-dimensional datasets can correspond to one or more physical objects located in a second physical environment at the second location. The second collection of three-dimensional datasets can include detected physical properties associated with the physical object. The detected physical properties can include, for example, color and shading, dimensional properties, geometric relationships, or any other spatial or optical properties. Accordingly, as the first virtual representation is being displayed (e.g. at step 530), the HMD device enables a user to view one or more physical objects within the displayed virtual environment. While the physical object is being viewed within the displayed virtual environment, the HMD can scan the physical object and generate the above-described second collection of three-dimensional datasets. It will be appreciated that the HMD device can scan any number of physical objects at multiple locations, thus generating multiple collections of three-dimensional datasets, each corresponding to a physical object.
At step 550, the HMD device can provide for display, at the first location, a second virtual representation of the physical object based on the second generated collection of three-dimensional datasets. The second virtual representation (e.g. virtual object) can be displayed to maintain any number of detected physical properties associated with the physical object, for example any geometric or dimensional properties. In some embodiments, the illumination of the second virtual representation can be altered, for example by the HMD device, to match that of the first location. In this way, the second virtual representation can be viewed under the illumination properties of the first location. In some embodiments, any number of virtual objects can be displayed at the first location, and further manipulated within the augmented reality environment.
Turning now to
At step 620, the HMD device at a second location can provide for display the generated first three-dimensional model representative of the first physical environment. The first three-dimensional model can be displayed to maintain any of the detected physical properties associated with the first physical environment. Further, the generated first three-dimensional model can be displayed such that a user perceives the model within a second physical environment at the second location. In some further embodiments, the generated first three-dimensional model is displayed as having the same illumination properties as the first physical environment at the first location.
Based on the generated first three-dimensional model being displayed within the second physical environment, at step 630, the HMD device can generate a second three-dimensional model representative of a physical object located in the second physical environment based on optical input data obtained and/or received by the optical input device. In some embodiments, the generated second three-dimensional model maintains any number of physical properties associated with the physical object located in the second physical environment.
At step 640, the HMD device can provide for display the second three-dimensional model representative of the physical object such that the user perceives the second three-dimensional model within the first physical environment when at the first location. In some embodiments, the HMD device can provide for display the second three-dimensional model representative of the physical object such that the user perceives the second three-dimensional model within the generated first three-dimensional model representative of the first physical environment, for example, when at the second location or a third location.
At step 650, the HMD device can transmit to a second HMD device (either directly or via a remote server device) at the first location, the generated second three-dimensional model representative of the physical object. At step 660, the second HMD device can provide for display the second three-dimensional model, such that the user perceives the second three-dimensional model within the first physical environment and under the illumination of the first physical environment.
Accordingly, various aspects of technology for facilitating intuitive collaboration on visual data in an augmented reality environment is described. It is understood that various features, sub-combinations, and modifications of the embodiments described herein are of utility and can be employed in other embodiments without reference to other features or sub-combinations. Moreover, the order and sequences of steps shown in the example methods 500 and 600 are not meant to limit the scope of the present invention in any way, and in fact, the steps can occur in a variety of different sequences within embodiments hereof. Such variations and combinations thereof are also contemplated to be within the scope of embodiments of the invention.
Having described various embodiments of the invention, an exemplary computing environment suitable for implementing embodiments of the invention is now described. With reference to
Embodiments of the invention can be described in the general context of computer code or machine-useable instructions, including computer-useable or computer-executable instructions, such as program modules, being executed by a computer or other machine, such as a personal data assistant, a smartphone, a tablet PC, or other handheld device. Generally, program modules, including routines, programs, objects, components, data structures, and the like, refer to code that performs particular tasks or implements particular abstract data types. Embodiments of the invention can be practiced in a variety of system configurations, including handheld devices, consumer electronics, general-purpose computers, more specialty computing devices. Embodiments of the invention can also be practiced in distributed computing environments where tasks are performed by remote-processing devices that are linked through a communications network. In a distributed computing environment, program modules can be located in both local and remote computer storage media including memory storage devices.
With reference to
Computing device 700 typically includes a variety of computer-readable media. Computer-readable media can be any available media that can be accessed by computing device 700 and includes both volatile and nonvolatile, removable and non-removable media. By way of example, and not limitation, computer-readable media can comprise computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVDs) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by computing device 700. Computer storage media does not comprise signals per se. Communication media typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media, such as a wired network or direct-wired connection, and wireless media, such as acoustic, RF, infrared, and other wireless media. Combinations of any of the above should also be included within the scope of computer-readable media.
Memory 712 includes computer storage media in the form of volatile and/or nonvolatile memory. The memory can be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard drives, optical-disc drives. Computing device 700 includes one or more processors 714 that read data from various entities such as memory 712 or I/O components 720. Presentation component(s) 716 presents data indications to a user or other device. Exemplary presentation components include a display device, speaker, printing component, vibrating component, and the like.
The I/O ports 718 allow computing device 700 to be logically coupled to other devices, including I/O components 720, some of which can be built in. Illustrative components include a microphone, joystick, game pad, satellite dish, scanner, printer, wireless device. The I/O components 720 can provide a natural user interface (NUI) that processes air gestures, voice, or other physiological inputs generated by a user. In some instances, inputs can be transmitted to an appropriate network element for further processing. An NUI can implement any combination of speech recognition, touch and stylus recognition, facial recognition, biometric recognition, gesture recognition both on screen and adjacent to the screen, air gestures, head and eye tracking, and touch recognition associated with displays on the computing device 700. The computing device 700 can be equipped with depth cameras, such as stereoscopic camera systems, infrared camera systems, RGB camera systems, and combinations of these, for gesture detection and recognition. Additionally, the computing device 700 can be equipped with accelerometers, gyroscopes, magnetometers, and the like that enable detection of motion, position, rotation, and/or orientation. The output of the accelerometers, gyroscopes, or magnetometers can be provided to the display of the computing device 700 to render immersive augmented reality, mixed-reality, or virtual reality.
Some embodiments of computing device 700 can include one or more radio(s) 724 (or similar wireless communication components). The radio 724 transmits and receives radio or wireless communications. The computing device 700 can be a wireless terminal adapted to receive communications and media over various wireless networks. Computing device 700 can communicate via wireless protocols, such as code division multiple access (“CDMA”), global system for mobiles (“GSM”), or time division multiple access (“TDMA”), as well as others, to communicate with other devices. The radio communications can be a short-range connection, a long-range connection, or a combination of both a short-range and a long-range wireless telecommunications connection. When we refer to “short” and “long” types of connections, we do not mean to refer to the spatial relation between two devices. Instead, we are generally referring to short range and long range as different categories, or types, of connections (i.e., a primary connection and a secondary connection). A short-range connection can include, by way of example and not limitation, a Wi-Fi® connection to a device (e.g., mobile hotspot) that provides access to a wireless communications network, such as a WLAN connection using the 802.11 protocol; a Bluetooth connection to another computing device is a second example of a short-range connection, or a near-field communication connection. A long-range connection can include a connection using, by way of example and not limitation, one or more of CDMA, GPRS, GSM, TDMA, and 802.16 protocols.
Many different arrangements of the various components depicted, as well as components not shown, are possible without departing from the scope of the claims below. Embodiments of the present invention have been described with the intent to be illustrative rather than restrictive. Alternative embodiments will become apparent to readers of this disclosure after and because of reading it. Alternative means of implementing the aforementioned can be completed without departing from the scope of the claims below. Certain features and sub-combinations are of utility and can be employed without reference to other features and sub-combinations and are contemplated within the scope of the claims.
