Patent Application
20230045759
The present disclosure is directed to configuring display modes of a three-dimensional (3D) call and providing affordances for improving image capturing in a 3D call.
Video conferencing has become a major way people connect. From work calls to virtual happy hours, webinars to online theater, people feel more connected when they can see other participants, bringing them closer to an in-person experience. However, video calls remain a pale imitation of face-to-face interactions. Understanding body language and context can be difficult with only a two-dimensional (“2D”) representation of a sender. Further, interpersonal interactions with video are severely limited as communication often relies on relational movements between participants.
Some artificial reality systems may provide the ability for users to engage in 3D calls, where a call participant can see a 3D representation of one or more other call participants. In such 3D calls, users can experience interactions that more closely mimic face-to-face interactions. For example, an artificial reality device can include a camera array that captures images of a sending call participant, reconstructs a hologram (3D model) representation of the sending call participant, encodes the hologram for delivery to an artificial reality device of a recipient call participant, which decodes and displays the hologram as a 3D model in the artificial reality environment of the recipient call participant. This may allow the recipient call participant to mover around the hologram, seeing it and interacting with it from different angles.
The techniques introduced here may be better understood by referring to the following Detailed Description in conjunction with the accompanying drawings, in which like reference numerals indicate identical or functionally similar elements.
Aspects of the present disclosure are directed to a 3D calling system that can provide 3D calls in various modes according to transitions and can provide affordances (i.e., visual or auditory cues to a user) to improve image capturing in the 3D call. In some cases, a 3D call can use an array of one or more cameras, incorporated in an external capture device (ECD), that is directed at a sending call participant. The live images of the sending call participant can be used to generate a hologram of the sending call participant for the 3D call. In other cases, the 3D calling system can generate an avatar representation of the sending call participant for use in the 3D call, which does not need images from an ECD to be generated.
The 3D calling system of a recipient in a 3D call can display the hologram or avatar of the sending call participant in a variety of ways, such as by making them “world-locked,” “ECD-locked,” or “body-locked.” World-locked virtual objects are positioned so as to appear stationary in the world, even when the call participant moves around in the artificial reality environment. ECD-locked virtual objects are positioned relative to the ECD, so as to appear at the same position relative to the ECD, despite the call participant’s movements. Body-locked virtual objects are positioned relative to the user of the artificial reality system, so as to appear at the same position relative to the user’s body, also despite the user’s movements.
In various implementations, the selection of a 3D call mode can be based on a hierarchy where a live world-locked or live ECD-locked mode is under a first branch for when the ECD of the sending call participant is in use and an avatar world-locked and an avatar body-locked mode are under a second branch for when the ECD of the sending call participant is not in use. The selection of the first or second branch can be based on contextual factors such as whether the ECD of the sending call participant is powered on and/or the camera capture system of the ECD is active, whether this ECD is positioned to capture images of the sending call participant, whether images captured by this ECD are of sufficient quality to form a live hologram, and/or whether there are sufficient network and processing resources to conduct a live hologram call. In the first branch, the selection of the live world-locked mode or the live ECD-locked mode can be based on determined movements of a second ECD of the recipient call participant (e.g., whether this ECD has moved a threshold amount, how fast it’s been moved, where it’s been moved, etc.) In the second branch, the selection of the avatar world-locked mode or the avatar body-locked mode can be based on a user selection between the modes. Additional details on the selection of an 3D call mode are provided below in relation to
The 3D calling system can also trigger various affordances for the sending call participant to improve the quality of the images captured by the sending call participant’s ECD - e.g., by capturing more of the sending call participant or changing the relative orientation between the ECD and sending call participant to place the sending call participant in a more optimal range of cameras of the ECD. An “affordance,” as used herein, is a signal to the user such as a displayed virtual object and/or an auditory cue, either signaling to the user that a current ECD configuration is non-optimal and/or providing instructions for an improved ECD configuration. For example, affordances can be provided in the 3D call to instruct the user on how to position the ECD to capture quality images (see e.g.,
Embodiments of the disclosed technology may include or be implemented in conjunction with an artificial reality system. Artificial reality or extra reality (XR) is a form of reality that has been adjusted in some manner before presentation to a user, which may include, e.g., virtual reality (VR), augmented reality (AR), mixed reality (MR), hybrid reality, or some combination and/or derivatives thereof. Artificial reality content may include completely generated content or generated content combined with captured content (e.g., real-world photographs). The artificial reality content may include video, audio, haptic feedback, or some combination thereof, any of which may be presented in a single channel or in multiple channels (such as stereo video that produces a three-dimensional effect to the viewer). Additionally, in some embodiments, artificial reality may be associated with applications, products, accessories, services, or some combination thereof, that are, e.g., used to create content in an artificial reality and/or used in (e.g., perform activities in) an artificial reality. The artificial reality system that provides the artificial reality content may be implemented on various platforms, including a head-mounted display (HMD) connected to a host computer system, a standalone HMD, a mobile device or computing system, a “cave” environment or other projection system, or any other hardware platform capable of providing artificial reality content to one or more viewers.
“Virtual reality” or “VR,” as used herein, refers to an immersive experience where a user’s visual input is controlled by a computing system. “Augmented reality” or “AR” refers to systems where a user views images of the real world after they have passed through a computing system. For example, a tablet with a camera on the back can capture images of the real world and then display the images on the screen on the opposite side of the tablet from the camera. The tablet can process and adjust or “augment” the images as they pass through the system, such as by adding virtual objects. “Mixed reality” or “MR” refers to systems where light entering a user’s eye is partially generated by a computing system and partially composes light reflected off objects in the real world. For example, a MR headset could be shaped as a pair of glasses with a pass-through display, which allows light from the real world to pass through a waveguide that simultaneously emits light from a projector in the MR headset, allowing the MR headset to present virtual objects intermixed with the real objects the user can see. “Artificial reality,” “extra reality,” or “XR,” as used herein, refers to any of VR, AR, MR, or any combination or hybrid thereof.
While some existing artificial reality systems can provide 3D calling, they fail to account for changing contexts during the 3D call. For example, these existing artificial reality systems generally provide a 3D call as either world-locked or body-locked holograms, which either fail or lose substantial quality if circumstances change, such as the receiving call participant moving around the displayed hologram or the sending call participant moving outside the view of the capturing cameras. The 3D calling system and methods described herein are expected to overcome these deficiencies in existing artificial reality systems by switching between four 3D calling modes: live world-locked, live ECD-locked, avatar world-locked, and avatar body-locked, depending on context and user selections in the call. The 3D calling system and methods described herein are further expected to overcome these deficiencies by detecting affordance triggers (such as the beginning of a 3D call, a user moving outside the range of capture cameras, or an object occluding the capture cameras) and, in response, providing a corresponding affordance (such as an indication of where to place the ECD, where to move to be in range of the capture cameras, or how the capture cameras are being occluded). These triggers and corresponding mode switches and affordances provide improved image quality in the field of 3D calling.
Several implementations are discussed below in more detail in reference to the figures.
Computing system 100 can include one or more processor(s) 110 (e.g., central processing units (CPUs), graphical processing units (GPUs), holographic processing units (HPUs), etc.) Processors 110 can be a single processing unit or multiple processing units in a device or distributed across multiple devices (e.g., distributed across two or more of computing devices 101-103).
Computing system 100 can include one or more input devices 120 that provide input to the processors 110, notifying them of actions. The actions can be mediated by a hardware controller that interprets the signals received from the input device and communicates the information to the processors 110 using a communication protocol. Each input device 120 can include, for example, a mouse, a keyboard, a touchscreen, a touchpad, a wearable input device (e.g., a haptics glove, a bracelet, a ring, an earring, a necklace, a watch, etc.), a camera (or other light-based input device, e.g., an infrared sensor), a microphone, or other user input devices.
Processors 110 can be coupled to other hardware devices, for example, with the use of an internal or external bus, such as a PCI bus, SCSI bus, or wireless connection. The processors 110 can communicate with a hardware controller for devices, such as for a display 130. Display 130 can be used to display text and graphics. In some implementations, display 130 includes the input device as part of the display, such as when the input device is a touchscreen or is equipped with an eye direction monitoring system. In some implementations, the display is separate from the input device. Examples of display devices are: an LCD display screen, an LED display screen, a projected, holographic, or augmented reality display (such as a heads-up display device or a head-mounted device), and so on. Other I/O devices 140 can also be coupled to the processor, such as a network chip or card, video chip or card, audio chip or card, USB, firewire or other external device, camera, printer, speakers, CD-ROM drive, DVD drive, disk drive, etc.
In some implementations, input from the I/O devices 140, such as cameras, depth sensors, IMU sensor, GPS units, LiDAR or other time-of-flights sensors, etc. can be used by the computing system 100 to identify and map the physical environment of the user while tracking the user’s location within that environment. This simultaneous localization and mapping (SLAM) system can generate maps (e.g., topologies, girds, etc.) for an area (which may be a room, building, outdoor space, etc.) and/or obtain maps previously generated by computing system 100 or another computing system that had mapped the area. The SLAM system can track the user within the area based on factors such as GPS data, matching identified objects and structures to mapped objects and structures, monitoring acceleration and other position changes, etc.
Computing system 100 can include a communication device capable of communicating wirelessly or wire-based with other local computing devices or a network node. The communication device can communicate with another device or a server through a network using, for example, TCP/IP protocols. Computing system 100 can utilize the communication device to distribute operations across multiple network devices.
The processors 110 can have access to a memory 150, which can be contained on one of the computing devices of computing system 100 or can be distributed across of the multiple computing devices of computing system 100 or other external devices. A memory includes one or more hardware devices for volatile or non-volatile storage, and can include both read-only and writable memory. For example, a memory can include one or more of random access memory (RAM), various caches, CPU registers, read-only memory (ROM), and writable non-volatile memory, such as flash memory, hard drives, floppy disks, CDs, DVDs, magnetic storage devices, tape drives, and so forth. A memory is not a propagating signal divorced from underlying hardware; a memory is thus non-transitory. Memory 150 can include program memory 160 that stores programs and software, such as an operating system 162, three-dimensional calling system 164, and other application programs 166. Memory 150 can also include data memory 170 that can include mappings of triggers 3D call mode changes, mappings of triggers to affordances, templates and data for displaying visual affordances and/or playing auditory affordances, configuration data, settings, user options or preferences, etc., which can be provided to the program memory 160 or any element of the computing system 100.
Some implementations can be operational with numerous other computing system environments or configurations. Examples of computing systems, environments, and/or configurations that may be suitable for use with the technology include, but are not limited to, XR headsets, personal computers, server computers, handheld or laptop devices, cellular telephones, wearable electronics, gaming consoles, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, or the like.
The electronic display 245 can be integrated with the front rigid body 205 and can provide image light to a user as dictated by the compute units 230. In various embodiments, the electronic display 245 can be a single electronic display or multiple electronic displays (e.g., a display for each user eye). Examples of the electronic display 245 include: a liquid crystal display (LCD), an organic light-emitting diode (OLED) display, an active-matrix organic light-emitting diode display (AMOLED), a display including one or more quantum dot light-emitting diode (QOLED) sub-pixels, a projector unit (e.g., microLED, LASER, etc.), some other display, or some combination thereof.
In some implementations, the HMD 200 can be coupled to a core processing component such as a personal computer (PC) (not shown) and/or one or more external sensors (not shown). The external sensors can monitor the HMD 200 (e.g., via light emitted from the HMD 200) which the PC can use, in combination with output from the IMU 215 and position sensors 220, to determine the location and movement of the HMD 200.
The projectors can be coupled to the pass-through display 258, e.g., via optical elements, to display media to a user. The optical elements can include one or more waveguide assemblies, reflectors, lenses, mirrors, collimators, gratings, etc., for directing light from the projectors to a user’s eye. Image data can be transmitted from the core processing component 254 via link 256 to HMD 252. Controllers in the HMD 252 can convert the image data into light pulses from the projectors, which can be transmitted via the optical elements as output light to the user’s eye. The output light can mix with light that passes through the display 258, allowing the output light to present virtual objects that appear as if they exist in the real world.
Similarly to the HMD 200, the HMD system 250 can also include motion and position tracking units, cameras, light sources, etc., which allow the HMD system 250 to, e.g., track itself in 3DoF or 6DoF, track portions of the user (e.g., hands, feet, head, or other body parts), map virtual objects to appear as stationary as the HMD 252 moves, and have virtual objects react to gestures and other real-world objects.
In various implementations, the HMD 200 or 250 can also include additional subsystems, such as an eye tracking unit, an audio system, various network components, etc., to monitor indications of user interactions and intentions. For example, in some implementations, instead of or in addition to controllers, one or more cameras included in the HMD 200 or 250, or from external cameras, can monitor the positions and poses of the user’s hands to determine gestures and other hand and body motions. As another example, one or more light sources can illuminate either or both of the user’s eyes and the HMD 200 or 250 can use eye-facing cameras to capture a reflection of this light to determine eye position (e.g., based on set of reflections around the user’s cornea), modeling the user’s eye and determining a gaze direction.
In some implementations, server 310 can be an edge server which receives client requests and coordinates fulfillment of those requests through other servers, such as servers 320A-C. Server computing devices 310 and 320 can comprise computing systems, such as computing system 100. Though each server computing device 310 and 320 is displayed logically as a single server, server computing devices can each be a distributed computing environment encompassing multiple computing devices located at the same or at geographically disparate physical locations.
Client computing devices 305 and server computing devices 310 and 320 can each act as a server or client to other server/client device(s). Server 310 can connect to a database 315. Servers 320A-C can each connect to a corresponding database 325A-C. As discussed above, each server 310 or 320 can correspond to a group of servers, and each of these servers can share a database or can have their own database. Though databases 315 and 325 are displayed logically as single units, databases 315 and 325 can each be a distributed computing environment encompassing multiple computing devices, can be located within their corresponding server, or can be located at the same or at geographically disparate physical locations.
Network 330 can be a local area network (LAN), a wide area network (WAN), a mesh network, a hybrid network, or other wired or wireless networks. Network 330 may be the Internet or some other public or private network. Client computing devices 305 can be connected to network 330 through a network interface, such as by wired or wireless communication. While the connections between server 310 and servers 320 are shown as separate connections, these connections can be any kind of local, wide area, wired, or wireless network, including network 330 or a separate public or private network.
Mediator 420 can include components which mediate resources between hardware 410 and specialized components 430. For example, mediator 420 can include an operating system, services, drivers, a basic input output system (BIOS), controller circuits, or other hardware or software systems.
Specialized components 430 can include software or hardware configured to perform operations for setting a mode controlling how a 3D call is presented by an artificial reality system and providing affordances for improving ECD capture of images. Specialized components 430 can include ECD in-use detector 434, ECD movement detector 436, world-locked drawing module 438, ECD-locked drawing module 440, body-locked drawing module 442, affordance module 444, and components and APIs which can be used for providing user interfaces, transferring data, and controlling the specialized components, such as interfaces 432. In some implementations, components 400 can be in a computing system that is distributed across multiple computing devices or can be an interface to a server-based application executing one or more of specialized components 430. Although depicted as separate components, specialized components 430 may be logical or other nonphysical differentiations of functions and/or may be submodules or code-blocks of one or more applications.
ECD in-use detector 434 can determine whether the ECD of a sending call participant is being used. When the ECD of a sending call participant is being used, a representation of the sending call participant can be a live hologram, whereas when the ECD of a sending call participant is now being used, a representation of the sending call participant can be an avatar. The ECD in-use detector 434 can make this determination based on whether the ECD of the sending call participant is powered on and/or the camera capture system of the ECD is active, whether this ECD is positioned to capture images of the sending call participant, whether images captured by this ECD are of sufficient quality to form a live hologram, and/or whether there are sufficient network and processing resources to conduct a live hologram call. Additional details on selecting an ECD in-use mode are provided below in relation to block 502 of
ECD movement detector 436 can determine whether movement characteristics of an ECD of a recipient call participant triggers a lock transition. The ECD movement detector 436 can monitor movement characteristics such as whether the ECD has moved a threshold amount, how fast it’s been moved, and/or where it’s been moved. Depending on these movement characteristics, a representation of the sending call participant (either a hologram or an avatar) can be positioned relative to a geographical point (i.e., be world-locked) or be positioned relative to the ECD (i.e., be ECD-locked). Additional details on determining the ECD movement characteristics are provided below in relation to block 506 of
World-locked drawing module 438 can draw an avatar or hologram representation of a sending call participant relative to a tracked geographical point even as the artificial reality device moves. Thus, the world-locked drawing module 438 can use photo tracking and other means of determining the relative position of the artificial reality device to the geographical position and, as the artificial reality device moves, can repeatedly update the avatar or hologram representation to appear as if it’s staying in the same location. Additional details on drawing a hologram or avatar in a world-locked position are provided below in relation to blocks 504 and 510 of
ECD-locked drawing module 440 can draw an avatar or hologram representation of a sending call participant relative to an ECD of a recipient call participant. Thus, the ECD-locked drawing module 440 can determined the relative position of the artificial reality device to the ECD and, as the artificial reality device and/or ECD moves, can repeatedly update the avatar or hologram representation to appear as if it’s staying in the position relative to the ECD. Additional details on drawing a hologram or avatar in an ECD-locked position are provided below in relation to block 508 of
Body-locked drawing module 442 can draw an avatar or hologram representation of a sending call participant relative to a body part or the artificial reality device of a recipient call participant. In some cases, the body-locked drawing module 442 can determined the relative position of the artificial reality device to the body part and, as the artificial reality device and/or body part moves, can repeatedly update the avatar or hologram representation to appear as if it’s staying in the position relative to the body part. In other cases, the body-locked drawing module 442 can display the hologram or avatar for the sending 3D call participant so as to appear consistently placed relative to the recipient 3D call participant by displaying the avatar at a consistent location in a display of the recipient 3D call participant, without updating its location according to how the artificial reality device is moved. Additional details on drawing a hologram or avatar in a body-locked position are provided below in relation to block 514 of
Affordance module 444 can display various affordances to help position the ECD and/or position the sending call participant relative to the ECD. In one case, the affordance can be an indication of where to place the ECD. In another case, the affordance can be a mini-model depicting miniature representations of both the sending call participant and the ECD. In a further case, the affordance can display a virtual boundary illustrating a portion of a capture range of the ECD. In yet a further case, the affordance can display a self-view of the sending call participant with a portion of the sending 3D call participant that is blocked from view of the ECD excluded from the self-view. In some cases, affordances can be auditory, such as a chime or recording. Additional details on providing affordances for ECD placement and adjustments are provided below in relation to
Those skilled in the art will appreciate that the components illustrated in
At block 502, process 500 can determine whether an event for transitioning to the ECD in-use mode is detected. In various implementations, an event for transitioning to the ECD in-use mode can include one or more of: the ECD of the sending call participant being powered on and/or the camera capture system of the ECD is active to take the yes branch to block 504; this ECD being powered off to take the no branch to block 510; process 500 determining that images captured by this ECD are above a quality threshold to take the yes branch to block 504; process 500 determining that images captured by this ECD are below the quality threshold to take the no branch to block 510; process 500 determining that images captured by this ECD depict at least a face of the sending call participant to take the yes branch to block 504; process 500 determining that images captured by the ECD do not include at least the face of the sending call participant to take the no branch to block 510; process 500 determining that available computing resources (e.g., memory, processor availability, network bandwidth, etc.) are above a resource threshold level to take the yes branch to block 504; or process 500 determining that at least one of the computing resources are below the resource threshold to take the no branch to block 510. In some implementations, combinations of these events have to exist to take the yes branch to block 504, such as the camera capture system of the ECD of the sending call participant being active, the quality being above the quality threshold, the images depicting the sending call participant, and there being sufficient computing resources; otherwise process 500 can take the no branch to block 510.
At block 504, process 500 can show the 3D call in a live world-locked mode. This can include initially positioning the hologram of the sending call participant in a world-locked location, which may be selected relative to the initial placement of the ECD, but will stay at that location if the ECD is moved while in the live world-locked mode. In other cases, the location of the hologram can be selected based on other real and/or virtual objects in the room. For example, this world-locked location can be placed above a detected local horizontal flat surface (such as above a table or above an area of the floor) a specified distance from the position of the recipient call participant, at an area where the recipient call participant is looking when the live world-locked mode was entered, etc. In some implementations, the location for the hologram of the sending call participant can be selected by the recipient call participant, either by directing an input to that location or by moving a current view of the hologram (e.g., with a grab and drag gesture) to the world-locked location that the recipient call participant desires. In some cases when an ECD in-use mode is entered, an affordance can be shown to the sending participant to help the user reposition herself and/or the ECD to better capture images of the sending participant (see e.g.,
At block 506, process 500 can determine whether an event is detected for transitioning how the 3D call is locked. In various implementations, the lock transition can be determined to occur when one or a combination of various movements of an ECD of the recipient call participant are above threshold levels. In cases where the ECD of the recipient call participant is not in use, process 500 can take the no branch from block 506. The various movements of the recipient call participant’s ECD can include one or more of: a distance of movement, a speed of movement, an angle of rotation, or whether the movement causes the ECD of the recipient call participant to approach a boundary (e.g., a boundary of a room or near a doorway or other portal). For example, the lock transition can occur when the ECD of the recipient call participant is moved a threshold distance such as 1, 2, or 3 meters; the lock transition can occur when this ECD is moved at a speed above .1, .2. or .4 meters/second; the lock transition can occur when this ECD is rotated at least 10, 30, or 90 degrees; or various areas can be identified as a boundary of a room, the artificial reality device can track where this ECD is within the room, and the lock transition can occur when this ECD is moved over, or within a threshold distance (e.g., .5, 1, or 2 meters) of, the boundary. In some implementations, the lock transition can occur when a specified combination of these events occurs (e.g., both the distance and angle events). Different threshold levels can be set for such a combination. For example, while movement of the ECD by two meters alone can trigger the lock transition, movement of the ECD by one meter and a rotation by 45 degrees may also trigger the lock transition.
At block 508, process 500 can show the 3D call in a live ECD-locked mode. The ECD-locked mode can anchor the hologram of the sending call participant to the ECD, such that as the ECD is moved the hologram of the sending call participant is moved (from the viewpoint of the recipient call participant) a corresponding amount.
At block 510, following process 500 determining at block 502 not to be in ECD in-use mode (i.e., being in an ECD not-in-use mode), process 500 can show the 3D call in an avatar world-locked mode. Because the ECD used to create a hologram of the sending participant is not in use, an avatar representing the sending participant can be displayed instead. The avatar can be a generic avatar, an avatar automatically created to resemble the sending participant, or an avatar with features selected by the sending participant, the recipient participant, or automatically (e.g., based on identified interests, characteristics, etc., of the sending participant). The avatar world-locked mode can include initially positioning the avatar representation the sending call participant in a world-locked location, which may be selected relative to the initial position of the recipient participant, but will stay at that location if the recipient participant moves while in the avatar world-locked mode. In other cases, the location of the avatar can be selected based on other real and/or virtual objects in the room. For example, this world-locked location can be placed above a detected local horizontal flat surface (such as above a table or above an area of the floor) a specified distance from the position of the recipient call participant, at an area where the recipient call participant is looking when the avatar world-locked mode was entered, etc. In some implementations, the location for the avatar of the sending participant can be selected by the recipient participant, either by directing an input to that location or by moving a current view of the avatar (e.g., with a grab and drag gesture) to the world-locked location that the recipient participant desires.
In some implementations, when system is not in the ECD in-use mode, this can be signaled to the sending participant, which may include a reason and/or an affordance for correcting the reason the ECD is not in use. For example, if the sending participant is out of view, or partially out of view of the cameras of the ECD, this can be signaled to the sending participant with an audio chime or recording affordance and/or through a visual affordance such as a grid virtual object showing where the cameras can capture (see e.g.,
At block 512, process 500 can determine whether an event is detected for transitioning how the 3D call is locked. The lock transition event, at block 512, can include identifying a manual, recipient selection for switching between the avatar representation of the sending participant being world-locked to being body-locked. Such a recipient selection can be signaled with a voice command, activating a control (e.g., button or menu item virtual object) in an artificial reality environment, performing a gesture mapped to the switch, etc.
At block 514, process 500 can show the 3D call in an avatar body-locked mode. The avatar being body locked can include causing the representation of the avatar to stay at a position relative to the body of the recipient participant, even as the recipient participant moves.
Following blocks 508 or 514, process 500 can return to block 502 to update whether the 3D call should stay in ECD in-use mode. These loops can continue while the 3D call is ongoing. Process 500 can end when the 3D call ends.
At block 602, process 600 can determine whether an affordance trigger has occurred. In various implementations, various events can be recognized as an affordance trigger, such as one or more of: entering an ECD in-use mode; identifying that relative positioning between a sending call participant and an ECD is preventing cameras of the ECD from capturing complete images of the sending call participant; determining that an object is occluding one or more cameras of the ECD, preventing it from capturing complete images of the sending call participant; determining that computing resources are below threshold levels; etc.
At block 604, process 600 can provide an affordance mapped to the affordance trigger determined to have occurred at block 602. The affordance trigger of entering an ECD in-use mode can be mapped to an affordance including providing an indication, in an artificial reality environment, of where the sending participant should place her ECD, e.g., with an overlay (see e.g.,
Following block 604, process 600 can return to block 602 to update whether an affordance trigger has occurred. This loop can continue while the 3D call is ongoing. Process 600 can end when the 3D call ends.
Reference in this specification to “implementations” (e.g., “some implementations,” “various implementations,” “one implementation,” “an implementation,” etc.) means that a particular feature, structure, or characteristic described in connection with the implementation is included in at least one implementation of the disclosure. The appearances of these phrases in various places in the specification are not necessarily all referring to the same implementation, nor are separate or alternative implementations mutually exclusive of other implementations. Moreover, various features are described which may be exhibited by some implementations and not by others. Similarly, various requirements are described which may be requirements for some implementations but not for other implementations.
As used herein, being above a threshold means that a value for an item under comparison is above a specified other value, that an item under comparison is among a certain specified number of items with the largest value, or that an item under comparison has a value within a specified top percentage value. As used herein, being below a threshold means that a value for an item under comparison is below a specified other value, that an item under comparison is among a certain specified number of items with the smallest value, or that an item under comparison has a value within a specified bottom percentage value. As used herein, being within a threshold means that a value for an item under comparison is between two specified other values, that an item under comparison is among a middle-specified number of items, or that an item under comparison has a value within a middle-specified percentage range. Relative terms, such as high or unimportant, when not otherwise defined, can be understood as assigning a value and determining how that value compares to an established threshold. For example, the phrase “selecting a fast connection” can be understood to mean selecting a connection that has a value assigned corresponding to its connection speed that is above a threshold.
As used herein, the word “or” refers to any possible permutation of a set of items. For example, the phrase “A, B, or C” refers to at least one of A, B, C, or any combination thereof, such as any of: A; B; C; A and B; A and C; B and C; A, B, and C; or multiple of any item such as A and A; B, B, and C; A, A, B, C, and C; etc.
Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Specific embodiments and implementations have been described herein for purposes of illustration, but various modifications can be made without deviating from the scope of the embodiments and implementations. The specific features and acts described above are disclosed as example forms of implementing the claims that follow. Accordingly, the embodiments and implementations are not limited except as by the appended claims.
Any patents, patent applications, and other references noted above are incorporated herein by reference. Aspects can be modified, if necessary, to employ the systems, functions, and concepts of the various references described above to provide yet further implementations. If statements or subject matter in a document incorporated by reference conflicts with statements or subject matter of this application, then this application shall control.
