Patent Grant
12126943
Light emitting diodes (LEDs) are used in a variety of applications. LEDs can be produced that emit light in a variety of colors and wavelengths, including light in the visible and infrared (IR) portions of the spectrum.
Virtual reality (VR) allows users to experience and/or interact with an immersive artificial environment, such that the user feels as if they were physically in that environment. For example, virtual reality systems may display stereoscopic scenes to users in order to create an illusion of depth, and a computer may adjust the scene content in real-time to provide the illusion of the user moving within the scene. When the user views images through a virtual reality system, the user may thus feel as if they are moving within the scenes from a first-person point of view. Similarly, mixed reality (MR) or augmented reality (AR) systems combine computer generated information (referred to as virtual content) with real world images or a real world view to augment, or add content to, a user's view of the world. The simulated environments of VR and/or the mixed environments of MR may thus be utilized to provide an interactive user experience for multiple applications, such as applications that add virtual content to a real-time view of the viewer's environment, interacting with virtual training environments, gaming, remotely controlling drones or other mechanical systems, viewing digital media content, interacting with the Internet, or the like.
Various embodiments of methods and apparatus for emitting light including light representing augmented reality (AR) content into environments such as rooms are described. Embodiments of a spatial light system are described that may, for example, be used to project AR content onto one or more surfaces of a room. In some embodiments, a spatial light system may include two or more light-emitting diode (LED) projection units for emitting light representing AR content onto surfaces within a room, onto object(s) within a room, or onto surfaces in a portion of a room. In some embodiments, a spatial light system may also be configured to emit diffuse light to illuminate a room, object(s) within a room, or a portion of a room when not emitting light representing AR content.
In some embodiments, a spatial light system may include a controller comprising one or more processors and one or more projection units. Embodiments of projection units are described that each include an active matrix LED array, an array of light pipes, and condensing optics (referred to as a condenser lens). The LED array may, for example, be an array of individually addressable LEDs. The LEDs may include white light-emitting LEDs and color-emitting LEDs. While embodiments are generally described in which projection units include light pipe arrays, other combination of light collection or light shaping optical elements may be used.
In some embodiments, two or more projection units may be connected to a flexible strip (referred to as a flex strip) that provides power and data (e.g., serial data) to the projection units. Two or more flex strips may be connected together. The flex strips of projection units provide a serially-connected, flexible modular architecture for spatial light systems that allow the projection units to be conformed to a variety of configurations and shapes. However, in some embodiments, the projection units may be mounted to a fixed surface that includes connections for power and/or data for the modules. In some embodiments, one or more other modules such as sensor modules (depth sensors, light sensors, motion sensors, cameras, etc.) may also be connected to the flex strips in addition to the projection units, for example to provide depth information for surfaces and object in the room, or to detect presence and motion of persons in the room.
In some embodiments, components of a spatial light system including but not limited to the projection units may be packaged or contained within an enclosure or “bulb-like” structure of a transparent or semi-transparent material (e.g., glass or plastic). In some embodiments, a spatial light system may be connected to electrical power, for example via a conventional light socket or via a conventional power plug. Embodiments of a spatial light systems as described herein, may, for example, be hung from a ceiling, or mounted on floor or table stands.
In some embodiments, a spatial light system may include a controller or control system that may, for example, provide power and video signals to the LED projection units. In some embodiments, a spatial light system may also include wireless technology that enables wireless communications between the spatial light system and other devices, such as an AR/VR system (e.g., an AR/VR headset or glasses), a mobile multipurpose device such as a smartphone, pad or tablet device, a smart TV, or a computer system. In some embodiments, a spatial light system may instead or also include technology that enables wired connection between the spatial light system and other devices.
In some embodiments, a spatial light system may include integrated sensors to detect objects and surfaces (e.g., people and specular surfaces such as mirrors or windows) within a room, and/or to detect light levels within a room. The sensors may also determine depth of objects or surfaces, and types of objects or surfaces within a room. In some embodiments, at least one of the sensors may use infrared (IR) light to detect objects or surfaces. Information about objects and surfaces within a room collected by the sensors may be fed back to a controller of the spatial light system, which may use the information in controlling operation of the projection units of the spatial light system.
A spatial light system may include a controller comprising one or more processors to which projection units are connected, for example via flex strips. The controller may drive or control the projection units on the flex strips to emit light into a room according to image content and room information. The room information may include, but is not limited to, depth information for surfaces in the room, lighting information for the room, specular information for surfaces or objects in the room, location and identification information for objects in the room, location, motion, and orientation information for persons in the room, or in general any mapping information that can be obtained for a room and its current contents. For example, one or more sensors may collect lighting information for existing artificial and natural light sources in a room, such as lamps, televisions, and windows. The room information may, for example, be used to determine depths at which “pixels” projected by the projection units are to be focused, objects or surfaces to be illuminated or not illuminated, and so on. In some embodiments, the room information may be obtained from an external device, for example via a wireless connection to the system. In these embodiments, the system controller may be relatively simple, projecting images onto surfaces based on the room information obtained from an external device. In some embodiments, the room information may be generated by the system controller from data obtained from one or more sensor modules of the system, for example sensor modules coupled to the flex strips on which the projection units are coupled. In these embodiments, the system controller may be relatively more complex to provide the needed processing power.
Image content to be projected by the spatial light system may be obtained from an external device, for example via a wireless connection, or alternatively may be generated based on image information stored locally in memory of the system. The spatial light system may, for example, project illuminating light, AR imagery, patterns, or other image content onto the floor, walls, and/or ceiling of a room based on the image content and room information. However, the system may also intelligently light or augment particular objects found within a room, and/or intelligently not illuminate certain surfaces or objects such as windows or television screens, or persons, within a room, based on the room information obtained from an external device or generated by the system controller from data collected by sensors.
In some embodiments, the room information may be used by the spatial light system to detect surfaces, objects, and motion within a room, and may also be used to identify particular objects including but not limited to specular surfaces such as mirrors, windows, and television screens, and persons within the room. This information may be used to adjust projection of light from the spatial light system. For example, upon detecting a person within the room, the spatial light system may lower the intensity of the light projected towards that person by one or more of the projection units to avoid “dazzling” the person with intense light. As another example, the spatial light system may adjust one or more of the projection units to focus the projected imagery at different depths as detected by the sensors. As yet another example, the spatial light system may lower the intensity of projected light, or not project light, towards a reflective surface to avoid unwanted reflective light from the reflective surface. As still yet another example, the spatial light system may project light towards an identified object or objects within the room to intentionally highlight or “spotlight” those objects. As still yet another example, a spatial light system may include one or more sensors that allow the system to track moving objects or persons in a room; the spatial light system may then project light with respect to the tracked object or person to augment that object or person, for example to project light in front of a moving toy car to provide the appearance of headlights of the car, or to “spotlight” a moving person in a room.
In some embodiments, the system may project 2D images of AR or other image content onto surfaces in the room, for example at least portions of the walls, ceiling, and floors, or onto objects within the room. In some embodiments, the image content projected by the system may be grayscale or color image content. Projected AR content may, for example, display swaying shadows of trees on the walls and ceiling, images of animals, birds flying, clouds, or of rain falling. Instead of or in addition to projecting image content in motion, still image content may be projected. In some embodiments, the system may also be configured to emit diffuse light to illuminate a room when not emitting light representing AR or other image content. In some embodiments, the system may emit spatial light in combination with other devices, for example to provide the expansion of television content. In some embodiments, the system may emit light to provide a low resolution version of a virtual environment that a person in the room is experiencing in virtual reality (e.g., using a headset or head-mounted device (HMD)) to give context to other people in the room without virtual reality headsets. In some embodiments, the system may emit light to complement a mixed reality environment being experienced by a person using an augmented reality headset by using lighting to improve contrast and colors viewed in the headset.
This specification includes references to “one embodiment” or “an embodiment.” The appearances of the phrases “in one embodiment” or “in an embodiment” do not necessarily refer to the same embodiment. Particular features, structures, or characteristics may be combined in any suitable manner consistent with this disclosure.
“Comprising.” This term is open-ended. As used in the claims, this term does not foreclose additional structure or steps. Consider a claim that recites: “An apparatus comprising one or more processor units . . . .” Such a claim does not foreclose the apparatus from including additional components (e.g., a network interface unit, graphics circuitry, etc.).
“Configured To.” Various units, circuits, or other components may be described or claimed as “configured to” perform a task or tasks. In such contexts, “configured to” is used to connote structure by indicating that the units/circuits/components include structure (e.g., circuitry) that performs those task or tasks during operation. As such, the unit/circuit/component can be said to be configured to perform the task even when the specified unit/circuit/component is not currently operational (e.g., is not on). The units/circuits/components used with the “configured to” language include hardware—for example, circuits, memory storing program instructions executable to implement the operation, etc. Reciting that a unit/circuit/component is “configured to” perform one or more tasks is expressly intended not to invoke 35 U.S.C. § 112, paragraph (f), for that unit/circuit/component. Additionally, “configured to” can include generic structure (e.g., generic circuitry) that is manipulated by software or firmware (e.g., an FPGA or a general-purpose processor executing software) to operate in manner that is capable of performing the task(s) at issue. “Configure to” may also include adapting a manufacturing process (e.g., a semiconductor fabrication facility) to fabricate devices (e.g., integrated circuits) that are adapted to implement or perform one or more tasks.
“First,” “Second,” etc. As used herein, these terms are used as labels for nouns that they precede, and do not imply any type of ordering (e.g., spatial, temporal, logical, etc.). For example, a buffer circuit may be described herein as performing write operations for “first” and “second” values. The terms “first” and “second” do not necessarily imply that the first value must be written before the second value.
“Based On” or “Dependent On.” As used herein, these terms are used to describe one or more factors that affect a determination. These terms do not foreclose additional factors that may affect a determination. That is, a determination may be solely based on those factors or based, at least in part, on those factors. Consider the phrase “determine A based on B.” While in this case, B is a factor that affects the determination of A, such a phrase does not foreclose the determination of A from also being based on C. In other instances, A may be determined based solely on B.
“Or.” When used in the claims, the term “or” is used as an inclusive or and not as an exclusive or. For example, the phrase “at least one of x, y, or z” means any one of x, y, and z, as well as any combination thereof.
Various embodiments of methods and apparatus for emitting light including light representing augmented reality (AR) content into environments such as rooms are described. Embodiments of a spatial light system are described that may, for example, be used to project AR content onto one or more surfaces of a room. In some embodiments, a spatial light system may include two or more light-emitting diode (LED) projection units for emitting light representing AR content onto surfaces within a room, onto object(s) within a room, or onto surfaces in a portion of a room. In some embodiments, a spatial light system may also be configured to emit diffuse light to illuminate a room, object(s) within a room, or a portion of a room when not emitting light representing AR content.
In some embodiments, two or more projection units may be connected to a flexible strip (referred to as a flex strip) that provides power and data (e.g., serial data) to the projection units. Alternatively, power and/or data for the projection units may come from another source. Two or more flex strips may be connected together. The flex strips of projection units provide a serially-connected, flexible modular architecture for spatial light systems that allow the projection units to be conformed to a variety of configurations and shapes as described herein. However, in some embodiments, the projection units described herein may be mounted to a fixed surface that includes connections for power and/or data for the modules. In some embodiments, one or more other modules such as sensor modules (depth sensors, light sensors, motion sensors, cameras, etc.) may also be connected to the flex strips in addition to the projection units, for example to provide depth information for surfaces and object in the room, or to detect presence and motion of persons in the room.
Embodiments of a spatial light system may include a controller comprising one or more processors to which the flex strips are connected and that drives or controls the projection units on the flex strips to emit light into a room according to image content and room information.
The room information may include, but is not limited to, depth information for surfaces in the room, lighting information for the room, specular information for surfaces or objects in the room, location and identification information for objects in the room, location, motion, and orientation information for persons in the room, or in general any mapping information that can be obtained for a room and its current contents. The room information may, for example, be used to determine depths at which “pixels” projected by the projection units are to be focused, objects or surfaces to be illuminated or not illuminated, and so on. In some embodiments, the room information may be obtained from an external device, for example via a wireless connection to the spatial light system. In these embodiments, the controller may be relatively simple, projecting images onto surfaces based on the room information obtained from an external device. In some embodiments, the room information may be generated by the controller from data obtained from one or more sensor modules coupled to the flex strips. In these embodiments, the controller may be relatively more complex to provide the needed processing power.
The image content to be projected may be obtained from an external device, for example via a wireless connection to the spatial light system, or alternatively may be generated based on image information stored locally in memory on the spatial light system. In some embodiments, the spatial light system may provide room illumination, as per conventional room lighting (e.g., light fixtures or light bulbs), as well as projecting AR or other imagery onto one or more surfaces in the room. A spatial light system may, for example, project illuminating light, AR imagery, patterns, or other image content onto the floor, walls, and/or ceiling of a room. However, the spatial light system may also intelligently light or augment particular objects found within a room, and/or intelligently not illuminate certain surfaces or objects such as windows or television screens, or persons, within a room.
The room information may be used by the spatial light system to detect surfaces, objects, and motion within a room, and may also be used to identify particular objects including but not limited to specular surfaces such as mirrors, windows, and television screens, and persons within the room. This information may be used to adjust projection of light from the spatial light system. For example, upon detecting a person within the room, the spatial light system may lower the intensity of the light projected towards that person by one or more of the projection units to avoid “dazzling” the person with intense light. As another example, the spatial light system may adjust one or more of the projection units to focus the projected imagery at different depths as detected by the sensors. As yet another example, the spatial light system may lower the intensity of projected light, or not project light, towards a reflective surface to avoid unwanted reflective light from the reflective surface. As still yet another example, the spatial light system may project light towards an identified object or objects within the room to intentionally highlight or “spotlight” those objects.
In some embodiments, the spatial light system may project 2D images of AR or other image content onto surfaces in the room, for example at least portions of the walls, ceiling, and floors, or onto objects within the room. In some embodiments, the image content projected by the spatial light system may be grayscale or color image content. Projected AR content may, for example, display swaying shadows of trees on the walls and ceiling, images of animals, birds flying, clouds, or of rain falling. Instead of or in addition to projecting image content in motion, still image content may be projected. In some embodiments, the spatial light system may also be configured to emit diffuse light to illuminate a room when not emitting light representing AR or other image content. In some embodiments, the spatial light system may replicate or simulate all or a portion of generic illumination that typical light sources such as floor, ceiling, wall, or tabletop lamps or light fixtures provide. In some embodiments, the system may emit spatial light in combination with other devices, for example to provide the expansion of television content. In some embodiments, the system may emit light to provide a low resolution version of a virtual environment that a person in the room is experiencing in virtual reality (e.g., using a headset or head-mounted device (HMD)) to give context to other people in the room without virtual reality headsets. In some embodiments, the system may emit light to complement a mixed reality environment being experienced by a person using an augmented reality headset by using lighting to improve contrast and colors viewed in the headset.
In some embodiments, the spatial light system controller generates or receives video signals (e.g., from an external device such as an AR/VR device, pad or tablet device, or smartphone) and processes the signals into separate video drives for each projection units so that the final image(s) projected by all the projection units is correctly stitched and continuous. The controller then provides the correct pulse width modulation to each LED pixel of each projection unit so that the correct light output is achieved to project the imagery.
In some embodiments, two or more spatial light systems in a room may be configured to communicate using light as a form of “wireless” information transfer medium. In some embodiments, a spatial light system may be configured to pulse light at a frequency that is not detectable to the human eye; the pulsed light can carry information about the device to one or more other spatial light systems or other devices in a room. The information conveyed may, for example, include information about the spatial light system's location in the room, current intensity, or any other relevant information about the device and room environment that the spatial light systems need to operate collaboratively. Further, a spatial light system may include light sensors to receive pulsed light signals from other devices in a room, and the spatial light system controller may be configured to generate and interpret the pulsed light signals. This allows two or more spatial light systems in a room to communicate in an open manner without having to physically connect the devices, and makes it easy to scale spatial light system installations in a room. Using pulsed light signals, two or more spatial light system devices just need to be able to “see” each other in a room to communicate directly via light that they already emit. Note that visible or non-visible wavelengths of light, or both, may be used as a pulsed light information transfer medium.
In some embodiments, light from a spatial light system may be used to power another spatial light system within a room, or to power other devices such as sensors within a room. Further, a spatial light system may include technology that allows the spatial light system to be powered by light from another spatial light system or other light source.
In some embodiments, components of a spatial light system including but not limited to the projection units may be packaged or contained within an enclosure or “bulb-like” structure of a transparent or semi-transparent material (e.g., glass or plastic). In some embodiments, a spatial light system may be connected to electrical power, for example via a conventional light socket or via a conventional power plug. Embodiments of a spatial light systems as described herein, may, for example, be hung from a ceiling, mounted on a wall or object, or mounted on floor or table stands.
While embodiments of a spatial light system are generally described as including flex strips with pluggable projection units (and possibly other modules such as sensors) that are applied to the outer surface of an object such as a cylinder or sphere, or to a curved or flat surface of an object such as a disk, and in which the flex strip provides connectivity between the modules and the controller, in some embodiments an object such as a cylinder, sphere, or disk may be manufactured with integrated wiring that connects the projection (and other) modules to the controller, and via which the controller drives the projection units.
While the controller is generally described as being a component of the spatial light system, and thus may be contained within the spatial light system enclosure, in some embodiments at least a portion of the controller may be external to the spatial light system enclosure. Further, in some embodiments, at least some of the functionality of the controller as described herein may be performed by a device external to the spatial light system that communicates with the system via a wired or wireless connection.
In some embodiments, in addition to room information that identifies surfaces, objects, and other information about a room in which the spatial light system is in, the spatial light system may obtain or store information about the layout and orientation of the projection units (and sensor modules if present) of the system. This information may provide a mapping between particular projection units and the content of the room, and may be used along with the room information in controlling the projection units, and the LEDs within projection units, when projecting light into the room environment.
In some embodiments, a spatial light system may include one or more actuators that can automatically move the system, for example under control of an external device. As an example, a lamp or disk spatial light system may move to illuminate different portions of a surface or room, or to track an object or person in motion. As another example, a cylinder or sphere spatial light system may be rotated within a room.
In some embodiments, a spatial light system or one or more projection units of a spatial light system may include at least one camera for tracking objects or persons within a room. In some embodiments, a camera may be implemented as a module that plugs into a flex strip. In some embodiments, at least one of the projection units may include an embedded camera. In some embodiments, the embedded camera may be confocal with the light projection mechanism of the projection unit; that is, the camera uses the same optical path as the light projection mechanism of the projection unit to capture images or video of a portion of a room. Tracking an object or person may, for example, allow the spatial light system to project light with respect to a moving object or person to augment that object or person, for example to project light in front of a moving toy car to provide the appearance of headlights of the car, or to “spotlight” a moving person in a room.
In some embodiments, an external device such as a smartphone may include applications or interfaces that allow a user to control operations of the spatial light system, for example to select content to be projected, to switch between projecting content and room illumination, to control the brightness of the system, to signal actuators to reposition the spatial light system, and so on.
The flex strip 130 includes wiring that provides control and image signals, as well as power, to a projection unit 110A via connection 111. The control and image signals selectively drive individual LEDs on LED array 114 to turn on or off individual ones of the LEDs, change the intensity or depth of focus, etc. of the LEDs. The LEDs may include white as well as color LEDs, for example as shown in
In some embodiments, there is one light pipe per LED in the light pipe array 116. The light pipes in the light pipe array 116 guide the light emitted by the LEDs and emit the light towards the condenser lens 118. Broadly defined, a condenser lens is an optical lens which renders a divergent beam from a point source into a parallel or converging beam to illuminate an object or surface. The condenser lens 118 affects the emitted light received from the light pipe array 116 to project pixels (or portions of pixels) onto a surface or object in the room. Note, however, that a projected “pixel” on a surface may be formed by light projected from two or more projection units 110. In other words, the light projected by a given projection unit 110 (e.g., 110A) may overlap with the light projected by one or more other (typically adjacent) projection units 110 (e.g., 110B) to form “pixels” on surface(s) within a room, all under control of controller 190; collectively the “pixels” form image(s) on the surface(s).
As previously mentioned, in some embodiments the individual projection units 110 may be controlled by controller 190 via a serial communications protocol through flex strip 130. In some embodiments, the serial communications protocol may be SDI (serial digital interface). However, other communications protocols may be used. In some embodiments, SDI may provide a 30 MHz rate at 16 bit color depth, and may maintain a 60 Hz per second frame rate.
In some embodiments, one or more projection units 110 of a spatial light system may be configured to pulse light at a frequency that is not detectable to the human eye; the pulsed light can carry information about the device to one or more other spatial light systems or other devices in a room. The information conveyed may, for example, include information about the spatial light system's location in the room, current intensity, or any other relevant information about the device and room environment that the spatial light systems need to operate collaboratively. Further, a spatial light system may include light sensors to receive pulsed light signals from other devices in a room, and the spatial light system controller 190 may be configured to generate and interpret the pulsed light signals. Note that visible or non-visible wavelengths of light, or both, may be used as a pulsed light information transfer medium.
In some embodiments, the spatial light system performs color mixing in the system. A given pixel on a surface may thus be formed of white light from one or more of the white LEDs in one or more of the projection units 110, or a mix of white and color light from the white and color LEDs in one or more of the projection units 110. In some embodiments, the LEDs may include LEDs configured to emit light in non-visible illumination wavelengths, or in any combination of wavelengths including but not limited to multiple white spectrums to form a broader spectrum of illumination.
As shown in
System 600 may include a controller comprising one or more processors to which projection units 610 are connected, for example via flex strips as described herein. The controller may drive or control the projection units 610 on the flex strips to emit light into a room according to image content and room information. In some embodiments, the controller may also be configured to cooperate with one or more other systems 600 in a room to collaboratively control lighting within the room. In some embodiments, two or more systems 600 in a room may communicate via pulsed visible or non-visible light signals to exchange information used in collaboratively controlling lighting within a room.
The room information may include, but is not limited to, depth information for surfaces in the room, lighting information for the room, specular information for surfaces or objects in the room, location and identification information for objects in the room, location, motion, and orientation information for persons in the room, or in general any mapping information that can be obtained for a room and its current contents. The room information may, for example, be used to determine depths at which “pixels” projected by the projection units are to be focused, objects or surfaces to be illuminated or not illuminated, and so on. In some embodiments, the room information may be obtained from an external device, for example via a wireless connection to the system 600. In these embodiments, the system 600 controller may be relatively simple, projecting images onto surfaces based on the room information obtained from an external device. In some embodiments, the room information may be generated by the system 600 controller from data obtained from one or more sensor modules of the system 600, for example sensor modules coupled to the flex strips on which the projection units 610 are coupled. In these embodiments, the system 600 controller may be relatively more complex to provide the needed processing power.
Image content to be projected by system 600 may be obtained from an external device, for example via a wireless connection to the system 600, or alternatively may be generated based on image information stored locally in memory of the system 600. In some embodiments, the system 600 may provide room illumination, as per conventional room lighting (e.g., light fixtures or light bulbs), as well as projecting AR or other imagery onto one or more surfaces in the room. System 600, for example, project illuminating light, AR imagery, patterns, or other image content onto the floor, walls, and/or ceiling of a room. However, the system 600 may also intelligently light or augment particular objects found within a room, and/or intelligently not illuminate certain surfaces or objects such as windows or television screens, or persons, within a room, based on the room information obtained from an external device or generated by the system 600 controller from data collected by sensors of the system 600.
In some embodiments, the room information may be used by the spatial light system 600 to detect surfaces, objects, and motion within a room, and may also be used to identify particular objects including but not limited to specular surfaces such as mirrors, windows, and television screens, and persons within the room. This information may be used to adjust projection of light from the spatial light system 600. For example, upon detecting a person within the room, the spatial light system 600 may lower the intensity of the light projected towards that person by one or more of the projection units 610 to avoid “dazzling” the person with intense light. As another example, the spatial light system 600 may adjust one or more of the projection units 610 to focus the projected imagery at different depths as detected by the sensors. As yet another example, the spatial light system 600 may lower the intensity of projected light, or not project light, towards a reflective surface to avoid unwanted reflective light from the reflective surface. As still yet another example, the spatial light system 600 may project light towards an identified object or objects within the room to intentionally highlight or “spotlight” those objects.
In some embodiments, the system 600 may project 2D images of AR or other image content onto surfaces in the room, for example at least portions of the walls, ceiling, and floors, or onto objects within the room. In some embodiments, the image content projected by the system 600 may be grayscale or color image content. Projected AR content may, for example, display swaying shadows of trees on the walls and ceiling, images of animals, birds flying, clouds, or of rain falling. Instead of or in addition to projecting image content in motion, still image content may be projected. In some embodiments, the system 600 may also be configured to emit diffuse light to illuminate a room when not emitting light representing AR or other image content.
In some embodiments, the system 600 may be configured to pulse light at a frequency that is not detectable to the human eye; the pulsed light can carry information about the system 600 to one or more other spatial light systems or other devices in a room. The information conveyed may, for example, include information about the system 600's location in the room, current intensity, or any other relevant information about the system 600 and room environment that the spatial light systems in a room need to operate collaboratively. Further, the system 600 may include light sensors to receive pulsed light signals from other devices in a room, and the system 600 controller may be configured to generate and interpret the pulsed light signals. Note that visible or non-visible wavelengths of light, or both, may be used as a pulsed light information transfer medium.
System 800 may include a controller comprising one or more processors to which projection units 810 are coupled, for example via flex strips as described herein. The controller 890 may drive or control the projection units 810 on the flex strips to emit light into a room according to image content and room information. The room information may be obtained from the device 880. The room information may include, but is not limited to, depth information for surfaces in the room, lighting information for the room, specular information for surfaces or objects in the room, location and identification information for objects in the room, location, motion, and orientation information for persons in the room, or in general any mapping information that can be obtained for a room and its current contents. The room information may, for example, be used to determine direction and depths at which “pixels” projected by the projection units 810 are to be focused, objects or surfaces to be illuminated or not illuminated, and so on. In some embodiments, the controller 890 may also be configured to cooperate with one or more other systems 800 in a room to collaboratively control lighting within the room. In some embodiments, two or more systems 800 in a room may communicate via pulsed visible or non-visible light signals to exchange information used in collaboratively controlling lighting within a room.
Image content to be projected by system 800 may be obtained from an external device 880, for example via a wireless connection to the system 800, or alternatively may be generated based on image information stored locally in memory of the system 800.
In some embodiments, the system 800 may be configured to pulse light at a frequency that is not detectable to the human eye; the pulsed light can carry information about the system 800 to one or more other spatial light systems or other devices in a room. The information conveyed may, for example, include information about the system 800's location in the room, current intensity, or any other relevant information about the system 800 and room environment that the spatial light systems in a room need to operate collaboratively. Further, the system 800 may include light sensors to receive pulsed light signals from other devices in a room, and the system 800 controller may be configured to generate and interpret the pulsed light signals. Note that visible or non-visible wavelengths of light, or both, may be used as a pulsed light information transfer medium.
System 800 may include a controller comprising one or more processors to which projection units 810 are coupled, for example via flex strips as described herein. The controller 890 may drive or control the projection units 810 on the flex strips to emit light into a room according to image content and room information. In some embodiments, the room information may be generated by the system 900 controller 990 from data obtained from one or more sensor modules 936 of the system 900, for example sensor modules 936 coupled to the flex strips on which the projection units 910 are coupled. The room information may include, but is not limited to, depth information for surfaces in the room, lighting information for the room, specular information for surfaces or objects in the room, location and identification information for objects in the room, location, motion, and orientation information for persons in the room, or in general any mapping information that can be obtained for a room and its current contents. The room information may, for example, be used to determine direction and depths at which “pixels” projected by the projection units 910 are to be focused, objects or surfaces to be illuminated or not illuminated, and so on.
In some embodiments, a spatial light system 900 may include at least one camera for tracking objects or persons within a room. In some embodiments, a camera may be implemented as a sensor module 936 that plugs into a flex strip. Alternatively, in some embodiments, at least one of the projection units 910 may include an embedded camera. In some embodiments, the embedded camera may be confocal with the light projection mechanism of the projection unit 910; that is, the camera uses the same optical path as the light projection mechanism of the projection unit 910 to capture images or video of a portion of a room. Tracking an object or person may, for example, allow the spatial light system to project light with respect to a moving object or person to augment that object or person, for example to project light in front of a moving toy car to provide the appearance of headlights of the car, or to “spotlight” a moving person in a room.
Image content to be projected by system 900 may be obtained from an external device 980, for example via a wireless connection to the system 900, or alternatively may be generated based on image information stored locally in memory of the system 900.
The controller 3090 may receive room information from an external device via wired or wireless signals to communications interface 3098, for example from a smartphone, tablet or pad device, laptop or other computer system, an AR/VR system, or other smart device such as a television. The room information may be stored to memory 3094. The controller 3090 may drive or control the projection units 3010 on the flex strips 3030 to emit light into a room according to image content and the room information obtained from an external device. The room information may include, but is not limited to, depth information for surfaces in the room, lighting information for the room, specular information for surfaces or objects in the room, location and identification information for objects in the room, location, motion, and orientation information for persons in the room, or in general any mapping information that can be obtained for a room and its current contents. The room information may, for example, be used to determine direction and depths at which “pixels” projected by the projection units 3010 are to be focused, objects or surfaces to be illuminated or not illuminated, and so on.
Image content to be projected by the spatial light system may be obtained from an external device, for example via a wireless connection through interface 3098, or alternatively may be generated by controller 3090 based on image information stored locally in memory 3094.
In some embodiments, controller 3090 may generate or render video or images based on the room information and image content that may be projected by projection units 3010 coupled to the controller 3090 via flex strips 3030 connected to the controller 3090 via serial interface 3099. In some embodiments, controller 3090 may also include memory 3094 that, for example, stores software that is executable by the controller 3090, data that may be used by the controller 3090 including but not limited to room information, and/or that includes buffer(s) for image or video content. In some embodiments, controller 3090 may also include one or more interfaces 3098 that may facilitate communication with an external device via a wired or wireless connection. In some embodiments, the external device may send image content (e.g., video) to be rendered and displayed by the spatial light system to the controller 3090 via interface 3098, and may also send signals to control operations of the spatial light system to the controller 3090 via interface 3098. The external device may be or may include any type of computing system or computing device, such as a desktop computer, notebook or laptop computer, pad or tablet device, smartphone, hand-held computing device, game controller, game system, AR/VR device such as a head-mounted device, smart device such as a smart television, and so on.
In some embodiments the individual projection units 3010 may be controlled by controller 3090 via a serial communications protocol through interface 3199 and flex strip(s) 3030. In some embodiments, the serial communications protocol may be SDI (serial digital interface). However, other communications protocols may be used. In some embodiments, SDI may provide a 30 MHz rate at 16 bit color depth, and may maintain a 60 Hz per second frame rate.
In various embodiments, controller 3090 may be a uniprocessor system including one processor 3092, or a multiprocessor system including several processors 3092 (e.g., two, four, eight, or another suitable number). Controller 3090 may include central processing units (CPUs) that implement any suitable instruction set architecture, and may execute instructions defined in that instruction set architecture. For example, in various embodiments controller 3090 may include general-purpose or embedded processors implementing any of a variety of instruction set architectures (ISAs), such as the x86, PowerPC, SPARC, RISC, or MIPS ISAs, or any other suitable ISA. In multiprocessor systems, each of the processors may commonly, but not necessarily, implement the same ISA. Controller 3090 may employ any microarchitecture, including scalar, superscalar, pipelined, superpipelined, out of order, in order, speculative, non-speculative, etc., or combinations thereof. Controller 3090 may include circuitry to implement microcoding techniques. Controller 3090 may include one or more processing cores that each execute instructions. Controller 3090 may include one or more levels of caches, which may employ any size and any configuration (set associative, direct mapped, etc.). In some embodiments, controller 3090 may include at least one graphics processing unit (GPU), which may include any suitable graphics processing circuitry. Generally, a GPU may render objects to be displayed into a frame buffer (e.g., one that includes pixel data for an entire frame). A GPU may include one or more graphics processors that may execute graphics software to perform a part or all of the graphics operation, or hardware acceleration of certain graphics operations. In some embodiments, controller 3090 may include one or more other components for processing and rendering video and/or images, for example image signal processors (ISPs), coder/decoders (codecs), etc.
Memory 3094 may include any type of memory, such as dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate (DDR, DDR2, DDR3, etc.) SDRAM (including mobile versions of the SDRAMs such as mDDR3, etc., or low power versions of the SDRAMs such as LPDDR2, etc.), RAMBUS DRAM (RDRAM), static RAM (SRAM), etc. In some embodiments, one or more memory devices may be coupled onto a circuit board to form memory modules such as single inline memory modules (SIMMs), dual inline memory modules (DIMMs), etc. Alternatively, the devices may be mounted with an integrated circuit implementing system in a chip-on-chip configuration, a package-on-package configuration, or a multi-chip module configuration.
In some embodiments, the spatial light system may include one or more sensor 3136 modules as well as projection units 3110; the sensors 3136 may collect data about the room that can be used to generate room information (e.g., surface depths, lighting, specular surfaces, motion, object detection and identification, etc.) The controller 3090 may obtain data from one or more of the sensors 3136 of the spatial light system and generate room information from the sensor data. The sensors 3136 may be, but are not necessarily coupled to the flex strips 3130. The room information may be stored to memory 3194. The controller 3090 may drive or control the projection units 3010 on the flex strips 3130 to emit light into a room according to image content and the room information generated from the sensor data. The room information may include, but is not limited to, depth information for surfaces in the room, lighting information for the room, specular information for surfaces or objects in the room, location and identification information for objects in the room, location, motion, and orientation information for persons in the room, or in general any mapping information that can be obtained for a room and its current contents. The room information may, for example, be used to determine direction and depths at which “pixels” projected by the projection units 3110 are to be focused, objects or surfaces to be illuminated or not illuminated, and so on.
In some embodiments, a spatial light system may include at least one camera for tracking objects or persons within a room. In some embodiments, a camera may be implemented as a sensor module 3136 that plugs into a flex strip 3130. Alternatively, in some embodiments, at least one of the projection units 3110 may include an embedded camera. In some embodiments, the embedded camera may be confocal with the light projection mechanism of the projection unit 3110; that is, the camera uses the same optical path as the light projection mechanism of the projection unit 3110 to capture images or video of a portion of a room. Tracking an object or person may, for example, allow the spatial light system to project light with respect to a moving object or person to augment that object or person, for example to project light in front of a moving toy car to provide the appearance of headlights of the car, or to “spotlight” a moving person in a room.
Image content to be projected by the spatial light system may be obtained from an external device, for example via a wireless connection through interface 3198, or alternatively may be generated by controller 3190 based on image information stored locally in memory 3194.
In some embodiments, controller 3190 may generate or render video or images based on the room information and image content that may be projected by projection units 3110 coupled to the controller 3190 via flex strips 3130 connected to the controller 3190 via serial interface 3199. In some embodiments, controller 3190 may also include memory 3194 that, for example, stores software that is executable by the controller 310, data that may be used by the controller 3190 including but not limited to room information, and/or that includes buffer(s) for sensor data and image or video content. In some embodiments, controller 3190 may also include one or more interfaces 3198 that may facilitate communication with an external device via a wired or wireless connection. In some embodiments, the external device may send image content (e.g., video) to be rendered and displayed by the spatial light system to the controller 3190 via interface 3198, and may also send signals to control operations of the spatial light system to the controller 3190 via interface 3198. The external device may be or may include any type of computing system or computing device, such as a desktop computer, notebook or laptop computer, pad or tablet device, smartphone, hand-held computing device, game controller, game system, AR/VR device such as a head-mounted device, smart device such as a smart television, and so on.
In some embodiments the individual projection units 3110 may be controlled by controller 3190 via a serial communications protocol through interface 3199 and flex strip(s) 3130. In addition, data from sensors 3136 may be obtained via the serial communications protocol through interface 3199 and flex strip(s) 3130. In some embodiments, the serial communications protocol may be SDI (serial digital interface). However, other communications protocols may be used. In some embodiments, SDI may provide a 30 MHz rate at 16 bit color depth, and may maintain a 60 Hz per second frame rate.
In various embodiments, controller 3190 may be a uniprocessor system including one processor 3192, or a multiprocessor system including several processors 3192 (e.g., two, four, eight, or another suitable number). Controller 3190 may include central processing units (CPUs) that implement any suitable instruction set architecture, and may execute instructions defined in that instruction set architecture. For example, in various embodiments controller 3190 may include general-purpose or embedded processors implementing any of a variety of instruction set architectures (ISAs), such as the x86, PowerPC, SPARC, RISC, or MIPS ISAs, or any other suitable ISA. In multiprocessor systems, each of the processors may commonly, but not necessarily, implement the same ISA. Controller 3190 may employ any microarchitecture, including scalar, superscalar, pipelined, superpipelined, out of order, in order, speculative, non-speculative, etc., or combinations thereof. Controller 3190 may include circuitry to implement microcoding techniques. Controller 3190 may include one or more processing cores that each execute instructions. Controller 3190 may include one or more levels of caches, which may employ any size and any configuration (set associative, direct mapped, etc.). In some embodiments, controller 3190 may include at least one graphics processing unit (GPU), which may include any suitable graphics processing circuitry. Generally, a GPU may render objects to be displayed into a frame buffer (e.g., one that includes pixel data for an entire frame). A GPU may include one or more graphics processors that may execute graphics software to perform a part or all of the graphics operation, or hardware acceleration of certain graphics operations. In some embodiments, controller 3190 may include one or more other components for processing and rendering video and/or images, for example image signal processors (ISPs), coder/decoders (codecs), etc.
Memory 3194 may include any type of memory, such as dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate (DDR, DDR2, DDR3, etc.) SDRAM (including mobile versions of the SDRAMs such as mDDR3, etc., or low power versions of the SDRAMs such as LPDDR2, etc.), RAMBUS DRAM (RDRAM), static RAM (SRAM), etc. In some embodiments, one or more memory devices may be coupled onto a circuit board to form memory modules such as single inline memory modules (SINMs), dual inline memory modules (DIMMs), etc. Alternatively, the devices may be mounted with an integrated circuit implementing system in a chip-on-chip configuration, a package-on-package configuration, or a multi-chip module configuration.
As indicated at 4010, the spatial light system may obtain image content to be projected into the room. The image content may be obtained from an external device, for example via a wireless connection, or alternatively may be generated based on image information stored locally in memory of the system.
As indicated at 4020, a spatial light system controller may drive the projection units according to the image content and room information to project the image content into at least a portion of the room. The system may, for example, project illuminating light, AR imagery, patterns, or other image content onto the floor, walls, and/or ceiling of a room. However, the system may also intelligently light or augment particular objects found within a room, and/or intelligently not illuminate certain surfaces or objects such as windows or television screens, or persons, within a room, based on the room information obtained from an external device.
In some embodiments, to drive the projection units, the spatial light system controller generates or receives video signals (e.g., from an external device such as an AR/VR device, pad or tablet device, or smartphone) and processes the signals into separate video drives for each projection units so that the final image(s) projected by all the projection units is correctly stitched and continuous. The controller then provides the correct pulse width modulation to each LED pixel of each projection unit so that the correct light output is achieved to project the imagery.
As indicated by the arrow returning from 4020 to 4010, the method may continue to obtain or generate image content and project the image content while the system is in use. While not specifically shown, additional or updated room information may be obtained from an external device while the system is in use; the additional or updated information may be used to adjust projection of the image content into the room.
As indicated at 4110, the spatial light system may obtain image content to be projected into the room. The image content may be obtained from an external device, for example via a wireless connection, or alternatively may be generated based on image information stored locally in memory of the system.
As indicated at 4120, a spatial light system controller may drive the projection units according to the image content and room information to project the image content into at least a portion of the room. The system may, for example, project illuminating light, AR imagery, patterns, or other image content onto the floor, walls, and/or ceiling of a room. However, the system may also intelligently light or augment particular objects found within a room, and/or intelligently not illuminate certain surfaces or objects such as windows or television screens, or persons, within a room, based on the room information obtained from an external device.
In some embodiments, to drive the projection units, the spatial light system controller generates or receives video signals (e.g., from an external device such as an AR/VR device, pad or tablet device, or smartphone) and processes the signals into separate video drives for each projection units so that the final image(s) projected by all the projection units is correctly stitched and continuous. The controller then provides the correct pulse width modulation to each LED pixel of each projection unit so that the correct light output is achieved to project the imagery.
As indicated by the arrow returning from 4120 to 4110, the method may continue to obtain or generate image content and project the image content while the system is in use. While not specifically shown, additional data may be obtained from the sensors while the system is in use; the additional or data may be used to adjust the room information and thus the projection of the image content into the room.
While embodiments are generally described in which the flexible strips are attached to a shaped object such as a sphere or cylinder to form a spatial light system, the flexible strips may be attached to other objects, or to walls, ceilings, floors, or other surfaces, within a room to provide custom lighting configurations with similar capabilities as the spatial light system configurations described herein.
Color Display Content
In some embodiments, color display content can be provided by the projector(s) in the AR light systems described herein by using an array of discrete red, green, and blue conventional LED sub-pixels. In some embodiments, a tunable white output may be achieved with selected color temperature conventional white LEDs, selectively driven within the LED arrays. In some embodiments, arrays of monolithic Indium Gallium Nitride (InGaN) tunable color LEDs can be used, for example as described in: Chen, J. C., et al. “Growth of monolithic full-color light-emitting diode and its applications.” Gallium Nitride Materials and Devices XIV. Vol. 10918. International Society for Optics and Photonics, 2019. In some embodiments, LEDs containing three different sets of quantum wells separated with intermediate carrier blocking layers may be used. In some embodiments, pulsed drive current containing different current amplitudes and widths, each creating a separate color, may be used. Alternatively a white output with tunable color temperature can be achieved.
In some embodiments, white light may be created using a UV source (often around 400 nm) which is then excited a phosphor that is deposited on the device window. The UV illumination is unconverted by the excited phosphor to create a broad spectrum of illumination that results in white light output. In some embodiments, the white light may be “tuned” by changing the phosphor in real-time (e.g., a tilting disc, color wheel, or other combination of beam-steering through appropriate filtering) to create slightly shifted white balance. In some embodiments, RGB light sources can be added in together and “mixed” to enhance or further shift the white spectrum being generated.
Spatial Light System Use Cases and Applications
The following describes several example use cases and applications for embodiments of a spatial light system as described herein. A spatial light system may be a device that lies somewhere between a projector and a light bulb. A spatial light system may be as bright as a light bulb, and may generate as high-quality light as a light bulb, but is higher resolution than a light bulb. However, a spatial light system may not be as high resolution as a conventional projector. Following from this, a spatial light system allows a lot of interesting use cases and applications.
In some embodiments, a spatial light system may be configured as a sphere that has a number of emitting pixels; each dot on the sphere corresponds to an LED that is projected onto the wall; the dots may be blended to generate imagery, for example sky and cloud images that an occupant of a room sees in the background, projected onto surfaces of the room by the spatial light system. The projected pixels on a wall may look fairly big and the imagery may be fairly low resolution when compared to pixels and imagery projected by a conventional projector.
Embodiments of a spatial light system may also rely on understanding the room that the system is in. Using room information obtained from an external source and/or from sensors of the system, light can be selectively projected or not projected onto particular surfaces or objects in the room. The room information may also be used to modulate brightness to avoid “hotspots” or similar undesirable lighting.
Embodiments of a spatial light system may be used to illuminate surfaces, floors, ceilings, or walls. Embodiments of a spatial light system may be used to create a spotlight that moves within a room without moving parts by turning on and off pixels. Embodiments of a spatial light system may be used to create architectural lighting, to highlight photographs or paintings, or to create accent lighting with different color temperatures, all without changing any hardware in the room.
Embodiments of a spatial light system may be used to create effects such rain, tree shadows, etc. that bring outside elements into the room, or to create caustics and abstract effects within a room or portion of a room,
Embodiments of a spatial light system also be used to interact with other devices in the room. Embodiments of a spatial light system may be used to expand or extend the content of devices, for example by expending the content of a television into the entire room.
In some embodiments, a spatial light system may emit spatial light in combination with other devices, for example to provide the expansion of television content. For example, embodiments of a spatial light system may generate content that can break out of a television; for example, a spatial light system may generate a Peter Pan-like shadow that walks around on the walls or ceiling of the room.
In some embodiments, the system may emit light to provide a low resolution version of a virtual environment that a person in the room is experiencing in virtual reality (e.g., using a headset or head-mounted device (HMD)) to give context to other people in the room without virtual reality headsets. In some embodiments, the system may emit light to complement a mixed reality environment being experienced by a person using an augmented reality headset by using lighting to improve contrast and colors viewed in the headset.
Embodiments of a spatial light system may create a virtual portal window. For example, for a person working from home, a virtual portal window may be generated to make a room at home feel more like an open office when working, or to provide a virtual window in an enclosed room.
Embodiments of a spatial light system may be controlled from an external device such as a smartphone, pad or tablet using touch, keystrokes, and/or voice commands. Using sensor technology and the ability to spotlight locations within a room, a voice command could be given requesting the location of an item such as keys; in response, the spatial light system could “shine a spotlight” on the keys.
Embodiments of a spatial light system may adjust the rendering of imagery within a room based on a detected point of view of a person within the room. Embodiments of a spatial light system may also affect the projected light based on the presence and movement of a person or persons within a room, for example to track and spotlight a moving person, or to not emit light towards the face of a person detected within the room.
Embodiments of a spatial light system may be used with a camera to improve photography by using emitted light to create bounced light, or different studio effects.
Embodiments of a spatial light system may be used to create cool or warm accent lighting within a room.
Most of these examples rely on embodiments of a spatial light system that includes projection units with multiple LEDs on a surface of a sphere. The projection unit technology may also be used to create a light bulb that projects imagery onto a localized object or surface. A light or lamp in this configuration may have the same or similar capabilities of understanding the room environment, and may provide a low resolution, bright projector based on LEDs that can do similar things as a sphere-shaped spatial light system.
Two or more spatial light systems may be used together in a room, and may interact with each other, either by direct communication via a wired or wireless connection or through an external device. Using room information obtained by spatial light system sensors or from an external device, each spatial light system may know where the other spatial light system(s) are located in the room, and may work together to complement each other, and to generate imagery from the combined light of the spatial light systems. In some embodiments, two or more spatial light systems may work in synchronization with each other, aware of their respective locations, orientations, and light output through wired or wireless communications channel(s), (e.g., ultra-wideband (UWB), cameras, sensors, pulsed visible or non-visible light signals, etc.) or through external device calibration, for example to create more room filling experiences, to avoid issues such as shadows, and/or to increase overall light output.
In addition to sphere and light bulb configurations, embodiments may be formed in other shapes, for example as cylinders or disks as described herein.
Extended Reality
A real environment refers to an environment that a person can perceive (e.g. see, hear, feel) without use of a device. For example, an office environment may include furniture such as desks, chairs, and filing cabinets; structural items such as doors, windows, and walls; and objects such as electronic devices, books, and writing instruments. A person in a real environment can perceive the various aspects of the environment, and may be able to interact with objects in the environment.
An extended reality (XR) environment, on the other hand, is partially or entirely simulated using an electronic device. In an XR environment, for example, a user may see or hear computer generated content that partially or wholly replaces the user's perception of the real environment. Additionally, a user can interact with an XR environment. For example, the user's movements can be tracked and virtual objects in the XR environment can change in response to the user's movements. As a further example, a device presenting an XR environment to a user may determine that a user is moving their hand toward the virtual position of a virtual object, and may move the virtual object in response. Additionally, a user's head position and/or eye gaze can be tracked and virtual objects can move to stay in the user's line of sight.
Examples of XR include augmented reality (AR), virtual reality (VR) and mixed reality (MR). XR can be considered along a spectrum of realities, where VR, on one end, completely immerses the user, replacing the real environment with virtual content, and on the other end, the user experiences the real environment unaided by a device. In between are AR and MR, which mix virtual content with the real environment.
VR generally refers to a type of XR that completely immerses a user and replaces the user's real environment. For example, VR can be presented to a user using a head mounted device (HMD), which can include a near-eye display to present a virtual visual environment to the user and headphones to present a virtual audible environment. In a VR environment, the movement of the user can be tracked and cause the user's view of the environment to change. For example, a user wearing a HMD can walk in the real environment and the user will appear to be walking through the virtual environment they are experiencing. Additionally, the user may be represented by an avatar in the virtual environment, and the user's movements can be tracked by the HMD using various sensors to animate the user's avatar.
AR and MR refer to a type of XR that includes some mixture of the real environment and virtual content. For example, a user may hold a tablet that includes a camera that captures images of the user's real environment. The tablet may have a display that displays the images of the real environment mixed with images of virtual objects. AR or MR can also be presented to a user through an HMD. An HMD can have an opaque display, or can use a see-through display, which allows the user to see the real environment through the display, while displaying virtual content overlaid on the real environment.
There are many types of devices that allow a user to experience the various forms of XR. Examples include HMDs, heads up displays (HUDs), projector-based systems, smart windows, tablets, desktop or laptop computers, smart watches, earbuds/headphones, controllers that may include haptic devices, and many others. As mentioned above, an HMD, or any of the other devices listed above may include opaque displays (e.g. liquid crystal displays (LCDs), organic light emitting diode (OLED) displays or micro-LED displays) or see through displays. A see through display can have a medium through which light is directed to a user's eyes. The medium can include one or more of a waveguide, hologram medium, optical combiner, optical reflector and other optical components. An image can be generated and propagated through the medium using a display source such as OLEDs, micro-LEDs, liquid crystal on silicon (LCOS), a light scanner, digital light projection (DLP).
Devices for XR may also include audio output devices such as speakers to present audio (including spatial audio) to users, haptics devices to stimulate the user's sense of touch, and other devices to stimulate any of the user's senses. Additionally, the device may include numerous sensors, including cameras, microphones, depth sensors, eye tracking sensors, environmental sensors, input sensors, and other sensors to allow the device to understand the user and the real environment.
The following clauses describe various examples embodiments consistent with the description provided herein.
Clause 1. A projection unit, comprising:
The methods described herein may be implemented in software, hardware, or a combination thereof, in different embodiments. In addition, the order of the blocks of the methods may be changed, and various elements may be added, reordered, combined, omitted, modified, etc. Various modifications and changes may be made as would be obvious to a person skilled in the art having the benefit of this disclosure. The various embodiments described herein are meant to be illustrative and not limiting. Many variations, modifications, additions, and improvements are possible. Accordingly, plural instances may be provided for components described herein as a single instance. Boundaries between various components, operations and data stores are somewhat arbitrary, and particular operations are illustrated in the context of specific illustrative configurations. Other allocations of functionality are envisioned and may fall within the scope of claims that follow. Finally, structures and functionality presented as discrete components in the example configurations may be implemented as a combined structure or component. These and other variations, modifications, additions, and improvements may fall within the scope of embodiments as defined in the claims that follow.
This application claims benefit of priority to U.S. Provisional Application Ser. No. 63/146,506, entitled “Spatial Light System,” filed Feb. 5, 2021, and which is hereby incorporated herein by reference in its entirety.
| Number | Name | Date | Kind |
|---|---|---|---|
| 7300179 | LaDuke et al. | Nov 2007 | B1 |
| 8317331 | Li | Nov 2012 | B2 |
| 8736675 | Holzbach | May 2014 | B1 |
| 8879782 | Garlington et al. | Nov 2014 | B2 |
| 9134593 | Worley, III | Sep 2015 | B1 |
| 9239627 | Kryze et al. | Jan 2016 | B2 |
| 9261762 | Kim | Feb 2016 | B2 |
| 10004984 | Voris et al. | Jun 2018 | B2 |
| 10297082 | Wilson et al. | May 2019 | B2 |
| 10388641 | Martin | Aug 2019 | B2 |
| 10921878 | Noris et al. | Feb 2021 | B2 |
| 20070092189 | Morejon | Apr 2007 | A1 |
| 20090051831 | Bierhuizen | Feb 2009 | A1 |
| 20100309390 | Plut | Dec 2010 | A1 |
| 20130342813 | Wang | Dec 2013 | A1 |
| 20140160442 | Lee | Jun 2014 | A1 |
| 20140185285 | Jorgensen | Jul 2014 | A1 |
| 20160109102 | Kim | Apr 2016 | A1 |
| 20160150201 | Kilche | May 2016 | A1 |
| 20180129167 | Maimone | May 2018 | A1 |
| 20190212642 | McMillan | Jul 2019 | A1 |
| 20190369298 | Ja | Dec 2019 | A1 |
| 20200081492 | Wang | Mar 2020 | A1 |
| 20200088358 | Minor | Mar 2020 | A1 |
| 20200236759 | Jurik | Jul 2020 | A1 |
| Entry |
|---|
| U.S. Appl. No. 63/058,240, filed Feb. 5, 2021, Felipe Bacim de Araujo e Silva, et al. |
| U.S. Appl. No. 17/240,447, filed Apr. 26, 2021, Michael D. Simmonds, et al. |
| U.S. Appl. No. 17/935,010, filed Sep. 23, 2022, Felipe Bacim de Araujo e Silva, et al. |
| Number | Date | Country | |
|---|---|---|---|
| 63146506 | Feb 2021 | US |
