Augmented reality lighting effects

Abstract

The present invention embraces a system, device, and method for adding lighting effects to augmented reality (AR) content (i.e., virtual objects). Light sensors in an augmented reality (AR) system monitor an environment's lighting conditions to acquire lighting data that can be used to create (or update) virtual light sources. Depth sensors in the AR system sense the environment to acquire mapping data that can be used to create a 3D model of the environment while tracking the system's location within the environment. Algorithms running on a processor may then add the virtual light sources to the 3D model of the environment so that, when AR content is created, lighting effects corresponding to the virtual light sources can be added. The resulting AR content with virtual lighting effects appear more realistic to a user.

Description

FIELD OF THE INVENTION

The present invention relates to augmented reality (AR) and more specifically, to a system for generating AR content with computer-generated lighting effects.

BACKGROUND

The use of AR devices is becoming more prevalent. These devices are typically worn on a user's head and are used to display information that augments the user's visual experience. When used in the workplace, a worker may use this information to analyze/understand their environment, leading to enhanced productivity and effectiveness.

The AR experience is created by presenting content (e.g., text, graphics, images, etc.) that overlay the user's field of view (FOV). This content is typically positioned so that it lends context to things (e.g., objects, people, etc.) within the user's immediate environment.

Lighting (e.g., light levels, shading, color, etc.) within a user's field of view (FOV) may change dramatically as the user moves. When AR content is displayed without regard to changes in lighting, it tends to look artificial, distracting, or worse, may become obscured. Therefore, a need exists for AR content that changes in appearance as the lighting in the user's environment changes.

SUMMARY

Accordingly, in one aspect, the present invention embraces an augmented reality (AR) system. The system includes a display to present AR content so that it overlaps with the AR system's perspective view of an environment. The system also includes one or more light sensors. The light sensors gather light data from light sources in the environment. The AR system also includes one or more depth sensors. The depth sensors gather mapping data of physical objects and light sources in the environment. Data from the one or more light sensors and the one or more depth sensors are fed to a computing device, included as part of the AR system. The computing device includes a processor that, when configured by software, can add lighting effects to AR content. To achieve this rendering, the software constructs a three-dimensional (3D) model of the environment using the mapping data. The software detects and characterizes the light sources in the environment using the light data. Virtual light sources are then created and added to the 3D model. The virtual light sources are used to add lighting effects to created AR content. The resulting AR content with lighting effects is then transmitted to the AR system's display for display to a user.

In an exemplary embodiment of the AR system, the lighting effects correspond to the environment's ambient light level.

In another exemplary embodiment of the AR system, the lighting effects correspond to the characteristics of the virtual light sources, as well as the position/orientation of the virtual light sources with respect to the AR content. In one possible embodiment, the characteristics of the virtual light sources include the direction of radiation. In another possible embodiment, the characteristics of the virtual light sources include light color. In still another possible embodiment, the characteristics of the virtual light sources include light intensity.

In another exemplary embodiment of the AR system, the AR content includes a graphical 3D object.

In another exemplary embodiment of the AR system, the at least one light sensor includes a charge-coupled device (CCD), and in one possible embodiment the detection of light sources includes comparing the CCD's pixel values to a threshold level.

In another exemplary embodiment of the AR system, the at least one depth sensor includes an optical 3D scanner.

In another exemplary embodiment of the AR system, the display includes a transparent plate that is positioned in front of the user's eye or eyes to allow a user to view the environment through the transparent plate. The transparent plate is arranged to display AR content to the user's eye (or eyes) so that the AR content appears superimposed on the user's view of the environment.

In another exemplary embodiment of the AR system, the display includes a liquid crystal display (LCD).

In another exemplary embodiment, the adding of virtual light sources to the 3D model of the environment is facilitated by simultaneous location and mapping (SLAM) techniques.

In another aspect, the present invention embraces a method for applying lighting effects to virtual objects for an augmented reality (AR) system. The method begins with the step of receiving lighting information and position information from at least one light sensor and at least one depth sensor respectively. Next, lighting effects are determined from the lighting information and the position information, and the lighting effects are applied to the virtual objects. The lighting effects are then updated if either the position information changes or the lighting information changes.

In an exemplary embodiment of the method for applying lighting effects to virtual objects for an AR system, the AR system includes a head mounted display (HMD).

In another exemplary embodiment of the method for applying lighting effects to virtual objects for an AR system, the at least one light sensor includes a CCD.

In another exemplary embodiment of the method for applying lighting effects to virtual objects for an AR system, the lighting effects include coloring, shading, and/or lightening at least a portion of a virtual object.

In another aspect, the present invention embraces an augmented reality (AR) device. The device includes a light sensor to gather light data for use in constructing virtual light sources. The device also includes a depth sensor to gather mapping data for constructing a 3D model of an environment. A processor is included with the device. The processor is communicatively coupled to the light sensor and the depth sensor. Software enables the processor to construct the 3D model of the environment including the virtual light sources. The processor is further enabled to create AR content, to which light effects are added. The lighting effects correspond to the virtual light sources.

In an exemplary embodiment of the AR device, the virtual light sources include a diffuse light source, a point light source, a directional light source, a distributed light source, and/or an ambient light source.

In another exemplary embodiment of the AR device, adding lighting effects to the AR content includes changing the AR content's color, intensity, and/or shading to appear illuminated by a light source.

The foregoing illustrative summary, as well as other exemplary objectives and/or advantages of the invention, and the manner in which the same are accomplished, are further explained within the following detailed description and its accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 graphically depicts, according to an embodiment of the present invention, a user wearing an exemplary AR device and an exemplary output of said AR device seen by the user.

FIG. 2 graphically depicts (i) an object illuminated by a light source, (ii) an AR system, and (ii) the output of the AR system including AR content with lighting effects according to an embodiment of the present invention.

FIG. 3 schematically depicts a system/method for applying lighting effects to AR content according to an embodiment of the present invention.

DETAILED DESCRIPTION

The present invention embraces a system, method, and device for enhancing a mixed reality experience. Mixed reality refers to the merging of real and virtual (i.e., computer generated) worlds. Augmented reality lies within the spectrum of mixed reality experiences.

Augmented reality (AR) systems allow a user to view and (in some cases) interact with an enhanced version of the physical world. AR systems combine a user's perspective view of the physical world (i.e., the user's environment) with virtual objects. The virtual objects may be overlaid and positioned within the user's perspective view to provide contextually relevant information.

Virtual objects may include graphics or text and may be presented in two dimensions (2D) and/or three dimensions (3D). The virtual objects (i.e., AR content) are continually updated (e.g., real time) to correspond with a user's change in perspective. As such, AR systems typically include body-worn cameras/displays (e.g., head mounted display) or hand-held cameras/displays (e.g., smartphone, tablet, etc.).

A head mounted display (HMD) may be part of an AR system. One possible HMD type is the video see-through HMD. Here, the environment is presented as a video stream to the user via a display (e.g., a liquid crystal display). Another possible HMD type is the optical see-through HMD (e.g., smart glasses), wherein the user looks through a transparent plate. The transparent plate is configured to display AR content so the AR content is overlaid with the user's perspective view of the environment.

An exemplary AR device is shown in FIG. 1. The AR device 1 is a smart glasses type HMD (e.g., MICROSOFT™ HOLOLENS™). When a user 2 wears the AR device like a pair of glasses, AR content 5 is presented to both eyes. This AR content may appear 3D resulting from the stereoscopic view and the display's ability to create “holograms” of virtual objects. The user's perspective view 3 of an environment 4 is displayed to a user with AR content 5 overlaid and positioned to help the user understand the environment 4.

The AR content 5 may change in response to movement of the AR device 1 within the environment (i.e., position). These changes typically occur in real time allowing a user to move freely while the AR content 5 updates appropriately to match changes in the user's perspective.

Tracking of the AR device's position/orientation is required to update the AR content 5 appropriately. Tracking may utilize or more sensors to determine the user's position/orientation. For example, inertial measurement sensors (e.g., gyroscope, accelerometer, magnetometer, etc.) may facilitate tracking. In addition, tracking may also utilize depth sensors.

Depth sensing may be used to create range images of the AR system's perspective. Range images are images with pixel values corresponding to the range between the AR system and points within the AR system's field of view.

Depth sensors (i.e., range cameras) may produce these range images using one of several possible techniques (e.g., stereo triangulation, sheet of light triangulation, structured light, time of flight, interferometry, coded aperture, etc.). Structure light depth sensors, for example, illuminate an environment with a specially designed light pattern (e.g., points, checkerboard, lines, etc.). The reflected light pattern is compared to a reference pattern to obtain a range image.

AR systems may include a camera to help tracking and mapping. This camera (e.g., CCD camera, CMOS camera, etc.) is typically aligned with the perspective view of the user. The images captured by the camera may be processed by processors running algorithms (such as simultaneous localization and mapping (SLAM)) to track and map. SLAM algorithms may aid in the creation of maps (i.e., models) of the environment, which include the locations of physical objects and/or light sources in the environment.

Detecting light sources for mapping may be accomplished using the camera or by using one of a variety of possible photo sensor types (e.g., photodiodes, phototransistors, etc.). For example, light levels measured by the light sensor (e.g., camera, photo sensor, etc.) may be compared to a threshold as part of a light-source detection process.

One challenge facing AR is creating virtual objects that appear real (i.e., as if they were part of the physical world). One aspect contributing to an object's realism is lighting. Physical objects are illuminated by a variety of light sources (e.g., diffuse light sources, point light sources, directional light sources, distributed light sources, ambient light source, etc.). This illumination creates lighting effects.

Lighting effects may include a change in an object's color (e.g., red light illuminates an object). Lighting effects may also include a change in an object's brightness (i.e., an object that reflects a high-intensity light may appear bright). Lighting effects may also include shadows on the object. For realism, a virtual object in a virtual environment should have the same lighting effects as a corresponding real object would have when placed in the same position in a corresponding physical environment.

As shown in FIG. 2, an object 7 illuminated by a light source 6 has lighting effects. Object surfaces closer to the light source 6 appear fully illuminated (i.e., bright), while object surfaces not facing the light source 6 are in shadow (i.e., dark). When this environment is observed using an AR device 1, the view seen by a user 3 may include a virtual object (i.e., AR content) 5. For the exemplary embodiment shown in FIG. 2, the AR content includes an arrow 5 (e.g., to indicate this particular package for some operation). Here, the AR-content arrow 5 appears illuminated by a virtual light source that corresponds to the physical light source 6. Since these lighting effects match the proximate object's lighting effects, the virtual object appears more realistic.

FIG. 3 schematically depicts an AR system 9. The system includes depth sensors (e.g., 3D optical scanners) 11 for mapping objects and light sources. The system also includes light sensors (e.g., a CCD) 12 for detecting and characterizing light sources 6 in the environment. The system further includes a display (e.g., a HMD) 13 to facilitate a user's view of the environment 4, wherein the view also includes AR content 5. The depth sensors 11, light sensors 12, and display 13 are typically integrated with a HMD worn by a user.

A computing device 10 is including as part of the AR system 9. The computing device may be integrated with an HMD worn by a user. In some possible embodiments, however, the computing device 10 may be communicatively coupled to the HMD but physically separate from the HMD body. The computing device has a processor enabled by software to add lighting effects to AR content. Exemplary processors suitable for the present invention include (but are not limited to) microprocessors, application-specific integrated circuits (ASIC), graphics processing units (GPU), digital signal processors (DSP), image processors, and multi-core processors. It is possible that the AR system uses one or more of these processors types.

The computing device 10 is configured to receive mapping data from the depth sensors 11. Algorithms running on the processor use the mapping data to construct (or update) a 3D model of the user's environment 20. The resulting 3D model includes the position of the AR system relative to the physical surfaces of objects (e.g., walls, furniture, objects, etc.).

The computing device 10 is also configured to receive light data from the light sensors 12. Algorithms running on the processor use the light data to detect and characterize the light sources 21. The results of these algorithms 21 include the position of the light sources (e.g., relative to the AR system and/or the 3D model) and lighting characteristics (e.g., color, intensity, directionality, etc.) of the light detected light sources. From this information, virtual light sources are created (or updated).

Algorithms then configure the computing device 10 to add the virtual light sources to the 3D model 22 so that, after AR content is created 23, lighting effects can be added 24. The AR content with lighting effects may then be transmitted from the computing device 10 to the display 13.

To supplement the present disclosure, this application incorporates entirely by reference the following commonly assigned patents, patent application publications, and patent applications:

• U.S. Pat. Nos. 6,832,725; 7,128,266;
• 7,159,783; 7,413,127;
• 7,726,575; 8,294,969;
• 8,317,105; 8,322,622;
• 8,366,005; 8,371,507;
• 8,376,233; 8,381,979;
• 8,390,909; 8,408,464;
• 8,408,468; 8,408,469;
• 8,424,768; 8,448,863;
• 8,457,013; 8,459,557;
• 8,469,272; 8,474,712;
• 8,479,992; 8,490,877;
• 8,517,271; 8,523,076;
• 8,528,818; 8,544,737;
• 8,548,242; 8,548,420;
• 8,550,335; 8,550,354;
• 8,550,357; 8,556,174;
• 8,556,176; 8,556,177;
• 8,559,767; 8,599,957;
• 8,561,895; 8,561,903;
• 8,561,905; 8,565,107;
• 8,571,307; 8,579,200;
• 8,583,924; 8,584,945;
• 8,587,595; 8,587,697;
• 8,588,869; 8,590,789;
• 8,596,539; 8,596,542;
• 8,596,543; 8,599,271;
• 8,599,957; 8,600,158;
• 8,600,167; 8,602,309;
• 8,608,053; 8,608,071;
• 8,611,309; 8,615,487;
• 8,616,454; 8,621,123;
• 8,622,303; 8,628,013;
• 8,628,015; 8,628,016;
• 8,629,926; 8,630,491;
• 8,635,309; 8,636,200;
• 8,636,212; 8,636,215;
• 8,636,224; 8,638,806;
• 8,640,958; 8,640,960;
• 8,643,717; 8,646,692;
• 8,646,694; 8,657,200;
• 8,659,397; 8,668,149;
• 8,678,285; 8,678,286;
• 8,682,077; 8,687,282;
• 8,692,927; 8,695,880;
• 8,698,949; 8,717,494;
• 8,717,494; 8,720,783;
• 8,723,804; 8,723,904;
• 8,727,223; D702,237;
• 8,740,082; 8,740,085;
• 8,746,563; 8,750,445;
• 8,752,766; 8,756,059;
• 8,757,495; 8,760,563;
• 8,763,909; 8,777,108;
• 8,777,109; 8,779,898;
• 8,781,520; 8,783,573;
• 8,789,757; 8,789,758;
• 8,789,759; 8,794,520;
• 8,794,522; 8,794,525;
• 8,794,526; 8,798,367;
• 8,807,431; 8,807,432;
• 8,820,630; 8,822,848;
• 8,824,692; 8,824,696;
• 8,842,849; 8,844,822;
• 8,844,823; 8,849,019;
• 8,851,383; 8,854,633;
• 8,866,963; 8,868,421;
• 8,868,519; 8,868,802;
• 8,868,803; 8,870,074;
• 8,879,639; 8,880,426;
• 8,881,983; 8,881,987;
• 8,903,172; 8,908,995;
• 8,910,870; 8,910,875;
• 8,914,290; 8,914,788;
• 8,915,439; 8,915,444;
• 8,916,789; 8,918,250;
• 8,918,564; 8,925,818;
• 8,939,374; 8,942,480;
• 8,944,313; 8,944,327;
• 8,944,332; 8,950,678;
• 8,967,468; 8,971,346;
• 8,976,030; 8,976,368;
• 8,978,981; 8,978,983;
• 8,978,984; 8,985,456;
• 8,985,457; 8,985,459;
• 8,985,461; 8,988,578;
• 8,988,590; 8,991,704;
• 8,996,194; 8,996,384;
• 9,002,641; 9,007,368;
• 9,010,641; 9,015,513;
• 9,016,576; 9,022,288;
• 9,030,964; 9,033,240;
• 9,033,242; 9,036,054;
• 9,037,344; 9,038,911;
• 9,038,915; 9,047,098;
• 9,047,359; 9,047,420;
• 9,047,525; 9,047,531;
• 9,053,055; 9,053,378;
• 9,053,380; 9,058,526;
• 9,064,165; 9,064,167;
• 9,064,168; 9,064,254;
• 9,066,032; 9,070,032;
• U.S. Design Patent No. D716,285;
• U.S. Design Patent No. D723,560;
• U.S. Design Patent No. D730,357;
• U.S. Design Patent No. D730,901;
• U.S. Design Patent No. D730,902;
• U.S. Design Patent No. D733,112;
• U.S. Design Patent No. D734,339;
• International Publication No. 2013/163789;
• International Publication No. 2013/173985;
• International Publication No. 2014/019130;
• International Publication No. 2014/110495;
• U.S. Patent Application Publication No. 2008/0185432;
• U.S. Patent Application Publication No. 2009/0134221;
• U.S. Patent Application Publication No. 2010/0177080;
• U.S. Patent Application Publication No. 2010/0177076;
• U.S. Patent Application Publication No. 2010/0177707;
• U.S. Patent Application Publication No. 2010/0177749;
• U.S. Patent Application Publication No. 2010/0265880;
• U.S. Patent Application Publication No. 2011/0202554;
• U.S. Patent Application Publication No. 2012/0111946;
• U.S. Patent Application Publication No. 2012/0168511;
• U.S. Patent Application Publication No. 2012/0168512;
• U.S. Patent Application Publication No. 2012/0193423;
• U.S. Patent Application Publication No. 2012/0203647;
• U.S. Patent Application Publication No. 2012/0223141;
• U.S. Patent Application Publication No. 2012/0228382;
• U.S. Patent Application Publication No. 2012/0248188;
• U.S. Patent Application Publication No. 2013/0043312;
• U.S. Patent Application Publication No. 2013/0082104;
• U.S. Patent Application Publication No. 2013/0175341;
• U.S. Patent Application Publication No. 2013/0175343;
• U.S. Patent Application Publication No. 2013/0257744;
• U.S. Patent Application Publication No. 2013/0257759;
• U.S. Patent Application Publication No. 2013/0270346;
• U.S. Patent Application Publication No. 2013/0287258;
• U.S. Patent Application Publication No. 2013/0292475;
• U.S. Patent Application Publication No. 2013/0292477;
• U.S. Patent Application Publication No. 2013/0293539;
• U.S. Patent Application Publication No. 2013/0293540;
• U.S. Patent Application Publication No. 2013/0306728;
• U.S. Patent Application Publication No. 2013/0306731;
• U.S. Patent Application Publication No. 2013/0307964;
• U.S. Patent Application Publication No. 2013/0308625;
• U.S. Patent Application Publication No. 2013/0313324;
• U.S. Patent Application Publication No. 2013/0313325;
• U.S. Patent Application Publication No. 2013/0342717;
• U.S. Patent Application Publication No. 2014/0001267;
• U.S. Patent Application Publication No. 2014/0008439;
• U.S. Patent Application Publication No. 2014/0025584;
• U.S. Patent Application Publication No. 2014/0034734;
• U.S. Patent Application Publication No. 2014/0036848;
• U.S. Patent Application Publication No. 2014/0039693;
• U.S. Patent Application Publication No. 2014/0042814;
• U.S. Patent Application Publication No. 2014/0049120;
• U.S. Patent Application Publication No. 2014/0049635;
• U.S. Patent Application Publication No. 2014/0061306;
• U.S. Patent Application Publication No. 2014/0063289;
• U.S. Patent Application Publication No. 2014/0066136;
• U.S. Patent Application Publication No. 2014/0067692;
• U.S. Patent Application Publication No. 2014/0070005;
• U.S. Patent Application Publication No. 2014/0071840;
• U.S. Patent Application Publication No. 2014/0074746;
• U.S. Patent Application Publication No. 2014/0076974;
• U.S. Patent Application Publication No. 2014/0078341;
• U.S. Patent Application Publication No. 2014/0078345;
• U.S. Patent Application Publication No. 2014/0097249;
• U.S. Patent Application Publication No. 2014/0098792;
• U.S. Patent Application Publication No. 2014/0100813;
• U.S. Patent Application Publication No. 2014/0103115;
• U.S. Patent Application Publication No. 2014/0104413;
• U.S. Patent Application Publication No. 2014/0104414;
• U.S. Patent Application Publication No. 2014/0104416;
• U.S. Patent Application Publication No. 2014/0104451;
• U.S. Patent Application Publication No. 2014/0106594;
• U.S. Patent Application Publication No. 2014/0106725;
• U.S. Patent Application Publication No. 2014/0108010;
• U.S. Patent Application Publication No. 2014/0108402;
• U.S. Patent Application Publication No. 2014/0110485;
• U.S. Patent Application Publication No. 2014/0114530;
• U.S. Patent Application Publication No. 2014/0124577;
• U.S. Patent Application Publication No. 2014/0124579;
• U.S. Patent Application Publication No. 2014/0125842;
• U.S. Patent Application Publication No. 2014/0125853;
• U.S. Patent Application Publication No. 2014/0125999;
• U.S. Patent Application Publication No. 2014/0129378;
• U.S. Patent Application Publication No. 2014/0131438;
• U.S. Patent Application Publication No. 2014/0131441;
• U.S. Patent Application Publication No. 2014/0131443;
• U.S. Patent Application Publication No. 2014/0131444;
• U.S. Patent Application Publication No. 2014/0131445;
• U.S. Patent Application Publication No. 2014/0131448;
• U.S. Patent Application Publication No. 2014/0133379;
• U.S. Patent Application Publication No. 2014/0136208;
• U.S. Patent Application Publication No. 2014/0140585;
• U.S. Patent Application Publication No. 2014/0151453;
• U.S. Patent Application Publication No. 2014/0152882;
• U.S. Patent Application Publication No. 2014/0158770;
• U.S. Patent Application Publication No. 2014/0159869;
• U.S. Patent Application Publication No. 2014/0166755;
• U.S. Patent Application Publication No. 2014/0166759;
• U.S. Patent Application Publication No. 2014/0168787;
• U.S. Patent Application Publication No. 2014/0175165;
• U.S. Patent Application Publication No. 2014/0175172;
• U.S. Patent Application Publication No. 2014/0191644;
• U.S. Patent Application Publication No. 2014/0191913;
• U.S. Patent Application Publication No. 2014/0197238;
• U.S. Patent Application Publication No. 2014/0197239;
• U.S. Patent Application Publication No. 2014/0197304;
• U.S. Patent Application Publication No. 2014/0214631;
• U.S. Patent Application Publication No. 2014/0217166;
• U.S. Patent Application Publication No. 2014/0217180;
• U.S. Patent Application Publication No. 2014/0231500;
• U.S. Patent Application Publication No. 2014/0232930;
• U.S. Patent Application Publication No. 2014/0247315;
• U.S. Patent Application Publication No. 2014/0263493;
• U.S. Patent Application Publication No. 2014/0263645;
• U.S. Patent Application Publication No. 2014/0267609;
• U.S. Patent Application Publication No. 2014/0270196;
• U.S. Patent Application Publication No. 2014/0270229;
• U.S. Patent Application Publication No. 2014/0278387;
• U.S. Patent Application Publication No. 2014/0278391;
• U.S. Patent Application Publication No. 2014/0282210;
• U.S. Patent Application Publication No. 2014/0284384;
• U.S. Patent Application Publication No. 2014/0288933;
• U.S. Patent Application Publication No. 2014/0297058;
• U.S. Patent Application Publication No. 2014/0299665;
• U.S. Patent Application Publication No. 2014/0312121;
• U.S. Patent Application Publication No. 2014/0319220;
• U.S. Patent Application Publication No. 2014/0319221;
• U.S. Patent Application Publication No. 2014/0326787;
• U.S. Patent Application Publication No. 2014/0332590;
• U.S. Patent Application Publication No. 2014/0344943;
• U.S. Patent Application Publication No. 2014/0346233;
• U.S. Patent Application Publication No. 2014/0351317;
• U.S. Patent Application Publication No. 2014/0353373;
• U.S. Patent Application Publication No. 2014/0361073;
• U.S. Patent Application Publication No. 2014/0361082;
• U.S. Patent Application Publication No. 2014/0362184;
• U.S. Patent Application Publication No. 2014/0363015;
• U.S. Patent Application Publication No. 2014/0369511;
• U.S. Patent Application Publication No. 2014/0374483;
• U.S. Patent Application Publication No. 2014/0374485;
• U.S. Patent Application Publication No. 2015/0001301;
• U.S. Patent Application Publication No. 2015/0001304;
• U.S. Patent Application Publication No. 2015/0003673;
• U.S. Patent Application Publication No. 2015/0009338;
• U.S. Patent Application Publication No. 2015/0009610;
• U.S. Patent Application Publication No. 2015/0014416;
• U.S. Patent Application Publication No. 2015/0021397;
• U.S. Patent Application Publication No. 2015/0028102;
• U.S. Patent Application Publication No. 2015/0028103;
• U.S. Patent Application Publication No. 2015/0028104;
• U.S. Patent Application Publication No. 2015/0029002;
• U.S. Patent Application Publication No. 2015/0032709;
• U.S. Patent Application Publication No. 2015/0039309;
• U.S. Patent Application Publication No. 2015/0039878;
• U.S. Patent Application Publication No. 2015/0040378;
• U.S. Patent Application Publication No. 2015/0048168;
• U.S. Patent Application Publication No. 2015/0049347;
• U.S. Patent Application Publication No. 2015/0051992;
• U.S. Patent Application Publication No. 2015/0053766;
• U.S. Patent Application Publication No. 2015/0053768;
• U.S. Patent Application Publication No. 2015/0053769;
• U.S. Patent Application Publication No. 2015/0060544;
• U.S. Patent Application Publication No. 2015/0062366;
• U.S. Patent Application Publication No. 2015/0063215;
• U.S. Patent Application Publication No. 2015/0063676;
• U.S. Patent Application Publication No. 2015/0069130;
• U.S. Patent Application Publication No. 2015/0071819;
• U.S. Patent Application Publication No. 2015/0083800;
• U.S. Patent Application Publication No. 2015/0086114;
• U.S. Patent Application Publication No. 2015/0088522;
• U.S. Patent Application Publication No. 2015/0096872;
• U.S. Patent Application Publication No. 2015/0099557;
• U.S. Patent Application Publication No. 2015/0100196;
• U.S. Patent Application Publication No. 2015/0102109;
• U.S. Patent Application Publication No. 2015/0115035;
• U.S. Patent Application Publication No. 2015/0127791;
• U.S. Patent Application Publication No. 2015/0128116;
• U.S. Patent Application Publication No. 2015/0129659;
• U.S. Patent Application Publication No. 2015/0133047;
• U.S. Patent Application Publication No. 2015/0134470;
• U.S. Patent Application Publication No. 2015/0136851;
• U.S. Patent Application Publication No. 2015/0136854;
• U.S. Patent Application Publication No. 2015/0142492;
• U.S. Patent Application Publication No. 2015/0144692;
• U.S. Patent Application Publication No. 2015/0144698;
• U.S. Patent Application Publication No. 2015/0144701;
• U.S. Patent Application Publication No. 2015/0149946;
• U.S. Patent Application Publication No. 2015/0161429;
• U.S. Patent Application Publication No. 2015/0169925;
• U.S. Patent Application Publication No. 2015/0169929;
• U.S. Patent Application Publication No. 2015/0178523;
• U.S. Patent Application Publication No. 2015/0178534;
• U.S. Patent Application Publication No. 2015/0178535;
• U.S. Patent Application Publication No. 2015/0178536;
• U.S. Patent Application Publication No. 2015/0178537;
• U.S. Patent Application Publication No. 2015/0181093;
• U.S. Patent Application Publication No. 2015/0181109;
• U.S. patent application Ser. No. 13/367,978 for a Laser Scanning Module Employing an Elastomeric U-Hinge Based Laser Scanning Assembly, filed February 2012 (Feng et al.);
• U.S. patent application Ser. No. 29/458,405 for an Electronic Device, filed Jun. 19, 2013 (Fitch et al.);
• U.S. patent application Ser. No. 29/459,620 for an Electronic Device Enclosure, filed Jul. 2, 2013 (London et al.);
• U.S. patent application Ser. No. 29/468,118 for an Electronic Device Case, filed Sep. 26, 2013 (Oberpriller et al.);
• U.S. patent application Ser. No. 14/150,393 for Indicia-reader Having Unitary Construction Scanner, filed Jan. 8, 2014 (Colavito et al.);
• U.S. patent application Ser. No. 14/200,405 for Indicia Reader for Size-Limited Applications filed Mar. 7, 2014 (Feng et al.);
• U.S. patent application Ser. No. 14/231,898 for Hand-Mounted Indicia-Reading Device with Finger Motion Triggering filed Apr. 1, 2014 (Van Horn et al.);
• U.S. patent application Ser. No. 29/486,759 for an Imaging Terminal, filed Apr. 2, 2014 (Oberpriller et al.);
• U.S. patent application Ser. No. 14/257,364 for Docking System and Method Using Near Field Communication filed Apr. 21, 2014 (Showering);
• U.S. patent application Ser. No. 14/264,173 for Autofocus Lens System for Indicia Readers filed Apr. 29, 2014 (Ackley et al.);
• U.S. patent application Ser. No. 14/277,337 for MULTIPURPOSE OPTICAL READER, filed May 14, 2014 (Jovanovski et al.);
• U.S. patent application Ser. No. 14/283,282 for TERMINAL HAVING ILLUMINATION AND FOCUS CONTROL filed May 21, 2014 (Liu et al.);
• U.S. patent application Ser. No. 14/327,827 for a MOBILE-PHONE ADAPTER FOR ELECTRONIC TRANSACTIONS, filed Jul. 10, 2014 (Hejl);
• U.S. patent application Ser. No. 14/334,934 for a SYSTEM AND METHOD FOR INDICIA VERIFICATION, filed Jul. 18, 2014 (Hejl);
• U.S. patent application Ser. No. 14/339,708 for LASER SCANNING CODE SYMBOL READING SYSTEM, filed Jul. 24, 2014 (Xian et al.);
• U.S. patent application Ser. No. 14/340,627 for an AXIALLY REINFORCED FLEXIBLE SCAN ELEMENT, filed Jul. 25, 2014 (Rueblinger et al.);
• U.S. patent application Ser. No. 14/446,391 for MULTIFUNCTION POINT OF SALE APPARATUS WITH OPTICAL SIGNATURE CAPTURE filed Jul. 30, 2014 (Good et al.);
• U.S. patent application Ser. No. 14/452,697 for INTERACTIVE INDICIA READER, filed Aug. 6, 2014 (Todeschini);
• U.S. patent application Ser. No. 14/453,019 for DIMENSIONING SYSTEM WITH GUIDED ALIGNMENT, filed Aug. 6, 2014 (Li et al.);
• U.S. patent application Ser. No. 14/462,801 for MOBILE COMPUTING DEVICE WITH DATA COGNITION SOFTWARE, filed on Aug. 19, 2014
• U.S. patent application Ser. No. 14/483,056 for VARIABLE DEPTH OF FIELD BARCODE SCANNER filed Sep. 10, 2014 (McCloskey et al.);
• U.S. patent application Ser. No. 14/513,808 for IDENTIFYING INVENTORY ITEMS IN A STORAGE FACILITY filed Oct. 14, 2014 (Singel et al.);
• U.S. patent application Ser. No. 14/519,195 for HANDHELD DIMENSIONING SYSTEM WITH FEEDBACK filed Oct. 21, 2014 (Laffargue et al.);
• U.S. patent application Ser. No. 14/519,179 for DIMENSIONING SYSTEM WITH MULTIPATH INTERFERENCE MITIGATION filed Oct. 21, 2014 (Thuries et al.);
• U.S. patent application Ser. No. 14/519,211 for SYSTEM AND METHOD FOR DIMENSIONING filed Oct. 21, 2014 (Ackley et al.);
• U.S. patent application Ser. No. 14/519,233 for HANDHELD DIMENSIONER WITH DATA-QUALITY INDICATION filed Oct. 21, 2014 (Laffargue et al.);
• U.S. patent application Ser. No. 14/519,249 for HANDHELD DIMENSIONING SYSTEM WITH MEASUREMENT-CONFORMANCE FEEDBACK filed Oct. 21, 2014 (Ackley et al.);
• U.S. patent application Ser. No. 14/527,191 for METHOD AND SYSTEM FOR RECOGNIZING SPEECH USING WILDCARDS IN AN EXPECTED RESPONSE filed Oct. 29, 2014 (Braho et al.);
• U.S. patent application Ser. No. 14/529,563 for ADAPTABLE INTERFACE FOR A MOBILE COMPUTING DEVICE filed Oct. 31, 2014 (Schoon et al.);
• U.S. patent application Ser. No. 14/529,857 for BARCODE READER WITH SECURITY FEATURES filed Oct. 31, 2014 (Todeschini et al.);
• U.S. patent application Ser. No. 14/398,542 for PORTABLE ELECTRONIC DEVICES HAVING A SEPARATE LOCATION TRIGGER UNIT FOR USE IN CONTROLLING AN APPLICATION UNIT filed Nov. 3, 2014 (Bian et al.);
• U.S. patent application Ser. No. 14/531,154 for DIRECTING AN INSPECTOR THROUGH AN INSPECTION filed Nov. 3, 2014 (Miller et al.);
• U.S. patent application Ser. No. 14/533,319 for BARCODE SCANNING SYSTEM USING WEARABLE DEVICE WITH EMBEDDED CAMERA filed Nov. 5, 2014 (Todeschini);
• U.S. patent application Ser. No. 14/535,764 for CONCATENATED EXPECTED RESPONSES FOR SPEECH RECOGNITION filed Nov. 7, 2014 (Braho et al.);
• U.S. patent application Ser. No. 14/568,305 for AUTO-CONTRAST VIEWFINDER FOR AN INDICIA READER filed Dec. 12, 2014 (Todeschini);
• U.S. patent application Ser. No. 14/573,022 for DYNAMIC DIAGNOSTIC INDICATOR GENERATION filed Dec. 17, 2014 (Goldsmith);
• U.S. patent application Ser. No. 14/578,627 for SAFETY SYSTEM AND METHOD filed Dec. 22, 2014 (Ackley et al.);
• U.S. patent application Ser. No. 14/580,262 for MEDIA GATE FOR THERMAL TRANSFER PRINTERS filed Dec. 23, 2014 (Bowles);
• U.S. patent application Ser. No. 14/590,024 for SHELVING AND PACKAGE LOCATING SYSTEMS FOR DELIVERY VEHICLES filed Jan. 6, 2015 (Payne);
• U.S. patent application Ser. No. 14/596,757 for SYSTEM AND METHOD FOR DETECTING BARCODE PRINTING ERRORS filed Jan. 14, 2015 (Ackley);
• U.S. patent application Ser. No. 14/416,147 for OPTICAL READING APPARATUS HAVING VARIABLE SETTINGS filed Jan. 21, 2015 (Chen et al.);
• U.S. patent application Ser. No. 14/614,706 for DEVICE FOR SUPPORTING AN ELECTRONIC TOOL ON A USER'S HAND filed Feb. 5, 2015 (Oberpriller et al.);
• U.S. patent application Ser. No. 14/614,796 for CARGO APPORTIONMENT TECHNIQUES filed Feb. 5, 2015 (Morton et al.);
• U.S. patent application Ser. No. 29/516,892 for TABLE COMPUTER filed Feb. 6, 2015 (Bidwell et al.);
• U.S. patent application Ser. No. 14/619,093 for METHODS FOR TRAINING A SPEECH RECOGNITION SYSTEM filed Feb. 11, 2015 (Pecorari);
• U.S. patent application Ser. No. 14/628,708 for DEVICE, SYSTEM, AND METHOD FOR DETERMINING THE STATUS OF CHECKOUT LANES filed Feb. 23, 2015 (Todeschini);
• U.S. patent application Ser. No. 14/630,841 for TERMINAL INCLUDING IMAGING ASSEMBLY filed Feb. 25, 2015 (Gomez et al.);
• U.S. patent application Ser. No. 14/635,346 for SYSTEM AND METHOD FOR RELIABLE STORE-AND-FORWARD DATA HANDLING BY ENCODED INFORMATION READING TERMINALS filed Mar. 2, 2015 (Sevier);
• U.S. patent application Ser. No. 29/519,017 for SCANNER filed Mar. 2, 2015 (Zhou et al.);
• U.S. patent application Ser. No. 14/405,278 for DESIGN PATTERN FOR SECURE STORE filed Mar. 9, 2015 (Zhu et al.);
• U.S. patent application Ser. No. 14/660,970 for DECODABLE INDICIA READING TERMINAL WITH COMBINED ILLUMINATION filed Mar. 18, 2015 (Kearney et al.);
• U.S. patent application Ser. No. 14/661,013 for REPROGRAMMING SYSTEM AND METHOD FOR DEVICES INCLUDING PROGRAMMING SYMBOL filed Mar. 18, 2015 (Soule et al.);
• U.S. patent application Ser. No. 14/662,922 for MULTIFUNCTION POINT OF SALE SYSTEM filed Mar. 19, 2015 (Van Horn et al.);
• U.S. patent application Ser. No. 14/663,638 for VEHICLE MOUNT COMPUTER WITH CONFIGURABLE IGNITION SWITCH BEHAVIOR filed Mar. 20, 2015 (Davis et al.);
• U.S. patent application Ser. No. 14/664,063 for METHOD AND APPLICATION FOR SCANNING A BARCODE WITH A SMART DEVICE WHILE CONTINUOUSLY RUNNING AND DISPLAYING AN APPLICATION ON THE SMART DEVICE DISPLAY filed Mar. 20, 2015 (Todeschini);
• U.S. patent application Ser. No. 14/669,280 for TRANSFORMING COMPONENTS OF A WEB PAGE TO VOICE PROMPTS filed Mar. 26, 2015 (Funyak et al.);
• U.S. patent application Ser. No. 14/674,329 for AIMER FOR BARCODE SCANNING filed Mar. 31, 2015 (Bidwell);
• U.S. patent application Ser. No. 14/676,109 for INDICIA READER filed Apr. 1, 2015 (Huck);
• U.S. patent application Ser. No. 14/676,327 for DEVICE MANAGEMENT PROXY FOR SECURE DEVICES filed Apr. 1, 2015 (Yeakley et al.);
• U.S. patent application Ser. No. 14/676,898 for NAVIGATION SYSTEM CONFIGURED TO INTEGRATE MOTION SENSING DEVICE INPUTS filed Apr. 2, 2015 (Showering);
• U.S. patent application Ser. No. 14/679,275 for DIMENSIONING SYSTEM CALIBRATION SYSTEMS AND METHODS filed Apr. 6, 2015 (Laffargue et al.);
• U.S. patent application Ser. No. 29/523,098 for HANDLE FOR A TABLET COMPUTER filed Apr. 7, 2015 (Bidwell et al.);
• U.S. patent application Ser. No. 14/682,615 for SYSTEM AND METHOD FOR POWER MANAGEMENT OF MOBILE DEVICES filed Apr. 9, 2015 (Murawski et al.);
• U.S. patent application Ser. No. 14/686,822 for MULTIPLE PLATFORM SUPPORT SYSTEM AND METHOD filed Apr. 15, 2015 (Qu et al.);
• U.S. patent application Ser. No. 14/687,289 for SYSTEM FOR COMMUNICATION VIA A PERIPHERAL HUB filed Apr. 15, 2015 (Kohtz et al.);
• U.S. patent application Ser. No. 29/524,186 for SCANNER filed Apr. 17, 2015 (Zhou et al.);
• U.S. patent application Ser. No. 14/695,364 for MEDICATION MANAGEMENT SYSTEM filed Apr. 24, 2015 (Sewell et al.);
• U.S. patent application Ser. No. 14/695,923 for SECURE UNATTENDED NETWORK AUTHENTICATION filed Apr. 24, 2015 (Kubler et al.);
• U.S. patent application Ser. No. 29/525,068 for TABLET COMPUTER WITH REMOVABLE SCANNING DEVICE filed Apr. 27, 2015 (Schulte et al.);
• U.S. patent application Ser. No. 14/699,436 for SYMBOL READING SYSTEM HAVING PREDICTIVE DIAGNOSTICS filed Apr. 29, 2015 (Nahill et al.);
• U.S. patent application Ser. No. 14/702,110 for SYSTEM AND METHOD FOR REGULATING BARCODE DATA INJECTION INTO A RUNNING APPLICATION ON A SMART DEVICE filed May 1, 2015 (Todeschini et al.);
• U.S. patent application Ser. No. 14/702,979 for TRACKING BATTERY CONDITIONS filed May 4, 2015 (Young et al.);
• U.S. patent application Ser. No. 14/704,050 for INTERMEDIATE LINEAR POSITIONING filed May 5, 2015 (Charpentier et al.);
• U.S. patent application Ser. No. 14/705,012 for HANDS-FREE HUMAN MACHINE INTERFACE RESPONSIVE TO A DRIVER OF A VEHICLE filed May 6, 2015 (Fitch et al.);
• U.S. patent application Ser. No. 14/705,407 for METHOD AND SYSTEM TO PROTECT SOFTWARE-BASED NETWORK-CONNECTED DEVICES FROM ADVANCED PERSISTENT THREAT filed May 6, 2015 (Hussey et al.);
• U.S. patent application Ser. No. 14/707,037 for SYSTEM AND METHOD FOR DISPLAY OF INFORMATION USING A VEHICLE-MOUNT COMPUTER filed May 8, 2015 (Chamberlin);
• U.S. patent application Ser. No. 14/707,123 for APPLICATION INDEPENDENT DEX/UCS INTERFACE filed May 8, 2015 (Pape);
• U.S. patent application Ser. No. 14/707,492 for METHOD AND APPARATUS FOR READING OPTICAL INDICIA USING A PLURALITY OF DATA SOURCES filed May 8, 2015 (Smith et al.);
• U.S. patent application Ser. No. 14/710,666 for PRE-PAID USAGE SYSTEM FOR ENCODED INFORMATION READING TERMINALS filed May 13, 2015 (Smith);
• U.S. patent application Ser. No. 29/526,918 for CHARGING BASE filed May 14, 2015 (Fitch et al.);
• U.S. patent application Ser. No. 14/715,672 for AUGUMENTED REALITY ENABLED HAZARD DISPLAY filed May 19, 2015 (Venkatesha et al.);
• U.S. patent application Ser. No. 14/715,916 for EVALUATING IMAGE VALUES filed May 19, 2015 (Ackley);
• U.S. patent application Ser. No. 14/722,608 for INTERACTIVE USER INTERFACE FOR CAPTURING A DOCUMENT IN AN IMAGE SIGNAL filed May 27, 2015 (Showering et al.);
• U.S. patent application Ser. No. 29/528,165 for IN-COUNTER BARCODE SCANNER filed May 27, 2015 (Oberpriller et al.);
• U.S. patent application Ser. No. 14/724,134 for ELECTRONIC DEVICE WITH WIRELESS PATH SELECTION CAPABILITY filed May 28, 2015 (Wang et al.);
• U.S. patent application Ser. No. 14/724,849 for METHOD OF PROGRAMMING THE DEFAULT CABLE INTERFACE SOFTWARE IN AN INDICIA READING DEVICE filed May 29, 2015 (Barten);
• U.S. patent application Ser. No. 14/724,908 for IMAGING APPARATUS HAVING IMAGING ASSEMBLY filed May 29, 2015 (Barber et al.);
• U.S. patent application Ser. No. 14/725,352 for APPARATUS AND METHODS FOR MONITORING ONE OR MORE PORTABLE DATA TERMINALS (Caballero et al.);
• U.S. patent application Ser. No. 29/528,590 for ELECTRONIC DEVICE filed May 29, 2015 (Fitch et al.);
• U.S. patent application Ser. No. 29/528,890 for MOBILE COMPUTER HOUSING filed Jun. 2, 2015 (Fitch et al.);
• U.S. patent application Ser. No. 14/728,397 for DEVICE MANAGEMENT USING VIRTUAL INTERFACES CROSS-REFERENCE TO RELATED APPLICATIONS filed Jun. 2, 2015 (Caballero);
• U.S. patent application Ser. No. 14/732,870 for DATA COLLECTION MODULE AND SYSTEM filed Jun. 8, 2015 (Powilleit);
• U.S. patent application Ser. No. 29/529,441 for INDICIA READING DEVICE filed Jun. 8, 2015 (Zhou et al.);
• U.S. patent application Ser. No. 14/735,717 for INDICIA-READING SYSTEMS HAVING AN INTERFACE WITH A USER'S NERVOUS SYSTEM filed Jun. 10, 2015 (Todeschini);
• U.S. patent application Ser. No. 14/738,038 for METHOD OF AND SYSTEM FOR DETECTING OBJECT WEIGHING INTERFERENCES filed Jun. 12, 2015 (Amundsen et al.);
• U.S. patent application Ser. No. 14/740,320 for TACTILE SWITCH FOR A MOBILE ELECTRONIC DEVICE filed Jun. 16, 2015 (Bandringa);
• U.S. patent application Ser. No. 14/740,373 for CALIBRATING A VOLUME DIMENSIONER filed Jun. 16, 2015 (Ackley et al.);
• U.S. patent application Ser. No. 14/742,818 for INDICIA READING SYSTEM EMPLOYING DIGITAL GAIN CONTROL filed Jun. 18, 2015 (Xian et al.);
• U.S. patent application Ser. No. 14/743,257 for WIRELESS MESH POINT PORTABLE DATA TERMINAL filed Jun. 18, 2015 (Wang et al.);
• U.S. patent application Ser. No. 29/530,600 for CYCLONE filed Jun. 18, 2015 (Vargo et al);
• U.S. patent application Ser. No. 14/744,633 for IMAGING APPARATUS COMPRISING IMAGE SENSOR ARRAY HAVING SHARED GLOBAL SHUTTER CIRCUITRY filed Jun. 19, 2015 (Wang);
• U.S. patent application Ser. No. 14/744,836 for CLOUD-BASED SYSTEM FOR READING OF DECODABLE INDICIA filed Jun. 19, 2015 (Todeschini et al.);
• U.S. patent application Ser. No. 14/745,006 for SELECTIVE OUTPUT OF DECODED MESSAGE DATA filed Jun. 19, 2015 (Todeschini et al.);
• U.S. patent application Ser. No. 14/747,197 for OPTICAL PATTERN PROJECTOR filed Jun. 23, 2015 (Thuries et al.);
• U.S. patent application Ser. No. 14/747,490 for DUAL-PROJECTOR THREE-DIMENSIONAL SCANNER filed Jun. 23, 2015 (Jovanovski et al.); and
• U.S. patent application Ser. No. 14/748,446 for CORDLESS INDICIA READER WITH A MULTIFUNCTION COIL FOR WIRELESS CHARGING AND EAS DEACTIVATION, filed Jun. 24, 2015 (Xie et al.).

In the specification and/or figures, typical embodiments of the invention have been disclosed. The present invention is not limited to such exemplary embodiments. The use of the term “and/or” includes any and all combinations of one or more of the associated listed items. The figures are schematic representations and so are not necessarily drawn to scale. Unless otherwise noted, specific terms have been used in a generic and descriptive sense and not for purposes of limitation.

Claims

1. An augmented reality (AR) system, comprising: a display for displaying AR content to a user, the AR content overlapping the AR system's perspective view of an environment;one or more light sensors for gathering light data from light sources in the environment;one or more depth sensors for gathering mapping data of (i) physical objects in the environment and (ii) the light sources in the environment; anda computing device communicatively coupled to the display, the one or more light sensors, and the one or more depth sensors, the computing device comprising a processor that is configured by software to: construct a three-dimensional (3D) model of the environment using the mapping data,detect and characterize the light sources in the environment using the light data,add virtual light sources to the 3D model of the environment, the virtual light sources corresponding to the light sources in the environment,create AR content corresponding to the AR system's perspective view of the environment, wherein the AR content is configured to be proximate a physical object in the AR system's perspective view of the environment,add lighting effects to the AR content, the lighting effects corresponding to the virtual light sources, wherein the lighting effects added to the AR content, by the computing device and without user intervention, match with physical lighting effects of the physical object, andtransmit the AR content with lighting effects to the display,wherein the physical object is a package and the AR content is configured to indicate the package.

2. The AR system according to claim 1, wherein the lighting effects correspond to the environment's ambient light level.

3. The AR system according to claim 1, wherein the lighting effects correspond to (i) the characteristics of the virtual light sources and (ii) the positions and orientations of the virtual light sources with respect to AR content.

4. The AR system according to claim 3, wherein the characteristics of the virtual light sources comprise direction of radiation.

5. The AR system according to claim 3, wherein the characteristics of the virtual light sources comprise light color.

6. The AR system according to claim 3, wherein the characteristics of the virtual light sources comprise light intensity.

7. The AR system according to claim 1, wherein the AR content comprises a graphical 3D object.

8. The AR system according to claim 1, wherein the at least one light sensor comprises a charge-coupled device (CCD).

9. The AR system according to claim 8, wherein the detection of light sources comprises comparing the CCD's pixel values to a threshold level.

10. The AR system according to claim 1, wherein the at least one depth sensor is an optical 3D scanner.

11. The AR system according to claim 1, wherein the display comprises a transparent plate that is (i) positioned in front of the user's eye or eyes, allowing the user to view the environment through the transparent plate and (ii) arranged to display AR content to the user's eye or eyes so that the AR content appears superimposed on the user's view of the environment.

12. The AR system according to claim 1, wherein the display comprises a liquid crystal display (LCD).

13. The AR system according to claim 1, the adding virtual light sources to the 3D model of the environment is facilitated by simultaneous location and mapping (SLAM) techniques.

14. A method for applying lighting effects to virtual objects for an augmented reality (AR) system, wherein a virtual object of the virtual objects is configured to be proximate a physical object in the AR system's perspective view of an environment, the method comprising: receiving lighting information from at least one light sensor;receiving position information from at least one depth sensor;determining lighting effects from the lighting information and the position information; andapplying lighting effects to the virtual objects, comprising matching, without user intervention, lighting effects applied to the virtual object with physical lighting effects of the physical object;updating the lighting effects if (i) the position information changes or (ii) the lighting information changes,wherein the physical object is a package and the virtual object is configured to indicate the package.

15. The method according to claim 14, wherein the AR system comprises a head mounted display (HMD).

16. The method according to claim 14, wherein the at least one light sensor is a charge-coupled device (CCD) camera.

17. The method according to claim 14, wherein the lighting effects comprise coloring, shading, and/or lightening at least a portion of a virtual object.

18. An augmented reality (AR) device, comprising: a light sensor to gather light data for constructing virtual light sources;a depth sensor to gather mapping data for constructing a three-dimensional (3D) model of an environment; anda processor communicatively coupled to the light sensor and the depth sensor, the processor configured by software to: construct the 3D model of the environment including virtual light sources,create AR content, wherein the AR content is configured to be proximate a physical object in the AR system's perspective view of the environment, andadd lighting effects to the AR content, the lighting effects corresponding to the virtual light sources, wherein the lighting effects added to the AR content, without user intervention, match with physical lighting effects of the physical object,wherein the physical object is a package and the AR content is configured to indicate the package.

19. The AR device according to claim 18, wherein the virtual light sources comprise a diffuse light source, a point light source, a directional light source, a distributed light source, and/or an ambient light source.

20. The AR device according to claim 18, wherein adding lighting effects to the AR content comprises changing the AR content's color, intensity, and/or shading so as to appear illuminated by a light source.