Patent Application
20150002540
The present application is based on, and claims priority from, Indian Application Number 2566/CHE/2012, filed on 28 Jun. 2012, the disclosure of which is hereby incorporated by reference herein.
The present embodiment relates to Augmented Reality (AR) systems and more particularly to Augmented Reality (AR) systems that enhance user experience by making systems more interactive.
Augmented Reality (AR) blurs the line between what's real and what's computer-generated by enhancing what users see, hear, feel and smell. There exist different techniques in which AR systems can be built like image overlay, superimposing real and virtual images, marker tracking, real object tracking etc.
Marker tracking techniques use a marker, which is an optically recognizable image or symbol and can be tracked using a live image capturing tool (such as a camera). The marker is visually recognized along with the position. Every marker is associated with a system generated image which is rendered on the marker position as seen through camera view. The system generated image can be moved by physically moving the marker, as long as the marker stays within the camera view. Existing technology uses marker tracking and recognition tools like AR Tool kit in creating marker based AR system. These toolkits can further render an associated system generated image (hence forth called system object) over the marker. But when multiple markers are used and are to be viewed at the same time, these toolkits can only render system objects against each marker, but are not able to make any correlation between the different system objects. Thus, user experience in applications like interactive learning, do-it-yourself instruction guides, games like jigsaw puzzle etc are not up to the desired level.
The object of the embodiments herein is to detect presence of multiple markers in AR systems at a given point in time and locate their position.
Another object of the embodiment is to determine relative positions of detected markers, calculate distance between detected markers and correlating them so as to dynamically render different objects for same marker on the screen.
Disclosed herein is a method of rendering virtual objects in an augmented reality environment, the method comprising identifying a plurality of markers in the augmented reality environment; calculating distance and relative positions of each of the plurality of markers in the augmented reality environment; and rendering dynamically different virtual objects corresponding to each of the plurality of markers based on the calculated distance and relative positions of each of the plurality of markers.
Also, disclosed herein is a device for rendering virtual objects in an augmented reality environment, wherein the device comprising an integrated circuit further comprising at least one processor; at least one memory having a computer program code within the circuit; the at least one memory and the computer program code configured to with the at least one processor cause the device to identify a plurality of markers in the augmented reality environment; calculate distance and relative positions of each of the plurality of markers in the augmented reality environment; and render dynamically different virtual objects corresponding to each of the plurality of markers based on the calculated distance and relative positions of each of the plurality of markers.
These and other aspects of the embodiments herein will be better appreciated and understood when considered in conjunction with the following description and the accompanying drawings. It should be understood, however, that the following descriptions, while indicating preferred embodiments and numerous specific details thereof, are given by way of illustration and not of limitation. Many changes and modifications may be made within the scope of the embodiments herein without departing from the spirit thereof, and the embodiments herein include all such modifications.
This embodiment is illustrated in the accompanying drawings, throughout which like reference letters indicate corresponding parts in the various figures. The embodiments herein will be better understood from the following description with reference to the drawings, in which:
a depicts a user device having Augmented Reality Multi Marker Positioning System (ARMMPS), which can detect multiple markers and correlate them, according to the embodiments disclosed herein;
b depicts display screen of user device that can render system objects according to processed data of position matrices, according to the embodiments disclosed herein;
a,
The embodiments herein and the various features and advantageous details thereof are explained more fully with reference to the non-limiting embodiments that are illustrated in the accompanying drawings and detailed in the following description. Descriptions of well-known components and processing techniques are omitted so as to not unnecessarily obscure the embodiments herein. The examples used herein are intended merely to facilitate an understanding of ways in which the embodiments herein can be practiced and to further enable those of skill in the art to practice the embodiments herein. Accordingly, the examples should not be construed as limiting the scope of the embodiments herein.
Referring now to the drawings, and more particularly to
a depicts a user device having Augmented Reality Multi Marker Positioning System (ARMMPS), which can detect multiple markers and correlate them (finds relative position of markers), according to the embodiments disclosed herein. Block diagram shows the user device 104 having the ARMMPS, which can detect and locate positions of real world multiple markers such as M1101, M2102, M3103 and so on. The ARMMPS further creates position matrices for each marker positions; processes data of position matrices, then renders virtual system objects for detected multiple markers, as shown in
The system comprises of marker detecting device (marker sensing device) such as optical sensors, live camera, still camera, infra red ray sensors, heat sensors, touch screens, proximity detectors. Markers can be two dimensional objects (2D) such as symbols, images, etc or three dimensional (3D) objects such as models, toys, fingers, faces, heat emitting, light emitting sources such as LED's etc. In one embodiment, the user can define his own marker.
b depicts the display screen 105 of the user device 104 which is implementing ARMMPS, according to the embodiments disclosed herein. Figure shows system objects1, system object2, system object3 rendered by the ARMMPS for markers M1101, M2102 and M3103 respectively. System objects rendered can be in 2D or 3D, which can be accessed from sources like system memory, pen drives, DVD's or any storage devices. In one embodiment user can choose or customize the object to be rendered for a marker. In one embodiment ARMMPS can render a predefined system object or a series of objects when it detects multiple markers satisfying a predefined condition.
The ARMMPS starts (301) execution by initializing and there after captures (302) image in which markers may be present. The ARMMPS module further analyzes (303) this image to detect markers present. The ARMMPS module compares (304) the latest image with the previous image and if a change has happened, then it creates (305) position matrices for each marker in the latest image and calculates (306) distance and relative positions of each marker and correlates markers. ARMMPS module uses the standard AR marker recognition tools (like AR toolkit, QCAR etc) to detect marker and get position matrix for each marker. This position matrix provides target area on the screen where the system object associated with the marker can displayed.
Details of processing module are discussed later in
The various actions in method 300 may be performed in the order presented, in a different order or simultaneously. Further, in some embodiments, some actions listed in
The various actions in method 400 may be performed in the order presented, in a different order or simultaneously. Further, in some embodiments, some actions listed in
The ARMMPS also correlates the position of each marker and if predefined conditions like solar eclipse, lunar eclipse are detected then, the ARMMPS will dynamically render different object to same marker. In this case a solar eclipse has happened when a moon has come in middle of sun and earth then the ARMMPS rendered a different object which is a sun with shade for marker M1 dynamically.
a,
a shows Illustrative example that considers position matrices which define target area on screen (where system object associated with marker is displayed). Target areas for markers here are the quadrilaterals as shown in
M1(M1x, M1y)=(S0.x+S1.x+S2.x+S3.x)/4, (S0.y+S1.y+S2.y+S3.y)/4
Similarly midpoint of target area of markers M2 (102) and M3 (103) can be calculated, and can be given by M2 (M2x, M2y), M3 (M3x, M3y) respectively.
b shows technique to check if all markers fall in a line. Technique used, initially considers M1 (M1x, M1y) and M2 (M2x, M2y) representing midpoints of target areas of M1 (101) and M2 (102) as seen in
y=m*x+c,
Where m=(M1y−M2y)/(M1x−M2x) and then rearranges above line equation as A*x+B*y+C=0
Calculates perpendicular distance (D) from point M3 (M3x, M3y) to line AB represented by equation A*x+B*y+C=0 which passes through M1 (M1x, M1y) and M2 (M2x, M2y). D gives perpendicular distance of M3 (M3x, M3y) from line AB passing through M1 (M1x, M1y) and M2 (M2x, M2y). With this available data, technique used here can further check if M3 (M3x, M3y) falls on the same line as M1 (M1x, M1y) and M1 (M1x, M1y) as described below.
Two consecutive corners of square target area of marker M1 (101) are S1 and S2
Length of one side of square S0, S1, S2, S3 (length)=√{square root over ((S1.x−S2.x)2+(S1.y−S2.y)2)}{square root over ((S1.x−S2.x)2+(S1.y−S2.y)2)}
Lengths of all sides of all target areas for all markers are equal as target area in illustrative example is a square.
Half of the length of a side of M3's (103) target area (d′)=length/2;
If D<d′ (less than) then we can say that M3's (103) target area position falls on the same line of M1 (M1x, M1y) and M2 (M2x, M2y).
c shows technique to check if one marker falls between the other two, according to the embodiments disclosed herein.
Here case where third midpoint M3 (M3x, M3y) can have various locations is illustrated. Two different positions M3 (M3x, M3y) or M3′ (M3′x, M3′y) that can be taken are considered. Technique calculates various distances between marker midpoints as mentioned below
Now checks if following condition is satisfied.
DM1M32<=DM2M32+DM1M22
Or
DM2M3′2<=DM1M3′2+DM1M22
Using this condition it can be decided if M3 (M3x, M3y) or M3′ (M3′x, M3′y) falls in the position in-between M1 (M1x, M1y) and M2 (M2x, M2y), and neither lies on the right side of the CD line nor the left side of the C′D′ line.
The embodiments disclosed herein can be implemented through at least one software program running on at least one hardware device and performing network management functions to control the elements. The elements shown in
The foregoing description of the specific embodiments will so fully reveal the general nature of the embodiments herein that others can, by applying current knowledge, readily modify and/or adapt for various applications such specific embodiments without departing from the generic concept, and, therefore, such adaptations and modifications should and are intended to be comprehended within the meaning and range of equivalents of the disclosed embodiments. It is to be understood that the phraseology or terminology employed herein is for the purpose of description and not of limitation. Therefore, while the embodiments herein have been described in terms of preferred embodiments, those skilled in the art will recognize that the embodiments herein can be practiced with modification within the spirit and scope of the embodiments as described herein.
