Patent Application
20250054241
The present invention relates to augmented reality technology, and more particularly to a method and a system for presenting real time live video streaming content.
The present invention covered by this disclosure is based on augmented reality technology.
Augmented reality (AR) is known as presenting content as a digital layer, where digital content is augmenting the “real” or the non-digital world (that is, reality) by superimposing supplemental information (such as, pictures, videos, three-dimensional (3D) models, and other sensory enhancements) onto the real-world environment. The AR technology overlays virtual objects onto the image of the real world, enhancing a user's perception of reality and providing a user with an immersive, interactive experience and has been used for a long period to fulfil many purposes such as communication within advertisement, education, healthcare, maintenance and construction. As AR technology has advanced it has been further used to present real time live video streaming content also known as live broadcasting. (NPL1)
Real time live video streaming is used as the term for capturing, transmitting, receiving, and presenting what has been captured. The same event is also covered by the term: live broadcasting. Although both being the same in aspect of presenting video content directly, the term real time live video streaming is used in this disclosure as it distances itself from broadcasting because its functionality is often extended by common features such as pausing, allowing the viewer to go back on the timeline to watch something previously presented, or even interacting with the real time live video stream such as commonly seen on social media where the viewer for example can like, comment, or even save a picture of the content whilst the content provider is live streaming the video in real time. Augmented reality (AR) real time live video streaming refers to real time live video streaming executed as augmented reality. In this disclosure, for the sake of simplicity, real time live video streaming by an AR device is the event of AR real time live video streaming.
Live streaming in its general term is considered streaming data, whereas real time live video streaming is referred to in this disclosure as the event where the representation of captured video content is presented immediately by streaming video content data to a device operable for receiving and presenting this live content in real time. Likewise live broadcasting, there will always be a small delay from what is playing out in front of the video capture device and what is presented to the end viewer comparable to how there will always be a delay within human biology between what is playing out and what is perceived or comprehended. The reason being the time it takes before light photons have been received through the eyes, a signal send to the brain by nerves, and processed by the brain representing an illusion of the world based on photon data. Real time live video streaming means that the process of capturing, transmitting, receiving, and presenting video content happens consequently in real time. Yet, with a natural and less significant delay.
An example of AR real time live video streaming is Microsoft HoloLens with Skype. The hardware HoloLens from Microsoft is an AR headset also known as a head mounted AR device whilst Skype is the software program. Together as an AR device, it is operable for capturing video content, transmitting it while simultaneously receiving video content data and presenting it as augmented reality in real time. (NPL2)
Video live streaming is a well-known communication tool to assure users about real life conditions and has been widely used by for example TV stations to display events such as news, sporting events, or peculiar weathers online as well as in marketing when assuring potential customers.
An example of assuring potential customers is by providing the option of observing part of a production associated to a product in real time which has been done to the alcoholic liquor “Linie Aquavit” by LINIE who utilized live streaming to prove to some doubting people that the liquor “Linie Aquavit” is being sailed across the equator twice to mature. The real time live video stream covered its entire sea journey around the world around the clock 24/7 combined with location data of the ship which could be experienced through the webpage LINIE.com/live/to ensure legitimacy. (NPL3)
All in all, the medium augmented reality and real time live video streaming as a tool of communication has been and is used to fulfil many purposes. Yet, when not executed rightly as for example within marketing it can result in poor confidentiality. Branding can be beneficial to the consumer but also superficial and manipulating as pictures and videos can easily represent different conditions than the originals. Consumer reviews can be affected by other factors and therefore providing a misguided review of the product as much as certificates and quality stamps can be wrong due to other reasons whereas corruption is a known one within certain industries and countries. A brand's credibility can be achieved through honesty which translates into transparency.
Except for the example of LINIE, none of the abovementioned examples of marketing create transparency to an extend where the user can be assured of what is presented to them independent of a brand's goodwill or other parameters.
Augmented reality is a media like any other media carrying a message from a sender to a receiver. Nonetheless, AR stands out from other media by augmenting the non-digital world with digital content.
While it is well known presenting real time live video streaming as well as it is well known to augment a visual object with digital graphical content known as augmented reality, it is not taught in the background art to combine the use of detectable information, namely AR tags, with real time live video streaming to be presented within a defined AR border comprised by an AR tag.
Therefore, while each of the different apparatuses, systems, and methods disclosed in the above references as background art are suitable for the uses and problems they intend to solve, there is an ongoing need for improvements in augmented reality real time live video streaming. A system and a method that can provide more control to the content provider and thereby limit the user experience to prevent infringing permits, overloading users with information, and in some cases creating transparency.
Accordingly, a primary object of the present invention is to provide an augmented reality (AR) system and method for presenting real time live video streaming content within at least one defined AR border activated by at least one AR tag, the system comprising;
The system, wherein the function of presenting real time live video streaming content upon detecting at least one AR tag is regulated consequent to detecting at least one or a combination of several AR tags.
Alongside the system, an object of the present invention is to provide a method for presenting real time live video streaming content within at least one defined AR border activated by at least one AR tag the method comprising;
Experiments with augmented reality real time live video streaming has started recently as AR technology evolves. Still, the content providers do not have the complete ability to control when, where, and how AR real time live video streaming content is played out which can be a crucial element for their business and their communication strategy. For example, when it is important to present real time live video streaming content within a specific time, place, and or connected to certain elements for the sake of creating strong associations between real time digital content and a certain product.
Examples of these problems would be when real time live video streaming content is used to legitimate the overall or part of a production associated with a related product. For example, in order to assure a potential customer parts of the original condition of a production while creating strong associations between the real time live video streaming content and the product itself. For example, when presenting a real time live video streaming of milk producing cows on associated milk packaging. However, current solutions of displaying real time live video streaming content as a standard and by augmented reality do not offer a way to control the experience of real time live video streaming content to be limited to be presented exclusively associated to a certain product as well as time, or place.
Another example of a technical problem is in a sports bar where there is a limited amount of space for screens and projectors displaying sporting events. The sports bar sells beverages, snacks, and food in exchange for allowing guests to watch sporting events but in return the sports bar must secure permits to show the events. Which technical solution can offer a bar guest to see specific real time live video streaming content of a specific sporting event without occupying space while securing not infringing sporting event copyrights?
The solution is real time live video streaming by use of an augmented reality technology allowing the content provider to control when, where, and how to present AR content by using augmented reality tags along with augmented reality borders (AR borders). An AR tag is combined with or directly connected to an AR border where the AR tag defines where and when content can be played out, whereas the AR border defines how by comprising a concrete frame as a limitation for content to be presented.
For example, a potential consumer in a convenience store can see a real time live video streaming of cows on the milk packaging but only on the actual milk packaging within a frame due to the limitations of the AR border framing a third of the packaging in the middle right below an upper text indicating the name of the product but above a lower label indicating the associated brand. The AR border defines exactly how the AR content is presented on the packaging by defining the space on the packaging where content can play out and the format. The real time live video streaming reflects good conditions of the cows which is thereby associated to the actual milk product and brand convincing the consumer to choose that product over similar products as these production conditions appeal to the user's morals and ethical convincement. The AR tag consisting of a graphical layout of the milk packaging comprises identification to the AR device and secures the necessity of the user to be in possessing of the actual milk packaging in order to experience the augmented reality live video streaming content. This benefits the seller as the potential customer must buy the milk product in order to experience the AR content at home.
Another example is in a sports bar where all screens are presenting sporting events. Some guests request to see a specific sporting event which is not displayed due to lack of space and screens. Through an AR device such as a smartphone or an AR headset the guests are able to enjoy the specific sporting event without the sports bar having to compromise one of the occupied screens. However, the AR tag and AR border secure that the real time live video streaming is only available and playable inside the sports bar as the AR tag and border are defined by actual location data of the bar. The AR tag is defined by coordinates of the Global Positioning System (GPS) location. The AR device detects the AR tag based on GPS data which makes real time live video streaming of the specific sporting event available to the user within that specific location in the sports bar. The AR border is comprised by the AR tag and is defining its frame by the same GPS data and the real time live video streaming is presented anywhere within the AR tag. Thereby, the real time live video streaming can be presented only within that frame specific to the local location of the sports bar. Once the guest moves outside the specific GPS-zone of the bar, the real time live video stream is no longer playable.
The AR tag in the bar could also be more specific to a graphical layout in the bar such as a poster of a beer product and to a specific time of the day. The AR tag thereby limits the experience of the sporting event to be presented associated to the poster during a specific time such as the happy hour of the bar.
The combination of AR tags and AR borders provides a technical solution to content providers to control where, when, and how augmented reality real time live video streaming is presented saving physical space and occupying screens.
Primary, the advantages effects of the invention are the possibility to control where, when, and how AR real time live video streaming content is played out. Where and when is controlled by the AR tag whilst how is depending on the defined AR border which is comprised by at least one AR tag.
By controlling where, when, and how presenting real time live video streaming content becomes an advantage when exclusivity is of a significant value. With AR tag, where and when provide the possibility to limit content to only be presented when certain requirements are fulfilled as the AR tag can be controlled. For example, if one content provider wants to limit the AR experience to a certain place and or a certain time. With a defined AR border the experience of how the real time live video streaming is presented is controlled which is an advantage when for example it brings a significant value to associate the real time live video streaming content with a product or service. For example, when presenting real time live video streaming of cows on a related milk product. The association of healthy organic free-range cows and a bottle of organic free-range milk is convincing by assuring the potential customer about good production conditions.
Moreover, an advantage of augmented reality real time live video streaming is the possibility to interact without depending on having to physically touch a button or screen by which the real time live video streaming is presented or by voice controlling the presentation device. For example, when the user wants to pause the real time live video stream on a milk packaging. AR is operable for interactivity by for instance the function of recognizing an object like a finger. By this means, activating the pause function by either physically pushing an augmented reality pause-button on the milk packaging with the recognized finger, holding the finger over the pause-button for a certain amount of time, or in another way indicate the wish of activating the pausing function saves the user for touching or giving a voice command to the AR device.
Another example when it is valuable not to be depending on touching a physical screen is when a user is wearing an AR headset while kneading a dough in a kitchen with sticky fingers and runs out of milk. Next to experiencing a real time live video streaming of cows on the milk packaging, AR is operable for common functions displayed as visual graphics as digital layers augmenting the milk packaging like the real time live video streaming. An example is a digital button or ikon for ordering more milk. It is an advantage to be able to interact with the real time live video streaming along with common features like reordering milk without having to touch a screen as the user thereby avoids staining the screen with the fingers.
Moreover, a further advantage of having functions within the real time live video streaming as augmented reality as well as functional ikons associated to the content is the forestall of having to create digital content in various formats for different displays.
It is to be understood that both the foregoing general description and the following detailed description are exemplary, and are intended to provide further explanation of the invention as claimed. Other advantages and features of the invention will be apparent from the following description, drawings, and claims.
The disclosure of invention set forth below, in connection with the accompanying drawings, is intended as a description of various embodiments and is not intended to represent the only embodiments in which the disclosure may be carried out or practiced. The disclosure includes specific details to provide a thorough understanding of the embodiments. Without regard to, it will be apparent to those skilled in the art that the disclosure without these specific details, in some instances, well-known structures and components are shown in simplified form for brevity of description.
The technical contents and detailed description of the present invention are described thereinafter according to a preferable embodiment, being not used to limit its executing scope. Any equivalent variation and modification made according to appended claims is all covered by the claims claimed by the present invention.
The functions and features of the invention believed to be novel are set forth with particularity in the appended claims. The invention itself, however, may be best understood by reference to the following detailed description of the invention, which describes exemplary embodiments of the invention, taken in conjunction with the accompanying drawings, in which:
In cooperation with attached drawings, the technical contents and detailed description of the present invention: A system and a method for live streaming by use of an augmented reality (AR) technology, are described thereinafter according to a preferable embodiment, being not used to limit its executing scope. Any equivalent variation and modification made according to appended claims is all covered by the claims claimed by the present invention.
Reference will now be made to the drawing figures to describe the present invention in detail.
Reference is made to
The present invention has particular utility in connection with a real time live video streaming on a product such as one of organic free-range cows on a milk packaging. Nevertheless, it is to be understood that the present invention can similarly be used in any number of augmented reality experiences such as watching a live sporting event or the like. For purposes of simplicity, reference will be made to the real time live video streaming of a bunch of cows. Specifically, a bunch of organic free-range cows representing the cows producing the milk of a milk product, alternatively an example of the invention used on coconuts in a grocery store and another example of the invention used with a sporting event in a sports bar, will follow throughout the ensuing description and following examples.
The augmented reality real time live video streaming system (100) enables detecting at least one augmented reality tag (300) with an AR border (301).
The system (100), generally comprises: (a) at least one AR tag (300) with at least one defined AR border (301); (b) an AR user device (401); (c) a Capture Device System (501); (d) a Data Management System (600) the Data Management System (600) being connected (602) to the AR user device (401) and the capture device (501) by a Network System (601).
Although controlling two different functions the function of AR tag (300) and the function of AR border (301) may be based on the same detectable information or may be based on different information.
The AR tag (300) also known as a marker may be any identifiable information such as a visual graphic, sound, light, temperature and other detectable physical properties as well as Global Positioning System (GPS) data and the like. A visual graphic may be a visual graphic such as an image, Quick Response (QR) code, the certain shape of an object, or a colour code. Sounds may be a certain sound frequency or a certain combination of frequencies being a sound code. Light may be a certain colour or frequency or a certain combination of frequencies being a light code. Global Positioning System data is the actual position or the position including a certain radius. The function of the AR tag (300) is to be detected by the AR user device (401) and by being so providing access for the AR user device (401) to present AR real time live video streaming content. The function of presenting real time live video streaming content upon detecting at least one AR tag (300) is regulated consequent to detecting at least one or a combination of numerous AR tags (300). Detecting a combination may be executed by detecting numerous AR tags consequent to each other, simultaneously, randomly, or the like. For example, detecting one AR tag being a visual graphic such as an image may fulfill the requirement to allow real time live video streaming content to be presented. Another example is the requirement of detecting an AR tag being a visual graphic as well as an AR tag in the form of a sound code simultaneously in order to fulfill the requirement to present real time live video streaming content.
The AR border (301) is a function technically defining how AR content can be presented. The AR border (301) is defining a frame in the non-digital environment also called the “real world” where AR real time live video streaming content can be presented. An AR border (301) definition of a frame may be the size and the position of the frame relative to the AR tag (300) or it may be within a limited area which can be limited by the same identifiable information that defines the AR tag (300). By way of example, the AR tag (300) may include a requirement of for example a certain combination of AR tags (300) in order to initiate the process of presenting AR real time live video streaming content within a defined frame of the AR border (301). The AR border (301) consists of software codes and its function can be achieved by many means of coding which is well-known to those skilled in the art. The function of detecting an AR tag (300) as well as the requirements of fulfilling a combination of AR tags (300) to present content also consist of software codes and the functions can be achieved by many means of coding which is well-known to those skilled in the art.
The AR user device (401) referred to as the AR device (401) may be a smart phone, tablet computer, AR headset, digital glasses, retinal projection glasses with holographic display or the like and possible for the future being digital contact lenses and neurone-devices reacting and interacting on neurons activity in the brain. The AR device (401) includes a capture device (403) such as a camera configured to capture a video signal or the like which can include a microphone (405,
In general, an ARSM (404,
The function of the ARSM (404) may also happen in a cloud well-known as the phenomenon cloud computing being one or several connected computer modules operable for receiving, processing, and transmitting data. Thereby, the AR device (401) may just be a device operable for capturing information, transmitting, receiving, and presenting data by the use of a capture device (403), a display unit (402), and a Network System (601). The full processing or part of the processing of data by the ARSM (404) may instead be processed externally online in a cloud or in another device connected to the AR device (401). For example, the AR device (401) may be a pair of AR glasses that only captures and presents AR content while sending all captured data by for example Bluetooth (1100) to a smartphone (900,
The AR device (401,
The Capture Device System (501) is operable for capturing content and transmitting it to the AR user device (401) through a Data Management System (600) or it may directly transmit data (600d,
The Data Management System (600,
The Network System (601) are well-known to those skilled in the art and include broadband, fiber-optic, cellular data, Wi-Fi, and other wireless access technologies, and as such shall not be discussed in detail hereinbelow.
Reference is made to
A user (400) walks into a convenience store to buy milk. Several milk products are available side by side on the shelf. The milk product (200) is contained within a packaging (201). The milk packaging (201) comprises an AR tag (300) being a visual graphic of a label of the milk producer. Furthermore, the AR tag (300) comprises an AR border (301). The AR border (301) is defining a frame in the format of 16:9 with the measurements of 6 cm×3.38 cm positioned right above the AR tag (300). The user (400) recognizes an ikon (203) indicating that this milk packaging (201) comprises an AR feature. The user (400) takes an AR user device (401) (referred to as an AR device) being a smartphone and opens an app, webpage or the like wherein the AR function is installed or available (401a) and the screen or display (402) of the smartphone (401) presents what the capture device (403) being a smartphone camera captures (402a). The user (400) points the camera (403) towards the milk packaging (201), (403a). The AR user device comprising an Augmented Reality Software Module (ARSM) (404) analyses captured data (404a) and detects the AR tag (300), processes the identified information (404b), and sends a request for real time live video content (404c) through a Network System (601) to a Data Management System (600), (600a). The Data Management System (600) receives the request (600a) and directs data (602) from the Capture Device System (501) to the AR user (401), (600c). The Capture Device System (501) captures and processes data simultaneously (501a) as data is transmitted (501b). The AR device (401) receives the data (402b) and processes it correlated to pre-sets of the AR tag (300) and the AR border (301), (402c) and presents the real time live video streaming content (406),(402d). However, instead of The Capture Device System (501) directly sends data stream (602) to the Data Management System (600), (600b) the Capture Device System (501) may instead send the data stream (602) directly to the AR device (401), (600d). The user (400) sees the real time live video streaming content on the AR device (401) appearing as a visual overlay atop the milk packaging (201). The consumer (400) likes the experience of seeing the healthy cows and makes a purchase decision and buys the associated milk product (200).
Appearing above and in relation to the real time live video streaming content atop of the packaging (201) is an ikon (204) or the like operable for carrying out common functions such as ordering the milk. Instead of bringing the milk product (200) home the user (400) pushes the ikon or in any other way activates the ikon (204). Upon activation of the ikon (204) a function is executed. The function may be adding the milk product (200) to an online shopping list for home delivery or the like.
Furthermore, the user (400) decides to go and see the represented cows (500,
Reference is made to
A user (400) walks around in a grocery store wearing an AR user device (401) being a pair of AR glasses. The user walks into the fruits and nuts area whereas a bunch of coconuts are displayed. The coconuts (200) do not have packaging. Under the coconuts is a speaker (800) playing a sound code being an AR tag (3001) with frequencies between 21-23 kHz. The user (400) is not capable of hearing those high frequencies. Yet, the AR user device (401) is capable of capturing the sound code (3001) through its capture device by a microphone (405) and detects the sound code (3001). The AR user device (401) further detects an AR tag being the shape of a coconut (3002) and an AR tag being the colour code of a coconut (3003) through a camera (403). The combination of the three AR tags being the sound code (3001), the shape of a coconut (3002), and the colour code of a coconut (3003) are the three AR tags which regulate if real time live video streaming content can be presented as this depends on the AR device (401) detecting all the AR tags consequent to each other. The AR tags being detected consequent to each other meets the requirement to allow access to receive real time live video streaming content.
The AR tag (3002) based on the shape of the coconut comprises an AR border (301). The AR border (301) is defining a frame with an oval shape positioned in the middle of the coconut (200) filling up 35% of the coconut shell (200). Upon detection of the AR tags (3001), (3002), and (3003) the AR device (401) transmits by Bluetooth information to a smartphone (900) operable for processing data by an ARSM (404,
The user (400) experiences real time live video streaming content of the coconut palm plantage and may swipe on the coconut, the content (406), to see real time live video streaming content from other capture devices (502,
Reference is made to
Two users (400) located in a sports bar wish to see a certain sporting event which is real time live video streamed on a certain channel between 4 μm and 6 pm. Due to lack of screens (1000) the sports bar offers access to the sporting event by augmented reality. The users (400) receive AR user devices (401) being AR head-mounted displays. The AR tags (300) are based on the GPS data of the location of the sports bar, the time 4 μm and 6 μm, and the visual graphic (300) of a beer advertising poster hanging on a wall behind a bar. The AR border (301) is also based on the visual graphic of the beer advertising poster (300) defining a frame (301) in the format of 16:9 with the measurements of 150 cm×84 cm positioned right next to the right side of the beer advertising poster. At 4 pm the users (400) experience watching the real time live video streaming content (406) of the sporting event presented atop of the wall next to the poster (300) through the AR devices (401) until 6 pm. It is not possible for a user to bring the visual graphic of a beer advertising poster (300) home to experience the sporting event (406) at home between 4 pm and 6 pm as one of the three AR tags being the GPS data related to the location of the sports bar will be missing once the user is outside the sports bar. Thereby, the experience of the real time live video streaming content is limited securing that the sports bar is not infringing permits as well as keeping the two bar quests in the bar to improve their bar experience and thereby provide increased revenue opportunities for the sports bar.
The above description of the disclosed embodiments including the examples is provided to enable any person skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles described herein can be applied to other embodiments without departing from the spirit or scope of the invention. Thus, it is to be understood that the description and drawings presented herein represent a presently preferred embodiment of the invention and are therefore representative of the subject matter which is broadly contemplated by the present invention. It is further understood that the scope of the present invention fully encompasses other embodiments that may become obvious to those skilled in the art and that the scope of the present invention is accordingly not limited.
The invention covered in this disclosure is applicable in any industry where real time live video streaming content playing out through the augmented reality media is relevant. As such, the applicability has no certain limits. Yet, the primary intention of this invention is to provide a system and a method for content providers of real time live video streaming content to control where, when, and how AR live streaming content is played out as this can be a crucial element for their business and their communication strategy. The retail and entertainment industry are the two industries to which the objectives of this invention regard. However, the industrial applicability of this invention is not limited to only these two industries as the function of the invention may be applicable in many more industries.
| Number | Date | Country | Kind |
|---|---|---|---|
| PA 2019 00228 | Feb 2019 | DK | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/DK2020/050046 | 2/20/2020 | WO |
