Augmented reality is a technology in which a person's conception of reality can be enhanced, typically through augmented sound, video or graphics displays. The augmentation is typically implemented via various technologies, such as a headset that may be worn by the person. One or more augmented views may be presented to the person through the headset.
The augmented reality headset typically includes a wearable computer and an optical display mounted to the headset. The wearable computer may include a wireless telecommunication capability, permitting a wireless connection from the wearable computer to a server computer. Because of the wireless telecommunication capability, the augmented reality headset may be used to interact with the server computer to accomplish various tasks.
Embodiments of the disclosure are directed to a method implemented on an augmented reality (AR) device, the method comprising: receiving an image of an object at the AR device; displaying the image on a display screen of the AR device; receiving one or more activating actions at the AR device; and as a result of the one or more activating actions: identifying the object; sending an identity of the object to a server computer; receiving information from the server computer regarding a percentage of completion towards a goal relating to the object; and visually displaying on the image of the object a representation of the percentage of completion towards the goal relating to the object.
In another aspect, an augmented reality (AR) electronic device comprises a processor; a camera; memory; and a display area, wherein, the AR electronic device is configured to display an image representing progress towards a goal, the image comprising an object to be purchased and an overlay covering at least part of the image of the object to be purchased, the overlay being of one or more colors or patterns, an extent to which the overlay covers the at least part of the image of the object to be purchased representing a percentage of a completion of a goal relating to the object to be purchased.
In yet another aspect, an augmented reality (AR) device comprises: a processing unit; and system memory, the system memory including instructions which, when executed by the processing unit, cause the AR device to: receive an image of an object; display the image on a display screen of the AR device; receive one or more activating actions at the AR device, each of the one or more activating actions being a verbal command, the one or more activating actions initiating a display on the AR device of progress towards a savings goal; and as a result of the one or more activating actions: obtain an identification for the object, the identification being obtained at the AR device via optical recognition software; identify a user of the AR device, an identification of the user of the AR device being performed via one or more of facial recognition, retinal scan, voice print or finger print scan; send an identity of the user and an identity of the savings goal to a server computer; receive information from the server computer regarding a percentage of completion towards the savings goal; and visually display a pattern or a color on a portion of the object, the pattern or the color covering up the portion of the object, the portion of the object being covered up with the pattern or color corresponding to the percentage of completion towards the savings goal.
The details of one or more techniques are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of these techniques will be apparent from the description, drawings, and claims.
The present disclosure is directed to systems and methods for using augmented reality (AR) to display progress towards a goal. The goal may be to purchase an item, such as a car, a house or a boat, to save for a specific event, for example a vacation or college, or a non-financial goal such as progress on an exercise program. Other types of goals are possible.
The AR device displays progress towards the goal by displaying an image of an object associated with the goal and coloring or shading the object to indicate progress towards the goal. For example, if the goal is to purchase a car, the AR device may display an image of a car. The image may be colored or shaded to indicate progress towards the goal of buying the car. For example, if 50 percent of an amount of money needed to purchase the car has been saved, one half of the image of the car may be colored or shaded and the color half of the image of the car is not colored or shaded. The 50 percent of the image that is colored or shaded indicates that there has been 50% progress towards the goal of purchasing the car.
The systems and methods are generally described for an AR device that may be worn or carried by a user. An example of an AR device that may be worn by the user is an AR headset. An example of an AR device that may be carried by the user is a smart telephone or tablet computer that includes AR components such as a processor, display and camera and an AR software application.
The AR headset includes a wearable computer, a camera and an optical display. The wearable computer includes a wireless telecommunication capability, permitting a wireless connection between the wearable computer and one or more server computers. The wearable computer also includes voice recognition capability, permitting the user to direct the wearable computer via voice commands. In addition, in some implementations, the wearable computer also includes biometric capability such as facial recognition, retinal scan capability, finger print and voice print capability.
One type of AR headset described in this disclosure is a smart glass type of headset, similar to eyeglasses, which may be worn by the user. The user may view AR images in the glass portion of the headset. An example of a smart glass headset is Google Glass, from Google Inc. of Mountain View, Calif.
In order for the AR device to display progress towards the goal, the user of the AR device needs to be identified. Once identified, account information for the user may be obtained. The user of the AR device is typically identified via one of several biometric methods including, facial recognition, retinal scan capability, finger print or voice print. Other methods for identifying the user may be used.
Using the systems and methods, information related to the goal is typically entered into a digital dashboard. The digital dashboard is a user interface of a computing device, for example a financial services software application accessible from a smart telephone or tablet computer. As explained in more detail later herein, a user may select a goal, select an amount to save for the goal and select an object from which to display progress towards the goal. For example, if the goal is to purchase a house, a picture of a house may be selected. If the goal is to purchase a car, a picture of a car may be selected. If the goal is to save for a vacation, a picture of a vacation scene may be selected. The picture selected is typically a generic representation for the object. For example, the picture of the car, house or vacation scene may be a picture of any car, house or vacation scene and need not be the actual car, house or vacation corresponding to the goal.
Using the digital dashboard, the user may also select an activating object. The activating object is an object, which when viewed by the AR device causes a display showing progress towards the goal to be displayed on the AR device. More than one activating object may be used for a goal. For example, if the goal is to purchase a car, the activating object may be a car, a key, or both the car and the key. Similarly, if the goal is to purchase a house, the activating object may be a house. However, each activating object may only be used for one goal. For example the key may be the activating object for the car or the house, but not for both. In a typical implementation, the activating object is a three-dimensional object such as an actual car, house or key as viewed via a camera on the AR device. In other implementations, the activating object may be a static two-dimensional object, such as a picture or photo of a car or house.
In an example implementation, the user may configure the digital dashboard for a goal of purchasing a car. In this example, a car and a key are both configured to be activating objects. If the user is walking down a street and sees a car, the viewing of the car may remind the user that he/she is saving to purchase a car and the user may be curious as to a current status of this goal. If the user is wearing or carrying an AR device, the user may view the car with the AR device and also issue an activating verbal command, such as “goal getter.” These actions may cause the AR device to display a colored or shaded overlay on the three-dimensional image of the viewed car showing progress towards the goal. In some implementations, a second activating command may be used—for example “car.” The second activating command may be used, for example, if optical recognition software on the AR device does not recognize the activating object. In other implementations other activating actions besides verbal commands may be used. For example, the user may initiate the display of progress towards the goal by touching one or more areas on the display screen of the AR device.
In another variation of this example implementation, the user may be sitting in his/her office and think about the car to be purchased. Rather than having to find an actual car to trigger the display, the user may view a key with the AR device and issue the same activating verbal commands, such as “goal getter” or “goal getter” and “car.” These actions also cause the AR device to display a representation of a percentage completion towards a goal. However, this time the representation of the percentage completion is shown on the viewed key. For example, a color or shaded overlay appears on the image of the key showing progress towards the goal. An extent to which the image of the key is colored or shaded indicates the percentage completion towards the goal.
By configuring the digital dashboard and using the activating objects, the user may see progress towards one or more goals simply by viewing the activating object. This is easier than needing to login to a software application in order to view the progress towards the goal. In addition, the use of an activating object and the display of the color or shaded image on the AR device may be considered to be more exciting or fun to certain users, particularly young and savvy technology users, than to login to a software application to obtain this progress information.
The AR software application runs on the electronic computing device. The AR software application is typically a financial software application that supports online banking applications. Example online banking applications include account access, bill pay, electronic banking statements, etc. The AR software application also provides support for the digital dashboard, as discussed later herein. In addition the AR software application supports AR display of financial and other data on the display screen of the AR device 102. The AR software application also supports biometric authentication for the user, generally one or more of facial recognition, retinal scan, finger print scan or voice recognition. Biometric information obtained at the AR device 102 is compared against a biometric profile of the user that is typically available on the AR device 102.
The example purchase goal 302 includes a pull-down list box 304. The pull-down list box 304 lists the names of purchase goals. Example purchase goals 306 that may be selected via the pull-down list box 304 include a car, a house, college, a vacation or other. More, fewer or other purchase goals 306 may be selected.
The example savings goal 308 is a dollar amount to save for a selected purchase goal 302. The user may enter the dollar amount to save into an edit box 310.
The example completion date 312 is typically a date by which the user expects to have completed savings for the savings goal 308. The user may enter the completion date into an edit box 314.
The example amount/percent to allocate 316 is a dollar amount or a percentage that the user decides to apply to the savings goal 308. The date to allocate 320 is a date on which the dollar amount or percentage is to be allocated to the savings goal 308. Allocation may comprise allocating a percentage of money for the savings goal 308 from a source of funds, for example the user's savings or checking account. Allocation may also comprise transferring a percentage of money to a special savings account assigned for the savings goal 308. The date to allocate 320 may be a specific date, a specific day of a month, a specific day or a week or some other interval or date the user chooses.
When the user allocates a percentage of money from the user's savings or checking account to be applied to the savings goal 308, the user builds up a reserve fund within the user's savings or checking account. This means that less of the user's funds are available for other purchases. When the user makes a purchase, for example via a debit card, that may cause the user's available funds to be exceeded, the user may receive an alert, for example an email or text message, indicating that the user's available funds are to be exceeded and that the reserve funds will need to be used.
Allocation of the percentage of money using the reserve fund may help the user budget for a goal without the user or bank needing to set up with or deal with a special savings account. However, in some cases users may prefer to set up a special savings account in which to allocate funds towards a savings goal.
The user may enter a dollar amount or a percentage to allocate into an edit box 318. When a dollar amount is entered into the edit box 318, the dollar amount may be allocated from the source of funds to the reserve fund or the special savings account. When a percentage is entered into the edit box 318, a percentage of an amount in the source of funds may be allocated to the reserve fund or to the special savings account. For example, if the source of funds is a checking account, and the user enters a percentage into the edit box 318, a percentage of the amount in the checking account may be allocated towards the savings goal 308 on the date to allocate 320. For example, if the user entered 5 percent into the edit box 318 and the user had $1,000 in the user's checking account, $50 would be allocated from the user's checking account to the reserve fund or transferred to the special savings account on the date to allocate 320.
In some implementations, the special savings account may be a savings account that the user opened to be dedicated to savings for special goals. In other implementations, the special savings account may be a temporary account associated with a specific savings goal for the user. The temporary account would be displayed when the user accesses his/her bank account and the user could transfer money from the temporary account to any other user account. However, when the user's goal is met, the user may transfer all the money saved out of the temporary account and the temporary account may be deleted. In other implementations, the special savings account may not be used. For example, money may be directly transferred into the user's checking account.
In some implementations, an amount/percent to allocate 316 may be automatically calculated. In these implementations, when a savings goal 308 and a completion date 312 are entered, the systems and methods may calculate an amount or percent of money needed to be obtained from a source of funds in order for the purchase goal 302 to be reached by the completion date 312. For example, if the user gets paid twice a month, a certain percentage or amount may be automatically withdrawn from the user's checking account each pay period in order to meet the purchase goal 302.
The source of funds 324 identifies where the money to be saved from the savings goal 308 is to be allocated from. For the example user interface 300, a source of funds 328 is selected via a check box 326. As shown, in
Other sources of funds 328 (not shown) in addition to checking and savings are possible. For example, one additional source may be a paycheck. For example, the user may choose to save a percentage from each paycheck toward the savings goal 308.
The activating object 338 is an object that is associated with the purchase goal 302. When the activating object 338 is viewed via the camera 202 on the AR device 102 and the appropriate activating command is issued, the AR device 102 displays a progress towards the purchase goal 302 on the viewed activating object 338. The progress is typically shown via a color or shading overlay on the activating object 338, as discussed in more detail later herein. However, the progress may also be shown in other ways, for example via a progress line across the activating object 338. A length of the progress line may correspond to a percentage progress towards the goal.
For the example user interface 300, the activating object 338 is selected via check boxes 340. An activating object 338 is selected by selecting one or more checkboxes. Example choices for the activating object include a car, a house, a winter or summer vacation scene, a book bag, a key and a cell phone. More, fewer or different activating objects.
Typically, at least one activating object 338 is similar to or the same as the purchase goal 302. For example, if the purchase goal 302 is a car, at least one activating object 338 may also be a car. For example, when a car is selected for the activating object 338 and the user views a car with the AR device and uses the appropriate verbal command to initiate the display, a display of progress towards the purchase goal 302 is shown on the image of the car on the AR device 102.
However, a user may want to trigger the display of progress without needing to go outside and look at a car. Therefore, the user may select a second activating object 338, for example a key, for the car. Then, when the user views a key with the AR device and uses the appropriate verbal command to initiate the display, the progress is displayed on the second activating object 338. For example, when the key is selected as an activating object 338 and the user views the key and uses the appropriate verbal command to initiate the display, the key fills up with a color or pattern indicating the percentage completion towards the purchase goal 302.
Similarly, the user may select a book bag or similar object when the purchase goal 302 is saving for college or the user may select a cell phone as an activating object for any purchase goal 302. In general, the user may select any activating object that the user typically carries, such as a cell phone, a key or a book bag, to trigger a display of progress towards the goal on the activating object 338. When the user selects these activating objects, the progress display is typically shown on the activating objects, for example on the cell phone, key or book bag.
The color or pattern for total 344 is a color or pattern to be used on the activating object 338 to indicate total progress to date towards the purchase goal 302. For example, if a solid color is selected, the selected color overlays an image of the activating object 338 to show current total progress towards the purchase goal 302, as explained later herein. For the example user interface 300, the color or pattern for total 344 is selected via a pull-down list box 346. One of a plurality of colors or patterns 348 may be selected.
The systems and methods also permit incremental progress towards the purchase goal 302 to be displayed. When incremental progress is displayed, progress towards the purchase goal 302 between time periods may be displayed. The color or pattern for increment 350 is a color or pattern to be used on the activating object 338 to display incremental progress towards the purchase goal. In some implementations, the color or pattern for increment 350 may be used alternately with the color or pattern for total 344 to show incremental progress towards a goal, as explained later herein. For the example user interface 300, the color or pattern for increment 350 is selected via a pull-down list box 352. One of a plurality of colors or patterns 354 may be selected. In some implementations (not shown), the user interface 300 may permit a selection of multiple incremental colors or patterns.
An increment checkbox 356 may be selected to activate a display of an incremental color or pattern. For the example user interface 300, when the increment checkbox 356 is not selected, progress towards purchase goal 302 is displayed using the color or pattern for total 344. When the increment checkbox 356 is selected, progress towards purchase goal 302 is displayed using both the color or pattern for total 344 and the color or pattern for increment 350, as explained later herein.
Pattern overlay 502 represents an incremental savings amount from a particular event. In some implementations, the event may represent a calendar date, such as the first day of a month. In other implementations, the event may represent a new payment to savings. For example, pattern overlay 404 may represent a percent completion towards savings goal 308 up to a last payment to savings and pattern overlay 502 may represent an added percent completion towards savings goal 308 as a result of the last payment to savings. In this case, pattern overlay 502 would indicate that an additional 11% was added towards savings goal 308 as a result of the last payment. The additional 11% represents a difference between the current 36% towards savings goal 308 and the previous 25% towards the savings goal. Other events are possible.
In some implementations, a different pattern or color is displayed each time an event occurs. In some implementations, there may be two distinct patterns or colors, for example pattern overlay 404 and pattern overlay 502, that are displayed alternately each time a payment event occurs. In other implementations, there may be a plurality of different patterns or colors that are used, each event being represented by a different pattern or color.
Progress displays similar to progress displays 400 and 500 may be used for other purchase goals 302. For example, pattern or color overlays may be used to show progress on an image of a house, a college campus, a vacation scene, etc., depending on the purchase goal 302. In some implementations, progress indications other than colors or shading may be used. For example, in some implementations, a progress line may be drawn through the activating object 338. A length of the progress line may represent progress towards the purchase goal 302.
The systems and methods may also be used for non-financial applications in addition to or in lieu of purchase goals. Some example non-financial applications include progress of a team in accomplishing a goal, progress towards completing tasks on a to-do list, progress towards an exercise goal, etc. Other non-financial goals are possible.
With non-financial goals, different objects to display and different activating objects may be used. For example, for a project goal, the object to display may be a computer and an activating object may be a pencil or a pen. Similarly for an exercise goal, an object to be displayed may be an image of a runner and the activating object may be a running shoe. Other objects to display and other activating objects are possible. In addition, for non-financial goals different events may be used for incremental progress displays. For example, each exercise session may be used as an event.
At operation 602, a digital dashboard is configured, for example the digital dashboard corresponding to user interface 300. The digital dashboard is typically configured via a financial software application on a computing device of the user. The computing device may be a smart telephone, tablet computer, laptop computer, desktop computer or other similar computing device.
At operation 604, the user views an activating object with AR device 102. The activating object may be any object configured on the digital dashboard to initiate a display of progress towards the goal. When the activating object is viewed on AR device 102, an image of the activating object is received at the camera 202 of AR device 102. In method 600, optical recognition software on AR device 102 identifies the image of the activating object. In some implementations, the image of the activating object may be sent to a server computer, for example server computer 104 for identification.
At operation 606, one or more activating actions are received at AR device 102. The activating action is typically a verbal command used to initiate the display of progress towards the goal on AR device 102. The verbal command is typically pre-programmed into AR device 102. For example, AR device 102 may be pre-programmed to recognize a verbal command of “goal getter” and initiate an application for displaying progress towards the goal when the verbal command “goal getter” is recognized and when the activating object is identified. In method 600, recognition of the “goal getter” command and identification of the activating object, for example a car or a key, identifies that progress towards the goal associated with the car or the key is to be displayed. In some implementations, a second activating command, for example “car,” may be used. The second activating command may be used, for example, if the image of the activating object is not recognized at AR device 102.
At operation 608, an identity of the user of AR device 102 is identified. The identification is typically implemented by biometric recognition using one or more of facial recognition, retinal scan, voice recognition and finger print scan.
At operation 610, a request for information regarding progress towards a goal for the user is sent to a server computer, for example to server computer 104. The request for information typically includes an identifier for the user, an identifier for the activating object and one or more fields indicating that the request is for progress towards a goal for the user. The identifier for the user may be a financial account identification number, for example the user's bank account number or a unique identifier assigned to the user.
At operation 612, the goal information is received from the server computer. In some implementations, the goal information may include specific data fields for items such as amount of goal, amount saved for goal and percent saved for goal. The AR software application on AR device 102 may use this information to render a display of progress towards the goal. In other implementations, the goal information may include an XML (extensible markup language) file that implements a rendering of a goal object and an overlay showing progress towards the goal.
At operation 614, an image, similar to images shown in
As illustrated in the example of
The mass storage device 714 is connected to the CPU 702 through a mass storage controller (not shown) connected to the system bus 722. The mass storage device 714 and its associated computer-readable data storage media provide non-volatile, non-transitory storage for the AR device 102. Although the description of computer-readable data storage media contained herein refers to a mass storage device, such as a hard disk or solid state disk, it should be appreciated by those skilled in the art that computer-readable data storage media can be any available non-transitory, physical device or article of manufacture from which the central display station can read data and/or instructions.
Computer-readable data storage media include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable software instructions, data structures, program modules or other data. Example types of computer-readable data storage media include, but are not limited to, RAM, ROM, EPROM, EEPROM, flash memory or other solid state memory technology, CD-ROMs, digital versatile discs (“DVDs”), other optical storage media, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by the AR device 102.
According to various embodiments of the invention, the AR device 102 may operate in a networked environment using logical connections to remote network devices through the network 720, such as a wireless network, the Internet, or another type of network. The AR device 102 may connect to the network 720 through a network interface unit 704 connected to the system bus 722. It should be appreciated that the network interface unit 704 may also be utilized to connect to other types of networks and remote computing systems. The AR device 102 also includes an input/output controller 706 for receiving and processing input from a number of other devices, including a touch user interface display screen, or another type of input device. Similarly, the input/output controller 706 may provide output to a touch user interface display screen or other type of output device.
As mentioned briefly above, the mass storage device 714 and the RAM 710 of the AR device 102 can store software instructions and data. The software instructions include an operating system 718 suitable for controlling the operation of the AR device 102. The mass storage device 714 and/or the RAM 710 also store software instructions, that when executed by the CPU 702, cause the AR device 102 to provide the functionality of the AR device 102 discussed in this document. For example, the mass storage device 714 and/or the RAM 710 can store software instructions that, when executed by the CPU 702, cause the AR device 102 to display received financial data on the display screen of the AR device 102.
Although various embodiments are described herein, those of ordinary skill in the art will understand that many modifications may be made thereto within the scope of the present disclosure. Accordingly, it is not intended that the scope of the disclosure in any way be limited by the examples provided.
Number | Name | Date | Kind |
---|---|---|---|
6031547 | Kennedy | Feb 2000 | A |
8180396 | Athsani et al. | May 2012 | B2 |
8239298 | Wilson et al. | Aug 2012 | B1 |
8438110 | Caiman et al. | May 2013 | B2 |
10210767 | Johansen | Feb 2019 | B2 |
10373383 | Werner | Aug 2019 | B1 |
20070203860 | Golden et al. | Aug 2007 | A1 |
20080033791 | Jones | Feb 2008 | A1 |
20080059267 | Hamilton | Mar 2008 | A1 |
20080082451 | Schneider | Apr 2008 | A1 |
20090237546 | Bloebaum et al. | Sep 2009 | A1 |
20090254859 | Arrasvuor et al. | Oct 2009 | A1 |
20090271287 | Halpern | Oct 2009 | A1 |
20090300513 | Nims | Dec 2009 | A1 |
20100185529 | Chesnut et al. | Jul 2010 | A1 |
20110145109 | Watson | Jun 2011 | A1 |
20110213684 | Tait | Sep 2011 | A1 |
20110258049 | Ramer et al. | Oct 2011 | A1 |
20120069131 | Abelow | Mar 2012 | A1 |
20120136762 | Wissenbach | May 2012 | A1 |
20120197689 | Hunter et al. | Aug 2012 | A1 |
20120197783 | Hunter et al. | Aug 2012 | A1 |
20120229625 | Caiman | Sep 2012 | A1 |
20120231424 | Caiman et al. | Sep 2012 | A1 |
20120231814 | Caiman et al. | Sep 2012 | A1 |
20120232937 | Caiman et al. | Sep 2012 | A1 |
20120232966 | Caiman et al. | Sep 2012 | A1 |
20120232968 | Caiman et al. | Sep 2012 | A1 |
20120232976 | Caiman et al. | Sep 2012 | A1 |
20120232977 | Caiman et al. | Sep 2012 | A1 |
20120233015 | Caiman et al. | Sep 2012 | A1 |
20120233032 | Caiman et al. | Sep 2012 | A1 |
20120233072 | Calman et al. | Sep 2012 | A1 |
20120264089 | Hoang et al. | Oct 2012 | A1 |
20130013476 | Knafelz et al. | Jan 2013 | A1 |
20130033522 | Calman et al. | Feb 2013 | A1 |
20130166332 | Hammad | Jun 2013 | A1 |
20130286048 | Sternitzke | Oct 2013 | A1 |
20140249916 | Verhaeghe | Sep 2014 | A1 |
20140279540 | Jackson | Sep 2014 | A1 |
20140358799 | Kingston et al. | Dec 2014 | A1 |
20150262183 | Gervais et al. | Sep 2015 | A1 |
20180032997 | Gordon | Feb 2018 | A1 |
20180150810 | Lee | May 2018 | A1 |
20190206130 | Ericson | Jul 2019 | A1 |
20200202313 | Eidam | Jun 2020 | A1 |
Entry |
---|
“highlight, v.”. OED Online. 2014. Oxford University Press. http://www.oed.com/view/Entry/86907? result=28rskey=l6dcJx& ( accessed Jan. 20, 2019). (Year: 2014). |
Highlight, v.' OED Online. 1989, Oxford English Dictionary, Second Edition http://www.oed.com/oed2/00106096 (accessed Jan. 20, 2019) (Year: 1989). |
Number | Date | Country | |
---|---|---|---|
Parent | 15471148 | Mar 2017 | US |
Child | 16734883 | US | |
Parent | 14278139 | May 2014 | US |
Child | 15471148 | US |