Patent Application
20140168258
A method, apparatus and computer program product are provided in accordance with an example embodiment of the present invention in order to augment an image of a location and, more particularly, to augment an image of a location with a representation of a transient object.
Services are available to provide images of a location to a user of computing device. For example, services are available that permit a user to identify a location, such as a street address, and to be presented with an image of the location. In this regard,
Additionally, services are available that permit a user of a computing device to capture an image, such as an image of their surroundings, and to then present the image that has been captured along with additional information. In this regard, the additional information may include information associated with the various objects within the image, such as the identification of various buildings, landmarks, streets or the like that appear within the image. As such, a user may readily obtain additional information regarding objects in their immediate vicinity in order to, for example, locate a particular place of business, obtain a menu from a nearby restaurant or the like.
A method, apparatus and computer program product are provided according to an example embodiment in order to augment an image of a location with a representation of a transient object. In this regard, the transient object is not otherwise included within the image of the location, but is anticipated to be positioned at the location during one or more periods of time. As such, the transient object may be represented by an augmented image that is generated in accordance with an example embodiment of the present invention in order to provide additional information to the user, such as information regarding a transient object that may only be positioned at the location during one or more periods of time.
In one embodiment, a method is provided that includes identifying an image of a location to be presented. For example, the identification of the image of the location may include causing the image of the location to be retrieved from a database or causing the image of the location to be captured. The method of this embodiment also includes identifying, with a processor, a transient object that is not in the image of the location, but that is anticipated to be at the location during at least one period of time. The method of this embodiment also includes causing the image of the location to be presented with augmentation that provides a representation of the transient object.
The transient object may have a predetermined appearance. As such, the method of this embodiment may include augmenting the image of the location with a representation having the predetermined appearance of the transient object. Alternatively, the method may include augmenting the image of the location with a representation of the transient object that has a generic appearance.
The method of one embodiment may also include receiving a request for information regarding the transient object and causing information to be presented regarding the at least one period of time during which the transient object is anticipated to be at the location. In one embodiment, one or more transient objects are anticipated to be at the location, such as during different periods of time. As such, the method of this embodiment may cause information to be presented by causing a schedule to be presented of the at least one period of time during which the one or more transient objects are anticipated to be at the location. In this embodiment, the method may also include receiving an indication of a selected time from the schedule, determining whether a respective transient object is anticipated to be at the location at the selected time and augmenting the image with the representation of the respective transient object that is anticipated to be at the location at the selected time. In this embodiment, the method augments the image only in instances in which the respective transient object is anticipated to be at the location at the selected time. The method of this embodiment may also cause the image of the location at the selected time to be presented with augmentation by augmenting the image of the location with the representation of a respective transient object that is anticipated to be at the location at the selected time.
In another embodiment, an apparatus is provided that includes at least one processor and at least one memory including computer program code with the at least one memory and the computer program code configured to, with the processor, cause the apparatus to at least identify an image of the location to be presented. The at least one memory and the computer program code may be configured to, with the processor, cause the apparatus of one embodiment to identify the image of the location by causing the image of the location to be retrieved from a database or causing the image of the location to be captured. The at least one memory and computer program code are configured to, with the processor, cause the apparatus of this embodiment to identify a transient object that is not in the image of the location, but that is anticipated to be at the location during at least one period of time. The at least one memory and the computer program code are further configured to, with the processor, cause the apparatus of this embodiment to cause the image of the location to be presented with augmentation that provides a representation of the transient object.
The transient object may have a predetermined appearance. As such, the at least one memory and the computer program code may be further configured to, with the processor, cause the apparatus to augment the image of the location with a representation having the predetermined appearance of the transient object. Alternatively, the at least one memory and the computer program code may be further configured to, with the processor, cause the apparatus to augment the image of the location with a representation of the transient object having a generic appearance.
The at least one memory and the computer program code may be further configured to, with the processor, cause the apparatus of one embodiment to receive a request for information regarding the transient object and cause information to be presented regarding the at least one period of time during which the transient object is anticipated to be at the location. One or more transient objects may be anticipated to be at the location, such as during different periods of time. The at least one memory and computer program code may be configured to, with the processor, cause the apparatus of this embodiment to cause information to be presented by causing a schedule to be presented of the at least one period of time during which the one or more transient objects are anticipated to be at the location. The at least one memory and the computer program code may also be configured to, with the processor, cause the apparatus of this embodiment to receive an indication of the selected time from the schedule, determine whether a respective transient object is anticipated to be at the location at the selected time and augment the image with the representation of the respective transient object that was anticipated to be the location at the selected time. In this embodiment, the at least one memory and the computer program code may be configured to, with the processor, cause the apparatus to augment the image only in an instance in which a respective transient object is anticipated to be at the location at the selected time. The at least one memory and the computer program code may also be configured to, with the processor, cause the apparatus of this embodiment to cause the image of the location at the selected time to be presented with augmentation by augmenting the image of the location with the representation of the respective transient object that was anticipated to be at the location at the selected time.
In a further embodiment, a computer program product is provided that includes at least one non-transitory computer-readable storage medium having computer-executable program code portions stored therein with the computer-executable program code portions including program code instructions for identifying an image of a location to be presented and identifying a transient object that is not in the image of the location, but that is anticipated to be at the location during at least one period of time. The computer-executable program code portions of this embodiment also include program code instructions for causing the image of the location to be presented with augmentation that provides a representation of the transient object.
The transient object may have a predetermined appearance. As such, the computer-executable program code instructions may also include program code instructions for augmenting the image of the location with a representation having the predetermined appearance of the transient object. Alternatively, the computer-executable program code portion may further include program code instructions for augmenting the image of the location with a representation of the transient object having a generic appearance.
The computer-executable program code portions of one embodiment may also include program code instructions for receiving a request for information regarding the transient object and causing information to be presented regarding the at least one period of time during which the transient object is anticipated to be at the location. One or more transient objects may be anticipated to be the location, such as during different periods of time. In this embodiment, the program code instructions for causing information to be presented may include program code instructions for causing a schedule to be presented of the at least one period of time during which the one or more transient objects are anticipated to be at the location. The computer-executable program code portions of this embodiment may also include program code instructions for receiving an indication of a selected time from a schedule, determining whether a respective transient object is anticipated to be at the location at the selected time and augmenting the image with the representation of the respective transient object that is anticipated to be at the location at the selected time. In this regard, the program code instructions may augment the image only in an instance in which a respective transient object is anticipated to be at the location at the selected time. In this embodiment, the program code instructions for causing the image of the location at the selected time to be presented with augmentation may include program code instructions for augmenting the image of the location with the representation of the respective transient object that is anticipated to be at the location at the selected time.
In yet another embodiment, an apparatus is provided that includes means for identifying an image of a location to be presented and means for identifying a transient object that is not in the image of the location, but that is anticipated to be at the location during at least one period of time. The apparatus of this embodiment also includes means for causing the image of the location to be presented with augmentation that provides a representation of the transient object.
Having thus described certain embodiments of the invention in general terms, reference will now be made to the accompanying drawings, which are not necessarily drawn to scale, and wherein:
Some embodiments of the present invention will now be described more fully hereinafter with reference to the accompanying drawings, in which some, but not all, embodiments of the invention are shown. Indeed, various embodiments of the invention may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will satisfy applicable legal requirements. Like reference numerals refer to like elements throughout. As used herein, the terms “data,” “content,” “information,” and similar terms may be used interchangeably to refer to data capable of being transmitted, received and/or stored in accordance with embodiments of the present invention. Thus, use of any such terms should not be taken to limit the spirit and scope of embodiments of the present invention.
Additionally, as used herein, the term ‘circuitry’ refers to (a) hardware-only circuit implementations (e.g., implementations in analog circuitry and/or digital circuitry); (b) combinations of circuits and computer program product(s) comprising software and/or firmware instructions stored on one or more computer readable memories that work together to cause an apparatus to perform one or more functions described herein; and (c) circuits, such as, for example, a microprocessor(s) or a portion of a microprocessor(s), that require software or firmware for operation even if the software or firmware is not physically present. This definition of ‘circuitry’ applies to all uses of this term herein, including in any claims. As a further example, as used herein, the term ‘circuitry’ also includes an implementation comprising one or more processors and/or portion(s) thereof and accompanying software and/or firmware. As another example, the term ‘circuitry’ as used herein also includes, for example, a baseband integrated circuit or applications processor integrated circuit for a mobile phone or a similar integrated circuit in a server, a cellular network device, other network device, and/or other computing device.
As defined herein, a “computer-readable storage medium,” which refers to a non-transitory physical storage medium (e.g., volatile or non-volatile memory device), can be differentiated from a “computer-readable transmission medium,” which refers to an electromagnetic signal.
A method, apparatus and computer program product are provided according to an example embodiment in order to augment an image of a location, such as the image of a streetscape as shown in
In accordance with an example embodiment of the present invention, an apparatus 20 is depicted that is specifically configured to augment an image of a location with a representation of a transient object. The apparatus may be embodied by a computing device. In one embodiment, the computing device may be embodied as a server, a personal computer, a computer workstation or the like. Alternatively, the computing device may be embodied as a mobile terminal, such as a portable digital assistant (PDA), mobile telephone, smartphone, pager, mobile television, gaming device, laptop computer, camera, tablet computer, touch surface, video recorder, audio/video player, radio, electronic book, positioning device (e.g., global positioning system (GPS) device), or any combination of the aforementioned, and other types of voice and text communications systems.
Regardless of its implementation, the apparatus 20 may include or otherwise be in communication with a processor 22 and a memory device 24, and optionally one or more other components as represented, for example, by the dashed outline of optional blocks in
In some embodiments, the processor 22 (and/or co-processors or any other processing circuitry assisting or otherwise associated with the processor) may be in communication with the memory device 24 via a bus for passing information among components of the apparatus. The memory device may be non-transitory and may include, for example, one or more volatile and/or non-volatile memories. In other words, for example, the memory device may be an electronic storage device (e.g., a computer readable storage medium) comprising gates configured to store data (e.g., bits) that may be retrievable by a machine (e.g., a computing device like the processor). The memory device may be configured to store information, data, content, applications, instructions, or the like for enabling the apparatus 20 to carry out various functions in accordance with an example embodiment of the present invention. For example, the memory device could be configured to buffer input data for processing by the processor. Additionally or alternatively, the memory device could be configured to store instructions for execution by the processor.
As noted above, the apparatus 20 may be embodied by a computing device. However, in some embodiments, the apparatus may be embodied as a chip or chip set. In other words, the apparatus may comprise one or more physical packages (e.g., chips) including materials, components and/or wires on a structural assembly (e.g., a baseboard). The structural assembly may provide physical strength, conservation of size, and/or limitation of electrical interaction for component circuitry included thereon. The apparatus may therefore, in some cases, be configured to implement an embodiment of the present invention on a single chip or as a single “system on a chip.” As such, in some cases, a chip or chipset may constitute means for performing one or more operations for providing the functionalities described herein.
The processor 22 may be embodied in a number of different ways. For example, the processor may be embodied as one or more of various hardware processing means such as a coprocessor, a microprocessor, a controller, a digital signal processor (DSP), a processing element with or without an accompanying DSP, or various other processing circuitry including integrated circuits such as, for example, an ASIC (application specific integrated circuit), an FPGA (field programmable gate array), a microcontroller unit (MCU), a hardware accelerator, a special-purpose computer chip, or the like. As such, in some embodiments, the processor may include one or more processing cores configured to perform independently. A multi-core processor may enable multiprocessing within a single physical package. Additionally or alternatively, the processor may include one or more processors configured in tandem via the bus to enable independent execution of instructions, pipelining and/or multithreading.
In an example embodiment, the processor 22 may be configured to execute instructions stored in the memory device 24 or otherwise accessible to the processor. Alternatively or additionally, the processor may be configured to execute hard coded functionality. As such, whether configured by hardware or software methods, or by a combination thereof, the processor may represent an entity (e.g., physically embodied in circuitry) capable of performing operations according to an embodiment of the present invention while configured accordingly. Thus, for example, when the processor is embodied as an ASIC, FPGA or the like, the processor may be specifically configured hardware for conducting the operations described herein. Alternatively, as another example, when the processor is embodied as an executor of software instructions, the instructions may specifically configure the processor to perform the algorithms and/or operations described herein when the instructions are executed. However, in some cases, the processor may be a processor of a specific device (e.g., a head mounted display) configured to employ an embodiment of the present invention by further configuration of the processor by instructions for performing the algorithms and/or operations described herein. The processor may include, among other things, a clock, an arithmetic logic unit (ALU) and logic gates configured to support operation of the processor.
In one embodiment, the apparatus 20 also includes a communication interface 26 embodied in either hardware or a combination of hardware and software that is configured to receive and/or transmit data from/to the apparatus, such as a computing device that embodies the apparatus. For example, in an embodiment in which the apparatus is embodied by a server, the communication interface may be configured to communicate with a client device operated by the user so as to receive user input provided via the client device and to cause images to be presented via the user interface of the client device. In this regard, the communication interface may include, for example, an antenna (or multiple antennas) and supporting hardware and/or software for enabling communications wirelessly. Additionally or alternatively, the communication interface may include the circuitry for interacting with the antenna(s) to cause transmission of signals via the antenna(s) or to handle receipt of signals received via the antenna(s). In some instances, the communication interface may alternatively or also support wired communication. As such, for example, the communication interface may include a communication modem and/or other hardware/software for supporting communication via cable, digital subscriber line (DSL), universal serial bus (USB) or other mechanisms.
The apparatus 20 may also optionally include a user interface 27, such as in instances in which the apparatus is embodied by a computing device with which the user interacts. In these embodiments, the user interface may be in communication with the processor 22 to provide output to the user and, in some embodiments, to receive an indication of a user input. For example, the user interface may include a display and, in some embodiments, may also include a keyboard, a mouse, a joystick, a touch screen, touch areas, soft keys, a microphone, a speaker, or other input/output mechanisms. The processor may comprise user interface circuitry configured to control at least some functions of one or more user interface elements such as a display and, in some embodiments, a speaker, ringer, microphone and/or the like. The processor and/or user interface circuitry comprising the processor may be configured to control one or more functions of one or more user interface elements through computer program instructions (e.g., software and/or firmware) stored on a memory accessible to the processor (e.g., memory 24, and/or the like).
In some example embodiments, such as in some instances in which the apparatus 20 is embodied by a computing device with which the user interacts, the apparatus may include an image capturing element, such as a camera 28, video and/or audio module, in communication with the processor 22. The image capturing element may be any means for capturing an image, video and/or audio for storage, display or transmission. As used herein, an image includes a still image as well as an image from a video recording. For example, in an example embodiment in which the image capturing element is a camera, the camera may include a digital camera capable of forming a digital image file from a captured image. As such, the camera may include all hardware (for example, a lens or other optical component(s), image sensor, image signal processor, and/or the like) and software necessary for creating a digital image file from a captured image. Alternatively, the camera may include only the hardware needed to view an image, while a memory device 24 of the apparatus stores instructions for execution by the processor in the form of software necessary to create a digital image file from a captured image. In an example embodiment, the camera may further include a processing element such as a co-processor which assists the processor in processing image data and an encoder and/or decoder for compressing and/or decompressing image data. The encoder and/or decoder may encode and/or decode according to, for example, a joint photographic experts group (JPEG) standard, a moving picture experts group (MPEG) standard, or other format.
In some embodiments, the apparatus 20 may include a positioning sensor 29, such as in some instances in which the apparatus is embodied by a computing device with which the user interacts. The positioning sensor may include, for example, a GPS sensor, an assisted global positioning system (A-GPS) sensor, a Bluetooth (BT)-GPS mouse, other GPS or positioning receivers or the like. However, in one example embodiment, the positioning sensor may include a pedometer or inertial sensor. In this regard, the positioning sensor may be capable of determining a location of the computing device that embodies the apparatus, such as, for example, longitudinal and latitudinal directions of the computing device, or a position relative to a reference point such as a destination or start point. Information from the positioning sensor may then be communicated to a memory 24 of the apparatus.
Referring now to
As shown in block 32 of
As shown in block 34 of
In one embodiment in which the transient object has a predetermined appearance, the apparatus 20, such as the processor 22, may augment the image of the location with a representation of the transient object that has the predetermined appearance of the transient object. As shown in
Alternatively, the apparatus 20, such as the processor 22, may augment the image of the location with a representation of the transient object that has a generic appearance. For example,
Although the representation of the transient object may be presented in the same fashion as the remainder of the image, such as with lines having the same thickness and shade, the apparatus 20, such as the processor 22, of one embodiment may augment the image of the location with a representation of the transient object that is depicted in a manner that is visually distinguishable from the remainder of the image at a location. For example, the representation of the transient object in
In one embodiment, the apparatus 20 may include means, such as the processor 22, the user interface 27 or the like, for receiving a request for information regarding the transient object. See block 38 of
In one embodiment, one or more transient objects are anticipated to be at the location. As such, the apparatus 20 may include means, such as the processor 22, the user interface 27 or the like, for causing information to be presented regarding the transient object by causing a timeline, e.g., a schedule, to be presented of one or more periods of time during which the one or more transient objects are anticipated to be at the location. In an instance in which two or more transient objects are anticipated to be at the location, the timeline, e.g., the schedule, that is presented may include one or more periods of time during which each of the respective transient objects are anticipated to be at the location. By way of example,
In this embodiment, the apparatus 20 may include means, such as the processor 22, the user interface 27 or the like, for receiving an indication of a selected time from the timeline, e.g., the schedule. See block 42 of
As shown in
By way of example, a user may virtually explore a location by providing the address of the location. In response, the apparatus 20, such as the processor 22, may retrieve an image of the location. Since the image of the location was captured in the early morning, the image does not include the food trucks that congregate at the location during lunch. The apparatus, such as the processor, of this embodiment may identify the food trucks that congregate at the location during lunch. Thus, the apparatus, such as the processor, may augment the image of the location so as to include a representation of the food trucks that are anticipated to be at the location. As such, the image of the location that is presented has been augmented with a representation of the food trucks even though the image of the location that is retrieved from the database does not include the food trucks.
By way of another example, a user may capture an image, such as a video recording, of a location. Since the image of the location is captured in the early morning, the image does not include the food trucks that congregate at the location during lunch. The apparatus 20, such as the processor 22, of this embodiment may identify the food trucks that congregate at the location during lunch. Thus, the apparatus, such as the processor, may augment the image of the location so as to include a representation of the food trucks that are anticipated to be at the location. As such, the image of the location that is presented has been augmented with a representation of the food trucks, even though the food trucks are not currently at the location.
In one embodiment in which the apparatus 20 is embodied by a server or other network entity, the apparatus, such as the processor, may identify an image of a location to be presented, such as in response to an inquiry from a mobile terminal. For example, a mobile terminal may provide an indication of a location, such as an address, the name of a landmark, location coordinates as provided by a GPS or the like. In response, the apparatus, such as the processor, may identify an image of the location, such as from memory 24, and may then provide the image of the location to the mobile terminal. The apparatus, such as the processor, embodied by a server or other network entity may also identify a transient object that is not in the image of the location, but that is anticipated to be at the location during at least one period of time. For example, the apparatus, such as the processor, may access the memory that may include information regarding transient object(s) that are anticipated to be at respective locations. As such, in an instance in which the apparatus, such as the processor, embodied by a server or other network entity, identifies a transient object that is not in the image of the location, but that is anticipated to be at the location during at least one period of time, the apparatus may provide information regarding the transient object to a mobile terminal. In one embodiment, the apparatus, such as the processor, the communications interface 26 or the like, may cause information defining a representation of the transient object to be provided to the mobile terminal such that the mobile terminal may thereafter display the image of the location with augmentation that provides a representation of the transient object, thereby causing the image of the location to be presented with augmentation that provides the representation of the transient object. In one embodiment, the apparatus, such as the processor, embodied by the server or other network entity may also retrieve information, such as a schedule, from memory and then provide information, such as the schedule, to the mobile terminal regarding at least one period of time during which the transient object is anticipated to be at the location.
As described above,
In some embodiments, certain ones of the operations above may be modified or further amplified as described below. Moreover, in some embodiments additional optional operations may also be included (some examples of which are shown in dashed lines in
Accordingly, blocks of the flowchart support combinations of means for performing the specified functions and combinations of operations for performing the specified functions for performing the specified functions. It will also be understood that one or more blocks of the flowchart, and combinations of blocks in the flowchart, can be implemented by special purpose hardware-based computer systems which perform the specified functions, or combinations of special purpose hardware and computer instructions.
Many modifications and other embodiments of the inventions set forth herein will come to mind to one skilled in the art to which these inventions pertain having the benefit of the teachings presented in the foregoing descriptions and the associated drawings. Therefore, it is to be understood that the inventions are not to be limited to the specific embodiments disclosed and that modifications and other embodiments are intended to be included within the scope of the appended claims. Moreover, although the foregoing descriptions and the associated drawings describe example embodiments in the context of certain example combinations of elements and/or functions, it should be appreciated that different combinations of elements and/or functions may be provided by alternative embodiments without departing from the scope of the appended claims. In this regard, for example, different combinations of elements and/or functions than those explicitly described above are also contemplated as may be set forth in some of the appended claims. Although specific terms are employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation.
