Patent Application
20050216464
The present invention relates in general to the processing of multimedia data using a computing device and more particularly to a system and method for automating the process of authoring a geographic travelogue of a trip including automatic integration, association, combination and creation of maps and other geographic data.
Throughout history people have shared with others accounts of their travels and adventures. These accounts, which typically are accompanied by an audio and visual presentation, are commonly referred to as “travelogues”. Before the invention of the photograph, early travelogues consisted primarily of the spoken word, along with artifacts or souvenirs collected during the travels. Later, travelogues included the lecture and souvenirs accompanied by a film, a video, slides and/or photographs of the trip.
Currently, the explosive growth of the Internet has provided a large forum for people to share their travelogues. A casual search of the Internet yields over half a million web sites pertaining to people displaying their travelogues. These travelogues typically are multimedia presentations of varying degree, which may contain some or all of the following: photographs, text, audio, maps and other types of multimedia data.
One problem, however, is that the production of these travelogues generally requires vast amounts of time. The travelogue must be tediously handcrafted such that each piece of multimedia data is positioned and incorporated manually. Depending on their complexity, the production of a single travelogue can take from hours to days. Even though these travelogues typically are labors of love for those wishing to share their travels, it is desirable to automate, assist, and otherwise simplify the travelogue authoring process.
The inclusion of maps and other geographical data adds significant interest and value to a travelogue. Maps and other geographical data greatly enhance a viewer's travelogue experience. A variety of maps may be used depending on the need, such as a city street map or a topographical map. Geographical data may be expressed in a variety of different reference systems, such as, for example, place names (e.g. Seattle, Wash.), longitude/latitude, and personal (e.g., “Grandma's house”).
One problem, however, with adding maps and other geographical data is that this increases the complexity of both the authoring and viewing processes. In particular, this requires a travelogue author to perform many tedious and time-consuming tasks. These tasks include, for example, selecting the type of maps to use, and correlating the multimedia data to the maps and other geographical data. It is desirable to automate and simply the inclusion of maps and other geographical data into travelogues.
Accordingly, there exists a need for a travelogue authoring system that automates and simplifies the authoring of travelogues produced on a computing device. In addition, there is a need for a travelogue authoring system and method that facilitates and automates the inclusion of maps and other geographical data along with other types of multimedia data into travelogues.
The invention disclosed herein includes a geographic travelogue authoring tool and method that automates the authoring of travelogue and allows the inclusion of a rich use of maps and other geographic data in combination with other multimedia data, such as text and images. The addition of maps in the travelogue greatly enhances the travelogue viewing experience. In addition, the automation of the authoring of these geographic travelogues greatly reduces the tedious nature of the task and decreases the amount of time a user spends authoring.
The geographic travelogue authoring tool obtains trip information about a trip, processes this trip information, and outputs a geographic travelogue that includes maps. The trip information includes subjects, content items, and tracks. A subject includes a person, an object, or sets thereof, which traveled together on a trip (or trips) over some interval of time. A content item includes any piece of displayable information (such as a block of text, an image, a video, or a link to a web page). Tracks include a record of where a subject traveled during the trip over some span of time. The authoring tool uses a geographic coder to mark (or tag) the content items with their associated geographic location from the trip to produce a geo-coded content item. A map selection module selects a map that corresponds to the geographic location of the geo-coded content item, and a content item and map layout module automatically incorporates the geo-coded content items, maps, and other multimedia data associated with the trip to produce the geographic travelogue.
The geographic travelogue authoring tool operates using the geographic travelogue authoring method. The geographic travelogue authoring method begins by obtaining content items that are associated with a trip and geographically codes these content items. This serves to associate a geographic location from the trip with the content item. A map then is selected based on the trip, and the map and the geo-coded content items are automatically integrated with other multimedia data (such as text, video, and photographs) to generate the geographic travelogue.
The geographic travelogue authoring tool and method include a number of geographically-oriented features that are beneficial when incorporating geographical data into a travelogue. These features include a location resolver that converts between various location reference systems. For example, if a map expresses location in a map location reference system and a geo-coded content item is expresses location in a content item location reference system, the location resolver converts the geo-coded content item to the map location reference system and vice versa.
The geographic travelogue authoring tool and method also include a clustering module that provides automatic clustering of content items by location. Based on the geo-coding, one or more clustering techniques are used to cluster the geo-coded content items. These clusters are sets of geo-coded content items that are identified as belonging to distinct clusters. The geographic travelogue authoring tool and method also include a cluster-integration module that can perform a number of operations on the clustered content items. For example, the appearance of content items on a map may be simplified by reducing the number of visual elements representing the content items. The cluster-integration module also may create a hierarchical organization of travelogue pages. The travelogue may also be split into multiple travelogues such that a “flat” set of travelogues is created for each of the clusters. Moreover, certain content items may also be excluded from a cluster. The cluster-integration module also includes the feature of automatically titling each cluster.
Another feature included in the geographic travelogue authoring tool and method is the map-selection module. The map-selection module automatically selects maps used in the geographic travelogue, including map type, size and shape. The map's dimensions and shape are appropriately selected based on the geo-coded content items. The map type is automatically selected based on the geo-coded content items. A content item and map-layout module provides an automated layout of content items and selected maps within the travelogue. Geo-coded content items are automatically arranged on an appropriate map such that each content item is on or near its associated geographic location. Visual cues, which show a relationship between a content item and its associated location, can also be included. These visual cues can be active visual cues, passive visual cues, or both. Active cues dynamically change based on interaction from a user, while passive visual cues are statically viewable.
The geographic travelogue authoring tool and method also include multiple subject-integration modules that uses the subjects, content items, and tracks to perform a variety of operations. One such operation is the automatic identification of geographic intersection between multiple subjects. Another operation is the automatic identification of geographic and temporal intersections, whereby multiple subjects converged to the same location at overlapping times. The module also features the automatic selection of a special set of content items based on geographic clustering and subject. Another feature is multi-faceted hierarchical organization of travelogue pages.
Other features included in the geographic travelogue authoring tool and method are a track incorporation module that performs the automatic incorporation of tracks into the travelogue. This allows a viewer to experience the tracks in a number of different ways. The module can align and overlay the tracks onto a selected map, and can “snap” the tracks on known landmarks on the map. Visual cues can also be used to correlate the tracks with content item, and can be active, passive, or both. The tracks may also be dynamically displayed such that they are animated to show a path taken by the subjects. Additional content items may be added using an additional content-item generator. The generator automatically creates additional content items relevant to an existing content item or cluster's location. Based on the analysis of the location, additional content items may be added (such as a hypertext link that links to a web page).
The present invention can be further understood by reference to the following description and attached drawings that illustrate aspects of the invention. Other features and advantages will be apparent from the following detailed description of the invention, taken in conjunction with the accompanying drawings, which illustrate, by way of example, the principles of the present invention.
Referring now to the drawings in which like reference numbers represent corresponding parts throughout:
In the following description of the invention, reference is made to the accompanying drawings, which form a part thereof, and in which is shown by way of illustration a specific example whereby the invention may be practiced. It is to be understood that other embodiments may be utilized and structural changes may be made without departing from the scope of the present invention.
I. Introduction
Travelogues typically are in the form of web sites featuring photographs, text, maps, and other types of geographical data. These travelogues are tediously and laboriously handcrafted by persons wishing to share tales of their travels and trips. At present, there are tools that allow a user to author travelogues manually containing photographs and text. However, there is a lack of authoring tools available that automates the authoring of travelogues and facilitates the inclusion of maps and other geographical information into these travelogues.
The geographical travelogue authoring tool and method described herein allow the incorporation of maps and other geographical data with text, photographs, and other multimedia data. The authoring tool and method include a number of geographically-oriented features that provide automation and are useful when blending the geographical data into the geographic travelogue. By way of example, content items are tagged or marked with a corresponding geographic location (“geo-coding”). Geo-coding content items allows automated matching of the items to maps. Correspondence between a map and a geo-coded content item is established using a location resolver. This location resolver is used in situations where the item's geo-coding is incompatible with a map. Automated clustering of content items by location allows sets of content items to be identified as belonging to distinct clusters. Using its rich array of features and a variety of data associated with a trip, the geographical travelogue authoring tool automates, assists, and otherwise simplifies the authoring of travelogues generated on a computing device. The result is an aesthetically-pleasing and professional travelogue suitable for viewing over the Web.
II. General Overview
The geographic travelogue authoring system and method facilitates the authoring of travelogues that incorporate map and other geographical data with other types of multimedia data.
More specifically, the geographic travelogue authoring system 100 facilitates the authoring of a travelogue by a user about a trip 110. Typically, this is a trip 110 (or trips) that the user has taken. Trip information is collected for input to the system 100. This trip information includes subjects 130, content items 140, and tracks 150. Each of these pieces of trip information is discussed in detail below.
The geographic travelogue authoring system 100 resides on a computing device 160. The trip information is entered into the computing device 160 for processing by the system 100. As explained below, at least some of the trip information is geographically coded for use by the system 100. Geographic coding (or “geo-coding”) means that a piece of trip information (such as a photograph or block of text) is tagged with its geographic location. The geographic travelogue authoring system 100 uses this geo-coding to automatically authoring certain portions of the travelogue 120. The result is the geographic travelogue of the trip 120.
III. Exemplary Operating Environment
The geographic travelogue authoring system 100 and method are designed to operate in a computing environment and on a computing device, such as the computing device 160 shown in
The geographic travelogue authoring system and method is operational with numerous other general purpose or special purpose computing system environments or configurations. Examples of well known computing systems, environments, and/or configurations that may be suitable for use with the geographic travelogue authoring system and method include, but are not limited to, personal computers, server computers, hand-held, laptop or mobile computer or communications devices such as cell phones and PDA's, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.
The geographic travelogue authoring system and method may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform particular tasks or implement particular abstract data types. The geographic travelogue authoring system and method may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices. With reference to
Components of the computer 210 may include, but are not limited to, a processing unit 220, a system memory 230, and a system bus 221 that couples various system components including the system memory to the processing unit 220. The system bus 221 may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. By way of example, and not limitation, such architectures include Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus also known as Mezzanine bus.
The computer 210 typically includes a variety of computer readable media. Computer readable media can be any available media that can be accessed by the computer 210 and includes both volatile and nonvolatile media, removable and non-removable media. By way of example, and not limitation, computer readable media may comprise computer storage media and communication media. Computer storage media includes volatile and nonvolatile removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.
Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by the computer 210. Communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.
Note that the term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of any of the above should also be included within the scope of computer readable media.
The system memory 230 includes computer storage media in the form of volatile and/or nonvolatile memory such as read only memory (ROM) 231 and random access memory (RAM) 232. A basic input/output system 233 (BIOS), containing the basic routines that help to transfer information between elements within the computer 210, such as during start-up, is typically stored in ROM 231. RAM 232 typically contains data and/or program modules that are immediately accessible to and/or presently being operated on by processing unit 220. By way of example, and not limitation,
The computer 210 may also include other removable/non-removable, volatile/nonvolatile computer storage media. By way of example only,
Other removable/non-removable, volatile/nonvolatile computer storage media that can be used in the exemplary operating environment include, but are not limited to, magnetic tape cassettes, flash memory cards, digital versatile disks, digital video tape, solid state RAM, solid state ROM, and the like. The hard disk drive 241 is typically connected to the system bus 221 through a non-removable memory interface such as interface 240, and magnetic disk drive 251 and optical disk drive 255 are typically connected to the system bus 221 by a removable memory interface, such as interface 250.
The drives and their associated computer storage media discussed above and illustrated in
Other input devices (not shown) may include a microphone, joystick, game pad, satellite dish, scanner, radio receiver, or a television or broadcast video receiver, or the like. These and other input devices are often connected to the processing unit 220 through a user input interface 260 that is coupled to the system bus 221, but may be connected by other interface and bus structures, such as, for example, a parallel port, game port or a universal serial bus (USB). A monitor 291 or other type of display device is also connected to the system bus 221 via an interface, such as a video interface 290. In addition to the monitor, computers may also include other peripheral output devices such as speakers 297 and printer 296, which may be connected through an output peripheral interface 295.
The computer 210 may operate in a networked environment using logical connections to one or more remote computers, such as a remote computer 280. The remote computer 280 may be a personal computer, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above relative to the computer 210, although only a memory storage device 281 has been illustrated in
When used in a LAN networking environment, the computer 210 is connected to the LAN 271 through a network interface or adapter 270. When used in a WAN networking environment, the computer 210 typically includes a modem 272 or other means for establishing communications over the WAN 273, such as the Internet. The modem 272, which may be internal or external, may be connected to the system bus 221 via the user input interface 260, or other appropriate mechanism. In a networked environment, program modules depicted relative to the computer 210, or portions thereof, may be stored in the remote memory storage device. By way of example, and not limitation,
IV. System and Associated Components
The geographic travelogue authoring system 100 shown in
Referring to
A subject includes a person, an object, or sets thereof, which traveled together on a trip (or trips) over some interval of time. A content item includes any piece of presentable information. For example, a content item could include a block of text, an image, a video, an audio clip, or a link to a web page. Tracks include a record of where a subject traveled during the trip over some span of time. Tracks may or may not be time stamped. Tracks are displayable on a map as the path that a subject traveled during the trip. Some or all of the subjects 130, content items 140, and tracks 150 are used by the geographic travelogue authoring system 100 in automatically or semi-automatically generating the geographic travelogue.
The geographic travelogue authoring system 100 includes a geographic coder 300 that geographically codes (“geo-coding”) the trip information, including the subjects 130, content items 140, and tracks 150. Typically, the content items 140 are the trip information that is most often geo-coded. A content item is said to be geo-coded if it is somehow associated with location metadata that indicates there is a relationship between the content item and a location. In other words, the content item is tagged (or marked) by its geographical location. For the purposes of this document, geo-coding includes an association between a content item and any type of geographic location data. This geographical location data is expressed in some type of location-reference system. This reference system may use various means to express location. For example, a location may be expressed using universally-established and understood means, such as longitude/latitude), a place name, a mail address, or using a personalized means, such as “Grandma's house”.
The output of the geographic coder 300 is a geo-coded content item 305. As stated above, the geo-coded content item 305 may have its location expressed in a number of different ways. A determination is made as to whether the geo-coded content item 305 needs its location resolved 310. If so, then a location resolver 315 is used to resolve the location. The location resolver 315 is any system that can convert between the various location reference systems (i.e., ways of indicating a location). For example, the location resolver 315 is capable of converting a content item that is a geo-coded by a mail address to a map that uses latitude/longitude. In this manner, the output of the location resolver 315 is a geo-coded content item with its location resolved 320. In addition, the location resolver can perform the reverse conversion, such as from latitude/longitude to the mail address. If the geo-coded content item 305 does not need its location resolved, the location resolver 315 is skipped.
The geo-coded content item with a resolved location 320 can be processed by any combination of the several modules contained in the geographic travelogue authoring system 100. In addition, these modules can also use the other types of trip information, such as the subjects 130 and the tracks 150. The following modules may use one or more of any combination of trip information as input to help in the generation of a geographic travelogue 120.
A clustering module 325 provides automatic clustering of geo-coded trip information based on geographic location. This groups each piece of trip information (such as a geo-coded content item with location resolved 320) into distinct clusters. Typically, a content item is processed by the clustering module 325 to produce a clustered content item 330. This clustered content item 330 is associated with a particular cluster usually containing other clustered content items. The geographic travelogue authoring system 100 also includes a clusters integration module 335. This integration module 335 automatically takes clusters found by the clustering module 325 and integrates each of the clusters into the geographic travelogue 120.
A map selection module 340 automatically selects and sizes a map based on the trip information. The output is a map 345 that becomes a part of the geographic travelogue 120. A content item and map layout module 350 provides automatic layout of the geo-coded content items 320 and the map 345.
A multiple-subject-integration module 355 integrates the trip information of several of the subjects 130 into the geographic travelogue 120. A tracks-incorporation module 360 automatically incorporates the tracks 150 such that the tracks are intelligently positioned within the geographic travelogue 120. Additional content-item generators 365 automatically create additional content items that are relevant to an existing location of a content item or a cluster. The result of processing the trip information by the above-mentioned modules of the geographic travelogue authoring system 100 is the geographic travelogue 120.
V. Operation
The geographic travelogue authoring system 100 discussed above operates using the geographic travelogue authoring method. The operation of the geographic travelogue authoring system 100 and method used therein shown in
In particular, as shown in
The clusters integration module 335 may also create a hierarchical organization of travelogue pages (box 720). By way of example, one page of the travelogue may include a map of Europe with clusters in London, Paris, and Berlin. Each of there clusters may lead to more focused pages. For instance, the London page may consist of clusters near Piccadilly Circus, Big Ben, and Buckingham Palace. In this example, the content items may also be hierarchically organized so that the majority of London images are only accessible from the London travelogue page. The travelogue may also be split into multiple travelogues (box 730). Thus, instead of a hierarchical organization, a “flat” set of travelogues can be created for each of the clusters.
Certain content items may also be excluded from a cluster (box 740). When one cluster contains a large amount of content items in relation to another cluster, this determination is made based upon at least two criteria. For example, if one cluster contains 99 content items and another contains only 1 content item, there are at least two options. One option is to eliminate some percentage of the 99 content items in the first cluster, because that cluster is over-represented. This process may be undertaken in a variety of ways, including, but not limited to a random selection process, or a representative-item selection process which creates sub-clusters of items based on some similarity metric and chooses a limited number from each cluster. A second option is to eliminate the single-item cluster altogether, as it is likely to be an outlier in the content-item data.
The clusters integration module 335 also includes the feature of automatically titling each cluster (box 750). This automatic titling is performed using appropriate place names. For example, if there is a cluster of 50 photographs in New York and 50 photographs in Washington, these two groups could be separated into two separate pages. Each of these two pages would be automatically labeled by place name. This may involve the use of the location resolver 315 to resolve location.
The map type is automatically selected based on the geo-coded content items (box 820). This means that the map type is selected to be appropriate for the location covered by the content items. For example, if the majority of content items in a travelogue appear to follow a trail in the mountains, a topographical map may be selected instead of a city street map. The output is a map having a type, size and shape that are suitable for displaying the geo-coded content items (box 830).
Next, visual cues may be added. Visual cues show a relationship between a content item and its associated location. These visual cues can be active visual cues (box 920), passive visual cues (box 930), or both. Active cues dynamically change based on interaction from a user. For example, a spot on a map may be highlighted upon a user moving a mouse cursor (“mousing over”) a content item displayed on a map. Passive visual cues are statically viewable. For instance, a line connecting a content item with its location on the map.
Another feature is the automatic identification of geographic and temporal intersections (box 1010). This means that multiple subjects converged to the same location and at overlapping times. In an alternate embodiment, a separate travelogue page is created for this intersection, and combines the content items of all subjects that geographically and temporally intersected. Alternatively, such events may be marked specially in any travelogues that contain that instant at that location, and link to other relevant travelogue pages.
The module 355 also features the automatic selection of a special set of content items based on geographic clustering and subject (box 1020). For example, for each cluster of geo-coded content items, a representative set of photographs may be selected such that each subject (such as a person) who traveled on the trip is equally represented. Another feature is multi-faceted hierarchical organization of travelogue pages (box 1030). By way of example, two parallel sets of travelogue pages may be created. One page may be hierarchically organized by location clusters, while the other page may be hierarchically organized by subjects.
The foregoing description of the invention has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form disclosed. Many modifications and variations are possible in light of the above teaching. It is intended that the scope of the invention be limited not by this detailed description of the invention, but rather by the claims appended hereto.
