Patent Application
20100082624
This disclosure relates to categorizing digital media, and more particularly to a system and method for categorizing digital media based on correspondence between characteristics of individual digital media items and characteristics associated with one or more calendar events.
Consumers have access to a wide variety of portable electronic devices that enable creation of digital media including, for example, digital cameras, digital video recorders, cell phones, smart phones, and digital sound recording devices, among others. These electronic devices are popular with consumers in part because they allow spontaneous creation of high-quality digital content whenever, and wherever the mood strikes. Still further, advances in digital media storage technology allow users to create and store large amounts of digital media. For example, a user can easily store thousands of high-quality pictures on a single flash memory storage chip. However, as capacities increase, the physical size of the media is decreasing. Where a 4″×5.75″×1″, 30 gigabyte hard drive was cutting edge in 1998, today a 32 gigabyte SD card has higher capacity, is less expensive, and is substantially smaller. And even so, other, smaller media are rapidly overtaking SD cards in capacity and cost.
Given these greater opportunities for digital storage, the ever expanding digital media libraries of most users have given rise to a desire to organize and often group the media items for better management. Often, users want to categorize their digital media based on several user specified criterion, such as time, date, and/or location of creation of the media. Also, the digital media can be categorized based on persons associated with the particular item of recorded digital media; for instance, groupings may be desired based on the persons appearing in photographs or the singers of songs. Other forms of categorizing digital media can be utilized as well. This categorizing process can be complicated and time consuming inasmuch as digital media libraries quickly grow beyond a manageable size to manually go through and label for categorization.
Many consumers utilize digital calendars. Frequently portable electronic devices such as PDAs and smart phones incorporate digital calendar features. Digital calendars may also take the form of computer programs that run locally on a user's desktop computer. Often portable electronic calendars synchronize with desktop computer based calendars. Alternatively, digital calendars can be stored remotely on a server and accessed through a web interface. In general, all digital calendars allow users to enter “events” into the calendar and which are defined based on periods in time. The present disclosure appreciates that digital media is often recorded/created in association with these events and it would be beneficial to be able to “categorize” media based on association with calendared events.
Additional features and advantages of the disclosure will be set forth in the description which follows, and in part will be obvious from the description, or may be learned through the practice of what is taught. The features and advantages of the disclosure may be realized and obtained by means of the instruments and combinations particularly pointed out in the patented claims. These and other features will become more fully apparent from the following description and the patented claims, or may be learned by the practice of that which is described.
This disclosure describes a system and method for categorizing digital media based on correspondence between characteristics of individual digital media items, such as photographs, and characteristics associated with one or more calendar events. Disclosed are systems, methods and computer readable media for categorizing such digital media based on correspondence between characteristics of individual media items and characteristics associated with one or more calendar events on a digital calendar such as iCal from Apple, Inc.
Aspects of the method described herein and the principles associated therewith are also applicable to the system and computer readable medium embodiments that are also described. Accordingly, a method of categorizing digital media (photographs) based on correspondence between characteristics of individual digital media items (date, time, location and/or people portrayed) and characteristics associated with one or more calendar events (date, time period, place and/or event attendees) is disclosed. The method includes acquiring, receiving and/or processing, for each of a plurality of digital media items, data representative of these types of characteristics of each of the respective digital media items. The method also includes acquiring, receiving and/or processing, for each of a plurality of calendar events, data representative of these similar types of characteristics of each of the respective calendar events. The method then includes relating a group of digital media items, such as photographs, together based on matching characteristics of each digital media item in the group to like or similar characteristics of a calendar event.
The matching characteristics of each digital media item in the group to that of a calendar event can include date and time.
The method may include another embodiment where the matching characteristics of each digital media item in the group to that of a calendar event includes date, time and likenesses {people, settings, things portrayed or other common feature(s) between items in the group}. In the instance of “likeness” representing people, likeness data for a calendar event can be derived from attendee data associated with the event. Likeness data for a digital media item can also be derived from facial recognition produced data. Additionally, likeness data for a digital media item can be derived from user-input identification data. For instance, a user can append metadata to a digital photograph specifying the persons depicted in the digital image.
In one embodiment, the matching characteristics of each digital media item in the group to that of a calendar event includes date, time and location. In this embodiment the location data for a calendar event can be derived from user-input location data, such as a specified meeting site in the calendar entry. The location data for a digital media item can be derived from GPS produced and associated data, or can be user-input.
The matching characteristic of each digital media item in the group to that of a calendar event can include an event label such as “Martha's Birthday.” An event label for a calendar event can be derived from user-input event data usually in the form of event title or subject. An event label for a digital media item would usually be derived from associated user-input data. Also, an event label for a digital media item can be derived from scene recognition data.
In a related aspect, once a group of media items has been defined as a group, the metadata defining, for example, the event label, can be used as input data to calendar the labeled event in a related calendar program application. For instance, if a group of photographs are each labeled as “Martha's Birthday” and time-wise the group spans the period from 6 pm to 10 pm, that data can be used to create a commensurate calendar entry, the subject of which is “Martha's Birthday” on the related calendar program.
Example image formats for digital media include JPG, GIF, and TIFF. Example video formats for digital media include WMV, AVI, MPG, and DIVX. A digital camera, for example, can embed data such as time, date, and location of creation of a digital media recording, such as a photograph.
In one aspect, the method categorizes digital media taken by a digital camera, digital media taken by a video recorder, and/or digital media taken by a digital sound recorder based on calendar events. The method applies to other types of digital media as well. As mentioned above, the calendar data can be stored locally and/or remotely. The principles described herein apply to any digital calendars from any provider.
In order to describe the manner in which the advantages and features of this disclosure can be obtained, a more particular description is provided below, including references to specific embodiments which are illustrated in the appended drawings. Understanding that these drawings depict only exemplary embodiments and are not therefore to be considered limiting, the subject matter will be described and explained with additional specificity and detail through the use of the accompanying drawings in which:
Various example embodiments of the categorization schemes for digital media are described in detail below. While specific implementations are discussed, it should be understood that this is done for illustration purposes only. A person skilled in the relevant art will recognize that other components and configurations may be used without parting from the spirit and scope of the disclosure.
With reference to
The computing device 100 further includes storage devices such as a hard disk drive 160, a magnetic disk drive, an optical disk drive, tape drive or the like. The storage device 160 is connected to the system bus 110 by a drive interface. The drives and the associated computer readable media provide nonvolatile storage of computer readable instructions, data structures, program modules and other data for the computing device 100. The basic components are known to those of skill in the art and appropriate variations are contemplated depending on the type of device, such as whether the device is a small, handheld computing device, a desktop computer, or a computer server.
Although the exemplary environment described herein employs the hard disk, it should be appreciated by those skilled in the art that other types of computer readable media which can store data that are accessible by a computer, such as magnetic cassettes, flash memory cards, digital versatile disks, cartridges, random access memories (RAMs), read only memory (ROM), a cable or wireless signal containing a bit stream and the like, may also be used in the exemplary operating environment.
To enable user interaction with the computing device 100, an input device 190 represents any number of input mechanisms, such as a microphone for speech, a touch-sensitive screen for gesture or graphical input, keyboard, mouse, motion input, speech and so forth. The device output 170 can also be one or more of a number of output mechanisms known to those of skill in the art. In some instances, multimodal systems enable a user to provide multiple types of input to communicate with the computing device 100. The communications interface 180 generally governs and manages the user input and system output. There is no restriction requiring operation on any particular hardware arrangement and therefore the basic features here may easily be substituted for improved hardware or firmware arrangements as they are developed.
For clarity of explanation, the illustrative system embodiment is presented as comprising (including, but not limited to) individual functional blocks (including functional blocks labeled as a “processor”). The functions these blocks represent may be provided through the use of either shared or dedicated hardware, including, but not limited to, hardware capable of executing software. For example the functions of one or more processors presented in
As noted above, the present disclosure enables the categorizing of digital media based on calendar events, such as events in iCal from Apple, Inc. Any digital media containing embedded information, such as time, date, and location, is contemplated as within the scope and spirit of this disclosure. Also, any calendar program capable of storing events is contemplated as within the scope of this disclosure.
Having discussed some fundamental system components and fundamental calendaring concepts, the present description turns to the exemplary method embodiment that is depicted. At least in part, the method is discussed in terms of a system that is configured to practice the method.
Accordingly,
As explained above, the system and method can also group other forms of digital media according to location where the digital media was created.
The system categorizes pictures having a date and time corresponding to a first event scheduled on Thursday 408 into group 5. The system categorizes pictures having a date and time corresponding to a second event scheduled on Thursday 410 into group 6. The system categorizes pictures having a date and time corresponding to an event scheduled on Friday 412 into group 7. The system categorizes pictures having a date and time corresponding to a first event scheduled on Saturday 414 into group 8. The system categorizes pictures having a date and time corresponding to a second event scheduled on Friday 412 into group 9. The system categorizes all digital media having a date and time corresponding to Tuesday, Wednesday, and Thursday as taken while on travel as a group 400. The system can employ various mechanisms to determine whether or not a picture of an event, for example, was taken during the scheduled event, for example using time, GPS location information, scene detection, face identification, and the like. As an example, if a user is calendared as being on vacation in the Bahamas and a picture taken during the scheduled vacation time includes snow, the system can determine that the picture is not at the scheduled location, using for example, scene detection and analysis tools. While this is an extreme example, the same fundamental principle applies to more subtle details in media content, as well.
The system groups a fifth set of digital pictures taken while attending a football match 506. The system groups a sixth set of digital pictures taken during a dinner reception 512. The system groups a seventh set of digital pictures taken while visiting a lake 514. The system additionally categorizes the second through fifth groups of digital pictures as occurring during travel corresponding to the travel calendar event listed on the calendar of
Turning back to
The system categorizes pictures containing images of person D 612 into another group. This group contains all images of D, whether D is alone or with other persons. In this example, the digital pictures containing D are found in the pictures taken of event 3 and event 4.
The system categorizes pictures containing images of persons A and E 614 into another group. In this example, the digital pictures containing A and E are found in the pictures taken of event 3 and event 4. The system categorizes one picture with unidentified persons from event 3 or event 4 into a default group labeled “Wednesday” 616, which is the date the picture was taken.
The system labels three images taken during event 5 containing unidentified persons only as “Thursday” 618, which is the date the pictures were taken. Additionally, the system categorizes and labels an image taken during event 6 containing unidentified persons into the same “Thursday” 618 group, which is the date the picture was taken.
Also,
For example, the system categorizes pictures taken at GPS location A 702 which correspond to the same location identified in the calendar into a group. This group of location A pictures includes pictures taken during first, sixth, seventh, and eighth events in the corresponding calendar.
The system categorizes pictures taken at GPS location B 704 which correspond to the same location identified in the calendar into a group. This group of location B pictures was taken during the second event in the corresponding calendar.
The system categorizes pictures taken at GPS location D 706 that correspond to the same location identified in the calendar into a group. This group of location D pictures includes pictures taken during the third or fourth events in the corresponding calendar. In this example, the system combines the overlapping events of event 3 and event 4 into one group for categorizing. Other methods of treating overlapping events can be implemented as well and as described hereinabove.
The system categorizes pictures taken at GPS location C 708 that correspond to the same location identified in the calendar into a group. This group of location C pictures was taken during the second, third, and fourth event in the corresponding calendar.
When location is the user specified criteria for media grouping, the system categorizes media items (pictures) having no GPS location information at the time of creation of the digital media into a group. In this example, the system groups pictures with no location information based on the day the pictures were taken, by default. As shown, the system categorizes pictures from event seven that have no location information into a group named “Saturday”. This label represents the day that the pictures having no location information were taken. Other default methods can be implemented to categorize pictures with no GPS location information. Although this example utilizes GPS signals to decipher location, the system can utilize other ways of recording location such as triangulation using cell phone towers, scene recognition and the like.
The system categorizes digital pictures taken on Tuesday 806 into a group. The group “4->8” 808 contains pictures taken in the four to eight time interval on Tuesday. The system categorizes digital pictures taken on Wednesday 800 of the corresponding calendar week into two groups. The first group “4-8” 812 contains pictures taken in the four to eight time interval on Wednesday. The second group “8->12” 814 contains pictures taken in the eight to twelve time interval on Wednesday.
The system categorizes digital pictures taken on Thursday 816 of the corresponding calendar week into two groups. The first group “4->8” 818 contains pictures taken in the four to eight time interval on Thursday. The second group “8->12” 820 contains pictures taken in the eight to twelve time interval on Thursday.
The system categorizes digital pictures taken on Saturday 800 of the corresponding calendar day into two groups. The first group “4->8” 802 contains pictures taken in the four to eight time interval on Saturday. The second group “8->12” 826 contains pictures taken in the eight to twelve time interval on Saturday.
Embodiments within the scope of the present disclosure may also include computer-readable media for carrying or having computer-executable instructions or data structures stored thereon. Such computer-readable media can be any available media that can be accessed by a general purpose or special purpose computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to carry or store desired program code means in the form of computer-executable instructions or data structures. When information is transferred or provided over a network or another communications connection (either hardwired, wireless, or combination thereof) to a computer, the computer properly views the connection as a computer-readable medium. A “tangible” computer-readable medium expressly excludes software per se (not stored on a tangible medium) and a wireless, air interface. Thus, any such connection is properly termed a computer-readable medium. Combinations of the above should also be included within the scope of the computer-readable media.
Computer-executable instructions include, for example, instructions and data that cause a general purpose computer, special purpose computer, or special purpose processing device to perform a certain function or group of functions. Computer-executable instructions also include program modules that are executed by computers in stand-alone or network environments. Generally, program modules include routines, programs, objects, components, and data structures and the like that perform particular tasks or implement particular abstract data types. Computer-executable instructions, associated data structures, and program modules represent examples of the program code means for executing steps of the methods disclosed herein. The particular sequence of such executable instructions or associated data structures represents examples of corresponding acts for implementing the functions described in such steps. Program modules may also comprise any tangible computer-readable medium in connection with the various hardware computer components disclosed herein, when operating to perform a particular function based on the instructions of the program contained in the medium.
Those of skill in the art will appreciate that other embodiments of this disclosure may be practiced in network computing environments with many types of computer system configurations, including personal computers, hand-held devices, multi-processor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, and the like. Embodiments may also be practiced in distributed computing environments where tasks are performed by local and remote processing devices that are linked (either by hardwired links, wireless links, or by a combination thereof) through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.
Although the above description may contain specific details, they should not be construed as limiting the claims in any way. Other configurations of the described embodiments are part of the scope of this disclosure. Accordingly, the patented claims and their legal equivalents shall only define the invention(s), rather than any specific examples described herein.
