The present technology relates to media item recommendations, and more specifically to recommendations of media items recently played back by members of a social relationships.
Music is an important part of interpersonal relationships, and one of the best ways to learn music is through one's interpersonal relationships. Because of this fact some technologies display songs that have recently been played back by members of a user's social network. However, such technologies give too much importance to the influence of the social network and many of the songs that are displayed are irrelevant to the user.
The above-recited and other advantages and features of the present technology will become apparent by reference to specific implementations illustrated in the appended drawings. A person of ordinary skill in the art will understand that these drawings only show some examples of the present technology and would not limit the scope of the present technology to these examples. Furthermore, the skilled artisan will appreciate the principles of the present technology as described and explained with additional specificity and detail through the use of the accompanying drawings in which:
Various examples of the present technology are discussed in detail below. While specific implementations are discussed, it should be understood that this is done for illustration purposes only. A person skilled in the relevant art will recognize that other components and configurations may be used without parting from the spirit and scope of the present technology.
The disclosed technology addresses the need in the art to provide media recommendations that while influenced by a user account's social network, the recommendations also take into account the user account's taste in that media. For example, some media services do present media items played back by people in a user account's social network, but many of these media items presented to the user account are irrelevant to the user account. They are irrelevant because the presented media items do not reflect the user account's taste. Such services fail to take into account the account's taste in at least two ways. First, such services fail to recognize that a user account's social graph can be wide ranging, and many of the members of that social graph may not share a particularly strong interpersonal relationship with the user of the user account, and more to the point, many of the members of that social graph may not share the same taste in media items as the user of the user account. Second, such services fail to recognize that even members of the social graph that have similar taste in media items as the user account do not have the same taste in media items and at least some media items played back by members of the social graph that have similar taste in media items as the user account will be irrelevant to the user account.
The present technology improves upon the above-described shortcomings of the present technology. For example the present technology can quickly determine members of a user account's social graph that have similar taste in media items to the profile of the user account and limit media items suggestions to only members of the social graph that have the most similar taste in media items. The present technology also recognizes and accounts for the fact that a person's taste in media items changes over time so determining members of the social graph that have the most similar taste in media items should be re-determined frequently. The present technology also filters media item suggestions to remove media items that are clearly not of interest to a user profile. Additionally, the present technology also recognizes that media items that have been experienced most recently are often of the most relevance and therefore, the present technology can prioritize such media items when making social recommendations. Even in view of the fact that the present technology takes into account these complicated factors, the present technology can provide social recommendations in an online process occurring in near real-time.
The socially influenced recommendations are derived from playback history of relevant members of users 104. Users 104 is representative of a population of users of media service 100.
Media service 100 stores user profile information in user profile database 110 for user 102 and users 104. Information stored in user profile database 110 can include user account identifiers, demographic information, payment information, records of media items purchased by or stored in user accounts, etc. in some embodiments user profile database 110 can also include additional information such as declared preferences (perhaps received during user account on boarding), followed artists (some embodiments users can select artists to follow and view public postings by those artists), and any other information.
Media service 100 also stores media playback data and interaction data in media database 112. For example media database 112 can include a listing of all media items played back by user accounts of media service 100. The listing of all media items played back by user accounts of media service 100 can include media items played from a user's media library, or media items played from a radio service or on-demand subscription service. The listing of all media items played back can also be associated with interaction data such as skip information, like, and dislikes of media item.
While user profile database 110 and media database 112 have been described with respect to specific types of data stored within the respective databases, persons of ordinary skill in the art will appreciate that all information described herein could be scored in one database or could be stored in more databases than those described or depicted in
The information stored in user profile database 110 and media database 112 can be used to determine user taste profiles and user item engagement profiles for user account 102 and each of user accounts 104 of media service 100. These profiles are then used to determine a representation for each of users 104 and user 102 which can be used to determine similarity representations between multiple users 104 and user 102, which then can be used to generate candidate media items, such as albums, playlists, and other collections of media items to be recommended to user 102 by social recommendations service 106. In some embodiments, candidate media items can include songs.
As addressed above, an aspect of the present technology is to be able to identify a subset of users 104 that have a similar representation to that of user 102. User representations can consist of both a summary of a user's taste profile and a summary of the user's recent playback history or engagement with media service 100. Broadly,
User's taste profiles can be determined by taste profile service 114 which can retrieve information stored in association with respective users' accounts and can analyze (202) the retrieved information.
In general, the retrieved taste profile information is information that is relevant to users taste in media, and while in some cases it might be derived from specific played back media items, it does not reflect the actual media items played back by the user account (this data is used to determine the engagement profile). An example of the retrieved taste profile information is illustrated in
Weighting factors, and relatedly, which data sources are considered most important can be chosen based on which data is the most statically significant (e.g., best sample size, regression analysis or machine learning points to a data source being a good indicator of media item taste).
While
While
Additionally while
Returning to
For each of the media items in the playback history, taste profile service can identify an associated artist and genre, etc. Both of these taste attributes can be normalized, aggregated, and then combined with other taste attributes.
This can be done, for example, by identifying a specific artist associated with a media item in the playback history of the user account, and determining (203) the percentage of all media items played back in the playback history of the user account that belong to that specific artist to achieve a normalized value for the user account's preference for that specific artist for that source of data.
In some embodiments, taste profile service can also determine an adventurousness ranking for a user account. This can determined from a declared interest of the user account, or by analyzing the playback history of a user account to determine a ratio of media items that are played back outside of a user's favorite genres, or how often a user account plays back media items that it has not played back before.
In some embodiments, the age of the data can be accounted for through a decay factor. For example, data derived for a media item that was played back a month ago can be weighted by a decay factor to reduce the relevance of the aged data on the calculated taste profile.
Other sources of data can also be analyzed (such as likes and dislikes, and other sources illustrated in
This technique can be repeated for each artist found in the user account's playback history.
The specific artist can be mapped (206) to a random number or identifier. The identifier or number can be randomly chosen, however once chosen, the identifier or number should remain consistent for the same artist. In some embodiments the mapping can be performed using locality sensitive hashing techniques. In such embodiments each media item belonging to a particular artist can be hashed to a number, identifier, or bucket.
The identifier or number for that specific artist can by multiplied by the score for that user's account preference for that artist to yield a compact score for that specific artist that is more easily and meaningfully aggregable. The compact scores for all artists are aggregated.
The technique described above is also utilized to determine a compact score for genre, or any other dimension in the data that is of interest.
After aggregated compact scores are determined for artists, genres and any other dimension in the data that is of interest, taste profile service 114 can aggregate (208) all the scores or numbers to yield (209) a compact taste profile representation. The taste profile representation can be further hashed or compressed using a thresholding operation to determine (209) a compact taste profile representation.
In addition to determining a taste profile for each user of media service 100, media service 100 also determines user engagement profiles for each user of media service 100. The user engagement profile is descriptive of media items that users of media service 100 have played back including a number of times playing back a media item, the time spent with that media item, and when the media item is a collection, the percentage of that collection that has been played back.
Both engagement services 116 and 117 are illustrated having three collections of media items which are three different time buckets (T1, T2, T3). One time bucket for media items just played back, for example within the last five or 10 minutes. Another time bucket is for media items played back recently, for example within the last few weeks. The last time bucket is historical for media items played back further in the past. These time buckets can be used to weight data derived from playback histories to give less influence to older data. For example, the relevance of the data stored in historical bucket can be associated with an exponential decay function that would give data derived from media items stored in the historical bucket ½ the weight after data is aged by two months, to give data aged four months ¼ of the weight, to give data aged 6 months ⅛ the weight and so forth.
User's engagement profiles can be determined by engagement services 116, 117 which can retrieve information stored in association with respective users' accounts and can analyze (
For each media item, engagement service 116, 117 can calculate (
The statistics for each category are then weighted (216) to yield an engagement metric for a particular media item. The user engagement score for a media item can be determined by the equation listed below where UE(u,i) is the engagement score for a user for a given media item, NormalPlays(u,i) is the normalized value for number of plays of the media item by the user, NormalTS(u,i) is the normalized value for time spent listening to the media item by the user, and % TracksPlayed(u,i) is the percentage of tracks played by the user for the media item. W1, W1, and W3 are weighting factors that are heuristically chosen and can be adjusted or learned over time (through machine learning, or trial and error). WAge is a weighting factor that will reduce the value of data for media items that are older as addressed above.
UE(u,i)=(W1*NormalPlays(u,i)+W2*NormalTS(u,i)+W3*% TracksPlayed(u,i))*WAge
While three statistics are mentioned above, persons of ordinary skill in the art will recognize that more or less statistics can be used. For example, user likes or dislikes, skips, purchases, etc. can be used and weighted as described above.
Persons of ordinary skill in the art will recognize that the equation given above is merely illustrative of one way of calculating a user engagement score for a user's engagement with media item but that many other methods are possible and the present technology is not limited to the equation given above.
Once an engagement score has been determined for each media item that the user has engaged with as recorded in the media database in association with their user account, the total scores can be aggregated to determine (218) a compact engagement score for the user.
One way of aggregating the engagement scores for each media item that the user has engaged with it to hash the media item to a hash value according to a locality sensitive hashing function, and the hashed value can be multiplied by the score given to the media item. The multiplied hash values for each media item can then be summed or otherwise aggregated (218) to yield the compact engagement score.
User representation service 118 can receive the taste profile representation from taste profile service 114 and engagement profile representation from engagement services 116, 117 for each user of media service 100 and store these representations in user representation database 120. In some embodiments, user representation service 118 can also determine (220) compact representations for each user that can also be stored in user representation database 120. In some embodiments the compact representations are determined (220) by hashing an aggregated value resulting from combining the taste profile representation and engagement profile representation for each user. Compact representation is a single, easily comparable value for each user of media service 100.
In some embodiments the compact representations are determined (220) by combining weighted values for the taste profile and the engagement profile. For example the compact representation can be represented by the equation below where D=compact representation, P1=Taste Profile, P2=Engagement Profile, W1=first weighting factor, and W2=second weighting factor.
D=W
1
P
1
+W
2
P
2
The preceding steps addressed above (202-220) and illustrated in
In some embodiments, the preceding steps addressed above (202-220) and illustrated in
In some embodiments a user device operated by user 102 requests (230) social recommendations from social recommendation service 106.
While in some embodiments, the present technology can be used to compare a first user's compact representation with all other users' compact representations to identify users that have similar representations (and therefore have similar tastes and engagement with media items), in some embodiments, comparisons are limited to within a first user's (e.g., user 102) social network.
Description of
Social recommendations service 106 can perform online processing to provide social recommendations to user account 102. Social recommendations service 106 can request social connections for user 102 from social graph database 130. Upon receiving social connections for user 102 from social graph database 130, social recommendations service can request compact representations for the social connections of user 102, and the compact representation for user account 102. Social recommendations service 106 can compare the compact representation of user account 102 to compact representations for the social connections of user 102 and identify (232) accounts that have a compact representation that are similar to the compact representation of user 102.
In some embodiments, compact representations that are similar are determined by taking the top “n” user accounts that are most similar to the compact representation of user account 102. In some embodiments, compact representations that are similar are determined by taking compact representations that vary by no more than a threshold parameter (e.g., a distance measurement, number of bits, or any other measurement of similarity between two values).
In some embodiments, the compact representations can be stored in user representation database 120 along with social graph information or in accordance with sets of social media groupings.
In some embodiments, rather than identifying (232) accounts at the time media item recommendations are requested by user account 102, it may also be possible to create an inverted index listing all users that have a similar representation for quick look up. The inverted index can be created in advance of request by user account 102 so that when social recommendations service 106 attempts to identify user accounts in user account's 102 social graph, the information can be looked up from the inverted index rather than determined through comparisons of compact representations.
Once accounts have been identified (232) that have a compact representation that is similar to the compact representation of user account 102, social recommendations service 106 can retrieve (234) media items that have been played back by the accounts having a compact representation that is similar to the user account 102. As illustrated in
After retrieving (234) the media items, media items that are not relevant to user account 102 are discarded. Media items can be considered not relevant when they belong to a genre that is not represented in the user's taste profile. However, in some embodiments, a criterion to determine whether media items are relevant to user account 102 can be adjusted when too few media items are retrieved (234).
Additionally, in some embodiments, a criterion to determine whether media items are relevant to user account 102 can be relaxed to introduce user account 102 to media items outside of the user account 102's normal preferences. For example, to avoid recommending media items that user account 102 is already playing back regularly (and thus influencing both user account 102's taste profile and engagement profile), social recommendations service 106 can retrieve media items from social connections with a compact representation that is less similar to the compact representation of user account 102 to result in a more diverse selection of media item candidates.
In some embodiments, a user account 102 might have too few user accounts in its social graph to make meaningful recommendations, or user account 102 has enough user accounts in its social graph, but the recommendations are too homogeneous, or there are not enough recommendations. In such embodiments, social recommendations service 106 can compare user account's 102 compact representation to all user accounts' compact representations to recommend other user accounts to follow and add to user account's 102 social graph. In some embodiments, social recommendations service 106 can recommend second degree connections (friends of friends) of user account to follow and add to user account's 102 social graph. In some embodiments, social recommendations service 106 can recommend third or further degree connections (friends of friends of friends) of user account to follow and add to user account's 102 social graph. In some embodiments, rather than recommending additional connections of user accounts to follow, social recommendations service 106 can pull media item recommendations from the additional user accounts without showing the additional connections of user account 102 to the user associated with user account 102.
In such embodiments where user account 102 either has too few connections, or its connections yield to few media item recommendations, social recommendations service 106 can search user representation database 120 to find additional media items to recommend to user account 102. However since user representation database 120 can be quite large and have millions of compact representations, there needs to be an efficient way to search user representation database 120.
As illustrated in
API layer 150 can include logic to receive the compact representation for user account 102 and can create individual queries (354) of key-value store 152 for similar compact representations to the compact representation for user account 102. In some embodiments this can be achieved by identifying specific compact representations to search for that are similar to the compact representation for user account 102. Since, in some embodiments, compact representations can be stored as a 32-bit number as illustrated in
Continuing with the example addressed above, if social recommendations service 106 was not able to identify enough media item recommendations for user 102, (for example, for user F who has no other users with the same taste profile), the process of
For each compact representation found in user representation database 120 that matches the individual queries, user representation database 120 can return (358) the values associated with those user representations, namely the user IDs and associated metadata that corresponds to the keys used in the individual queries. In
The metadata associated with each user in user representation database can include a minimal amount of information such as the respective user's top three favorite genres and top three favorite artists. This metadata can be used by social recommendations service 106, to help it rank media item candidates as addressed below. The metadata can be important because, while two representations that are the same except for a one bit difference should reflect similar tastes, some small differences could be substantial to the listener. The metadata can be used to find key similarities between two users, or between the user account and the media item candidates so that the media items recommended to user account 102 are more likely to be good recommendations for user account 102,
In some embodiments, the users identified as a result of the method illustrated in
In some embodiments, media service 100 can retrieve recommendations from second degree connections (friends of friends) of user account by looking up a social graph for a friend in user 102's social graph in social graph database 130. Media items that have been that have been played back by the second degree connection can be added to media item recommendation candidates to be recommended to user account 102.
Whether media item recommendation candidates are identified from user accounts in user 102's social network having a similar taste profile as user 102, or media item recommendation candidates are retrieved from a broader collection of users (further degrees of separation in user 102's social graph, or users outside of user 102's social graph but that have a similar compact representation as user 102), the media item candidates can be ranked (236) according to a ranking criterion. Ranking criteria can include:
After the media items have been ranked social recommendations service 106 can present (238) the retrieved media items in a user interface on a client device associated with the user account 102 ordered from highest rank to lowest. In some embodiments, only a certain number of highest ranked media items will be presented (238) in the user interface.
In some embodiments, the client device associated with the user account 102 can request badges from badge service 134 representing users of user account 102's social network that have recently played back the media items presented (238) to be displayed over the representations of the presented media items. For example
In some embodiments, the media items having the highest ranks can be combined into a playlist and presented as a media item in the user interface on the client device associated with the user account 102.
Based on the similarity values, the social graph visualization service can cluster (418) the identified accounts and the first account according to the similarity values and present (420) the visualization of the of the first user and the identified accounts that have a social connection arranged according to their similarity values.
The matrix shows that Asher 502 has a relatively similar representation as to Madelyn 504, wherein the matrix reflects a distance representation of 5.7 between the two user accounts. This is in contrast with Anthony 506 and Kaylee 508 for which the matrix reflects a distance representation of 18 and 17 respectively between these user accounts and Asher 502.
These distance representations are further reflected in
In the embodiments addressed above with respect to
In some embodiments computing system 600 is a distributed system in which the functions described in this disclosure can be distributed within a datacenter, multiple datacenters, a peer network, etc. In some embodiments, one or more of the described system components represents many such components each performing some or all of the function for which the component is described. In some embodiments, the components can be physical or virtual devices.
Example system 600 includes at least one processing unit (CPU or processor) 610 and connection 605 that couples various system components including system memory 615, such as read only memory (ROM) and random access memory (RAM) to processor 610. Computing system 600 can include a cache of high-speed memory connected directly with, in close proximity to, or integrated as part of processor 610.
Processor 610 can include any general purpose processor and a hardware service or software service, such as services 632, 634, and 636 stored in storage device 630, configured to control processor 610 as well as a special-purpose processor where software instructions are incorporated into the actual processor design. Processor 610 may essentially be a completely self-contained computing system, containing multiple cores or processors, a bus, memory controller, cache, etc. A multi-core processor may be symmetric or asymmetric.
To enable user interaction, computing system 600 includes an input device 645, which can represent any number of input mechanisms, such as a microphone for speech, a touch-sensitive screen for gesture or graphical input, keyboard, mouse, motion input, speech, etc. Computing system 600 can also include output device 635, which can be one or more of a number of output mechanisms known to those of skill in the art. In some instances, multimodal systems can enable a user to provide multiple types of input/output to communicate with computing system 600. Computing system 600 can include communications interface 640, which can generally govern and manage the user input and system output. There is no restriction on operating on any particular hardware arrangement and therefore the basic features here may easily be substituted for improved hardware or firmware arrangements as they are developed.
Storage device 630 can be a non-volatile memory device and can be a hard disk or other types of computer readable media which can store data that are accessible by a computer, such as magnetic cassettes, flash memory cards, solid state memory devices, digital versatile disks, cartridges, random access memories (RAMs), read only memory (ROM), and/or some combination of these devices.
The storage device 630 can include software services, servers, services, etc., that when the code that defines such software is executed by the processor 610, it causes the system to perform a function. In some embodiments, a hardware service that performs a particular function can include the software component stored in a computer-readable medium in connection with the necessary hardware components, such as processor 610, connection 605, output device 635, etc., to carry out the function.
For clarity of explanation, in some instances the present technology may be presented as including individual functional blocks including functional blocks comprising devices, device components, steps or routines in a method embodied in software, or combinations of hardware and software.
Any of the steps, operations, functions, or processes described herein may be performed or implemented by a combination of hardware and software services or services, alone or in combination with other devices. In some embodiments, a service can be software that resides in memory of a client device and/or one or more servers of a content management system and perform one or more functions when a processor executes the software associated with the service. In some embodiments, a service is a program, or a collection of programs that carry out a specific function. In some embodiments, a service can be considered a server. The memory can be a non-transitory computer-readable medium.
In some embodiments the computer-readable storage devices, mediums, and memories can include a cable or wireless signal containing a bit stream and the like. However, when mentioned, non-transitory computer-readable storage media expressly exclude media such as energy, carrier signals, electromagnetic waves, and signals per se.
Methods according to the above-described examples can be implemented using computer-executable instructions that are stored or otherwise available from computer readable media. Such instructions can comprise, for example, instructions and data which cause or otherwise configure a general purpose computer, special purpose computer, or special purpose processing device to perform a certain function or group of functions. Portions of computer resources used can be accessible over a network. The computer executable instructions may be, for example, binaries, intermediate format instructions such as assembly language, firmware, or source code. Examples of computer-readable media that may be used to store instructions, information used, and/or information created during methods according to described examples include magnetic or optical disks, solid state memory devices, flash memory, USB devices provided with non-volatile memory, networked storage devices, and so on.
Devices implementing methods according to these disclosures can comprise hardware, firmware and/or software, and can take any of a variety of form factors. Typical examples of such form factors include servers, laptops, smart phones, small form factor personal computers, personal digital assistants, and so on. Functionality described herein also can be embodied in peripherals or add-in cards. Such functionality can also be implemented on a circuit board among different chips or different processes executing in a single device, by way of further example.
The instructions, media for conveying such instructions, computing resources for executing them, and other structures for supporting such computing resources are means for providing the functions described in these disclosures.
Although a variety of examples and other information was used to explain aspects within the scope of the appended claims, no limitation of the claims should be implied based on particular features or arrangements in such examples, as one of ordinary skill would be able to use these examples to derive a wide variety of implementations. Further and although some subject matter may have been described in language specific to examples of structural features and/or method steps, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to these described features or acts. For example, such functionality can be distributed differently or performed in components other than those identified herein. Rather, the described features and steps are disclosed as examples of components of systems and methods within the scope of the appended claims.
This application claims priority to U.S. provisional application No. 62/514,228, filed on Jun. 2, 2017, which is expressly incorporated by reference herein in its entirety.
Number | Date | Country | |
---|---|---|---|
62514228 | Jun 2017 | US |