Patent Application
20180253696
This application relates to the technical fields of software and/or hardware technology and, in one example embodiment, to system and method to generate job recommendations using co-viewership scores in the context of an on-line social network system.
An on-line social network may be viewed as a platform to connect people and share information in virtual space. An on-line social network may be a web-based platform, such as, e.g., a social networking web site, and may be accessed by a use via a web browser or via a mobile application provided on a mobile phone, a tablet, etc. An on-line social network may be a business-focused social network that is designed specifically for the business community, where registered members establish and document networks of people they know and trust professionally. Each registered member may be represented by a member profile. A member profile may be represented by one or more web pages, or a structured representation of the member's information in XML (Extensible Markup Language), JSON (JavaScript Object Notation) or similar format. A member's profile web page of a social networking web site may emphasize employment history and education of the associated member. An on-line social network may store include one or more components for matching member profiles with those job postings that may be of interest to the associated member.
Embodiments of the present invention are illustrated by way of example and not limitation in the figures of the accompanying drawings, in which like reference numbers indicate similar elements and in which:
A method and system to generate recommendations using co-viewership scores in an on-line social network system is described. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of an embodiment of the present invention. It will be evident, however, to one skilled in the art that the present invention may be practiced without these specific details.
As used herein, the term “or” may be construed in either an inclusive or exclusive sense. Similarly, the term “exemplary” is merely to mean an example of something or an exemplar and not necessarily a preferred or ideal means of accomplishing a goal. Additionally, although various exemplary embodiments discussed below may utilize Java-based servers and related environments, the embodiments are given merely for clarity in disclosure. Thus, any type of server environment, including various system architectures, may employ various embodiments of the application-centric resources system and method described herein and is considered as being within a scope of the present invention.
For the purposes of this description the phrases “an on-line social networking application,” “an on-line social network system,” and “an on-line social network service” may be referred to as and used interchangeably with the phrase “an on-line social network” or merely “a social network.” It will also be noted that an on-line social network may be any type of an on-line social network, such as, e.g., a professional network, an interest-based network, or any on-line networking system that permits users to join as registered members. For the purposes of this description, registered members of an on-line social network may be referred to as simply members.
Each member of an on-line social network is represented by a member profile (also referred to as a profile of a member or simply a profile). A member profile may include or be associated with links that indicate the member's connection to other members of the social network. A member profile may also include or be associated with comments or recommendations from other members of the on-line social network, with links to other network resources, such as, e.g., publications, etc. The profile information of a social network member may include various information such as, e.g., the name of a member, current and previous geographic location of a member, current and previous employment information of a member, information related to education of a member, etc. The on-line social network system also maintains information about various companies, as well as so-called job postings. A job posting, also referred to as merely “job” for the purposes of this description, is an electronically stored entity that includes information that an employer may post with respect to a job opening.
The information in a job posting may include, e.g., industry, company, job position, required and/or desirable skills, geographic location of the job, etc. Member profiles and job postings are represented in the on-line social network system by feature vectors. The features in the feature vectors may represent, e.g., a job industry, a professional field, a job title, a company name, professional seniority, geographic location, etc.
The on-line social network system includes a recommendation system configured to select one or more job postings for presentation to a member based on criteria that indicates that a particular job posting is likely to be of interest to the member. The criteria that indicates that a particular job posting is likely to be of interest to the member, in one embodiment, is associated with a relevance value.
When a new login session is initiated for a member in the on-line social network system, the recommendation system generates respective relevance values for pairs comprising a member profile representing the member in the on-line social network system and a job posting. The relevance values are generated, in one embodiment, using a statistical model (referred to as the relevance model for the purposes of this description). A relevance value reflects the likelihood that a member represented by the member profile applies for a job represented by the job posting. Those job postings, for which their respective relevance values for a particular member profile are equal to or greater than a predetermined threshold value, are presented to that particular member, e.g., on the news feed page of the member or on some other page provided by the on-line social networking system.
In some embodiments, the recommendation system is configured to use signals indicating that multiple members have viewed the same job posting to determine whether a particular job posting should be recommended to a certain member. The recommendation system includes or in cooperation with an events monitor, which is configured to monitor, in the on-line social network system, job view events. A job view event, for the purposes of this description, indicates a viewing of a job posting by a member represented by a member profile. As described above, recommended job postings may be presented to a member via various user interfaces generated by the on-line social network system, such as, e.g., via the news feed page generated for the member. An example of a job view event is activation of a web link that references a job posting. Another example of a job view event is activation of a visual control (e.g., a “view” button) presented next to the reference a job posting. The job view events are used to generate, for a job posting, viewership data. Viewership data generated for a job posting comprises identifications of profiles in the on-line social network system that represent respective members who viewed the job posting via a user interface provided by the an on-line social network system. The viewership data may also include a time stamp indicating when a particular member viewed a particular job posting.
Viewership data generated for respective of job postings is stored in a database maintained in the on-line social network system and is used by the recommendation engine to determine that a job posting should be selected for presentation to a member as a recommended job. In one embodiment, subsequent to a request to generate job recommendations for a member represented by a subject member profile, the recommendation engine first determines a set of job postings that have been previously recommended to that member based on the respective relevance scores generated for those jobs. For these previously recommended job postings, the recommendation engine accesses their respective viewership data and identifies a set of co-viewed job postings—those job postings that were viewed by at least one member who also viewed a job from the set of previously recommended jobs.
For each job from the set of co-viewed jobs, the recommendation engine generates a co-viewership score with respect to the subject member profile. Based on these co-viewership scores, the recommendation engine selects, from the set of co-viewed jobs, one or more additional job postings to be included in the set of recommendations for presentation to the member.
A co-viewership score for a co-viewed job with respect to the subject member profile may be derived based on the number of member profiles referenced in respective viewership data of both the co-viewed job and one of the previously recommended job postings.
In some embodiments, the co-viewership score for a pair comprising a subject member profile and a particular job posting is determined by applying a graph analysis algorithm to a so-called co-viewership graph. The co-viewership graph is a tripartite graph. The first set of nodes in the co-viewership graph comprises nodes representing member profiles in the on-line social network system. The second set of nodes comprises nodes representing those job postings recommended to at least one member represented by a member profile represented by a node from the first set of nodes. The third set of nodes represents job postings associated with respective viewership data that includes a reference to at least one member profile that is also referenced in viewership data associated with a job posting represented by a node in the second set of nodes. An edge between a node from the first set of nodes and a node from the second set of nodes indicates that the job represented by the node in the second set of nodes was previously recommended to the member represented by the node from the first set of nodes. A weight assigned to an edge between a node, u from the second set of nodes and a node, v from the third set of nodes is calculated based on the co-viewership of job postings u and v. This weight could be obtained as the ratio of a numerator term denoting the co-views of jobs u and v to a denominator normalization term. The numerator term can be obtained in different ways, for example as (1) the number of times the job v was viewed by a member after viewing job u, or (2) the number of times jobs u and v were viewed by the same member within a specified time window. The denominator normalization term could be computed, for instance, as (1) the number of times job u was viewed, (2) the number of times job v was viewed, (3) the sum of these two counts, or (4) the maximum of these two counts.
Co-viewership scores generated for job postings with respect to a subject profile representing a member in the on-line social network system can be used beneficially to recommend to a member additional jobs, specifically jobs that have their respective job poster values greater than or equal to an underperformance threshold value. A job poster value (JPV) represents a value that is still owed to the job poster (e.g., to the company that posted the job with the on-line social network service) at a certain point in time. A job posting entity is typically charged a fee for submitting a job posting to the on-line social network system, which creates an expectation on the part of the job posting entity that the job posting would receive a certain amount of value for the fee that was charged. This value is a job poster value, and it may be affected by the level of interest in the job expressed by qualified members of the on-line social network system. The level of interest may be gauged by the number of applications received with respect to the job posting, the number of views and the number of impressions. A job posting that has received fewer applications and views after a certain period since it was posted with the on-line social network service may be described as having a higher job poster value. A job that has received a greater number of applications from qualified candidates and a greater number of views after a certain period since it was posted with the on-line social network service may be described as having a lower job poster value.
Returning to the discussion of the co-viewership graph, in one embodiment, the job postings selected to be represented by respective nodes in the third set of nodes of the co-viewership graph are only those job postings that have their respective job poster values less than or equal to an underperformance threshold value. Examples of a graph analysis algorithm that the recommendation system can use to generate a co-viewership score using the co-viewership graph are described further below.
For example, a co-viewership score for an additional job posting with respect to a subject member profile may be calculated, using the co-viewership graph described above, as probability of a random walk starting at a node (from the first set of nodes) representing the subject member profile and reaching the node representing the additional job posting (from the third set of nodes). In other words, what is being considered is a Markov chain with nodes representing underperforming job postings in the third set of nodes as the absorbing states. The absorbing states are ranked based on the probability of reaching that state from the node representing the subject member profile. Another example graph analysis algorithm that the recommendation system can use to generate a co-viewership score is a decreasing function (such as reciprocal) of the graph commute time between a node (from the first set of nodes) representing the subject member profile and the node representing the additional job posting (from the third set of nodes). Yet another example graph analysis algorithm involves computing the number of length 2 paths from a node representing the subject member profile and the node representing the additional job posting. Any of these graph analysis algorithms can utilize edge weights for deriving the co-viewership scores.
A co-viewership graph is generated and stored in the on-line social network system. It is updated based on a pre-determined schedule and/or, e.g., as the status of one or more new job postings are identified as underperforming based on their respective JPVs or as one or more new job postings are no longer identified as underperforming based on their respective JPVs.
In one embodiment, the recommendation system selects some of the job postings to be included in the set of job recommendations based on their respective relevance values, and some additional jobs (from those that have been identified as underperforming based on their respective JPVs) based on their respective co-viewership scores. The recommendation system, in one embodiment, reserves a certain number of slots in the set of job recommendations specifically for additional underperforming jobs and fills these slots with job postings that have the highest job posting similarity values. In some embodiments, the recommendation system generates, for each candidate job posting, a combined score based on its respective relevance score its co-viewership score. The recommendation system then uses these combined scores to rank the candidate job postings and includes the top-ranking ones into the set of job recommendations. A combined score for a job posting j with respect to a member profile m can be generated as C(r(m,j), s(m,j)), where r(m,j) is the relevance score and s(m,j) is the co-viewership score. C(,) could be a monotonically increasing function in two variables such as C(x,y)=x(1+y); C(x,y)=x.exp(y), etc.
The job posting similarity values, in some embodiments, are used as features to learn coefficients for a machine learning algorithm such as, e.g., logistic regression, where the associated model is trained on a ground truth dataset of (member profile, underperforming job posting) tuples.
An example recommendation system may be implemented in the context of a network environment 100 illustrated in
The client systems 110 and 120 may be capable of accessing the server system 140 via a communications network 130, utilizing, e.g., a browser application 112 executing on the client system 110, or a mobile application executing on the client system 120. The communications network 130 may be a public network (e.g., the Internet, a mobile communication network, or any other network capable of communicating digital data). As shown in
The events monitor 210 is configured to monitor and collect job view events in the on-line social network system 142 of
The viewership data generator 220 is configured to determine, based on the job view events monitored and collected by the events monitor 210, viewership data for a particular job posting. The viewership data associated with a job posting comprises a set of references to profiles in the on-line social network system 142 that represent respective members who viewed that job posting via a user interface provided by the on-line social network system 142.
The co-viewership detector 230 is configured to examine viewership data associated with a job posting in order to determine that the job posting was viewed by a member who also viewed a so called recommended job posting, which is a job posting that was previously recommended, to a specific member represented by a subject member profile, based on a relevance score generated for the recommended job posting with respect to the subject member profile that represented the specific member in the on-line social network system 142. As explained above, the relevance score reflects a likelihood that the member applies for a job represented by the recommended job posting. The co-viewership detector 230 may be utilized in the process of generating the co-viewership graphs, e.g., to determine whether a job posting should be represented by a node in the co-viewership graph.
The co-viewership score generator 240 is configured to generate a co-viewership score for a pair comprising a subject member profile and a particular job posting. A co-viewership score for a pair comprising a subject member profile and a particular job posting, in one embodiment, is a number of member profiles that are referenced in viewership data associated the particular job posting and are also referenced in viewership data associated with a job posting that was previously recommended to the member represented by the subject member profile.
In some embodiment, the co-viewership score generator 240 is configured to access a co-viewership graph, which was described above and which may be stored in the database 150 of
The recommendation generator 250 is configured to select a particular job posting to be included in a set of recommendations for a subject member profile, based on the co-viewership score generated for that subject member profile and that particular job posting, for presentation to the member represented by the subject member profile. The recommendation generator 250 may performs operations to select the particular job posting to be included in the set of recommendations in response to detecting a new login session commenced with respect to the member represented by the subject member profile.
The presentation module 260 is configured to cause presentation, on a display device, references to respective job postings selected by the recommendation generator 250 for presentation to a member. Some operations performed by the system 200 may be described with reference to
As shown in
The various operations of example methods described herein may be performed, at least partially, by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Whether temporarily or permanently configured, such processors may constitute processor-implemented modules that operate to perform one or more operations or functions. The modules referred to herein may, in some example embodiments, comprise processor-implemented modules.
Similarly, the methods described herein may be at least partially processor-implemented. For example, at least some of the operations of a method may be performed by one or more processors or processor-implemented modules. The performance of certain of the operations may be distributed among the one or more processors, not only residing within a single machine, but deployed across a number of machines. In some example embodiments, the processor or processors may be located in a single location (e.g., within a home environment, an office environment or as a server farm), while in other embodiments the processors may be distributed across a number of locations.
The example computer system 400 includes a processor 402 (e.g., a central processing unit (CPU), a graphics processing unit (GPU) or both), a main memory 404 and a static memory 406, which communicate with each other via a bus 404. The computer system 400 may further include a video display unit 410 (e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)). The computer system 400 also includes an alpha-numeric input device 412 (e.g., a keyboard), a user interface (UI) navigation device 414 (e.g., a cursor control device), a disk drive unit 416, a signal generation device 418 (e.g., a speaker) and a network interface device 420.
The disk drive unit 416 includes a machine-readable medium 422 on which is stored one or more sets of instructions and data structures (e.g., software 424) embodying or utilized by any one or more of the methodologies or functions described herein. The software 424 may also reside, completely or at least partially, within the main memory 404 and/or within the processor 402 during execution thereof by the computer system 400, with the main memory 404 and the processor 402 also constituting machine-readable media.
The software 424 may further be transmitted or received over a network 426 via the network interface device 420 utilizing any one of a number of well-known transfer protocols (e.g., Hyper Text Transfer Protocol (HTTP)).
While the machine-readable medium 422 is shown in an example embodiment to be a single medium, the term “machine-readable medium” should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, and/or associated caches and servers) that store the one or more sets of instructions. The term “machine-readable medium” shall also be taken to include any medium that is capable of storing and encoding a set of instructions for execution by the machine and that cause the machine to perform any one or more of the methodologies of embodiments of the present invention, or that is capable of storing and encoding data structures utilized by or associated with such a set of instructions. The term “machine-readable medium” shall accordingly be taken to include, but not be limited to, solid-state memories, optical and magnetic media. Such media may also include, without limitation, hard disks, floppy disks, flash memory cards, digital video disks, random access memory (RAMs), read only memory (ROMs), and the like.
The embodiments described herein may be implemented in an operating environment comprising software installed on a computer, in hardware, or in a combination of software and hardware. Such embodiments of the inventive subject matter may be referred to herein, individually or collectively, by the term “invention” merely for convenience and without intending to voluntarily limit the scope of this application to any single invention or inventive concept if more than one is, in fact, disclosed.
Certain embodiments are described herein as including logic or a number of components, modules, or mechanisms. Modules may constitute either software modules (e.g., code embodied (1) on a non-transitory machine-readable medium or (2) in a transmission signal) or hardware-implemented modules. A hardware-implemented module is tangible unit capable of performing certain operations and may be configured or arranged in a certain manner. In example embodiments, one or more computer systems (e.g., a standalone, client or server computer system) or one or more processors may be configured by software (e.g., an application or application portion) as a hardware-implemented module that operates to perform certain operations as described herein.
In various embodiments, a hardware-implemented module may be implemented mechanically or electronically. For example, a hardware-implemented module may comprise dedicated circuitry or logic that is permanently configured (e.g., as a special-purpose processor, such as a field programmable gate array (FPGA) or an application-specific integrated circuit (ASIC)) to perform certain operations. A hardware-implemented module may also comprise programmable logic or circuitry (e.g., as encompassed within a general-purpose processor or other programmable processor) that is temporarily configured by software to perform certain operations. It will be appreciated that the decision to implement a hardware-implemented module mechanically, in dedicated and permanently configured circuitry, or in temporarily configured circuitry (e.g., configured by software) may be driven by cost and time considerations.
Accordingly, the term “hardware-implemented module” should be understood to encompass a tangible entity, be that an entity that is physically constructed, permanently configured (e.g., hardwired) or temporarily or transitorily configured (e.g., programmed) to operate in a certain manner and/or to perform certain operations described herein. Considering embodiments in which hardware-implemented modules are temporarily configured (e.g., programmed), each of the hardware-implemented modules need not be configured or instantiated at any one instance in time. For example, where the hardware-implemented modules comprise a general-purpose processor configured using software, the general-purpose processor may be configured as respective different hardware-implemented modules at different times. Software may accordingly configure a processor, for example, to constitute a particular hardware-implemented module at one instance of time and to constitute a different hardware-implemented module at a different instance of time.
Hardware-implemented modules can provide information to, and receive information from, other hardware-implemented modules. Accordingly, the described hardware-implemented modules may be regarded as being communicatively coupled. Where multiple of such hardware-implemented modules exist contemporaneously, communications may be achieved through signal transmission (e.g., over appropriate circuits and buses) that connect the hardware-implemented modules. In embodiments in which multiple hardware-implemented modules are configured or instantiated at different times, communications between such hardware-implemented modules may be achieved, for example, through the storage and retrieval of information in memory structures to which the multiple hardware-implemented modules have access. For example, one hardware-implemented module may perform an operation, and store the output of that operation in a memory device to which it is communicatively coupled. A further hardware-implemented module may then, at a later time, access the memory device to retrieve and process the stored output. Hardware-implemented modules may also initiate communications with input or output devices, and can operate on a resource (e.g., a collection of information).
The various operations of example methods described herein may be performed, at least partially, by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Whether temporarily or permanently configured, such processors may constitute processor-implemented modules that operate to perform one or more operations or functions. The modules referred to herein may, in some example embodiments, comprise processor-implemented modules.
Similarly, the methods described herein may be at least partially processor-implemented. For example, at least some of the operations of a method may be performed by one or processors or processor-implemented modules. The performance of certain of the operations may be distributed among the one or more processors, not only residing within a single machine, but deployed across a number of machines. In some example embodiments, the processor or processors may be located in a single location (e.g., within a home environment, an office environment or as a server farm), while in other embodiments the processors may be distributed across a number of locations.
The one or more processors may also operate to support performance of the relevant operations in a “cloud computing” environment or as a “software as a service” (SaaS). For example, at least some of the operations may be performed by a group of computers (as examples of machines including processors), these operations being accessible via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., Application Program Interfaces (APIs).)
Thus, a method and system to generate recommendations using co-viewership scores in an on-line social network system has been described. Although embodiments have been described with reference to specific example embodiments, it will be evident that various modifications and changes may be made to these embodiments without departing from the broader scope of the inventive subject matter. Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense.
