1. Field
Disclosed herein are systems and methods for identifying experts with particular talents among participants in an online social network.
2. Related Art
The last few years have seen significant growth in the number of business users of online social networking services like Chatter®, which is offered by salesforce.com, inc. These users may be within the same company or other organization (intra-company) or may span organizations. While business use of social networks increases the opportunity to interact and collaborate with other users, the number, geographic distribution or organizational structure of people participating in the social network may make it difficult to identify an “expert” with particular talents who is available to answer questions. User applied talent-labels or other keyword identifiers can flag a relevant subject matter, but this does not automatically identify users by their degree of talent in a specific area.
It is desirable to provide a tool for identifying experts with particular talents that are accessible within a user's social network.
Disclosed herein are systems and methods for identifying experts with particular talents among participants in an online social network. Identification of experts can take into account multiple factors. One factor that can be applied is self-nomination. Individuals who offer to help others can be expected to offer assistance in areas where they have strong talents. However, self-nomination may be perceived as biased. Another factor that can be applied is endorsements by others. A co-worker making an endorsement can be encouraged to supply narrative that describes experience with the nominee's actual work. Endorsements accompanied by narrative can be given greater weight than simple endorsements without a narrative. Another factor that can be applied is data mining by semantic analysis or other techniques, taking advantage of access to work activity databases that describe a nominee's work activities. These databases may be fielded databases. One or more proficiency scores can be derived from these and other factors. Proficiency scores can be based on individual factors or combinations of factors. The proficiency scores can be used when responding to queries or generally when compiling expertise inventories or talent banks.
The technology disclosed provides more useful identification of subject matter experts. It also provides a quantification of their expertise across particular talents. Instead of rating an expert for their general competence, participants in a social network can identify experts with the particular expertise that they are seeking.
Other aspects and advantages of the present invention can be seen on review of the drawings, the detailed description and the claims, which follow.
Implementations of the technology described are provided with reference to the
The technology disclosed assists with scoring particular talents of participants in an online social network and with identification of experts with those talents. The system and method described categorized talents. The talents can apply to a category of intra-company specific tasks, such as “Apex Code,” “API Verbs,” “database.com,” etc. The talents can apply to a relatively more general category of industry-wide skills, such as “JavaScript”. Additional examples of first and second categories of talents appear in
The talent engine scores expertise per talent-label, rather than per individual. This granular level of scoring helps identify experts with particular talents, as opposed to generally strong performers or well-liked helpers. The scoring incorporates multiple factors that assist in talent evaluation.
A user interacting with the talent engine can search for relevant talent-labels in a variety of ways. Users viewing query results can drill down to see more detailed profiles of individual experts.
An individual profile can list talent-labels and talent-label points for a nominee. In some implementations, the talent-labels could be organized by category, level of expertise, points, alphabetically or in any useful order. Individual profiles can incorporate external biographical data, in addition to talent evaluation data.
During operation, users interact with the talent engine 117 through computing devices 143, 147. The computing devices 143, 147 and the talent engine 117 each include memory for storage of data and software applications, a processor for accessing data and executing application, and components that facilitate communication over the communication network 125. The computing devices 143, 147 execute applications, such as Web browsers (e.g., Web browser 144 executing on computing device 143, and app 148 executing on computing device 147) that allow users to make nominations, to make endorsements and to formulate queries and submit them to the talent engine 117. The talent engine 117 receives talent information and talent queries that may have originated with the computing devices 143, 147 and updates or queries various databases 113, 121, 131. The talent engine 117 processes new nominations and connects nominees with talent-labels 113. The talent engine 117 further accumulates talent-label points 121 specific to talent-labels for the nominees. This includes interacting with users and can include processing data from other subsystems such as the semantic analysis engine 137. Talent engine 117 can be part of an overall system that has access to at least one fielded database that describes work activities 127 of the nominees. For example, in response to a query to identify relevant talent-labels received from computing device 143, the talent engine 117 can access the talent-labels database 113 transmit a talent-labels selection page to be displayed in browser 144 executing on the computing device 143. As another example, in response to a query that specifies selected talent-labels received from computing device 143, the talent engine 117 can access the talent-label points database 121 and transmit a list of experts with their relevant talent-labels and corresponding talent-label points.
In one implementation, the network 125 includes the Internet. The network 125 can also utilize dedicated or private communication links that are not necessarily part of the Internet. In one implementation, the network 125 uses standard communication technologies, protocols and/or inter-process communication technologies.
The client computing device 143, 147 executes an application such as a Web browser 144 or an app 148, which interacts with the talent engine 117. The client computing device 143, 147 may be for example a desktop computer, a laptop, a tablet computer, a mobile phone, or any other type of computing device.
The talent engine 117 can be implemented in hardware, firmware, or software running on hardware. The talent engine 117 is described in more detail below with respect to
The databases 113, 121, 127, 131 can be collectively stored on multiple computers or storage devices. A single, multitenant database that handles multiple object types can store some or all of the internal databases for talent-labels 113, talent-label points 121 and work activities 127.
The talent engine 117 provides graphic user interfaces for display on the client device 143, 147. The graphic user interfaces facilitate submission of nominations, endorsements and queries by the user. Examples of some of these interfaces appear in
The nomination processor 225 accepts nominations and self nominations. The nominees are proposed as experts, typically with a plurality of talent-labels. While it is possible for nominee to have a single talent-label, it is expected that experts will have multiple talent-labels, in a well-organized labeling system. It is expected that experts will have talent-labels in multiple categories, including a first, relatively specific category and a second category that is relatively broader than the first. For instance, the first category may be talents related to company tasks or systems and the second category may be talents applicable across an industry. Additional categories could be provided for grouping talents. Preferably, talents belong to one category or another. In some implementations, talents could belong to more than one category. Nominees are members of a social network or other organization, so the nomination processor can verify the identity of nominees. Nominations can be self nominations or endorsements that apply a talent-label to a nominee that had not previously been applied. The nomination processor 225 can rely on the label processor 235 to process labels the company nominations.
The label processor 235 interacts with the talent-labels database 113. It generates user interfaces that can be provided to user for identification of relevant talent-labels to apply to a nominee. A user can use an interface to search for relevant labels in any of the ways described above. For example, the interface may allow the user to interact with a checkbox list or to enter labels and suggest auto completions, for instance in a drop down menu. The label processor 235 also accepts new labels during nomination and updates the talent-label database 113 accordingly. The label processor 235 also can be used by the endorsement processor 245 to identify relevant talent-labels or to create custom talent-labels.
The endorsement processor 245 interacts with the talent-label points database 121. It generates one or more user interfaces that can be provided to a user to endorse a nominee for specific talent-labels. A user can use an interface to search for relevant labels or to create new labels. The endorsement processor 245 can encourage a co-worker to write a narrative or story that describes the co-worker's experience with the nominee's skills corresponding to a specific talent-label. An endorsement can be accepted without a narrative or with the related narrative. A related narrative can be analyzed semantically or otherwise for connection to the talent-label being applied to the nominee. The user could be prompted to give an improved description in the narrative, based on the analysis, or the system could use the analysis without further interaction with the user. An endorsement with narrative can be given greater weight than an endorsement without a narrative. Endorsements are one of the factors considered by the point accumulation processor 255.
The point accumulation processor 255 accumulates talent-label points 121 for particular nominees. The talent-label points 121 can be accumulated across a variety of factors. For instance, self nominations can be awarded points. Co-worker endorsements can be awarded points and more points awarded for the endorsements accompanied by a narrative. A semantic analysis engine 137 or other analysis engine can communicate results of evaluating work activities 127 to the point accumulation processor 255. The point accumulation processor 255 can maintain details of factors that contribute to talent-label points 121. A running total of talent-label points 121 can be maintained, subtotals of talent-label points by factor can be maintained, or the talent-label points can be calculated on-the-fly. The point accumulation processor can calculate one or more proficiency scores based on the talent-label points.
For particular talent-label 113, the talent-label points 121 can be calculated according to the following formula:
TLP=(a*SN)+(b*END1)+(c*END2)+(d*SEM), where
Useful weights can be identified without undue experimentation. In one implementation, a weight of 20 can be applied to self nomination, a weight of 10 can be applied to endorsements with descriptions, a weight of five can be applied to endorsements without descriptions, and a weight of one or two can be applied to instances of talent application identified by a semantic analysis engine 137 or other analytical engine that evaluates work activities 127 of the nominee. In some categories, weights assigned to factors can diminish with elapsed time. Alternatively, increasing weights can be applied as time progresses, effectively depreciating the value of earlier contributions to talent-label points. Results of point accumulation processing 255 can be used by the query processor 265 to respond to certain types of queries.
The query processor 265 interacts with users to identify relevant talent-labels, to identify experts with accumulated talent-label points for selected talent-labels, to provide detailed information about identified experts, and to facilitate communications with experts. The communications with experts can involve a handoff to another engine such as a chatter engine that is separate from the talent engine 117. The operation of the talent engine 117 can be described or reference to the flowchart in
At step 325, a nomination processor 225 receives nominations. Nominations associate talent-labels with nominees. In subsequent steps, talent-label points will accumulate for the nominee's particular talent-labels. In some implementations, receiving nominations 325 includes processing labels 337. Processing labels can include selecting standard talent-labels, selecting previously entered custom talent-labels, or creating and selecting new custom talent-labels. The talent-labels can be organized by keyword, hierarchy, or both. The talent-labels can further be clustered or associated with one another by a recommendation and/or collaborative filtering engine.
At step 355, the point accumulation processor 255 accumulates points for nominee talent-labels 355. In subsequent steps, talent-label points can be used to respond to queries or to generate talent inventories. In some implementations, accumulating points for nominee talent-labels 355 includes receiving endorsements 347, receiving narratives 348 and receiving analytical hits 349. Data received can be stored in the talent-label points database 121. Receiving endorsements 347 can be proceeded by generating a user interface, leading to receiving results of user interaction with the interface. The user interface can be generated by the talent engine 117 or presentation layer. Receiving endorsements 347 includes assigning talent-labels to nominees or commenting on previously assigned talent-labels. Receiving narratives 348 can accompany receiving endorsements 347. Receiving analytical hits 349 can involve interaction with the semantic analysis engine 137 or other analytical engine. The analytical hits received can be coming via a link to a work activities database 127. The analytical hits received can have equal weights or can be assigned varying weights by the engine from which they are received.
At step 365, the query processor 265 response to queries 365. Responses can be generated for a variety of queries. For example, responses can be generated for queries seeking to identify talent-label candidates 367. Candidate talent-labels can be identified responsive to keyword queries, expansion of the hierarchy, or partial entry of the keyword. Keyword searches can be supported, with or without thesaurus support. A recommendation engine using collaborative filtering or other techniques can recommend talent-labels responsive either to a keyword search or to selection of one or more talent-labels. A taxonomy can be provided to facilitate selection of labels and branches of multiple labels. Selected talent-labels can be the basis for searching for experts.
Responses to queries by talent-label can identify nominees with accumulated talent-label points applied to selected talent-labels 368. Either a query processor 265 or presentation layer can present a user with an interface that identifies nominees and their relevant talent-label points. The query processor 265 can receive request for detailed information regarding particular nominees. Responding to queries 365 can include providing nominee details about 369. This can include a list of a user's talent-labels and their talent-label points. It can include the text of endorsements relevant for selected talent-labels. It can include external biographical information 131 from another source, such as LinkedIn or Facebook. The overall process flow described can be supported by the data structures depicted in
The talent-label points database 121 can be organized by nominee 465. Nominees have multiple talent-labels 477. Talent-label points are accumulated for a talent-label 477 based on a variety of factors. For instance, talent-label points 121 can be accumulated for nominations 487, endorsements 488 or analytical results 489. In the talent-label points database 121, denomination or reference to nomination 487 can be stored. For a particular talent-label 477, nomination associated points 487 can be accumulated on a weighted or unweighted basis. Endorsements or references to endorsements 488 can include identity of the endorsing social network member and accompanying narrative. Analytical results or references to analytical results 489 can include results returned by the semantic analysis engine 137 or other analytical engine. Endorsement and analytical associated points can be accumulated on a weighted or unweighted basis. The talent-label database 113 and talent-label points database 121 report the same overall database structure. They can be organized as separate tables or as different object types within the same table. With the underlying structure in mind, we turn to example interfaces.
In order to standardize available areas of expertise, a user having appropriate privileges can select a set syntax or define a talent-label for a skill. In some implementations, every user can create talent-labels. When a user edits his or her profile and types in a skill, the standardized label may appear as an available choice. Alternatively or additionally, a user can list skills without using a standardized list. In this fashion, a user can endorse him- or herself as an expert in specific areas.
While self-endorsement can be an effective way to identify expertise, it may not be perceived as unbiased. In an embodiment, a first user may endorse a second user for a specific area of expertise, i.e., a peer endorsement. The endorsement may be for a skill from a list of standardized areas of expertise, or the endorsement may be for an unlisted skill.
In some implementations, a user may be identified as an expert by his various posts to the online social network, i.e., based on semantic analysis credit. For example, if the user responds to questions on database.com, which is an on-line database service offered by salesforce.com, inc., then the user may be considered an expert on database.com. Similarly, if other users tend to “like” or otherwise indicate approval of the user's posts, these actions may be used to identify the user as an expert. Semantic credit for expertise may be based upon interaction with one or more records, bugs, or other data objects in an enterprise system. Generally, information of this sort can be stored in one or more work activities databases 127 where available to the semantic analysis center 137 or other analytical engine. Work activity databases typically are fielded databases that are more amenable to semantic analysis than free text documents.
Other areas of the user profile 500 can include user summary information 540, user future activities information 550, user current activities information 560 and user past activities information 570. The summary information 540 can be a user provided expertise summary. The future activities information 550 can be a user supplied description or it can be automatically generated from work records. The current activities information 560 can be user supplied or automatically generated from work records. The past activities information 570 can be obtained from an external biographical source, such as LinkedIn or Facebook, or an internal social network-related biographical source. The source can be maintained by social network members or by a human relations or publicity department. The proficiency scores displayed this example interface combine a variety of factors.
Self-endorsement, peer endorsement, and semantic credit can each be effective ways for rating a user's level of expertise in specific areas. However, it can be more effective to combine two or three of these factors.
In the illustration, example weights are given for self endorsement 687, peer endorsements 688 and semantic credit 689. The example weight for self endorsement or self-nomination 687 is 20 points. A nominee only nominates himself once for a particular talent-label. The example weight for peer endorsements 688 is 10 points. In some implementations, the weight assigned to peer endorsements 688 may vary, depending on whether the endorsements are accompanied by a narrative. In other implementations, endorsements could be required to be accompanied by a narrative. Endorsement narratives can be analytically or semantically evaluated to ensure that they make sense for the talent-label that is being endorsed. Display of proficiency scores using these weights is illustrated in
In some implementations, the parents of these interfaces can be rearranged. The order of columns can be changed. Vertical orientations can be changed to horizontal orientations. Positions of things on the interface pages may be user configurable.
In some implementations, a user can search for an expert by typing in the desired subject matter in the search box 721, and/or using a “leaderboard” as is shown in
By posting a question to an expert's newsfeed, the question can be made public such that other users viewing the expert's newsfeed may also respond. Alternatively or additionally, the expert may respond and may post the answer as a comment, thereby allowing other users to view the answer.
Following a satisfactory answer, the user seeking expert advice can then endorse the expert, thereby contributing to the expert's peer endorsement rating, and subsequently, proficiency score.
In this fashion, locating and promoting users as experts can be made “social” in an online social network context.
User interface input devices 1319 may include a keyboard, pointing devices such as a mouse, trackball, touchpad, or graphics tablet, a scanner, a touchscreen incorporated into the display, audio input devices such as voice recognition systems, microphones, and other types of input devices. In general, use of the term “input device” is intended to include all possible types of devices and ways to input information into computer system 1300 or onto communication network 125.
User interface output devices 1337 may include a display subsystem, a printer, a fax machine, or non-visual displays such as audio output devices. The display subsystem may include a cathode ray tube (CRT), a flat-panel device such as a liquid crystal display (LCD) or light emitting diode display (LED), which may be touch sensitive, a projection device, or some other mechanism for creating a visible image. The display subsystem may also provide non-visual display such as via audio output devices. In general, use of the term “output device” is intended to include all possible types of devices and ways to output information from computer system 1300 to the user or to another machine or computer system.
Storage subsystem 1325 stores programming and data constructs that provide the functionality of some or all of the modules described herein, including the logic to identify related queries that represent different information requests according to the processes described herein. These software modules are generally executed by processor 1331 alone or in combination with other processors.
Memory 1321 used in the storage subsystem can include a number of memories including a main random access memory (RAM) 1324 for storage of instructions and data during program execution and a read only memory (ROM) 1323 in which fixed instructions are stored. A file storage subsystem 1325 can provide persistent storage for program and data files, and may include a hard disk drive, a floppy disk drive along with associated removable media, a CD-ROM drive, an optical drive, or removable media cartridges. The modules implementing the functionality of certain implementations may be stored by file storage subsystem 1325 in the storage subsystem 1324, or in other machines accessible by the processor.
Bus subsystem 1330 provides a mechanism for letting the various components and subsystems of computer system 1300 communicate with each other as intended. Although bus subsystem 1330 is shown schematically as a single bus, alternative embodiments of the bus subsystem may use multiple busses.
Computer system 1300 can be of varying types including a workstation, server, computing cluster, blade server, server farm, or any other data processing system or computing device. Due to the ever-changing nature of computers and networks, the description of computer system 1300 depicted in
In one implementation the method is described that includes identifying particular technical talent within an organization. This method includes receiving talent nominations that include two or more talent-labels that identify a nominee's technical talents in a first category and a second category, wherein talent-labels in the first category are more specific to the organization to which the nominee belongs than talent-labels in the second category. In some implementations, more than two categories can be used to group talent-labels. The first category can list intra-company task talents and the second category can list industry wide talents. Examples of talents these categories are given above. The method can further include accumulating talent points for the nominees' talent-labels, including assigning talent-label points for at least co-worker endorsements and from semantic analysis of intra-company fielded databases.
This method and other implementations the technology disclosed can each optionally include one or more the following features.
The method can include responding to queries for particular technical talents by identifying one or more relevant talent-labels, one or more nominees for each relevant talent-label, and the identified nominees' talent-label points.
An additional category of talent-labels can be included. For example, talent-labels in the category of personal or recreational skills can be collected.
Talent-labels can be created on demand while collecting a particular talent nomination or endorsement.
A co-worker endorsement can be accompanied by a narrative, either optionally or as a requirement of making an endorsement. The narrative can be qualified semantic analysis of the narrative's content.
Talent-label points can be assigned for self-nomination.
Talent-label points can be calculated according to the formulas given above.
Responding to the queries can include searching the talent-labels by keyword and/or presenting a taxonomy of talent-labels.
Responding to follow-up queries can include providing details on one or more nominees, including the particular nominee's talent-labels, talent-label points and a self-description of the particular nominee. The self-description can be obtained from an external biographical source.
Other implementations may include a non-transitory computer readable storage medium storing instructions executable on a processor to perform any of the methods described. Implementations also may include systems including memory and one or more processors operable to execute instructions, stored in the memory, to perform any of the methods described.
While the present invention is disclosed by reference to the preferred embodiments and examples detailed above, it is to be understood that these examples are intended in an illustrative rather than in a limiting sense. It is contemplated that modifications and combinations will readily occur to those skilled in the art, which modifications and combinations will be within the spirit of the invention and the scope of the following claims.
This application claims the benefit of U.S. Provisional Patent Application No. 61/558,017, entitled, “Systems and Methods for Identifying Experts,” filed on 10 Nov. 2011. The provisional application is hereby incorporated by reference for all purposes.
Number | Date | Country | |
---|---|---|---|
61558017 | Nov 2011 | US |