Patent Application
20200394724
Virtual hooliganism and misbehavior on online services and content publishing platforms appears to be increasing. Mainstream social networking platforms are seeking to solve problems associated with content moderation. However, moderating content requires a significant amount of resources and technology. While automated and semi-automated moderation strategies and tools are improving, offenders find means to bypass conventionally programmed logic. In addition, employing agencies or hiring human resources for manual moderation is expensive. Another problem with human moderation is the psychological impact that objectionable content has on moderators.
In an embodiment, a method of selecting an intervention includes, based on user content submitted to a platform, determining whether the content includes objectionable content. The method further includes, based on the objectionable content, associating a content persona classification with the objectionable content. The method further includes, based on the content persona classification, selecting an intervention.
In an embodiment, a method includes receiving user content submitted to a platform. The method further includes determining whether the user content is associated with a product or is associated with a review. The method further includes associating a persona classification with the user content. The method further includes determining whether the user content includes objectionable content. The method further includes, based on the persona classification, the objectionable content, and whether the user content is associated with a product, selecting an intervention.
In an embodiment, a method includes receiving user content associated with a product. The method further includes, based on the user content, associating, with one or more of a set of classifications, the user content based on at least a set of defined personas, objectionability, and validity. The method further includes, based on the one or more of the set of classifications that the user content is associated with, selecting an intervention.
Some online platforms and organizations struggle to police input received as reviews for inaccurate and/or offensive content. Some have shut down commenting sections which harms the user experience because these reviews/comments can be a crucial decision-making factor for members within these online communities. Shutting down discourse does not just stifle the creation of an engaged social community around the product or service, it also makes honest participants within the community feel disenfranchised.
In an embodiment, interventions are made after the creation of offensive content. Distinct personas of users, and content characteristics may be identified that lead to the development or reinforcement of these personas. Intervention strategies that are personalized based on the channel of interaction as well as the mapped persona are also identified. By identifying and deploying individualized intervention strategies that work best for a particular persona, users are encouraged to engage in corrective behaviors.
Platform interactions 150 is operatively coupled to data management system 140 (and interaction data 141, in particular.) Data management system 140 is operatively coupled to content classifier 110 and user persona classifier 120. Data management system 140 is operatively coupled to content classifier 110 to provide at least customer information data and associated platform interaction data 141 to content classifier 110. Content classifier 110 is operatively coupled to data management system 140, user persona classifier 120, and intervention mapping system 130. Intervention mapping system 130 is operatively coupled to messaging module 131 and intervention communication channels 132.
Data management system 140 serves as a central database linked a platform that contains information about the customer and the customers platform interactions 150. Data management system 140 may collect and/or store, for example, the following kinds of data: (1) clickstream data—all interactions that customers engage in with the platform or companion applications to on a daily basis; (2) user data—insights obtained about the user about their demographics, preferences, interests and patterns; (3) product data—on selling platforms, information about the product and corresponding sellers; and/or (4) persona data—the classified personas associated with content and/or platform interactions.
Role classifier 111 of content classifier 110 may use structural syntax to determine whether the content or platform interaction (e.g., from data management system 140) is a product listing or a review of a product listing. If it is a review, then the content persona classifier 112 would be provided the content or platform interaction in order to associate a persona with the content. If the review has characteristics that can be attributed to a persona's characteristics, then the content is classified and associated with a persona and the identified persona is shared with the user persona classification module 120. If the content is a product listing, then a persona is not associated with the content.
Objectionable content detection 113 may use a set of Natural Language Processing (NLP) algorithms to identify distinct forms of objectionable content. A magnitude or intensity of the objectionable content is also associated with the content. Validity detection 114 processes the product listing and reviews to determine whether the content includes indicators of invalid content such as misaligned subject (geography, content, audience), expired content, and/or prohibited content. When multiple indicators of invalid content are associated with the content, a priority for intervention is increased.
User persona classifier 120 may classify a reviewer into, for example, one or more of the following reviewer personas: subjective, objective, misaligned, trigger, misguided, expressive, high-returner, non-owner, and offensive. A subjective reviewer persona may be selected when the review is highly descriptive in nature but does not address a real issue. An objective reviewer persona may be selected when the review is non-descriptive in nature and identifies a real issue.
A misaligned reviewer persona may be selected when it is determined there is one or more of a review-geographical misalignment, a review-product misalignment, and/or a review-seller misalignment. A review-geographical misalignment may be selected when, for example, a review states something about a product and that statement does not hold true in the geographical area associated with the reviewer and/or review, or does not hold true for the geographical areas that the product is intended for (e.g., right-hand car steering wheels in the United States.) A review-product misalignment may be selected when, for example, the review is posted to a different product than the review addresses (i.e., posted to the wrong product listing.) A review-seller misalignment may be selected when, for example, the review addresses the correct product but was posted to the wrong seller's profile and/or listing.
A trigger reviewer persona may be selected when, for example, the review itself is not deemed objectionable, has indications that it will lead to backlash from other reviewers due to how the review is framed, and/or other subtleties. For example, a review that states “ . . . only dogs would buy this product . . . ” when it is not a pet related product may trigger a response from those who liked the product and understand the statement as equivalent to being called a ‘dog.’
A misguided reviewer persona may be selected when, for example, there is a mismatch between a rating score and the review comment. For example, the review may state that this is a ‘great product, everyone should buy one . . . ” but the reviewer only gave the product a “1-star” out of 5 rating. An expressive reviewer persona may be selected when, for example, the review is overly expressive and verbal. A high-returner reviewer persona may be selected when, for example, the reviewer often tries then returns products. A non-owner reviewer persona may be selected when, for example, it is determined the reviewer has not bought the specific product from the platform they are posting the review on. An offensive reviewer persona may be selected when, for example, the review contains hate speech or other offensive material/language.
User persona classifier 120 may classify a seller into, for example, one or more of the following seller personas: misaligned, prohibited, trigger product, trigger description, expired/invalid, objectionable, and offensive. A misaligned seller persona may be selected when, for example, it is determined there is one or more of an item-geographical misalignment, an item-category misalignment, an item-content misalignment, and/or a description-audience misalignment.
An item-geographical misalignment may be selected when, for example, the item/product is marked as being permitted to be sold into a geography where it is not allowed to be sold. An item-category misalignment may be selected when, for example, the item/product has been placed into the wrong category (e.g., eating utensils placed into the ‘cleaning supplies’ category.) An item-content misalignment may be selected when, for example, the product description does not match the item/product it is associated with (e.g., the description for a ‘box of rubber bands’ talks about ‘decorative stone tile’.) A description-audience misalignment may be selected when, for example, the item/product description contains content that might not be suitable for the audience the product is restricted to.
A prohibited seller persona may be selected when, for example, an attempt to list a product/item that is prohibited (according to platform policies) is made. A seller trigger product persona may be selected when, for example, a product has the possibility of leading to trigger comments, but which itself cannot be banned. For example, if a product has certain side-effects upon consumption, and the seller hasn't made those side-effects clear. The existence of those undisclosed side effects could lead to a backlash.
A seller trigger description persona may be selected when, for example, the product description is determined to have a likelihood of leading to backlash in the reviews of the product. For example, a general use product with a description that includes ‘ . . . this product is great for men . . . ” may lead to reaction by women using the product. An expired/invalid product persona may be selected when, for example, the item/product/price has expired or is otherwise no longer available.
An objectionable description persona may be selected when, for example, the item/product description includes objectionable language. An offensive description persona may be selected when, for example, the item/product description includes hate speech.
Intervention mapping system 130 selects messaging module 131 to be provided to the user (i.e., reviewer or seller) via a communication channel 132 based on the user persona classification provided by user persona classifier 120. Intervention mapping system 130 selects messaging module 131 to be provided to the user (i.e., reviewer or seller) and selects the communication channel 132 based further on inputs from objectionable content detection 113 and validity detection 114. In particular, communication channels 132 are selected from the general groups of: (1) on-platform notification, and (2) off-platform channels. The on-platform notifications may include, but are not limited to: notification icon, direct messaging chat (e.g., chatbot), a popup with a required action, and/or a popup without a required action. The off-platform notifications may include, but are not limited to: SMS (i.e., text message), e-mail, and/or voice call. The channel 132 selected may be based on the persona selected by user persona classifier 120. Table 1 details an example persona to channel selection mapping.
A popup with a required action is an on-platform intervention that a user has to act upon to in order to proceed further in the platform. This messaging strategy may be used for a highly sensitive offense, or where an action necessarily needs to be taken on the user's end. A popup with a required action may also be used for trigger reviews and trigger product descriptions. The platform cannot remove the trigger reviewer persona content because the content itself has nothing overtly offensive, because removing the trigger reviewer persona content would typically be against the platform's policies. Similarly, for trigger products/descriptions a mandatory action is required on the seller's end to ensure the seller has made the product/description as ‘trigger proof’ as possible.
A popup without a required action is an intervention that a user has to acknowledge to in order to proceed further in the platform. Typically, when a popup without required action is selected, the platform has already taken an action on its own, but brings this action to the user's notice so that the user acknowledges the action, learns more about the offense, and provides an option for the user to raise an exception. A popup without a required action may be used when a seller uploads a prohibited product, or an invalid/expired product. A popup without a required action may also be used in cases of offensive (hate speech) detection in product description or review content.
A chatbot may be used for providing guidance so that users may take actions on less time critical issues. For example, when a seller uploads a product in a misaligned category, or a reviewer writes a review of misaligned product, a chatbot assistant can help provide guidance to help correct the mistake. This also helps ensure that the issue has been identified correctly and is dealt with by bringing the issue to the user's notice. A chatbot may be used for a subjective reviewer. For example, the user may be asked to take a survey about the product associated with the review. This survey may help them convert the review into an objective one. A chatbot may be used to make a misguided reviewer aware of the rating system of the platform. A chatbot may be used to lead a frequent returner through a qualitative intercept to understand the frequent returner's pain points (issues) and thus possibly change the frequent returning behavior. A chatbot may be used for an expressive reviewer to make sure that the expressive reviewer really intended (in hindsight) the strong response to a particular product that was initially posted.
A notification may be used more as an acknowledgement than actionable intervention. A notification may be used for an objective reviewer to express appreciation of the objectivity of the review and to reinforce good behavior. A notification may be used for notifying non-owner reviewers that the review will be tagged as a non-owner review to thereby bring their non-owner classification to the notice of the other platform users. An SMS may be used as a notification and may also be used for a reviewer who writes an objective review.
A voice call may be used when the user may need an explanation of a particular offense. This may occur for offense that may not be apparent, or may be highly sensitive (e.g., trigger reviews, trigger products, and trigger descriptions.) A voice call may be used for instances where qualitatively understanding a user's pain point is necessary too (e.g., a frequent returner.)
An email may be selected as a communication channel where actionable items can be sent, but where immediacy is not of concern and ambiguity is not likely to occur. In an embodiment, instances where a chatbot is used will also have an associated email communication. In cases where we use a popup without required action is used, an email may be sent as formal communication. In these cases, the platform may have already taken an action and therefore immediacy is not a priority. An email may be sent (e.g., in addition to a call) as form of a formal communication.
Messaging module 131 generates personalized messages based on the intervention strategy identified and inserts it into a template appropriate to the channel that is identified for intervention. Messaging module 131 chooses and personalizes a messaging template based on the intervention strategy that was selected. The factors that messaging module 131 may consider include, but are not limited to: whether the user a seller or reviewer; whether the user is a new offender or a habitual/repeat offender; the seller's persona classification; the intensity of the identified objectionable content; the type of triggered validation-based criteria and associated priority; and/or, the selected channel of intervention. Messaging module 131 receives the selected channel for the intervention and returns the intervention to intervention mapping module 130 for provision a selected channel 132.
Based on the objectionable content, a content persona classification is associated with the objectionable content (204). For example, content classifier 110 (and content persona classifier, in particular) may, based on the content, associate a persona with the content. If the user is a reviewer, for example, one or more of the following reviewer personas may be associated: subjective, objective, misaligned, trigger, misguided, expressive, high-returner, non-owner, and offensive. If the user is a seller, for example, one or more of the following seller personas may be associated: misaligned, prohibited, trigger product, trigger description, expired/invalid, objectionable, and offensive.
Based on the content persona classification, an intervention is selected (206). For example, based on a user persona selected by user persona classifier 120, which is based on the content persona provided by content persona 112, intervention mapping 130 may select an intervention. The intervention(s) may be, for example, selected from those given in Table 1.
Whether the user content is associated with a product or is associated with a review is determined (304). For example, role detection 111 may determine whether the content is directed to a product that a different user is selling, or is directed to a product that the user is selling. A persona classification is associated with the user content (306). For example, content classifier 110 (and content persona classifier, in particular) may, based on the content and the role determined in box 304, associate a persona with the content. If the role is determined to be a reviewer, for example, one or more of the following reviewer personas may be associated: subjective, objective, misaligned, trigger, misguided, expressive, high-returner, non-owner, and offensive. If the role is determined to be a seller, for example, one or more of the following seller personas may be associated: misaligned, prohibited, trigger product, trigger description, expired/invalid, objectionable, and offensive.
Whether the user content includes objectionable content is determined (308). For example, content classifier 110 (and objectionable content detection and/or validity detection 114, in particular) may determine whether platform interactions 150 from a user that is stored in data management system 140 (and interaction data 141, in particular) includes objectionable content. The objectionable content may include, for example, content that has one or more characterizations as: subjective, misaligned, trigger, misguided, expressive, non-owner, offensive, prohibited, trigger product, trigger description, expired/invalid, and/or otherwise objectionable.
An intervention is selected based on whether the user content is associated with a product, the persona classification, and the objectionable content (310). For example, based on the determined role, the persona classification, and the objectionable content, intervention mapping 130 may select an intervention. The intervention(s) may be, for example, selected from those given in Table 1.
The user content is associated with one or more of a set of classifications based on at least a set of defined personas, objectionability, and validity (404). For example, based on one or more personas selected by content persona classifier 112, the output of objectionable content detection 113, and the output of validity detection 114, user persona classifier may select a classification. This classification may be, for example, selected from the personas given in Table 1.
An intervention is selected based on the one or more classifications that the user content is associated with (406). For example, based on a user persona selected by user persona classifier 120 intervention mapping 130 may select and intervention. The intervention(s) may be, for example, selected from those given in Table 1.
The user's role is determined (504). For example, role detection 111 may determine whether the content is directed to a product that a different user is selling (reviewer role), or is directed to a product that the user is selling (seller role). If the content is associated with the reviewer role, then flow proceeds to box 506. If the content is associated with the seller role, then flow proceeds to box 524.
If the content is associated with the reviewer role, then the content is classified as objective or subjective (506). For example, content persona classifier 112 may classify the content as objective or subjective. Example pseudocode for this classification procedure is given in Table 2. Note that TextBlob (referenced in Table 2) is a Python programming language library for processing textual data. It provides a consistent API for diving into common natural language processing (NLP) tasks such as noun phrase extraction, part-of-speech tagging, sentiment analysis, classification (e.g., naive Bayes, decision tree), language translation and detection, tokenization (e.g., splitting text into words and sentences), word and phrase frequencies, parsing, n-grams, word inflection (pluralization and singularization) and lemmatization, and spelling correction.
The content is classified as owner or non-owner (508). For example, content persona classifier 112 may label the content as having been posted by an owner or a non-owner. Example pseudocode for this classification procedure is given in Table 3.
It is determined whether the content is misaligned (510). For example, content persona classifier 112 may label the content as being misaligned as to geography, misaligned as to seller, and/or misaligned as to product. Example pseudocode for this classification procedure is given in Table 4.
It is determined whether the content is misguided (512). For example, content persona classifier 112 may label the content as being misguided where the rating does not accord with the language in the review. Example pseudocode for this classification procedure is given in Table 5.
It is determined whether the content is expressive (514). For example, content persona classifier 112 may label the content as being expressive when the review is overly expressive and verbal. Example pseudocode for this classification procedure is given in Table 6.
It is determined whether the content is objectionable (516). For example, objectionable content detection 113 may use a set of Natural Language Processing (NLP) algorithms to identify distinct forms of objectionable content—as trigger content. A magnitude or intensity of the objectionable content is also associated with the content. Example pseudocode for this classification procedure is given in Table 7.
After it is determined whether the content is objectionable, flow proceeds to box 540 to select an intervention. Once the user's role is determined in box 504, if the content is associated with the seller role, then flow proceeds to box 524 where it is determined whether the content is prohibited (524). Prohibited content is when, for example, an attempt is made to list a product/item that is prohibited (according to platform policies). It is determined whether the content is misaligned (526). For example, conent persona 112 may classify content as misaligned when, for example, it is determined there is one or more of an item-geographical misalignment, an item-category misalignment, an item-content misalignment, and/or a description-audience misalignment. Example pseudocode for this classification procedure is given in Table 8.
It is determined whether the content is invalid (528). For example, content classifier 110 (and validity detection 114, in particular) may detect when, for example, the item/product/price has expired or is otherwise no longer available. It is determined whether the content is objectionable (530). For example, objectionable content detection 113 may use a set of Natural Language Processing (NLP) algorithms to identify distinct forms of objectionable content—such as trigger content. A magnitude or intensity of the objectionable content is also associated with the content. Example pseudocode for this classification procedure is given in Table 9.
Flow then proceeds to box 540. In box 540, an intervention is selected (540). The intervention may be selected, for example, according to Table 1. The intervention is then performed (542). The intervention may be performed, for example, via a channel selected according to Table 1.
An example use of processing node 600 includes implementing intervention system 100, content classifier 110, user persona classifier 120, intervention mapping system 130, messaging module 131, intervention communication channels 132, data management system 140, platform interaction 150, their components, and/or implementing the methods described herein (e.g., the processes described herein with reference to
The exemplary systems and methods described herein can be performed under the control of a processing system executing computer-readable codes embodied on a computer-readable recording medium or communication signals transmitted through a transitory medium. The computer-readable recording medium is any data storage device that can store data readable by a processing system, and includes both volatile and nonvolatile media, removable and non-removable media, and contemplates media readable by a database, a computer, and various other network devices.
Examples of the computer-readable recording medium include, but are not limited to, read-only memory (ROM), random-access memory (RAM), erasable electrically programmable ROM (EEPROM), flash memory or other memory technology, holographic media or other optical disc storage, magnetic storage including magnetic tape and magnetic disk, and solid state storage devices. The computer-readable recording medium can also be distributed over network-coupled computer systems so that the computer-readable code is stored and executed in a distributed fashion. The communication signals transmitted through a transitory medium may include, for example, modulated signals transmitted through wired or wireless transmission paths.
The above description and associated figures teach the best mode of the invention. The following claims specify the scope of the invention. Note that some aspects of the best mode may not fall within the scope of the invention as specified by the claims. Those skilled in the art will appreciate that the features described above can be combined in various ways to form multiple variations of the invention. As a result, the invention is not limited to the specific embodiments described above, but only by the following claims and their equivalents.
