Headlines are the first, and sometimes the only, opportunity to capture a user's attention. As a result, special time is spent in educating news writers on how to write headlines. This is often considered more art than science. Historically, the motivation for writing engaging headlines was to sell more newspapers—from a time when consumers would only glimpse a paper from inside a machine and decide whether to purchase based on the headline. Today, in the context of digital media, the rewards for writing engaging headlines translate into user clicks of the headlines within digital environments, such as web pages, mobile apps, Twitter streams, or discussion threads.
Content promoters, such as publishers, aggregators, content platform providers, and marketers, all wish for promoted content to have the most engaging (yet accurate) headlines possible. It has been shown that even though measures can be taken in advance to train staff to write better headlines, ultimately it is not possible to predict how well a headline will perform in terms of its click-through-rate (“CTR”) with a particular audience in a particular digital medium or venue.
Therefore, in order to find the best performing headline, a content promoter would have to hire writers or editors who are properly trained, and have them write several alternative headlines for each new piece of content. Then each of the alternative headlines would have to be rotated while tracking performance in order to determine which alternative headline yielded the highest CTR. For subsequent impressions or placements, the content promoter could then use the “winning” headline(s).
This manual headline generation process would be slow and expensive. Trained writers would have to be hired to produce all of these headlines. Additionally, since digital content is syndicated and aggregated across different time zones around the clock, a content promoter would need such writers ready at every moment.
Unfortunately, there are currently no systems for automatically and effectively generating alternative headlines for content.
While methods, apparatuses, and computer-readable media are described herein by way of examples and embodiments, those skilled in the art recognize that methods, apparatuses, and computer-readable media for generating headlines are not limited to the embodiments or drawings described. It should be understood that the drawings and description are not intended to be limited to the particular form disclosed. Rather, the intention is to cover all modifications, equivalents and alternatives falling within the spirit and scope of the appended claims. Any headings used herein are for organizational purposes only and are not meant to limit the scope of the description or the claims. As used herein, the word “may” is used in a permissive sense (i.e., meaning having the potential to) rather than the mandatory sense (i.e., meaning must). Similarly, the words “include,” “including,” and “includes” mean including, but not limited to.
Applicant has discovered and developed new technology which, given a piece of content in a document, such as a web page or a blog post, can automatically generate an original headline, or one or more alternative headlines to the original headline. The term “headline”, as used herein refers to any text that serves as topical indicator of the content, such as article headlines, subject lines of email messages, chapter names, or other headers in a document.
Headline can also refer to titles and sub-titles in a document. For example, the methods described herein can be used to generate a new sub-title or an alternative sub-title for each sub-section of text in a document. In this case, each sub-section of text can be processed separately using the described methods and systems to generate the sub-titles, even for sub-sections where the author did not originally have a sub-title. The generated sub-titles can then be used as a synopsis for the entire document (for example, by collapsing the text of the document to just the sub-titles).
Headline, as used herein, can also refer to a portion of the text which is designed to attract a reader's attention but is not necessarily at the top of an article or document, such a snippet of text which is enlarged and presented alongside an article. For example, magazines and online articles sometimes take snippets of an article and blow them up larger—this is aimed at grabbing the attention of a reader or potential reader and pulling them into the article. The methods and systems described herein can be used to generate new or alternative snippets for articles and other documents.
The headline generation methods and systems described herein can also be utilized to provide feedback or grades to authors and editors (such as by incorporation into the editing environment or content management system) and to suggest possible headlines or score suggested headlines before the article is even published. For example, an author or editor could suggest a headline for an article and the methods described herein can be used to score the suggested headline and indicate whether there are other possible headlines (based on the text of the document) with higher scores. This feedback can be used by the author or editor to reformulate the headline and/or improve future headlines.
At step 101, a content section of the document is identified. Identifying the content section can include isolating the zones or sections of the document which are applicable to the headline, since not all of the document sections are necessarily applicable.
In a digital document, such as a web page, some sections are ads, some sections are listings of other articles published that day which have nothing to do with the headline, some sections contain the copyright notices of the publisher, some sections are navigation menus, etc. Since the main body of text in the document is used as the raw material for new headlines, it is important to properly identify the content section of the document to make sure that extraneous or irrelevant content is not used to generate headlines.
This section can be identified through a variety of methods. For example, the content section can be identified based on the layout of the document and pre-existing rules regarding the likely location of the content section. In another example, the content section can be identified by analyzing the text of the document and comparing it to the original headline to identify which section of the document contains overlap with the original headline. The content section can also be identified by a purely textual analysis, such by counting the number of words or sentences in each section. Additionally, the content section can be identified based on syntax or grammatical features. For example, a section with complete sentences and multiple paragraphs can be flagged as a content section while a section with short or incomplete sentences, multiple images, multiple links, and/or little text can be identified as a non-content section, such as an advertisement section or a related article section. Of course, more than one section in the document can be identified as a content section.
Returning to
Headline performance can be measured by a performance indicator such as click-through rate (CTR), conversion to action, time spent on site, and/or download rate. Performance indicators can include any metric which measures user engagement. For example, a performance indicator can be a mouse-over rate, which is the rate at which users move the mouse over a particular item. In another example, if an item (such as a link) on web page includes a quick survey/poll or other interactive user interface element, the performance indicator can measure how often users click a poll response or interact with the user interface element.
User engagement can be measured by any actions the user takes with regard to an item, such as a headline, that indicate the user is engaging with a page containing the item. For example, if a headline contains a link to a document and a small blurb from the document, a performance indicator can include detecting how often users scroll down a web page to read the headline and/or blurb.
When selecting a sentence in a document (or multiple sentences in a document) the characteristics considered can be those which correlate to positive headline performance or negative headline performance.
Characteristics correlated with positive headline performance can mean characteristics correlated with headlines that have a performance indicator above a predetermined threshold. For example, characteristics associated with headlines that have a CTR greater than 0.02% may be considered characteristics associated with positive headline performance.
Characteristics correlated with negative headline performance can mean characteristics correlated with headlines that have a performance indicator below a predetermined threshold. For example, characteristics associated with headlines that have a CTR less than 0.001% may be considered characteristics associated with negative headline performance.
Additionally, characteristics can be weighted according to the degree they are correlated with positive or negative headline performance. For example, the weights can be negative or positive, reflecting whether the characteristic is correlated with negative headline performance or positive headline performance.
For example, headlines starting with the word “Tips” can be correlated with a low CTR. As a result, any sentences starting with the word “tips” would have that characteristic be negatively weighted when computing a score for the sentence, as will be described further.
The one or more characteristics can include semantic characteristics (such characteristics having to do with meaning or topic) and grammatical characteristics (such as characteristics relating to how a sentence is written without regard to specific subject matter). Both kinds of characteristics may be desirable with regard to performance indicators such as CTR. Additionally, a characteristic can also be the existence of one or more particular words or phrases in the sentence.
An example of semantic characteristics can be sex, privacy, scandal, Edward Snowden, etc. These semantic characteristics can change over time (such as monthly, daily, or hourly). These are not keywords but topics which can be represented by many variations in phrasing. For example, the “privacy” topic can include sub-topics or associated terms such as “intrusion” or “fourth amendment.”
Additionally, certain topics can be more strongly associated with headline performance (and positive performance indicators), and the strength of that association can wax and wane over time.
Another sort of semantic characteristic would be high-level abstract characteristics such as, implied grade-level of vocabulary used in the sentence (for example whether a 12th grade or 4th grade level vocabulary is used). This characteristic can be determined or estimated based on how many words are used that are typical of readers at higher or lower grade levels. For example, if the word “sophistry” is correlated only with readers at an 11th grade level or higher, then a sentence containing the word “sophistry” can have an implied grade-level of 11th grade (assuming no major inconsistencies with other words in the sentence). Furthermore, if there is a correlation between low CTRs (such as CTRs below 0.0005%) and sentences written at grade-levels higher than 8th grade level, then this characteristic can be negatively weighted when the score for the sentence is determined.
Grammatical characteristics can include characteristics such as the use of one-digit natural numbers, or use of the number “five,” or use of any comparatives or superlatives, or use of particular superlatives such as “ugliest,” or use of certain prefatory words such as “tips” or “how to.” These characteristics can be general or specific, and can also be weighted.
A general characteristic can reference an entire class of words or constructions, such as “positive comparatives.” That includes phrases such as “better, faster, stronger, easier, smarter, more intelligent,” etc. A specific characteristic can be “equivalents of ‘faster,’” which can include “faster, quicker, speedier,” etc.
Additionally, grammatical characteristics can include a number or type of phrases in a sentence, a number of words, a number of punctuation marks, an average word length in characters or syllables, a number of connectives (such as and, but, it, not), sentence structure, parts-of-speech for each of the words, or any other grammar related characteristics.
Additionally, the system can receive input based on historical performance indicators, such as historical CTR data associated with particular headlines, to tailor the process by favoring characteristics, semantic or grammatical, that have been correlated with better performing headlines.
In this way, the optimization can be specific to real human behavior on a specific content network, which can differ from one audience to the next. This also means the headlines can be constructed differently than, for example, headlines constructed for Search Engine Optimization. For example, if the goal of the headlines is to increase the rate at which certain articles are forwarded or shared by users (such as in a social network), then the characteristics which are historically correlated with headlines for articles having a high rate of sharing can be positively weighted, resulting in headlines intended to optimize sharing of the article.
For example, the characteristics can be weighted separately by class of characteristic, such that grammatical characteristics are weighted less than semantic characteristics, such as at a 2:3 ratio. So if a particular sentence includes one negative grammatical characteristic (meaning a characteristic correlated with poor headline performance) with a score of −5 and one positive semantic characteristic with a score of +3, then the total score for the sentence would be 2*(−5)+3*(+3)=−10+9=−1, since both types of characteristics are weighted.
Of course, the scores for each of the sentences can also be determined based on a score associated with each characteristics in the sentence, without any separate weighting of each characteristic. For example, if a sentence includes two positive characteristics with scores of 3, and 4.2, and one negative characteristic with a score of −1.2, then the total score for the sentence can be 3+4.2−1.2=6.
At step 302, the sentences in the content section(s) can optionally be ranked according to the assigned scores. This step can also be omitted. For example, if only one sentence is being selected to generate a headline, then this step can be omitted and the highest scoring sentence can be selected.
At step 303, the top N sentences in the content section(s) are selected as the seed sentences from which to construct one or more headlines, where N is any positive number less than or equal to the total number of sentences in the content section(s). For example, one sentence can be selected, five sentences can be selected, or ten sentences can be selected. The number of sentences selected can also be based on a score threshold. For example, all sentences which have a total score greater than a predetermined amount, such as an amount provided by the user, can be selected. After the sentence(s) are selected, the relevant portions of each of the sentence(s) are extracted.
At step 402 at least one of a subject of the trigger attribute and an object of the trigger attribute are identified. The subject of the trigger attribute and/or the object of the trigger attribute can be identified based on rules relating to at least one of syntax, grammar, parts-of-speech, and punctuation. This can include analyzing the sentence grammatically, left and right of the trigger attribute, to determine a “window” within which a complete concept (such as a verb with object and/or subject nouns) is likely expressed. Syntax parsing and punctuation can be used to determine this window. For example, commas often delimit segments of a longer sentence in a way that encapsulates a particular concept and commas can be used to separate the sentence into portions and thereby select the portion containing the trigger attribute.
At step 403 a portion of the sentence is extracted based at least in part on the location of the trigger attribute and the location of at least one of the subject of the trigger attribute and the object of the trigger attribute. For example, a portion of the sentence can be extracted that includes the trigger attribute and at least one of the subject of the trigger attribute and the object of the trigger attribute and any words between, while leaving out any words not located between. The extracted portion can be the earlier determined window as described above.
As shown in sentence 503 the sentence is concatenated based on the positions of the commas and the location of the trigger attribute so that we are left with the portion containing the trigger attribute.
As indicated in sentence 504, “Apple” is identified as the subject of the verb “launch” and “new line of wearable devices” is identified as the object of the verb “launch.” These identifications can be made based on rules relating to at least one of syntax, grammar, parts-of-speech, and punctuation as described above. For example, “Apple” can be identified as the subject since it is the immediately preceding noun phrase and “line of wearable devices” can be identified as the object since it is the immediately succeeding noun phrase.
Therefore, the beginning of the window for the portion of the sentence is the word “Apple” and the end of the window is the word “devices.” All of the words in this window are extracted, leaving the portion of the sentence shown at 505.
At step 601, one or more words are removed from the portion of the sentence. The one or more words can include unnecessary adjectives, adverbs, and/or prepositional phrases. However, not all unnecessary adjectives, adverbs, and/or prepositional phrases are deleted. The determination of which words are deleted can be based on the desirability of each of the words in a final headline. In other words, if certain adjectives, adverbs, and/or prepositional phrases are correlated with positive headline performance, then they can be kept in the sentence portion. Otherwise, any words that are not logically and/or grammatically necessary to convey the concept of the sentence portion can be deleted.
At step 602, one or more verbs can be converted into a different tense. For example, a verb can be converted into an active tense or a gerund. Of course, other tenses can be used and these examples are provided for illustration only. A verb can be converted into any tense that is correlated with positive headline performance. For example, if the past tense of a particular verb has a greater correlation with positive headline performance, then that verb can be converted into past tense.
At step 603, one or more additional operations can be performed on the sentence portion to generate the alternative headline. These additional operations can include, for example, reordering of words in the portion of the sentence or addition of connectives between words in the sentence portion.
As shown in portion 702 the words “an entirely” are removed as being unnecessary. Of course, if any of these words were correlated with positive headline performance, then they could be kept. For example, if the word “new” was considered unnecessary (since, for example, “launch” already implies a new product) but the word “new” was correlated with positive headline performance, then it would not be removed. Furthermore, additional words can also be removed. For example, the prepositional phrase “of wearable devices” could also be removed for a shorter headline.
As shown in portion 703, the verb “launch” is converted into present tense and the previous word “would” is accordingly removed as no longer necessary or grammatically correct. After these changes are implemented, the headline 704 reads “Apple launching new line of wearable devices.”
One or more alternative headlines can be generated for an original headline using the processes described above. The one or more alternative headlines can be based on one or more sentences in the content portion of document, thereby utilizing the author's own words to generate the alternative headlines. The one or more alternative headlines can be rotated as the headline for the document and the results (in terms of performance indicators such as CTR) can be recorded for each alternative headline. Based on these results, a winning alternative headline can be selected from the one or more alternative headlines to permanently replace the original headline.
When generating an alternative headline, a score can be generated for the original headline using the sentence scoring processes described earlier, and the score for the original headline can be compared to that of the generated alternative headline to determine the improvement. Optionally, only alternative headlines that improve the score from the original headline by a predetermined threshold can be utilized or suggested to replace the original headline.
Additionally, the method for generating alternative headlines described above can also be utilized to generate original headlines. In this case, documents without headlines can be received and the processes described above can be used to identify the content section of the documents, identify one or more sentences as seed sentences, extract one or more portions from the seed sentences, and generate one or more possible headlines for the document.
Furthermore, as discussed earlier, the methods and systems for generating headlines described herein can be to generate sub-titles for sub-sections of text in a document or snippets of text to be enlarged or presented alongside an article or document or sections of a document. For example, a separate snippet or sub-title can be generated for each sub-section of text in a document based on the text in that sub-section. Additionally, as described earlier, the sentence scoring processes can be used to grade headlines suggested by authors or editors in pre-publication setting and the headline generation process can be used to provide possible alternative headlines with higher scores.
One or more of the above-described techniques can be implemented in or involve one or more computer systems.
With reference to
A computing environment may have additional features. For example, the computing environment 800 includes storage 840, one or more input devices 850, one or more output devices 860, and one or more communication connections 890. An interconnection mechanism 870, such as a bus, controller, or network interconnects the components of the computing environment 800. Typically, operating system software or firmware (not shown) provides an operating environment for other software executing in the computing environment 1000, and coordinates activities of the components of the computing environment 800.
The storage 840 may be removable or non-removable, and includes magnetic disks, magnetic tapes or cassettes, CD-ROMs, CD-RWs, DVDs, or any other medium which can be used to store information and which can be accessed within the computing environment 800. The storage 840 may store instructions for the software 880.
The input device(s) 850 may be a touch input device such as a keyboard, mouse, pen, trackball, touch screen, or game controller, a voice input device, a scanning device, a digital camera, remote control, or another device that provides input to the computing environment 800. The output device(s) 860 may be a display, television, monitor, printer, speaker, or another device that provides output from the computing environment 800.
The communication connection(s) 890 enable communication over a communication medium to another computing entity. The communication medium conveys information such as computer-executable instructions, audio or video information, or other data in a modulated data signal. A modulated data signal is a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media include wired or wireless techniques implemented with an electrical, optical, RF, infrared, acoustic, or other carrier.
Implementations can be described in the general context of computer-readable media. Computer-readable media are any available media that can be accessed within a computing environment. By way of example, and not limitation, within the computing environment 800, computer-readable media include memory 820, storage 840, communication media, and combinations of any of the above.
Of course,
Having described and illustrated the principles of our invention with reference to the described embodiment, it will be recognized that the described embodiment can be modified in arrangement and detail without departing from such principles. It should be understood that the programs, processes, or methods described herein are not related or limited to any particular type of computing environment, unless indicated otherwise. Various types of general purpose or specialized computing environments may be used with or perform operations in accordance with the teachings described herein. Elements of the described embodiment shown in software may be implemented in hardware and vice versa.
In view of the many possible embodiments to which the principles of our invention may be applied, we claim as our invention all such embodiments as may come within the scope and spirit of the following claims and equivalents thereto.
This application claims priority to U.S. Provisional Application No. 61/825,993, filed May 21, 2013, the disclosure of which is hereby incorporated by reference in its entirety.
Number | Date | Country | |
---|---|---|---|
61825993 | May 2013 | US |