Patent Application
20120036147
The present invention relates generally to computer network-based messaging systems, including so-called chat rooms, and more specifically to a method of filtering undesirable words and phrases in a chat room and to a messaging system including a message filter incorporating such method.
With the evolution of increasingly sophisticated Internet tools and the widespread availability of broadband Internet connections, the World Wide Web (“Web”) experience is moving steadily beyond the passive dissemination of information, towards real-time interaction between simultaneous users. Virtual communities exist for groups that share every conceivable interest, hobby, or profession. Increasingly more people of all ages use the Internet as a place to meet and to interact with other people for work and for play. As a consequence, chat rooms have become ubiquitous on the Internet, and accordingly, the maintenance of behavioral standards and safety, especially for young people and minors, is becoming a major societal concern.
Consequently, administrators of chat sites face challenges related to maintaining standards of conduct to prevent a site from degenerating into a forum for types of discussion that were never intended. One such challenge is how to maintain standards within an environment like the Internet where the participants are anonymous and therefore cannot be held accountable with traditional methods. Around-the-clock real-time monitoring by the administrator is not economically feasible for most Internet businesses.
One common approach is to use basic word filters to eliminate offensive words and profanity from the chat conversation. Such filters, often referred to as “black lists” or “exclusive” filters, generally involve blocking words and/or phrases contained in a black list or exclusive dictionary. Unfortunately such simplistic exclusive dictionary approaches can never be exhaustive and are easily outwitted by the well-known practice of “obfuscation.” This practice, also known by other labels such as “creative misspelling,” involves users purposely misspelling words or phrases in a chat message in order to prevent the detection of the words or phrases by exclusive word filters, while still permitting the meaning of the words to be understood by human readers.
Another limitation of these exclusive filters is that, depending on the needs of a chat site, certain words and phrases that are neither profanity nor generally offensive may need to be discouraged in order to preserve certain specific site standards. For example, in a community site for children who do not fully grasp the importance of password safety, phrases like “what's your password,” “gimme your pass,” and “my password is” need to be discouraged. Furthermore, these needs arise dynamically out of the needs of a community and continually evolve. Accordingly, the ongoing task of maintaining an exclusive dictionary of all prohibited words and phrases requires devoting a tremendous amount of resources to constantly monitoring chat room conversations and updating the exclusive dictionary as new offensive language evolves. This is not practical or economically feasible for most Internet businesses. Furthermore, there will always be some delay in updating the exclusive dictionary based on monitoring, since it is nearly impossible for the administrator to anticipate all of the creative misspellings and word combinations that will need to be blocked.
Another approach is to use a more restrictive form of message filtering or chat filtering, known is “white list” or “inclusive” filtering. Such filters prevent the use of offensive words, as well as the use of word obfuscation, since they only allow words contained in a list of permitted words that have been preapproved for use in chatting. This list of permitted words in often referred to as a “white list” or “inclusive dictionary.” However, in order to effectively block the use of offensive language, the inclusive dictionary must be kept relatively small, which can significantly hinder the ability of users to effectively chat with each other. Further, users often adopt “workarounds” by using permitted words in a context that implies offensive or otherwise inappropriate messages.
As described in U.S. Patent Application Publication No. 2009/0228557 to Ganz et al., a hybrid or multiple-layer chat filter was developed which uses a combination of an exclusive dictionary and an inclusive dictionary. While the system described therein overcomes many of the disadvantages of using either method alone, as with other inclusive dictionary-based filters, users are still limited to the words provided in the inclusive diction when formulating a message.
According to an aspect of the present invention, there is provided a computer network-based messaging system for controlling content of messages sent by users in the messaging system. The messaging system comprises: an electronic data storage device containing a database including a plurality of permitted words and at least one replacement text element corresponding to at least one of said plurality of permitted words; a message filter that receives an entered word or phrase input by a user on a messaging device as part of a message, wherein the message filter uses a processor to replace the entered word in the message with a replacement text element stored in the database that matches the entered word; and a message sending part that transmits the message including the replacement text element over a data network for viewing by at least one other user only if every word in the message is: a permitted word included in the database, and/or a word that forms part or all of a replacement text element included in the database. The message sending part transmits the message including the replacement text element even though one or more words contained in the replacement text element are not permitted words included in the database.
According to an aspect of the present invention, there is provided a method of filtering and modifying text messages comprising steps of: receiving a word or phrase as part of a message entered by a user on a data entry device; using a processor for comparing the received word to a plurality of permitted words contained in a database stored in a data storage device; using the processor for determining whether the received word matches one of the plurality of permitted words that has a corresponding replacement text element stored in the database, and retrieving the corresponding replacement text element; based on said determining, automatically using the processor for replacing the received word in the message with the retrieved replacement contained text element; and using the processor for deleting the replacement text element in response to receiving an editing command entered by the user on the data entry device, and preventing any portion of the text element from being deleted from the message unless the entire replacement text element is deleted.
As described above, a limitation of any chat filter that employs an inclusive dictionary (also referred to as a “white list”) is that users are limited to using words included in the inclusive dictionary, which can considerably impede effective communication between users. As described above, users often develop workarounds that circumvent such a filter by combining several permitted words to create offensive or otherwise inappropriate messages. As described in U.S. Patent Application Publication No. 2009/0228557 to Ganz et al. (hereinafter “Ganz '557”), which is hereby incorporated by reference in its entirety, an inclusive dictionary can optionally be used together with an exclusive dictionary (also referred to as a “black list”) to provide a multiple layer or hybrid chat filter. In such a hybrid system, the size of the inclusive dictionary can be expanded to include more words and provide additional flexibility, since offensive phrases composed of several permitted words can be blocked by the exclusive dictionary. Nonetheless, creative users will continue to develop workaround phrases to create offensive messages. The administrator of a chat filter may face the difficult choice of removing certain desirable words from the inclusive dictionary in order to circumvent such workarounds, or to face the challenge of keeping up with users' workaround attempts by continually supplementing the exclusive dictionary as new workarounds are discovered.
According to the present invention, an additional mechanism has been developed to allow the filter administrator to provide more flexibility to users by adding additional words to the inclusive dictionary, which are desirable for use in chatting but are often used as part of workaround phrases to communicate offensive messages. In particular, according to an aspect of the present invention, any word in the inclusive dictionary can be associated with replacement text. The associated replacement text may be another word or a phrase that is, for example, a synonym of the associated word listed in the inclusive dictionary. Thus, when a user types a word that is included in the inclusive dictionary and has replacement text, the word is automatically substituted with the replacement text. In the case where the replacement text is a synonym, if the user was using the word for its intended meaning, then the replacement text should approximate the user's original intent and the user can proceed to send the message without significant inconvenience. On the other hand, if the user was using the word as part of a workaround, such as a substitute for a phonetic equivalent that is not included in the inclusive dictionary, then the user's workaround attempt will have been foiled by the substitution of the replacement text. As an alternative, the system may also be designed so that the substitution occurs after the user instructs the system to send the message without notifying the user that the substitution has been made. As a result, only the recipient of the message will see the replacement text, making it more difficult for users to develop new workarounds.
In an example chat system having an inclusive dictionary filter, a chat room intended for young children might attempt to prevent users from revealing their gender by not including gender specific words such as “girl” in the inclusive dictionary. As a workaround, a creative user might substitute the word “abroad” for “a girl”, allowing her to send the message “I am abroad”, signifying her gender by implying the slang term “broad” as an indication of her female gender. However, by eliminating term “abroad” from the inclusive dictionary, the chat administrator will also inadvertently be preventing a well-intentioned user from using the term “abroad”, for example, to communicate that they are traveling outside of his or her home country. Thus, according to an aspect of the present invention, by including the word “abroad” in the inclusive dictionary and associating it with the replacement text “overseas”, when a user types “I am abroad” the message will automatically be changed to read “I am overseas.” This effectively prevents a user from using the term “abroad” as part of a workaround with significantly inconveniencing a user who is trying to use the term “abroad” for its normal meaning.
Throughout this application, inclusive dictionaries containing only individual words are described. It should be appreciated that, if desired, such inclusive dictionary can also contain multiple-word phrases, each having corresponding replacement text.
Referring to
As illustrated in
Referring still to
As one of ordinary skill in the art will appreciate, a suitable inclusive dictionary can be composed and updated as needed by an administrator of the messaging system 10. An editing tool may be provided to the system for this purpose. For purposes of illustration, a simplified examples of the inclusive dictionary 20 described above, is shown below in Table 1.
As is apparent from the example of Table 1, which shows an excerpted portion of an inclusive dictionary (also referred to as a “word database”), the replacement text entry for each permitted word is optional, so some permitted words do not have corresponding replacement text. As desired, such as to speed up processing, words in the inclusive dictionary may be flagged or otherwise identified as having corresponding replacement text.
It should be appreciated that the schematic diagram of
According to one embodiment of the present invention, as illustrated in the simplified flow diagram of
It should also be appreciated that the illustrated order of processing of steps 100, 102 and 104 are merely representative, and that it is contemplated that steps 100, 102 and 104 could be performed in another suitable order or be performed concomitantly.
It is also contemplated to be within the scope of the present invention that any suitable combination of inclusive and exclusive dictionaries (such as in the hybrid system described in Ganz '557) may be used in combination with the present invention, such that at least one inclusive dictionary would be provided with replacement text as described herein. It is also contemplated that the some or all of the functions of the chat filter 14, as described herein, could be performed by the chat client 26 or other component on the sender's computer 12. Further, it is contemplated that the message server 16 could be eliminated entirely from messaging system 10 such that, for example, the sender's computer 12 and a recipient's computer 24 function in a so-called peer-to-peer mode.
As used herein, the term “type” or “typing” refers generally to methods for manually entering text into data entry devices by selecting individual letters or symbols or other characters. Typing can refer, for example, to entry of text by a user using a traditional keyboard, a keypad, a touch screen, a mouse, or the like, to select letters. Further, as used herein, the term “word” refers to a set of one or more letters, symbols or other characters employed by a user to represent an element of language that can be understood by another user. Words can be, for example, traditional spellings contained in a generally accepted dictionary, such as the Oxford English Dictionary, and/or informal, inventive, nontraditional spellings of words or phrases devised to communicate more efficiently or creatively with other users, including symbols and numeric characters.
The simplified flowchart of
If, at step 202, the typed word does not have associate replacement text in the inclusive dictionary 20, the process proceeds to step 210. At step 210, the chat filter 14 determines if the typed word matches a permitted word listed in the inclusive dictionary (which has no replacement text). If not, then at step 212 the typed word is displayed to the user as highlighted text to indicate that it has been rejected. For example, a rejected typed word can be displayed in a color (e.g. orange) different from the color (e.g. white) of other text in the message being typed. Other suitable methods of highlighting the word may be used as well. The sender may be permitted to continue typing more words (i.e. at step 200), but the message will not be transmitted so long as it contains any rejected (i.e. highlighted) words.
Once the sender is satisfied with the message and it contains only permitted words (step 210) and/or replacement text contained in the inclusive dictionary (step 206) he or she may instruct the message entry interface to transmit the message at step 214. Then, at step 216 the message is transmitted via the Internet 18 to another computer or messaging device, such as the recipient's computer 24, and displayed to another user or users.
It should be appreciated that the computers 12, 24 and messaging devices described herein can be replaced by any suitable device having a microprocessor and data storage that is capable of inputting, transmitting, receiving and displaying a message to a user, for example a desktop, laptop or tablet computer, a cellular phone, a PDA, or the like. The messaging server, as referred to herein, can be any suitable computer system known in the art, such as the one described in U.S. Patent Application Publication No. 2009/0204908 A1 to Ganz, which is incorporated by reference herein. Further, the terms “computer network”, “data network”, “Internet” and the like, as used herein includes any suitable network, combination of networks, or other infrastructure for transmitting data between microprocessor-based devices, including wired, fiber optic, wireless networks and the like.
With reference to the example screenshots of
According to an example embodiment, the send function on the sender's computer 12 will be disabled whenever the text box 30 contains prohibited words. Specifically, the SEND button 32 on the sender's computer is disabled and becomes “grayed out” as long as at least one word in the text box remains highlighted. The sender must remove or change the prohibited word(s) to eliminate the highlighted text. Once the text box 30 no longer contains any highlighted words, the send function is re-enabled and the SEND button 32 becomes active again, allowing the sender to click or otherwise trigger the SEND button 32 to request transmission of the message by the messaging server 16.
According to an alternative embodiment, the disabling of the SEND button and/or the highlighting of prohibited words can be eliminated. It should be appreciated that once a sender knows what words or phrases are prohibited, by virtue of inclusion in the inclusive dictionary, it may become easier for the sender to circumvent the message filtering. Therefore, it is contemplated that the administrator of a chat site may wish to keep some or all of the contents of the inclusive dictionary more secretive. One way that this can be accomplished by eliminating the above-described highlighting function. The highlighting function could be eliminated entirely, or highlighting could be performed only for designated words. Thus, although the SEND button 32 would remain disabled as long as prohibited words or phrases were contained in the text box 30, one or more prohibited phrases may be contained in the text box 30 without highlighting. Since there may be prohibited words in the text box 30 that are not highlighted, the sender will have to modify the message through trial and error until the SEND button 32 becomes re-enabled.
As a further possible modification to the chat room 34, to help keep the contents of the inclusive dictionary more secretive, both the highlighting function and the disabling of the SEND button 32 could be eliminated. In other words, the system would permit the sender to click or otherwise trigger the SEND button 32 even if prohibited words were present in the text box 30, but not actually transmit the message. The message could then be cleared from the text box 30, and other steps could be taken to give the sender the impression that the message was sent, even though the message would not actually be transmitted to any recipient computers. Thus, it would be more difficult for a sender to determine exactly how messages are being filtered by the messaging system 10. As a further alternative, when the user clicks the SEND button 32 with any prohibited words in the text box 30, the message could be censored before it is transmitted, so that a receipt computer displays a modified version of the message, such as with some or all of the words removed, obscured, and/or replaced by replacement text. It should be appreciated by one of ordinary skill in the art that one or more of the above-described alternatives and modifications, as well as other modifications to the operation of the chat room 34, can be utilized as desired without departing from the scope of the present invention.
It should also be appreciated that the text filtering and replacement techniques described herein are not applicable only to messaging systems and chat rooms as presently disclosed. It will be appreciated that the described techniques can be readily adapted to be used in other environments and with other kinds of linguistic systems, such as email systems, word processing systems, voicemail systems, and the like, or wherever it may be desirable to filter, censor or otherwise limit the use of language.
Further, it is contemplated that the replacement text techniques described herein could be used without an inclusive dictionary-based filter and/or together with other kinds of filtering systems and methods, for example with so-called exclusive dictionary-based filters.
Furthermore, it should be evident that this disclosure is provided by way of example and that various changes may be made by adding, modifying or eliminating details without departing from the fair scope of the teaching contained in this disclosure. The invention is therefore not limited to particular details of this disclosure except to the extent that the following claims are necessarily so limited.
