Patent Grant
11907275
The invention relates generally to the field of communications. One aspect of the invention relates to a communications server apparatus for processing text data for disabbreviation of text units. Another aspect of the invention relates to a communications device, and a system, for processing text data for disabbreviation of text units. Other aspects of the invention relate to a method for processing text data for disabbreviation of text units and computer programs and computer program products comprising instructions for implementing the method.
Data processing in communications systems is well known to the art. One example of data processing used in communications systems is processing of the data and information which facilitate text-based communications, as opposed to audio-based communications. Previously considered techniques have addressed processing text data in order for communications systems to work as efficiently as possible, and to reduce bandwidth use and computational processing.
Some such techniques address text units (such as words) in text-based communications, by processing data elements of the text data. Certain previously considered text data processing techniques have for example attempted to determine whether text units appearing in text-based communications are canonical to a text corpus, database or dictionary. Others have addressed determining whether non-canonical text units can be converted into canonical forms.
However, such previously considered methods have typically used rudimentary techniques for comparing data for non-canonical and canonical text units, or have provided highly complex techniques which nevertheless are susceptible to false positives and negatives. Such methods have had particular difficulty with abbreviated text units, such as abbreviations of common words typically found in text-based communications.
Aspects of the invention are as set out in the independent claims. Some optional features are defined in the dependent claims.
Implementation of the techniques disclosed herein may provide significant technical advantages. For example, far greater accuracy can be achieved in decoding or determining a correct or canonical text unit or word for an abbreviated text unit in text data from a text-based communication.
In at least some implementations, the techniques disclosed herein allow for decoding or disabbreviating text units which would otherwise be indecipherable, or at least too difficult for previously considered techniques to address, without an unacceptable level of false positives/negatives. Moreover, these techniques allow for the greater accuracy and efficiency of any following data processing, such as textual analysis for facilitating features of a user interface or other feature of a communications device, compression or packetisation of communications, translation of text, and the like.
In an exemplary implementation, the functionality of the techniques disclosed herein may be implemented in software running on a handheld communications device, such as a mobile phone. The software which implements the functionality of the techniques disclosed herein may be contained in an “app”—a computer program, or computer program product—which the user has downloaded from an online store. When running on the, for example, user's mobile telephone, the hardware features of the mobile telephone may be used to implement the functionality described below, such as using the mobile telephone's transceiver components to establish the secure communications channel for receiving text-based communications, and using the mobile telephone's processor(s) to determine candidate text for abbreviated text units in the text data.
The invention will now be described, by way of example only, and with reference to the accompanying drawings in which:
Referring first to
Communications server apparatus 102 may be a single server as illustrated schematically in
Service provider communications device 104 may comprise a number of individual components including, but not limited to, one or more microprocessors 128, a memory 130 (e.g. a volatile memory such as a RAM) for the loading of executable instructions 132, the executable instructions defining the functionality the service provider communications device 104 carries out under control of the processor 128. Service provider communications device 104 also comprises an input/output module 134 allowing the service provider communications device 104 to communicate over the communications network 108. User interface 136 is provided for user control. If the service provider communications device 104 is, say, a smart phone or tablet device, the user interface 136 will have a touch panel display as is prevalent in many smart phone and other handheld devices. Alternatively, if the service provider communications device is, say, a conventional desktop or laptop computer, the user interface may have, for example, conventional computing peripheral devices such as display monitors, computer keyboards and the like. The service provider communications device may, for example, be a device managed by a text data processing service provider.
User communications device 106 may be, for example, a smart phone or tablet device with the same or a similar hardware architecture to that of service provider communications device 104.
Further, there is also provided a method, performed in a communications server apparatus 102 processing text data for disabbreviation of text units, the method comprising, under control of a processor 116 of the server apparatus: receiving (202) text data comprising at least one text data element associated with an abbreviated text unit; comparing (204) the at least one text data element with a plurality of candidate text data elements from a representation of a given text database, each candidate text data element associated with a respective candidate text unit in the database; determining (206) values for a similarity measure between the at least one text data element and the candidate text data elements; processing (208) candidate text data elements to select candidate text data elements with associated candidate text units having an ordered relationship with the abbreviated text unit; and using (210) the similarity measure values and the candidate text data element selections to nominate an associated candidate text unit as a disabbreviated text unit for the abbreviated text unit.
Yet further, there is also provided a communications system processing text data for disabbreviation of text units, comprising communications server apparatus (102), at least one user communications device (106) and communications network equipment (104, 108) operable for the communications server apparatus and the at least one user communications device to establish communication with each other therethrough, wherein the at least one user communications device (104, 106) comprises a first processor and a first memory, the at least one user communications device being configured, under control of the first processor, to execute first instructions stored in the first memory to: receive text data comprising at least one text data element associated with an abbreviated text unit, and wherein the communications server apparatus (102) comprises a second processor and a second memory, the communications server apparatus being configured, under control of the second processor, to execute second instructions stored in the second memory to: compare (204) the at least one text data element with a plurality of candidate text data elements from a representation of a given text database, each candidate text data element associated with a respective candidate text unit in the database; determine (206) values for a similarity measure between the at least one text data element and the candidate text data elements; process (208) candidate text data elements to select candidate text data elements with associated candidate text units having an ordered relationship with the abbreviated text unit; and use (210) the similarity measure values and the candidate text data element selections to nominate an associated candidate text unit as a disabbreviated text unit for the abbreviated text unit.
As noted above, techniques described herein relate to processing text data to decode or disabbreviate text units, such as words, found in a text-based communication or message, which have been abbreviated. The disabbreviation allows for example the units or words to be interpretable by a further processing step, such as text analysis or translation, or to be displayable in the interpreted, non-abbreviated form to a user of a communications device receiving the communication or message, for example via a display device of the communications device.
Instances of abbreviation of words or text units in communications messages are found in a variety of settings and media, but are particularly used in text-based communications, such as email, text or SMS messages, messages via social media platforms and the like, between users of computers and electronic devices. For instance, when typing short messages on handheld electronic devices which are to be transmitted to a recipient, users often type a simplified version of a word or phrase when they assume that the meaning of the word/phrase will be clear to the recipient in the simplified form. For example:
Ad hoc shortenings are especially prevalent in some languages. Typical patterns include leaving out some characters, such as vowels: for example other forms of ‘thanks’ as noted above; in Indonesian ‘sy sdh smp’ meaning ‘saya sudah sampai’ (I already arrived). Other patterns may include, in languages which use them, omitting diacritical marks: in Vietnamese, ‘5 phut’ meaning ‘five phút’ (five minutes).
Techniques described herein are directed to processing the text data and/or the data representing or underlying the text units (words, syllables, morphemes, and the like) in such messages, in order for example to convert abbreviated forms of words into their unabbreviated forms, thereby mapping abbreviated or non-canonical input text to the proper canonical form.
As noted above, the techniques described herein provide technical advantages in the data processing and communications realms, such as increasing efficiency and providing greater accuracy for follow-on text data processing applications. They also of course allow for easier interpretation of messages by users. Other potential applications for these techniques are:
One broad form example technique aims to use a combination of two or more of:
For example, in its simplest form step c. can be completed on the basis of selecting the candidate word/phrase which is the most frequently found in the body of reference work: e.g. the word/phrase which has the highest number of instances in Wikipedia.
For step a. one option is to train a heuristic model on a text database, and to compare the abbreviated input text unit to the text database by comparing it to the modelled data. For example, in a vector space model of a text database (as described in more detail below), a vector found for the input text unit can be compared to neighbouring vectors for canonical words in the modelled database, and a score derived for each candidate word/phrase. The most likely canonical form of the input text may be the word/phrase with the highest score.
One example of a similarity measure for step a. above is as follows.
Suppose character replacement for a text unit (in this case a word) W gives the candidate targets {W1, W2, . . . Wn}. That is, all of the Wi can be converted into W by removing characters (we can for instance stipulate that the removed characters are vowels) or diacritics. A comparison of the Wi to W can return direct matches. Then a similarity score sim(A, B) can be used to select which of the Wi is most similar to the source word W. That is, for each of the Wi, we calculate sim(W, Wi) and select the Wi with the largest similarity score.
In one technique, a number of similar candidates may be selected, for further processing (such as steps b. and/or c.). For example, candidates may be classified or ranked by similarity score, and processed in a ranked order, or only those above a given similarity score threshold processed.
It should be noted that there are many such text similarity measures known to the art. Some work by comparing how similar the distributions of words are in a given text corpus or database.
One way to do this is to construct a vector space model of a text corpus. As is well known to the art this can be done by representing the text corpus in a multi-dimensional space, with each word or text unit having a separate dimension, and a count made of the frequency of each word in the corpus to give the vector value. For any input text unit, the corresponding vector can be found in the vector space, and then a measure of the similarity of that vector to neighbouring vectors can be computed. For instance, a cosine similarity can be calculated—a representation of the angle between two vectors in the vector space.
Thus a word which in a given corpus is often found with another word (such as an abbreviation of that word, removing letters or diacritical marks) will have a high value for cosine similarity in the vector space model having the corresponding vectors for the two words.
Another similarity measure can compute a simple orthographical or lexical similarity between text units; for example, are the text units similar length, do they have the same number of vowels, consonants, and the like.
Of the data elements Ei, there is a text data element or element(s) Ex (306) which is associated with the text unit “ppl” (308) from the message (“2 ppl, 2 luggage . . . ”). Thus the data element(s) Ei can be used in the text data processing, in order to find a disabbreviation for the text unit “ppl”. In one example, the processing step will involve finding the vector associated with “ppl” in the vector space model trained on the corpus text, and finding neighbours by cosine similarity for that vector.
For a technique in an example of step c. above, a frequency of occurrence of candidate text units in a text database is determined and used for nominating an associated candidate text unit. For example, a Wikipedia corpus may be used, and the frequency of the input text unit in that corpus can be used to help determine which of a number of candidates (for example, those suggested by cosine similarity of neighbouring vectors for the vector associated with the input text unit) is the best choice.
For the vector space model, it can be advantageous to train or generate the model from a corpus which is relevant to the text inputs which are likely to require disabbreviation. For example, a Wikipedia corpus will contain few abbreviations such as ‘thx’; however if the corpus is application specific, for example using a corpus of text-based messages as the training data, where it is text-based messages that will be interpreted, this is likely to have a similar population of abbreviations. Moreover, if the corpus is relevant—for example, if a set of driver messages is used to train a vector space model for later analysing driver messages—results should be improved still further.
However, for frequency of occurrence techniques, the preferred corpus may be a standardised set, so that canonical words are more likely to be present in the relevant context. Therefore in one technique, the text database for the determination of the frequency of occurrence of the associated candidate text unit is a different text database from that for the vector space model. This has the other advantage that such a standardized corpus may be publicly available.
In a technique for step b. above, a candidate text unit for disabbreviation may be selected if it has an ordered relationship with the abbreviated text unit; for example if characters of the abbreviated text unit are a partially ordered set of (or for) the characters of the candidate text unit, or characters of the abbreviated text unit are in a similar order to characters of the candidate text unit, or consonants of the abbreviated text unit are the same or similar to consonants of the candidate text unit, or in the same or similar order.
One difference between these techniques and previous techniques is that most similarity measures are symmetric, i.e., sim(a, b)=sim(b, a). This property is typically undesirable for text normalization, because (for example) we always want to map “dmn” to “dimana”, but we never want to map “dimana” to “dmn”. So we want sim(“dmn”, “dimana”) to be high, but sim(“dimana”, “dmn”) to be low. This can be implemented by only considering pairs where adding vowels back in maps the source to the target. This can be implemented as a filter. So for example, “dimana” can be converted to “dmn” just by removing letters, in this case vowels, so “dimana” is considered a potential replacement for “dmn”. The converse does not hold, so “dmn” cannot be considered a potential replacement for “dimana”.
This combination of similarity measures, of filtering by ordering relationship, and optionally discriminating by frequency in a (different) corpus provides accurate results even when the reduced form of the input word is many characters distant from the target word. Previously considered techniques have found these circumstances difficult; for those that just use vector neighbors, some words would come higher than the correct form; consonant filtering and corpus frequency weighting can corroborate the correct word. Mere orthographic distance may be high for some candidates which are actually correct disabbreviations. Techniques describe herein can allow dissimilar words to be selected as candidates, if they also pass the ordering and frequency comparison stages.
The payload data components of the data record can be processed in the manner described herein in order to find a disabbreviation for “ppl”, and the payload edited or a new data record (422, 426) formed to now include (424) the data components for the disabbreviated text unit “people”.
The input word ‘brp’ is received (502). The first stage is the vector space model or word-embedding similarity step. Here the corpus to be used to train or generate the vector space model is a set of reviews for a travel company, named “Grab”. The reviews are likely to be written in a similar vernacular to the messages being received for disabbreviation.
Words in Grab reviews 504 (the word corpus of users reviewing their journeys, the drivers, so on) are pre-mapped to n-dimensional vectors 506. This is done as a pre-processing stage (as is the word counting for the Wikipedia comparison in 516, 518—see below).
In an alternative the vector model can be trained on a combination of the Grab reviews, and the Wikipedia corpus—this gives a combination of the vernacular expected in the messages, but also the range of the Wikipedia corpus in case some words in either corpus are missing.
Nearest neighbours are identified (508) from the vector model. Scoring is done by a cosine similarity calculation. The following stage (510) listing the candidates can then set a cut off threshold to give, say, the 10 closest hits. This gives a list of the closest neighbours and their respective similarity scores (similarity to the input text).
At this stage optionally an orthographic similarity score can also be calculated, and either used in parallel, or a combined score with the cosine similarity used. The use of orthographic similarity can enable shortenings of words to be compared with the corresponding words in Wikipedia and Grab reviews, even if they don't appear in the Grab reviews. The use of orthographic similarity at this stage may improve the effectiveness of the overall processing technique by reducing the complexity of the following filter stage 512, for example by reducing the number of candidates for filtering.
These results from the similarity score(s) are then filtered by ordering relationship, for example here to select only those which can be derived from the input text by adding something (e.g characters—consonants, vowels) or diacritical marks (diacritics). In this example (512) the filter is for targets with the same consonants in the same order, with one or more vowels added. The results (514) are therefore reduced again.
Next, the Indonesian Wikipedia corpus (516) having been used to find word frequency counts (518), the similarity score is multiplied by some function of the count from Wikipedia. In this example (520) the similarity score for each word pair (input word, each of the closest neighbours) is multiplied by the logarithm of the number of times this word appears in the Wikipedia corpus. The log (2+count) is used for a number of reasons: a word which appears 10 times more than another word is obviously significant, but not necessarily 10 times more significant; the add delta constant allows for useless results for zero instances—log (0) is considered to be not defined, and the steepness of the log curve is useful for low input numbers: log (1) is zero, so that everything will start from at least 2. A word from Grab reviews might have zero instances in the Wikipedia corpus, so 2 is added to it before taking the log. In alternatives, other types of weightings can also be done, e.g. using a square root, or another monotonically increasing function with positive y intercept.
The highest scoring result (522) from this final stage is taken as the most likely canonical form, and therefore as the nominated candidate text.
It will be appreciated that the invention has been described by way of example only. Various modifications may be made to the techniques described herein without departing from the spirit and scope of the appended claims. The disclosed techniques comprise techniques which may be provided in a stand-alone manner, or in combination with one another. Therefore, features described with respect to one technique may also be presented in combination with another technique.
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/SG2019/050266 | 5/15/2019 | WO |
| Publishing Document | Publishing Date | Country | Kind |
|---|---|---|---|
| WO2020/231323 | 11/19/2020 | WO | A |
| Number | Name | Date | Kind |
|---|---|---|---|
| 8930813 | McMaster | Jan 2015 | B2 |
| 9355084 | Doornenbal | May 2016 | B2 |
| 10140272 | Bao | Nov 2018 | B2 |
| 10303746 | Periasamy | May 2019 | B1 |
| 10643276 | Loddo | May 2020 | B1 |
| 11010535 | Periasamy | May 2021 | B1 |
| 11017180 | Gandhi | May 2021 | B2 |
| 11138382 | Podgorny | Oct 2021 | B2 |
| 20070220037 | Srivastava et al. | Sep 2007 | A1 |
| 20080059421 | Baartman et al. | Mar 2008 | A1 |
| 20090248401 | Grabarnik et al. | Oct 2009 | A1 |
| 20100145676 | Rogers | Jun 2010 | A1 |
| 20120084076 | Boguraev et al. | Apr 2012 | A1 |
| 20160041990 | Dabah et al. | Feb 2016 | A1 |
| 20160098386 | Rangarajan Sridhar | Apr 2016 | A1 |
| 20160103808 | Anders et al. | Apr 2016 | A1 |
| 20160350652 | Min et al. | Dec 2016 | A1 |
| 20170091164 | Bao et al. | Mar 2017 | A1 |
| 20170193091 | Byron et al. | Jul 2017 | A1 |
| 20180089172 | Needham | Mar 2018 | A1 |
| Number | Date | Country |
|---|---|---|
| 104731771 | Jun 2015 | CN |
| 2010-152561 | Jul 2010 | JP |
| 2018-055491 | Apr 2018 | JP |
| Entry |
|---|
| European Search Report for Application No. 19928628.7, dated Nov. 15, 2022 (11 pages). |
| Glass, et al. “Language Independent Acquisition of Abbreviations”, Arxiv. Org, Cornell University Library, 201 Olin Library Cornell University Ithaca, NY 14853, Sep. 23, 2017, (9 pages). |
| Charbonnier, et al., “Using Word Embeddings for Unsupervised Acronym Disambiguation”, Proceedings of the 27th International Conference on Computational Linguistics, Aug. 20, 2018, pp. 2610-2619. |
| Written Opinion for PCT/SG2019/050266, dated Aug. 23, 2019, 5 pages. |
| Damdoo, et al., “Probabilistic Language Model for Template Messaging Based on Bi-Gram”, Conference: IEEE-International Conference on Advances in Engineering, Science and Management, Conference Date: Mar. 30-31, 2012, 3 pages. |
| Jose, et al. “Lexico-Syntactic Normalization Model for Noisy SMS Text”, Conference: 2014 International Conference on Electronics, Communication and Computational Engineering (ICECCE), Conference Date; Nov. 17-18, 2014, 3 pages. |
| International Search Report for PCT/SG2019/050266, dated Aug. 23, 2019, 4 pages. |
| De Vries, “Indonesian Abbreviations and Acronyms”, In Bijdragen tot de Taal-, Land-en Volkenkunde 126, (1970), No. 3, Leiden, pp. 338-346, 10 pages. |
| Fodden, “ IBM Granted Patent for . . . Dict.?”, Website: Slaw, Canada's online Legal Magazine Publication, Date: Dec. 31, 2009, 2 pages. |
| Hidayatullah, et al., “Pre-processing Tasks in Indonesian Twitter Messages”, International Conference on Computing and Applied Informatics 2016, IOP Conf. Series: Journal of Physics: Conf. Series 801 (2017), 012072, pp. 1-6, 7 pages. |
| “Bahasa Indonesia SMS Abbreviations Explained”, Bahasa Indonesia SMS Abbreviations Explained—Bail Manual, Apr. 24, 2018, pp. 1-8, 8 pages. |
| Japanese Notice of Reasons for Refusal for Application No. 2021-568104, dated Feb. 28, 2023 (19 pages). |
| Sekiguchi, Yuichiro, “Clipped Word Extraction using Blog Documents,” DBSJ Letters vol. 6, No. 2, Sep. 2007, pp. 21-24. |
| Number | Date | Country | |
|---|---|---|---|
| 20220237217 A1 | Jul 2022 | US |
