Patent Grant
10740572
The present invention generally relates to language translation systems, and more particularly to a method of providing more comprehensive parallel linguistic corpora for such systems.
Modern language translation systems rely on a variety of technologies to enable quick and accurate translation of phrases and sentences. Among these technologies are natural language processing and cognitive analysis. Natural language processing (NLP) is a field of computer science, artificial intelligence, and linguistics concerned with the interactions between computers and human (natural) languages, not limited to English. Many challenges in NLP involve natural language understanding, that is, enabling computers to derive meaning from human or natural language input, and others involve natural language generation allowing computers to respond in a manner familiar to a user. For example, a non-technical person may input a natural language question to a computer system, and the system intelligence can provide a natural language answer which the user can hopefully understand. NLP uses text analysis such as a text annotator that searches text documents (corpora) and analyzes them relative to a defined set of tags. The text annotator can generate linguistic annotations within the document to tag concepts and entities that might be buried in the text. NLP can also involve relationship extraction, i.e., detecting semantic relations within a set of artifacts. Examples of advanced computer systems that use natural language processing include virtual assistants, Internet search engines, and deep question answering systems.
A cognitive system (sometimes referred to as deep learning, deep thought, or deep question answering) is a form of artificial intelligence that uses machine learning and problem solving. Cognitive systems often employ neural networks although alternative designs exist. A modern implementation of artificial intelligence is the IBM Watson™ cognitive technology, which applies advanced natural language processing, information retrieval, knowledge representation, automated reasoning, and machine learning technologies to the field of open domain question answering. Such cognitive systems can rely on existing corpora and analyze them in various ways in order to extract answers relevant to a query, such as person, location, organization, and particular objects, or identify positive and negative sentiment. Different techniques can be used to analyze natural language, identify sources, find and generate hypotheses, find and score evidence, and merge and rank hypotheses. Models for scoring and ranking the answer can be trained on the basis of large sets of question (input) and answer (output) pairs. The more algorithms that find the same answer independently, the more likely that answer is correct, resulting in an overall score or confidence level.
Machine language translation systems typically implement phrased-based translations that have limited sensitivity to morphological, syntactical and/or semantic differences between the source and target languages. The process of customizing (i.e., training) a phrased-based statistical machine translation system commonly uses parallel (bilingual) corpora—the exact same material written in two different languages—to prioritize the statistical hits of correct translations. There exist large corpora linking multiple languages, for example, the twenty-one languages in which the European Parliament proceedings are recorded, the six language set (English-French-Spanish-Russian-Chinese-Arabic) of United Nations resolutions, and various government proceedings such as Canadian which are recorded in both English and French. The Christian bible is also very useful as it was, until recently, the most widely translated book, particularly in many obscure languages.
The present invention in at least one embodiment is generally directed to a method of building a parallel linguistic corpus for a translation system by receiving robot plans which contain verbal instructions in different languages relating to control of kinematic robots, identifying a first one of the robot plans that includes a first verbal instruction in a first language, identifying a second one of the robot plans that includes a second verbal instruction in a second language different from the first language, determining that the first robot plan and the second robot plan are directed to a substantially identical task, and responsively storing the first verbal instruction in association with the second verbal instruction as part of the parallel linguistic corpus. For example, the first verbal instruction can be a first plan name for the first robot plan and the second verbal instruction can be a second plan name for the second robot plan. The plans can have steps such that the first robot plan includes a first series of steps having first step names and the second robot plan includes a second series of steps having second step names, in which case the first step names can be stored in respective association with the second step names as part of the parallel linguistic corpus. If the first robot plan has a missing step as compared to the second robot plan, a request can be issued to a linguistic expert to provide an additional step name in the first language for the missing step. The robot plans can be modeled as finite state automata having a plurality of links representing sensory states and a plurality of nodes representing motor actions. The determination of plan similarity would thus be based on a comparison of a first finite state automata of the first robot plan to a second finite state automata of the second robot plan. Plans can be considered as directed to a similar task when they each include substantially the same actions in substantially the same order. A parallel linguistic corpus so built can be applied by using it to train a cognitive translation system, so the cognitive translation system can translate a request from the first language into the second language.
The above as well as additional objectives, features, and advantages in the various embodiments of the present invention will become apparent in the following detailed written description.
The present invention may be better understood, and its numerous objects, features, and advantages of its various embodiments made apparent to those skilled in the art by referencing the accompanying drawings.
The use of the same reference symbols in different drawings indicates similar or identical items.
Modern high-quality language translation systems rely on parallel corpora for training. For some language pairs, such as English-French, this is easy to arrange since there are so many sources of parallel corpora. However, for other language pairs (especially in developing or third-world nations) there are no such parallel documents, or there are very few. This is especially true if the countries where the native speakers reside are far apart geographically. For instance, there is very little parallel material between Quechua (Peru) and Swahili (Tanzania) or Vietnamese. Moreover, there is some difficulty in aligning the words within existing corpora due to head-modifier order, complex case structure, gender, honorifics, ergativity, irregular verbs, and idioms.
It would, therefore, be desirable to devise an improved method of supplying parallel linguistic corpora for translation systems. It would be further advantageous if the method could be easily applied to a wider range of available languages to acquire multifaceted corpora. The present invention in its various embodiments achieves these and other advantages by using plans for kinematic robots to generate bilingual reference sources. If robots have been trained in similar tasks in two different languages, then the verbal instructions associated with these tasks can form the needed parallel corpus. If two plans are keyed off linguistic phrases that are in different languages but are equivalent then recognizing that the plans are substantially the same suggests that these two phrases are parallel. Moreover, if a particular procedure is broken down into multiple steps then these steps (or sub-steps) can also be put into correspondence to enlarge the size of the parallel corpus. This approach enables machine translation between two languages even if there are no governmental or cultural documents which are written in both languages.
With reference now to the figures, and in particular with reference to
MC/HB 16 also has an interface to peripheral component interconnect (PCI) Express links 20a, 20b, 20c. Each PCI Express (PCIe) link 20a, 20b is connected to a respective PCIe adaptor 22a, 22b, and each PCIe adaptor 22a, 22b is connected to a respective input/output (I/O) device 24a, 24b. MC/HB 16 may additionally have an interface to an I/O bus 26 which is connected to a switch (I/O fabric) 28. Switch 28 provides a fan-out for the I/O bus to a plurality of PCI links 20d, 20e, 20f These PCI links are connected to more PCIe adaptors 22c, 22d, 22e which in turn support more I/O devices 24c, 24d, 24e. The I/O devices may include, without limitation, a keyboard, a graphical pointing device (mouse), a microphone, a display device, speakers, a permanent storage device (hard disk drive) or an array of such storage devices, an optical disk drive which receives an optical disk 25 (one example of a computer readable storage medium) such as a CD or DVD, and a network card. Each PCIe adaptor provides an interface between the PCI link and the respective I/O device. MC/HB 16 provides a low latency path through which processors 12a, 12b may access PCI devices mapped anywhere within bus memory or I/O address spaces. MC/HB 16 further provides a high bandwidth path to allow the PCI devices to access memory 18. Switch 28 may provide peer-to-peer communications between different endpoints and this data traffic does not need to be forwarded to MC/HB 16 if it does not involve cache-coherent memory transfers. Switch 28 is shown as a separate logical component but it could be integrated into MC/HB 16.
In this embodiment, PCI link 20c connects MC/HB 16 to a service processor interface 30 to allow communications between I/O device 24a and a service processor 32. Service processor 32 is connected to processors 12a, 12b via a JTAG interface 34, and uses an attention line 36 which interrupts the operation of processors 12a, 12b. Service processor 32 may have its own local memory 38, and is connected to read-only memory (ROM) 40 which stores various program instructions for system startup. Service processor 32 may also have access to a hardware operator panel 42 to provide system status and diagnostic information.
In alternative embodiments computer system 10 may include modifications of these hardware components or their interconnections, or additional components, so the depicted example should not be construed as implying any architectural limitations with respect to the present invention. The invention may further be implemented in an equivalent cloud computing network.
When computer system 10 is initially powered up, service processor 32 uses JTAG interface 34 to interrogate the system (host) processors 12a, 12b and MC/HB 16. After completing the interrogation, service processor 32 acquires an inventory and topology for computer system 10. Service processor 32 then executes various tests such as built-in-self-tests (BISTs), basic assurance tests (BATs), and memory tests on the components of computer system 10. Any error information for failures detected during the testing is reported by service processor 32 to operator panel 42. If a valid configuration of system resources is still possible after taking out any components found to be faulty during the testing then computer system 10 is allowed to proceed. Executable code is loaded into memory 18 and service processor 32 releases host processors 12a, 12b for execution of the program code, e.g., an operating system (OS) which is used to launch applications and in particular the parallel linguistic corpus generation program of the present invention, results of which may be stored in a hard disk drive of the system (an I/O device 24). While host processors 12a, 12b are executing program code, service processor 32 may enter a mode of monitoring and reporting any operating parameters or errors, such as the cooling fan speed and operation, thermal sensors, power supply regulators, and recoverable and non-recoverable errors reported by any of processors 12a, 12b, memory 18, and MC/HB 16. Service processor 32 may take further action based on the type of errors or defined thresholds.
The present invention may be a system, a method, and/or a computer program product. The computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present invention.
The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.
Computer readable program instructions described herein can be downloaded to respective computing/processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and/or a wireless network. The network may comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. A network adapter card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing/processing device.
Computer readable program instructions for carrying out operations of the present invention may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the “C” programming language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present invention.
Aspects of the present invention are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer readable program instructions.
These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the function/act specified in the flowchart and/or block diagram block or blocks.
The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions/acts specified in the flowchart and/or block diagram block or blocks.
The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts or carry out combinations of special purpose hardware and computer instructions.
Computer system 10 carries out program instructions for a process that uses novel analysis techniques to build parallel linguistic corpora which can in turn be used by cognitive translation services. Accordingly, a program embodying the invention may additionally include conventional aspects of various cognitive system tools, and these details will become apparent to those skilled in the art upon reference to this disclosure.
Referring now to
Robot plan 50 is provided in any particular language X and can include a variety of information. At the minimum, robot plan 50 includes some verbal instruction which can be used to identify the particular task or type of task that is to be carried out. The task information can be a plan name or other linguistic phrase. For example, for a recreational robot a task might be “Kick the ball”. This plan name is provided in the particular language X of this plan. Accordingly, when two plans are keyed off linguistic phrases in different languages and it can be recognized that the plans are the substantially identical (e.g., substantially the same actions in substantially the same order), then these two phrases can be considered parallel. The two tasks from a candidate pair of plans need not be exactly the same to invoke association. Substantially identical tasks can also qualify for correspondence of instructions. The question of how similar the tasks must be is resolved by the system designer, e.g., as long as it is discernable that some fraction of the steps involved in the task are the equivalent.
In this implementation robot plan 50 has additional information including, but not limited to, a robot identifier, time-ordered steps for at least one procedure with corresponding step names, motor actions of kinematic robot 52, and sensory states of kinematic robot 52. The identifying information may for example include a numeric value (robot ID number), or a robot name. This identifier may be explicit within the robot plan electronic document or may be inferred from a file name of the document. The robot identifier can help buttress the conclusion that two plans are directed to the same task. i.e., when they are applied to the same robot. Step names can be provided when a procedure is broken down into pieces. These steps (or sub-steps) can also be put into fine-grained correspondence to enlarge the size of the parallel corpus.
The robot plans are fed to plan comparison logic 64 which may reside in a computer such as computer system 10. Plan comparison logic 64 may be algorithmic in nature, or may be embodied in a separate cognitive system programmed to carry out natural language programming in multiple languages, i.e., the cognitive system can generate a confidence score regarding the semantic similarity different plans and if the score exceeds a predetermined threshold then the two plans are considered sufficiently similar to warrant pairing. Similarity can also be based on fixed rules. The results are multiple pairs 66a, 66b, 66c of robot plans, each pair comprising robot plans in two different languages. Plan pairs for the same language pair (e.g., Quechua and Swahili) are stored in a single parallel corpus 68 for later use by the cognitive translation system.
In one implementation, plans are represented as finite state automata (FSA) with links representing sensory states and nodes representing motor actions. If the hardware of two systems is similar, then the representation of the sensory states and motor actions will be comparable, irrespective of the language used to create the plan. This allows two FSAs to be put into correspondence in a node-to-node and link-to-link fashion. If source language annotations have been associated with each node and each link, these can be considered to have the same semantic import. Thus the associated phrases in the different teaching languages can be used to form a parallel corpus between the two languages.
As seen in the example of
The idea is that users will teach the robot (from scratch) to do things in whatever native language they use. This instruction will generally consist of a verb-like object and associated noun-like objects (which are typically taught independently, e.g., by pointing). After teaching, the phrase in their native language is “grounded” in a particular set of robot actions. Because the invention assumes that the robots are identical everywhere in the world (e.g., Sears model T3201) then the actions are independent of language. That is, the same sequence of machine codes (e.g., “fcn-134→fcn-933→fcn-005→fcn-784”) is produced no matter where in the world the robot is. This sequence forms the key for matching parts of one language to parts of another language. So, for example, the Maori robot knows the “meaning” of the language phrase in terms of what actions to perform.
While there may be n primitive actions, there may be as many as n4 four-step procedures. Therefore matching sequences is more powerful than just matching individual primitives.
The present invention recognizes that in some cases near identical plans might not have the exact same number of steps, i.e., one plan could still have extra actions. So a (n+1) step procedure in one language (L1) may have a near match with an n step procedure in another language (L2). In this case, the plan comparison logic 64 could issue a request to a teacher of language L2 (or other subject matter expert) for a description of the missing step in the overall procedure. At that point, it can update the language correspondence corpus.
Once parallel linguistic corpora have been so aggregated, they can be applied as depicted in
For example, an English-speaking doctor may have asked a Maori person “What hurts” (translated to Maori). The Maori person responds “waewae” which is a term unknown to the doctor, and so is input as the query to the translation program. From the phrase associated with nodes A1=A1′ in
The present invention may be further understood with reference to the chart of
The present invention thereby offers a significant improvement in the technical field of cognitive translation. In the future as more domestic and industrial robots are deployed to an ever-wider variety of areas and taught procedures in native languages, the invention will build increasingly more sophisticated linguistic corpora. This leveraging of data is easily exploited as an interlingua which helps relate two languages. The approach of the present invention enables machine translation between two languages even if there are no governmental or other documents which are written in both languages. Moreover, the sub-steps of a plan provide a natural fine-grained alignment of the two sets of linguistic terms.
Although the invention has been described with reference to specific embodiments, this description is not meant to be construed in a limiting sense. Various modifications of the disclosed embodiments, as well as alternative embodiments of the invention, will become apparent to persons skilled in the art upon reference to the description of the invention. For example, while the invention has been described in the context of a recreational robot, it is of course useful with industrial robots. The invention could theoretically be expanded beyond kinematic robots to purely software bots as well, provided those bots have plans which can be similarly analyzed. It is therefore contemplated that such modifications can be made without departing from the spirit or scope of the present invention as defined in the appended claims.
| Number | Name | Date | Kind |
|---|---|---|---|
| 7475390 | Berstis et al. | Jan 2009 | B2 |
| 7493559 | Wolff et al. | Feb 2009 | B1 |
| 8122424 | Berstis et al. | Feb 2012 | B2 |
| 20050005266 | Datig | Jan 2005 | A1 |
| 20050065652 | Sakagami | Mar 2005 | A1 |
| 20050273317 | Brill | Dec 2005 | A1 |
| 20080262826 | Pacull | Oct 2008 | A1 |
| 20090299729 | Quirk et al. | Dec 2009 | A1 |
| 20130103390 | Fujita et al. | Apr 2013 | A1 |
| 20150248401 | Ruvini et al. | Sep 2015 | A1 |
| Number | Date | Country | |
|---|---|---|---|
| 20190340249 A1 | Nov 2019 | US |
