Patent Application
20170344538
Computer programs generally include a graphical user interface through which a computer communicates both state information and prompts for input. To the extent that this communication is in the form of a natural language, such as English, a computer program typically is “localized” to make the computer program accessible in multiple natural languages. In other words, possible outputs generated by the computer program are stored as translations in multiple natural languages. To support localization, a computer program generally is written so that outputs are defined by an identifier of a message and an identifier of the natural language of the output. The computer program then includes a data structure that stores, for each message identifier, the corresponding message in each of several natural languages.
As an example, a common exercise in an introduction to computer programming is to write a computer program that outputs the text “Hello, world”. Such a computer program typically would include an instruction that included a command, such as “output:”, and the phrase “Hello, world”, i.e., “output: ‘Hello, world’”. (Note this example does not use an actual computer programming language). To localize this computer program, one could write instead “output: message[x, y]”, where “message” is a command that returns a message using an identifier “x” of the desired message as an index in a data structure, and where “y” corresponds to a selected natural language. For example, message “1” can represent a message corresponding to the desired “Hello, world” output. Other values can represent other messages of the computer program. In this example, for “y”, let English be “1” and let French be “2”. A data structure is created to store “Hello, world” as message[1,1] and “Bonjour, le monde” as message [1,2].
In practice, the localization of a computer program into a large number of natural languages typically is performed by a team of translators, not a computer programmer. The computer programmer, or a team of programmers, typically provides the initial content for messages of a computer program in one natural language. Then, the data structures which store these messages are provided to translators. The translators often use a computer program that accesses these data structures and updates the data structures with translations input by the translators. The computer program typically presents messages from these data structures through a graphical user interface to a translator, and receives input from the translator providing translations for the messages. The translations then are stored in the data structure.
This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is intended neither to identify key or essential features, nor to limit the scope, of the claimed subject matter.
One challenge for translators, when localizing software, is providing accurate translations for idiomatic expressions, words with multiple meanings, and other ambiguous terms and phrases. For example, if a message in a computer game includes the English word “block”, a translator would need to determine whether the word was intended to signify, for example, a defensive action of a person fighting, or, an object, such as a building component. To address such challenges, contextual information from a computer program being localized is provided to the translator to help the translator to resolve ambiguities in translation. Herein, the computer program being localized is called the target computer program.
Context data from the target computer program is generated by processing outputs presented by the graphical user interface during actual use of the computer program and which are captured during that use. The captured outputs are processed to create the context data. Generally, the context data associates strings, that can be matched to selected text of messages from the graphical user interface of the target computer program, with other content, such as images or audio, that provide context from the graphical user interface of the target computer program. In an example implementation, the context data can be in the form of strings, recognized from images or audio of the graphical user interface of the target computer program captured during actual use of the target computer program, and associations between the strings and the images which are associated with the recognized strings.
A translation editing tool has, as one of its inputs, access to this context data. The translation editing tool also has, as another of its inputs, access to message data of the target computer program to be translated. The message data includes messages, in a first natural language, of the graphical user interface of the target computer program. The translation editing tool presents messages to be translated from the message data through a graphical user interface to a user. In response to selected text of a message, the translation editing tool accesses the context data to retrieve content derived from the graphical user interface of the target computer program and associated with a recognized string that matches the selected text. The retrieved content is presented to the translator through a graphical user interface of the translation editing tool in association with the selected text. Given such a presentation, a translator has contextual information to inform how to translate a message into a second natural language. Text based on input from the translator, in the second natural language, can be stored in the message data.
The computer system includes a capture tool that runs on a first computer running a target computer program. The target computer program has message data, including messages, in a first natural language, of the graphical user interface of the target computer program. The target computer program provides one or more outputs, such as display data at a refresh rate, for its graphical user interface. Other outputs can include audio data, for example. The capture tool accesses the outputs, such as the display data, of the target computer program and stores the accessed outputs in an output file herein called a capture file.
The computer system also includes an analysis tool that runs on a second computer and receives the capture file from the capture tool as an input. The analysis tool processes the capture file to extract context data. For example, the analysis tool can process images to recognize strings in the images. In this example, the analysis tool stores data associating the recognized strings with those images as the context data.
The computer system also includes a translation editing tool that runs on a third computer and receives, as a first input, the context data generated from use of the target computer program and receives, as a second input, the message data of the target computer program. The translation editing tool, in response to translator input, updates the message data with translations of the messages of the target computer program.
An example workflow using such a computer system includes having a first user use the target computer program on the first computer. The capture tool runs during this use and captures data, such as display data, output by the target computer program. After the capture file is created by the capture tool, the capture file is transmitted to the second computer. The analysis tool is run on the second computer to process the capture file and generate the context data. The context data and the message data for the target computer program are transmitted to the third computer. The translation editing tool is run on the third computer using the context data and the message data for the target computer program.
The capture file can be encrypted and the analysis tool can be configured to require authentication before the contents of the capture file can be accessed. The context data and/or message data also can be encrypted, and the translation editing tool can be configured to require authentication before that data can be accessed through the translation editing tool.
In the following description, reference is made to the accompanying drawings which form a part hereof, and in which are shown, by way of illustration, specific example implementations. Other implementations may be made without departing from the scope of the disclosure.
This example computer system includes a first computer 100 running a target computer program 104. The first computer also runs a capture tool 102. The first computer can be implemented using a general purpose computer such as described below in connection with
The target computer program can be any computer program that can be executed on the first computer. Typical examples of such target computer programs include, but are not limited to, computer games, productivity applications such as word processing, spreadsheet, presentation, drawing and other end user applications, operating systems, or any other computer program.
The capture tool captures outputs, such as display data, for the graphical user interface of the target computer program during execution of the target computer program during actual use of the target computer program by a user. The capture tool can be any computer program that can be executed on the first computer, or connected to an output for a presentation device for the first computer, that can capture the output data of the graphical user interface of the target computer program during execution of the target computer program. Other outputs in addition to display data, such as audio data, also can be captured. Data captured by the capture tool can be stored, for example, in a data file in local storage of the first computer, and then transferred to a second computer 110. As an example alternative, the capture tool can store a capture file directly to storage of the second computer. The term “capture file” as used herein is intended to mean any data structure used to store the output data captured from the use of the target computer program.
The example computer system of
The analysis tool is a computer program that executes on the second computer 110. In one example implementation described herein, the analysis tool processes images stored in the capture file. In this implementation, the analysis tool can include an optical character recognition (OCR) component such as an OCR software development kit available from many software publishers, such as, but not limited to: Microsoft Corporation (the MICROSOFT OCR Library for Windows Runtime, or an OCR library that is part of the Windows SDK as Universal Windows Platform (UWP) API), ABBYY (the ABBYY Cloud OCR SDK), Aquaforest Ltd. (the AQUAFOREST OCR SDK), Nuance Communications, Inc. (the OMNIPAGE capture SDK) and Lead Technologies, Inc. (the LEADTOOLS OCR SDK). The OCR component can be on another computer and accessible by the second computer over a computer network or can reside on the second computer.
In another example implementation, the analysis tool processes audio stored in the capture file. This implementation can be an alternative to, or in addition to, processing of images stored in the capture file. In this implementation, the analysis tool can include a speech recognition component, such as a speech recognition software development kit (SDK) available from many software publishers, such as, but not limited to: Microsoft Corporation (the MICROSOFT Speech Platform SDK), Nuance Communications, Inc. (the DRAGON SDK), SRI International (the EDUSPEAK and DYNASPEAK SDK's), and others.
The computer system also includes a third computer 120. The third computer runs translation editing tool 122, described in more detail below in connection with
The computer networks 130, 140 can be any computer network supporting interaction between the end user computers and the shared storage system, such as a local area network or a wide area network, whether private and/or publicly accessible, and can include wired and/or wireless connectivity. The computer network can be implemented using any of a number of available network communication protocols, including but not limited to Ethernet and TCP/IP.
In one example deployment, the second computer can include a plurality of computers implementing one or more server computers and accessible by a plurality of first computers and a plurality of third computers. Such one or more server computers can implement a service employing the analysis tool to generate context data for multiple target computer programs for use by multiple translation editing tools. In one example deployment, the second computer can be configured to allow the third computer to remotely access the context data for a target computer program without transferring all of the context data to third computer.
In one example deployment, the first and second computers can be the same computer. In another example deployment, the second and third computers can be the same computer. In another example deployment, the first, second and third computers are the same computer.
Referring now to
The target computer program 200 has message data 202 including messages for the graphical user interface of the target computer program. The messages generally include text strings in at least a first natural language. The message data can be stored in the form of one or more data files. The target computer program has an output that provides output data 204, such as display data and/or audio, for its graphical user interface. The output data 204 is presented on one or more presentation devices 206, such as a display or speakers.
The capture tool 210 accesses the output data 204 from the target computer program 200 and stores the accessed output data in a capture file 212. For display data, the sample rate of images stored by the capture tool can be equal to or less than a refresh rate of the display data from the target computer program. Any audio data can be recorded at an audio sample rate based on a device used for recording.
The analysis tool 220 receives the capture file 212 as an input. The analysis tool processes the capture file to generate context data 222, which is data derived from the graphical user interface of the target computer program. For example, the analysis tool can recognize strings in the captured data and associate the recognized strings with content derived from the capture file. For example, strings can be recognized in display data using optical character recognition techniques. Strings can be generated by recognizing speech in any recorded audio data. Such strings can be associated with images to which they correspond, e.g., an image from which the string was recognized, or an image displayed at a time corresponding to audio from which the string was recognized. The analysis tool outputs context data 222. In one example implementation, the context data can be stored in the form of a database file that associates recognized strings with other context data, such as an image and a location in that image, providing a database of context information.
The translation editing tool 230 receives the context data 222 as a first input and the message data 202 of the target computer program as a second input. A dashed line 234 indicates that a copy of the message data 202 of the target computer program is provided to the translation editing tool for modification. The translation editing tool has a graphical user interface through which output data 238 is presented to a user, e.g., a translator. The output data can include images from the context data 222 corresponding to strings matching text selected from the message data 232 by the translator. In response to user input 236 of a translation, the translation editing tool updates the message data, as indicated by the updated message data 232, with translations of the messages of the target computer program based on user input. The updated message data 232 can be combined with the target computer program to provide a localized target computer program.
Referring now to
The first computer, using the capture tool, captures 300 output data from a target computer program during actual use of the target computer program. The capture tool can store 302 the captured output data in a data file, herein called a capture file. The capture file is made accessible 304 to an analysis tool, for example by transferring the capture file to a second computer that runs the analysis tool. The analysis tool then processes 306 the capture file to generate the corresponding context data. In turn the context data is made accessible 308 to the translation editing tool. Similarly, the message data for the target computer program is made accessible to the translation editing tool. For example, the context data and message data can be packaged into an encrypted data file that is transferred to the computer that runs the translation editing tool. The translation editing tool, during use in editing translations of the message data of the target computer program, presents 310 content from the context data in response to input of selected text from the message data. The translation editing tool then processes user input of translated messages to update 312 the message data for the target computer program.
More details of an example implementation of such a computer system and its operation will be described below in connection with
The strings recognized in images from the graphical user interface can be represented as one or more records for recognized strings 504. For each recognized string (e.g., “hello world”), a record stores a reference to an image ID 508, which is a unique identifier of the image in which the recognized string is found, and coordinates 510. Coordinates 510 represent at least the location, such as x and y coordinate values, where the recognized string 504 is found within the image having the image ID 508. The coordinates 510 also can include size information for a bounding box in an image that includes the recognized string, such as a height and a width. Each recognized string can have a unique ID 506. For example, the unique ID can be programmatically calculated from the recognized string and the coordinates. In such way, the same string at the same coordinates cannot be added to this database multiple times. A number of characters 507 in the string can be stored. Additionally, a scale factor 509 applied to stored image data also can be stored. While
The context data also can include image data 512, which can include the image data from the capture file or can include a reference to a separate data file storing the images, or to multiple image files that store the image data. Each image ID 508 associated with a recognized string 504 identifies an image stored in this image data. The image ID can be a numerical identifier, or file name, or other information representing the image. In implementations where each image is stored as a separate image file, the image ID can be the image file name. The context data also can include references to any corresponding audio data or other data from, or derived from, the capture file.
The capture file containing output data captured from the target computer program can be stored as a data file containing a sequence of the captured images. The data file can have the format of a conventional video data file. As such, the data file also can store audio data as in a conventional video data file. Alternatively, audio data also can be stored in a separate data file.
The capture file can be encrypted and the analysis tool can be configured to require authentication before the contents of the capture file can be accessed. The context data and/or message data also can be encrypted, and the translation editing tool can be configured to require authentication before this data can be accessed through the translation editing tool.
Turning now to
The capture tool can be implemented using any tool that can capture and store data output by the target computer program running on a computer to a presentation device of the computer. For example, a screen capture application or a recording device connected to a display output of the computer running the target computer program can capture a sequence of images output by an application. A video camera also can record a video of a display. As another example, an audio capture application or other recording device connected to an audio output of the computer can capture audio data. A microphone also can record audio output by the computer.
Recording by the capture tool is activated 600. A user initiates 602 use of the target computer program, and uses 604 the target computer program for a period of time. The start of the use of the target computer and the start of recording can occur in any order. The use of the target computer program should include one or more scenarios or states of the target computer program for which a translator will be completing a translation. The capture tool can terminate 606 recording, in response to any of a variety of events. For example, termination can occur after a set period of time automatically, or can be in response to a user instruction, or can be in response to termination of use of the target computer program. After terminating recording, the capture file including the captured output data can be made accessible 608 to the analysis tool, for example by transferring the capture file to a computer that runs the analysis tool.
Turning now to
The analysis tool includes a control module 700 which accesses a capture file 702 for processing. In this example, the control module 700 accesses selected images from the capture file 702, and provides an image identifier 704 for each individual image 706 from the capture file that is selected for processing. Only a selection of frames, instead of all of the frames, from the capture file can be processed. For example, a sampling of two frames per second can be used to select frames from the capture file. For some target computer programs which do not have frequent updates in the graphical user interface, even fewer frames can be selected for processing. Each image 706 is input to an optical character recognition (OCR) module 708 which processes the image to extract any strings 710 recognized in the image. A string is a data type implemented as a sequence of characters used in a written form of a natural language, and can include numbers and/or symbols, and can be represented using a form of character encoding. The optical character recognition module 708 can receive external inputs 712, such as user input or text from the message data of the target computer program, or other data, to assist in recognizing strings in an image. The recognized strings 710 can be output as one or more strings, with each string comprising one or more words, and may be associated with coordinates indicating a position in the image at which the string is located. Any conventional optical character recognition program that processes an input image and generates one or more strings per image can be used for this purpose. The recognized string 710 for an image, and the image identifier 704 for that image, are input to an output module 714. The output module 714 generates and stores records 720 that associate data about recognized strings 718, such as the recognized string and coordinates of each string, with an image identifier 704 for the image in which the recognized string was found, in the context data 716.
More particularly, each recognized string can have associated data, such as, but not requiring or limited to, an image identifier of the image, coordinates of the string in the image, and a unique identifier. The output module adds a record for each recognized string into the context data, such as one described in
Usually, sequential images generated by the graphical user interface of the target computer program, as captured in the capture file, are very similar and contain many of the same strings in the same position. Various techniques can be used to avoid duplicate entries in the context data for the same recognized string. As an example, if having many of the same strings with same coordinates in the context data are not preferred, the output module can compute a unique identifier from at least the recognized string and its coordinates. The output module can ensure that only one record is stored for each such unique identifier.
Images of the graphical user interface of the target computer program also can consume a substantial amount of computer storage. To reduce storage requirements for the context data, the output module can apply a scale ratio, which can be a user setting or an automatic setting, and which can be adjustable per image. The scale ratio represents a desired ratio of the size of the original image from the capture file to the size of its corresponding image in the context data. The output module can resize images from the capture file to smaller size images, in terms of pixel resolution, that are stored in the context data. The output module also can apply data compression to further reduce the amount of data used to represent each image. The output module also can store any information about the scale ratio and/or any compression format in the context data.
Each image can have multiple text areas. An OCR module may detect such multiple text areas as individual strings. For example,
To improve quality of matching in the translation editing tool, in one implementation, the output module, in addition to adding the individual strings, adds combinations of concatenated neighboring strings to the context data. A variety of techniques can be used to generate concatenations of strings, such as adding all possible combinations of two or more adjacent lines. Partial strings also may be used. By adding such combinations of strings to the context data, multiline string matching can be performed when the context data is used in the translation editing tool. As shown in
Turning now to
To compare the selected text to the recognized strings stored in the database, the context access module can apply any of a number of string comparison algorithms. For example, because an OCR module can introduce errors in recognized strings, the string comparison algorithm can be implemented so as to execute approximate, or “fuzzy”, matches. As an example, the context data can be implemented using a database that supports full-text searching operations with query operators that can be applied to strings. An approximate matching mechanism can be implemented with query operators that find one or more substrings within a string. For example, a “near” operator can identify strings in which two substrings are near each other: the operation “TermA Near/N TermB” means that TermA and TermB are less than N+1 words apart from each other. As another example, a “match” operation can identify strings including a substring: the operation [Match “Tokyo”] can retrieve records having the string “Tokyo Station” or “Center of Tokyo”. Some systems use a wildcard operator to provide a similar result. As an example, an SQLite database is a relational database that supports full-text search with a “Near” operator. In this example, an initial query on the context data, given the selected text, can retrieve a set of candidate entries. For example, if the selected text is a single word or other string, a “match” query can be applied to the database to retrieve all entries that begin with or that contain the word or string. If the selected text includes multiple words or strings, a “near” query can be built from the words of the selected text and applied to the database to retrieve all entries that contain the words in approximately the same order. Additionally, the candidate entries can be limited by the number of characters in the recognized string as compared to the number of characters in the selected text. For each of the candidate entries, a similarity or distance metric between the recognized string of the entry and the selected text is calculated. A variety of other similarity or distance metrics can be used. For example, any edit distance metric can be used. An example distance metric that can be used is a Levenshtein distance metric. Approximate matches having a measure of distance or similarity over a given threshold can be selected and sorted based on this measure by the context access module, and presented in sorted order by the translation editing tool.
An illustrative example of an image of a graphical user interface for a translation editing tool will now be described in connection with
The graphical user interface 1000 includes a message data display area 1002 for displaying message data from the target computer program. In this example, the message data is in the form of a structured data file, such as an XML file or spreadsheet, which can be readily transformed into a display of rows 1012 and columns 1010. Thus the message data display area includes multiple rows, one for each source message to be translated. In each row, the message data display area includes, for a given message, the word or words 1014 of the message in a first natural language, the word or words for its corresponding translation 1016 into a second natural language, and optionally other information 1018 related to the message. Such other information can include, but is not limited to, message identification information (e.g., a file name for a data file from which message data was retrieved, a message identifier), and status information related to the translation (e.g., has the message been translated and/or has the translation been approved). Any row of the message data display area can be selected in response to user input as a currently selected message; all of, or a portion of, the text within a currently selected message can be selected. The translation editing tool may automatically select a message or portion of a message.
The graphical user interface 1000 also can include an editing display area 1004 for allowing a user to enter a translation for a currently selected message. The editing display area 1004 can include, for example, a source pane 1020 and a translation pane 1022. The source pane can display text in the source natural language for the currently selected message. The translation pane can be an input field through which a user can enter text in the target natural language as the translation to be associated with the currently selected message.
The graphical user interface 1000 also includes a content display area 1006 for presenting content received based on the context data. For example, content display area 1006 can display images resulting from a search of the context data for a recognized string that approximately matches the selected text from the currently selected message. In this content display area 106, an image can be displayed in an image pane 1030. Using the coordinate data for the recognized string in the context data, the recognized string can be highlighted on the displayed image, for example by drawing a box around that text. Additional data about the displayed image can be displayed in a data pane 1032. In this example interface, the data pane includes a search count indicator 1034 which conveys to the user the number of matches to the currently selected text. For example, the search count indicator 1034 can include a total number of images or screens, and an indicator of which image from that set is currently being viewed. The recognized string corresponding to the currently displayed image can be displayed at 1036. A confidence indicator 1038 can be displayed as a value indicative of how well the currently selected text matches the recognized string corresponding to the displayed image, which may be a function of the similarity or distance metric used to identify this string as a match. The actual text 1040 in the source natural language of the currently selected message also can be displayed in the data pane, adjacent to the recognized string, to assist the user in determining how well the currently displayed image corresponds to the currently selected text.
Turning now to
The translation editing tool also can have a mode of operation in which an individual reviews and approves translations. Such a mode of operation is similar to the operation described in
Having now described an example implementation,
The computer can be any of a variety of general purpose or special purpose computing hardware configurations. Some examples of types of computers that can be used include, but are not limited to, personal computers, game consoles, set top boxes, hand-held or laptop devices (for example, media players, notebook computers, tablet computers, cellular phones, personal data assistants, voice recorders), server computers, multiprocessor systems, microprocessor-based systems, programmable consumer electronics, networked personal computers, minicomputers, mainframe computers, and distributed computing environments that include any of the above types of computers or devices, and the like.
With reference to
A computer storage medium is any medium in which data can be stored in and retrieved from addressable physical storage locations by the computer. A computer storage medium thus can be a volatile or nonvolatile memory, or a removable or non-removable storage device. Memory 1204, removable storage 1208 and non-removable storage 1210 are all examples of computer storage media. Some examples of computer storage media are RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optically or magneto-optically recorded storage device, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices. Computer storage media and communication media are mutually exclusive categories of media.
Computer 1200 may also include communications connection(s) 1212 that allow the computer to communicate with other devices over a communication medium. Communication media typically transmit computer program instructions, data structures, program modules or other data over a wired or wireless substance by propagating a modulated data signal such as a carrier wave or other transport mechanism over the substance. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal, thereby changing the configuration or state of the receiving device of the signal. By way of example, and not limitation, communication media includes wired media, such as metal or other electrically conductive wire that propagates electrical signals or optical fibers that propagate optical signals, and wireless media, such as any non-wired communication media that allows propagation of signals, such as acoustic, electromagnetic, electrical, optical, infrared, radio frequency and other signals. Communications connections 1212 are devices, such as a wired network interface, wireless network interface, radio frequency transceiver, e.g., WiFi 1270, cellular 1274, long term evolution (LTE) or Bluetooth 1272, etc., transceivers, navigation transceivers, e.g., global positioning system (GPS) or Global Navigation Satellite System (GLONASS), etc., or other devices 1276 that interface with communication media to transmit data over and receive data from the communication media.
The computer 1200 may have various input device(s) 1214 such as a pointer device, keyboard, touch-based input device, pen, camera, microphone, sensors, such as accelerometers, thermometers, light sensors and the like, and so on. The computer 1200 may have various output device(s) 1216 such as a display, speakers, and so on. Such devices are well known in the art and need not be discussed at length here. Various input and output devices can implement a natural user interface (NUI), which is any interface technology that enables a user to interact with a device in a “natural” manner, free from artificial constraints imposed by input devices such as mice, keyboards, remote controls, and the like.
Examples of NUI methods include those relying on speech recognition, touch and stylus recognition, gesture recognition both on screen and adjacent to the screen, air gestures, head and eye tracking, voice and speech, vision, touch, gestures, and machine intelligence, and may include the use of touch sensitive displays, voice and speech recognition, intention and goal understanding, motion gesture detection using depth cameras (such as stereoscopic camera systems, infrared camera systems, and other camera systems and combinations of these), motion gesture detection using accelerometers or gyroscopes, facial recognition, three dimensional displays, head, eye, and gaze tracking, immersive augmented reality and virtual reality systems, all of which provide a more natural interface, as well as technologies for sensing brain activity using electric field sensing electrodes (EEG and related methods).
The various storage 1210, communication connections 1212, output devices 1216 and input devices 1214 can be integrated within a housing with the rest of the computer, or can be connected through various input/output interface devices on the computer, in which case the reference numbers 1210, 1212, 1214 and 1216 can indicate either the interface for connection to a device or the device itself as the case may be.
A computer generally includes an operating system, which is a computer program that manages access to the various resources of the computer by applications. There may be multiple applications. The various resources include the memory, storage, input devices and output devices, such as display devices and input devices as shown in
The various modules, tools, or applications, and data structures and flowcharts of
Alternatively, or in addition, the functionality of one or more of the various components described herein can be performed, at least in part, by one or more hardware logic components. For example, and without limitation, illustrative types of hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Program-specific Integrated Circuits (ASICs), Program-specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.
Accordingly, in one aspect, a computer comprises a graphical user interface module executing on a processing unit of the computer and comprising: a first input accessing message data from computer storage accessible to the computer, the message data comprising data corresponding to messages, in a first natural language, of a graphical user interface of a target computer program, an output providing output data, generated using at least the message data, for presentation on a presentation device, and a second input receiving user input. The computer also comprises a context access module executing on a processing unit of the computer and operative, in response to selected text corresponding to a message in the message data, to access context data from computer storage accessible to the computer, wherein the context data comprises data derived from the graphical user interface of the target computer program, to retrieve content associated with a string that matches the selected text, and having an output providing the retrieved content to the graphical user interface module. The graphical user interface module is operative, in response to receiving the retrieved content, to generate the output data including the retrieved content in association with the selected text. The graphical user interface module is operative, in response to an input with respect to a selected message, to update data for the selected message in the message data based on the input.
In another aspect, a computer-implemented process performed by a computer program executing on a computer, comprises accessing message data from computer storage accessible to the computer, the message data comprising messages, in a first natural language, of a graphical user interface of a target computer program. An input indicative of selected text corresponding to a message in the message data is received. Context data is accessed from computer storage accessible to the computer, wherein the context data comprises data derived from the graphical user interface of the target computer program, to retrieve content associated with a string that matches the selected text. Output data, including the retrieved content in association with the selected text, is generated for presentation on a presentation device. Input with respect to a selected message is received, and data for the selected message in the message data is updated based on the input.
In another aspect, a computer includes means for accessing context data, derived from the graphical user interface of the target computer program, to retrieve content associated with a string that matches the selected text from messages data, and means for presenting the context data in association with the selected text. The computer can store text in the message data for a translation corresponding to the selected text.
In another aspect, a computer includes means for processing output data from a target computer program to generate context data, wherein the context data comprises data derived from output data of the target computer program, including strings recognized from the output data associated with content derived from the output data.
In any of the foregoing aspects, the input with respect to a selected message can be a translation, into a second natural language, of the selected message. The translation is stored in the message data as the translation in the second natural language of the selected message.
In any of the foregoing aspects, the input with respect to a selected message can be an indication of approval of a translation, and data indicative of the approval of the translation is stored in the message data.
In any of the foregoing aspects, the output data can include display data of the graphical user interface of the target computer program. The display data can be processed using optical character recognition to recognize strings in the display data.
In any of the foregoing aspects, the output data can include audio data of the graphical user interface of the target computer program. The audio data can be processed using speech recognition to recognize strings in the audio data.
In any of the foregoing aspects, the data derived from the graphical user interface of the target computer program can include strings, recognized from outputs of the graphical user interface of the target computer program captured during actual use of the target computer program, and associations between the strings and images of the graphical user interface of the target computer program captured during the actual use of the target computer program.
In any of the foregoing aspects, the display data can include an editing pane displaying the selected text corresponding to the message in the message data and an image pane displaying images retrieved from the context data. A plurality of retrieved images can be displayed sequentially in the image pane.
In any of the foregoing aspects, the graphical user interface can include an indication of confidence of a match between selected text and a string corresponding to the retrieved content.
In any of the foregoing aspects, the context data can include, for a recognized string from an image, a location of the string in the image. When an image corresponding to a string that matches the selected text is displayed, a highlight of the string can be displayed in the image.
In any of the foregoing aspects, the context data can be stored as a database associating strings with content derived from the graphical user interface of the target computer program. The database can be stored for access using queries on recognized strings in a structured query language.
In any of the foregoing aspects, accessing context data includes matching selected text to recognized strings in the context data. In one implementation, an edit distance metric is used to measure similarity between the selected text and recognized strings in the context data. In one implementation, a set of candidate recognized strings is limited by string length based on string length of the selected text.
In another aspect, an article of manufacture includes at least one computer storage medium, and computer program instructions stored on the at least one computer storage medium. The computer program instructions, when processed by a processing system of a computer, the processing system comprising one or more processing units and storage, configures the computer as set forth in any of the foregoing aspects and/or performs a process as set forth in any of the foregoing aspects.
Any of the foregoing aspects may be embodied as a computer system, as any individual component of such a computer system, as a process performed by such a computer system or any individual component of such a computer system, or as an article of manufacture including computer storage in which computer program instructions are stored and which, when processed by one or more computers, configure the one or more computers to provide such a computer system or any individual component of such a computer system.
It should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific implementations described above. The specific implementations described above are disclosed as examples only.
