Providing human-generated and machine-generated trusted translations

Abstract

Customers having a translation project to select a translation method from a variety of options, ranging from a completely human translation to a completely automated translation. For human translations, translation job information may be communicated through one or more network service modules which execute within a network service application, such as a web-based networking application. A network service module may register a user having an account with the network service application as a translator and communicate translation jobs to the user. One or more users who express interest in performing the translation are selected to perform a translation job, each job comprising at least a portion of the translation project. After a user provides a translation for the translation job, the translation is analyzed to generate a trust level prediction for the translation. A user translation profile may be updated after each translation to reflect the user's performance.

Description

BACKGROUND OF THE INVENTION

Translations of documents from a source one language to a target language are typically handled by people or by machines. Translation services usually focus on one or the other, but not both. Most machine translation services do not provide a choice of machine translation or human translation options to a customer.

Further, human translations are typically more expensive than machine translations and peer reviewed. The human translation quality is usually confirmed by review of one or more other people. Inevitably, the translation skills of different reviewers will vary from person to person. Therefore, the quality of a peer reviewed translation may be subjective based on the skill of the particular reviewer, and may vary between translation. This results in an uneven measure of human translation reviews and inconsistent quality.

SUMMARY OF THE INVENTION

The present technology allows customers having a translation project to select a translation method from a variety of options, ranging from a completely human translation to a completely automated translation. The human translation option may include one or more users translating portions of a translation project. The users may be harnessed through one or more network service modules (end point modules) which execute within a network service application, such as a web-based networking application. A network service module may register a user having an account with the network service application as a translator and communicate translation jobs to the user. One or more users who express interest in performing the translation are selected to perform a translation job, each job comprising at least a portion of the translation project. After a user provides a translation for the translation job, the translation is analyzed to generate a trust level prediction for the translation. A user translation profile may be updated after each translation to reflect the user's performance. User performance may be based on an automatically generated trust score, failure to provide a translation and other information. The translation and rating information is provided to the user and to the network service module within the web service application.

An embodiment may provide a trusted translation by receiving a human-generated translation of a document from a source language to a target language. A trust level prediction of the human-generated translation may be generated. The trust level may be associated with a translational accuracy of the human-generated translation. The human-generated translation and the trust level may then be output.

An embodiment may process a translation job by receiving user information from a user having an account with a network service. The user information may be received by a network service module. Translation job information may be received from a remote application by the network service module. The translation job information may be provided to the user.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an exemplary system for providing a trusted translation.

FIG. 2 is a block diagram of an exemplary application server module.

FIG. 3 is a flow chart of an exemplary method for providing a trusted translation.

FIG. 4 is a flow chart of an exemplary method for providing a trusted human translation.

FIG. 5 is a flow chart of an exemplary method for providing a translation.

FIG. 6 is a flow chart of an exemplary method for handling a translation job.

FIG. 7 is a flow chart of a computing device for implementing the present technology.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

The present technology allows customers having a translation project to select a translation method from a variety of options, ranging from a completely human translation to a completely automated translation. The human translation option may include one or more users translating portions of a translation project. The users may be harnessed through one or more network service modules (end point modules) which execute within a network service application, such as a web-based networking application. A network service module may register a user having an account with the network service application and communicate translation jobs to the user.

A user may request to perform one or more jobs that are communicated to the user. One or more users who express interest in performing the translation are selected to perform a translation job, each job comprising at least a portion of the translation project. User selection may depend on past user translation performance, ability to complete past translations on time, and other information. Once a user provides a translation for the translation job, the translation is analyzed to generate a trust level prediction for the translation. A user translation profile may be updated after the translation or failure to complete the translation to reflect the user's performance. The translation and rating information is provided to the user and to the network service module within the web service application.

FIG. 1 is a block diagram of an exemplary system for providing a trusted translation. System 100 of FIG. 1 includes application server 110, network server 120, network 130, customer computing device 140, user computing devices 150 and 160, network service servers 170 and 180, and content destination server 190.

Application server 110 may be implemented as one or more servers and may process requests received over network 130 via network server 120. Application server 110 may communicate with network server 120 and may include translation application 112, training data set 114, and human translation module 116. The functionality of application server 110 is discussed in more detail below with respect to the methods of FIGS. 3-5.

Translation application 112 may include one or more applications executed by a processor to generate a machine translation. Translation application 112 may also be executed to predict a translation quality of a human generated translation or machine translation, provide an interface to customers for requesting and managing translation projects, and other functions.

Human translation module 116 may be executed to communicate with remote network service modules installed on remote machines (for example, modules executing within applications on network service servers). The human translation module may manage user translation profiles and communicate translation job information, user translation profile information, and other data with a network service module.

Network server 120 communicates with application server 110 and network 130. The network server may be implemented as one or more machines that process requests for application server 110 and provide responses to requesting entities. The responses may be generated by application server 110 or network server 120. When network 130 is implemented as the Internet, network server 120 may be implemented as a web server.

Network 130 may be implemented as a local area network, wide area network, private network, public network, an intranet, the Internet, or a combination of these networks.

Customer computing device 140 may be utilized by a customer 142 to request services provided by application server 110. In particular, customer computing device 140 may transmit a request to translation application 112 to configure a translation project, select a type of translation, configure network service modules for the human translation, and complete a translation.

The present technology may utilize one or more users to implement crowd sourced-based human translations. Users 152 and 162 may access translation job information provided by a network service module and submit translations to application server 110 through computing devices 150-160. Each of computing devices 140, 150 and 160 may communicate with other devices over network 130. Though only two user computing devices are illustrated, more or fewer user computing devices may be present in system 100.

As illustrated, a system 100 may include one or more network service servers, such as servers 170 and 180, which may include applications 172 and 182, respectively. Each of network service servers 170 and 180 may be implemented as one or more servers which provide a service over network 130. The service may be accessed by network service users or a machine translation customer over network 130. Network service server 170 may include an application 172. The application may implement a network service over network 130, such as for example a networking service, business service, or some other service. Examples of web services provided by an application 172 include Facebook, Linked In, Pro-Z, and other services.

One or more endpoint modules may be embedded within or execute to communicate with an application within a network service. Network service module 174 may be imbedded within application 172 and may interact with users which subscribe to and/or have an account with a service provided by application 172. When executed, network service module 174 may register a user with the network service module, provide information regarding translation jobs, and select one or more users to perform a translation. The implementation of registering a user, publishing a translation job, and selecting users to perform translations may vary depending on the platform and application in which the network service module 174 executes within. Operation for network service module 174 is discussed in more detail below with respect to FIG. 6.

Network service module 184 may execute within application 182, and may operate similarly to network service module 174. Though system 100 illustrates one network service module within each application, other variations of applications within a server and network service modules associated with an application are possible.

Content destination server may communicate with network 130 and receive translated content to be published or stored. The content destination server 190 may be implemented as a customer web site, an online document, or some other content, and may publish translations generated by crowd sourcing or machine translation.

FIG. 2 is a block diagram of an exemplary application server module. In some embodiments, the application of FIG. 2 provides more detail for the application modules of application server 110 in FIG. 1. A human translation module 116 may identify jobs in a translation project, as well as manage communications, parameters, and other portions of a crowd-sourced translation project. The human translation module 116 may communicate job information to a network service module based on a translation project configured by a customer. Human translation module 116 may also manage user translation profiles. The translation profiles may be updated based on a status of a translation, a trust score generated based on a completed translation, and other information such as for example subject matter expertise for a particular user translator.

Translation application 112 may include communications module 205, interface module 210, translation engine 215, and quality prediction engine 220. Translation application 112 may be executed to manage communication with a customer through customer computing device 140, interface with remote devices, translate a document to generate translated content, and provide a quality prediction of a user generated translation.

Communications module 205 may be executed to allow an indication to be received via a user interface to provide a translation of a document from a source language to a target language, as well as a prediction of a trust level of the translation. The communications module 205 may also facilitate accessing the document to be translated such as in response to an indication by a user.

The interface module 210 can be executed to provide a graphical user interface through a network browser, for example as a content page provided by a web browser, which enables a user to request the translation and corresponding trust-level prediction.

The translation engine 215 comprises a machine translation engine capable of translating from a source language to a target language. Such translation capability may result from training the translation engine 215 on various training data. In some embodiments, the translation application 112 may include more than one translation engines 215. Additionally, the translation engine 215 may be based on statistical-translation techniques, non-statistical-translation techniques, or a combination thereof.

As depicted in FIG. 2, the translation engines 215 is associated with the training dataset 225. According to other exemplary embodiments, the translation engine 215 can be associated with any number of training datasets. The training dataset 225 may comprise documents in source languages and corresponding translations of those documents in target languages (i.e., parallel corpora). The translated documents may be human-generated or machine-generated. The training dataset 225 may be domain-specific or generic.

According to exemplary embodiments, the quality prediction engine 220 is executable to predict a trust level of a translation provided by the translation engine 215. The trust-level prediction is indicative of translational accuracy of translations generated by the translation engine 215. The trust level is predicted independent of a human-generated translation or other human intervention.

An example of translation application 112 is described in U.S. patent application Ser. No. 12/572,021, entitled “Providing Machine-Generated Translation and Corresponding Trust Levels,” which was filed on Oct. 1, 2009, and is incorporated herein by reference in its entirety.

The interaction between the translation service provided by application server 110 and network server 120, a customer requesting a translation, and users providing crowd source translation is discussed in more detail with respect to FIGS. 3-6 and may be summarized as follows. A translation project may be communicated to translation application 112, for example by customer 142 using customer computing device 140 to define a project through translation application 112. The details of the translation job may be provided by customer computing device 140 over network 130 to translation application 112, for example through a web page provided by translation application 112. Once the project configuration is completed, information regarding translation jobs consisting of one or more portions of the translation project may be transmitted to selected network service modules embedded in network services. Selected users who wish to perform the translation jobs are paired with the jobs by network service modules which receive job information. Users obtain access to the content to be translated and either complete a translation or do not complete a translation under a specified set of criteria. A trust score may be generated based on the translation, or failure to complete a translation. A user translation profile may also be updated. The translation is reported to a content destination server and feedback regarding a translation is provided to the translating user.

FIG. 3 is a flow chart of an exemplary method for providing a trusted translation. The method of FIG. 3 may be performed by portions of system 100 in the system of FIG. 1. First, a translation request is received at step 310. The request may be received by application server 110 from a customer with documents to be translated. For example, the translation may be received through an interface such as a web page interface provided by translation application 112 over network 130.

A list of human and automated translation options may be provided at step 320. The list of translation options may be provided through an interface provided by translation application 112. The options may allow the customer to select between a fully automated translation, a machine translation with post-editing performed by a human, crowd source generated translation (non-professional human translation), and professional human translation for converting documents in a source language to documents in a target language.

A selection of a translation option is received from a customer at step 330. Hence, a single translation service provided by the present technology allows a user to select any of several translation methods. The selection may be received through an interface provided by translation application 112. The translation is generated and provided per the customer request at step 340. If selected, the professional translation may be performed by a professional translator as requested by a customer. A requested machine translation may be performed by translation engine 215 of translation application 112. A machine translation with a post-edits may be performed in the same way, however a human may review the translation provided by translation engine 215. A crowd source application may be provided by users registered with an end point application embedded in a network service. Crowd source translation is discussed in more detail below with respect to FIGS. 4-6.

For each translation provided by the present technology, whether by human translation or machine translation, a trust score may be predicted for the translation. Hence, whether the translation is generated by a user, by machine translation, or a combination of these, the present technology may generate a trust score to predict a trust level of the resulting translation. The trust score may be generated by quality prediction engine 220 as discussed above.

At step 330 in the method of FIG. 3, a customer may indicate that a translation should be performed by crowd sourcing. In this case, a customer may provide details to configure parameters associated with the crowd sourced translation by one or more nonprofessional translators. The method of FIG. 4 outlines an exemplary method for allowing a customer to configure a translation to be performed by multiple nonprofessional translators (i.e., via crowd sourcing).

FIG. 4 is a flow chart of an exemplary method for providing a trusted human translation. The method of FIG. 4 provides more detail for step 340 in the method of FIG. 3, for example when a customer requests a translation generated via crowd sourcing. The method of FIG. 4 provides exemplary steps for configuring parameters for a crowd-sourced translation project. Configuration of such a translation project may include the steps of FIG. 4, fewer steps, and additional configuration steps.

A source language and translation language selection are received at step 410. For example, when a customer configures a translation project, the customer may indicate that the translation should be performed from an English source language to a French target language.

A crowd sourcing location selection may be received at step 420. When the source language and translation language selection is received at step 410, the present technology may determine the locations of available translators that may perform the translation. For example, the translators may be located in a particular country, a particular state, or some other location. The locations may indicate availability of users registered with an end point application and able to perform the requested translation.

A network service module selection is received at step 430. After a customer has selected the language translation pair, the customer may select types of application network service modules to publish translation job information. Types of network service module may correspond to the network service application in which the network service module operates. Examples of a network service module types may include a Facebook network service module, Linked-In network service module, Pro-Z network service module, Second Life network service module. Human translation module 116 may maintain a table or list of network service module types which correspond to a particular language pair, location, and other data. The table or list may be updated upon receiving user registration details from a network service module at a particular network service application.

Translation parameters may be received at step 440. The translation parameters may include general translation parameters and end point specific parameters. For example, translation parameters may include a message to send to potential translator users, whether or not to enable e-mail notification of available translation jobs by a network service module, and whether or not to post available translation jobs to a module content page. Additional translation parameters may include payment terms, currency, type of reward to receive, and other compensation information. For example, the compensation for translation may include cash, virtual goods having value within the application, rewards, a thank you button, gift vouchers, and other items of real or virtual value. The compensation for a translation may vary per end point value. Additional translation parameters may include a project budget and project timing information, such as for example a time limit and maximum budget for which to spend on the particular translation. Other project-related items such as review and recurrence may also be configured as a translation parameter.

A translation may be processed at step 450. Processing a translation may include sending translation notification information to a network service module, selection of one or more users to perform the translation by the network service module, having the selected users perform the translation, and processing the translation by application server 110. Trust scores may be generated for translations performed by users. Processing a translation is discussed in more detail below with respect to the methods of FIGS. 5 and 6. FIG. 5 describes processing a translation from the point of view of application server 110, and FIG. 6 describes processing a translation from the point of view of a network service module.

FIG. 5 is a flow chart of an exemplary method for providing a translation. The method of FIG. 5 provides more detail for step 450 of the method of FIG. 4 from the context of translation application 112.

First, content to be translated is identified at step 510. The content to be translated may be a document which is accessed from a remote location. The document to be translated may also be identified based on the location of the content rather than retrieving the content itself. The identification may include receiving the location of the content, or the content itself, by translation application 112.

A new content notification is transmitted by application server 110 to network service modules at step 520. The new content notification is an indication of one or more translation jobs. A new content notification may include a customer message for a user, a location and/or a link to the content to be translated and information to enable a network service module select a translator, such as a classification of the document content to be translated. The network service modules selected at step 430 may receive the new content notification.

Source content to be translated is provided to selected users by application server 110 at step 530. A network service module may receive the content notification sent at step 520, provide the information to one or more users, receive indications from one or more users interested in performing a translation, selecting one or more users to perform the translation, and provide the selected users with job information for accessing the content from application server 110. The job information regarding the content to translate may include application server contact information or other information to provide the user with access to the content to be translated. In some embodiments, a user may be provided with a link to an interface (provided by translation application 112) provided by application server 110 which provides the content to be translated.

The translation or detection of a no-translation event is received at step 540. Once a selected user completes a translation job, the user may indicate this to the application server, for example, through an interface in which the translation is being provided. If the translation is not completed within a certain amount of time, or for some other reason a user fails to complete a translation, a detect no-translation event is triggered. The no-translation event indicates the user failed to complete a translation of the content. The detect no-translation event may be associated with an expiration of a period of time (for example, a maximum allowed time in which to complete the translation), a cancellation of the translation by the user, or some other event. Upon detecting a no-translation event, the translation job will be re-sent to the selected network service modules to reassign the translation job.

A trust level prediction of the human translation provided by the user is generated at step 550. Quality prediction engine 220 may generate the trust score based on the translation provided by the user. If a selected user does not complete a translation, the trust level prediction is not generated.

The user translation record is updated at step 560. The user translation record may reflect a user's predicted trust score, translations not completed, customer feedback on user translations, and other data. For example, user trust levels may be averaged to generate an overall score. The user's failure to complete a translation may reduce the overall score. In some embodiments, the higher the level of accuracy and the more consistently a user completes a translation, the better the translation overall score.

The complete translation is provided to a destination server at step 570. In some embodiments, the translation may be provided by translation application 112 to content destination server 190. The translation provided may be based on the predicted trust level score determined at step 550 (i.e., the translation may be provided only if the predicted trust level meets a minimum threshold level, for example 95%).

The translation predicted trust score may be reported at step 580. The predicted trust score may be reported to the user to inform the user of their translation quality. Overall score information may also be provided to the user. The predicted trust score and/or overall score may also be provided to the network service module to enable the module to select the most qualified interested users for future translation jobs.

FIG. 6 is a flow chart of an exemplary embodiment for handling a translation job. The method of FIG. 6 may be performed by a network service module within an application, such as for example network service module 174 within application 172.

A user registers with a network service module at step 610. To register with a network service module, the user may first register with an application 172 which network service module 174 is embedded within or communicates with. Once registered with application 172, or having an account with application 172, a user may register with a network service module associated with the application. Registration with a network service module may include providing a user name, contact information, e-mail, translation skills (e.g., languages able to perform translations in), indication of other network service modules in which the user is registered, and other information.

A new translation job notification may be received by a network service module from translation application 112 at step 620. The information received for a new translation job may be the same information sent by translation application 112 at step 520 in the method of FIG. 5. The notification may include, for example, a message from a customer to prospective user translators, language pair information, a location for a user to access content to be translated, and other data.

New job notifications are reported by a network service module to registered users at step 630. The new job notification may be reported to users registered with the particular network service module that receives the new translation job notification. The reporting may be made by e-mail, message notification, or some other communication sent by a network service module to a registered user. The notification may also be made by reporting the new job notification data to a page associated with the network service module within the application in which it is embedded, such as for example a bulletin board, content page, favorites page, or some other location.

A request may be received from one or more registered users by a network service module to perform a translation job at step 640. The translation job information is provided to one or more selected users at step 650. The users selected to perform the translation may by the requesting users which have the highest overall score, the first users to reply to the job notification, or may be selected in some other manner. The translation job information provided to the user may include a link with additional information to start the translation job. When a user selects the link to access the content to be translated, translation application 112 may receive this request as discussed above with respect to step 530 in the method of FIG. 5. The user may then access the content to be translated and provide a translation. If the user does not complete the translation within a specified period of time, a no-translation event may be triggered and the translation job may be reported by an end-point module at step 630.

FIG. 7 is a flow chart of a computing device for implementing the present technology. In some embodiments, the system of FIG. 7 may be used to implement application server 110, network server 120, computing devices 140, 150 and 160, network service servers 170 and 180, and content destination server 190. The computing system 700 (also referred to as computer system 700) includes one or more processors 710 and main memory 720. Main memory 720 stores, in part, instructions and data for execution by processor 710. Main memory 720 can store the executable code when in operation. The computing system 700 further includes a mass storage device 730, portable storage medium drive(s) 740 (also referred to as portable storage device 740), output devices 750, user input devices 760, a graphics display 770 (also referred to as display system 770), and peripheral device(s) 780.

The components shown in FIG. 7 are depicted as being connected via a single bus 790. The components may be connected through one or more data transport means. The processor 710 and the main memory 720 may be connected via a local microprocessor bus, and the mass storage device 730, the peripheral devices 780, the portable storage medium drive(s) 740, and display system 770 may be connected via one or more input/output (I/O) buses.

The mass storage device 730, which may be implemented with a magnetic disk drive or an optical disk drive, is a non-volatile storage device for storing data and instructions for use by the processor 710. The mass storage device 730 can store the system software for implementing embodiments of the present technology for purposes of loading that software into the main memory 720.

The portable storage device 740 operates in conjunction with a portable non-volatile storage medium, such as a floppy disk, compact disk, digital video disc, or USB storage device, to input and output data and code to and from the computer system 700 of FIG. 7. The system software for implementing embodiments of the present technology may be stored on such a portable medium and input to the computer system 700 via the portable storage device 740.

The input devices 760 provide a portion of a user interface. The input devices 760 may include an alpha-numeric keypad, such as a keyboard, for inputting alpha-numeric and other information, or a pointing device, such as a mouse, a trackball, stylus, or cursor direction keys. Additionally, the computing system 700 as shown in FIG. 7 includes the output devices 750. Suitable output devices include speakers, printers, network interfaces, and monitors.

The display system 770 may include a liquid crystal display (LCD) or other suitable display device. The display system 770 receives textual and graphical information, and processes the information for output to the display device.

The peripheral device(s) 780 may include any type of computer support device to add additional functionality to the computer system. The peripheral device(s) 780 may include a modem or a router.

The components contained in the computer system 700 of FIG. 7 are those typically found in computer systems that may be suitable for use with embodiments of the present technology and are intended to represent a broad category of such computer components that are well known in the art. Thus, the computer system 700 of FIG. 7 can be a personal computer, hand held computing device, telephone, mobile computing device, workstation, server, minicomputer, mainframe computer, or any other computing device. The computer can also include different bus configurations, networked platforms, multi-processor platforms, etc. Various operating systems can be used including Unix, Linux, Windows, Macintosh OS, Palm OS, webOS, Android, iPhone OS and other suitable operating systems.

It is noteworthy that any hardware platform suitable for performing the processing described herein is suitable for use with the technology. Computer-readable storage media refer to any medium or media that participate in providing instructions to a central processing unit (CPU), a processor, a microcontroller, or the like. Such media can take forms including, but not limited to, non-volatile and volatile media such as optical or magnetic disks and dynamic memory, respectively. Common forms of computer-readable storage media include a floppy disk, a flexible disk, a hard disk, magnetic tape, any other magnetic storage medium, a CD-ROM disk, digital video disk (DVD), any other optical storage medium, RAM, PROM, EPROM, a FLASHEPROM, any other memory chip or cartridge.

Various forms of transmission media may be involved in carrying one or more sequences of one or more instructions to a CPU for execution. A bus carries the data to system RAM, from which a CPU retrieves and executes the instructions. The instructions received by system RAM can optionally be stored on a fixed disk either before or after execution by a CPU.

While various embodiments have been described above, it should be understood that they have been presented by way of example only, and not limitation. The descriptions are not intended to limit the scope of the technology to the particular forms set forth herein. Thus, the breadth and scope of a preferred embodiment should not be limited by any of the above-described exemplary embodiments. It should be understood that the above description is illustrative and not restrictive. To the contrary, the present descriptions are intended to cover such alternatives, modifications, and equivalents as may be included within the spirit and scope of the technology as defined by the appended claims and otherwise appreciated by one of ordinary skill in the art. The scope of the technology should, therefore, be determined not with reference to the above description, but instead should be determined with reference to the appended claims along with their full scope of equivalents.

Claims

1. A method for providing a trusted translation, comprising: receiving a human-generated translation of a document from a source language to a target language, the human-generated translation being produced by a person;generating a trust level prediction of the human-generated translation based at least in part on a mapping by executing a quality-prediction engine stored in memory, the trust level prediction of the human-generated translation associated with translational accuracy of the human-generated translation, the quality-prediction engine being calibrated, the calibration including: obtaining a plurality of opinions for a plurality of sample translations performed by the person, each of the opinions from a human and indicating a perceived trust level of corresponding sample translations, andusing the quality prediction-engine to determine a trust level of each of the plurality of sample translations,determining a relationship between the plurality of opinions and the trust levels of each of the plurality of sample translations, andtuning the mapping to minimize any difference between the plurality of opinions and the trust levels of each of the plurality of sample translations; andoutputting the human-generated translation and the trust level prediction of the human-generated translation.

2. The method of claim 1, further comprising providing an interface configured to receive a translation selection of one of a plurality of translation methods, the plurality of translation methods including both a human translation method and a machine translation method.

3. The method of claim 1, wherein outputting includes reporting information based on the trust level prediction to a user that generated the translation.

4. The method of claim 1, wherein outputting includes reporting information based on the trust level prediction to a module executing within a remote application.

5. The method of claim 1, further comprising transmitting translation information to a module executing within a remote application.

6. The method of claim 5, wherein the translation information includes a description of the translation to be performed.

7. The method of claim 5, wherein the translation information includes information for initiating access to the content to be translated.

8. The method of claim 1, further comprising receiving configuration information for a translation to be performed by humans.

9. The method of claim 8, wherein the configuration information includes a selection of one or more modules from a plurality of modules within one or more remote applications.

10. The method of claim 9, wherein the remote applications are network service applications executing on remote application servers.

11. A non-transitory computer readable storage medium having embodied thereon a program, the program being executable by a processor to perform a method for providing a trusted translation, the method comprising: receiving a human-generated translation of a document from a source language to a target language, the human-generated translation being produced by a person;generating a trust level prediction of the human-generated translation based at least in part on a mapping, the trust level associated with translational accuracy of the human-generated translation;calibrating, the calibrating including: obtaining a plurality of opinions for a plurality of sample translations performed by the person, each of the opinions from a human and indicating a perceived trust level of corresponding sample translations, andusing the quality prediction-engine to determine a trust level of each of the plurality of sample translations,determining a relationship between the plurality of opinions and the trust levels of each of the plurality of sample translations, andtuning the mapping to minimize any difference between the plurality of opinions and the trust levels of each of the plurality of sample translations; andoutputting the human-generated translation and the trust level prediction of the human-generated translation.

12. The non-transitory computer readable storage medium of claim 11, the method further comprising providing an interface configured to receive a translation selection of one of a plurality of translation methods, the plurality of translation methods including both a human translation method and a machine translation method.

13. The non-transitory computer readable storage medium of claim 11, wherein outputting includes reporting information based on the trust level prediction to a user that generated the translation.

14. The non-transitory computer readable storage medium of claim 11, wherein outputting includes reporting information based on the trust level prediction to a module executing within a remote application.

15. The non-transitory computer readable storage medium of claim 11, the method further comprising transmitting translation information to a module executing within a remote application.

16. The non-transitory computer readable storage medium of claim 15, wherein the translation information includes a description of the translation to be performed.

17. The non-transitory computer readable storage medium of claim 15, wherein the translation information includes information for initiating access to the content to be translated.

18. The non-transitory computer readable storage medium of claim 11, the method further comprising receiving configuration information for a translation to be performed by humans.

19. The non-transitory computer readable storage medium of claim 18, wherein the configuration information includes a selection of one or more modules from a plurality of modules within one or more remote applications.

20. The non-transitory computer readable storage medium of claim 19, wherein the remote applications are network service applications executing on remote application servers.

21. The method of claim 1, further comprising: receiving user information from a user having an account with a network service by executing a network service module stored in memory;receiving translation job information from a remote application by the network service module; andproviding the translation job information to the user.

22. The method of claim 21, wherein the user information includes user translation information.

23. The method of claim 21, wherein receiving user information includes registering the user with the network service module.

24. The method of claim 21, further comprising communicating the translation job information to users of the network service for which user information was received.

25. The method of claim 21, further comprising receiving a request from one or more users to perform the translation by the network service module.

26. The method of claim 21, further comprising selecting one or more users to perform the translation by the network service module.

27. The method of claim 21, further comprising receiving trust score information regarding a translation performed by a user.