Patent Grant
10360306
This application claims the benefit of the French patent application No. 14 57324 filed on Jul. 29, 2014, the entire disclosure of which is incorporated herein by reference.
The present disclosure relates to the field of aircraft maintenance operations. In particular, the disclosure herein concerns a method and a system for monitoring an intervention by a maintenance engineer on an aircraft when the maintenance engineer is remotely guided by a maintenance support engineer.
A Line Maintenance Engineer (LME) performing maintenance operations on an aircraft may be remotely guided by a Remote Maintenance Support Staff (RMSS) maintenance support engineer located in a maintenance center. The maintenance support engineer may provide instructions directly to the maintenance engineer, regarding the manipulations to be performed, and guide the maintenance engineer precisely in the execution of tricky tasks. Guidance is given by voice communication by mobile phones or optionally by walkie-talkie if the maintenance support engineer is located in close proximity to the aircraft or by VHF if the distance between the support engineer and the maintenance engineer is very great. At the end of maintenance activity, the maintenance support engineer must validate and take responsibility for the actions performed by the maintenance engineer. Thus, the support engineer can ensure that the instructions issued by same have been correctly understood and followed by the maintenance engineer. Similarly, the support engineer must ensure that nothing further has been carried out during the maintenance intervention.
In addition, during a complex intervention, the assistance of the support engineer enables the efficiency of the maintenance engineer to be increased. Indeed, the maintenance engineer may be confronted with a new problem requiring expertise that may already be known to the support engineer or which can be easily found by the latter from documents and/or from other support engineers.
However, the airport environment is noisy, which may impair understanding during voice communication between the maintenance engineer and the support engineer. Moreover, in the context of information exchange, the part of interpretation is important, which may distort both the understanding of the instructions received by the maintenance engineer and the response of same to the maintenance support engineer.
The subject matter of the present disclosure is therefore to remedy the aforementioned disadvantages by providing a method and a system for monitoring the intervention by a maintenance engineer on an aircraft, that are simple to implement and that can automatically detect any inconsistency between the instructions given by the maintenance support engineer and the responses of the maintenance engineer.
The present disclosure is aimed at automatically detecting and reporting any inconsistency in understanding between a maintenance engineer and a remote support engineer exchanging information and concerns a system for monitoring maintenance intended for monitoring an intervention by a maintenance engineer on an aircraft remotely guided by a maintenance support engineer, comprising:
The disclosure herein is also aimed at the detection of any divergence of maintenance actions with respect to a theoretical maintenance procedure described in maintenance documents and/or with respect to changes of states of the aircraft.
Other features and advantages of the system and method according to the disclosure herein will appear better on reading the description given below, as a non-restrictive guide, with reference to the accompanying drawings in which:
A principle of the disclosure herein comprises monitoring an intervention by a maintenance engineer on an aircraft remotely guided by a maintenance support engineer by automatically comparing the instructions given orally by the maintenance support engineer with the oral responses of the maintenance engineer in order to ensure the mutual understanding of the participants and warn same in the event of misunderstanding.
In accordance with the disclosure herein, the maintenance monitoring system 1 comprises first and second communication terminals 3, 5, a voice recognition device 7, a semantic analysis device 9, and a warning device 11.
The communication terminals 3, 5 are, for example, mobile phones, walkie-talkies, laptop computers, etc. A first mobile communication terminal 3 is intended for a maintenance engineer 13 located in or around an aircraft 15 and a second communication terminal 5 is intended for a maintenance support engineer 17 located in a maintenance center 19. These first and second communication terminals 3, 5 enable the maintenance engineer 13 and remote maintenance support engineer 17 to exchange information between them.
The voice recognition device 7 is in a known way a computer application for analyzing speech for transcribing same in the form of a text. Similarly, the semantic analysis device 9 is in a known way a computer application for establishing the meaning of a message using the sense of the words in the text (see, for example, ‘semantic analysis’ in Wikipedia).
Each communication terminal 3, 5 normally includes a receiving/transmitting unit (e.g. a modem), an input unit (e.g. a microphone, a keyboard), a processing unit, a memory and an output unit (e.g. earphones, headphone set, a screen). It will be noted that the memory may comprise a computer program including code instructions suitable for the implementation of the monitoring method according to the disclosure herein.
In step E1, a maintenance engineer 13 who has to intervene on an aircraft 15 calls a maintenance support engineer 17 via the first communication terminal 3 and the voice recognition device 7 to assist him in the performance of a maintenance task. The maintenance support engineer 17 then guides the maintenance engineer 13 in performing this via the second communication terminal 5 and the voice recognition device 7.
Indeed, the maintenance support engineer 17, on the basis of the maintenance tasks in the aircraft's documentation 21, gives oral instructions to the maintenance engineer 13. It will be noted that the designations of the components of the aircraft 15 or the environment thereof, the functions or parameters of the aircraft, tools and maintenance actions are described by a standard aeronautical vocabulary limited to the task to be performed. Similarly, the instructions and the exchange between the maintenance support engineer 17 and the maintenance engineer 13 follow the same codes. The vocabulary used is generally reduced to the strict minimum. Thus, the maintenance support engineer 17 and the maintenance engineer 13 use in their sentences a number of keywords (type of action, generic name of the system, equipment number associated with the action) listed in a predefined dictionary of keywords which is actually an extract from the maintenance documentation. The instructions may, for example, take the following form: ‘deactivate the Fuel system’, ‘dismantle panel VUxxx’, etc.
The maintenance engineer 13 executes the instruction and orally gives the response or the result of the latter's action. The result may, for example, take the following form: ‘Fuel System deactivated’, “VUxxx panel dismantled”, etc.
In step E2, the voice recognition device 7 is configured for translating the oral guidance instructions given by the maintenance support engineer 17 and the oral responses given by the maintenance engineer 13 relating to the execution of the instructions into textual instructions and textual responses.
Advantageously, the voice recognition device 7 is configured for retranscribing the instructions and results onto screens 31, 51 of the first and second communication terminals 3, 5. Thus, the maintenance engineer 13 and the maintenance support engineer 17 have the possibility if necessary of correcting the instructions and results by relaunching the processing procedure in the voice recognition device.
In step E3, the semantic analysis device 9 is intended to carry out a semantic analysis and an identification of keywords or key phrases. This device 9 is capable if necessary of extracting keywords from the instructions and results retranscribed by the voice recognition device 7 and performing the semantic comparison between these keywords or the phrases per se. It will be noted that the comparison is made on all the words of the instruction (i.e. subject, verb, complement, etc.).
Thus, the semantic analysis device 9 is configured for comparing the textual instructions with the textual responses in order to ensure the mutual understanding of the participants 13, 17 and optionally to detect any inconsistency between the instructions and the responses. For example, if the system name or the panel number mentioned by the maintenance support engineer 17 has been misunderstood by the maintenance engineer 13, the semantic analysis device 9 will be able to detect it.
The voice recognition 7 and semantic analysis 9 devices are self-adaptive and are capable of self-learning thereby increasing the overall efficiency of voice recognition and the detection of anomalies.
In event of failure during voice recognition or the detection of anomalies or of misunderstanding between the participants, the semantic analysis device 9 transmits these anomalies to the warning device 11.
Indeed, in step E4, the warning device 11 is configured for generating a warning 23 when an inconsistency is detected between the instructions and the responses. Advantageously, the warning device 11 is configured for sending the warning 23 simultaneously to the maintenance support engineer 17 and to the maintenance engineer 13. According to one variant, the warning device 11 is configured for sending the warning 23 only to one of them.
Moreover, depending on the seriousness of the anomaly, the warning device 11 may use different ways of possible warnings and of different levels. Indeed, the warning device 11 may activate display and/or audible and/or light and/or vibration devices on the communication terminals 3, 5 carried by the maintenance support engineer 17 and/or the maintenance engineer 13. For example, sounds may be emitted by the mobile terminal or by a headphone set worn by the maintenance support engineer 17 and/or the maintenance engineer 13 and optionally, the light and/or vibration devices are triggered in the communication terminals 3, 5.
Furthermore, it will be noted that the voice recognition 7, semantic analysis 9 and warning 11 devices are modules which advantageously may be included in the first 3 and/or the second 5 communication terminal. These modules 7, 9, 11 are, for example, implemented by one or more application(s) integrated into the communication terminal(s). In a variant, the voice recognition 7 and semantic analysis 9 devices may be installed on a remote device.
The maintenance monitoring system 1 comprises a content comparator 25 in addition to the voice recognition 7, semantic analysis 9, warning 11 devices and communication terminals 3, 5. The content comparator 25 is suitable for comparing the instructions and responses with the information contained in the document 21 describing the maintenance procedures.
Steps E1 through E3 are identical to those of
In a step E31, the content comparator 25 is configured for comparing at least one portion of the textual instructions and results with theoretical maintenance task data contained in the maintenance procedure document 21 in order to detect any inconsistency. It will be noted that this step may advantageously be carried out in parallel with step E2.
More particularly, the content comparator 25 is used to compare the instructions given by the maintenance support engineer 17 with the maintenance task data contained in the procedure document 21 in order to avoid the maintenance support engineer 17 leading the maintenance engineer 13 into error.
In the event of detecting any divergence or inconsistency between theoretical maintenance task data on the one hand and the textual instructions and results on the other, the content comparator 25 transmits these inconsistencies to the warning device 11.
In step E41, the warning device 11 is configured for generating a warning 23 when an inconsistency is detected between the instructions and the responses and/or when an inconsistency is detected between theoretical maintenance task data with the textual instructions and results.
The maintenance monitoring system 1 comprises a status comparator 27 in addition to the content comparator 25, the voice recognition 7, semantic analysis 9, warning 11 devices and communication terminals 3, 5. The status comparator 27 is suitable for comparing the parameters coming from the aircraft with the actions requested in the maintenance procedures.
Indeed, the maintenance center 19 in which the maintenance support engineer 17 is located obtains in real time from the aircraft 15 a certain amount of information which is useful for performing this comparison. As an example, this information comprises the status of the aircraft systems (activated, not activated, etc.), failures, downloads/uploads performed and much other information. Thus, a number of actions carried out by the maintenance engineer 13 on the aircraft 15 are visible through the information received by the maintenance support engineer 17.
Steps E1 through E31 are identical to those of
In step E32, the status comparator 27 is configured for comparing a set of parameters of the aircraft received (arrow E5) in real time from the aircraft 15 with theoretical maintenance task data contained in the procedure document 21 in order to ensure the correct execution of operations carried out by the maintenance engineer 13.
Steps E31 and E32 are used to ensure that the oral exchanges (instructions and responses) correspond correctly to what is required in the maintenance procedure and reflect the current state of the aircraft.
In the event of detecting any divergence or inconsistency between theoretical maintenance task procedure data and the aircraft's parameters, the status comparator 27 transmits these inconsistencies to the warning device 11.
In step E42, the warning device 11 is configured for generating a warning 23 when an inconsistency is detected between the instructions and the responses and/or when an inconsistency is detected between theoretical maintenance task data with the textual instructions and results, and/or when an inconsistency is detected between theoretical maintenance task procedure data and the aircraft's parameters.
Thus, a warning is generated in the event of failure during voice recognition or semantic analysis, in the event of misunderstanding between the interlocutors 13, 17, in the event of non-compliance with procedures by the maintenance support engineer 17 (an instruction error) or by the maintenance engineer 13 (error in carrying out an instruction), and when the results of voice recognition or comparisons do not allow it to be concluded that the planned task has been successfully completed.
According to yet another embodiment, the maintenance monitoring system 1 comprises an automatic translation device in addition to the devices illustrated in any of
The present disclosure enables the number of errors during the execution of maintenance to be reduced and checks that the maintenance actions requested and guided by the maintenance support engineer have actually been carried out by the maintenance engineer. This enables the maintenance support engineer to endorse the actions performed with confidence.
The subject matter disclosed herein can be implemented in software in combination with hardware and/or firmware. For example, the subject matter described herein can be implemented in software executed by a processor or processing unit. In one exemplary implementation, the subject matter described herein can be implemented using a computer readable medium having stored thereon computer executable instructions that when executed by a processor of a computer control the computer to perform steps. Exemplary computer readable mediums suitable for implementing the subject matter described herein include non-transitory devices, such as disk memory devices, chip memory devices, programmable logic devices, and application specific integrated circuits. In addition, a computer readable medium that implements the subject matter described herein can be located on a single device or computing platform or can be distributed across multiple devices or computing platforms.
While at least one exemplary embodiment of the invention(s) is disclosed herein, it should be understood that modifications, substitutions and alternatives may be apparent to one of ordinary skill in the art and can be made without departing from the scope of this disclosure. This disclosure is intended to cover any adaptations or variations of the exemplary embodiment(s). In addition, in this disclosure, the terms “comprise” or “comprising” do not exclude other elements or steps, the terms “a” or “one” do not exclude a plural number, and the term “or” means either or both. Furthermore, characteristics or steps which have been described may also be used in combination with other characteristics or steps and in any order unless the disclosure or context suggests otherwise. This disclosure hereby incorporates by reference the complete disclosure of any patent or application from which it claims benefit or priority.
| Number | Date | Country | Kind |
|---|---|---|---|
| 14 57324 | Jul 2014 | FR | national |
| Number | Name | Date | Kind |
|---|---|---|---|
| 5926790 | Wright | Jul 1999 | A |
| 20030202638 | Eringis | Oct 2003 | A1 |
| 20040143428 | Rappaport | Jul 2004 | A1 |
| 20070083300 | Mukheriee | Apr 2007 | A1 |
| 20070100583 | Yano et al. | May 2007 | A1 |
| 20070213988 | Hanson | Sep 2007 | A1 |
| 20110122890 | Johnson | May 2011 | A1 |
| 20120053777 | Meisels | Mar 2012 | A1 |
| 20130127980 | Haddick | May 2013 | A1 |
| 20130278631 | Border | Oct 2013 | A1 |
| 20140207419 | Messinger | Jul 2014 | A1 |
| 20150120388 | Tan | Apr 2015 | A1 |
| 20150242800 | Ellenby | Aug 2015 | A1 |
| Number | Date | Country |
|---|---|---|
| 2 917 195 | Dec 2008 | FR |
| Entry |
|---|
| French Search Report for Application No. 1457324 dated Jan. 13, 2015. |
| Number | Date | Country | |
|---|---|---|---|
| 20160031570 A1 | Feb 2016 | US |
