Patent Application
20230136853
This application claims priority to Chinese Patent Application No. 202111274278.4 filed on Oct. 29, 2021, the contents of which are incorporated by reference herein.
The subject matter herein generally relates to an industrial maintenance field, in particular, relates to a method for maintaining equipment by remote operation and assistance and an electronic device.
Maintenance works on different equipment become more complex as the equipment becomes more complex. After a placement machine fails, senior engineers are required to perform maintenance, and perhaps experts and manufacturers are required to provide assistance. Maintenance skills of on-site maintenance personnel are uneven, and maintenance efficiency is low, the equipment utilization rate is affected. Sparse experts, high cost of technical assistance, long training cycle for new recruits, and lack of experience will lead to problems such as low maintenance efficiency, heavy time-consumption and labor-intensive work, with poor safety.
Furthermore, traditional training methods are not efficient, and enterprises often need to spend a lot of training time and expenses. Learners cannot be productive right away, thus consuming the time and energy of teaching staff. At present, although there are a few application softwares in the field of industrial maintenance that can provide auxiliary maintenance functions, the maintenance needs of specific equipment such as placement machines are not met, nor can it meet the maintenance needs of specific faults. In addition, most of these softwares need to be connected to the Internet to run, and for some enterprises with high security and confidentiality requirements, the maintenance site is not connected to an external network.
Implementations of the present disclosure will now be described, by way of embodiment, with reference to the attached figures.
It will be appreciated that for simplicity and clarity of illustration, where appropriate, reference numerals have been repeated among the different figures to indicate corresponding or analogous elements. In addition, numerous specific details are set forth in order to provide a thorough understanding of the embodiments described herein. However, it will be understood by those of ordinary skill in the art that the embodiments described herein can be practiced without these specific details. In other instances, methods, procedures, and components have not been described in detail so as not to obscure the related relevant feature being described. Also, the description is not to be considered as limiting the scope of the embodiments described herein. The drawings are not necessarily to scale and the proportions of certain parts may be exaggerated to better illustrate details and features of the present disclosure.
The present disclosure, including the accompanying drawings, is illustrated by way of examples and not by way of limitation. Several definitions that apply throughout this disclosure will now be presented. It should be noted that references to “an” or “one” embodiment in this disclosure are not necessarily to the same embodiment, and such references mean “at least one”.
The term “module”, as used herein, refers to logic embodied in hardware or firmware, or to a collection of software instructions, written in a programming language, such as, Java, C, or assembly. One or more software instructions in the modules can be embedded in firmware, such as in an EPROM. The modules described herein can be implemented as either software and/or hardware modules and can be stored in any type of non-transitory computer-readable medium or other storage device. Some non-limiting examples of non-transitory computer-readable media include CDs, DVDs, BLU-RAY, flash memory, and hard disk drives. The term “comprising” means “including, but not necessarily limited to”; it specifically indicates open-ended inclusion or membership in a so-described combination, group, series, and the like.
At block 201, in response to a failure notification from an electronic device, obtaining on-site information, and querying corresponding maintenance guidance information according to the on-site information.
In one embodiment, the method is applied to electronic equipment, and the electronic equipment includes, but is not limited to, a mounter machine. When the electronic equipment fails, for example, the mounter machine cannot apply a software patch or an update as a solution, or the placement machine cannot accurately patch components to corresponding position, defective products are produced. When personnel on a production line find the above fault in the mounter machine, a failure notification is issued to notify the maintenance personnel to go to repair the faulty equipment.
In one embodiment, when the electronic device fails, the electronic device generates a failure notification. Exemplarily, when the electronic device cannot be patched correctly, a patch error fault prompt will be displayed on a display screen of the electronic device.
In one embodiment, the backend 105 includes a maintenance guidance sub-module 401.
In one embodiment, the remote expert terminal 107 manages the maintenance guidance information by the backend 105 of the cloud server 104. In one embodiment, the remote expert terminal 107 supplements or adds the original maintenance guidance information.
In one embodiment, the terminal device 101 scans the two-dimensional code of the faulty equipment by the AR glasses 102, obtains the historical maintenance information of the faulty equipment from the cloud server 104, and determines the maintenance guidance information from the historical maintenance information according to selection by user. In one embodiment, the historical maintenance information includes maintenance records of the faulty equipment and maintenance instruction information of the maintenance records. When the historical maintenance information of the faulty equipment is acquired, user determines keywords according to the location where the electronic device has failed, and inputs the keywords into a search box of the AR glasses 102 to search for the corresponding historical maintenance information. When the corresponding historical maintenance information cannot be found, the terminal device 101 returns to the search box and continues to search by the search box. For example, for an operation failure of the mounter machine, the user determines keywords such as “power failure”, and enters the keywords into the search box of the AR glasses 102 to make a search. When finding the corresponding historical maintenance information, the terminal device 101 retrieves the maintenance guidance information corresponding to the historical maintenance information, and assists the user in repairing the faulty equipment.
In one embodiment, the terminal device 101 scans the two-dimensional code of the faulty equipment by the AR glasses 102, obtains the historical maintenance information of the faulty equipment from the cloud server 104, and matches the on-site information with the historical maintenance information, and determines that the historical maintenance information matching the on-site information is the maintenance instruction information. When the historical maintenance information of the faulty equipment is obtained, the terminal device 101 matches the real-time video or picture of the faulty equipment collected by the AR glasses 102 with the historical maintenance information picture or video of the AR glasses 102, and determines the historical maintenance information matching with the real-time video or picture of the faulty equipment, and the real-time video or picture of the faulty equipment is collected by the AR glasses 102. For example, by parsing the real-time videos of the faulty equipment collected by the AR glasses 102, the terminal device 101 intercepts the real-time videos to obtain key video parts, inputs the key video parts into the matching box, and obtains the historical maintenance information matching with the real-time video of the faulty equipment, and takes the historical maintenance information matching with the real-time video of the faulty equipment as the maintenance guidance information.
At block 202, determining whether the maintenance guidance information is existed.
In one embodiment, when the keyword is entered into the search box, if the corresponding historical maintenance information can be obtained, it is determined that the maintenance guidance information exists. If the corresponding maintenance guidance information is not obtained, it is determined that the maintenance guidance information does not exist.
In one embodiment, if the historical maintenance information of the faulty equipment contains historical maintenance information that matches the real-time video of the faulty equipment collected by the AR glasses 102, it is determined that maintenance guidance information exists. If the historical maintenance information of the faulty equipment does not match the faulty equipment collected by the AR glasses 102, the real-time video not matching the historical maintenance information will determine that the maintenance guidance information does not exist.
At block 203, when the maintenance guidance information exists, displaying or playing the maintenance guidance information by the AR glasses 102.
In one embodiment, when the maintenance guidance information is existing, the terminal device 101 calls up the maintenance guidance information from the cloud server 104 by the AR glasses 102, displays or plays the content of the maintenance guidance information, and enables repair of the faulty equipment. In one embodiment, the maintenance guidance information is displayed on the AR glasses 102, so that dedicated/experienced users can be tasked to repair the faulty equipment and improve maintenance efficiency.
In one embodiment, when the user repairs the faulty equipment according to the maintenance guidance information, the AR glasses 102 synchronously collects and records user operation data of an maintenance operation in real time, tracks user operation data and compares the user operation data with the maintenance guidance information. For example, by comparing multiple images of point of fault of the faulty equipment within a preset time, or multiple frames of images captured from the video and images of the parts, with the parts of the fault point included in the maintenance guidance information in a non-fault state of the faulty equipment, this determines whether the operation data is abnormal. If the tracking data compared to the maintenance guidance information is abnormal, a warning message will be output to remind the user that there is an error in the maintenance operation, or in a maintenance procedure or a maintenance method, and that the error should be corrected in time. For example, the electronic equipment cannot attract the iron sheet, resulting in the product not being affixed with the corresponding iron sheet. The user may obtain the solution according to the maintenance guidance information as being that a nozzle of the electronic equipment needs to be replaced. When the user replaces the nozzle according to the maintenance guidance information and the replaced nozzle is inconsistent with the nozzle model properly used by the electronic equipment, the warning message will be output to remind the user that there is an error in the maintenance operation, and check whether the maintenance operation is correct until the user replaces the nozzle with a nozzle properly used in the electronic equipment, than the maintenance procedure is completed.
At block 204, when the maintenance guidance information does not exist, establishing a communication connection with the remote expert terminal 107 by the AR glasses 102, and obtaining real-time maintenance guidance suggestions from the remote expert terminal 107.
In one embodiment, when the maintenance guidance information does not exist, the user cannot carry out maintenance of the faulty equipment, the terminal device 101 establishes a communication connection with the remote expert terminal 107 by the AR glasses 102. Further, the AR glasses 102 transmits the collected on-site information by the first 5G intranet 103 to the cloud server 104 to save the on-site information and the remote expert terminal 107 can obtain the on-site information from the cloud server 104. After the terminal device 101 establishes a communication connection with the remote expert terminal 107, the remote expert terminal 107 calls the on-site information from the cloud server 104 by the second 5G intranet 106, communicates with the user according to the on-site information, and gives maintenance guidance suggestions.
In one embodiment, the AR glasses 102 establish a network connection with the remote expert terminal 107 by the 5G intranet. The user of the AR glasses 102 initiates a video call with a remote expert of the remote expert terminal 107 by the AR glasses 102. When the remote expert receives the video call, a video conference call room is established. In the video conference call room, the user or the remote expert can invite other users or other remote experts to join. When the user who originally initiated the video call chooses to end the video call, the video conference call room will be dismissed, and all the invited remote experts or other users will exit the video conference call room.
In one embodiment, all 5G intranets use adaptive transmission mode to transmit the on-site information. The adaptive transmission mode can dynamically adjust the definition of the video in the on-site information according to the bandwidth of the 5G intranet, preventing the remote expert terminal 107 from being stuck or hanging when viewing the real-time video or picture of the faulty equipment.
In one embodiment, the call records generated after the AR glasses 102 are used to establish the communication connection with the remote expert terminal 107 are saved to the cloud server 104 and managed by the backend 105 of the cloud server 104, including: saving valuable call records and the on-site information, saving valuable call records and on-site information, and deleting worthless call records and on-site information. In one embodiment, the valuable call records or the on-site information include, but are not limited to, significant reference and information on parts of the electronic equipment that frequently fails. Worthless call records or on-site information include, but are not limited to, no significant reference being found and information which is irrelevant.
The above are only specific embodiments of the present application, but are not to limit the protection scope of the present application. For those of ordinary skill in the art, improvements can be made without departing from the inventive concept of the present application, but these all belong to the protection scope of the present application.
Those skilled in the art can understand that the schematic diagram shown in
The at least one processor 602 may be a Central Processing Unit (CPU), and may also be a general-purpose processor, a Digital Signal Processors (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The at least one processor 602 can be a microprocessor or the at least one processor 602 can also be any conventional processor, etc. The at least one processor 602 is the control center of the electronic device 6, using various interfaces and lines to connect various parts of the entire electronic device 6.
The storage 601 can be used to store the computer program 603, and the at least one processor 602 implements the electronic program by executing the computer program 603 stored in the storage 601 and calling up the data stored in the storage 601. The storage 601 may mainly include a stored program area and a stored data area, wherein the stored program area may store an operating system, and at least one application program required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The storage data area may store data (such as audio data) created in the use of the electronic device 6, etc. In addition, the storage 601 may include non-volatile storage such as a hard disk, an internal memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card (Flash Card), at least one disk storage device, flash memory device, or other non-volatile solid state storage device.
The exemplary embodiments shown and described above are only examples. Even though numerous characteristics and advantages of the present disclosure have been set forth in the foregoing description, together with details of the structure and function of the present disclosure, the disclosure is illustrative only, and changes may be made in the detail, including in matters of shape, size, and arrangement of the parts within the principles of the present disclosure, up to and including the full extent established by the broad general meaning of the terms used in the claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 202111274278.4 | Oct 2021 | CN | national |
