Patent Application
20220236860
The present invention relates to a visualization system and a method thereof, and more particularly to a visualization system based on artificial intelligence inference and a method thereof.
In recent years, with the popularization and rapid development of artificial intelligence (AI), various applications that combine artificial intelligence have sprung up. However, there are certain thresholds for using artificial intelligence, so how to use artificial intelligence more conveniently has become one of the problems that manufacturers urgently want to solve.
Generally speaking, the conventional method of using artificial intelligence requires the user to train the model first, and then store the trained model in the file directory of the inference system, and then the inference system selects the trained model for deployment of application programming interface (API) services. However, the conventional inference system does not have a visual interface part, the user must link the source data with the API service by a program and check the identification result on a graphical interface of the program. In other words, when the user wants to apply AI to a new situation, the user needs to re-train a new model, the conventional method does not permit the user to directly deploy the model by a dragging manner, and the user cannot quickly and intuitively view the recognition results and accuracy of the applied model. Therefore, the conventional method has a problem of insufficient convenience in model selection and operation.
According to above-mentioned contents, what is needed is to develop an improved technical solution to solve the conventional technical problem of insufficient convenience in model selection and operation.
The present invention discloses a visualization system based on artificial intelligence inference. The visualization system includes a storage module, an initialization module, a loading module, an executing module and display module. The storage module is configured to store at least one recommended template, a plurality of image data sets from different sources, a plurality of AI models trained with different identification algorithms, and a plurality of dashboards. The at least one recommended template comprises specified at least one of the plurality of image data sets, specified at least one of the plurality of AI models and specified at least one of the plurality of dashboards. The initialization module is connected to the storage module and configured to, in initial, generate a graphical user interface (GUI) to display the plurality of image data sets, and permit to drag and drop the displayed image data set to a candidate block of the graphical user interface as a dragged unit. The loading module is connected to the storage module and the initialization module configured to select one, which comprises the dragged unit, of the at least one recommended template, and load the specified image data set, the specified AI model and the specified dashboard comprised in the selected recommended template. The executing module is connected to the loading module and configured to when an execution command is triggered, input the loaded image data set to the loaded AI model to perform an inference calculation, and generate an inference result based on the inference calculation, and detect whether the selected recommended template has a precision. When the selected recommended template has a precision, the executing module directly load the precision, and when the selected recommended template does not have the precision, the executing module calculates the precision corresponding to the inference result, and set the calculated precision as the precision of the selected recommended template. The display module is connected to the loading module and the executing module, and configured to use the loaded dashboard to display the inference result and the precision, which is directly loaded or calculated, on the GUI.
Furthermore, the present invention discloses a visualization method based on artificial intelligence inference, and the visualization method including following steps of: providing at least one recommended template, a plurality of image data sets from different sources, a plurality of AI models trained with different identification algorithms, and a plurality of dashboards, wherein the at least one recommended template comprises specified at least one of the plurality of image data sets, specified at least one of the plurality of AI models and specified at least one of the plurality of dashboards; in initial, generating a graphical user interface to display the plurality of image data sets, and permitting to drag and drop one of the plurality of displayed image data sets to a candidate block of the graphical user interface as a dragged unit; selecting one, comprising the dragged unit, of the at least one recommended template, and loading the specified image data set, the specified AI model and the specified dashboard of the selected recommended template; when an execution command is triggered, inputting the loaded image data set into the loaded AI model to perform an inference calculation, and generating an inference result based on the inference calculation; detecting whether the selected recommended template has a precision, and when the selected recommended template has the precision, directly loading the precision, and when the selected recommended template does not have the precision, calculating the precision corresponding to the inference result, and setting the calculated precision as the precision of the selected recommended template; using the loaded dashboard to display the inference result and the precision, which is directly loaded or calculated, on the graphical user interface.
According to above-mentioned system and method of the present invention, the difference between the system and method of the present invention and the conventional technology is that in the system and method of the present invention the GUI can provide a user to drag and select the image data set, and load and display the recommended template matching the selection result, the recommended template automatically specifies the AI model and the dashboard for the selected image data set, and after the inference calculation is performed, the inference result and the precision of the recommended template are displayed as the basis of adjusting the recommended template.
The aforementioned technical solution of the present invention can achieve the technical effect of improving convenience in model selection and operation.
The structure, operating principle and effects of the present invention will be described in detail by way of various embodiments which are illustrated in the accompanying drawings.
The following embodiments of the present invention are herein described in detail with reference to the accompanying drawings. These drawings show specific examples of the embodiments of the present invention. These embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. It is to be acknowledged that these embodiments are exemplary implementations and are not to be construed as limiting the scope of the present invention in any way. Further modifications to the disclosed embodiments, as well as other embodiments, are also included within the scope of the appended claims.
These embodiments are provided so that this disclosure is thorough and complete, and fully conveys the inventive concept to those skilled in the art. Regarding the drawings, the relative proportions and ratios of elements in the drawings may be exaggerated or diminished in size for the sake of clarity and convenience. Such arbitrary proportions are only illustrative and not limiting in any way. The same reference numbers are used in the drawings and description to refer to the same or like parts. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. As used herein, the term “or” includes any and all combinations of one or more of the associated listed items.
It will be acknowledged that when an element or layer is referred to as being “on,” “connected to” or “coupled to” another element or layer, it can be directly on, connected or coupled to the other element or layer, or intervening elements or layers may be present. In contrast, when an element is referred to as being “directly on,” “directly connected to” or “directly coupled to” another element or layer, there are no intervening elements or layers present.
In addition, unless explicitly described to the contrary, the words “comprise” and “include”, and variations such as “comprises”, “comprising”, “includes”, or “including”, will be acknowledged to imply the inclusion of stated elements but not the exclusion of any other elements.
The environment where the present invention is applied is described before illustration of the visualization system based on artificial intelligence inference and a method thereof. The present invention applies a GUI to permit a user to drag and drop image data sets from different sources, for example, images of traffic flow, images of parts or images of the defect parts, and also permit the user to select the image data set from different sources at the same time, for example, the user can select the images of parts and the images of the defect parts at the same time, so that the suitable one of the recommended templates can be automatically loaded according to the selected image data sets, and the AI model appropriate for the image data sets can be used. As a result, the present invention can improve convenience in AI model selection and operation.
The visualization system based on artificial intelligence inference and a method thereof of the present invention will hereinafter be described in more detail with reference to the accompanying drawings. Please refer to
The initialization module 120 is connected to the storage module 110, and configured to in initial, generate a graphical user interface (GUI) to display the image data sets, and permit to drag and drop the displayed image data sets to a candidate block of the GUI as a dragged unit. In actual implementation, displaying the image data sets on the GUI is performed by image blocks, and different image blocks represent different image data sets, respectively. The user can use a cursor to drag and drop the image block representing for the selected image data set, to achieve the purpose of selecting the image data set, and the image block dragged to the candidate block is used as the dragged unit. Furthermore, a data link relationship between the image data set and the AI model in the candidate block is permitted to re-adjust by a dragging-and-dropping manner, and the inference calculation is performed again based on the re-adjusted data link relationship.
The loading module 130 is connected to the storage module 110 and the initialization module 120 and configured to screen out and select the recommended template which includes the dragged unit, and then load the specified image data set, the AI model and the dashboard of the selected recommended template based on the selected recommended template. For example, suppose that a recommended template includes the specified image data set being part A, the AI model being YOLO, the dashboard being a bar chart, when the image data set represented by the dragged unit is also images of part A, the recommended template is selected to load because of including the image data set being the part A.
The executing module 140 is connected to the loading module 130, and when an execution command is triggered, the executing module 140 is configured to input the loaded image data set to the loaded AI model, so that inference calculation is performed and an inference result is generated based on the inference calculation. The executing module 140 also detects whether the selected recommended template has a precision, if the selected recommended template has the precision, the precision is directly loaded; otherwise, the precision corresponding to the inference result is calculated, and the calculated precision is set as the precision of the selected recommended template. In actual implementation, an image block or graphical button can be generated on the GUI for the user to click, and when the image block or the graphical button is clicked, the execution command is triggered to perform evaluation and inference. Furthermore, the calculation of the precision of the recommended template can be implemented by using confusion matrix or other similar performance measure index, and even the calculation result can be stored as the history record corresponding to the recommended template. Furthermore, when the precision is lower than a preset value, the corresponding recommended template can be loaded to display the specified image data set, the specified AI model and the specified dashboard thereof in the candidate block as the dragged units, and the user is permitted to add, delete or adjust the dragged units.
The display module 150 is connected to the loading module 130 and the executing module 140, and configured to use the loaded dashboard to display the inference result and the precision, which is directly loaded or calculated, on the GUI together. For example, in a condition that the dashboard is a bar chart, the inference result and the precision can be digitized and then expressed in a form of the bar chart. In actual implementation, the dashboard can display various messages in a dashboard at the same time.
It is to be noted that it is to be particularly noted that, in actual implementation, the modules of the present invention can be implemented by various manners, including software, hardware or any combination thereof, for example, in an embodiment, the module can be implemented by software and hardware, or one of software and hardware. Furthermore, the present invention can be implemented fully or partly based on hardware, for example, one or more module of the system can be implemented by integrated circuit chip, system on chip (SOC), a complex programmable logic device (CPLD), or a field programmable gate array (FPGA). The concept of the present invention can be implemented by a system, a method and/or a computer program. The computer program can include computer-readable storage medium which records computer readable program instructions, and the processor can execute the computer readable program instructions to implement concepts of the present invention. The computer-readable storage medium can be a tangible apparatus for holding and storing the instructions executable of an instruction executing apparatus. The computer-readable storage medium can be, but not limited to electronic storage apparatus, magnetic storage apparatus, optical storage apparatus, electromagnetic storage apparatus, semiconductor storage apparatus, or any appropriate combination thereof. More particularly, the computer-readable storage medium can include a hard disk, a RAM memory, a read-only-memory, a flash memory, an optical disk, a floppy disc or any appropriate combination thereof, but this exemplary list is not an exhaustive list. The computer-readable storage medium is not interpreted as the instantaneous signal such as a radio wave or other freely propagating electromagnetic wave, or electromagnetic wave propagated through waveguide, or other transmission medium (such as optical signal transmitted through fiber cable) or electric signal transmitted through electric wire. Furthermore, the computer readable program instruction can be downloaded from the computer-readable storage medium to each calculating/processing apparatus, or downloaded through network, such as internet network, local area network, wide area network and/or wireless network, to external computer equipment or external storage apparatus. The network includes copper transmission cable, fiber transmission, wireless transmission, router, firewall, switch, hub and/or gateway. The network card or network interface of each calculating/processing apparatus can receive the computer readable program instructions from network, and forward the computer readable program instruction to store in computer-readable storage medium of each calculating/processing apparatus. The computer program instructions for executing the operation of the present invention can include source codes or object code programmed by assembly language instructions, instruction-set-structure instructions, machine instructions, machine-related instructions, micro instructions, firmware instructions or any combination of one or more programming language. The programming language include object oriented programming language, such as Common Lisp, Python, C++, Objective-C, Smalltalk, Delphi, Java, Swift, C#, Perl, Ruby, and PHP, or regular procedural programming language such as C language or similar programming language. The computer readable program instruction can be fully or partially executed in a computer, or executed as independent software, or partially executed in the client-end computer and partially executed in a remote computer, or fully executed in a remote computer or a server.
Please refer to
In an embodiment, a step 270 can be performed after the step 260. When a creation command is executed, the image data sets, the AI models and the dashboards are permitted to drag and drop to the candidate block, the data pre-processing is then performed on the image data set, and image features of the pre-processed image data set are analyzed, so that the AI model can be selected to be the new recommended template based on the image features. The data pre-processing can improve the identification speed and the precision, to facilitate to select the appropriate AI model based on the image content.
The embodiment of the present invention will be described in following paragraphs with reference to
Please refer to
Please refer to
According to above-mentioned contents, the difference between the present invention and conventional technology is that in the system and method of the present invention the GUI can provide a user to drag and select the image data set, and load and display the recommended template matching the selection result, the recommended template automatically specifies the AI model and the dashboard for the selected image data set, and after the inference calculation is performed, the inference result and the precision of the recommended template are displayed as the basis of adjusting the recommended template. Therefore, the aforementioned technical solution of the present invention can solve the conventional technical problems, so as to achieve the technical effect of improving convenience in model selection and operation.
The present invention disclosed herein has been described by means of specific embodiments. However, numerous modifications, variations and enhancements can be made thereto by those skilled in the art without departing from the spirit and scope of the disclosure set forth in the claims.
