Patent Application
20230060459
The present disclosure relates to a classification technology and, specifically to an image object classification optimizing method and system.
The development and application of machine learning have gradually become an apparent trend. A large amount of data (or information) can be used to train machine learning models, and the trained models can be used to obtain certain prediction information.
Object classification technology is widely used. Many image classification optimizing methods adopt neural networks as a basic architecture, such as classifying digital data generated by converting pictures.
The conventional object classification model usually emphasizes that it can provide a classification result with the highest reliability. However, it will ignore other possible classification results and is not suitable for specific applications.
In light of this, it is necessary to provide a technical solution different from the past to solve a prior art problem.
One objective of the present disclosure is to provide an image object classification optimizing method that can optimize a classification result of image objects and favorably suitable for digital files containing multiple image categories.
Another objective of the present disclosure is to provide an image object classification optimizing system that can optimize a classification result of image objects and favorably suitable for digital files containing multiple image categories.
Another objective of the present disclosure is to provide a tangible, non-transitory, computer readable medium that can optimize a classification result of image objects and favorably suitable for digital files containing multiple image categories.
To achieve the above objective, one aspect of the present disclosure provides an image object classification optimizing method, executed by a processor coupled to a memory, including: providing an image file including at least one image object; performing a process of characteristics enhancement on the image object; performing a process of characteristics classification on the enhanced image object by an odd number of two-dimensional masks whose sizes are sequentially doubled, based on a plurality of characteristic parameters of a preferred classification model, to generate a plurality of classification results; and estimating variabilities of the plurality of classification results, sorting the variabilities, and selecting at least one of the classification results whose variability is lower than a variation tolerance as at least one optimization result according to the sorting result.
In one embodiment of the present disclosure, the process of characteristics classification is performed by three two-dimensional masks with sizes of 13×13, 26×26, and 52×52.
In one embodiment of the present disclosure, before performing the process of characteristics enhancement, the method further includes: converting information of pixels of the image file from a two-dimensional array to a one-dimensional array.
In one embodiment of the present disclosure, the process of characteristics enhancement is performed on the image object converted into the one-dimensional array, through a one-dimensional mask, and a number of elements in the one-dimensional mask is the same as a number of elements in a two-dimensional mask.
In one embodiment of the present disclosure, before converting the information of pixels of the image file from the two-dimensional array to the one-dimensional array, the method further includes: converting the image file from an original size to a compressed size according to a compression ratio.
In one embodiment of the present disclosure, there is a difference interval between the original size and the compressed size of the image file, and the information of pixels in the difference interval is expressed as a specific graphic feature to reform the image file.
In one embodiment of the present disclosure, the plurality of classification results includes a plurality of image categories, and the plurality of image categories includes a plurality of schematic diagrams of characteristics of an electronic component.
Another aspect of the present disclosure provides an image object classification optimizing system including a processor coupled to a memory storing at least one instruction configured to be executed by the processor to perform the above method.
Another aspect of the present disclosure provides a tangible, non-transitory, computer readable medium storing instructions that cause a computer to execute the above method.
The image object classification optimizing method, system, and tangible non-transitory computer readable medium of the present disclosure are provided for providing an image file including at least one image object; performing a process of characteristics enhancement on the image object; performing a process of characteristics classification on the enhanced image object by an odd number of two-dimensional masks whose sizes are sequentially doubled, based on a plurality of characteristic parameters of a preferred classification model, to generate a plurality of classification results; and estimating variabilities of the plurality of classification results, sorting the variabilities, selecting at least one of the classification results whose the variability is lower than a variation tolerance as at least one optimization result according to the sorting result. Thus, after the above object classification optimization process, the classification results of image objects can be effectively optimized, which is beneficial to digital files containing multiple image categories.
The following description of the various embodiments is provided to illustrate the specific embodiments of the present disclosure. Furthermore, directional terms mentioned in the present disclosure, such as upper, lower, top, bottom, front, rear, left, right, inner, outer, side, surrounding, central, horizontal, lateral, vertical, longitudinal, axial, radial, uppermost, and lowermost, which only refer to the direction of drawings. Therefore, the directional terms configured as above are for illustration and understanding of the present disclosure and are not intended to limit the present disclosure.
Please refer to
For example, the image object classification optimizing system may be configured to include a processor and a memory coupled to the processor, wherein the memory stores at least one instruction executed by the processor to perform an image object classification optimizing method provided by another aspect of the present disclosure, which is illustrated as the following but is not limited here.
Another aspect of the present disclosure that provides an image object classification optimizing method that is executed by a processor coupled to a memory and includes: providing an image file, e.g., being converted from a document file, wherein the image file includes at least one image object; performing a process of characteristics enhancement on the image object; performing a process of characteristics classification on the enhanced image object by an odd number of two-dimensional masks whose sizes are sequentially doubled, based on a plurality of characteristic parameters of a preferred classification model, to generate a plurality of classification results; and estimating variabilities of the plurality of classification results, sorting the variabilities, and selecting at least one of the classification results whose variability is lower than a variation tolerance as at least one optimization result according to the sorting result. The following examples illustrate the sample states that can be implemented to understand the relevant content, but not limited here.
For example, as shown in
As shown in
The external data can also be the document file, and the processor can convert the document file to provide the image file. The image file is at least provided in the manner mentioned above and is not limited here. For example, the document file can be a digital document file containing various graphic examples, such as different drawing views of electronic components, and text examples, such as explanatory text. For example, “*. pdf,” a portable document format, e.g., the content of the file can include technical documents (datasheets) related to electronic components, but not limited here.
The digital document file can also be in other file formats, e.g., other file-formats, including graphics and text, such as “*.doc” or “*.odt.” Additionally, the external data can also include other data, such as tables. The image file can be in compressed or uncompressed file format, such as “*.jpg,” but it is not limited here. The image file can also be in other file formats, such as “*.png” or “*.bmp.”
The following image files can refer to file data or screen content in which data of the file is displayed on a display device and can also be called pictures. Subsequently, step S2 can be performed.
Optionally, as shown in
Optionally, as shown in
Optionally, as shown in
For example, one 1×9 one-dimensional mask can be adopted to replace one 3×3 two-dimensional mask, but not limited here, e.g., the one-dimensional mask can be re-sized to 1×36 for characteristics enhancement of at least one object in the one-dimensional image file according to actual application. For example, information of pixels in one-dimensional array is processed to enhance characteristics in a sliding window manner by using one 1×9 one-dimensional mask.
For example, a layer transfer method in a transfer learning technology can be used, e.g., a pre-train weight method used for object detection may be adopted. For example, coco data set can be adopted as preset parameters of a one-dimensional mask, make the mask more sensitive to detect lines and patterns, so that the mask can effectively highlight the difference between lines, patterns, and image background when the mask slides in the one-dimensional array of the image file, in which the calculating manner is understandable by those skilled in the art. Subsequently, step S5 can be performed.
Optionally, as shown in
An application scenario of electronic components is taken as an example. Schematic diagrams of various features of electronic components can be classified, such as appearance features, electrical features, and application features.
For example, directional views of electronic components may be adopted. Each of the drawing-view diagrams (such as top-view, bottom-view, and side-view) has line segments and geometries for showing the components' size. In addition, pin-assignment diagrams (also called such as pin configurations, pinout diagrams, pin function diagrams etc., e.g., integrated circuit (IC) diagrams showing the feature of pins, hereinafter referred to as IC diagrams) of the electronic components may be adopted. Each of the pin-assignment diagrams of the IC package has geometries showing pins and text.
For example, circuit diagrams may be adopted. For example, the circuit formed by connecting symbols of electronic components has geometric graphs and lines. Also, characteristic curve diagrams may be adopted. For example, characteristic curve diagrams of the voltage or current of electronic components have waveform information formed by continuously extending lines.
For example, signal timing diagrams may be adopted. For example, each of the clock's timing diagrams, input, and output signals of electronic components has a closed block waveform composed of continuously extending lines used to present a continuous waveform relationship. The main difference between the signal timing diagram and the characteristic curve diagram is there are more closed blocks and turning lines, but not limited here.
In addition, other image object classification scenarios, such as a mobile phone manual, may include a schematic diagram of appearance functions and a schematic diagram of screen functions. Graphic characteristics of the mobile phone manual can also be analyzed as a basis for classifying image objects.
As shown in
It should be noted that, by simultaneously using odd-numbered two-dimensional masks with sizes that are doubly enlarged in sequence, such as using three two-dimensional masks with sizes of 13×13 (smaller), 26×26 (medium), and 52×52 (larger), which are not limited here, it is suitable for detecting various image objects in the image file and performing a characteristics classification process to generate the classification results. The various image objects may be larger-sized image objects (such as large tables, IC diagrams, and circuit diagrams), medium-sized image objects (such as drawing-view diagrams, reference signs of size, and small tables), or smaller-sized image objects (such as reference signs of pins, texts, and symbols of electronic components). In addition, the characteristics classification process can be suitable for the above various graphic examples that are possible in various specification documents of electronic components, but is not limited here.
To enable a person to understand the features of embodiments of the present disclosure, the following is an example of inputting a technical file (datasheet) of an electronic component as a document file and illustrates the classification process of the objects in the above embodiments. Still, it is not intended to be used as a limit.
For example, some specification documents of electronic components will be used as examples to illustrate a process that three two-dimensional masks with dimensions of 13×13, 26×26, and 52×52 are adopted to detect image objects in the specification documents of electronic components.
An example of method may include compressing pictures that need to be detected and classified, wherein the compressed pictures' size is 512×512 pixels, there are 35,000 pictures in a data set, and the proportion of the object's size is shown in Table 1 as below.
Please refer to
First, a 13×13 two-dimensional mask is taken as an example for detecting smaller objects and finer features in the picture shown in
In another aspect, a 26×26 two-dimensional mask is taken as an example being a general-sized mask for detecting medium objects in the picture shown in
In another aspect, a 52×52 two-dimensional mask is taken as an example for detecting lager objects, so that global features can be seen from the large objects, avoiding a situation where only small features can be detected by a small mask. For example, the 52×52 two-dimensional mask can be used to detect large objects, such as those objects which can be detected by the 52×52 two-dimensional mask, in the picture shown in
It can be understandable from the above examples; the above embodiments of the present disclosure can be used to automatically select objects by a machine. For example, the processed results of the three different sized masks can be combined to avoid using a single mask resulting in the processed results that are too biased. By masks with various sizes, different sized objects in the picture can be detected as an important reference for subsequent variability analysis.
As shown in
It should be noted that, by estimating a plurality of variabilities of the classification results, it can be known that adopting variabilities of the odd-numbered classification results, such as variabilities of the classification-trusted levels, generated by simultaneously using odd-numbered (e.g., three) two-dimensional masks with different sizes. The variabilities of the classification-trusted levels can be sorted. According to a quality condition, such as selected by a variation tolerance value from a large amount of variability statistics data, e.g., take any value in a range from 1×10−2 to 1×10−4 as a threshold, such as 1×10−2, 1×10−3, 1×10−4, at least one of the classification results, which variability is less than the variation tolerance value, can be selected as at least one optimization result. For example, there are two of three variabilities lower than the threshold, which means that the classification results are more closer, i.e., more likely to be the desired category. Therefore, the classification results corresponding to these two variabilities can be used as the optimization result.
The above examples illustrate the image object classification optimizing method of the present disclosure, so that embodiments of the present disclosure can be summarized as follows.
Optionally, in one embodiment, the process of characteristics classification is performed by three two-dimensional masks with sizes of 13×13, 26×26, and 52×52. In this way, image objects can be automatically filtered, so that objects with different sizes in the picture can be detected.
Optionally, in one embodiment, before performing the process of characteristics enhancement of the image object, the image object classification optimizing method further includes: converting information of pixels of the image file from a two-dimensional array to a one-dimensional array. In this way, when a computer is used for image data processing, an index of the data can be used for processing to speed up the processing speed.
Optionally, in one embodiment, the process of characteristics enhancement is performed on the image object converted into the one-dimensional array through a one-dimensional mask, and a number of elements in the one-dimensional mask is the same as a number of elements in a two-dimensional mask. In this way, sliding window processing can be performed by the one-dimensional mask, so as to perform the characteristics enhancement operation of the image object.
Optionally, in one embodiment, before converting the information of pixels of the image file from the two-dimensional array to the one-dimensional array, the method further includes: converting the image file from an original size to a compressed size according to a compression ratio. In this way, an image file with less data can be formed for subsequent processing, and the number of pixels to be processed can be reduced, which is beneficial to reduce the amount of calculation and the time required for processing.
Optionally, in one embodiment, there is a difference interval between the original size and the compressed size of the image file, and the information of pixels in the difference interval is expressed as a specific graphic feature to reform the image file. In this way, it can be used in application scenarios that need to process a specific size of image. When information of the pixels of the image file is compressed, the specific size of image (such as the degree that can be used by humans or machines to distinguish the content) can be maintained for batch processing.
Optionally, in one embodiment, the plurality of classification results include a plurality of image categories, and the plurality of image categories includes a plurality of schematic diagrams of characteristics of an electronic component, such as drawing views of electronic components, IC diagrams of the electronic components, circuit diagrams, characteristic curve diagrams, and signal timing diagrams, but are not limited here. In this way, different image categories, such as drawing views of electronic components, IC diagrams of the electronic components, circuit diagrams, characteristic curve diagrams, and signal timing diagrams, can be effectively classified, which is helpful for a related person to accelerate data interpretation, analysis, and related development schedule.
In another aspect, the present disclosure further provides an image object classification optimizing system, which includes a processor coupled to a memory storing at least one instruction configured to be executed by the processor to perform the above-mentioned image object classification optimizing method. The coupling manner can be wired or wireless.
For example, the image object classification optimizing system can be configured as an electronic device with data processing functions. The electronic device can be cloud platform machines, servers, desktop computers, notebook computers, tablets, or smartphones, but is not limited here, to perform the above-mentioned image object classification optimizing method, which is not described here again.
In another aspect, the present disclosure further provides a tangible, non-transitory, computer readable medium, storing instructions that cause a computer to execute operations including: providing an image file including at least one image object; performing a process of characteristics enhancement on the image object; performing a process of characteristics classification on the enhanced image object by an odd number of two-dimensional masks whose sizes are sequentially double, based on a plurality of characteristic parameters of a preferred classification model, to generate a plurality of classification results; and estimating variabilities of the plurality of classification results and sorting the variabilities, and selecting at least one of the classification results whose variability is lower than a variation tolerance as at least one optimization result according to the sorting result.
After the instructions are loaded and executed by the computer, the computer can execute the aforementioned image object classification optimizing method. For example, several program instructions can be implemented by using existing programming languages to implement the above-mentioned image object classification optimizing methods, such as Python combining with Numpy, Matplotlib, and TENSORFLOW packages, but not limited here.
In another aspect, the present disclosure further provides a computer-readable medium, such as an optical disc, a flash drive, or a hard disk, but not limited here. It should be understandable that the computer-readable medium can further be configured as other forms of computer data storage medium, e.g., cloud storage (such as ONEDRIVE, GOOGLE Drive, AZURE Blob, or a combination thereof), or data server, or virtual machine. The computer can read the program instructions stored in the computer-readable medium. After the computer loads and executes the program instructions, the computer can complete the above-mentioned image object classification optimizing method.
In summary, the image object classification optimizing method, system, and tangible non-transitory computer readable medium of the present disclosure are provided for providing an image file including at least one image object; performing a process of characteristics enhancement on the image object; performing a process of characteristics classification on the enhanced image object by an odd number of two-dimensional masks whose sizes are sequentially double, based on a plurality of characteristic parameters of a preferred classification model, to generate a plurality of classification results; and estimating variabilities of the plurality of classification results, sorting the variabilities, and selecting at least one of the classification results whose variability is lower than a variation tolerance as at least one optimization result according to the sorting result. Thus, after the above object classification optimization process, the classification results of image objects can be effectively optimized, which is beneficial to digital files containing multiple image categories.
Although the present disclosure has been disclosed in preferred embodiments, which are not intended to limit the disclosure, those skilled in the art can make various changes and modifications without departing from the spirit and scope of the disclosure. Therefore, the scope of protection of the present disclosure is defined as definitions of the scope of the claims.
