Patent Application
20230162466
The disclosure relates to the field of computer technology, and in particular, to a method and a computer system for displaying an identification result.
In the field of computer technology, a variety of applications (APPs) for identifying objects to be identified are available, such as applications for identifying plants. These applications usually receive images from users (including static images, dynamic images, videos, etc.), and identify the objects to be identified in the images based on the identification model established by artificial intelligence technology to obtain identification results. For instance, the identification result obtained when the object is a living creature may be its species. The image from the user usually includes at least a portion of the object to be identified, for example, the image photographed by the user includes stems, leaves, and flowers of the plant to be identified. The identification result may completely match or match to a high degree with the object to be identified in the image, or may have a low degree of matching with the object to be identified in the image. The identification results are usually displayed in the form of pictures.
The disclosure aims to provide a method and a computer system for displaying an identification result.
According to the first aspect of the disclosure, the disclosure provides a method for displaying an identification result, and the method includes the following steps. An image capable of presenting at least a portion of an object to be identified is received, and at least a portion of the object to be identified presented by the image is identified. In response to obtaining an identification result, a first picture presenting the identification result is displayed. The first picture includes marks targeting a portion or a plurality of portions of the identification result.
According to the second aspect of the disclosure, the disclosure provides a method for displaying an identification result, and the method includes the following steps. An image capable of presenting at least a portion of an object to be identified is received, and at least a portion of the object to be identified presented by the image is identified. In response to obtaining an identification result, a fifth picture or a plurality of fifth pictures related to the identification result are displayed, and each of the fifth pictures corresponds to a portion of the identification result.
According to the third aspect of the disclosure, the disclosure provides a method for displaying an identification result, and the method includes the following steps. An image capable of presenting a first portion of an object to be identified is received, and the first portion is identified. In response to obtaining an identification result, an eighth picture presenting a first portion of the identification result is displayed. The eighth picture further presents a second portion of the identification result that is different from the first portion.
According to the fourth aspect of the disclosure, the disclosure provides a computer system for displaying an identification result, and the computer system includes a processor or a plurality of processors and a memory or a plurality of memories. The memory or the plurality of memories are configured to store a series of computer-executable instructions and computer-accessible data associated with the series of computer-executable instructions. When the series of computer-executable instructions are executed by the processor or the plurality of processors, the processor or the plurality of processors are enabled to perform the abovementioned method.
According to the fifth aspect of the disclosure, the disclosure provides a non-transitory computer readable storage medium. The non-transitory computer readable storage medium stores a series of computer-executable instructions, and when the series of computer-executable instructions are executed by a computer apparatus or a plurality of computer apparatuses, the computer apparatus or the plurality of computer apparatuses are enabled to perform the abovementioned method.
Other features of the disclosure and advantages thereof will become apparent from the following detailed description of exemplary embodiments of the disclosure with reference to the accompanying drawings.
The accompanying drawings, which form a part of the specification, illustrate embodiments of the disclosure and together with the description serve to explain the principles of the disclosure.
The disclosure may be more clearly understood from the following detailed description with reference to the accompanying drawings described as follows.
Note that in the embodiments described below, the same reference numerals are used in common between different figures to denote the same parts or parts having the same function, and repeated description thereof is omitted. In this specification, similar numbers and letters are used to denote similar items, and therefore, once an item is defined in one figure, it does not require further discussion in subsequent figures.
Various exemplary embodiments of the disclosure are described in detail below with reference to the accompanying drawings. It should be noted that the relative arrangement of the components and steps, the numerical expressions and numerical values set forth in these embodiments do not limit the scope of the disclosure unless specifically stated otherwise. In the following description, in order to better explain the disclosure, numerous details are set forth, however it will be understood that the disclosure may be practiced without these details.
The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the disclosure, its application or uses in any way. In all examples shown and discussed herein, any specific value should be construed as illustrative only and not as limiting.
Techniques, methods, and apparatuses known to a person having ordinary skill in the art may not be discussed in detail, but where appropriate, such techniques, methods, and apparatuses should be considered part of the specification.
In some cases, a user inputs an image of all or a portion of an object to be identified into an application capable of performing object identification in order to obtain information about the object to be identified. For instance, when the object to be identified is a plant, the image may include any one or a combination of any one of the roots, stems, leaves, flowers, fruit, and seeds of the plant to be identified, and each of these items may be the entirety or a portion of such an item. The image may be previously stored by the user, photographed in real time, or downloaded from the Internet. The image may include any form of visual presentation, such as a static image, a dynamic image, and a video. The image may be captured using an apparatus including a camera, such as a mobile phone, a tablet computer, etc. The object to be identified may also be any other object except plants, such as an animal, a mineral, fungi, and the like.
An application capable of implementing the method 100 may receive the image from the user and perform object identification based on the image. Identification may include any known method of image-based object identification. For instance, an object to be identified in an image may be identified by a computing apparatus and a pre-trained (or “trained”) object identification model to obtain an identification result (e.g., a species). An identification model may be established based on a neural network (e.g., a deep convolutional neural network (CNN) or a deep residual network (Resnet), etc.). For instance, a certain number of image samples labeled with the species name of the plant are obtained for each plant species, that is, a training sample set. These image samples are used to train the neural network until the output accuracy of the neural network meets the requirements. The image may also be preprocessed before object identification based on the image. Preprocessing may include normalization, brightness adjustment, or noise reduction, and so on. Noise reduction processing may highlight the description of the characteristics in the image and make the characteristics more distinct.
In a specific example, the received image may be as shown in
In an embodiment, the displayed pictures presenting the identification result may be as shown in
The picture shown in
The mark or the portion targeted by the mark may be manipulated. The operation may include clicking, double-clicking, touching, pressing, stretching and zooming, sliding, etc. For instance, the user may click on the region framed by the rectangular frame as shown in
In an embodiment, in response to the mark or the portion targeted by the mark being operated, information related to an object having a characteristic of the portion of the identification result targeted by the mark is displayed. For instance, after the identification result as shown in
The “selection” described here is operation performed by the user. The application executing the method 100 may allow the user to perform positive selection, that is, to select one or more characteristics of the portion to be retained through operation such as clicking and may also allow the user to perform negative selection, that is, delete one or more characteristics of the portions that the user wants to ignore through operation such as clicking.
A characteristic usually refers to the shape of a specific portion of the object or the identification result. For instance, when the portion is a leaf of a plant, the shape characteristic of the leaf may include heart shape, kidney shape, egg shape, oval shape, triangle shape, circle shape, fan shape, sword shape, oblong shape, needle shape, bar shape, diamond shape, and the like. It should be understood that the leaf portion of the plant may also have other categories of characteristics, such as texture characteristics, edge characteristics (smooth or burrs), solitary/opposite characteristics, and the like. In the database, classification may be performed according to each shape characteristic of leaves, that is, the species of plants with this characteristic are stored under the classification of each shape characteristic. Correspondingly, for each characteristic of each portion including the roots, stem, leaves, flowers, fruit, and seeds, the species of the object whose portions have the characteristics may be stored under the characteristic classification (including the name of the species, pictures, text introduction, etc.). According to each characteristic possessed by the portion of the identification result selected by the user, the common species (i.e., the intersection of the species stored under these characteristic classifications) under the classification of these characteristics is selected, that is, the output result that may be displayed when the application of the method 100 is executed in the embodiment.
In an embodiment, in response to obtaining the identification result, pictures (e.g., the third picture and the seventh picture in the claims may be implemented as such pictures) corresponding to the image are also displayed, for example, the received image itself, a partial picture of the image, a thumbnail image of the image, and the like. In an embodiment, it is difficult for the object identification model to obtain an identification result whose degree of matching with the object to be identified meets the requirements based on the received image, that is, no identification result is obtained. In response to no identification result being obtained, the application performing the method 100 may display one or more pictures of one or more objects similar to the object to be identified (e.g., the fourth picture in the claims may be implemented as such a picture). For instance, pictures of other species that are similar to sunflowers may be outputted when one or more portions of a sunflower are included in the image but not identified. For another instance, if the image includes multiple portions of a plant, and it is difficult to find an identification result that matches all the portions, an identification result that matches only some portions may be outputted.
In other embodiments, there may be situations where the identification result does not match one or more portions of the object to be identified. In such a case, each portion of the identification result may be outputted, and the degree of matching of each portion may be marked. For instance, when identifying several characteristics of sunflower roots, stem, leaves, flowers, and fruit, the identification results of roots, stem, leaves, and flowers are correct, but the identification result of fruit may be incorrect. The application that executing the above method may automatically mark the correct portion (that is, the portion with a high degree of matching) with √ (acting as an example only, and other words, symbols, or pictures, etc. may also be used for marking). For the portion that is incorrectly identified (that is, the portion with a low degree of matching), this portion may be automatically marked with × (acting as an example only, and other words, symbols, or pictures, etc. may also be used for marking). In another example, the user may be allowed to mark the correctness and incorrectness, or the user may be allowed to modify the correctness and incorrectness of automatic mark provided by the application. Further, according to the marked correct or incorrect portions, the application may display the objects having all the characteristics of the portions that are correctly identified for the user's reference. The user may select the most similar result to the object to be identified from these objects.
With reference to
An exemplary screen 610 displaying an identification result is shown in
Another exemplary screen 620 displaying the identification result is shown in
The picture corresponding to the received image may not be displayed on the screen, but only the identification result may be displayed. Another exemplary screen 630 displaying the identification results is shown in
It should be understood that the pictures in any of the above regions 61 to 65 may be appended with text description, for example, may be displayed in the form shown in
Various pictures involved in the embodiments of the disclosure, such as the picture presenting the entirety of the identification result, the pictures presenting a plurality of portions of the identification result, the detailed picture presenting a portion of the identification result, the picture presenting the position of a portion of the identification result in the identification result, the picture presenting a portion of the identification result corresponding to a portion of the object to be identified in the received image, etc., may all be obtained from the abovementioned training sample set. Usually in the above training sample set, there are multiple samples (usually a large number of samples) for a species. For each species, a sample may be determined in advance as a representative picture of the species. The representative picture is preferably selected when it is necessary to display the entire picture or pictures of multiple portions of the identification result, or when it is necessary to display the picture of a portion of the identification result corresponding to a portion of the object to be identified in the received image. A representative picture may also be determined in advance for each portion of each species, and when a picture of a specific portion of the identification result is required to be displayed, the representative picture is preferentially selected.
The one or plurality of storage devices 710 may be configured to store any of the data described above, including but not limited to: received images, neural network models, individual sample sets/sample libraries, databases recording the characteristics of various plants, application program files, and the like. The one or plurality of computing devices 730 may be configured to perform one or more of the methods 100, 200, and 300, and/or one or more steps of the one or more of the methods 100, 200, and 300. The one or plurality of electronic apparatuses 720 may be configured to provide a service to a user, which may display pictures and screens 610 to 630 as shown in
The network or bus 740 may be any wired or wireless network and may also include cables. The network or bus 740 may be part of the Internet, the World Wide Web, a specific intranet, a wide area network, or a local area network. The network or bus 740 may utilize standard communication protocols such as Ethernet, WiFi, HTTP, etc., protocols that are proprietary to one or more companies, and various combinations of the foregoing protocols. The network or bus 740 may also include but not limited to an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.
Each of the one or plurality of electronic apparatuses 720 and the one or plurality of computing devices 730 may be configured similarly to a system 800 shown in
The one or plurality of electronic apparatuses 720 may also include one or more cameras for capturing still images or recording video streams, as well as all components for connecting these elements to each other. The one or plurality of electronic apparatuses 720 may each include a full-sized personal computing device, but they may alternatively include mobile computing devices capable of wirelessly exchanging data with a server over a network such as the Internet. For instance, the one or plurality of electronic apparatuses 720 may be a mobile phone, or a device such as a PDA with wireless support, a tablet PC, or a netbook capable of obtaining information via the Internet. In another example, the one or plurality of electronic apparatuses 720 may be a wearable computing system.
The instruction 821 may be any instruction set to be executed directly by the one or plurality of processors 810, such as a machine code, or any instruction set to be executed indirectly, such as a script. The terms “instructions”, “applications”, “processes”, “steps”, and “programs” may be used interchangeably in the specification. The instruction 821 may be stored in an object code format for direct processing by the one or plurality of processors 810 or may be stored as any other computer language, including scripts or collections of independent source code modules that are interpreted on demand or compiled in advance. The instruction 821 may include an instruction that cause, for example, one or plurality of the processors 810 to function as various neural networks in the specification. The functions, methods, and routines of the instruction 821 are explained in detail elsewhere in the specification.
The one or plurality of memories 820 may be any temporary or non-transitory computer readable storage medium capable of storing content accessible by the one or plurality of processors 810, such as a hard drive, a memory card, ROM, RAM, DVD, CD, USB memory, writable memory, read-only memory, and the like. One or more of the one or plurality of memories 820 may include a distributed storage system. The instruction 821 and/or data 822 may be stored on a number of different storage devices that may be physically located in the same or different geographic locations. One or more of the one or plurality of memories 820 may be connected to the one or plurality of processors 810 via a network and/or may be directly connected to or incorporated into any one of the one or plurality of processors 810.
The one or plurality of processors 810 may retrieve, store, or modify data 822 in accordance with the instruction 821. The data 822 stored in the one or plurality of memories 820 may include at least a portion of one or more of the items stored in the one or plurality of storage devices 710 described above. For instance, although the subject matter described in the specification is not limited to any particular data structure, the data 822 may also be stored in a computer register (not shown), in a relational database as a table or XML document with many different fields and records. The data 822 may be formatted in any computing device readable format, such as, but not limited to, binary values, ASCII, or Unicode. In addition, the data 822 may also include any information sufficient to identify relevant information, such as numbers, descriptive text, proprietary codes, pointers, references to data stored in other memory, such as at other network locations, or information used by functions to compute relevant data.
The one or plurality of processors 810 may be any conventional processor, such as a commercially available central processing unit (CPU), a graphics processing unit (GPU), or the like. Alternatively, the one or plurality of processors 810 may also be special-purpose components, such as application specific integrated circuits (ASICs) or other hardware-based processors. Although not required, the one or plurality of processors 810 may include specialized hardware components to perform specific computational processes faster or more efficiently, such as image processing of images and the like.
The one or plurality of processors 810 and the one or plurality of memories 820 are schematically shown in the same box in
The term “A or B” in the specification includes “A and B” and “A or B”, but not exclusively “A” or only “B” unless specifically stated otherwise.
In the disclosure, reference to “one embodiment” or “some embodiments” means that a feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment and at least some embodiments of the disclosure. Therefore, presence of the phrases “in one embodiment” and “in some embodiments” in various places in the disclosure are not necessarily referring to the same embodiment or embodiments. Besides, the characteristics, structures, or features may be combined in any suitable combination and/or sub-combination in one or more embodiments.
As used herein, the word “exemplary” means “serving as an example, instance, or illustration” rather than as a “model” to be exactly reproduced. Any implementation illustratively described herein is not necessarily to be construed as preferred or advantageous over other implementations. Further, the disclosure is not to be bound by any expressed or implied theory presented in the preceding technical field, background, summary, or specific embodiments.
In addition, specific terms may also be used in the following description for reference purposes only, and are thus not intended to be limiting. For instance, the terms “first”, “second”, and other such numerical terms referring to structures or elements do not imply a sequence or order unless the context clearly indicates otherwise. It should also be understood that the term “including/comprising” when used in the specification indicates the presence of the indicated feature, integer, step, operation, unit, and/or component, but does not exclude the presence or addition of one or more other features, integers, steps, operations, units and/or components, and/or combinations thereof.
In the disclosure, the terms “component” and “system” are intended to refer to a computer-related entity, hardware, a combination of hardware and software, software, or software in execution. For instance, a component may be but not limited to a process, an object, an executable state, a thread of execution, and/or a program, etc. running on a processor. By way of examples, both an application running on a server and the server may be one component. The one or more components may reside within an executing process and/or thread, and a component may be localized on one computer and/or distributed between two or more computers.
A person having ordinary skill in the art may know that the boundaries between the operations described above are merely illustrative. Multiple operations may be combined into a single operation, a single operation may be distributed among additional operations, and operations may be performed at least partially overlapping in time. Further, alternative embodiments may include multiple instances of a particular operation, and the order of operations may be changed in other various embodiments. However, other modifications, changes, and substitutions are equally possible. Therefore, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense.
In addition, the embodiments of the disclosure may also include the following examples.
1. A method for displaying an identification result, including:
receiving an image capable of presenting at least a portion of an object to be identified and identifying at least a portion of the object to be identified presented by the image; and
displaying a first picture presenting an identification result in response to obtaining the identification result, where the first picture includes one or a plurality of marks targeting one or a plurality of portions of the identification result.
2. The method according to 1, further including:
displaying a second picture and/or associated text presenting the portion of the identification result targeted by the mark in response to the mark or the portion targeted by the mark being operated.
3. The method according to 1, further including:
displaying information related to an object having a characteristic of the portion of the identification result targeted by the mark in response to the mark or the portion targeted by the mark being operated.
4. The method according to 1, where: the first picture is a picture presenting entirety of the identification result.
5. The method according to 1, where: the first picture is a picture presenting one or a plurality of portions of the identification result, and the presented one or plurality of portions of the identification result are:
a portion of the identification result corresponding to at least one portion of the object to be identified; and/or
portions of the identification result other than the portion corresponding to the at least one portion of the object to be identified.
6. The method according to 2, where: the second picture is a picture presenting details of a corresponding portion of the identification result and/or a picture presenting a position of the corresponding portion of the identification result in the identification result.
7. The method according to 1, where: the mark includes a combination of one or more of an area mark, a lead mark, a text mark, a symbol mark, and a picture mark.
8. The method according to 1, further including:
further displaying a third picture corresponding to the image in response to obtaining the identification result; and
displaying one or a plurality of fourth pictures of one or a plurality of objects in proximity to the object to be identified in response to no identification result being obtained.
9. A method for displaying an identification result, including:
receiving an image capable of presenting at least a portion of an object to be identified and identifying at least a portion of the object to be identified presented by the image; and
displaying a plurality of fifth pictures related to an identification result in response to obtaining the identification result, where each of the fifth pictures corresponds to a portion of the identification result.
10. The method according to 9, where: the fifth pictures are pictures presenting details of corresponding portions of the identification result and/or pictures presenting positions of the corresponding portions of the identification result in the identification result.
11. The method according to 9, further including: displaying a sixth picture presenting a plurality of portions of the identification result.
12. The method according to 9, further including:
displaying a sixth picture presenting entirety of the identification result in response to obtaining the identification result and a degree of matching between the identification result and the object to be identified being greater than a first threshold, where the sixth picture includes at least one mark targeting at least a portion of the identification result presented in the sixth picture; and
displaying the fifth picture corresponding to the portion of the fifth picture of the identification result targeted by the mark in response to the mark or the portion targeted by the mark being operated.
13. The method according to 12, further including:
further displaying information related to an object having a characteristic of a portion or a plurality of portions of the identification result targeted by a mark or a plurality of marks in response to the mark or the plurality of marks in the at least mark or the portion or the plurality of portions targeted by the mark or the plurality of marks being operated.
14. The method according to 12, where: the mark includes a combination of one or more of an area mark, a lead mark, a text mark, a symbol mark, and a picture mark.
15. The method according to 9, where: displaying the fifth picture in response to obtaining the identification result and a degree of matching between the identification result and the object to be identified being less than a second threshold, where a portion of the identification result corresponding to the fifth picture matches a corresponding portion of the object to be identified.
16. The method according to 9, further including:
displaying text associated with the fifth picture in association with the fifth picture.
17. The method according to 9, further including:
highlighting the fifth pictures and/or text associated with the fifth pictures in response to the fifth pictures being operated.
18. The method according to 9, further including:
further displaying a seventh picture corresponding to the image in response to obtaining the identification result.
19. A method for displaying an identification result, including:
receiving an image capable of presenting a first portion of an object to be identified and identifying the first portion; and
displaying an eighth picture presenting a first portion of an identification result in response to obtaining the identification result, where the eighth picture further presents a second portion of the identification result that is different from the first portion.
20. The method according to 19, where: the second portion presented by the eighth picture is manipulatable, and the method further includes:
displaying a ninth picture presenting the second portion and/or text associated with the second portion in response to the second portion being operated.
21. A computer system for displaying an identification result, including:
a processor or a plurality of processors; and
a memory or a plurality of memories, where the memory or the plurality of memories are configured to store a series of computer-executable instructions and computer-accessible data associated with the series of computer-executable instructions,
where when the series of computer-executable instructions are executed by the processor or the plurality of processors, the processor or the plurality of processors are enabled to perform the method according to any one of 1 to 20.
22. A non-transitory computer readable storage medium, where the non-transitory computer readable storage medium stores a series of computer-executable instructions, and when the series of computer-executable instructions are executed by a computer apparatus or a plurality of computer apparatuses, the computer apparatus or the plurality of computer apparatuses are enabled to perform the method according to any one of 1 to 20.
Although some specific embodiments of the disclosure are described in detail by way of examples, a person having ordinary skill in the art should know that the above examples are provided for illustration only and not for the purpose of limiting the scope of the disclosure. The various embodiments disclosed herein may be combined arbitrarily without departing from the spirit and scope of the disclosure. It will also be understood by a person having ordinary skill in the art that various modifications may be made to the embodiments without departing from the scope and spirit of the disclosure. The scope of the disclosure is defined by the appended claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 202011271432.8 | Nov 2020 | CN | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/CN2021/123714 | 10/14/2021 | WO |
