Identification of key points in multimedia data elements

Information

  • Patent Grant
  • 11037015
  • Patent Number
    11,037,015
  • Date Filed
    Thursday, October 27, 2016
    8 years ago
  • Date Issued
    Tuesday, June 15, 2021
    4 years ago
Abstract
A system and method for method for identifying key points in a multimedia data element (MMDE). The method includes: identifying, via a computer vision system, a plurality of candidate key points in the MMDE, wherein a size of each candidate key point is equal to a predetermined size, wherein a scale of each candidate key point is equal to a predetermined scale; analyzing the plurality of candidate key points to determine a set of properties for each candidate key point; comparing the sets of properties of the plurality of candidate key points; and selecting, based on the comparison, a plurality of key points from among the candidate key points.
Description
TECHNICAL FIELD

The present disclosure relates generally to analyzing multimedia content, and particularly to identifying key points in multimedia data elements.


BACKGROUND

With the abundance of multimedia data made available through various means in general and the Internet and world-wide web (WWW) in particular, there is a need for effective ways of analyzing and identifying such multimedia data. Such multimedia data may include, for example, images, graphics, video streams, video clips, video frames, photographs, images of signals, and the like.


Analyzing such multimedia content may be challenging at best due to the huge amount of information that needs to be examined. Helpful and vital data analysis becomes time intensive due to the amount of data that must be processed. As a result, data analysis may be a low priority or ignored entirely.


Even identifying multimedia content elements included in multimedia content is a challenging problem. Existing solutions may include processing, analyzing, and understanding the multimedia content based on one or more decisions. A theme of development for many such existing solutions has been to replicate the abilities of human vision by electronically perceiving and recognizing multimedia content items.


The existing solutions are limited in the ability to identify multimedia content elements that are received for the first time. In particular, many existing solutions require comparing portions of multimedia content to known multimedia content elements to identify any matching multimedia content elements. Thus, unknown multimedia content elements or multimedia content elements that have otherwise never been received before may not be successfully recognized.


Additionally, existing solutions are often highly sensitive to changes in the received multimedia content elements. Consequently, minor changes in the multimedia content due to, for example, differences during capturing, may result in otherwise identical multimedia content elements being unrecognizable. For example, taking pictures of a car at different angles (e.g., one from the rear right side and another from the front left side) may result in the car being unrecognizable by existing solutions for one or more of the pictures.


Other existing solutions rely on metadata to identify multimedia content elements.


Use of such metadata typically relies on information from, e.g., users. Thus, the metadata may not be sufficiently defined to fully describe the multimedia content and, as a result, may not capture all aspects of the multimedia content. For example, a picture of a car may be associated with metadata representing a model of the car, but other pictures of the car may not be associated with metadata designating the owners.


Further, the existing solutions often include analyzing each and every pixel of the multimedia content and matching those pixels to pixels in a database. This analysis consumes a significant amount of computing resources.


It would therefore be advantageous to provide a solution that would overcome the deficiencies of the prior art.


SUMMARY

A summary of several example embodiments of the disclosure follows. This summary is provided for the convenience of the reader to provide a basic understanding of such embodiments and does not wholly define the breadth of the disclosure. This summary is not an extensive overview of all contemplated embodiments, and is intended to neither identify key or critical elements of all embodiments nor to delineate the scope of any or all aspects. Its sole purpose is to present some concepts of one or more embodiments in a simplified form as a prelude to the more detailed description that is presented later. For convenience, the term “some embodiments” may be used herein to refer to a single embodiment or multiple embodiments of the disclosure.


Certain embodiments disclosed herein include a method for identifying key points in a multimedia data element (MMDE). The method comprises: identifying, via a computer vision system, a plurality of candidate key points in the MMDE, wherein a size of each candidate key point is equal to a predetermined size, wherein a scale of each candidate key point is equal to a predetermined scale; analyzing the plurality of candidate key points to determine a set of properties for each candidate key point; comparing the sets of properties of the plurality of candidate key points; and selecting, based on the comparison, a plurality of key points from among the candidate key points.


Certain embodiments disclosed herein also include a non-transitory computer readable medium having stored thereon instructions for causing one or more processing units to execute a method, the method comprising: identifying, via a computer vision system, a plurality of candidate key points in the MMDE, wherein a size of each candidate key point is equal to a predetermined size, wherein a scale of each candidate key point is equal to a predetermined scale; analyzing the plurality of candidate key points to determine a set of properties for each candidate key point; comparing the sets of properties of the plurality of candidate key points; and selecting, based on the comparison, a plurality of key points from among the candidate key points.


Certain embodiments disclosed herein also include a system for identifying key points in a multimedia data element (MMDE). The system comprises: a processing circuitry; and a memory, the memory containing instructions that, when executed by the processing circuitry, configure the system to: identify, via a computer vision system, a plurality of candidate key points in the MMDE, wherein a size of each candidate key point is equal to a predetermined size, wherein a scale of each candidate key point is equal to a predetermined scale; analyze the plurality of candidate key points to determine a set of properties for each candidate key point; compare the sets of properties of the plurality of candidate key points; and select, based on the comparison, a plurality of key points from among the candidate key points.





BRIEF DESCRIPTION OF THE DRAWINGS

The subject matter disclosed herein is particularly pointed out and distinctly claimed in the claims at the conclusion of the specification. The foregoing and other objects, features, and advantages of the disclosed embodiments will be apparent from the following detailed description taken in conjunction with the accompanying drawings.



FIG. 1 is a schematic diagram of a system for identifying key points in multimedia data elements according to an embodiment.



FIG. 2 is a schematic diagram of a properties generation unit according to an embodiment.



FIG. 3 is a flowchart illustrating a method for identifying key points in multimedia data content according to an embodiment.



FIG. 4 is a flowchart illustrating a method for generating properties based on candidate key points according to an embodiment.



FIG. 5 is a flow chart illustrating a method for selecting key points from among candidate key points according to an embodiment.



FIG. 6 is an example simulation of identifying key points in an image multimedia data content.



FIG. 7 is an example data plot utilized to illustrate determining a location property of a candidate key point.



FIG. 8 is an example simulation of determining a rotation property of a candidate key point.



FIG. 9 is an example data plot utilized to illustrate determining a size property of a candidate key point.



FIG. 10 is an example data plot utilized to illustrate determining a pixelation property of a candidate key point.



FIG. 11 is an example data plot utilized to illustrate determining a key point based on an analysis of a set of properties of each of a plurality of candidate key points.





DETAILED DESCRIPTION

It is important to note that the embodiments disclosed herein are only examples of the many advantageous uses of the innovative teachings herein. In general, statements made in the specification of the present application do not necessarily limit any of the various claimed embodiments. Moreover, some statements may apply to some inventive features but not to others. In general, unless otherwise indicated, singular elements may be in plural and vice versa with no loss of generality. In the drawings, like numerals refer to like parts through several views.


The various disclosed embodiments include a method and system for identifying key points in multimedia data elements. The identified key points may be utilized to identify multimedia content elements in the multimedia data elements. A multimedia data element is analyzed to identify candidate key points. The candidate key points are analyzed to determine a set of properties for each candidate key point. Key points are selected based on the determined sets of properties.



FIG. 1 shows an example schematic diagram of a system 100 for identifying key points in multimedia data elements (MMDEs) according to an embodiment. The system 100 includes an interface 110, a processing circuitry 120, a memory 130, a properties generator (PG) 140, and a storage unit (SU) 150. The memory 130 may be volatile (e.g., RAM, etc.), non-volatile (e.g., ROM, flash memory, etc.), or a combination thereof.


Key points are areas within an MMDE of predetermined size and scale that are determined by the system 100 to be the best representations of elements shown in the MMDE. A key point is an area of interest within the MMDE. Key points can be utilized to allow for efficient identification of elements shown in the MMDE by, for example, computer vision systems.


As an example, for a picture of a cat lying on grass, portions of the picture in which the cat or part of the cat is shown may be considered of stronger interest than portions in which only grass is shown. Thus, the area in the picture showing the cat is a key point. As another example, for a picture of a sunset reflected in the ocean, portions of the picture in which both the sun and ocean appear may be considered key points, while portions featuring only the sun or the ocean may not be considered key points.


The key points may be determined based on at least one candidate key point identified in an MMDE. In an embodiment, the identified candidate key points may be selected randomly from among points in the MMDE identified during the analysis. In another embodiment, the candidate key points may be identified based on at least one predetermined key point representation rule. As a non-limiting example, a key point representation rule may include a distance threshold such that only one of any two points having a distance between the two points less than the distance threshold are is selected as a candidate key point. A key point representation rule is described herein below.


MMDEs may be received through the interface 110. The interface 110 may be, but is not limited to, a network interface. As an example, the interface 110 may be a network interface for receiving MMDEs from one or more data sources (not shown) over a network (not shown). The data sources may be, for example, servers (e.g., web servers) or other sources of data including MMDEs. Each MMDE may be, but is not limited to, an image, a graphic, a video stream, a video clip, a video frame, a photograph, and an image of signals (e.g., spectrograms, phasograms, scalograms, etc.), combinations thereof, and portions thereof.


The properties generator 140 is configured to generate a set of properties for each candidate key point. The properties are scalable measures enabling evaluation of each candidate key point as well as determination of key points from among the candidate key points. The properties may include, but are not limited to, a location of a candidate key point within an MMDE, a rotation of a candidate key point within the MMDE, a size of the candidate key point relative to the MMDE, a pixelation of the candidate key point, combinations thereof, and the like. In an embodiment, the properties generator 140 may be further configured to identify benchmarking metrics utilized for determining properties of the candidate key points. For example, for an image, benchmarking metrics may include a white color against which other colors in the image may be compared. The benchmarking metrics utilized and properties determined may be based on a type of the MMDE. For example, metrics for an image may differ from metrics for audio.


The location of the candidate key point may be represented in an XY diagram, wherein the point (0,0) represents one or more edges of the MMDE. The size of a candidate key point is a size of a multimedia content element The rotation of the candidate key point is an angle at which a multimedia content element located at the candidate key point is tilted with respect to a baseline and may be determined respective of, for example, 8 different benchmarking metrics representing different rotations. The baseline may be further determined based on other multimedia content elements of the MMDE. The pixelation may be represented, e.g., in two rectangles (e.g., a 6×3 black rectangle and a 3×6 white rectangle).


The properties generator 140 may be further configured to store the generated properties in the storage unit (SU) 150. The properties generator 140 is described further herein below with respect to FIG. 2.


The processing circuitry 120 is configured to receive one or more MMDEs through the interface 110 and to determine candidate key points for each of the received MMDEs. The processing circuitry 120 is further configured to cause the properties generator 140 to generate the set of properties for each determined candidate key point and to retrieve the generated sets of properties. Based on the retrieved properties, the processing circuitry 120 is configured to determine key points from among the candidate key points.


The processing circuitry 120 is typically coupled to the memory 130. The processing circuitry 120 may comprise or be a component of a processor (not shown) or an array of processors coupled to the memory 130. The memory 130 contains instructions that can be executed by the processing circuitry 120. The instructions, when executed by the processing circuitry 120, cause the processing circuitry 120 to perform the various functions described herein. The one or more processors may be implemented with any combination of general-purpose microprocessors, multi-core processors, microcontrollers, digital signal processors (DSPs), field programmable gate array (FPGAs), programmable logic devices (PLDs), controllers, state machines, gated logic, discrete hardware components, dedicated hardware finite state machines, or any other suitable entities that can perform calculations or other manipulations of information.


The processing circuitry 120 may also include machine-readable media for storing software. Software shall be construed broadly to mean any type of instructions, whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise. Instructions may include code (e.g., in source code format, binary code format, executable code format, or any other suitable format of code). The instructions, when executed by the one or more processors, cause the processing system to perform the various functions described herein.


In another embodiment, the processing circuitry 120 can be realized as an array of computational cores, each core having properties that are at least partly statistically independent from other cores of the plurality of computational cores. Such cores are generated or otherwise configured to obtain maximal independence, i.e., the projection from a signal space should generate a maximal pair-wise distance between any two cores' projections into a high-dimensional space. Further, the cores are optimally designed for the type of signals, i.e., the cores should be maximally sensitive to the spatio-temporal structure of the injected signal, for example, and in particular, sensitive to local correlations in time and space. Thus, in some cases a core represents a dynamic system, such as in state space, phase space, edge of chaos, etc., which is uniquely used herein to exploit their maximal computational power. In addition, the computational cores are optimally designed with regard to invariance to a set of signal distortions, of interest in relevant applications. A detailed description of processes for generating, configuring, and operating any array of computational cores is discussed in more detail U.S. Pat. No. 8,655,801 assigned to the common assignee, which is hereby incorporated by reference for all the useful information they contain.


It should be understood that the embodiments disclosed herein are not limited to the specific architecture illustrated in FIG. 1, and other architectures may be equally used without departing from the scope of the disclosed embodiments. Moreover, in an embodiment, there may be a plurality of systems 100 operating as described hereinabove and configured to either have one as a standby, to share the load between them, or to split the functions between them.



FIG. 2 is an example flow diagram illustrating image processing by the properties generator 140 according to an embodiment. In the example embodiment shown in FIG. 2, the properties generator 140 includes a location determination circuit (LDC) 141, a rotation determination circuit (RDC) 142, a size determination circuit (SDC) 143, and a pixelation determination circuit (PDC) 144. In an optional implementation, the properties generator 140 also includes a comparator 145. Further, in the example embodiment shown in FIG. 2, a MMDE is processed in the following order: by the location determination circuit 141, by the rotation determination circuit 142, by the size determination circuit 143, by the pixelation determination circuit 144, and by the comparator 145.


The location determination circuit 141 is configured to determine a location of a candidate key point in a MMDE. The location may be relative to the MMDE. To this end, when the MMDE is an image, the location may be expressed as a pair of, e.g., X and Y coordinates (X,Y). The origin (0,0) may be any point in the MMDE. As a non-limiting example, the origin may be at the bottom left corner of the MMDE such that points in the MMDE are at coordinates (0,0), (100,150), and any coordinates in between such as, but not limited to, (0,90), (50,0), (75,75), (99,149), (80,120), and so on. When the MMDE is audio, the location may be expressed as a moment or period of time in the audio file. For example, for an audio clip that is 5 minutes (300 seconds) long, the location may be, but is not limited to, 0 seconds (start), 30 seconds, 100 seconds, 267 seconds, 150.2 seconds, 300 seconds (end), and the like.


The rotation determination circuit 142 is configured to determine a rotation of candidate key points in MMDEs. As a non-limiting example, the rotation determination circuit 142 is configured to identify edges of MMDEs, thereby enabling matching of the rotation of the MMDEs based on the respective edges thereof. The size determination circuit 143 is configured to determine a size of candidate key points in MMDEs. The pixelation determination circuit is configured to 144 determines a pixelation of candidate key points in MMDEs. As a non-limiting example, in an image of a couple hugging in front of the Eiffel tower, the pixelation of the portion of the image showing the couple is higher than the pixelation of the portion of the image showing the Eiffel tower in the background. Each of the location determination circuit 141, the rotation determination circuit 142, the size determination circuit is configured to 143, and the pixelation determination circuit 144 may determine its respective properties based on characteristics of elements in the MMDE. To this end, each circuit, 142, 143, or 144 may be configured to identify at least one benchmarking metric based on the MMDE and to compare elements in the MMDE to the benchmarking metric.


Each benchmarking metric may be a metric representing a particular rotation, size, or pixelation of an MMDE, and may be utilized as a point of comparison by, for example, the rotation determination circuit 142, the size determination circuit 143, or the pixelation determination circuit 144, respectively. To this end, each of the rotation, size, and pixelation of a MMDE may be determined relative to at least one corresponding benchmarking metric. As a non-limiting example, if a text element in an image is identified as being in a particular character set (e.g., letters of the English alphabet), the rotation determination circuit 142 may be configured to determine a rotation of the text element with respect to a benchmarking metric text element in the same character set (e.g., a horizontally oriented text element using English alphabet letters).


In an embodiment, the properties generator 140 may include a comparison unit 145. The comparison unit 145 compares a set of properties of each candidate key point. The comparison may be utilized to determine whether a candidate key point should be selected as a key point. To this end, the comparator 145 may be configured to compare scores of properties of the same type (e.g., scores for locations of different candidate key points, scores for rotations of different candidate key points, and the like).


In an embodiment, each, some, or all of the location determination circuit 141, the rotation determination circuit 142, the size determination circuit 143, the pixelation determination circuit (PDC) 144, and the comparator 145 may comprise or be a component of a processor (not shown) or an array of processors Examples for such processor or processors are provided above.


It should be noted that the flow diagram shown in FIG. 2 is merely an example and does not limit any of the disclosed embodiments. In particular, a MMDE may be processed by any of the location determination circuit 141, the rotation determination circuit 142, the size determination circuit 143, and the pixelation determination circuit 144, either in series or in parallel, and may be processed by each circuit in any order. As a non-limiting example, the MMDE may be processed by the location determination circuit 141, the rotation determination circuit 142, the size determination circuit 143, and the pixelation determination circuit 144 simultaneously. As another non-limiting example, the MMDE may be processed by the circuits in the following order: by the rotation determination circuit 142, by the pixelation determination circuit 144, by the size determination circuit 143, and by the location determination circuit 141. Additionally, other circuits for determining properties of MMDEs (not shown), such as a color determination circuit, may be equally used in addition to or instead of any of the circuits 141, 142, 143, or 144.



FIG. 3 is an example flowchart 300 illustrating a method for identifying key points in a MMDE according to an embodiment. In an embodiment, the method may be performed by the system 100.


At S310, a MMDE is received. The MMDE may be received via an interface (e.g., the interface 110).


At S320 the MMDE is analyzed to identify candidate key points. In an embodiment, S320 may include image-based recognition of the MMDE. In a further embodiment, the image-based recognition may begin at the edges of the MMDE and continue to the center. As an example, if the MMDE is an image, the analysis may begin at the outermost points in the image. As another example, if the MMDE is audio, the analysis may begin at the beginning and end times for the audio.


In an embodiment, the identified candidate key points may be selected randomly from among points in the MMDE identified during the analysis. In another embodiment, the candidate key points may be identified based on at least one predetermined key point representation rule. As an example, a key point representation rule may include a distance threshold (e.g., a distance between points in an image or video, a length of time in audio, etc.). If two points in an MMDE are separated by a distance less than the distance threshold, only one of the points may be identified as a candidate key point.


At S330, a set of properties is determined for each identified candidate key point. Determination of properties for candidate key points is described further herein below with respect to FIG. 4.


At S340, the properties for each candidate key point are compared. In an embodiment, comparing the properties further includes determining a property score for each property of each candidate key point. The property scores may be determined based on comparison of characteristics of each property such as, but not limited to, intensity, distance from a center point of the MMDE, color, angle of rotation, a combination thereof, and the like. The property scores may be determined further based on benchmarking metrics for such characteristics. In an embodiment, higher property scores indicating a greater likely significance of the candidate key point. As an example, the location scores for a particular candidate point may be 3, 7, and 8, respectively, with 1 representing the lowest likelihood of significance (e.g., toward the outer edges of the MMDE) and 10 representing the highest likelihood of significance (e.g., closest to the center of the MMDE).


At S350, key points are selected from among the identified candidate key points.


The key points may be selected based on the determined sets of properties via, e.g., comparison of the properties' respective scores. Selecting key points among candidate key points is described further herein below with respect to FIG. 5.


At optional S360, it is checked whether additional key points are required and, if so, execution continues with S320; otherwise, execution terminates. In an embodiment, upon selecting a key point in a particular area of the MMDE, additional key points may be checked for within, or in proximity to, the area of the key point.



FIG. 4 is an example flowchart S340 illustrating a method for determining a set of properties for candidate key points in an MMDE according to an embodiment. In an embodiment, each of the properties may be determined by comparing benchmarking metrics to one or more characteristics of the candidate key point.


At S410, a location of a candidate key point is determined. The location of the candidate key point may be determined by identifying a center point of the MMDE and determining a distance from the center point to the candidate key point. At S420, a rotation of the candidate key point may be determined. The rotation may be determined based on edges identified in the MMDE. At S430, a size of the candidate key point may be determined. At S440, a pixilation of the candidate key point may be determined.


At S450, it may be determined whether properties of additional candidate key points are required and, if so, execution continues with S410; otherwise execution terminates. In an embodiment, the determination may be based on an MMDE identification rule. The MMDE identification rule indicates at least one condition for successful identification of multimedia content elements and may be based on, but not limited to, an event (e.g., identification of a concept related to the MMDE), a threshold (e.g., a number of sets of properties for candidate key points), a combination thereof, and the like. To this end, in an embodiment in which identification of a concept related to the MMDE is indicated by the MMDE identification rule, S450 may further include determining whether a concept can be identified based on the properties determined thus far.


It should be noted that FIG. 4 is described herein above with respect to location, rotation, size, and pixelation properties merely for simplicity purposes and without limitation on the disclosed embodiments. More, fewer, or other properties may be utilized without departing from the scope of the disclosure. As an example, a color property for an image may also be determined. As another example, a volume property for an audio file may be determined, and the rotation and pixelation properties may not be determined for the audio file.



FIG. 5 is an example flowchart S350 illustrating a method for selecting key points in a MMDE according to an embodiment.


At S510, sets of properties of candidate key points in the MMDE are obtained. The sets of properties for each candidate key point may include the properties determined as described herein above with respect to FIG. 4.


At S520, the sets of properties are compared to identify relatively high sets of properties. The relatively high sets of properties are identified to determine the most descriptive candidate key points. In an embodiment, S520 includes determining a property score for each property. Each property score may be determined based on relative values for properties of the candidate key points. In a further embodiment, S520 may also include determining an average property score for properties of each set of properties. In yet a further embodiment, relatively high sets of properties may be sets of properties having average property scores above a predetermined threshold.


At optional S530, at least one budget parameter may be retrieved. The budget parameter is a quantitative limitation on the maximum amount of key points that may be selected for the MMDE and is typically utilized to ensure efficient key point identification by restricting the number of key points that need to be identified, thereby conserving computing resources. The budget may be the same for all MMDEs, may differ for different types of MMDEs, and the like. In an embodiment, the budget may be retrieved from the storage unit 150.


At S540, key points to be selected are determined based on the comparison. The number of key points determined may be limited based on the budget.



FIG. 6 is an example simulation of a selection of candidate key points in a MMDE. In the example simulation of FIG. 6, an image 600 includes a cat. The image 600 may be received via an interface and analyzed. Based on the analysis, candidate key points 610-1 through 610-4 are identified. A set of properties is generated for each of the candidate key points 610. The sets of properties are compared. For example, the location property of candidate key point 610-1 is relatively low as compared to candidate point 610-3 because it is closer to the center of the image 600. Additionally, the pixelation property of candidate key point 610-2 may be relatively low as compared to candidate key point 610-4.



FIG. 7 is an example data plot 700 utilized to illustrate determining a location property of a candidate key point. The example data plot 700 includes a set of data points 710 and a subset of data points 720 among the set of data points 710. Location properties are determined for candidate key points. The location properties may be displayed in the data plot 700 as an XY graph where the data point (0,0) represents a point on an edge of a MMDE and where the points farthest from the data point (0,0) represent the points closest to the center of the MMDE. In the example data plot 700, each point (x,y) represents, e.g., a horizontal and vertical distance, respectively, of the candidate key point to the center of the MMDE when the MMDE is an image. In other example data plots, each point (x,y) may represent, e.g., a horizontal and vertical distance, respectively, of the candidate key point to one or more of the edges of the image MMDE. For other types of multimedia content elements (e.g., video, audio, etc.), each of the X-axis and the Y-axis may represent metrics such as, but not limited to, amount of time from half of the total time of the audio or video, a horizontal or vertical distance to or from a center of an image-based portion of the video, a distance of a line to or from the center of the image-based portion of the video, and the like.


The candidate key points with the strongest responses (i.e., location properties) may be selected. The strongest response key points may be determined by comparing the location properties among the candidate key points 710 and assigning a location score to each of the candidate key points 710. As an example, the strongest response candidate key points are associated with points of the subset 720.



FIG. 8 is an example simulation 800 of determining a rotation property of a candidate key point. The simulation 800 is based on an image 810 of a superhero character standing in front of a city background and includes candidate key points 820 and 830. A rotation property may be determined for each of the candidate key points 820 and 830. The determination may include comparing rotations among the candidate key points and determining a rotation scale based on the comparison. The determination may further include analyzing the rotation of each candidate key point respective of the rotation scale and assigning a rotation score to each candidate key point. As an example, the candidate key point 820 may be assigned a lower rotation score than that of candidate key point 830 because the rotation of the candidate key point 820 (i.e., a point on the side of a vertically oriented building) is less than the rotation of the candidate key point 830 (i.e., a point on an angle of a bent elbow).



FIG. 9 is an example data plot 900 utilized to illustrate determining size properties of candidate key points. The example data plot 900 includes data points 910 and a data point cluster 920. Each of the data points 910 represents a size property of a candidate key point, with the data point (0,0) representing the lowest size property. The data points of the key-data point cluster 920 belong to the cluster that is the most distant point cluster from the data point (0,0) and its data point have the highest size properties among candidate key points. Thus, the largest candidate key points are associated with a higher size score and are the candidate key points associated with the data point cluster 920.



FIG. 10 is an example data plot 1000 utilized to illustrate determining pixilation properties of candidate key points. The pixilation properties represent visual quality of the MMCE as determined based on, e g., resolution. The example data plot 1000 includes data points 1010 and a data point cluster 1020 where the data point (0,0) represents the lowest pixilation among candidate key points. The data points of data point cluster 1020 belong to the cluster that is the most distant point cluster from the data point (0,0) and its data point have the highest pixilation among candidate key points.



FIG. 11 is an example data plot 1100 utilized to illustrate selecting key points based on an analysis of a set of properties of each of a plurality of candidate key points. The example data plot 1100 includes points 1110 and 1120. Each point 1110 and 1120 represents a candidate key point having a set of properties. In an example implementation, the set of properties may include 2 or 4 properties. The set of properties for each candidate key point is analyzed to determine which candidate key points are to be utilized as key points. As an example, the candidate key point may be assigned da score for each property, and one or more candidate key points having a highest average score from among a plurality of candidate key points may be selected as key points.


The various embodiments disclosed herein can be implemented as hardware, firmware, software, or any combination thereof. Moreover, the software is preferably implemented as an application program tangibly embodied on a program storage unit or computer readable medium consisting of parts, or of certain devices and/or a combination of devices. The application program may be uploaded to, and executed by, a machine comprising any suitable architecture. Preferably, the machine is implemented on a computer platform having hardware such as one or more central processing units (“CPUs”), a memory, and input/output interfaces. The computer platform may also include an operating system and microinstruction code. The various processes and functions described herein may be either part of the microinstruction code or part of the application program, or any combination thereof, which may be executed by a CPU, whether or not such a computer or processor is explicitly shown. In addition, various other peripheral units may be connected to the computer platform such as an additional data storage unit and a printing unit. Furthermore, a non-transitory computer readable medium is any computer readable medium except for a transitory propagating signal.


All examples and conditional language recited herein are intended for pedagogical purposes to aid the reader in understanding the principles of the disclosed embodiment and the concepts contributed by the inventor to furthering the art, and are to be construed as being without limitation to such specifically recited examples and conditions. Moreover, all statements herein reciting principles, aspects, and embodiments of the disclosed embodiments, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Additionally, it is intended that such equivalents include both currently known equivalents as well as equivalents developed in the future, i.e., any elements developed that perform the same function, regardless of structure.

Claims
  • 1. A method for identifying key points in an image, comprising: receiving the image;identifying, via a computer vision system and after the receiving of the image, a plurality of candidate key points in the image, wherein all candidate key points of the plurality of candidate key points have only a single scale that equals a predetermined scale;analyzing the plurality of candidate key points to determine a set of properties for each candidate key point; wherein the set of properties comprises a distance from a global center of the image;comparing between the sets of properties of the plurality of candidate key points;
  • 2. The method of claim 1, further comprising: identifying, in the image, a plurality of points having a predetermined size and the predetermined scale, wherein the candidate key points are selected from among the identified plurality of points.
  • 3. The method of claim 2, wherein the candidate key points are selected randomly from among the identified plurality of points.
  • 4. The method of claim 2, wherein the candidate key points are identified based on at least one key point representation rule.
  • 5. The method of claim 1, wherein each set of properties further comprises a pixelation of the candidate key point.
  • 6. The method of claim 1, further comprising: receiving a budget parameter indicating a maximum number of key points to be identified, wherein a number of the determined key points is less than the maximum number of key points.
  • 7. The method according to claim 1 wherein each set of properties further comprises a rotation of the candidate key point.
  • 8. The method according to claim 1 wherein each one of the candidate key points comprises multiple pixels.
  • 9. A non-transitory computer readable medium having stored thereon instructions for causing one or more processing units to execute a method, the method comprising: receiving the image;identifying, via a computer vision system and after the receiving of the image, a plurality of candidate key points in an image, wherein all candidate key points of the plurality of candidate key points have only a single scale that equals a;analyzing the plurality of candidate key points to determine a set of properties for each candidate key point;comparing between the sets of properties of the plurality of candidate key points;
  • 10. A system for identifying key points in an image, comprising: a processing circuitry; and a memory, the memory containing instructions that, when executed by the processing circuitry, configure the system to: receive the image;identify, via a computer vision system and after a reception of the image, a plurality of candidate key points in the image, wherein all candidate key points of the plurality of candidate key points have only a single scale that equals a;analyze the plurality of candidate key points to determine a set of properties for each candidate key point;compare between the sets of properties of the plurality of candidate key points;wherein the set of properties comprises a distance from a global center of the image; andselect, based on the comparison, a plurality of key points from among the candidate key points;wherein the system is further configured to:determine a score for each property of each set of properties for each candidate key point, wherein each score is determined by comparing at least one characteristic of the candidate key point to at least one characteristic of each other candidate key point; and
  • 11. The system of claim 10, wherein the system is further configured to: identify, in the image, a plurality of points having a predetermined size and the predetermined scale, wherein the candidate key points are selected from among the identified plurality of points.
  • 12. The system of claim 11, wherein the candidate key points are selected randomly from among the identified plurality of points.
  • 13. The system of claim 11, wherein the candidate key points are identified based on at least one key point representation rule.
  • 14. The system of claim 10, wherein each set of properties further comprise a pixelation of the candidate key point.
  • 15. The system of claim 10, wherein the system is further configured to: receive a budget parameter indicating a maximum number of key points to be identified, wherein a number of the determined key points is less than the maximum number of key points.
CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Application No. 62/267,398 filed on Dec. 15, 2015, the contents of which are hereby incorporated by reference.

US Referenced Citations (344)
Number Name Date Kind
4733353 Jaswa Mar 1988 A
4932645 Schorey et al. Jun 1990 A
4972363 Nguyen et al. Nov 1990 A
5214746 Fogel et al. May 1993 A
5307451 Clark Apr 1994 A
5412564 Ecer May 1995 A
5436653 Ellis et al. Jul 1995 A
5568181 Greenwood et al. Oct 1996 A
5638425 Meador et al. Jun 1997 A
5745678 Herzberg et al. Apr 1998 A
5763069 Jordan Jun 1998 A
5806061 Chaudhuri et al. Sep 1998 A
5852435 Vigneaux et al. Dec 1998 A
5870754 Dimitrova et al. Feb 1999 A
5873080 Coden et al. Feb 1999 A
5887193 Takahashi et al. Mar 1999 A
5978754 Kumano Nov 1999 A
5991306 Burns et al. Nov 1999 A
6052481 Grajski et al. Apr 2000 A
6070167 Qian et al. May 2000 A
6075881 Foster et al. Jun 2000 A
6076088 Paik et al. Jun 2000 A
6122628 Castelli et al. Sep 2000 A
6128651 Cezar Oct 2000 A
6137911 Zhilyaev Oct 2000 A
6144767 Bottou et al. Nov 2000 A
6147636 Gershenson Nov 2000 A
6163510 Lee et al. Dec 2000 A
6243375 Speicher Jun 2001 B1
6243713 Nelson et al. Jun 2001 B1
6275599 Adler et al. Aug 2001 B1
6329986 Cheng Dec 2001 B1
6381656 Shankman Apr 2002 B1
6411229 Kobayashi Jun 2002 B2
6422617 Fukumoto et al. Jul 2002 B1
6507672 Watkins et al. Jan 2003 B1
6523046 Liu et al. Feb 2003 B2
6524861 Anderson Feb 2003 B1
6550018 Abonamah et al. Apr 2003 B1
6557042 He et al. Apr 2003 B1
6594699 Sahai et al. Jul 2003 B1
6601026 Appelt et al. Jul 2003 B2
6611628 Sekiguchi et al. Aug 2003 B1
6618711 Ananth Sep 2003 B1
6640015 Lafruit Oct 2003 B1
6643620 Contolini et al. Nov 2003 B1
6643643 Lee et al. Nov 2003 B1
6665657 Dibachi Dec 2003 B1
6681032 Bortolussi et al. Jan 2004 B2
6704725 Lee Mar 2004 B1
6711293 Lowe Mar 2004 B1
6732149 Kephart May 2004 B1
6742094 Igari May 2004 B2
6751363 Natsev et al. Jun 2004 B1
6751613 Lee et al. Jun 2004 B1
6754435 Kim Jun 2004 B2
6763069 Divakaran et al. Jul 2004 B1
6763519 McColl et al. Jul 2004 B1
6774917 Foote et al. Aug 2004 B1
6795818 Lee Sep 2004 B1
6804356 Krishnamachari Oct 2004 B1
6813395 Kinjo Nov 2004 B1
6819797 Smith et al. Nov 2004 B1
6901207 Watkins May 2005 B1
6985172 Rigney et al. Jan 2006 B1
7013051 Sekiguchi et al. Mar 2006 B2
7020654 Najmi Mar 2006 B1
7023979 Wu et al. Apr 2006 B1
7139372 Chakravorty et al. Nov 2006 B2
7158681 Persiantsev Jan 2007 B2
7215828 Luo May 2007 B2
7260564 Lynn et al. Aug 2007 B1
7299261 Oliver et al. Nov 2007 B1
7302117 Sekiguchi et al. Nov 2007 B2
7313805 Rosin et al. Dec 2007 B1
7340358 Yoneyama Mar 2008 B2
7353224 Chen et al. Apr 2008 B2
7376672 Weare May 2008 B2
7409208 Clare et al. Aug 2008 B1
7433895 Li et al. Oct 2008 B2
7464086 Black et al. Dec 2008 B2
7529659 Wold May 2009 B2
7657100 Gokturk et al. Feb 2010 B2
7660468 Gokturk et al. Feb 2010 B2
7801893 Gulli Sep 2010 B2
7860895 Scofield Dec 2010 B1
7933407 Keidar et al. Apr 2011 B2
8098934 Vincent Jan 2012 B2
8108881 Hamel Jan 2012 B2
8266185 Raichelgauz et al. Sep 2012 B2
8275764 Jeon Sep 2012 B2
8312031 Raichelgauz et al. Nov 2012 B2
8315442 Gokturk et al. Nov 2012 B2
8345982 Gokturk et al. Jan 2013 B2
RE44225 Aviv May 2013 E
8457827 Ferguson et al. Jun 2013 B1
8468164 Paleja et al. Jun 2013 B1
8495489 Everingham Jul 2013 B1
8510734 Criddle et al. Aug 2013 B2
8527978 Sallam Sep 2013 B1
8634980 Urmson Jan 2014 B1
8655801 Raichelgauz et al. Feb 2014 B2
8655878 Kulkarni et al. Feb 2014 B1
8781152 Momeyer Jul 2014 B2
8782077 Rowley Jul 2014 B1
8799195 Raichelgauz et al. Aug 2014 B2
8799196 Raichelquaz et al. Aug 2014 B2
8818916 Raichelgauz et al. Aug 2014 B2
8868861 Shimizu et al. Oct 2014 B2
8886648 Procopio et al. Nov 2014 B1
9165406 Gray et al. Oct 2015 B1
9298763 Zack Mar 2016 B1
9323754 Ramanathan et al. Apr 2016 B2
9440647 Sucan Sep 2016 B1
9466068 Raichelgauz et al. Oct 2016 B2
9646006 Raichelgauz et al. May 2017 B2
9679062 Schillings et al. Jun 2017 B2
9734533 Givot Aug 2017 B1
9807442 Bhatia et al. Oct 2017 B2
9984369 Li et al. May 2018 B2
10133947 Yang Nov 2018 B2
10347122 Takenaka Jul 2019 B2
10491885 Hicks Nov 2019 B1
20010019633 Tenze Sep 2001 A1
20010038876 Anderson Nov 2001 A1
20020010682 Johnson Jan 2002 A1
20020010715 Chinn et al. Jan 2002 A1
20020019882 Bokhani Feb 2002 A1
20020032677 Morgenthaler et al. Mar 2002 A1
20020037010 Yamauchi Mar 2002 A1
20020038299 Zernik et al. Mar 2002 A1
20020072935 Rowse et al. Jun 2002 A1
20020087530 Smith et al. Jul 2002 A1
20020107827 Benitez-Jimenez et al. Aug 2002 A1
20020113812 Walker et al. Aug 2002 A1
20020129140 Peled et al. Sep 2002 A1
20020147637 Kraft et al. Oct 2002 A1
20020157116 Jasinschi Oct 2002 A1
20020163532 Thomas Nov 2002 A1
20020174095 Lulich et al. Nov 2002 A1
20030037010 Schmelzer Feb 2003 A1
20030089216 Birmingham et al. May 2003 A1
20030093790 Logan et al. May 2003 A1
20030101150 Agnihotri May 2003 A1
20030105739 Essafi et al. Jun 2003 A1
20030200217 Ackerman Oct 2003 A1
20030217335 Chung et al. Nov 2003 A1
20030229531 Heckerman et al. Dec 2003 A1
20040059736 Willse Mar 2004 A1
20040091111 Levy May 2004 A1
20040111432 Adams et al. Jun 2004 A1
20040117638 Monroe Jun 2004 A1
20040153426 Nugent Aug 2004 A1
20040230572 Omoigui Nov 2004 A1
20040267774 Lin et al. Dec 2004 A1
20050021394 Miedema et al. Jan 2005 A1
20050114198 Koningstein et al. May 2005 A1
20050131884 Gross et al. Jun 2005 A1
20050163375 Grady Jul 2005 A1
20050172130 Roberts Aug 2005 A1
20050177372 Wang et al. Aug 2005 A1
20050193015 Logston Sep 2005 A1
20050238198 Brown Oct 2005 A1
20050249398 Khamene et al. Nov 2005 A1
20050256820 Dugan et al. Nov 2005 A1
20050262428 Little et al. Nov 2005 A1
20050289163 Gordon et al. Dec 2005 A1
20050289590 Cheok et al. Dec 2005 A1
20060033163 Chen Feb 2006 A1
20060100987 Leurs May 2006 A1
20060120626 Perlmutter Jun 2006 A1
20060217828 Hicken Sep 2006 A1
20060236343 Chang Oct 2006 A1
20060242130 Sadri Oct 2006 A1
20060248558 Barton Nov 2006 A1
20060251339 Gokturk Nov 2006 A1
20060253423 McLane et al. Nov 2006 A1
20060288002 Epstein et al. Dec 2006 A1
20070022374 Huang et al. Jan 2007 A1
20070038614 Guha Feb 2007 A1
20070061302 Ramer et al. Mar 2007 A1
20070074147 Wold Mar 2007 A1
20070083611 Farago et al. Apr 2007 A1
20070130159 Gulli et al. Jun 2007 A1
20070156720 Maren Jul 2007 A1
20070196013 Li Aug 2007 A1
20070244902 Seide et al. Oct 2007 A1
20070253594 Lu et al. Nov 2007 A1
20070298152 Baets Dec 2007 A1
20080049789 Vedantham et al. Feb 2008 A1
20080072256 Boicey et al. Mar 2008 A1
20080109433 Rose May 2008 A1
20080152231 Gokturk Jun 2008 A1
20080159622 Agnihotri et al. Jul 2008 A1
20080165861 Wen Jul 2008 A1
20080166020 Kosaka Jul 2008 A1
20080201299 Lehikoinen et al. Aug 2008 A1
20080201314 Smith et al. Aug 2008 A1
20080201361 Castro et al. Aug 2008 A1
20080228995 Tan et al. Sep 2008 A1
20080237359 Silverbrook et al. Oct 2008 A1
20080253737 Kimura Oct 2008 A1
20080263579 Mears et al. Oct 2008 A1
20080270373 Oostveen et al. Oct 2008 A1
20080270569 McBride Oct 2008 A1
20080294278 Borgeson Nov 2008 A1
20080307454 Ahanger et al. Dec 2008 A1
20080313140 Pereira et al. Dec 2008 A1
20090022472 Bronstein Jan 2009 A1
20090024641 Quigley et al. Jan 2009 A1
20090034791 Doretto Feb 2009 A1
20090043637 Eder Feb 2009 A1
20090043818 Raichelgauz Feb 2009 A1
20090080759 Bhaskar Mar 2009 A1
20090125544 Brindley May 2009 A1
20090172030 Schiff et al. Jul 2009 A1
20090216761 Raichelgauz Aug 2009 A1
20090220138 Zhang et al. Sep 2009 A1
20090245573 Saptharishi et al. Oct 2009 A1
20090254572 Redlich et al. Oct 2009 A1
20090278934 Ecker Nov 2009 A1
20090282218 Raichelgauz et al. Nov 2009 A1
20100042646 Raichelqauz Feb 2010 A1
20100082684 Churchill Apr 2010 A1
20100104184 Bronstein Apr 2010 A1
20100111408 Matsuhira May 2010 A1
20100125569 Nair May 2010 A1
20100162405 Cook Jun 2010 A1
20100198626 Cho et al. Aug 2010 A1
20100284604 Chrysanthakopoulos Nov 2010 A1
20100306193 Pereira Dec 2010 A1
20100318493 Wessling Dec 2010 A1
20100318515 Ramanathan et al. Dec 2010 A1
20100325138 Lee et al. Dec 2010 A1
20100325581 Finkelstein et al. Dec 2010 A1
20110029620 Bonforte Feb 2011 A1
20110038545 Bober Feb 2011 A1
20110055585 Lee Mar 2011 A1
20110164180 Lee Jul 2011 A1
20110164810 Zang et al. Jul 2011 A1
20110246566 Kashef Oct 2011 A1
20110276680 Rimon Nov 2011 A1
20110296315 Lin et al. Dec 2011 A1
20120070078 Sharma Mar 2012 A1
20120095850 Hamel Apr 2012 A1
20120133497 Sasaki May 2012 A1
20120167133 Carroll Jun 2012 A1
20120179642 Sweeney et al. Jul 2012 A1
20120179751 Ahn Jul 2012 A1
20120185445 Borden et al. Jul 2012 A1
20120197857 Huang Aug 2012 A1
20120221470 Lyon Aug 2012 A1
20120227074 Hill et al. Sep 2012 A1
20120239690 Asikainen et al. Sep 2012 A1
20120239694 Avner et al. Sep 2012 A1
20120299961 Ramkumar et al. Nov 2012 A1
20120301105 Rehg et al. Nov 2012 A1
20120331011 Raichelgauz et al. Dec 2012 A1
20130066856 Ong et al. Mar 2013 A1
20130067364 Berntson et al. Mar 2013 A1
20130089248 Remiszewski Apr 2013 A1
20130103814 Carrasco Apr 2013 A1
20130139151 Criddle et al. May 2013 A1
20130159298 Mason et al. Jun 2013 A1
20130191823 Davidson et al. Jul 2013 A1
20130212493 Krishnamurthy Aug 2013 A1
20130226820 Sedota, Jr. Aug 2013 A1
20130226930 Amgren et al. Aug 2013 A1
20130283401 Pabla et al. Oct 2013 A1
20140025692 Pappas Jan 2014 A1
20140059443 Tabe Feb 2014 A1
20140095425 Sipple Apr 2014 A1
20140111647 Atsmon Apr 2014 A1
20140125703 Roveta May 2014 A1
20140147829 Jerauld May 2014 A1
20140152698 Kim et al. Jun 2014 A1
20140169681 Drake Jun 2014 A1
20140176604 Venkitaraman et al. Jun 2014 A1
20140193077 Shiiyama et al. Jul 2014 A1
20140201330 Lozano Lopez Jul 2014 A1
20140250032 Huang et al. Sep 2014 A1
20140282655 Roberts Sep 2014 A1
20140300722 Garcia Oct 2014 A1
20140330830 Raichelgauz et al. Nov 2014 A1
20140341476 Kulick et al. Nov 2014 A1
20140379477 Sheinfeld Dec 2014 A1
20150009129 Song Jan 2015 A1
20150033150 Lee Jan 2015 A1
20150100562 Kohlmeier et al. Apr 2015 A1
20150117784 Lin Apr 2015 A1
20150120627 Hunzinger et al. Apr 2015 A1
20150134688 Jing May 2015 A1
20150254344 Kulkarni et al. Sep 2015 A1
20150286742 Zhang et al. Oct 2015 A1
20150324356 Gutierrez et al. Nov 2015 A1
20150363644 Wnuk Dec 2015 A1
20160007083 Gurha Jan 2016 A1
20160026707 Ong et al. Jan 2016 A1
20160210525 Yang Jul 2016 A1
20160221592 Puttagunta Aug 2016 A1
20160342683 Kwon Nov 2016 A1
20160342863 Kwon Nov 2016 A1
20160357188 Ansari Dec 2016 A1
20170017638 Satyavarta et al. Jan 2017 A1
20170032257 Sharifi Feb 2017 A1
20170041254 Agara Venkatesha Rao Feb 2017 A1
20170109602 Kim Apr 2017 A1
20170150047 Jung May 2017 A1
20170154241 Shambik et al. Jun 2017 A1
20170255620 Raichelgauz Sep 2017 A1
20170262437 Raichelgauz Sep 2017 A1
20170323568 Inoue Nov 2017 A1
20180081368 Watanabe Mar 2018 A1
20180101177 Cohen Apr 2018 A1
20180157916 Doumbouya Jun 2018 A1
20180158323 Takenaka Jun 2018 A1
20180204111 Zadeh Jul 2018 A1
20190005726 Nakano Jan 2019 A1
20190039627 Yamamoto Feb 2019 A1
20190043274 Hayakawa Feb 2019 A1
20190045244 Balakrishnan Feb 2019 A1
20190056718 Satou Feb 2019 A1
20190065951 Luo Feb 2019 A1
20190188501 Ryu Jun 2019 A1
20190220011 Della Penna Jul 2019 A1
20190317513 Zhang Oct 2019 A1
20190364492 Azizi Nov 2019 A1
20190384303 Muller Dec 2019 A1
20190384312 Herbach Dec 2019 A1
20190385460 Magzimof Dec 2019 A1
20190389459 Berntorp Dec 2019 A1
20200004248 Healey Jan 2020 A1
20200004251 Zhu Jan 2020 A1
20200004265 Zhu Jan 2020 A1
20200005631 Visintainer Jan 2020 A1
20200018606 Wolcott Jan 2020 A1
20200018618 Ozog Jan 2020 A1
20200020212 Song Jan 2020 A1
20200050973 Stenneth Feb 2020 A1
20200073977 Montemerlo Mar 2020 A1
20200090484 Chen Mar 2020 A1
20200097756 Hashimoto Mar 2020 A1
20200133307 Kelkar Apr 2020 A1
20200043326 Tao Jun 2020 A1
Foreign Referenced Citations (9)
Number Date Country
1085464 Jan 2007 EP
0231764 Apr 2002 WO
2003067467 Aug 2003 WO
2005027457 Mar 2005 WO
2007049282 May 2007 WO
2014076002 May 2014 WO
2014137337 Sep 2014 WO
2016040376 Mar 2016 WO
2016070193 May 2016 WO
Non-Patent Literature Citations (46)
Entry
The International Search Report and the Written Opinion for PCT/US2016/059111, ISA/RU, Moscow, Russia, dated Jan. 19, 2017.
Hogue, “Tree Pattern Inference and Matching for Wrapper Induction on the World Wide Web”, Master's Thesis, Massachusetts institute of Technology, 2004, pp. 1-106.
Lau et al., “Semantic Web Service Adaptation Model for a Pervasive Learning Scenario”, 2008 IEEE Conference on Innovative Technologies in Intelligent Systems and Industrial Applications, 2008, pp. 98-103.
McNamara et al., “Diversity Decay in opportunistic Content Sharing Systems”, 2011 IEEE International Symposium on a World of Wireless, Mobile and Multimedia Networks, pp. 1-3.
Odinaev et al, “Cliques in Neural Ensembles as Perception Carriers”, Technion—Institute of Technology, 2006 International Joint Conference on neural Networks, Canada, 2006, pp. 285-292.
Queluz, “Content-Based Integrity Protection of Digital Images”, SPIE Conf. on Security and Watermarking of Multimedia Contents, San Jose, Jan. 1999, pp. 85-93.
Santos et al., “SCORM-MPEG: an Ontology of Interoperable Metadata for multimediaand E-Learning”, 23rd International Conference on Software, Telecommunications and Computer Networks (SoftCom), 2015, pp. 224-228.
“Computer Vision Demonstration Website”, Electronics and Computer Science, University of Southampton, 2005, USA.
Boari et al, “Adaptive Routing for Dynamic Applications in Massively Parallel Architectures”, 1995 IEEE, Spring 1995, pp. 1-14.
Burgsteiner et al., “Movement Prediction from Real-World Images Using a Liquid State machine”, Innovations in Applied Artificial Intelligence Lecture Notes in Computer Science, Lecture Notes in Artificial Intelligence, LNCS, Springer-Verlag, BE, vol. 3533, Jun. 2005, pp. 121-130.
Cernansky et al, “Feed-forward Echo State Networks”, Proceedings of International Joint Conference on Neural Networks, Montreal, Canada, Jul. 31-Aug. 4, 2005, pp. 1-4.
Fathy et al, “A Parallel Design and Implementation for Backpropagation Neural Network Using MIMD Architecture”, 8th Mediterranean Electrotechnical Conference, 19'96. MELECON '96, Date of Conference: May 13-16, 1996, vol. 3 pp. 1472-1475, vol. 3.
Freisleben et al, “Recognition of Fractal Images Using a Neural Network”, Lecture Notes in Computer Science, 1993, vol. 6861, 1993, pp. 631-637.
Garcia, “Solving the Weighted Region Least Cost Path Problem Using Transputers”, Naval Postgraduate School, Monterey, California, Dec. 1989.
Hogue, “Tree Pattern Inference and Matching for Wrapper Induction on the World Wide Web”, Master's Thesis, Massachusetts Institute of Technology, Jun. 2004, pp. 1-106.
Howlett et al, “A Multi-Computer Neural Network Architecture in a Virtual Sensor System Application”, International journal of knowledge-based intelligent engineering systems, 4 (2). pp. 86-93, 133N 1327-2314.
Hua et al., “Robust Video Signature Based on Ordinal Measure”, Image Processing, 2004, 2004 International Conference on Image Processing (ICIP), vol. 1, IEEE, pp. 685-688, 2004.
International Search Report and Written Opinion for PCT/US2016/050471, ISA/RU, Moscow, RU, dated May 4, 2017.
International Search Report and Written Opinion for PCT/US2016/054634, ISA/RU, Moscow, RU, dated Mar. 16, 2017.
International Search Report and Written Opinion for PCT/US2017/015831, ISA/RU, Moscow, RU, dated Apr. 20, 2017.
Johnson et al, “Pulse-Coupled Neural Nets: Translation, Rotation, Scale, Distortion, and Intensity Signal Invariance for Images”, Applied Optics, vol. 33, No. 26, 1994, pp. 6239-6253.
Ortiz-Boyer et al, “CIXL2: A Crossover Operator for Evolutionary Algorithms Based on Population Features”, Journal of Artificial Intelligence Research 24 (2005) Submitted Nov. 2004; published Jul. 2005, pp. 1-48.
Santos et al., “SCORM-MPEG: an Ontology of Interoperable Metadata for multimediaand E-Leaming”, 23rd International Conference on Software, Telecommunications and Computer Networks (SoftCom), 2015, pp. 224-228.
Scheper et al, “Nonlinear dynamics in neural computation”, ESANN'2006 proceedings—European Symposium on Artificial Neural Networks, Bruges (Belgium), Apr. 26-28, 2006, d-side publication, ISBN 2-930307-06-4, pp. 1-12.
Schneider et al, “A Robust Content based Digital Signature for Image Authentication”, Proc. ICIP 1996, Lausane, Switzerland, Oct. 1996, pp. 227-230.
Stolberg et al, “HIBRID-SOC: A Mul Ti-Core Soc Architecture for Mul Timedia Signal Processing”, 2003 IEEE, pp. 189-194.
Theodoropoulos et al, “Simulating Asynchronous Architectures on Transputer Networks”, Proceedings of the Fourth Euromicro Workshop on Parallel and Distributed Processing, 1996. PDP '96, pp. 274-281.
Verstraeten et al, “Isolated word recognition with the Liquid State Machine: a case study”, Department of Electronics and Information Systems, Ghent University, Sint-Pietersnieuwstraat 41, 9000 Gent, Belgium, Available onlline Jul. 14, 2005, pp. 521-528.
Wang et al., “Classifying Objectionable Websites Based onImage Content”, Stanford University, pp. 1-12.
Ware et al, “Locating and Identifying Components in a Robot's Workspace using a Hybrid Computer Architecture” Proceedings of the 1995 IEEE International Symposium on Intelligent Control, Aug. 27-29, 1995, pp. 139-144.
Whitby-Strevens, “The transputer”, 1985 IEEE, pp. 292-300.
Wilk et al., “The Potential of Social-Aware Multimedia Prefetching on Mobile Devices”, International Conference and Workshops on networked Systems (NetSys), 2015, pp. 1-5.
Yanagawa et al, “Columbia University's Baseline Detectors for 374 LSCOM Semantic Visual Concepts”, Columbia University Advent Technical Report # 222-2006-8, Mar. 20, 2007, pp. 1-17.
Yanagawa et al., “Columbia University's Baseline Detectors for 374 LSCOM Semantic Visual Concepts”, Columbia University Advent Technical Report #222, 2007, pp. 2006-2008.
Zhou et al, “Ensembling neural networks: Many could be better than all”, National Laboratory for Novel Software Technology, Nanjing University, Hankou Road 22, Nanjing 210093, PR China, Available online Mar. 12, 2002, pp. 239-263.
Zhou et al, “Medical Diagnosis With C4.5 Rule Preceded by Artificial Neural Network Ensemble”, IEEE Transactions on Information Technology in Biomedicine, vol. 7, Issue: 1, Mar. 2003, pp. 37-42.
Zhu et al., “Technology-Assisted Dietary Assesment”, Proc SPIE. Mar. 20, 2008, pp. 1-15.
Zou et al., “A Content-Based Image Authentication System with Lossless Data Hiding”, ICME 2003, pp. 213-216.
Ma Et El. (“Semantics modeling based image retrieval system using neural networks” 2005 (Year: 2005).
Jasinschi et al., A Probabilistic Layered Framework for Integrating Multimedia Content and Context Information, 2002, IEEE, p. 2057-2060. (Year: 2002).
Jones et al., “Contextual Dynamics of Group-Based Sharing Decisions”, 2011, University of Bath, p. 1777-1786. (Year: 2011).
Iwamoto, “Image Signature Robust to Caption Superimpostion for Video Sequence Identification”, IEEE, pp. 3185-3188 (Year: 2006).
Cooperative Multi-Scale Convolutional Neural, Networks for Person Detection, Markus Eisenbach, Daniel Seichter, Tim Wengefeld, and Horst-Michael Gross Ilmenau University of Technology, Neuroinformatics and Cognitive Robotics Lab (Year; 2016).
Chen, Yixin, James Ze Wang, and Robert Krovetz. “CLUE: cluster-based retrieval of images by unsupervised learning.” IEEE transactions on Image Processing 14.8 (2005); 1187-1201. (Year: 2005).
Wusk et al (Non-Invasive detection of Respiration and Heart Rate with a Vehicle Seat Sensor; www.mdpi.com/journal/sensors; Published: May 8, 2018). (Year: 2018).
Chen, Tiffany Yu-Han, et al. “Glimpse: Continuous, real-time object recognition on mobile devices.” Proceedings of the 13th ACM Confrecene on Embedded Networked Sensor Systems. 2015. (Year: 2015).
Related Publications (1)
Number Date Country
20170169305 A1 Jun 2017 US
Provisional Applications (1)
Number Date Country
62267398 Dec 2015 US