Touch may require movements to be made with fingertips in order to sense information about what the fingers are touching. The nature of these movements may be optimized to extract the tactile properties of an object that may be useful for identifying the object. Experimental psychologists have observed a number of useful types of exploratory movements that humans make when identifying objects by touch, such as hefting, enclosing, applying pressure, and sliding. (Lederman, S J, and R L Klatzky. 1987. “Hand Movements: a Window Into Haptic Object Recognition.” Cognitive Psychology 19: 342-368.). However, even within these discrete sets of movements, there may be many ways in which these movements can be executed to collect information. For instance, different combinations of forces and sliding trajectories could be made when performing a sliding movement. Given the large number of possible movements and variations in parameters, it may be impractical to perform every possible movement to collect every piece of information before identifying what is being touched. Similar problems may arise during any type of diagnostic task when it may be impractical to collect all information before making a decision. For example, the definitive diagnosis of a disease given an initial set of symptoms could benefit from a very large number of possible tests, each of which takes a significant amount of time and money to perform. Physicians use a subjective process called differential diagnosis to estimate the probability of each possible diagnosis and the potential of each available test to differentiate among them. It would be advantageous to have an objective method to determine the most efficient sequence of tests to arrive at a final diagnosis.
Human skin contains a variety of neural transducers that sense mechanical strain, vibrations, and thermal information (Jones, L A, and S J. Lederman. 2006. Human Hand Function. New York, N.Y.: Oxford University Press, USA.; Vallbo, A B, and R S Johansson. 1984. “Properties of Cutaneous Mechanoreceptors in the Human Hand Related to Touch Sensation.” Human Neurobiology 3 (1): 3-14.). The skin and its sensory transducers are highly evolved and specialized in structure, and the glabrous skin found on the palmar surface of the human hand, and in particular the fingertip, may possess a higher density of cutaneous receptors than the hairy skin on the rest of the body (Vallbo, Å B, and R S Johansson. 1978. “The Tactile Sensory Innervation of the Glabrous Skin of the Human Hand.” In Active Touch, the Mechanism of Recognition of Objects by Manipulation, edited by G Gordon, 29-54. Oxford: Pergamon Press Ltd.; Johansson, R S, and A B Vallbo. 1979. “Tactile Sensibility in the Human Hand: Relative and Absolute Densities of Four Types of Mechanoreceptive Units in Glabrous Skin.” Journal of Physiology 286 (1): 283.). A device that mimics these sensory capabilities has been described in a form factor that has size, shape and mechanical properties similar to a human fingertip (U.S. Pat. Nos. 7,658,110, 7,878,075, 8,181,540 and 8,272,278). Other tactile sensors designed to replicate human touch have been described in a number of literature reviews covering several decades of research (Nicholls, H R, and M H Lee. 1989. “A Survey of Robot Tactile Sensing Technology.” International Journal of Robotics Research 8 (3): 3-30.; Howe, R D. 1994. “Tactile Sensing and Control of Robotic Manipulation.” Advanced Robotics 8 (3): 245-261.; Lee, M H, and H R Nicholls. 1999. “Tactile Sensing for Mechatronics—a State of the Art Survey.” Mechatronics 9: 1-31.; Dahiya, R S, G Metta, M Valle, and G Sandini. 2010. “Tactile Sensing—From Humans to Humanoids.” IEEE Transactions on Robotics 26 (1): 1-20.).
Another approach is artificial texture recognition with tactile sensors (Tada, Y, K Hosoda, and M Asada. 2004. “Sensing Ability of Anthropomorphic Fingertip with Multi-Modal Sensors.” In Proc. IEEE International Conference on Intelligent Robots and Systems, 1005-1012.; Mukaibo, Y, H Shirado, M Konyo, and T Maeno. 2005. “Development of a Texture Sensor Emulating the Tissue Structure and Perceptual Mechanism of Human Fingers.” In Proc. IEEE International Conference on Robotics and Automation, 2565-2570. IEEE.; Hosoda, K, Y Tada, and M Asada. 2006. “Anthropomorphic Robotic Soft Fingertip with Randomly Distributed Receptors.” Robotics and Autonomous Systems 54 (2): 104-109.; de Boissieu, F, C Godin, B Guilhamat, D David, C Serviere, and D Baudois. 2009. “Tactile Texture Recognition with a 3-Axial Force MEMS Integrated Artificial Finger.” In Proc. Robotics: Science and Systems, 49-56.; Sinapov, J, and A Stoytchev. 2010. “The Boosting Effect of Exploratory Behaviors.” In Proc. Association for the Advancement of Artificial Intelligence, 1613-1618.; Giguere, P, and G Dudek. 2011. “A Simple Tactile Probe for Surface Identification by Mobile Robots.” IEEE Transactions on Robotics 27 (3): 534-544.; Oddo, C M, M Controzzi, L Beccai, C Cipriani, and M C Carrozza. 2011. “Roughness Encoding for Discrimination of Surfaces in Artificial Active-Touch.” IEEE Transactions on Robotics 27 (3): 522-533.; Jamali, N, and C Sammut. 2011. “Majority Voting: Material Classification by Tactile Sensing Using Surface Texture.” IEEE Transactions on Robotics 27 (3): 508-521.; Sinapov, J, V Sukhoy, R Sahai, and A Stoytchev. 2011. “Vibrotactile Recognition and Categorization of Surfaces by a Humanoid Robot.” IEEE Transactions on Robotics 27 (3): 488-497.; Chu, V, I McMahon, L Riano, C G McDonald, Q He, J M Perez-Tejada, M Arrigo, et al. 2013. “Using Robotic Exploratory Procedures to Learn the Meaning of Haptic Adjectives.” In Proc. IEEE International Conference on Robotics and Automation.). The sliding movements humans make when identifying surface texture (Lederman, S J, and R L Klatzky. 1987. “Hand Movements: a Window Into Haptic Object Recognition.” Cognitive Psychology 19: 342-368.) may be executed with these sensors over a number of textures to identify which characteristics make them unique. Various approaches to producing these movements have been explored, including using anthropomorphic hands (Tada, Y, K Hosoda, and M Asada. 2004. “Sensing Ability of Anthropomorphic Fingertip with Multi-Modal Sensors.” In Proc. IEEE International Conference on Intelligent Robots and Systems, 1005-1012.; Hosoda, K, Y Tada, and M Asada. 2006. “Anthropomorphic Robotic Soft Fingertip with Randomly Distributed Receptors.” Robotics and Autonomous Systems 54 (2): 104-109.; Oddo, C M, M Controzzi, L Beccai, C Cipriani, and M C Carrozza. 2011. “Roughness Encoding for Discrimination of Surfaces in Artificial Active-Touch.” IEEE Transactions on Robotics 27 (3): 522-533.; Jamali, N, and C Sammut. 2011. “Majority Voting: Material Classification by Tactile Sensing Using Surface Texture.” IEEE Transactions on Robotics 27 (3): 508-521.; Chu, V, I McMahon, L Riano, C G McDonald, Q He, J M Perez-Tejada, M Arrigo, et al. 2013. “Using Robotic Exploratory Procedures to Learn the Meaning of Haptic Adjectives.” In Proc. IEEE International Conference on Robotics and Automation.), 2-axis plotting machines (de Boissieu, F, C Godin, B Guilhamat, D David, C Serviere, and D Baudois. 2009. “Tactile Texture Recognition with a 3-Axial Force MEMS Integrated Artificial Finger.” In Proc. Robotics: Science and Systems, 49-56.), robotic arms (Sinapov, J, V Sukhoy, R Sahai, and A Stoytchev. 2011. “Vibrotactile Recognition and Categorization of Surfaces by a Humanoid Robot.” IEEE Transactions on Robotics 27 (3): 488-497.), or manual sliding (Giguere, P, and G Dudek. 2011. “A Simple Tactile Probe for Surface Identification by Mobile Robots.” IEEE Transactions on Robotics 27 (3): 534-544.). Previous studies employed a fixed exploration sequence for collecting data, which, after processing, was fed into a machine learning classifier that sought to identify the texture. One exception was (Jamali, N, and C Sammut. 2011. “Majority Voting: Material Classification by Tactile Sensing Using Surface Texture.” IEEE Transactions on Robotics 27 (3): 508-521.), who repeated the same sliding movement until the classification reached a desired confidence.
Using additional exploratory movements has been demonstrated to improve performance (Sinapov, J, V Sukhoy, R Sahai, and A Stoytchev. 2011. “Vibrotactile Recognition and Categorization of Surfaces by a Humanoid Robot.” IEEE Transactions on Robotics 27 (3): 488-497.). However, executing every possible movement to gain all information about an object may be impractical, so these systems were restricted to a small number of preprogrammed exploratory movements. This approach may only provide marginal performance accuracies when using a small number of highly distinctive surfaces that would be trivial for a human observer to discriminate. Examples of classification performance in previous literature include: 62% over 10 textures (de Boissieu, F, C Godin, B Guilhamat, D David, C Serviere, and D Baudois. 2009. “Tactile Texture Recognition with a 3-Axial Force MEMS Integrated Artificial Finger.” In Proc. Robotics: Science and Systems, 49-56.), 89.9-94.6% over 10 textures (Giguere, P, and G Dudek. 2011. “A Simple Tactile Probe for Surface Identification by Mobile Robots.” IEEE Transactions on Robotics 27 (3): 534-544.), 95% over 20 textures (Sinapov, J, V Sukhoy, R Sahai, and A Stoytchev. 2011. “Vibrotactile Recognition and Categorization of Surfaces by a Humanoid Robot.” IEEE Transactions on Robotics 27 (3): 488-497.), 97.6% over 3 textures (Oddo, C M, M Controzzi, L Beccai, C Cipriani, and M C Carrozza. 2011. “Roughness Encoding for Discrimination of Surfaces in Artificial Active-Touch.” IEEE Transactions on Robotics 27 (3): 522-533.), and 95% over 8 textures (Jamali, N, and C Sammut. 2011. “Majority Voting: Material Classification by Tactile Sensing Using Surface Texture.” IEEE Transactions on Robotics 27 (3): 508-521.).
Loeb et al., 2011, (Loeb, G E, G A Tsianos, J A Fishel, N Wettels, and S Schaal. 2011. “Understanding Haptics by Evolving Mechatronic Systems.” Progress in Brain Research 192: 129-144.), suggested the general desirability of selecting exploratory movements incrementally according to the most likely identity of the object being explored but, provided no examples or methods to do so.
An object investigation and classification system may include an object test system, a data storage system, and a data processing system. The object test system may receive a command to perform at least one action with a test object, perform the at least one action with the test object, and return test information indicative of at least one percept resulting from the at least one action. The data storage system may contain an experience database containing data indicative of multiple classifications and, for each classification, at least one action that was performed with at least one previously-observed reference object having this classification, and at least one percept value that is based in whole or in part on the test information resulting from the at least one action. The data processing system may: a) for each of multiple different classifications, compute or receive an initial prior probability that a test object falls within the classification; b) determine at least one action that should be performed with the test object to obtain at least one percept about the test object that is likely to enable the classification of the test object to be more accurately determined based on the initial prior probabilities and the data within the experience database; c) cause the object test system to perform the at least one action with the test object; d) receive test information from the object test system indicative of at least one percept resulting from the at least one action with the test object; e) compute at least one percept value; f) for each of multiple different classifications, determine a posterior probability that the test object falls within the classification based on the initial prior probability, the at least one percept value, and data within the experience database; g) determine whether any of the posterior probabilities meets or exceeds a threshold; h) if none of the posterior probabilities meet or exceed the threshold, repeat b) through i), substituting the posterior probabilities determined in f) for the initial prior probabilities in b); and/or i) when one or more of the posterior probabilities meets or exceeds the threshold, output information indicative of one or more of the classifications that correspond to the one or more posterior probabilities that meets or exceeds the threshold.
The data in the experience database may include data indicative of a distribution of percept values for at least one of the percepts resulting from an action that has been performed multiple times in association with a given classification or a given previously observed reference object.
The threshold may not be the same during all of the repetitions of g).
The data processing system may add data about at least one of the percepts indicated by the received test information to the experience database.
The data processing system may determine which of the percepts indicated by the received test information should have data about them added to the experience database based on the degree to which the at least one action that led to each percept caused a change in the probability that the test object has one or more of the classifications.
The data processing system may determine if the percept values of at least one percept resulting from the at least one action performed with the test object is substantially unlike the percept values of the same at least one percept resulting from the same at least one action associated with any of the classifications described in the experience database.
The data processing system may add the percept values of at least one percept resulting from the at least one action with the test object to the experience database in association with a new classification that was not in the experience database when the data processing system determines that the percept values of at least one percept resulting from at least one action performed with the test object is substantially unlike the percept values of the same at least one percept resulting from the same at least one action associated with any of the classifications described in the experience database.
The data processing system may determine at least one additional action to perform with the test object to obtain at least one percept associated with the at least one additional action and add the percept value of the at least one percept resulting from the at least one additional action to the experience database.
The initial prior probabilities may be the same.
The initial prior probabilities may be weighted based on the number of times each classification has been associated with a previously-observed reference object in the experience database.
The experience database may also contain data indicative of a time when each percept was obtained. The initial prior probabilities may be weighted based on the time each percept was obtained.
The experience database may also contain data indicative of a location where each percept was obtained. The initial prior probabilities may be weighted based on the location each percept was obtained.
The determines at least one action that should be performed may include:
a) for each classification, computing a probability density function that describes a distribution of percept values expected for a percept resulting from an action that has been performed multiple times in association with the classification; b) computing a degree to which two different probability density functions for two different classifications result in similar distributions of the percept values of the same percept when performing the same action; c) multiplying the degree computed in b) by the prior probability that the test object has each classification used to compute the degree in b); d) repeating b and c for all other possible pairs of classifications; e) summing the results of all of the multiplications in c); f) repeating a-e for each of the other combinations of actions and percepts; and g) selecting the action that yields the lowest summing value in e) for any percept as the action to be performed.
The previously performed actions with the test object may be given less preference in being re-selected as the action to be performed if the previously performed action was unsuccessful in producing percept values that help discriminate between the most likely classifications.
The object test system may include at least one controllable actuator that performs the at least one action with the test object, and at least one tactile sensor that interacts with the test object and returns the test information indicative of at least one percept resulting from the at least one action.
At least one action with the test object may include sliding across a surface of the test object or contacting the test object with a varying force.
The data processing system may process the test information indicative of the at least one percept to indicate a type of surface texture on the test object, a degree of roughness or smoothness of the test object, a degree of coarseness or fineness of the test object, a degree of hardness or softness of the test object, a degree to which the test object has a springiness or dampens; and/or a thermal property of the test object.
A non-transitory, tangible, computer-readable storage medium may contain a program of instructions that cause a computer system running the program of instructions to perform one, any sub-combination, or all of the functions of the data processing system as described herein.
These, as well as other components, steps, features, objects, benefits, and advantages, will now become clear from a review of the following detailed description of illustrative embodiments, the accompanying drawings, and the claims.
The drawings are of illustrative embodiments. They do not illustrate all embodiments. Other embodiments may be used in addition or instead. Details that may be apparent or unnecessary may be omitted to save space or for more effective illustration. Some embodiments may be practiced with additional components or steps and/or without all of the components or steps that are illustrated. When the same numeral appears in different drawings, it refers to the same or like components or steps.
Illustrative embodiments are now described. Other embodiments may be used in addition or instead. Details that may be apparent or unnecessary may be omitted to save space or for a more effective presentation. Some embodiments may be practiced with additional components or steps and/or without all of the components or steps that are described.
The following definitions apply to both this specification and the claims:
An ACTOR is a component that can interact with an OBJECT so as to cause it to generate information.
A SENSOR is a component that can detect or receive information from an OBJECT.
An OBJECT is any physical thing that is capable of interacting with one or more ACTORS and producing information that can be detected or received by one or more SENSORS.
A CLASSIFICATION is any descriptive attribute of an OBJECT.
A CLASSIFICATION SET is a grouping of two or more mutually exclusive CLASSIFICATIONS.
A PERCEPT is an abstraction of information from the one or more SENSORS when performing an ACTION that reflects a characteristic of the OBJECT.
An ACTION is a particular interaction that an ACTOR can perform with an OBJECT.
An EXPERIENCE DATABASE is a database that contains data indicative of previously executed ACTIONS with OBJECTS and PERCEPTS that result from the actions, as well as and one or more CLASSIFICATIONS of each OBJECT.
A PROBABILITY of a CLASSIFICATION of an OBJECT is a likelihood that the given OBJECT falls within the CLASSIFICATION. The PROBABILITY may be expressed in any form, such as a number between zero and one or a percentage.
A PROBABILITY DENSITY FUNCTION for a given PERCEPT and a given ACTION with a given CLASSIFICATION is the relative likelihood of the PERCEPT taking on a given value when performing the ACTION with an OBJECT that is representative of the CLASSIFICATION.
A PRIOR PROBABILITY of a CLASSIFICATION is the PROBABILITY of the CLASSIFICATION computed prior to performing an ACTION and abstracting a PERCEPT.
A POSTERIOR PROBABILITY of a CLASSIFICATION is the PROBABILITY of the CLASSIFICATION computed after performing an ACTION and abstracting a PERCEPT.
Various object investigation and classification systems for investigating and determining one or more classifications of an object are now described. An object investigation and classification system may include a physical apparatus with a tactile sensor to interact mechanically and to sense tactile information from the object in order to determine its classifications by touch. The object investigation and classification system may be capable of performing one or more actions with an object and detecting or receiving sensory information that may be computed to determine the value of one or more percepts that correlate with one or more classifications. The object investigation and classification system may have the ability to perform a large number of possible actions such that it may be impractical or time consuming for the object investigation and classification system to efficiently perform all of these actions before attempting to resolve the one or more classifications of the object. To efficiently sequence actions to perform in determining the classification(s) of an object within a single classification set consisting of mutually exclusive classifications, the object investigation and classification system may start with a set of initial prior probabilities describing the various probabilities that an object possesses each of the classifications in that classification set. The object investigation and classification system may utilize an experience database that may include records associating actions and percept values from previously-observed reference objects representative of various classifications in order to determine the optimal action that is expected to result in sensory information and consequent percept values that would be most likely to disambiguate between the most probable classification(s) of the object. The object investigation and classification system may decide to execute this optimal action once it is determined and receive sensory information that can be computed into one or more percept values. After execution of this action and computation of these percept values, the object investigation and classification system may combine the computed percept values and prior probabilities with information in the experience database to determine the posterior probabilities describing the updated probabilities that an object possesses each of the classifications in the classification set given this new information. The object investigation and classification system may iterate this process of selecting optimal queries, substituting posterior probabilities for prior probabilities until a desired confidence in a single classification within the one or more classification sets is reached.
An object may have one or more classifications corresponding with one or more classification sets. A classification may describe any descriptive attribute of the object, such as a name reflecting its identity, a category of objects which it is a part of, a feature that it possesses, or an affordance (as defined by Gibson, J. J., 1977, “The Theory of Affordances” in Perceiving, Acting, and Knowing, eds. Robert Shaw and John Bransford, ISBN 0-470-99014-7). For instance, the object may be a piece of shirt fabric with the following classifications: soft, fabric, cloth, flannel, capable of being used as a filter, 140-count twill weave double napped Canton flannel.
A classification may be applied to one or more objects. A classification set contains two or more classifications that are mutually exclusive. Classification from different classification sets may or may not overlap with, be a subset or superset of, be mutually exclusive with, or be mutually inclusive with each other across more than one previously-observed reference objects.
Referring to
Still referring to
Still referring to
Still referring to
Still referring to
Still referring to
Still referring to
Still referring to
The object investigation and classification system 100 may be able to perform a large number of actions with its one or more actors 130 and to abstracting a large number of percepts derived from information detected or received by the one or more sensors 140 during such actions. Executing all of these actions may be too numerous, impractical, or time-consuming for the object investigation and classification system 100 to perform before identifying the one or more classifications of the test object 120. The data processing system 101 may intelligently sequence the actions to be performed by the one or more actors 130 to determine efficiently the one or more classifications of the test object 120 that the object investigation and classification system 100 seeks to identify.
Still referring to
Referring to
The algorithm may be implemented by the data processing system 101.
Still referring to
Still referring to
Referring to the box below, ai is a given action, pj is a given percept that can be computed from sensory information obtained from performing the action ai, cm is a given classification,
Still referring to
Still referring to
Still referring to
Still referring to
The above algorithm describes a method that may be capable of intelligently sequencing actions to identify efficiently a single classification within a classification set. Various approaches may be taken to solve for two or more classifications within two or more corresponding classification sets. One approach may be to set the initial prior probabilities 211 for the two or more classifications within the two or more classification sets, then use the above described algorithm to select the optimal actions to identify the classification within the first classification set while also using the posterior estimator 214 to update the probabilities of the other classifications within the other classification sets with these selected actions and resulting percept values. Upon converging on a classification within the first classification set, the process may be repeated for the remaining classifications to be identified in the remaining classification sets. Another variation to the approach above may be to select the optimal action to identify the classification within the classification set that has the highest probability that has yet to exceed the threshold across all possible classifications across all classification sets seeking to be identified. Another approach may be to converge the two or more classification sets into a single classification set by combining all possible combinations of each classification between all classification sets seeking to be identified. For instance, if one classification set contained the three mutually exclusive colors red, green and blue and another classification set contained the two mutually exclusive shapes circle and square, the new classification set would contain 6 mutually exclusive merged classifications: red-circle, red-square, green-circle, green-square, blue-circle, and blue-square. This merged set may be treated as a single classification set; however, depending on the relationships between classifications within the classification sets, some of these combined classifications may be mutually exclusive. Instead, the new classification set may consist of all observed instances of possible combinations of each classification between all classification sets seeking to be identified.
The algorithm for investigating and determining the classification of an object that may be implemented by the data processing system 101 may benefit from an experience database 113 that contains a large amount of previous experience describing the values of one or more percepts that arise from actions that are useful for disambiguating possible classifications within a classification set. Because the useful actions and percepts for disambiguating possible classifications within a classification set may not be known prior to collecting this previous experience, the object investigation and classification system 100 could be initially programmed to perform a small number of repetitions over many possible candidate actions with the test object 120 and compute values of one or more percepts for each candidate action to be stored in the experience database 113 to gain an initial set of information describing previously-observed reference objects. Through executing the above-described algorithm in future encounters and adding experiences of actions and percepts determined to be useful in disambiguating possible classifications of a test object 120 within a classification set, the experience database 113 may automatically acquire a larger amount of experience in a smaller set of actions and percepts found to be useful.
Estimating probability density functions of a given percept for a given action for a given classification from a limited amount of information in the experience database 113 may result in errors between the estimated probability density function and the true probability density function of that percept for a given action and given classification. If the previous experience in the experience database 113 is limited, these errors may become more severe when the posterior estimator 214 attempts to compute the posterior probabilities based on more than one percept at a time. This phenomenon is known as the curse of dimensionality (Jain, A K, R P W Duin, and J. Mao. 2000. “Statistical Pattern Recognition: a Review.” IEEE Transactions on Pattern Analysis and Machine Intelligence 22 (1): 4-37.) and is familiar to those skilled in the arts of statistical pattern recognition. To circumvent this, the posterior estimator 214 may initially analyze only a single percept for a given action when computing posterior probabilities, considering only the optimal percept as determined by the action selector 212, and then gradually increase the number of percepts to contribute to the computation of the posterior probabilities by the posterior estimator 214 as more experience is obtained in the experience database 113. All percepts obtained when conducting an action, including those that may not be used by the above algorithm, may still be added to the experience database 113 after a classification is identified by the convergence calculator 215.
The total anticipated ambiguity for a given percept and a given action as computed by the action selector 212 could consider the number of times the given action and resulting percept has been performed on the test object 120 that the object investigation and classification system 100 is currently investigating and use this information to provide a more accurate prediction of the total anticipated ambiguity. For instance, if the action selector 212 suggests a particular action and percept to be optimal based on information contained in the experience database 113 and current prior probabilities, yet the object investigation and classification system 100 has previously performed this action and computed a value for this percept on the test object 120 that it is currently investigating without yielding useful information to disambiguate likely classifications of the object, this information may indicate that the total anticipated ambiguity as computed by the action selector should be higher. To accomplish this the total anticipated ambiguity as computed by the action selector could include a penalty factor based on the number of times the given action has been repeated, such as, but not limited to, raising the total anticipated ambiguity to a power that becomes greater than one according to the number of times this has occurred without producing significant changes in the posterior probabilities. This may serve to encourage the action selector 212 to select new actions if the selected actions are not helping the object investigation and classification system 100 converge on a given classification.
To avoid the need for supervised learning to label classifications, the data processing system 101 may be programmed create new classifications based on the distributions of actions and percepts in the experience database 113. The data processing system 101 may be programmed to use cluster analysis to identify if the distribution of percepts computed while performing actions with a test object 120 are likely to correspond to previously-experienced objects of one or more existing classifications or if they are more likely to represent one or more novel classifications. If the latter, then the data processing system 101 may add one or more new classifications to the experience database 113, with each such new classification containing the previously experienced actions and percepts (or a statistical summary thereof) that are most closely associated with the new one or more classifications using methods familiar to those skilled in the art of cluster analysis and unsupervised learning. Prior to or after adding the one or more new classifications to the experience database 113, the data processing system 101 may also perform one or more additional actions and obtain one or more percepts resulting from each action and add this information to the experience database 113 to obtain additional information on the new classification.
A prior probability or a posterior probability for a given classification may be a number between zero and one corresponding to the likelihood that the test object 120 possesses the given classification or it may be any other value on any scale that correlates with the likelihood that the test object 120 possesses the given classification.
The one or more linear actuators 330 may be capable of moving the tactile sensor 340 to perform actions that are similar to the movements humans make when exploring objects by touch. The tactile sensor 340 may have sensory capabilities, sensitivity, resolution, and mechanical properties and may interact with objects being explored in ways that are similar to those of human fingertips. The one or more feedback controllers 150 may make use of information from the tactile sensor 340 or from multiple tactile sensors instrumented on the one or more linear actuators 330 to control exploratory movement. Signal processing algorithms may measure percepts that correspond to the linguistic descriptions of percepts observed by humans when identifying tactile properties of objects. The experience database 113 may be used and may include previous experience exploring similar objects and may include and associate descriptors of each exploratory movement with linguistic measurements of object properties. A biologically inspired decision making process may be employed for determining an optimal next exploratory movement to make. The object test system 102 may apply these tactile exploration, measurement and perception technologies in quality control, product design, and in other fields.
Still referring to
The tactile sensor 340 may have sensory capabilities, sensitivity, resolution, and mechanical properties and may interact with objects being explored similar to those of a human fingertip. The tactile sensor 340 may be the BioTac (made by SynTouch LLC, Los Angeles, Calif.). The ability of the tactile sensor 340 to produce interactions between the tactile surface 320 that are similar to those that would be produced when interacting with a human fingertip may benefit from the tactile sensor 340 having similar mechanical properties as the human fingertip, such as similar compliance, shape, and size. The tactile sensor 340 may also or instead have similar features as the human fingertip, such as a fingernail that facilitates the transduction of shear forces applied to the fingerpad, and fingerprints that enhance vibrations sensed when sliding the tactile sensor 340 over a textured surface. The ability of the tactile sensor 340 to perceive sensations similar to those a human may perceive when exploring the tactile surface 320 may benefit from the tactile sensor 340 having sensory modalities similar to those found in human skin, such as sensitivity to contact location, normal and shear forces, vibrations, and/or temperature.
The one or more feedback controllers 150 that make use of information from the tactile sensor 340 or sensors instrumented on the one or more linear actuators 330 to control exploratory movements may include the ability to control the specified force of the one or more linear actuators 330 in the normal axis. The specified force in the one or more linear actuators 330 in the normal axis may be controlled using the one or more feedback controllers 150 that may make use of sensory information from the tactile sensor 340 or other sensors instrumented on the one or more linear actuators 330 in the normal axis, such as force plates, motor current sensors, strain gages, or other technologies familiar to those skilled in the art of force measurement. The tactile sensor 340 may be a fluid-filled tactile sensor capable of sensing fluid pressure. The fluid pressure in the tactile sensor 340 may be used to stabilize contact force by adjusting the position of the one or more linear actuators 330 in the normal axis by means of a feedback controller that maintains the fluid pressure reading at a constant value.
The signal processing strategies for the measurement of percepts that correspond to the linguistic descriptions of percepts observed by humans when identifying tactile properties of objects may be computed from the sensory information obtained from the tactile sensor 340 and/or from other sensors contained in the one or more linear actuators 330, such as position encoders, strain gages, motor current sensors, force plates, or other technologies familiar to those skilled in the art of actuation. Examples of linguistic descriptions of percepts observed by humans have been observed in prior art and may include, but are not limited to: properties relating to surface texture including, but not limited, to roughness, coarseness, slipperiness, or regularity; properties relating to compliance, including, but not limited to, hardness or springiness; and/or properties relating to thermal properties, including, but not limited to thermal effusivity, thermal conductivity, and heat capacity. The percept of texture roughness may be computed by analyzing the logarithm of the variance of vibrations measured by the tactile sensor 340 when sliding over a surface. The percept of texture coarseness can be computed by transforming the vibrations measured by the tactile sensor 340 into the frequency domain with a fast Fourier transform and multiplying the power of the individual frequency bins by their respective frequency and summing all respective terms, then dividing the result by the total power of all frequency bins, and taking the logarithm of the result. The percept of texture slipperiness can be computed from the inverse of the tangential load of the actuator when sliding at a specified contact force. The percept of texture regularity can be computed from the diffusivity of the spectral content in the Fourier transform when the tactile sensor 340 slides over a tactile surface. The percept of hardness can be computed as the inverse of mechanical compliance, which may be determined by measuring the displacement of the tactile sensor 340 as the force between the tactile sensor 340 and the test object 120 is changed by the one or more linear actuators 330. The percept of thermal effusivity can be computed from the measurement of temperature over time in the tactile sensor 340 that is heated above ambient temperature. Other signal processing strategies, such as those described in (Lin, C H, T W Erickson, J A Fishel, undefined author, N Wettels, and G E Loeb. 2009. “Signal Processing and Fabrication of a Biomimetic Tactile Sensor Array with Thermal, Force and Microvibration Modalities.” In Proc. IEEE International Conference on Robotics and Biomimetics, 129-134.; Su, Z, J A Fishel, T Yamamoto, and G E Loeb. 2012. “Use of Tactile Feedback to Control Exploratory Movements to Characterize Object Compliance.” Frontiers in Neurorobotics 6(7): 1-12.; Fishel, J A, and G E Loeb. 2012. “Bayesian Exploration for Intelligent Identification of Textures.” Frontiers in Neurorobotics 6(4): 1-20.; Chu, V, I McMahon, L Riano, C G McDonald, Q He, J M Perez-Tejada, M Arrigo, et al. 2013. “Using Robotic Exploratory Procedures to Learn the Meaning of Haptic Adjectives.” In Proc. IEEE International Conference on Robotics and Automation.) for characterizing tactile properties of texture, compliance and thermal properties may also be used.
The one or more linear actuators 330 may be precision components designed to produce smooth motions with high accuracy and repeatability with low mechanical vibration such that the variability and noise of the one or more linear actuators 330 produce variability of percept values that is similar to those that would be computed if the linear actuators were not moving. Such precision components may include, but are not limited to, actuators with precision cross roller bearings, actuators with air bearings, actuators with hydraulic cylinders and other actuators and mechanical linkages familiar to those skilled in the art of mechatronic design. The fidelity of sensor information collected from the tactile sensor 340 may benefit from the low background noise levels produced from such precision components as discussed below. An example of a suitable linear actuator may include the ANT95-75-L (Aerotech, Pittsburg, Pa.) or other similar product families. The apparatus may have two linear actuators 330, one to control the movement in the direction normal to the tactile surface 320 and another to control the movement in the direction tangential to the tactile surface 320. However, the actuators need not be linear and alternative technologies such as rotary actuators may be used as familiar to those skilled in the art of mechatronic design.
The above described object investigation and classification system 100 incorporating the tactile sensor 340 and linear actuators 330 was used in an experiment to test its ability to identify a single texture within a set of 117 different textures. (Fishel, J A, and G E Loeb. 2012. “Bayesian Exploration for Intelligent Identification of Textures.” Frontiers in Neurorobotics 6(4): 1-20.). The data processing system 101 was programmed to control the one or more linear actuators 330 to produce three different actions that consisted of sliding motions at different contact forces and velocities (0.20N contact force, 6.31 cm/s sliding velocity; 0.5N contact force, 2.51 cm/s sliding velocity; and 1.26N contact force, 1 cm/s sliding velocity). The data processing system 101 was programmed to receive test information from the tactile sensor 340 and additional sensors within the one or more linear actuators 330 and to compute the following three percepts from each action: i) the percept of texture roughness, which was computed by analyzing the logarithm of the variance of vibrations measured by the tactile sensor 340 when sliding over a surface; ii) the percept of texture coarseness, which was computed by transforming the vibrations measured by the tactile sensor 340 into the frequency domain with a fast Fourier transform and multiplying the power of the individual frequency bins by their respective frequency and summing all respective terms, then dividing the result by the total power of all frequency bins, and taking the logarithm of the result; and iii) the percept of texture slipperiness, which was computed from the inverse of the tangential load on the actuator when sliding at the specified contact force. Using the algorithm described above, the object investigation and classification system 100 obtained 95.4% accuracy over 117 textures, 99.6% accuracy discriminating between pairs of nearly identical textures (in comparison, human subjects were only able to obtain 81.3% performance in discriminating these pairs of nearly identical textures), and 100% performance over 10 textures intentionally selected due to their dissimilarity. In comparison, other methods were found to have only marginal performance accuracies when using a small number of highly distinctive surfaces that would be trivial to discriminate for a human observer or for the object investigation and classification system 100. Examples of classification performance in previous literature include: 62% over 10 textures (de Boissieu, F, C Godin, B Guilhamat, D David, C Serviere, and D Baudois. 2009. “Tactile Texture Recognition with a 3-Axial Force MEMS Integrated Artificial Finger.” In Proc. Robotics: Science and Systems, 49-56.), 89.9-94.6% over 10 textures (Giguere, P, and G Dudek. 2011. “A Simple Tactile Probe for Surface Identification by Mobile Robots.” IEEE Transactions on Robotics 27 (3): 534-544.), 95% over 20 textures (Sinapov, J, V Sukhoy, R Sahai, and A Stoytchev. 2011. “Vibrotactile Recognition and Categorization of Surfaces by a Humanoid Robot.” IEEE Transactions on Robotics 27 (3): 488-497.), 97.6% over 3 textures (Oddo, C M, M Controzzi, L Beccai, C Cipriani, and M C Carrozza. 2011. “Roughness Encoding for Discrimination of Surfaces in Artificial Active-Touch.” IEEE Transactions on Robotics 27 (3): 522-533.), and 95% over 8 textures (Jamali, N, and C Sammut. 2011. “Majority Voting: Material Classification by Tactile Sensing Using Surface Texture.” IEEE Transactions on Robotics 27 (3): 508-521.).
These tactile exploration, measurement, and perception technologies may be used in quality control and product design and in other fields. The above described system that seeks to mimic biological processes for identifying the classification of an object based on its tactile properties may be used to provide quantitative measures of human tactile perception. The object investigation and classification system 100 may be used in product development applications to determine if one product has a similar feel to another product. In order to determine the attribute of a particular feel, the object investigation and classification system 100 may acquire an experience database that is organized according to classifications reflecting various possible feels. Such an experience database may be accumulated by performing various actions and recording the resulting percept values obtained with reference objects already classified according to the various possible feels by humans who may have experience in making such classifications. Design and manufacturing industries for garments, paper goods, consumer electronics, cosmetics, skin and hair care products, prepared foods, and other products commonly employ humans with expertise in classifying objects and materials according to feel. Classification according to feel may be useful when designing a product that seeks to mimic the feel of another product, or restore the feel of an object that has been damaged using various surface treatments. The above described object investigation and classification system 100 may also be useful for determining which combinations of classifications have desirable or undesirable traits that can be identified in consumer preference studies. The above described system may also be useful in applications of quality control.
Referring to
Referring to
Unless otherwise indicated, the data processing system 101, the object test system 102, and the data storage system 103 may each be implemented with the same or a different computer system configured to perform the functions that have been described herein for the component. Each computer system may include one or more processors, tangible memories (e.g., random access memories (RAMs), read-only memories (ROMs), and/or programmable read only memories (PROMS)), tangible storage devices (e.g., hard disk drives, CD/DVD drives, and/or flash memories), system buses, video processing components, network communication components, input/output ports, and/or user interface devices (e.g., keyboards, pointing devices, displays, microphones, sound reproduction systems, and/or touch screens).
Each computer system may include one or more computers at the same or different locations. When at different locations, the computers may be configured to communicate with one another through a wired and/or wireless network communication system.
Each computer system may include software (e.g., one or more operating systems, device drivers, application programs, and/or communication programs). When software is included, the software includes programming instructions and may include associated data and libraries. When included, the programming instructions are configured to implement one or more algorithms that implement one or more of the functions of the computer system, as recited herein. The description of each function that is performed by each computer system also constitutes a description of the algorithm(s) that performs that function.
The software may be stored on or in one or more non-transitory, tangible storage devices, such as one or more hard disk drives, CDs, DVDs, and/or flash memories. The software may be in source code and/or object code format. Associated data may be stored in any type of volatile and/or non-volatile memory. The software may be loaded into a non-transitory memory and executed by one or more processors.
The components, steps, features, objects, benefits, and advantages that have been discussed are merely illustrative. None of them, nor the discussions relating to them, are intended to limit the scope of protection in any way. Numerous other embodiments are also contemplated. These include embodiments that have fewer, additional, and/or different components, steps, features, objects, benefits, and advantages. These also include embodiments in which the components and/or steps are arranged and/or ordered differently.
For example, the data processing system 101 and the data storage system 103 may be combined into a single component such as a microcomputer. For example, the one or more feedback controllers 150 may be part of the object test system 102 or part of the data processing system 101.
Unless otherwise stated, all measurements, values, ratings, positions, magnitudes, sizes, and other specifications that are set forth in this specification, including in the claims that follow, are approximate, not exact. They are intended to have a reasonable range that is consistent with the functions to which they relate and with what is customary in the art to which they pertain.
All articles, patents, patent applications, and other publications that have been cited in this disclosure are incorporated herein by reference.
The phrase “means for” when used in a claim is intended to and should be interpreted to embrace the corresponding structures and materials that have been described and their equivalents. Similarly, the phrase “step for” when used in a claim is intended to and should be interpreted to embrace the corresponding acts that have been described and their equivalents. The absence of these phrases from a claim means that the claim is not intended to and should not be interpreted to be limited to these corresponding structures, materials, or acts, or to their equivalents.
The scope of protection is limited solely by the claims that now follow. That scope is intended and should be interpreted to be as broad as is consistent with the ordinary meaning of the language that is used in the claims when interpreted in light of this specification and the prosecution history that follows, except where specific meanings have been set forth, and to encompass all structural and functional equivalents.
Relational terms such as “first” and “second” and the like may be used solely to distinguish one entity or action from another, without necessarily requiring or implying any actual relationship or order between them. The terms “comprises,” “comprising,” and any other variation thereof when used in connection with a list of elements in the specification or claims are intended to indicate that the list is not exclusive and that other elements may be included. Similarly, an element preceded by an “a” or an “an” does not, without further constraints, preclude the existence of additional elements of the identical type.
None of the claims are intended to embrace subject matter that fails to satisfy the requirement of Sections 101, 102, or 103 of the Patent Act, nor should they be interpreted in such a way. Any unintended coverage of such subject matter is hereby disclaimed. Except as just stated in this paragraph, nothing that has been stated or illustrated is intended or should be interpreted to cause a dedication of any component, step, feature, object, benefit, advantage, or equivalent to the public, regardless of whether it is or is not recited in the claims.
The abstract is provided to help the reader quickly ascertain the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In addition, various features in the foregoing detailed description are grouped together in various embodiments to streamline the disclosure. This method of disclosure should not be interpreted as requiring claimed embodiments to require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed embodiment. Thus, the following claims are hereby incorporated into the detailed description, with each claim standing on its own as separately claimed subject matter.
This application is based upon and claims priority to U.S. provisional patent application 61/750,674, entitled “Apparatus and Method for Characterizing, Discriminating, and Identifying Objects Based on Tactile Properties,” filed Jan. 9, 2013, attorney docket number 085936-0022. The entire content of this application is incorporated herein by reference.
This invention was made with government support under Contract No. D11PC20121, awarded by the Defense Advanced Research Projects Agency (DARPA). The government has certain rights in the invention.
Number | Date | Country | |
---|---|---|---|
61750674 | Jan 2013 | US |