Patent Application
20200065394
The present disclosure relates to analysis of data to determine the deception/trustworthiness of an individual.
The Internet enables individuals to participate in the creation and sharing of content in various forms of unstructured data, for example, creating editable text documents, spreadsheets, sharing calendars, notes, chats, pictures, videos, voice recordings, etc. Unstructured content includes, for example, pictures/images, audio recordings, videoconferencing, etc. These types of data elements are considered unstructured because there is an absence of a predefined data model or are not organized in a predefined manner. Applications such as Google Docs, Flickr, and Facebook allow individuals to distribute and share unstructured content. Also, there are products that enable the management, search, and analysis of unstructured data such as IBM's® Watson solutions, NetOwl®, LogRhythm®, ZL Technologies, SAS®, Inxight®, etc. These solutions can extract structured data from unstructured content for business intelligence or analytics and are for general use. However, these products do not detect the deception of an individual by analyzing their answers to questions that are contained in one or more different types of unstructured content such as video, audio recordings, documents, images, etc.
An exemplary embodiment of the present disclosure provides a method for detecting deception of an individual, the method including: receiving, in a server that includes at least one processor device and a memory, a first data item from a computing device of the individual, wherein the first data item represents one or more answers to one or more questions presented to the individual by the computing device; converting, by the server, the first data item to structured data if the first data item is unstructured data; and determining, by the server, probability of deception of the individual in their one or more answers based on analysis of the structured data from the first data item.
An exemplary embodiment of the present disclosure provides a server configured to detect deception of an individual. The server includes: a memory; and at least one processor device, wherein the server is configured to: receive a first data item from a computing device of the individual, wherein the first data item represents one or more answers to one or more questions presented to the individual by the computing device, convert the first data item to structured data if the first data item is unstructured data, and determine probability of deception of the individual in their one or more answers based on analysis of the structured data from the first data item.
The scope of the present disclosure is best understood from the following detailed description of exemplary embodiments when read in conjunction with the accompanying drawings, wherein:
The present disclosure is directed to a system and method for collecting unstructured data and detecting deception of an individual 100 by analyzing their answers to questions that are contained in one or more different types of unstructured content such as video, audio recordings, documents, images, etc. Specifically, the system and method detects deception using a multi-layered model based on unstructured data such as audio recordings, telephonic conversations, video streams, or text documents such as email, SMS, chats logs, etc. The analysis of such unstructured data can include the use of specific methods for a particular data type, such as psycholinguistics, advanced analytics, cognitive analysis, etc. These methods will convert unstructured data into structured data that is inputted into the multi-layer model that detects deception with a certain level of confidence. In the multi-layer model, the different types of unstructured content are combined to determine a probability of deception in the content analyzed. The probability of deception can be expressed, for example, using a number larger than zero and less or equal to one, with zero indicating no deception. Alternatively, the probability of deception can be expressed based on a letter grade, word, color, or in any other manner. In an exemplary embodiment, the probability of deception is calculated for each of the answers collected during the interview of the individual 100, then it is aggregated for each of the competencies or characteristics that are being evaluated in the assessment, and finally an overall value of deception is calculated for the entire completed interview of the individual 100 (e.g., a candidate for a job, a potential person to date, person questioned by law enforcement/government, etc.). In an exemplary embodiment, some or all of the analysis of the unstructured data can be performed by artificial intelligence.
The computing device 110 uses the stored application 120 to perform a real-time interview of the individual 100 which can be, for example, a recording (audio and/or video) or a collection of one-way interactions with the interviewed individual 100. The computing device 110, running the application 120, presents to the individual 100 a set of questions (e.g., with an initial predefined order) and a related instruction on how the answer to the question is to be captured. For example, the answer to the question could be an answer to a multiple choice question, a written text answer to the question inputted by a keyboard or touchscreen, an audio recording of the answer, or a video recording of the answer. In an exemplary embodiment, during the interview, the next question can be selected according to the previous answer. The presenting of questions and the capturing of their answers allows for the collection of unstructured data elements that are inputted into the multi-layer deception model module 204 for deception detection.
The computing device 110 establishes a connection with the server 130 that contains the set of questions that can be presented to the individual 100. The computing device 110 can include one or more of a keyboard, a microphone, and a video camera. The system checks for the availability of the keyboard, microphone and video camera and it configures the interview (i.e., questions) for the individual 100 accordingly. To check which devices among the keyboard, microphone, and video camera are available in the computing device 110, the application 120 uses the available APIs in the supported operating systems (OS). Depending on the type of processing elements (keyboard, microphone, video camera, etc.) that are available in the computing device 110, the answering mode is configured for each of the questions that will be part of the interview. The server 130 can receive text, audio and video data items from the computing device 110. If one processing element is missing (for example, the computing device 110 does not have a video camera), a message is sent/displayed to the individual 100 and the individual 100 can decide to continue the interview with the related restraint (i.e., no video recording) or pause and fix the problem to have a more comprehensive evaluation.
In an exemplary embodiment, the server 130 is configured to detect deception of an individual 100, and the server 130 includes at least one memory 220 and at least one processor device 218. In
In an exemplary embodiment, when the first data item is unstructured data, the server 130 is configured to convert the first data item to structured data, extract parts of the unstructured data or identify characteristics of the unstructured data, and analyze the unstructured data of the first data item.
In an exemplary embodiment, when the first data item is the audio recording of the individual 100 providing the answer to the one or more questions, the server 130 is configured to generate a transcript of the audio recording, analyze the transcript for indications of deception, and analyze the audio recording for indications of deception. The server 130 is also configured to compare a deception event at a time in the transcript to a corresponding time in the audio recording to determine the probability of the deception.
In an exemplary embodiment, when the first data item is the video recording of the individual 100 providing the answer to the one or more questions, the server 130 is configured to separate recorded audio corresponding to the video recording from the video recording, generate a transcript of the recorded audio, and analyze the transcript of the recorded audio for indications of deception. The server 130 is also configured to analyze the audio recording for indications of deception, and analyze the video recording for indications of deception. In addition, the server 130 is configured to compare a deception event at a time in the transcript to a corresponding time in the recorded audio and a corresponding time in the video recording to determine the probability of the deception.
In an exemplary embodiment, the server 130 is configured to receive a second data item from the computing device 110 of the individual 100. The second data item represents one or more answers to one or more questions presented to the individual 100 by the computing device 110. For example, the second data item can be an answer to a multiple choice question, an answer to the one or more questions provided by the individual 100 in the form of text, an audio recording of the individual 100 providing an answer to the one or more questions, or a video recording of the individual 100 providing an answer to the one or more questions. The server 130 is also configured to convert the second data item to structured data if the second data item is unstructured data; and determine the probability of deception of the individual 100 based on the structured data from the first data item and the structured data from the second data item.
In an exemplary embodiment, the first data item is a first type of data, and the second data item is a second type of data. In an exemplary embodiment, the first type of data is one of text data, audio data, or video data and the second type of data is one of text data, audio data, or video data, and the first type of data is different than the second type of data. For example, the first data item could be an answer to a multiple choice question and the second data item could be an answer to the one or more questions provided by the individual 100 in the form of text. For example, the first data item could be an answer to the one or more questions provided by the individual 100 in the form of text and the second data item could be an audio recording of the individual 100 providing an answer to the one or more questions. For example, the first data item could be an audio recording of the individual 100 providing an answer to the one or more questions, and the second data item could be a video recording of the individual 100 providing an answer to the one or more questions. Any other combination is possible.
In an exemplary embodiment, the server 130 is configured to compare structured data from the first data item with structured data from the second data item. For example, the server 130 could compare structured data from a first text data item with structured data from a second text data item.
In an exemplary embodiment, the server 130 is configured to receive a third data item from the computing device 110 of the individual 100. The third data item represents one or more answers to one or more questions presented to the individual 100 by the computing device 110. The server 130 is configured to convert the third data item to structured data if the third data item is unstructured data. Also, the server 130 is configured to determine the probability of deception of the individual 100 based on the structured data from the first data item, the structured data from the second data item, and the structured data from the third data item. The third data item can be an answer to a multiple choice question, an answer to the one or more questions provided by the individual 100 in the form of text, an audio recording of the individual 100 providing an answer to the one or more questions, or a video recording of the individual 100 providing an answer to the one or more questions. In an exemplary embodiment, the first data item, the second data item, and the third date item can all be different types of data (e.g., the first data item could be an answer to a multiple choice question, the second data item could be an answer to the one or more questions provided by the individual 100 in the form of text, and the third data item could be an audio recording of the individual 100 providing an answer to the one or more questions). Any combination of three different data items among the four different data types is possible.
In an exemplary embodiment, the server 130 is configured to receive a fourth data item from the computing device 110 of the individual 100. The fourth data item represents one or more answers to one or more questions presented to the individual 100 by the computing device 110. The server 130 is also configured to convert the fourth data item to structured data if the fourth data item is unstructured data. Also, the server 130 is configured to determine the probability of deception of the individual 100 based on the structured data from the first data item, the structured data from the second data item, the structured data from the third data item, and the structured data from the fourth data item. In an exemplary embodiment, the first data item, the second data item, the third date item, and the fourth data item can all be different types of data (e.g., the first data item could be an answer to a multiple choice question, the second data item could be an answer to the one or more questions provided by the individual 100 in the form of text, the third data item could be an audio recording of the individual 100 providing an answer to the one or more questions, and the fourth data item could be a video recording of the individual 100 providing an answer to the one or more questions).
In an exemplary embodiment, the first data item is an answer to a multiple choice question provided by the individual 100, the second data item is an answer to the one or more questions provided by the individual 100 in the form of text, the third data item is an audio recording of the individual 100 providing an answer to the one or more questions, and the fourth data item is a video recording of the individual 100 providing an answer to the one or more questions.
In an exemplary embodiment, the first data item is in a form of a data file (e.g., audio file, video file, etc.) and the second data item is in a form of a data file (e.g., audio file, video file, etc.).
In an exemplary embodiment, the server 130 is configured to determine whether the computing device 100 has a microphone, video camera, and keyboard or touch screen, and based on this determination the server 130 is configured to determine whether a response to a question presented to the individual will be in the form of an answer to a multiple choice question provided by the individual 100, an answer to a question provided by the individual 100 in the form of text, an audio recording of the individual 100 providing an answer to a question, or a video recording of the individual 100 providing an answer to a question.
In an exemplary embodiment, the methodology used to define the rules implemented in the Competency Based Assessment Rules Module 202 consists of three questions for each competency that is being evaluated, and is shown in
When a rule is triggered, its execution is recorded in a file for the Competency Based Assessment Rules Module 202. This file is communicated as an input to the Multi-Layer Deception Model Module 204. See S408 of
In an exemplary embodiment, at step S410 of
In
At step S508, using the Psycholinguistics Module 208, an analysis to extract personality traits like openness, extraversion, and agreeableness is performed. Then, these personality traits are correlated to each of the competencies evaluated using the Competency Based Assessment Rules Module 202. The objective of step S508 is to identify strong correlations or potential deviations between competency scores and the extracted personality traits. Using these inputs, a set of rules are defined that will target and identify deviations in the input data. There are two types of rules in the Psycholinguistics Module 208: direct and indirect relation rules. Using direct relation rules, there is a direct mapping between one of the competencies evaluated by the Competency Based Assessment Rules Module 202 and a personality trait extracted from the analysis of the text elements. For example, competency leadership can have values associated with an introvert or an extrovert, and this is also a personality trait that can be extracted from the text analysis. For indirect relation rules, there is no direct relation, but the trait is an aspect of the competency. For example, an extrovert leader can also show openness as a personality trait. In an exemplary embodiment, the extraction of personality traits from text analysis can be performed using third party services (i.e., an API) such as Watson Personality Insights from IBM. In step S510, calculated values from previous process steps are fed into the Multi-Layer Deception Model Module 204, and combined with the rest of the inputs from all data types and data elements. The Multi-Layer Deception Model Module 204 will correlate the different inputs and run the model to output a final Deception Probability Ranking Matrix 1202 shown in
At step S512 of
The disclosed system uses cross-references in the unstructured data items captured during the individual's interview to increase the deception detection certainty. When analyzing audio data items, cross-referencing is performed by generating an audio transcript (step S606 in
As shown in
There are some steps of analysis that are common to the three different data types (i.e., open-ended text, an audio recording, and a video recording). These steps are shown in
Next, step S1102 is repeated, and features are extracted from the data item according to the data type. Step S1106 includes identifying and analyzing deception cross-references among data items (e.g., identifying deception cues in text and in the co-located time window in the audio file). Step S1108 includes running psycholinguistic analysis on text data elements. Once all relevant features are extracted, these are fed into a machine learning model (e.g. a machine learning model in the Machine Learning Module 216), for example, a random forest or neural networks, etc. These models are trained using a historical dataset and the output is a confidence value on the individual's response, or deception probability. In step S1110, all of these confidence values form the Deception Probability Matrix 1202, an example of which is shown in
An example of the process flow of
Video recording=0.7 (high probability of deception)
Audio recording=0.3 (low probability of deception)
Cross reference=0.5 (medium probability of deception)
In this example, there is a high probability of deception resulting from the isolated analysis of the video recording (0.7), but separating and analyzing the audio from the video results in a low probability (0.3) of deception. Therefore, it can be considered a lower probability of deception (0.5) when the two data items are considered for the same data item.
In an exemplary embodiment, the converting includes analyzing the unstructured data of the first data item and extracting parts of the unstructured data or identifying characteristics of the unstructured data.
In an exemplary embodiment, the probability of deception is a number value that indicates a confidence level of the deception.
In an exemplary embodiment, the first data item is an answer to a multiple choice question, the first data item is an answer to the one or more questions provided by the individual 100 in the form of text, the first data item is an audio recording of the individual 100 providing an answer to the one or more questions, or the first data item is a video recording of the individual 100 providing an answer to the one or more questions.
In an exemplary embodiment, when the first data item is the audio recording of the individual 100 providing the answer to the one or more questions, the method includes: generating a transcript of the audio recording, analyzing the transcript for indications of deception, analyzing the audio recording for indications of deception, and comparing a deception event at a time in the transcript to a corresponding time in the audio recording to determine the probability of the deception.
In an exemplary embodiment, when the first data item is the video recording of the individual 100 providing the answer to the one or more questions, the method includes: separating recorded audio corresponding to the video recording from the video recording, generating a transcript of the recorded audio, analyzing the transcript of the recorded audio for indications of deception, analyzing the audio recording for indications of deception, and analyzing the video recording for indications of deception. The method also includes comparing a deception event at a time in the transcript to a corresponding time in the recorded audio and a corresponding time in the video recording to determine the probability of the deception.
In an exemplary embodiment, the method includes receiving, in the server 130, a second data item from the computing device 110 of the individual 100. The second data item represents one or more answers to one or more questions presented to the individual 100 by the computing device 110. The method also includes converting, by the server 130, the second data item to structured data if the second data item is unstructured data. The determining of the probability of deception of the individual 100 is based on the structured data from the first data item and the structured data from the second data item.
In an exemplary embodiment, the first data item is a first type of data, and the second data item is a second type of data.
In an exemplary embodiment, the first type of data is one of text data, audio data, or video data and the second type of data is one of text data, audio data, or video data, and the first type of data is different than the second type of data.
In an exemplary embodiment, the method includes comparing, by the server 130, structured data from the first data item with structured data from the second data item.
In an exemplary embodiment, the method includes receiving, in the server 130, a third data item from the computing device 110 of the individual 100. The third data item represents one or more answers to one or more questions presented to the individual 100 by the computing device 110. The method also includes converting, by the server 130, the third data item to structured data if the third data item is unstructured data. The determining of the probability of deception of the individual 100 is based on the structured data from the first data item, the structured data from the second data item, and the structured data from the third data item.
In an exemplary embodiment, the method includes receiving, in the server 130, a fourth data item from the computing device 110 of the individual 100. The fourth data item represents one or more answers to one or more questions presented to the individual 100 by the computing device 110. The method also includes converting, by the server 130, the fourth data item to structured data if the fourth data item is unstructured data. The determining of the probability of deception of the individual 100 is based on the structured data from the first data item, the structured data from the second data item, the structured data from the third data item, and the structured data from the fourth data item.
In an exemplary embodiment, the first data item is an answer to a multiple choice question provided by the individual 100, the second data item is an answer to the one or more questions provided by the individual 100 in the form of text, the third data item is an audio recording of the individual 100 providing an answer to the one or more questions, and the fourth data item is a video recording of the individual 100 providing an answer to the one or more questions.
In an exemplary embodiment, the first data item is in a form of a data file and the second data item is in a form of a data file.
In an exemplary embodiment, the server 130 determines whether the computing device 100 has a microphone, video camera, and keyboard or touch screen, and based on this determination the server 130 determines whether a response to a question presented to the individual will be in the form of an answer to a multiple choice question provided by the individual 100, an answer to a question provided by the individual 100 in the form of text, an audio recording of the individual 100 providing an answer to a question, or a video recording of the individual 100 providing an answer to a question.
In an exemplary embodiment, the disclosed system can be used to evaluate the competencies of the individual. For example, to assess the leadership of the individual 100. For example, the individual could be asked to rate their leadership skill, and they could rate themselves as a 5 out of 5, and if there is not detected deception, it can be determined that the individual 100 does indeed have a level of leadership. In an exemplary embodiment, the disclosed system can be used to determine psychological profile of an individual. For example, the individual's 100 answers to specific questions could indicate whether the individual is an introvert, extrovert, etc.
A hardware processor device as discussed herein may be a single hardware processor, a plurality of hardware processors, or combinations thereof. Hardware processor devices may have one or more processor “cores.” The term “non-transitory computer readable medium” as discussed herein is used to generally refer to tangible media such as a memory device 220 and main memory 1404.
Various embodiments of the present disclosure are described in terms of this exemplary computing device 1400. After reading this description, it will become apparent to a person skilled in the relevant art how to implement the present disclosure using other computer systems and/or computer architectures. Although operations may be described as a sequential process, some of the operations may in fact be performed in parallel, concurrently, and/or in a distributed environment, and with program code stored locally or remotely for access by single or multi-processor machines. In addition, in some embodiments the order of operations may be rearranged without departing from the spirit of the disclosed subject matter.
Hardware processor 1402 may be a special purpose or a general purpose processor device. The hardware processor device 1402 may be connected to a communications infrastructure 1410, such as a bus, message queue, network, multi-core message-passing scheme, etc. The network shown in
Data stored in the computing device 1400 (e.g., in the memory 1404) may be stored on any type of suitable computer readable media, such as optical storage (e.g., a compact disc, digital versatile disc, Blu-ray disc, etc.), magnetic tape storage (e.g., a hard disk drive), or solid-state drive. An operating system can be stored in the memory 1404.
In an exemplary embodiment, the data may be configured in any type of suitable database configuration, such as a relational database, a structured query language (SQL) database, a distributed database, an object database, etc. Suitable configurations and storage types will be apparent to persons having skill in the relevant art.
The computing device 1400 may also include a communications interface 1412. The communications interface 1412 may be configured to allow software and data to be transferred between the computing device 1400 and external devices. Exemplary communications interfaces 1412 may include a modem, a network interface (e.g., an Ethernet card), a communications port, a PCMCIA slot and card, etc. Software and data transferred via the communications interface 1412 may be in the form of signals, which may be electronic, electromagnetic, optical, or other signals as will be apparent to persons having skill in the relevant art. The signals may travel via a communications path 1414, which may be configured to carry the signals and may be implemented using wire, cable, fiber optics, a phone line, a cellular phone link, a radio frequency link, etc.
Memory semiconductors (e.g., DRAMs, etc.) may be means for providing software to the computing device 1400. Computer programs (e.g., computer control logic) may be stored in the memory 1404. Computer programs may also be received via the communications interface 1412. Such computer programs, when executed, may enable computing device 1400 to implement the present methods as discussed herein. In particular, the computer programs stored on a non-transitory computer-readable medium, when executed, may enable hardware processor device 1402 to implement the methods illustrated by
The computing device 1400 may also include a display interface 1406 that outputs display signals to a display unit 1408, e.g., LCD screen, plasma screen, LED screen, DLP screen, CRT screen, etc.
Where the present disclosure is implemented using software, the software may be stored in a computer program product or non-transitory computer readable medium and loaded into one or more of the computing device 100 and the server 130 using a removable storage drive or a communications interface.
Thus, it will be appreciated by those skilled in the art that the disclosed systems and methods can be embodied in other specific forms without departing from the spirit or essential characteristics thereof. The presently disclosed embodiments are therefore considered in all respects to be illustrative and not restricted. It is not exhaustive and does not limit the disclosure to the precise form disclosed. Modifications and variations are possible in light of the above teachings or may be acquired from practicing of the disclosure, without departing from the breadth or scope. Reference to an element in the singular is not intended to mean “one and only one” unless explicitly so stated, but rather “one or more.” Moreover, where a phrase similar to “at least one of A, B, or C” is used in the claims, it is intended that the phrase be interpreted to mean that A alone may be present in an embodiment, B alone may be present in an embodiment, C alone may be present in an embodiment, or that any combination of the elements A, B and C may be present in a single embodiment; for example, A and B, A and C, B and C, or A and B and C.
No claim element herein is to be construed under the provisions of 35 U.S.C. 112(f) unless the element is expressly recited using the phrase “means for.” As used herein, the terms “comprises,” “comprising,” or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. The scope of the invention is indicated by the appended claims rather than the foregoing description and all changes that come within the meaning and range and equivalence thereof are intended to be embraced therein.
