Patent Application
20220108714
The present disclosure relates to automated detection of speech-based cognitive impairment symptoms, and in particular detection of Alzheimer's Disease (AD) from speech samples.
Research related to the automatic detection of Alzheimer's Disease (AD) is important, given the high prevalence of AD and the high cost of traditional diagnostic methods. Since AD significantly affects the content and acoustics of spontaneous speech, natural language processing (NLP) and machine learning (ML) provide promising techniques for reliably detecting AD. There has been a recent proliferation of classification models for AD, but these vary in the datasets used, model types and training and testing paradigms.
Generally, NLP and ML approaches to AD detection from speech are based on hand-crafted engineering of several hundred clinically-relevant features including certain acoustic features and linguistic features from speech transcripts. While the use engineered features makes a ML model easier to interpret, the feature engineering process is time and resource-consuming since it does require domain knowledge, may be subject to biases in data, and may result in highly relevant features being overlooked.
In drawings which illustrate by way of example only embodiments of the present application,
Alzheimer's Disease (AD) is a progressive neurodegenerative disease that causes problems with memory, thinking, and behavior. Currently, AD affects over 40 million people worldwide with high costs of acute and long-term care. Conventional forms of diagnosis are both time consuming and expensive; as a result, many people living with AD do not receive a timely diagnosis. Based on the clinical observation that information indicative of cognition can be obtained from spontaneous speech elicited using pictures, speech analysis, natural language processing (NLP), and machine learning (ML) have been to distinguish between speech from healthy and cognitively impaired participants in datasets including semi-structured speech tasks such as picture description. Such methods may serve as quick, objective, and non-invasive assessments of an individual's cognitive status which could be developed into more accessible tools to facilitate clinical screening and diagnosis.
Generally, NLP and ML approaches to AD detection from speech are based on hand-crafted engineering of several hundred clinically-relevant features including certain acoustic features (such as zero-crossing rate, Mel-frequency cepstral coefficients etc.) and linguistic features (such as proportions of various parts-of-speech (POS) tags) from speech transcripts. Engineered features have the advantage of producing more easily interpretable ML model decisions, since the selection of features is informed by domain knowledge; it was also believed to be advantageous to include representations of speech in different modalities (both acoustic and linguistic). Additionally, feature engineering may have potentially lower computational resource requirements when used with conventional ML models. However, a feature engineering process may in fact be prone to biases in data, and despite the use of domain knowledge, bears a risk of missing highly relevant features. Further, the feature engineering process can be expensive and time-consuming because it requires clinical expertise. Successful models generally involve extraction of several hundreds of features. For example, Fraser et al. (2016) extracted 370 linguistic and acoustic features from picture descriptions in the DementiaBank dataset; Balagopalan et al. (2018) augmented DementiaBank data with multi-task healthy data to improve accuracy of their ML model, employing 480 linguistic and acoustic features. DementiaBank (Becker et al., 1994) is a longitudinal dataset of speech commonly used for assessing cognitive impairment containing 473 narrative picture descriptions, where each participant describes a picture shown to them.
Transfer learning, or in other words, utilizing language representations from huge pre-trained neural models that learn robust representations for text, has become ubiquitous in NLP. A popular transfer learning model is BERT (Devlin et al., 2019), which trains “contextual embeddings” wherein a representation of a sentence (or transcript) is influenced by the context in which the words occur in sentences. This model, including its derivatives (e.g., ALBERT, RoBERTa, DistilBERT) (Lan et al., 2019; Liu, Ott et al., 2019; Sanh et al., 2019) offers enhanced parallelization and better modeling of long-range dependencies in text and as such, has achieved state-of-the-art performance on a variety of tasks in NLP. BERT uses powerful attention mechanisms to encode global dependencies between the input and output. Fine-tuning BERT for a few epochs can potentially attain good performance even on small datasets. However, unlike conventional ML models employed for AD detection, BERT takes embeddings as input rather than engineered features, and thus may be felt not to be conducive to an AD or similar impaired speech detection task. Critiques of BERT in this regard include the lack of direct interpretability and the fact that it is pre-trained on a corpus of healthy language. Further, the original version of the BERT model could take only text input, and thus could not use the acoustic modality of speech, which was generally believed important for detecting AD. Thus, BERT and similar transfer learning models may not have been previously used for developing predictive models for AD detection despite its success in NLP with healthy speech. As set out below, it was surprisingly discovered that use of a BERT model fine-tuned for the AD detection test using a relatively small AD training dataset could deliver at least as good or numerically better (more accurate) results compared to conventional ML models used for AD detection, without the need for domain knowledge or feature extraction, and using linguistic information only—without the use of acoustic features. It was also discovered that a noticeable improvement in the accuracy of a fine-tuned BERT model could be achieved with the addition of a limited set of engineered features, significantly smaller than those sets mentioned above, and again without the use of acoustic features. Eliminating feature engineering steps in whole or in part, and eliminating the extraction of acoustic features, may reduce the computational resources required to pre-process speech data as input to an AD detection ML model.
The examples below include experiments supporting the above conclusions, employing different training/test datasets. Existing studies that have addressed differences between AD and non-AD speech and worked on developing speech-based AD biomarkers are often descriptive rather than predictive. Thus, they often overlook common biases in evaluations of AD detection methods, such as repeated occurrences of speech from the same participant, variations in audio quality of speech samples, and imbalances of gender and age distribution in the used datasets. Thus, existing ML models may be prone to the biases introduced in available data. In addition, the performance of the previously developed predictive AD-detection models has been evaluated using either random train/test split or a cross-validation technique, which may result in artificially increased reported performance of ML models (i.e., overfitting) as compared to their evaluation on a held out unseen dataset, especially when it comes to smaller and unbalanced datasets. Accordingly, the systems and methods described below include models trained or fine-tuned on a new common dataset that was introduced to better compare model performance, the ADReSS Challenge (Luz et al., 2020). The ADReSS dataset comprises a demographically (age and gender) balanced speech dataset of speech from AD and non-AD participants describing a picture.
A BERT text sequence classification model was evaluated using the dataset for the AD Recognition through Spontaneous Speech (ADReSS) Challenge (Luz et al., 2020). The ADReSS dataset consists of 156 speech recordings and associated transcripts from non-AD (N=78) and AD (N=78) English-speaking participants. Speech was elicited from participants through the Cookie Theft picture from the Boston Diagnostic Aphasia exam (Goodglass et al., 2001). Speech transcripts were manually transcribed and annotated per the CHAT protocol (MacWhinney, 2000), and included speech segments from both the participant and an investigator.
In contrast to other speech datasets for AD detection such as DementiaBank's English Pitt Corpus (Becker et al., 1994), the ADReSS challenge dataset was matched for age and gender to minimize risk of bias in the prediction tasks. Characteristics of the patients in each group are set out in Tables 1-3 below. Contrasted with DementiaBank, the ADReSS dataset is significantly smaller (156 recordings/transcripts compared to 551) but demographically balanced.
Recordings were acoustically enhanced by the challenge organizers with stationary noise removal and audio volume normalization was applied across all speech segments to control for variation caused by recording conditions such as microphone placement. The speech dataset was divided into a training set and an unseen held out test set.
The ADReSS Challenge included a baseline linguistic model. For additional comparison with BERT, classifier models were trained on manually-engineered features identified using domain knowledge. Only the portion of transcripts corresponding to the participant were used, and all participant speech segments corresponding to a single picture description were combined for extracting acoustic features.
A total of 509 engineered features were extracted from the transcripts and associated audio files. These features are summarized in Tables 4-6 below (the column # Features indicates the number of features extracted for each feature type). These features were identified as indicators of cognitive impairment in previous literature, and thus encode domain knowledge. Briefly, these features may be divided into three higher-level categories:
The lexico-syntactic, acoustic, and semantic features were extracted at transcript level and classified using four conventional machine learning models: support vector machine (SVM), neural network (NN), random forest (RF), and naïve Bayes (NB) (see scikit-learn.org/stable/). All hyperparameters were tuned to the best possible setting by searching within a grid of possible parameter values using 10-fold cross validation on the ADReSS challenge “train” set.
The SVM was trained with a radial basis function kernel with kernel coefficient (γ) 0.001, and regularization parameter set to 100.
The NN consisted of two layers of 10 units each (both the number of units and number of layers were varied while tuning for the optimal hyperparameter setting). The ReLU activation function was used at each hidden layer. The model was trained using the Adam optimization algorithm (Kingma and Ba, 2014) for 200 epochs and with a batch size of number of samples in train set in each fold. All other parameters were set to the default value.
The RF classifier fit 200 decision trees and considered √{square root over (features)} when looking for the best split. The minimum number of samples required to split an internal node was 2, and the minimum number of samples required to be at a leaf node was 2. Bootstrap samples were used when building trees. All other parameters were set to the default value.
The Gaussian Naive Bayes classifier was fit with balanced priors and variance smoothing coefficient set to 1 e-10, and all other parameters set to default.
Feature selection was performed by choosing the top-k number of features, based on ANOVA F-value between label/features. The number of features was jointly optimized with the classification model parameters.
A combined text sequence classification model (i.e., base BERT model with classification layer) was implemented using the open-source PyTorch library (pytorch.org). In order to leverage the language information encoded by BERT (Devlin et al., 2019), the pre-trained BERT model weights were used to initialize the classification model. All experiments are based on the bert-base-uncased variant (Devlin et al., 2019), which consists of 12 layers, each having a hidden size of 768 and 12 attention heads, with a maximum input length of 512 tokens. Linear scheduling was used for the learning rate, which was initially set to 2e-5, and the Adam optimization algorithm was used. Fine-tuning for AD detection employed cross-entropy loss.
While the base BERT model was pre-trained with (healthy) sentence pairs, the input used to fine-tune the model for performing AD detection consisted of speech transcripts from the ADReSS train set. Each speech transcript comprised several transcribed utterances, which were tokenized and delimited with start and separator special tokens from the BERT vocabulary at the beginning and end of each utterance, respectively (i.e., [CLS] and [SEP]), following Liu and Lapata (2019). This ensured that utterance boundaries were encoded, since cross-utterance information such as coherence and utterance transitions is considered important for reliable AD detection.
Following Devlin et al. (2019), an aggregate transcript representation was extracted from the base BERT model for each transcript. This aggregate representation was the final hidden state corresponding to the first start ([CLS]) token in the transcript, which is an embedding pooling information across all tokenized units in the transcript. This final hidden state summarized information across all tokens in the transcript using BERT's self-attention mechanism. This embedding was passed to the classification layer (Devlin et al., 2019; Wolf et al., 2019).
In tuning the hyperparameters, the number of epochs was optimized at 10 by varying the number from 1 to 12 during cross validation. As noted above, Adam optimization and linear scheduling for the learning rate were used. The learning rate and other parameters were set based on prior work on fine-tuning BERT (Devlin et al., 2019; Wolf et al., 2019).
Two cross-validation (CV) strategies were used: leave-one-subject-out CV (LOSO CV) and 10-fold CV at transcript level. Evaluation metrics with LOSO CV were determined for all models except fine-tuned BERT for direct comparison to ADReSS Challenge baselines. No LOSO CV was performed for fine-tuned BERT due to computational constraints; instead, 10-fold CV to compare the feature-based classification models with fine-tuned BERT. Values of performance metrics for each model were averaged across three runs with different random seeds in all cases.
Three predictions were generated with different seeds from each hyperparameter-optimized classifier trained on the complete ADReSS train set, then a majority prediction was produced to avoid overfitting. Task performance was evaluated primarily using accuracy scores, since the ADReSS train and test sets were known to be demographically balanced. Table 7 sets out the classification performance of all models evaluated on the ADReSS train set via 10-fold CV. Precision, recall, specificity, and F1 are also included with respect to the positive class (AD).
As can be seen from Table 7, BERT numerically outperformed all domain knowledge-based machine learning models with respect to all metrics, with an average accuracy of 81.8%. SVM was found to be the best-performing domain knowledge-based model. However, accuracy of the fine-tuned BERT model was not significantly higher than that of the SVM classifier based on an Kruskal-Wallis H-test (H=0.4838, p>0.05). The Kruskal-Wallis H-test was employed here and with the performance-comparisons discussed below, since it was observed that accuracy was not normally distributed on varying the random seed during training or inference.
The performance results on the unseen, held out ADReSS challenge test set are set out in Table 8 below:
As can be seen in Table 8, the results follow the trend of the cross-validated performance in Table 7 in terms of accuracy, with the fine-tuned BERT model outperforming the best feature-based classification model SVM with an accuracy of 83.33%, but not significantly so (H=2.4, p>0.05). The accuracy with the BERT-based classification model described above ranged between 85.14 and 81.25%.
These results demonstrate that a fine-tuned BERT model may perform the same, or numerically better, than classifier models conventionally selected for AD and similar detection tasks, which are reliant on engineered features defined by domain knowledge. Further, the feature-based and BERT-based classification models described above performed significantly better than the linguistic baseline provided in the ADReSS challenge, showing the importance of linguistic features for detecting AD-related differences.
A feature differentiation analysis was performed to identify the most differentiating features between AD and non-AD speech in the ADReSS training set. In order to study statistically significant differences in linguistic/acoustic phenomena, independent t-tests were performed between feature means for each class in the ADReSS training set, following the methodology followed by Balagopalan et al. (2020). Eighty-seven features were found to be significantly different between the two groups at p<0.05. Seventy-nine of these were text-based lexico-syntactic and semantic features, while eight were acoustic (including temporal). These eight acoustic features included the number of long pauses, pause duration, and mean/skewness/variance-statistics of various MFCC coefficients. However, after Bonferroni correction for multiple testing, it was found that 13 features were significantly different between AD and non-AD speech at p<9e-5, and none of these features were acoustic (including temporal features). In Table 9 below, μAD and μnon-AD show the means of the 13 significantly different features at p<9e-5 after Bonferroni correction for the AD and non-AD group, respectively.
The features that differentiate the AD and non-AD groups largely indicate semantic impairments in AD, reflected in the types of words used and the content of their picture descriptions. Many of the differentiating features reflect findings in prior literature suggesting that despite the ADReSS dataset being more demographically balanced, many of the previous findings are maintained. In addition, the differentiating features are consistent with other previous clinical literature documenting decreased specificity and information content in AD. For example, the features relating to the content units in the picture and the cosine similarity between utterances and picture content units show that the picture descriptions produced in AD have fewer relevant content words and that the words used are less semantically related to the themes of the picture. Lower average cosine distance in AD signifies more repetition in speech. These findings are also consistent with previous studies reporting reduced information content and coherence in AD. Other differentiating features related to the use of shorter words, and increased use of pronouns, adverbs, and words not found in the dictionary. These features may all reflect the use of less specific and simpler language, and reflecting previous findings of decreased specificity of language in AD. However, while Fraser et al. (2016) found differences in acoustic features, none of those findings survived Bonferroni correction can be seen above. Without wishing to be bound by theory, these findings may indicate that a demographically balanced dataset reduces the acoustic differences between groups. Further, without wishing to be bound by theory, the finding that linguistic (semantic and lexico-syntactic) features are particularly differentiating between AD and non-AD classes may explain why the BERT-based classification model, trained and fine-tuned only on linguistic features, attained performance well above random chance.
Additionally, multi-scale attention visualizations of the BERT classification model fine-tuned for AD detection as described above were produced using the BertViz library (Vig, 2019), since attention patterns may assist in interpreting model decisions, given that self-attention is an important component of BERT-based models. Attention weights for the first [CLS] token were visualized for both AD and healthy speech transcripts (the visualization may be found in Balagopalan et al., 2021, the entirety of which is incorporated herein by reference). It was found that attention weights were often attributed to a few important information content units such as such as “water,” “boy,” etc., which have been identified to be important speech indicators of AD in prior work (Fraser et al., 2016). Sometimes, attention weights were attributed to pauses and fillers, such as “uh” and “um”, and to the sentence separator tokens. Without wishing to be bound by theory, this may represent counting the number of utterances in the transcript.
The possibility of augmenting a BERT-based model was studied by executing probing tasks on embedded representations extracted from different BERT layers.
The same initial BERT-based classification model was also fine-tuned for the task of AD detection at utterance level using the DementiaBank dataset. For fine-tuning the BERT model, each transcript was divided into individual utterances. Each utterance was treated as a sample, similar to the methodology followed by Karlekar et al. (2018). However, only data from the picture description task was used for the intended classification task, because other speech tasks were exclusively performed by participants with AD. Each sentence spoken by the participant was associated with the diagnosis label of the participant, thus increasing the sample size to 5103 utterances, which were split at about 82%/9%/9% for train/validation/test respectively. The utterances were bounded by a start [CLS] and end [SEP] token, as before. Table 10 below sets out the split details.
The representations of the first classification ([CLS]) token from each layer of the fine-tuned BERT classification model were probed (Jawahar et al., 2019). Multi-Layer Perceptrons (MLPs) were trained using these embedded representations as input to predict the following five properties:
These properties were selected since it had been found that features similar to these properties were important for AD detection from picture descriptions (Fraser et al., 2016; Yancheva et al., 2015). For example, variations in proportions of various production rules from the constituency parse representation, which are features of syntactic type, were found to be an important characteristic of impaired speech, and presence of informative content words or units such as “cookie” or “boy” while describing the picture has been mentioned as an important characteristic. Such informative content words are identified by clinicians, and are not arbitrarily defined.
Gridsearch hyperparameter optimization was performed to arrive at the optimal parameter settings using the validation set. It was observed that performance on the syntactic task of tree-depth prediction was low, as was the performance on the word content prediction task. These probing results are set out in Table 11 below, with boldface indicating the worst performance in each feature type.
WordContent
22.47
4
Surface
TreeDepth
36.14
6
Syntactic
Based on the above results, it was identified that engineered features capturing presence of informative content words in utterances, and syntactic tree depths may improve AD detection performance in combination with BERT. Thus, 119 features were extracted at utterance level: 117 syntactic and two word-content based features.
The two word-content features extracted were:
The 117 syntactic features included depth-related features of the constituency parse representations, the height of the constituency parse-tree, proportion of verb-phrases, and proportion of production rules of type “adjective phrase followed by adjective”.
Several input settings were compared to see the effect of these features:
It was found that fine-tuned BERT models were able to perform well above chance for the utterance-level AD detection task. It was also observed that the augmented BERT model—a combination of the BERT classification model with the selected engineered features—attained the highest accuracy, about 13% accuracy points higher than using classification models using features alone, and about 5% higher than fine-tuned BERT alone. These results are set out in Table 12 below, where FS1 denotes the feature set discussed above, and boldface indicates the highest performance.
0.64
0.76
0.86
While the engineered features were selected using probing tasks, the identification of additional features may be replaced by other augmentation methods, such as generating explanations (Mothilal et al., 2020).
Individual user systems 100 may be computer systems in clinical or consultation settings, communicating with the remote service 130 over a wide area network (e.g., the Internet). It is contemplated that users may implement clinic or patient management software locally, and that preferably, personally identifying information (PII) of a patient—which can include recorded utterances—is stored securely, and even locally, to avoid accidental dissemination of PII by third-party services. This is reflected in the example data processing system of
The clinical system 110 in this example is configured to receive the recorded speech (1), convert the recorded speech to text using a suitable speech recognition module 112 as would be known to those skilled in the art, and to provide the recognized speech (2) to a pre-processing module 114. The pre-processing functions executed by the module 114 may comprise speech recognition and/or feature extraction to identify linguistic or acoustic features, if feature extraction is required to generate a feature set as input to a ML classification system. The pre-processing functions may alternatively or additionally comprise speech recognition and/or tokenization or other preparation of the recognized speech data. The audio data corresponding to the recorded speech may also be pre-processed, for example to remove noise. Alternatively, in some implementations, a transcript of the subject's speech may be produced manually, and the featurization may be carried out manually as well. This may occur locally (e.g., in the clinical setting). The results of the pre-processing are transmitted (3) to the remote analysis service 130. Alternatively, no pre-processing is carried out, and instead the recorded speech 10 is transmitted (3) to the remote analysis service 130. Any data transmitted to the remote service 130 may be completely or partially anonymized, for example identified only using a patient identification number.
The remote analysis service 130 may implement both training (which, in the context of a BERT-based classification model, may comprise primarily or only fine-tuning and hyperparameter optimization since the initial parameters of a pre-trained BERT model may be employed) and classification functions employing module 200, which executes the model. It will be appreciated, however, that that actual training and/or fine-tuning of the classification model may be carried out outside the illustrated data processing system, with the resultant model packaged and imported into the remote analysis service 130 for execution by a classification module 210, e.g., a server providing a REST (REpresentational State Transfer) web service). In a classification task, the remote service 130 receives the speech input 10 or the pre-processed speech input data; performs any pre-processing that may still be required; then applies the resultant data as input to the module 200 or 210 to produce a resultant classification output, which may then be transmitted (4) over the network to the user system 100.
In the AD task without augmentation, an aggregate sequence representation 230 for each input transcript is obtained, and would be provided as input directly to the classification layer 250 (not indicated in
Thus, there is provided a computer-implemented method, system and computer-readable medium configured for detecting speech impairment indicative of Alzheimer's Disease (AD) from a speech sample, by providing input transcribed speech to a classification model, the input comprising at least one utterance, the classification model comprising a fine-tuned, pre-trained Bidirectional Encoder Representations from Transformer (BERT)-based model and a classification layer, the fine-tuning using training data comprising transcribed speech, a portion of the training data identified as associated with AD and a portion of the training data identified as not associated with AD; and obtaining a classification of the transcribed speech input as either associated with AD or not associated with AD.
In one aspect, the BERT-based model is pre-trained on healthy speech sentence pairs. The BERT-based model may be an original BERT model or a variant.
In another aspect, input does not comprise acoustic or temporal features. Such features may comprise information about pauses and fillers, fundamental frequency, duration of audio, duration of spoken segment of audio, zero-crossing rate statistics, and mel-frequency cepstral coefficient statistics.
In a further aspect, the input comprises a plurality of utterances.
The training data may be demographically balanced, and may be significantly smaller than a training data set used to pre-train the BERT-based model. The training data may comprise a set of transcribed speeches each comprising at least one utterance bounded by a start token and an end token. The training data may comprise sets comprising a plurality of utterances.
The utterances of the input and training data may be delimited or bounded by a start token, which may be a [CLS] token, and an end token, which may be a [SEP] token. Each utterance is tokenized.
In an aspect, speech not associated with AD is healthy speech.
In another aspect, obtaining the classification comprises: obtaining an aggregate transcript representation of the input from the BERT-based model; and providing the aggregate transcript representation to the classification layer to obtain the classification.
In a further aspect, each utterance is bounded by a start token and an end token as described above, and the aggregate transcript representation is a final hidden state corresponding to the first start token of the input.
The classification layer may be a linear layer or a non-linear layer. It may be a dense layer, being fully-connected with its respective preceding (input) and following (output) layers.
In still another aspect, a feature vector is obtained for the input comprising syntactic features and word content features, wherein obtaining the classification comprises providing a concatenation of an aggregate transcript representation from the BERT-based model and the feature vector to the classification layer to obtain the classification.
The feature vector may comprise utterance-level syntactic features and utterance-level word-content features. The syntactic features may comprise information about syntactic tree depths associated with the input. These features may be depth-related features of constituency parse representations, height of a constituency parse-tree, proportion of verb-phrases, and proportion of production rules of type “adjective phrase followed by adjective”. The word-content features may comprise a value or flag, such as a Boolean, indicating presence of informative content units and a total number of informative content units in each utterance. The informative content units may be clinically defined.
As set out above, the present disclosure also provides use of a computer-implemented classification module comprising a fine-tuned BERT-based model and a classification layer for detecting speech impairment indicative of AD from input transcribed speech.
In one aspect, the classification module is configured to obtain a feature vector for the input comprising syntactic features and word content features which is concatenated with an aggregate transcript representation from the BERT-based model and provided to the classification layer to obtain a classification of the input as indicative of AD or not indicative of AD. The BERT-based model, input, utterances, classification layer and feature vector may be as described above. The pre-training and fine-tuning of the BERT-based model may be as described above.
The examples and embodiments are presented only by way of example and are not meant to limit the scope of the subject matter described herein. Individual features of each example or embodiment presented above may be combined, in whole or in part, with individual features of other examples or embodiments. Some steps or acts in a process or method may be reordered or omitted, and features and aspects described in respect of one embodiment may be incorporated into other described embodiments. Variations of these examples will be apparent to those in the art and are considered to be within the scope of the subject matter described herein.
The data employed by the systems, devices, and methods described herein may be stored in one or more data stores. The data stores can be of many different types of storage devices and programming constructs, including but not limited to RAM, ROM, flash memory, programming data structures, programming variables, and so forth. Code adapted to provide the systems and methods described above may be provided on many different types of computer-readable media including but not limited to computer storage mechanisms (e.g., CD-ROM, diskette, RAM, flash memory, computer hard drive, etc.) that contain instructions for use in execution by one or more processors to perform the operations described herein. The media on which the code may be provided is generally considered to be non-transitory or physical.
Computer components, software modules, engines, functions, and data structures may be connected directly or indirectly to each other in order to allow the flow of data needed for their operations. The data processing and computer systems described above may be provided in a single location, or may be distributed in a cloud environment, using techniques known to those skilled in the art. Those skilled in the art will know how to implement the particular examples described above in the systems and methods of
Use of any particular term should not be construed as limiting the scope or requiring experimentation to implement the claimed subject matter or embodiments described herein. Any suggestion of substitutability of the data processing or computer systems or environments for other implementation means should not be construed as an admission that the invention(s) described herein are abstract, or that the data processing systems or their components are non-essential to the invention(s) described herein.
A portion of the disclosure of this patent document contains material which is or may be subject to one or more of copyright, design, or trade dress protection, whether registered or unregistered. The rightsholder has no objection to the reproduction of any such material as portrayed herein through facsimile reproduction of this disclosure as it appears in the Patent Office records, but otherwise reserves all rights whatsoever.
This application claims priority to U.S. Provisional Application No. 63/086,778 filed Oct. 2, 2020, and to U.S. Provisional Application No. 63/120,093 filed Dec. 1, 2020, the entireties of which are incorporated herein by reference.
| Number | Date | Country | |
|---|---|---|---|
| 63086778 | Oct 2020 | US | |
| 63120093 | Dec 2020 | US |
