This invention relates in general to the use of brain health programs utilizing brain plasticity to enhance human performance and correct neurological disorders, and more specifically, to a method for assessing participant thresholds for respective exercises.
Almost every individual has a measurable deterioration of cognitive abilities as he or she ages. The experience of this decline may begin with occasional lapses in memory in one's thirties, such as increasing difficulty in remembering names and faces, and often progresses to more frequent lapses as one ages in which there is passing difficulty recalling the names of objects, or remembering a sequence of instructions to follow directions from one place to another. Typically, such decline accelerates in one's fifties and over subsequent decades, such that these lapses become noticeably more frequent. This is commonly dismissed as simply “a senior moment” or “getting older.” In reality, this decline is to be expected and is predictable. It is often clinically referred to as “age-related cognitive decline,” or “age-associated memory impairment.” While often viewed (especially against more serious illnesses) as benign, such predictable age-related cognitive decline can severely alter quality of life by making daily tasks (e.g., driving a car, remembering the names of old friends) difficult.
In many older adults, age-related cognitive decline leads to a more severe condition now known as Mild Cognitive Impairment (MCI), in which sufferers show specific sharp declines in cognitive function relative to their historical lifetime abilities while not meeting the formal clinical criteria for dementia. MCI is now recognized to be a likely prodromal condition to Alzheimer's Disease (AD) which represents the final collapse of cognitive abilities in an older adult. The development of novel therapies to prevent the onset of this devastating neurological disorder is a key goal for modern medical science.
The majority of the experimental efforts directed toward developing new strategies for ameliorating the cognitive and memory impacts of aging have focused on blocking and possibly reversing the pathological processes associated with the physical deterioration of the brain. However, the positive benefits provided by available therapeutic approaches (most notably, the cholinesterase inhibitors) have been modest to date in AD, and are not approved for earlier stages of memory and cognitive loss such as age-related cognitive decline and MCI.
Cognitive training is another potentially potent therapeutic approach to the problems of age-related cognitive decline, MCI, and AD. This approach typically employs computer- or clinician-guided training to teach subjects cognitive strategies to mitigate their memory loss. Although moderate gains in memory and cognitive abilities have been recorded with cognitive training, the general applicability of this approach has been significantly limited by two factors: 1) Lack of Generalization; and 2) Lack of enduring effect.
Lack of Generalization: Training benefits typically do not generalize beyond the trained skills to other types of cognitive tasks or to other “real-world” behavioral abilities. As a result, effecting significant changes in overall cognitive status would require exhaustive training of all relevant abilities, which is typically infeasible given time constraints on training.
Lack of Enduring Effect: Training benefits generally do not endure for significant periods of time following the end of training. As a result, cognitive training has appeared infeasible given the time available for training sessions, particularly from people who suffer only early cognitive impairments and may still be quite busy with daily activities.
As a result of overall moderate efficacy, lack of generalization, and lack of enduring effect, no cognitive training strategies are broadly applied to the problems of age-related cognitive decline, and to date they have had negligible commercial impacts. The applicants believe that a significantly innovative type of training can be developed that will surmount these challenges and lead to fundamental improvements in the treatment of age-related cognitive decline. This innovation is based on a deep understanding of the science of “brain plasticity” that has emerged from basic research in neuroscience over the past twenty years which only now through the application of computer technology can be brought out of the laboratory and into the everyday therapeutic treatment.
Some cognition improvement exercises, such as embodiments of the Tell Us Apart exercise in the HiFi program described herein, are designed to force participants to identify rapid spectro-temporal patterns (brief synthesized formant transitions) in order to classify consonants by place of articulation under conditions of backward masking from a following vowel. The spectral characteristics of these syllables (as dictated by formant frequencies) closely parallel the patterns that occur in natural productions of the sounds, and they can usually be identified as the speech sounds they are intended to represent. However, since formant frequencies constitute only a (comparatively informative) subset of the range of acoustic cues that accompany human productions of the consonants, sounds synthesized in this way do not closely resemble natural speech in a general sense.
As a result, many participants may be unable to match these synthesized sounds, presented in isolation, with the intended syllables based on their previous linguistic experience, and are therefore unable to progress through the easiest levels of the exercise, which almost certainly involve sound distinctions that are well above their actual thresholds for detection.
More generally, in exercises that use synthesized speech to target specific neurological deficits, it is desired that the effectiveness of a task not be severely limited by the overall naturalness of the speech stimuli, since it is often necessary to reduce the acoustic cues available to the listener to a small, carefully controlled set. Thus, a way is needed to help listeners attend to the set of cues relevant to a synthetic speech distinction so that they can reliably identify sounds and progress through the exercise.
Therefore, what is needed is an overall training program that will significantly improve fundamental aspects of brain performance and function relevant to the remediation of the neurological origins and consequences of age-related cognitive decline. Additionally, improved means for helping listeners attend to the set of cues relevant to a synthetic speech distinction to reliably identify sounds and progress through exercises that utilize such distinctions.
Each of the exercises described generally utilizes one or more types of audial stimuli with characteristic attributes that the participant (i.e., the aging adult) must perceive and respond to in performing trials. Each participant generally has a threshold with respect to each type of stimuli such that when stimuli are presented with intensities below this threshold, the participant is unlikely to exceed some specified performance level, e.g., is likely to answer incorrectly some specified percentage of the trials, e.g., 69%.
However, prior art embodiments of such cognition enhancement exercises do not facilitate the determination of this threshold for participants. Thus, systems and methods for assessing participant thresholds for cognition enhancement exercises are desired.
Various embodiments of a method for determining an aging adult's ability to classify formant transition and segment duration information in making phonetic categorizations, such as in the Tell Us Apart exercise described herein.
Since (1) the range of implementations of these contrasts in speech is not readily described with respect to a single dimension along which a detection threshold can be measured, and (2) their realization in the Tell Us Apart exercise emphasis levels does not relate to detection difficulty in a monotonic manner that is sufficiently uniform across participants and contrasts, a participant's success in the Tell Us Apart exercise can currently best be estimated by measuring overall performance across a range of stimuli that is representative of the contrasts and difficulty included in the Tell Us Apart exercise. In other words, an assessment of the aging adult's success in the Tell Us Apart exercise may be estimated by performing a modified version of the exercise (e.g., without visual or auditory feedback) with a restricted but representative data set. More specifically, a single percent correct identification score for a pre-selected (restricted but representative) stimulus set may be determined.
One embodiment of a method for estimating or predicting an aging adult's success with respect to a cognitive enhancement exercise, such as the Tell Us Apart exercise described herein, utilizing a computing device to present aural presentations to the adult, and to record responses from the adult, is presented. Note that in various embodiments, some of the method elements may be performed concurrently, in a different order than shown, or may be omitted. Additional method elements may also be performed. As shown, the method may operate as follows:
A plurality of confusable pairs of phonemes may be provided, each of the phonemes having a consonant portion and a vowel portion. A plurality of stimulus levels, which may include emphasis levels, for computer processing of the plurality of confusable pairs of phonemes may be provided. A stimulus level may specify various timing and/or tonal aspects of a synthesized or computer processed phoneme (or word containing a phoneme) to make discriminating between the phonemes more or less difficult. For example, the plurality of stimulus levels may include stimulus levels that stretch the consonant portion of the phonemes, that vary the relative loudness of the consonant and vowel portions of the phonemes, and/or that vary the gap between the consonant and vowel portions of the phonemes, among others.
A representative subset of the plurality of confusable pairs of phonemes may be selected for presentation to the aging adult. In other words, a portion of the plurality of confusable pairs of phonemes may be selected that broadly or substantially covers or represents the range of phonetic attributes of the plurality of confusable pairs of phonemes. For example, the confusable pairs of phonemes may be selected to represent a spectrum of articulation points, including, for example, back of throat, tongue and pallet, and lip generated consonants. As another example, some of the confusable pairs of phonemes may be selected to represent a frequency spectrum of vowels. In one embodiment, at least one of the representative subset of the plurality of stimulus levels may assist the aging adult in discriminating between the consonant and vowel portion of the one of the phonemes being aurally presented. For example, at least one of the plurality of stimulus levels may emphasize and stretch both the consonant and vowel portions of the one of the phonemes.
A representative subset of the plurality of stimulus levels may be selected for use with the selected representative subset of confusable pairs of phonemes. Said another way, a portion of the plurality of stimulus levels may be selected that substantially covers or represents the range of attributes of the plurality of stimulus levels. For example, stimulus levels that cover a range of the stimulus attributes described above may be selected as a representative subset, e.g., that stretch the consonant portion of the phonemes, that vary the relative loudness of the consonant and vowel portions of the phonemes, and/or that vary the gap between the consonant and vowel portions of the phonemes, among others.
Each confusable pair of phonemes from the representative subset of the plurality of confusable pairs of phonemes may be processed or considered at each stimulus level of the representative subset of the plurality of stimulus levels. Specifically, for each confusable pair of phonemes from the representative subset of the plurality of confusable pairs of phonemes, and for each stimulus level of the representative subset of the plurality of stimulus levels, the following described method elements may be performed.
In one embodiment, icons for each phoneme from the confusable pair may be graphically presented on the computing device. In other words, icons for each phoneme from the confusable pair may be graphically presented on the computing device, where, for example, each icon may textually and phonetically represent the respective phoneme. For example, buttons may be displayed that are respectively labeled with the phonemes of the confusable pair. Thus, the icons may be or include visual representations of the phonemes on the computing device. Note that in preferred embodiments, the visual representations are independently selectable by the aging adult.
A computer generated one of the phonemes from the confusable pair may be aurally presented on the computing device, the computer generation corresponding to the stimulus level. In other words, one of the phonemes from the confusable pair may be synthesized and aurally or audibly presented in accordance with the stimulus level (of the representative subset of the plurality of stimulus levels). Note that in some embodiments, the term “computer generated” may indicate that the phonemes are generated algorithmically by the computing device rather than simply processing recorded speech. Moreover, in some embodiments, the aurally presenting may be performed by a selected one of a plurality of synthesized speakers, where in this use, “speaker” refers to a source of speech, such as a human speaker, not a device for presenting general sounds, such as a stereo speaker. In one embodiment, the aurally presented computer generated one of the phonemes may be randomly selected from the confusable pair. In one embodiment, the aurally presenting the phoneme may be performed via headphones coupled to the computing device, although any other means, e.g., computer speakers, may be used as desired.
The adult may be required to select one of the icons, specifically, the icon corresponding to the aurally presented phoneme. In other words, the adult may be required to select the aurally presented phoneme by selecting the corresponding icon. For example, the adult may have to move a selection tool, e.g., a computer mouse, over one of the icons, and indicate the selection, e.g., by clicking a button on the mouse while the cursor is over the icon. Note that any other means of selection are also contemplated.
Then, the correctness or incorrectness of the adult's icon selection may be recorded, i.e., whether the adult correctly selected an icon corresponding to the aurally presented one of the phonemes may be recorded, thereby generating response results. In other words, the method may include recording whether the adult correctly selected an icon corresponding to the aurally presented one of the phonemes, e.g., for later analysis. Thus, the method elements of graphically presenting, aurally presenting, requiring, and recording may be performed for each confusable pair of phonemes in the representative subset of the plurality of confusable pairs of phonemes, at each stimulus level in the representative subset of the plurality of stimulus levels.
A success rate for the adult may be determined based on the response results, where the success rate includes or is an estimate of the adult's success rate with respect to the provided plurality of confusable pairs of phonemes at the provided plurality of stimulus levels. In other words, a single score or metric may be determined based on the recorded responses of the adult, where, because the confusable pairs of phonemes and the stimulus levels were respective representative subsets of the pluralities of confusable pairs of phonemes and stimulus levels, the determined success rate may be indicative (i.e., predictive) of how well the adult would perform with the provided pluralities of confusable pairs of phonemes and stimulus levels.
In one embodiment, the method may further include repeating the method elements of graphically presenting, aurally presenting, requiring, and recording, in an iterative manner for each confusable pair of phonemes of the representative subset of the plurality of confusable pairs of phonemes, at each stimulus level of the representative subset of the plurality of stimulus levels, where determining the success rate for the adult may be further based on response results from this repeating. In other words, the graphically presenting, aurally presenting, requiring, and recording, may be performed for each phoneme pair at each stimulus level more than once, and the total results used to determine the success rate.
For example, in one embodiment, for each phoneme contrast (confusable pair of phonemes)/level of the respective subsets, the adult may be asked or required to identify a specified number of items, e.g., 30, (randomly) selected from the same confusable pair. In some embodiments, illustrative practice examples may be presented first. For example, in one embodiment, the adult may first hear a specified number of randomly ordered practice items (e.g., 10), for which answers may be provided, after which the above representative trials may be performed for assessment.
Note that in preferred embodiments, no visual or auditory feedback may be provided to the adult, for example, to minimize learning effects during the test procedure. This is an important difference between the above described assessment method and the regular (Tell Us Apart) exercise.
Other features and advantages of the present invention will become apparent upon study of the remaining portions of the specification and drawings.
Referring to
Now referring to
Before providing a detailed description of the present invention, a brief overview of certain components of speech will be provided, along with an explanation of how these components are processed by subjects. Following the overview, general information on speech processing will be provided so that the reader will better appreciate the novel aspects of the present invention.
Referring to
Also shown are formants for a phoneme /ba/. This phoneme contains an upward sweep frequency component 308, at approximately 2 khz, having a duration of approximately 35 ms. The phoneme also contains an upward sweep frequency component 310, at approximately 1 khz, during the same 35 ms period. Following the stop consonant portion /b/ of the phoneme, is a constant frequency vowel portion 314 whose duration is approximately 110 ms.
Thus, both the /ba/ and /da/ phonemes begin with stop consonants having modulated frequency components of relatively short duration, followed by a constant frequency vowel component of longer duration. The distinction between the phonemes exists primarily in the 2 khz sweeps during the initial 35 ms interval. Similarity exists between other stop consonants such as /ta/, /pa/, /ka/ and /ga/.
Referring now to
With the above general background of speech elements, and how subjects process them, a general overview of speech processing will now be provided. As mentioned above, one problem that exists in subjects is the inability to distinguish between short duration acoustic events. If the duration of these acoustic events are stretched, in the time domain, it is possible to train subjects to distinguish between these acoustic events. An example of such time domain stretching is shown in
In
Another method that may be used to help subjects distinguish between phonemes is to emphasize selected frequency envelopes within a phoneme. Referring to
A third method that may be used to train subjects to distinguish short duration acoustic events is to provide frequency sweeps of varying duration, separated by a predetermined interval, as shown in
Although a number of methodologies may be used to produce the stretching and emphasis of phonemes, of processing speech to stretch or emphasize certain portions of the speech, and to produce sweeps and bursts, according to the present invention, a complete description of the methodology used within HiFi is described in Appendix G, which should be read as being incorporated into the body of this specification.
Appendices H, I and J have further been included, and are hereby incorporated by reference to further describe the code which generates the sweeps, the methodology used for incrementing points in each of the exercises, and the stories used in the exercise Story Teller.
Each of the above described methods have been combined in a unique fashion by the present invention to provide an adaptive training method and apparatus for enhancing memory and cognition in aging adults. The present invention is embodied into a computer program entitled HiFi by Neuroscience Solutions, Inc. The computer program is provided to a participant via a CD-ROM which is input into a general purpose computer such as that described above with reference to
Referring to
High or Low
Referring now to
Referring now to
In one embodiment, the participant is presented with two or more frequency sweeps, each separated by an inter-stimulus-interval (ISI). For example, the sequence of frequency sweeps might be (UP, DOWN, UP). The participant is required, after the frequency sweeps are auditorily presented, to indicate the order of the sweeps by selecting the blocks 1002, 1004, according to the sweeps. Thus, if the sequence presented was UP, DOWN, UP, the participant would be expected to indicate the sequence order by selecting the left block 1002, then right block 1004, then left block 1002. If the participant correctly indicates the sweep order, as just defined, then they have correctly responded to the trial, the score indicator increments, and a “ding” is played to indicate a correct response. If the participant incorrectly indicates the sweep order, then they have incorrectly responded to the trial, and a “thunk” is played to indicate an incorrect response. With the above understanding of training with respect to the exercise HIGH or LOW, specifics of the game will now be described.
A goal of this exercise is to expose the auditory system to rapidly presented successive stimuli during a behavior in which the participant must extract meaningful stimulus data from a sequence of stimulus. This can be done efficiently using time order judgment tasks and sequence reconstruction tasks, in which participants must identify each successively present auditory stimulus. Several types of simple, speech-like stimuli are used in this exercise to improve the underlying ability of the brain to process rapid speech stimuli: frequency modulated (FM) sweeps, structured noise bursts, and phoneme pairs such as /ba/ and /da/. These stimuli are used because they resemble certain classes of speech. Sweeps resemble stop consonants like /b/ or /d/. Structured noise bursts are based on fricatives like /sh/ or /f/, and vowels like /a/ or /i/. In general, the FM sweep tasks are the most important for renormalizing the auditory responses of participants. The structured noise burst tasks are provided to allow high-performing participants who complete the FM sweep tasks quickly an additional level of useful stimuli to continue to engage them in time order judgment and sequence reconstruction tasks.
This exercise is divided into two main sections, FM sweeps and structured noise bursts. Both of these sections have: a Main Task, an initiation for the Main Task, a Bonus Task, and a short initiation for the Bonus Task. The Main Task in FM sweeps is Task 1 (Sweep Time Order Judgment), and the Bonus Task is Task 2 (Sweep Sequence Reconstruction). FM Sweeps is the first section presented to the participant. Task 1 of this section is closed out before the participant begins the second section of this exercise, structured noise bursts. The Main Task in structured noise bursts is Task 3 (Structured Noise Burst Time Order Judgment), and the Bonus Task is Task 4 (Structured Noise Burst Sequence Reconstruction). When Task 3 is closed out, the entire Task is reopened beginning with easiest durations in each frequency. The entire Task is replayed.
Task 1—Main Task: Sweep Time Order Judgment
This is a time order judgment task. Participants listen to a sequential pair of FM sweeps, each of which can sweep upwards or downwards. Participants are required to identify each sweep as upwards or downwards in the correct order. The task is made more difficult by changing both the duration of the FM sweeps (shorter sweeps are more difficult) and decreasing the inter-stimulus interval (ISI) between the FM sweeps (shorter ISIs are more difficult).
Stimuli consist of upwards and downwards FM sweeps, characterized by their base frequency (the lowest frequency in the FM sweep) and their duration. The other characteristic defining an FM sweep, the sweep rate, is held constant at 16 octaves per second throughout the task. This rate was chosen to match the average FM sweep rate of formants in speech (e.g., ba/da). A pair of FM sweeps is presented during a trial. The ISI changes based on the participant's performance. There are three base frequencies:
Initially, a “training” session is provided to illustrate to the participant how the exercise is to be played. More specifically, an upward sweep is presented to the participant, followed by an indication, as shown in
Referring now to
Referring to
Referring to
Referring now to
Referring now to
Choosing a frequency, duration (category) and ISI: The first time in: the participant begins by opening duration index 1 (80 ms) in frequency index 1 (500 Hz). The starting ISI is 600 ms when opening a duration and the ISI step size index when entering a duration is 1.
Beginning subsequent sessions: The participant moves to a new frequency unless the participant has completed less than 20 trials in Task 1 of the previous session's frequency.
Returning from Task 2 (bonus task): The participant will be switching durations, but generally staying in the same frequency.
Switching frequencies: The frequency index is incremented, cycling the participant through the frequencies in order by frequency index (500 Hz, 1000 Hz, 200 Hz, 500 Hz, etc.). If there are no open durations in the new frequency, the frequency index is incremented again until a frequency is found that has an open duration. If all durations in all frequencies have been closed out, Task 1 is closed. The participant begins with the longest open duration (lowest duration index) in the new frequency.
Switching durations: Generally, the duration index is incremented until an open duration is found (the participant moves from longer, easier durations to shorter, harder durations). If there are no open durations, the frequency is closed and the participant switches frequencies. A participant switches into a duration with a lower index (longer, easier duration) when 10 incorrect trials are performed at an ISI of 1000 ms at a duration index greater than 1.
Progression within a duration changes in ISI: ISIs are changed using a 3-up/1-down adaptive tracking rule: Three consecutive correct trials equals advancement—ISI is shortened. One incorrect equals retreat—ISI is lengthened. The amount that the ISI changes is adaptively tracked. This allows participants to move in larger steps when they begin the duration and then smaller steps as they approach their threshold. The following steps sizes are used:
When starting a duration, the ISI step index is 1 (50 ms). This means that 3 consecutive correct trials will shorten the ISI by 50 ms and 1 incorrect will lengthen the ISI by 50 ms-3up/1down. The step size index is increased after every second Sweeps reversal. A Sweeps reversal is a “change in direction”. For example, three correct consecutive trials shortens the ISI. A single incorrect lengthens the ISI. The drop to a longer ISI after the advancement to a shorter ISI is counted as one reversal. If the participant continues to decrease difficulty, these drops do not count as reversals. A “change in direction” due to 3 consecutive correct responses counts as a second reversal.
A total of 8 reversals are allowed within a duration; the 9th reversal results in the participant exiting the duration; the duration remains open unless criteria for stable performance have been met. ISI never decreases to lower than 0 ms, and never increases to more than 1000 ms. The tracking toggle pops the participant out of the Main Task and into Task Initiation if there are 5 sequential increases in ISI. The current ISI is stored. When the participant passes initiation, they are brought back into the Main Task. Duration re-entry rules apply. A complete description of progress through the exercise High or Low is found in Appendix A.
To allow the text of this specification to be presented clearly, the details relating to progression methodology, processing, stimuli, etc., for each of the exercises within HiFi have been placed in Appendices to this specification. However, applicants consider the appendices to be part of this specification. Therefore, they should be read as part of this specification, and as being incorporated within the body of this specification for all purposes.
Stretch and Emphasis Processing of Natural Speech in HiFi
In order to improve the representational fidelity of auditory sensory representations in the brain of trained individuals, natural speech signals are initially stretched and emphasized. The degree of stretch and emphasis is reduced as progress is made through the exercise. In the final stage, faster than normal speech is presented with no emphasis.
Both stretching and emphasis operations are performed using the Praat (v. 4.2) software package (http://www.fon.hum.uva.nl/praat/) produced by Paul Boersma and David Weenink at the Institute for Phonetic Sciences at the University of Amsterdam. The stretching algorithm is a Pitch-Synchronous OverLap-and-Add method (PSOLA). The purpose of this algorithm is lengthen or shorten the speech signal over time while maintaining the characteristics of the various frequency components, thus retaining the same speech information, only in a time-altered form. The major advantage of the PSOLA algorithm over the phase vocoder technique used in previous versions of the training software is that PSOLA maintains the characteristic pitch-pulse-phase synchronous temporal structure of voiced speech sounds. An artifact of vocoder techniques is that they do not maintain this synchrony, creating relative phase distortions in the various frequency components of the speech signal. This artifact is potentially detrimental to older observers whose auditory systems suffer from a loss of phase-locking activity. A minimum frequency of 75 Hz is used for the periodicity analysis. The maximum frequency used is 600 Hz. Stretch factors of 1.5, 1.25, 1 and 0.75 are used.
The emphasis operation used is referred to as band-modulation deepening. In this emphasis operation, relatively fast-changing events in the speech profile are selectively enhanced. The operation works by filtering the intensity modulations in each critical band of the speech signal. Intensity modulations that occur within the emphasis filter band are deepened, while modulations outside that band are not changed. The maximum enhancement in each band is 20 dB. The critical bands span from 300 to 8000 Hz. Bands are 1 Bark wide. Band smoothing (overlap of adjacent bands) is utilized to minimize ringing effects. Band overlaps of 100 Hz are used. The intensity modulations within each band are calculated from the pass-band filtered sound obtained from the inverse Fourier transform of the critical band signal. The time-varying intensity of this signal is computed and intensity modulations between 3 and 30 Hz are enhanced in each band. Finally, a full-spectrum speech signal is recomposed from the enhanced critical band signals. The major advantage of the method used here over methods used in previous versions of the software is that the filter functions used in the intensity modulation enhancement are derived from relatively flat Gaussian functions. These Gaussian filter functions have significant advantages over the FIR filters designed to approximate rectangular-wave functions used previously. Such FIR functions create significant ringing in the time domain due to their steepness on the frequency axis and create several maxima and minima in the impulse response. These artifacts are avoided in the current methodology.
The following levels of stretching and emphasis are used in HiFi:
Referring now to
Applicants believe that auditory systems in older adults suffer from a degraded ability to respond effectively to rapidly presented successive stimuli. This deficit manifests itself psychophysically in the participant's poor ability to perform auditory stimulus discriminations under backward and forward masking conditions. This manifests behaviorally in the participant's poor ability to discriminate both the identity of consonants followed by vowels, and vowels preceded by consonants. The goal of Tell us Apart is to force the participant to make consonant and vowel discriminations under conditions of forward and backward masking from adjacent vowels and consonants respectively. This is accomplished using sequential phoneme identification tasks and continuous performance phoneme identification tasks, in which participants identify successively presented phonemes. Applicants assume that older adults will find making these discriminations difficult, given their neurological deficits. These discriminations are made artificially easy (at first) by using synthetically generated phonemes in which both 1) the relative loudness of the consonants and vowels and/or 2) the gap between the consonants and vowels has been systematically manipulated to increase stimulus discriminability. As the participant improves, these discriminations are made progressively more difficult by making the stimuli more normal.
Referring now to
Referring to
Referring to
Match It
Goals of the exercise Match It! include: 1) exposing the auditory system to substantial numbers of consonant-vowel-consonant syllables that have been processed to emphasize and stretch rapid frequency transitions; and 2) driving improvements in working memory by requiring participants to store and use such syllable information in auditory working memory. This is done by using a spatial match task similar to the game “Concentration”, in which participants must remember the auditory information over short periods of time to identify matching syllables across a spatial grid of syllables.
Match It! has only one Task, but utilizes 5 speech processing levels. Processing level 1 is the most processed and processing level 5 is normal speech. Participants move through stages within a processing level before moving to a less processed speech level. Stages are characterized by the size of the spatial grid. At each stage, participants complete all the categories. The task is a spatial paired match task. Participants see an array of response buttons. Each response button is associated with a specific syllable (e.g., “big”, “tag”), and each syllable is associated with a pair of response buttons. Upon pressing a button, the participant hears the syllable associated with that response button. If the participant presses two response buttons associated with identical syllables consecutively, those response buttons are removed from the game. The participant completes a trial when they have removed all response buttons from the game. Generally, a participant completes the task by clicking on various response buttons to build a spatial map of which buttons are associated with which syllables, and concurrently begins to click consecutive pairs of responses that they believe, based on their evolving spatial map, are associated with identical syllables. The task is made more difficult by increasing the number of response buttons and manipulating the level of speech processing the syllables receive.
Stages: There are 4 task stages, each associated with a specific number of response buttons in the trial and a maximum number of response clicks allowed:
Categories: The stimuli consist of consonant-vowel-consonant syllables or single phonemes:
Category 1 consists of easily discriminable CV pairs. Leading consonants are chosen from those used in the exercise Tell us Apart and trailing vowels are chosen to make confusable leading consonants as easy to discriminate as possible. Category 2 consists of easily discriminable CVC syllables. Stop, fricative, and nasal consonants are used, and consonants and vowels are placed to minimize the number of confusable CVC pairs. Categories 3, 4, and 5 consist of difficult to discriminate CVC syllables. All consonants are stop consonants, and consonants and vowels are placed to maximize the number of confusable CVC syllables (e.g., cab/cap).
Referring now to
Referring now to
Referring now to
Referring now to
Referring now to
Sound Replay
Applicants believe that the degraded representational fidelity of the auditory system in older adults causes an additional difficulty in the ability of older adults to store and use information in auditory working memory. This deficit manifests itself psychophysically in the participant's poor ability to perform working memory tasks using stimuli presented in the auditory modality. The goals of this exercise therefore include: 1) To expose the participant's auditory system to substantial numbers of consonant-vowel-consonant syllables that have been processed to emphasize and stretch the rapid frequency transitions; and 2) To drive improvements in working memory by requiring participants to store and use such syllable information in auditory working memory. These goals are met using a temporal match task similar to the neuropsychological tasks digit span and digit span backwards, in which participants must remember the auditory information over short periods of time to identify matching syllables in a temporal stream of syllables.
Sound Replay has a Main Task and Bonus Task. The stimuli are identical across the two Tasks in Sound Replay. In one embodiment, the stimuli used in Sound Replay is identical to that used in Match It. There are 5 speech processing levels. Processing level 1 is the most processed and processing level 5 is normal speech. Participants move through stages within a processing level before moving to a less processed speech level. At each stage, participants complete all categories.
A task is a temporal paired match trial. Participants hear a sequence of processed syllables (e.g., “big”, “tag”, “pat”). Following the presentation of the sequence, the participant sees a number of response buttons, each labeled with a syllable. All syllables in the sequence are shown, and there may be buttons labeled with syllables not present in the sequence (distracters). The participant is required to press the response buttons to reconstruct the sequence. The Task is made more difficult by increasing the length of the sequence, decreasing the ISI, and manipulating the level of speech processing the syllables receive. A complete description of the flow through the various stimuli and processing levels is found in Appendix D.
Referring now to
Referring now to
Listen and Do
Applicants believe that a degraded representational fidelity of the auditory system in older adults causes an additional difficulty in the ability of older adults to store and use information in auditory working memory. This deficit manifests itself behaviorally in the subject's poor ability to understand and follow a sequence of verbal instructions to perform a complex behavioral task. Therefore, goals of the exercise Listen and Do include: 1) exposing the auditory system to a substantial amount of speech that has been processed to emphasize and stretch the rapid frequency transitions; and 2) driving improvements in speech comprehension and working memory by requiring participants to store and use such speech information. In this task, the participant is given auditory instructions of increasing length and complexity.
The task requires the subject to listen to, understand, and then follow an auditory instruction or sequence of instructions by manipulating various objects on the screen. Participants hear a sequence of instructions (e.g., “click on the bank” or “move the girl in the red dress to the toy store and then move the small dog to the tree”). Following the presentation of the instruction sequence, the participant performs the requested actions. The task is made more difficult by making the instruction sequence contain more steps (e.g., “click on the bus and then click on the bus stop”), by increasing the complexity of the object descriptors (i.e., specifying adjectives and prepositions), and manipulating the level of speech processing the instruction sequence receives. A complete description of the flow through the processing levels in the exercise Listen and Do is found in Appendix E.
Referring now to
Referring now to
Referring now to
Story Teller
Applicants believe that the degraded representational fidelity of the auditory system in older adults causes an additional difficulty in the ability of older adults to store and use information in auditory working memory. This deficit manifests itself behaviorally in the participant's poor ability to remember verbally presented information. Therefore applicants have at least the following goals for the exercise Story Teller: 1) to expose the participant's auditory system to a substantial amount of speech that has been processed to emphasize and stretch the rapid frequency transitions; and 2) to drive improvements in speech comprehension and working memory by requiring participants to store and recall verbally presented information. This is done using a story recall task, in which the participant must store relevant facts from a verbally presented story and then recall them later. In this task, the participant is presented with auditory stories of increasing length and complexity. Following the presentation, the participant must answer specific questions about the content of the story.
The task requires the participant to listen to an auditory story segment, and then recall specific details of the story. Following the presentation of a story segment, the participant is asked several questions about the factual content of the story. The participant responds by clicking on response buttons featuring either pictures or words. For example, if the story segment refers to a boy in a blue hat, a question might be: “What color is the boy's hat?” and each response button might feature a boy in a different color hat or words for different colors. The task is made more difficult by 1) increasing the number of story segments heard before responding to questions 2) making the stories more complex (e.g., longer, more key items, more complex descriptive elements, and increased grammatical complexity) and 3) manipulating the level of speech processing of the stories and questions. A description of the process for Story Teller, along with a copy of the stories and the stimuli is found in Appendix F.
Referring now to
Progressions in HiFi Assessments
Exercise based assessments are designed to assess a participant's threshold with respect to stimuli on a given exercise. The assessment can be used to assess or determine a pre-training threshold that can then be used to calibrate the program to an individual's capabilities on various exercises as well as serve as a baseline measure to which post-training thresholds can be compared. Comparison of pre-training to post-training thresholds may be used to determine the gains made as a function of training with the cognition enhancement exercises described herein.
In some embodiments, exercise based assessments may be similar or even identical to the actual exercises in appearance with the exception of the rewards and points systems. Since assessments are not designed to be training tools, but rather assessment tools, the rewards screen, progress bar, and points are not needed. However, the bells and thunks associated with correct and incorrect responses may remain, i.e., may still be used.
There are various approaches whereby such thresholds may be assessed, such as, for example, the well known QUEST (Quick Estimation by Sequential Testing) threshold method, which is an adaptive psychometric procedure for use in psychophysical experiments, or a related method, referred to as the ZEST (Zippy Estimation by Sequential Testing) procedure or method, among others, although it should be noted that such methods have not heretofore been utilized in cognition enhancement exercise assessments as described herein.
The ZEST procedure is a maximum-likelihood strategy to estimate a subject's threshold in a psychophysical experiment based on a psychometric function that describes the probability a stimulus is detected as a function of the stimulus intensity. For example, consider a cumulative Gaussian psychometric function, F(x−T), for a 4-alternative-forced-choice (afc) task with a 5% lapsing rate, with proportion correct (ranging from 0-1) plotted against intensity of the stimulus (ranging from 0-5). The threshold is defined to be the mean of the Gaussian distribution—a value yielding 60% success rate, which corresponds to an intensity of 2.
The method may make some assumptions about the psychophysics:
1. The psychometric function has the same shape, except a shift along the stimulus intensity axis to indicate different threshold value.
2. The threshold value does not change from trial to trial.
3. Individual trials are statistically independent.
The primary idea of the ZEST procedure is as follows: given a prior probability density function (P.D.F.) centered around the best threshold guess, x, this P.D.F. is adjusted after each trial by one of two likelihood functions, which are the probability functions that the subject will respond “yes” or “no” to the stimulus at intensity as a function of threshold. Since the psychometric function has a constant shape and is of the form F(x−T), fixing the intensity x and treating threshold T as the independent variable, the “yes” likelihood, p=F(−(T−x)), is thus the mirror image of the psychometric function about the threshold, and the “no” likelihood function is then simply 1−p.
The P.D.F. is updated using Bayes' rule, where the posterior P.D.F. is obtained by multiplying the prior P.D.F. by the likelihood function corresponding to the subject's response to the trial's stimulus intensity. The mean of the updated (or posterior) P.D.F. is then used as the new threshold estimate and the test is repeated with the new estimate until the posterior P.D.F. satisfies a confidence interval criteria (e.g. standard deviation of posterior P.D.F.<predetermined value) or a maximum number of trials is reached.
In one example of the ZEST procedure, a single trial of a 4-afc experiment is performed, with x=2.5 (intensity) as the initial threshold guess. If the subject responds correctly, the next trial is placed at the mean of the corresponding posterior P.D.F., ˜x=2.3; if the response is incorrect, the next trial is placed at the mean of the corresponding P.D.F., ˜x=2.65.
In some embodiments of the exercised based assessments described herein, a 2-stair ZEST procedure may be employed, where two independent tracks with starting values, preferably, encompassing the true threshold, each running its own ZEST procedure, are randomly interleaved in the threshold seeking procedure. In addition to their individual termination criterion, the difference between the two stairs may also be required to be within a specified range, e.g., the two stairs may be constrained to be a predetermined distance apart. An exemplary implementation of this method is described below with respect to the High or Low threshold assessment.
As used herein, the parameters required for ZEST may include the mean of the prior P.D.F. (threshold estimate), the standard deviation of the prior P.D.F. (spread of threshold distribution), the standard deviation of the cumulative Gaussian distribution (slope of psychometric function), the maximum number of trials to run, and a confidence level and interval. Additionally, in one embodiment, the trial-by-trial data saved for analysis may include: the track used, the stimulus intensity presented, the subject's response, the mean of posterior P.D.F., and the standard deviation of the posterior P.D.F., as well as any other data deemed necessary or useful in assessing the participant's threshold.
In various embodiments, an assessment performed with respect to a given exercise may be performed independently of the training exercise, or, alternatively, may be performed in conjunction with the training exercise (although it should be noted that in either case, the primary aspects and mechanisms of the exercise are performed). For example, in some embodiments, a first assessment may be performed, e.g., using a representative set or subset of data for the exercise, then the exercise may be performed as a training exercise, e.g., using the full or regular data set for the exercise, after which another, second, assessment may be performed, e.g., again using the representative set or subset of data for the exercise. A comparison of the adult's performance on the pre (exercise) and post (exercise) assessments may be used to gauge the effectiveness of the training (middle) exercise. An exemplary embodiment using this approach is described in detail below with respect to the Match It assessment, although it should be noted that this approach may be used regarding any of the exercises described herein, or others. The following describes exemplary embodiments of exercises implementing threshold assessments specific to the respective exercises. It should be noted that the particular implementations described may be used with respect to other, e.g., similar, exercises, as well.
High or Low Threshold Assessment
The following threshold assessment method is based on the High or Low exercise described above with reference to
A primary purpose of the High or Low threshold assessment is to determine the smallest duration of tone sweeps in a timer order judgment task that a person can respond correctly to above a statistical threshold. The High or Low assessment may be similar to the High or Low exercise with respect to visual presentation, where the differences between the assessment and the exercise lie (at least primarily) in the movement or progression through the task and the data that are obtained from this movement for the assessment. The task is designed to obtain a threshold, which is a statistical rather than an exact quantity. For the purposes of this task, the threshold is defined as the smallest duration of tone sweep (in milliseconds) at which a participant will fail to respond correctly a specified percentage, e.g., 69%, of all trials for a serial order judgment task. In a preferred embodiment, being a computer based task, the High or Low assessment may use the ZEST procedure to progress or move through the task, adjust the duration of the tone sweeps to be presented, and determine the statistical threshold.
As noted above, many aspects of the High or Low assessment may generally be similar, or possible even identical, to the High or Low exercise with respect to visual presentation. However, some aspects of the exercise version of High or Low are not necessary in the High or Low assessment. For example, the progress bar normally displayed on the upper left hand corner, the points bar, and the reward area are not necessary, and so may be omitted. The features or assets that remain the same include the buttons and the “ding” and “thump” sounds that play after a participant responds correctly or incorrectly. Also identical to the exercise version, may be the stimulus presentation.
In 3502, first and second tracks may be initialized with respective durations based on an initial anticipated threshold, where the initial anticipated threshold is an initial estimate or guess of a duration for frequency sweeps corresponding to a specified performance level of the adult, e.g., a stimulus duration at which the adult fails to respond correctly some specified percentage of the time, e.g., 69%. For example, in one embodiment, the first track may be initialized to a first duration that is below the initial anticipated threshold, e.g., preferably just slightly below the initial anticipated threshold, and the second track may be initialized to a second duration that is (e.g., slightly) above the initial anticipated threshold. Thus, the initial durations of the two tracks may straddle the initial anticipated threshold.
In 3504, upward and downward frequency sweeps associated with respective “up” and “down” icons may be provided. For example, a first frequency sweep that increases in frequency over time may be provided, and associated with a first icon, e.g., a button that displays an up arrow (see, e.g.,
In 3506, at least two frequency sweeps may be aurally presented to the adult utilizing the first frequency sweep, the second frequency sweep, or a combination of the first and second frequency sweeps, in accordance with the duration of a specified one of either the first track or the second track. In other words, one of the tracks may be selected or otherwise determined, and the frequency sweeps may be presented with durations of the selected track. In one embodiment, the aurally presenting may include randomly selecting at least two frequency sweeps to be presented, utilizing combinations of the first frequency sweep and the second frequency sweep. In one embodiment, the first frequency sweep may be referred to as UP, and the second frequency sweep may be referred to as DOWN, and the aurally presenting at least two frequency sweeps may include any of the following possible combinations: UP-UP, UP-DOWN, DOWN-UP, and DOWN-DOWN. Of course, other sequences of sweeps are also contemplated, and any such sequence may be used as desired, e.g., UP-DOWN-UP, DOWN-DOWN-UP-DOWN, and so forth. Note that the aural presentations may be made via any of a variety of means, such as, for example, via headphones attached to the computing device, speakers, and so forth.
Note that the frequency sweeps are presented (sequentially) with an inter-stimulus-interval (ISI), i.e., a specified time interval between successive frequency sweeps. In preferred embodiments, the initial anticipated threshold, the first duration, the second duration, and the (to be determined) threshold each includes a respective sweep duration, and a respective inter-stimulus-interval (ISI). In other words, the term “duration” may refer to the actual sweep duration and the ISI, and so may be a compound parameter or value.
The frequency ranges for the sweeps may be specified as desired, e.g., based on typical (aging) adult hearing frequency responses. For example, in some embodiments, if the sweep duration is above 80 ms, the frequency range for the sweep may be approximately 1000 Hz to 2428 Hz. If the sweep duration is below 80 ms, the frequency range for the sweep may be 16 octaves-per-second with minimum of 1000 Hz. Thus, for example, if the sweep duration for a frequency sweep is 70 ms, the frequency range may be 1000 Hz to 2174 Hz. It should be noted, however, that these particular values and relationships for the sweeps are meant to be exemplary only, and that other values may be used as desired.
Moreover, in one embodiment, the sweep duration and inter-stimulus-interval may be co-varied in the ratio of 1:1. In other words, the sweep duration and inter-stimulus-interval may have the same value, or in some embodiments, may retain the same ratio when varied.
In 3508, the adult may be required to respond to the at least two frequency sweeps by indicating, utilizing the icons, an order in which the at least two frequency sweeps were presented. In other words, the adult may, in response to hearing the sequence of frequency sweeps, indicate the perceived order of the sweeps via the two icons. For example, in the case of the two sweep sequence UP-DOWN, the adult should indicate the order by pressing the “up” icon, and then the “down” icon. For a three sweep sequence, e.g., DOWN-DOWN-UP, the adult should press the “down” icon twice, then the “up” icon, and so forth.
In one embodiment, the requiring may include providing a period of time in which the adult is to select the icons in the order in which the at least two frequency sweeps were presented, selection of the icons made by the adult placing a cursor over a icon and clicking a mouse, wherein each mouse click is recorded as a selection, recording the selections made by the adult, and recording whether the adult correctly identified the order in which the at least two frequency sweeps were presented.
The duration of the specified track may then be modified, based on the adult's response, as indicated in 3510. For example, the duration of the track may be modified in accordance with a maximum likelihood procedure, such as a QUEST (quick estimation by sequential testing) threshold procedure, and/or a ZEST (Zippy Estimation by Sequential Testing) threshold procedure, although other threshold procedures may be used as desired. In one embodiment, for each track, modifying the duration of the specified track based on the adult's response may include increasing the duration if the adult responds incorrectly, and decreasing the duration if the adult responds correctly. As noted above, modifying the duration of a track may include modifying the frequency sweep duration and/or the ISI. Thus, for each trial (in a given track), the duration of the sweep for that trial may be determined by the performance of the previous trial for that track. In other words, the adult's response to the stimulus (frequency sweep) determines that track's next sweep duration via a maximum likelihood method.
In 3512, the above presenting (3506), requiring (3508), and modifying (3510), may be repeated one or more times in an iterative manner to determine respective final durations for the first track and the second track. For example, in one embodiment, trials in the first track and the second track may be performed in an alternating manner, or, alternatively, trials may be performed in the first track and the second track randomly with equal probability. Thus, over numerous trials, the number of trials performed in each track should be equal, or at least substantially equal. In preferred embodiments, the presenting, requiring, and modifying, may be repeated until the durations of the first track and the second track have converged to values within a specified confidence interval, and where the values are within a specified distance from each other, or, until a specified number of trials have been conducted for each track. In other words, the repetition may continue until either some maximum number of trials has been performed, or until convergence conditions for the tracks have been met, both singly, and together. For example, each track may be required converge to a respective duration value (which may include both the sweep duration and the ISI for the track), and the convergent values for the two tracks may be required to be within some distance or interval of each other.
In 3514, a threshold for the adult may be determined based on the respective final durations for the first track and the second track, where the threshold is or specifies the duration associated with the specified performance level of the adult. For example, as mentioned above, the determined threshold may specify the duration (sweep duration and/or ISI) at which the adult fails to respond correctly some specified percentage of the trials, e.g., 69%, although it should be noted that any other percentage may be used as desired. In one embodiment, the threshold for the adult may be determined by averaging the respective final durations for the first track and the second track.
In some embodiments, the presenting, requiring, and modifying may compose performing a trial, and certain information may be save on a per trial basis. For example, In one embodiment, for each trial, the method may include saving one or more of: which track was used in the trial, the duration used in the trial, the direction and order of sweeps presented to the adult in the trial, the series of icons used in the adult's response (e.g., UP-button, DOWN-button, UP-button, and so forth), the correctness or incorrectness of the adult's response, the mean of a posterior probability distribution function for the maximum likelihood procedure, and the standard deviation of the posterior probability distribution function for the maximum likelihood procedure.
Additionally, in some embodiments, various parameters for the maximum likelihood procedure besides the respective (initial) durations of the two tracks may be initialized, such as, for example, the standard deviation of a cumulative Gaussian psychometric function for the maximum likelihood procedure, and/or the standard deviation of a prior threshold distribution for the maximum likelihood procedure. For example, in one embodiment, the following values may be used: initial duration values or estimates for the two tracks: 100 ms and 200 ms; standard deviation of cumulative Gaussian psychometric function: 0.14; Standard deviation of prior threshold distribution: 0.15. Exemplary values for the confidence interval and width are: confidence interval: 0.95; confidence interval width: 0.2, although it should be noted that other values may be used as desired.
In one embodiment, the method may include determining the initial anticipated threshold. For example, the initial anticipated threshold may be determined based on one or more of: the age of the adult, calibration trials performed by the adult, and/or calibration trials performed by other adults, e.g., in a “pilot” program, although it should be noted that any other type of information may also be used as desired to determine the initial anticipated threshold. In some embodiments, the method may also include performing a plurality of practice trials, i.e., prior to performing the method elements described above. For example, in one embodiment, the initial 20 (or any other number desired) trials may be considered practice and not included in the analysis or assessment of the threshold. Thus, while the initial 20 trials may follow a ZEST (or ZEST-like) stepping procedure, the two stairs may be reset on the 21st trial to the initial track values. As indicated above, the assessment may end when either the two tracks have converged to within a given confidence interval with both tracks also within a certain distance apart or when a maximum number of trials (e.g., 100), evenly conducted between the two tracks, have been performed. The average of the two tracks' ending stage values may thus yield the adult's threshold.
Tell Us Apart Assessment
The following assessment method is based on the Tell Us Apart exercise described above with reference to FIGS. 16-19.
1811 A primary purpose of the purpose of the Tell Us Apart assessment is to determine a participant's (i.e., an aging adult's) ability to classify formant transition and segment duration information in making phonetic categorizations. However, since (1) the range of implementations of these contrasts in speech is not readily described with respect to a single dimension along which a detection threshold can be measured, and (2) their realization in the Tell Us Apart exercise emphasis levels does not relate to detection difficulty in a monotonic manner that is sufficiently uniform across participants and contrasts, a participant's success in the Tell Us Apart exercise can currently best be estimated by measuring overall performance across a range of stimuli that is representative of the contrasts and difficulty included in the Tell Us Apart exercise. In other words, an assessment of the aging adult's success in the Tell Us Apart exercise may be estimated by performing a modified version of the exercise (e.g., without visual or auditory feedback) with a restricted but representative data set. More specifically, a single percent correct identification score for a pre-selected (restricted but representative) stimulus set may be determined.
In 3702, a plurality of confusable pairs of phonemes may be provided, each of the phonemes having a consonant portion and a vowel portion.
In 3704, a plurality of stimulus levels, which may comprise emphasis levels, for computer processing of the plurality of confusable pairs of phonemes may be provided. A stimulus level may specify various timing and/or tonal aspects of a synthesized or computer processed phoneme (or word containing a phoneme) to make discriminating between the phonemes more or less difficult. For example, the plurality of stimulus levels may include stimulus levels that stretch the consonant portion of the phonemes, that vary the relative loudness of the consonant and vowel portions of the phonemes, and/or that vary the gap between the consonant and vowel portions of the phonemes, among others.
In 3706, a representative subset of the plurality of confusable pairs of phonemes may be selected for presentation to the aging adult. In other words, a portion of the plurality of confusable pairs of phonemes may be selected that broadly or substantially covers or represents the range of phonetic attributes of the plurality of confusable pairs of phonemes. For example, the confusable pairs of phonemes may be selected to represent a spectrum of articulation points, including, for example, back of throat, tongue and pallet, and lip generated consonants. As another example, some of the confusable pairs of phonemes may be selected to represent a frequency spectrum of vowels. In one embodiment, at least one of the representative subset of the plurality of stimulus levels may assist the aging adult in discriminating between the consonant and vowel portion of the one of the phonemes being aurally presented. For example, at least one of the plurality of stimulus levels may emphasize and stretch both the consonant and vowel portions of the one of the phonemes.
In 3708, a representative subset of the plurality of stimulus levels may be selected for use with the selected representative subset of confusable pairs of phonemes. Said another way, a portion of the plurality of stimulus levels may be selected that substantially covers or represents the range of attributes of the plurality of stimulus levels. For example, stimulus levels that cover a range of the stimulus attributes described above in 3704 may be selected as a representative subset, e.g., that stretch the consonant portion of the phonemes, that vary the relative loudness of the consonant and vowel portions of the phonemes, and/or that vary the gap between the consonant and vowel portions of the phonemes, among others.
As indicated in 3710 and 3712, each confusable pair of phonemes from the representative subset of the plurality of confusable pairs of phonemes may be processed or considered at each stimulus level of the representative subset of the plurality of stimulus levels. Specifically, for each confusable pair of phonemes from the representative subset of the plurality of confusable pairs of phonemes, and for each stimulus level of the representative subset of the plurality of stimulus levels, the following described method elements of 3714-3720 may be performed.
As indicated in 3714, icons for each phoneme from the confusable pair may be graphically presented on the computing device. In other words, icons for each phoneme from the confusable pair may be graphically presented on the computing device, where, for example, each icon may textually and phonetically represent the respective phoneme. For example, buttons may be displayed that are respectively labeled with the phonemes of the confusable pair, as illustrated in
In 3716, a computer generated one of the phonemes from the confusable pair may be aurally presented on the computing device, the computer generation corresponding to the stimulus level. In other words, one of the phonemes from the confusable pair may be synthesized and aurally or audibly presented in accordance with the stimulus level (of the representative subset of the plurality of stimulus levels). Note that in some embodiments, the term “computer generated” may indicate that the phonemes are generated algorithmically by the computing device rather than simply processing recorded speech. Moreover, in some embodiments, the aurally presenting may be performed by a selected one of a plurality of synthesized speakers, where in this use, “speaker” refers to a source of speech, such as a human speaker, not a device for presenting general sounds, such as a stereo speaker. In one embodiment, the aurally presented computer generated one of the phonemes may be randomly selected from the confusable pair. In one embodiment, the aurally presenting the phoneme may be performed via headphones coupled to the computing device, although any other means, e.g., computer speakers, may be used as desired.
In 3718, the adult may be required to select one of the icons, specifically, the icon corresponding to the aurally presented phoneme. In other words, the adult may be required to select the aurally presented phoneme by selecting the corresponding icon. For example, the adult may have to move a selection tool, e.g., a computer mouse, over one of the icons, and indicate the selection, e.g., by clicking a button on the mouse while the cursor is over the icon. Note that any other means of selection are also contemplated.
In 3720, the correctness or incorrectness of the adult's icon selection may be recorded, i.e., whether the adult correctly selected an icon corresponding to the aurally presented one of the phonemes may be recorded, thereby generating response results. In other words, the method may include recording whether the adult correctly selected an icon corresponding to the aurally presented one of the phonemes, e.g., for later analysis. Thus, method elements 3714-3720 may be performed for each confusable pair of phonemes in the representative subset of the plurality of confusable pairs of phonemes, at each stimulus level in the representative subset of the plurality of stimulus levels.
In 3722, a success rate for the adult may be determined based on the response results, where the success rate includes or is an estimate of the adult's success rate with respect to the provided plurality of confusable pairs of phonemes at the provided plurality of stimulus levels. In other words, a single score or metric may be determined based on the recorded responses of the adult, where, because the confusable pairs of phonemes and the stimulus levels were respective representative subsets of the pluralities of confusable pairs of phonemes and stimulus levels, the determined success rate may be indicative (i.e., predictive) of how well the adult would perform with the provided pluralities of confusable pairs of phonemes and stimulus levels.
In one embodiment, the method may further include repeating the method elements of 3714-3720, specifically, the graphically presenting, aurally presenting, requiring, and recording, in an iterative manner for each confusable pair of phonemes of the representative subset of the plurality of confusable pairs of phonemes, at each stimulus level of the representative subset of the plurality of stimulus levels, where determining the success rate for the adult may be further based on response results from this repeating. In other words, the graphically presenting, aurally presenting, requiring, and recording, may be performed for each phoneme pair at each stimulus level more than once, and the total results used to determine the success rate.
For example, in one embodiment, for each phoneme contrast (confusable pair of phonemes)/level of the respective subsets, the adult may be asked or required to identify a specified number of items, e.g., 30, (randomly) selected from the same confusable pair. In some embodiments, illustrative practice examples may be presented first. For example, in one embodiment, the adult may first hear a specified number of randomly ordered practice items (e.g., 10), for which answers may be provided, after which the above representative trials may be performed for assessment.
Note that in preferred embodiments, no visual or auditory feedback may be provided to the adult, for example, to minimize learning effects during the test procedure. This is an important difference between the above described assessment method and the regular (Tell Us Apart) exercise.
Stimuli Specifications
Note that, as described above, the representative subset of confusable pairs of phonemes and the representative subset of stimulus levels, collectively referred to as a test set, preferably contains stimuli that are representative of the contrasts and difficulty levels that HiFi participants (performers of the exercise(s) disclosed herein) are exposed to, and sufficiently novel that improvements cannot be attributed solely to learning specific to the set of sounds in HiFi. The following describes one exemplary test set.
In one embodiment, three contrasts (confusable phoneme pairs) may be included in the assessment. For example, two, e.g., bo/do and gi/ki, may be included in the (regular) Tell Us Apart exercise, where one, e.g., gi/ki, may be produced by a novel (female) synthesized speaker. In this particular embodiment test set, an additional (third) contrast, e.g., ba/ga may be included, using the same synthesized speaker as the (regular) Tell Us Apart exercise. Exemplary stimulus or emphasis levels used in the assessment may include (for each of the phoneme pairs): bo/do: 10, 5, 1; gi/ki: 10, 6, 2; and ba/ga/ 10, 5, 0, although it should be noted that these test set stimulus levels (and confusable phoneme pairs) are meant to be exemplary only, and are not intended to limit the assessment method to any particular set (or subset) of phonemes or stimulus levels.
Match It Assessment
A primary purpose of the Match It assessment described herein is to assess an aging adult's integration of visual-spatial and auditory memory. The Match It assessment is very similar to the Match It exercise described above, but with several key distinctions, as will be described below.
A general goal of the Match It assessment described herein is to provide a tool to compare pre and post training performance of the aging adult with respect to a visual-spatial and auditory memory exercise, such as the Match It exercise described above. In one exemplary embodiment, the method may use and present data in accordance with a representative subset of the many variables (speech processing level, stimulus category and grid size) used in the regular training exercise a specified number of times, e.g., 5 times, pre and post to allow for a consistent comparison of the adult's performance before and after training. A secondary goal of the Match It assessment method described herein is to develop more advanced analysis than is currently possible given the level of detail in a trial-by-trial database. Additionally, an attempt may be made to quantify memory-span and analyze strategies employed in this task.
Note that for assessment purposes, in some embodiments, certain visual aspects of the interface of the exercise version of Match It are not necessary, thus, since the method is not concerned with conveying progress to participants who are taking the assessment, all assets, e.g., GUI/auditory indicators, relating to progress are unnecessary (see, e.g.,
In 3802, at least two pairs of response buttons for selection by the adult may be graphically presented on a computing device, where the at least two pairs of response buttons compose a grid, and where the grid is selected from a first plurality of grids. In other words, a grid of buttons may be graphical presented to the adult, where the grid is from a first plurality of grids, e.g., selected for use in the assessment.
As noted above, in the assessment portion of the method (directed to the first plurality of grids), i.e., for trials performed with respect to the first plurality of grids, no visual or auditory indication of the adult's progress may be provided.
In 3804, the adult may be required to select one response button of the at least two pairs of response buttons from the presented grid, e.g., by clicking on a button with a pointing device, such as a mouse, although any other means of selecting the button may be used as desired.
In 3806, upon selection of one response button of the at least two pairs of response buttons by the adult, one of a plurality of computer generated stimuli that has been associated with the selected one response button of the at least two pairs of response buttons may be aurally presented, e.g., via speakers or headphones coupled to the computing device. For example, in one embodiment, the computer generated stimuli may include syllables that may be audibly presented to the adult, e.g., kiss, dish, etc. In another embodiment, the computer generated stimuli may include consonant-vowel phonemes, e.g., ki, na, etc. In yet other embodiments, the computer generated stimuli may include a plurality of consonant-vowel-consonant syllables, e.g.,fig, lab, etc.
In 3808, the adult may be required to select another response button of the at least two pairs of response buttons, i.e., where the other response button is different from the button selected in 3804.
In 3810, upon selection of another response button of the at least two pairs of response buttons by the adult, another one of the plurality of computer generated stimuli that has been associated with the selected another response button of the at least two pairs of response buttons may be aurally presented. Thus, each time the adult selects a response button, the stimulus associated with the button may be aurally presented to the adult.
In 3812, the two selected response buttons may be removed if the adult has consecutively selected two response buttons of the at least two pairs of response buttons that have had associated with them a desired pairing, removing those two selected buttons from those presented, e.g., removing the two buttons from the grid. In one embodiment, the desired pairing includes associating the same computer generated aural stimuli with the two consecutively selected response buttons. Thus, for example, in one embodiment, if the stimuli for the two selected buttons match, the buttons may be removed from the grid.
There are numerous ways in which the response buttons may be associated with the computer generated stimuli. For example, in one embodiment, desired pairs of stimuli may be associated with pairs of response buttons prior to graphically presenting the buttons. Alternatively, a stimulus may not be associated with a response button until after the response button is selected. Not associating a stimuli with a response button until after the response button is selected may prevent the adult from accidentally consecutively selecting two response buttons of the at least two pairs of response buttons with the desired pairing, prior to previously hearing the stimuli for each of the selected response buttons. Thus, in some embodiments, a stimulus may be associated with a response button only after the response button has been selected. In one embodiment, the method may include detecting if a chance pairing of response buttons occurs, and if so, changing an association of a stimulus with a selected response button to prevent the chance pairing.
In 3814, the method elements 3804-3812 may be repeated until all of the at least two pairs of response buttons have been removed from the grid. In one embodiment, the adult may complete a trial when they have removed all of the at least two pairs of response buttons that have pairs of stimuli associated with them. In another embodiment, a trial may be ended if the adult fails to remove all of the at least two pairs of response buttons within a predetermined number of selections. In one embodiment, a trial may include the presentation of such a plurality of response buttons (in a grid), and may continue as described above until either all of the presented response buttons have been removed, or the adult exceeds a predetermined number of selections for the trial. In other words, a trial may include the above-described method elements repeated until either all the buttons have been matched and removed, or until some maximum number of selections by the adult have been made.
As indicated in 3816, in some embodiments, the method elements 3802-3814 may be performed for each of the remaining grids of the first plurality of grids, and a first measure of the adult's performance with respect to the first plurality of grids determined. For example, in one embodiment, the first measure may include the average number of selections made by the adult to clear a grid, e.g., to match all the stimulus pairs of the grid, over all the grids of the first plurality of grids. Thus, the first measure may characterize the adult's performance with regard to the first plurality of grids.
In 3818, the method elements 3802-3814 may be performed for each grid of a second plurality of grids. In other words, the steps of graphically presenting (3802) through repeating (3814) may be performed for each grid of a second plurality of grids.
In one embodiment, the second plurality of grids may be directed to training the adult, i.e., via the Match It exercise described above, whereas the first plurality of grids may be directed to assessment or estimation of the adult's capabilities. For example, in one embodiment, the first plurality of grids may include grids that are representative of the grids in the second plurality of grids, e.g., may include grids in accordance with a representative sample of parameter values for the exercise used in training the adult. For example, in one embodiment, the first plurality of grids may have a first processing level, e.g., level 5 (which may be the level closest to human speech), whereas the grids used in training (the second plurality of grids) may include grids at all processing levels, ranging from heavily processed synthetic speech (e.g., level 1) through natural or near-natural processed speech (e.g., level 5).
For example, in one embodiment, performing the steps of graphically presenting through repeating for each grid of a second plurality of grids may include providing a plurality of processing levels for processing the computer generated stimuli, and beginning with a first processing level for aurally presenting the computer generated stimuli which makes the stimuli easy to distinguish. As the adult successfully pairs the stimuli, the amount of processing may be altered towards a last processing level, where, for example, the last processing level is that of normal speech. Additionally, in some embodiments, as the adult successfully pairs the stimuli, the number of response buttons presented in a trial may be increased. Thus, during training (with the second plurality of grids), the trials may be made more difficult as the adult progresses.
In some embodiments, to prevent the adult's memory of grid layout from trial to trial from influencing the adult's performance, the first plurality of grids may include various subsets of grids with different parameters, e.g., size, processing level, stimulus category, etc., where consecutive trials may use grids from different subsets. For example, in one embodiment, the first plurality of grids may include a first subset of grids, referred to as primary stimuli, where each grid has a first size and a first category of stimuli, and a second subset of grids, referred to as secondary stimuli, where each grid has a second size and a second category of stimuli. Thus, trials with respect to the first plurality of grids may alternate between the first subset of grids (primary stimuli) and the second subset of grids (secondary stimuli) to avoid effects of spatial memory of an immediately previous trial.
In 3820, the above-described method elements 3802-3814 may be performed (again) for each grid of the first plurality of grids. In other words, the steps of graphically presenting (3802) through repeating (3814) may be performed for each grid of the first plurality of grids. A second measure of the adult's performance with respect to the first plurality of grids may then be determined based on the performing the steps of graphically presenting through repeating for each grid of the first plurality of grids. In other words, trials directed to the assessment grids (the first plurality of grids) may be performed a second time (after the training trials directed to the second plurality of grids), and the adult's performance characterized, e.g., by an average number of selections made by the user to clear each grid.
Finally, in 3822, a measure of the adult's improvement may be determined based on the first measure and the second measure. For example, a ratio of the first measure to the second measure may be computed, where a value greater than 1 indicates improvement, presumably due to the intervening training session. In another embodiment, a difference between the two measures may be computed, indicating a difference in average performance from the first assessment to the second assessment. Of course, these particular measures are meant to be exemplary only, and are not intended to limit the types of measures nor the means for determining such measures to any particular type or technique.
Stimuli Specifications
The following describes exemplary stimulus data sets suitable for use in the above-described method. It should be noted, however, that the data described is meant to be exemplary only, and is not intended to limit the data used in the invention to any particular data sets or attributes.
In one embodiment, the primary stimuli or first subset of the first plurality of grids may be characterized by a grid size of 4×4, with 16 total sounds (8 pairs of distinct sounds). Moreover, the computer generated stimuli may have a sound processing level that is closest to human speech, e.g., a sound processing level of 5, from a range of 1 to 5, where a processing level of 1 is heavily processed to emphasize distinguishing attributes of the aurally presented stimuli. The stimulus category may be set at category level 2, which may include the syllables: ‘fig’, ‘rib’, ‘sit’, ‘kiss’, ‘bill’, ‘dish’, ‘nut’, ‘chuck’, ‘rug’, ‘dust’, ‘pun’, ‘gum ’,‘bash’, ‘can’, ‘gash’, ‘mat’, ‘lab’, and ‘nag’, although in other embodiments, other syllables may be used as desired.
In one embodiment, the secondary stimulus or second subset of the first plurality of grids may be characterized by a grid size of 3×2, with 6 total sounds (3 pairs of distinct sounds). The computer generated stimuli may have a sound processing level that is closest to human speech, e.g., a sound processing level of 5, from a range of 1 to 5, where, as described above, a processing level of 1 is heavily processed to emphasize distinguishing attributes of the aurally presented stimuli. The stimulus category may be set at category level 4 from among a plurality of stimulus categories, where category 4 may include the words (syllables): ‘buck’, ‘bud’, ‘but’, ‘cup’, ‘cut’, ‘duck’, ‘dug ’, ‘pug’, ‘pup’, ‘tub’, ‘tuck’, ‘tug’, ‘bug’, ‘cud’, ‘puck’, ‘dud’, ‘gut’, ‘guck’, although in other embodiments, other words may be used as desired.
In one exemplary embodiment, during the assessment portion of the method, the adult may be presented with 9 grids, 5 primary grids and 4 secondary grids, where, as described above, beginning with a primary grid, the grids may alternate between primary and secondary, the purpose of alternating being to remove or ameliorate any spatial memory of the previous trial. Of course, other numbers of grids may be used as desired.
In moving through the task or method, the adult may click a ‘start’ button on the screen to begin. Once presented with the first grid, the adult may click or select each button in any order or sequence. As the adult clicks on matching buttons in succession, the buttons may be removed from the screen. Once all the buttons are gone, the adult may be presented with a ‘start’ button to begin the next grid. In other embodiments, once each grid is completed, the next grid may be presented automatically, i.e., without the need for the adult to click or select a start button.
Sound Replay Threshold Assessment
A primary purpose of the Sound Replay threshold assessment described herein is to determine the number of items a participant can remember in a serial order, specifically, the number of syllables, which corresponds to stage, that a person can respond correctly to above a statistical threshold. The Sound Replay threshold assessment is very similar to the Sound Replay exercise described above, but with several key distinctions, as will be described below. For example, the Sound Replay assessment may be similar to the Sound Replay exercise with respect to visual presentation. At least some of the differences between the assessment and the exercise may be with regard to movement through the task and the data that are obtained from this movement for the assessment. In a preferred embodiment, the task (of the assessment) is designed to obtain a threshold, which is a statistical rather than an exact quantity. For the purposes of this task, “threshold” is defined as the number of sounds (e.g., stage) at which a participant will fail to respond correctly some specified percentage, e.g., 47.5%, of trials. Sound Replay assessment is similar to neuropsychological tasks “digit span” and “digit span backwards”, in which subjects must remember the auditory information over short periods of time to identify matching syllables in a temporal domain. Being a computer based task, Sound Replay assessment may use the ZEST algorithm to move through the task, adjust the number of sounds to be presented, and determine the statistical threshold, as described below in more detail.
In the exemplary embodiments described below, a primary goal is to statistically determine the stage value where a participant will fail to respond correctly 47.5% of trials, as mentioned above.
In 4002, first and second tracks may be initialized with respective stage values based on an initial anticipated threshold, where a stage specifies a set of stimulus attributes, and where the initial anticipated threshold specifies an initial estimate of stimulus attributes corresponding to a specified performance level of the adult, e.g., stimulus attributes at which the adult fails to respond correctly some specified percentage of the time, e.g., 47.5%. For example, in one embodiment, the first track may be initialized to a stage value that is below the initial anticipated threshold, e.g., preferably just slightly below the initial anticipated threshold, and the second track may be initialized to a second stage value that is (e.g., slightly) above the initial anticipated threshold. Thus, the initial stage values of the two tracks may straddle the initial anticipated threshold. In one embodiment, the method may include determining the initial anticipated threshold, e.g., based on one or more of: the age of the adult, calibration trials performed by the adult, and/or calibration trials performed by other adults, among others.
In some embodiments, an initialization process may operate to initialize other items as well. For example, the initialization process may include initializing one or more of: the first and second tracks' initial stage values (as indicated above), a standard deviation of a cumulative Gaussian psychometric function for the maximum likelihood procedure, or a standard deviation of a prior threshold distribution for the maximum likelihood procedure.
Note that there may be a specified number of stages used in the method, e.g., stages 1-8, where each stage specifies or is associated with stimulus attributes, as indicated above. For example, in one embodiment, the attributes may include one or more of: a respective number of distinct syllables in the plural set of syllables, a respective number of repeat syllables in the plural set of syllables, a respective number of distractor buttons presented in a trial, and/or a respective inter-stimulus-interval (ISI), denoting a time interval between successive syllables, among others.
In 4004, a plural set of syllables from a first plurality of syllables may be aurally presented on the computing device in accordance with the stage value of a specified one of either the first track or the second track, where the plural set of syllables are presented serially, e.g., where serially indicates that the plural set of syllables are aurally presented, one at a time, e.g., one after another, until all of the syllables in the plural set of syllables have been presented. In preferred embodiments, the aural presentations may be made via headphones attached to the computing device, although speakers may be used as desired. In some embodiments, the syllables may include consonant-vowel-consonant syllables, and/or may include phonemes. In one embodiment, the syllables may be processed to emphasize and stretch rapid frequency transitions, although in some embodiments, the processing used may be minimal, or even none, e.g., to approximate natural human speech.
In 4006, after the aurally presenting of 4004, the plural set of syllables may be graphically presented on the computing device, e.g., via a GUI. For example, in one embodiment, the graphically presenting may include providing a graphical icon (e.g., a graphical response button) for each one of the plural set of syllables, and displaying graphical icons on the computing device that correspond to the aurally presented syllables. The displayed graphical icons may be selectable by the adults using a pointer on the computing device. Moreover, in some embodiments, the graphically presenting may also include graphically presenting distracter syllables along with the first plurality of syllables. The distracter syllables may be provided to the adult to allow the adult to make incorrect selections, i.e., may present incorrect choices for selection by the adult.
In 4008, the adult may be required to select on the computing device the graphically presented syllables corresponding to an order in which they were aurally presented. In other words, the adult may be required to select the graphical icons or buttons representing each aurally presented syllable in the order in which they were presented. Note that the distracter syllables mentioned above may thus be provided to the adult to make the requiring more difficult, since the graphically presented icons or buttons include these incorrect responses among those corresponding to the aurally presented syllables.
In 4010, the stage value of the specified track may be modified based on the adult's response. Modifying the stage value of the specified track based on the adult's response may include modifying the stage value in accordance with a maximum likelihood procedure, such as, for example, a QUEST (quick estimation by sequential testing) threshold procedure, or a ZEST (zippy estimation by sequential testing) threshold procedure, described above. In one embodiment, for each track, modifying the stage value of the specified track based on the adult's response may include increasing the stage value if the adult responds correctly in a specified percentage of trials, and decreasing the stage value if the adult responds incorrectly in the specified percentage of trials. For example, in one exemplary embodiment, if the player obtains a 90% or over correct rate on the first 10 trials, the track's initial values may be increased, and if the player obtains a 90% or over incorrect rate on the first 10 trials, the track's initial values may be decreased, and the assessment may resume, or start over, with the new initial values.
Note that while each stage is discrete, i.e., has an integer value, the stage values determined in the method may not always be integers, and may not always be modified or adjusted in integer amounts. For example, for each iteration of the assessment process described herein (or periodically per some specified number of trials), the method may determine a real (non-integral) stage value. This value may be the true updated stage value, but its rounded value may used to specify the stage to use for the next trial, e.g., a stage value of 3.4 may specify use of stage 3 for the trial, while a stage value of 3.6 may specify use of stage 4 for the trial. In one embodiment, the initial stage values may chosen such that if they are too easy (or too difficult), the next stage used will immediately be incremented (or decremented) one full stage. For example, assuming an adjustment increment/decrement of approximately 0.2, if 3.6 (which specifies stage 4) were found to be too difficult, the next value would be about 3.4 (specifying stage 3), whereas if 4 had been specified as the initial stage value and the trial were found to be too difficult, the next value (3.8) would still round to 4, and so no effective change would occur. Thus, the initial stage values may be set to real values.
As noted above, the initial anticipated threshold, the first stage value, the second stage value, and the threshold (determined below) may each specify or be associated with one or more of: a respective number of distinct syllables in the plural set of syllables, a respective number of repeat syllables in the plural set of syllables, a respective number of distractor buttons presented in a trial, and/or a respective inter-stimulus-interval (ISI), denoting a time interval between successive syllables. Thus, for each track, increasing the stage value may include one or more of: increasing the respective number of distinct syllables in the plural set of syllables, increasing the respective number of repeat syllables in the plural set of syllables, increasing the respective number of distractor buttons presented in a trial, or decreasing the respective ISI (the time interval between successive syllables). Similarly, for each track, decreasing the stage value may include one or more of: decreasing the respective number of distinct syllables in the plural set of syllables, decreasing the respective number of repeat syllables in the plural set of syllables, decreasing the respective number of distractor buttons presented in a trial, or increasing the respective inter-stimulus-interval. Thus, increasing the stage value for a track may operate to make subsequent trials performed in that track more difficult, while decreasing the stage value for a track may operate to make subsequent trials performed in that track less difficult. Note that the adult's response to the trial conducted at a track's value may thus determine that track's next stage value via the maximum likelihood method.
In 4012, the aurally presenting, graphically presenting, requiring, and modifying (i.e., method elements 4004-4010) may be repeated (i.e., performed) one or more times in an iterative manner with respect to other plural sets of syllables from the first plurality of syllables to determine respective final stage values for the first track and the second track.
Finally, in 4014, a threshold for the adult may be determined based on the respective final stage values for the first track and the second track, where the threshold is or includes the stage value associated with the specified performance level of the adult.
In various embodiments, repeating the presenting, requiring, and modifying, one or more times in an iterative manner may include performing trials in the first track and the second track in an alternating manner, or performing trials in the first track and the second track randomly, e.g., with equal probability. Note that in either approach, over many trials the number of trials performed in each track are equal or at least approximately equal. Moreover, in accordance with QUEST or ZEST threshold determination techniques, repeating the presenting, requiring, and modifying, one or more times in an iterative manner may include repeating the presenting, requiring, and modifying, until the stage values of the first track and the second track have converged to values within a specified confidence interval, and where the values are within a specified distance from each other, or, until a specified number of trials have been conducted for each track (e.g., 20 per track).
In preferred embodiments, determining a threshold for the adult based on the respective final stage values for the first track and the second track may include averaging the respective final stage values for the first track and the second track to determine the threshold for the adult. For example, averaging the respective final stage values for the first track and the second track to determine the threshold for the adult may include rounding the average of the respective final stage values for the first track and the second track to determine the threshold for the adult. Thus, a stage value of 4.6 corresponds to stage 5, a stage value of 4.2 corresponds to stage 4, and so forth.
In one embodiment, the presenting, requiring, and modifying described above may compose performing a trial. Certain information may be saved with respect to each trial performed, e.g., for operation of the method, and/or for subsequent analysis of the assessment. For example, for each trial, one or more of the following may be saved: which track was used in the trial, the stage value used in the trial, the series of syllables presented to the adult, the button to syllable assignments, the series of button selections by the adult, the correctness or incorrectness of the adult's response, the mean of a posterior probability distribution function for the maximum likelihood procedure, and the standard deviation of the posterior probability distribution function for the maximum likelihood procedure. Of course, any other type of information may also be saved as desired.
As indicated above, the adult may interact with the assessment process via a GUI presented on the computing device, similar to the GUI used in the training exercise (Sound Replay). However, some aspects of the exercise version of Sound Replay may not be necessary in the Sound Replay assessment. For example, a progress bar, points bar, and reward area included in the training exercise GUI (of Sound Replay) are not necessary, since progress/reward indicators are not used in the assessment. However, in some embodiments, some of the GUI elements or assets may remain the same, such as the response buttons and the “ding” and “thump” sounds that play after an adult responds correctly or incorrectly. The stimulus presentation may also be identical to the exercise version. In one embodiment, initially the only button in the screen is the start button. However, because the adult will not already know the number of sounds to be presented for each trial, the number of sounds to be played may be displayed in the space of the response buttons.
After the participant clicks on the start button, the sounds (syllables) may be played and the buttons displayed in random order, as described above.
Stimuli Specifications
The following describes exemplary stimulus data sets suitable for use in the above-described method. It should be noted, however, that the data described is meant to be exemplary only, and is not intended to limit the data used in the invention to any particular data sets or attributes.
In one embodiment, for assessment purposes, the sound processing level for presentation of the syllables may be set at level 5, where level 5 is the level closest to human speech. This is in contrast to the processing levels used in the normal training exercise (Sound Replay), ranging from heavily processed synthetic speech (e.g., level 1), e.g., to emphasize distinguishing attributes of the aurally presented stimuli, through natural or near-natural processed speech (e.g., level 5).
In some embodiments, the stimulus category may be set at category level 2, from among a plurality of available categories, where category 2 may include the syllables: ‘fig’, ‘rib’, ‘sit’, ‘kiss’, ‘bill’, ‘dish’, ‘nut’, ‘chuck’, ‘rug’, ‘dust’, ‘pun’, ‘gum’,‘bash’, ‘can’, ‘gash’, ‘mat’, ‘lab’, and ‘nag’, although in other embodiments, other syllables may be used as desired.
As noted above, in preferred embodiments, the stage parameters may be the same as the stage parameters for the Sound Replay exercise. Exemplary stage parameters and their ranges are provided below:
Initialization Values
In one embodiment, initial values for the assessment process may be set at:
A primary purpose of the Listen and Do threshold assessment described herein is to determine the number of instructions (associated with a stage value) an aging adult can remember and perform in a serial order correctly above a statistical threshold. The Listen and Do assessment is similar to the Listen and Do exercise described above with respect to visual presentation. In preferred embodiments, the differences between the assessment and the training exercise are primarily in the movement or progression through the task and the data obtained for the assessment. The task is designed to determine a threshold, which is a statistical rather than an exact quantity. For the purposes of this task, “threshold” is defined, as the number of instructions (e.g., stage) at which a participant fails to respond correctly some specified percentage of trials, e.g., 47.5%, with processing level and stimulus category fixed. Note that the Listen and Do assessment described herein may be similar to neuropsychological tasks “digit span” and “digit span backwards”, in which subjects must remember auditory information over short periods of time. Being a computer-based task, Listen and Do assessment preferably uses the ZEST algorithm (described above) to move through the task, adjust the number of instructions to be presented, and determine the statistical threshold, as will be described below in detail.
In the exemplary embodiments described below, a primary goal of the Listen and Do assessment is to statistically determine the stage value where a participant will fail to respond correctly 47.5% of trials, as mentioned above.
In 4302, first and second tracks may be initialized with respective stage values based on an initial anticipated threshold, where a stage specifies a set of stimulus attributes, and where the initial anticipated threshold specifies an initial estimate of stimulus attributes corresponding to a specified performance level of the adult, e.g., stimulus attributes at which the adult fails to respond correctly some specified percentage of the time, e.g., 47.5%. For example, in one embodiment, the first track may be initialized to a stage value that is below the initial anticipated threshold, e.g., preferably just slightly below the initial anticipated threshold, and the second track may be initialized to a second stage value that is (e.g., slightly) above the initial anticipated threshold. Thus, the initial stage values of the two tracks may straddle the initial anticipated threshold. In one embodiment, the method may include determining the initial anticipated threshold, e.g., based on one or more of: the age of the adult, calibration trials performed by the adult, and/or calibration trials performed by other adults, among others.
In some embodiments, an initialization process may operate to initialize other items as well. For example, the initialization process may include initializing one or more of: the first and second tracks' initial stage values (as indicated above), a standard deviation of a cumulative Gaussian psychometric function for the maximum likelihood procedure, or a standard deviation of a prior threshold distribution for the maximum likelihood procedure.
In 4304, a sequence of one or more auditory instructions that have been processed by the computer may be presented, e.g., via headphones and/or speakers connected to the computing device, where the auditory instructions indicate a corresponding sequence of actions to be performed by the adult. The actions may include or specify manipulations of graphical objects displayed on a display of a computing device. Note that in preferred embodiments, the graphical objects displayed with respect to a sequence of instructions may include a confusable superset of objects referred to in the instructions, i.e., various of the objects may be easily confused by the adult.
The presenting of the sequence of instructions may be made in accordance with the stage value of a specified one of either the first track or the second track. Note that there may be a specified number of stages used in the method, where each stage specifies or is associated with stimulus attributes, as indicated above. For example, in one embodiment, the attributes may include or specify a respective number of distinct instructions in the sequence of instructions, i.e., the length of the sequence. Thus, the initial stage values for the two tracks, the initial anticipated threshold, and the threshold (determined below) may each specify or be associated with a respective number of distinct instructions in the sequence of instructions.
In one embodiment, the sequence of instructions may be selected from a plurality of sequences of instructions, where the plurality of sequences of instructions has a specified stimulus category, i.e., where the plurality of sequences of auditory instructions is from a specified category from a plurality of categories, each category specifying a type of sequences of instructions. For example, in the Listen and Do training exercise described above, sequences of instructions from a variety of categories are used, such as: category 1, where each of the instructions is of lower complexity and is directed to stimulus discrimination where the adult is to select a respective graphical object with a pointing device; category 2, where each of the instructions is of higher complexity and is directed to stimulus discrimination where the adult is to select a respective graphical object with a pointing device, wherein the respective graphical object is qualified by one or more adjectives and/or one or more prepositional phrases; category 3, where each of the instructions is of lower complexity and is directed to memory, and includes an instruction to move a respective first graphical object with respect to a respective second graphical object; and category 4, where each of the instructions is of higher complexity and is directed to memory, and includes an instruction to move a respective first graphical object with respect to a respective second graphical object, where one or both of the respective first graphical object and the respective second graphical object are qualified by one or more adjectives and/or one or more prepositional phrases.
In a preferred embodiment, the sequences used in the Listen and Do assessment described herein may be sequences from category 2. In other words, each of the instructions (in the sequence) may include an instruction to select a respective graphical object with a pointing device, where the respective graphical object may be qualified by one or more adjectives and/or one or more prepositional phrases. Of course, in other embodiments, other sequences of instructions, possibly in other categories, may be used as desired.
As noted above, in some embodiments, the aurally presented instruction sequences may be processed to some degree, e.g., to facilitate understanding by the adult. For example, in the Listen and Do training exercise, there are a number of different processing levels, ranging from heavily processed synthetic speech (e.g., level 1), e.g., to emphasize distinguishing attributes of the aurally presented stimuli, through natural or near-natural processed speech (e.g., level 5), where, for example, the level of processing of the auditory instructions may be based one or more of: 1) modifying a rate at which at least a portion of the auditory instructions are played, while maintaining characteristic pitch-pulse-phase synchronous temporal structure of voiced speech sounds, and 2) emphasizing portions of the auditory instructions, using band-modulation deepening to selectively enhance relatively fast-changing events in the voiced speech sounds.
In preferred embodiments, for assessment purposes, the sound processing level for presentation of the instructions may be set at level 5, where level 5 is the level closest to human speech. In other words, each of the plurality of sequences of auditory instructions may be aurally presented with a specified processing level substantially corresponding to natural speech. This is in contrast to the multiple processing levels used in the normal training exercise (Listen and Do). Of course, in other embodiments, any processing levels may be used as desired. Thus, in some embodiments, each of the plurality of sequences of auditory instructions may have a respective difficulty based a stage, corresponding to the number of instructions in the sequence, and a level of processing of the auditory instructions, where the level of processing corresponds to the degree to which the auditory instructions have been processed to enhance clarity of the auditory instructions.
In 4306, input may be received from the adult, where the input manipulates the graphical objects on the display. For example, the adult may click on various items displayed in a GUI presented on a display of the computing device.
In one embodiment, the method may include indicating whether the adult correctly performed the sequence of actions. For example, a respective sound indicating correctness or incorrectness may be presented to the user, e.g., a “ding” indicating correctness, and a “thunk” indicating incorrectness of the response. Additionally, or alternatively, a respective graphical indication of correctness or incorrectness of the response may be presented, e.g., a “success” image, icon, or animation.
In 4308, the stage value of the specified track may be modified based on the adult's response. For example, modifying the stage value of the specified track based on the adult's response may include modifying the stage value in accordance with a maximum likelihood procedure, e.g., in accordance with a QUEST (quick estimation by sequential testing) threshold procedure, or a ZEST (zippy estimation by sequential testing) threshold procedure, as described above in some detail.
In one embodiment, for each track, modifying the stage value of the specified track based on the adult's response may include increasing the stage value if the adult responds correctly in a specified percentage of trials, and decreasing the stage value if the adult responds incorrectly in the specified percentage of trials. For example, similar to the Sound Replay assessment described above, in one exemplary embodiment, if the player obtains a 90% or over correct rate on the first 10 trials, the track's initial values may be increased, and if the player obtains a 90% or over incorrect rate on the first 10 trials, the track's initial values may be decreased, and the assessment may resume, or start over, with the new initial values.
As also described above, it should be noted that while each stage is discrete, i.e., has an integer value, the stage values determined in the method may not always be integers, and may not always be modified or adjusted in integer amounts. For example, for each iteration of the assessment process described herein (or periodically per some specified number of trials), the method may determine a real (non-integral) stage value. This value may be the true updated stage value, but its rounded value may used to specify the stage to use for the next trial, e.g., a stage value of 3.4 may specify use of stage 3 for the trial, while a stage value of 3.6 may specify use of stage 4 for the trial. In one embodiment, the initial stage values may chosen such that if they are too easy (or too difficult), the next stage used will immediately be incremented (or decremented) one full stage. For example, assuming an adjustment increment/decrement of approximately 0.2, if 3.6 (which specifies stage 4) were found to be too difficult, the next value would be about 3.4 (specifying stage 3), whereas if 4 had been specified as the initial stage value and the trial were found to be too difficult, the next value (3.8) would still round to 4, and so no effective change would occur. Thus, the initial stage values may be set to real values.
As noted above, the initial anticipated threshold, the first stage value, the second stage value, and the threshold (determined below) may each specify or be associated with a respective number of distinct instructions in the sequence of instructions. Thus, for each track, increasing the stage value may include increasing the number of instructions in the sequence. Similarly, for each track, decreasing the stage value may include decreasing the number of instructions in the sequence. Increasing the stage value for a track may operate to make subsequent trials performed in that track more difficult, while decreasing the stage value for a track may operate to make subsequent trials performed in that track less difficult. Note that the adult's response to the trial conducted at a track's value may thus determine that track's next stage value via the maximum likelihood method.
In 4310, the presenting, receiving, and modifying described above may be performed with respect to each of a plurality of sequences of auditory instructions in an iterative manner to determine respective final stage values for the first track and the second track. In other words, method elements 4304-4308 may be performed for each of a plurality of instruction sequences to determine final stage values of the two tracks.
For example, in one embodiment, performing the presenting, receiving, and modifying, one or more times in an iterative manner may include performing trials in the first track and the second track in an alternating manner, or performing trials in the first track and the second track randomly with equal probability. As noted above, in either approach, over many trials the number of trials performed in each track are equal or at least approximately equal. Moreover, in accordance with QUEST or ZEST threshold determination techniques, repeating the presenting, receiving, and modifying, one or more times in an iterative manner may include repeating the presenting, receiving, and modifying, until the stage values of the first track and the second track have converged to values within a specified confidence interval, and where the values are within a specified distance from each other, or, until a specified number of trials have been conducted for each track (e.g., 20 per track).
In 4312, a threshold for the adult may be determined based on the respective final stage values for the first track and the second track, where the threshold is or includes the stage value associated with the specified performance level of the adult. In other words, a threshold stage value may be determined that specifies stimulus attributes at which the adult will generally fail some specified percentage of the time, e.g., 47.5%.
In preferred embodiments, determining a threshold for the adult based on the respective final stage values for the first track and the second track may include averaging the respective final stage values for the first track and the second track to determine the threshold for the adult. For example, averaging the respective final stage values for the first track and the second track to determine the threshold for the adult may include rounding the average of the respective final stage values for the first track and the second track to determine the threshold for the adult. Thus, a stage value of 4.6 corresponds to stage 5, and a stage value of 4.2 corresponds to stage 4.
In one embodiment, the presenting, receiving, and modifying described above may compose performing a trial. Certain information may be saved with respect to each trial performed, e.g., for operation of the method, and/or for subsequent analysis of the assessment. For example, for each trial, one or more of the following may be saved: which track was used in the trial, the stage value used in the trial, the series of auditory instructions presented to the adult, button-to-sound assignments, the series of selections by the adult, the correctness or incorrectness of the adult's response, the mean of a posterior probability distribution function for the maximum likelihood procedure, and the standard deviation of the posterior probability distribution function for the maximum likelihood procedure. Of course, any other type of information may also be saved as desired.
As indicated above, the adult may interact with the assessment process via a GUI presented on the computing device, similar to the GUI used in the training exercise (Listen and Do). However, some aspects of the exercise version of Listen and Do may not be necessary in the Listen and Do assessment. For example, a progress bar, points bar, and reward area included in the training exercise GUI (of Listen and Do) are not necessary, since progress/reward indicators may not be used in the assessment. However, in some embodiments, some of the GUI elements or assets may remain the same, such as the response buttons and the “ding” and “thump” sounds that play after an adult responds correctly or incorrectly. In some embodiments, the stimulus presentation may also be identical to the exercise version. In one embodiment, initially the only button in the screen is the start button. However, because the adult will not already know the number of instructions to be presented for each trial, in some embodiments, the number of instructions to be presented (and followed) may be displayed at the beginning of each trial so the adult knows how many instructions to expect to receive. After the adult clicks on the start button, the instructions may be played and the interface may be activated after the entire set of instructions is given, i.e., the various graphic objects may be selectable by the adult to carry out the instructions.
Stimuli Specifications
The following describes exemplary stimulus data sets suitable for use in the above-described assessment method. It should be noted, however, that the data described is meant to be exemplary only, and is not intended to limit the data used in the invention to any particular data sets or attributes.
In one embodiment, for assessment purposes, the sound processing level for presentation of the instructions may be set at level 5, where level 5 is the level closest to human speech. This is in contrast to the processing levels used in the normal training exercise (Listen and Do), ranging from heavily processed synthetic speech (e.g., level 1), e.g., to emphasize distinguishing attributes of the aurally presented stimuli, through natural or near-natural processed speech (e.g., level 5).
In preferred embodiments, for assessment purposes, the sound processing level for presentation of instructions may be set at level 5, which is the level in which sounds are not emphasized but simply sped up. To avoid problems the adult might have with dragging and dropping items, in preferred embodiments, the stimulus category may be set at category 2, where the adult is only asked to select (i.e., click on) various objects on the interface. Thus, drag and drop commands may not be used for assessment purposes. Note that category 2 includes characters that are descriptive, e.g. the girl in the purple (green, red) dress, the tall (short) police officer, Washington (Lincoln) library, etc.). Said another way, in some embodiments, the stimulus category may be set at category 2, from among a plurality of available categories (see above), where, as described above, category 2 may include instructions to identify or select (e.g., click on) graphical objects or items presented on the display of the computing device, where the graphical objects in the instructions are modified by one or more adjectives and/or one or more prepositional phrases, although in other embodiments, other types of instructions may be used as desired.
As noted above, in preferred embodiments, the stage parameters may be the same as the stage parameters for the Listen and Do exercise, and may represent or correspond to the number of instructions given. For example, an exemplary stage 3 sequence of instructions may include three qualified selection instructions, e.g., click on the brown dog and then click on man in the red hard hat and then click on the sunshine bakery.
Initialization Values
In one embodiment, initial values for the assessment process may be set at:
However, it should be noted that in other embodiments, other values may be used as desired.
Assessment and Exercise Performance
Maximum likelihood procedure based psychophysical threshold determination, such as described above with respect to various exemplary cognitive training exercises, may facilitate more effective use of such exercises by establishing a stimulus intensity for each exercise that is substantially optimal for improving the cognitive skills of the adult subject.
In 4602, a psychophysical threshold for an aging adult with respect to stimuli in a cognitive training exercise may be determined. As noted above, the threshold may comprise a stimulus intensity value associated with a specified performance level of the adult, and may be determined using a maximum likelihood procedure, such as, for example, a QUEST (quick estimation by sequential testing) threshold procedure, or a ZEST (zippy estimation by sequential testing) threshold procedure, as described above.
For example, similar to the assessments described above, in one embodiment, determining the psychophysical threshold may include initializing a first track to a first intensity value that is below an initial anticipated threshold, where the initial anticipated threshold comprises or includes an initial estimate of a stimulus intensity value for stimuli corresponding to a specified performance level of the adult. A second track may be initialized to a second stimulus intensity value that is above the initial anticipated threshold. A stimulus may then be presented to the adult via the computing device in accordance with the stimulus intensity value of a specified one of either the first track or the second track. For example, the stimuli may be presented via one or more of: headphones attached to the computing device, speakers attached to the computing device, and/or a display device attached to the computing device. A response to the stimulus may then be received from the adult via the computing device, and the stimulus intensity value of the specified track modified based on the adult's response in accordance with the maximum likelihood procedure. For example, in one embodiment, for each track, modifying the stimulus intensity value of the specified track based on the adult's response may include increasing the stimulus intensity value if the adult responds correctly in a specified percentage of trials, and decreasing the stimulus intensity value if the adult responds incorrectly in the specified percentage of trials.
In one embodiment, an indication of whether the adult responded correctly to the stimulus may be provided. For example, a respective sound and/or graphical indication indicating correctness or incorrectness may be presented. Additionally, or alternatively, points may be awarded (or possibly subtracted) based on the correctness of the adult's response.
This presenting, receiving, and modifying may be performed with respect to each of a plurality of stimuli in an iterative manner to determine respective final stimulus intensity values for the first track and the second track, after which a threshold for the adult may be determined based on the respective final stimulus intensity values for the first track and the second track, where the threshold is or includes the stimulus intensity value associated with the specified performance level of the adult. In other words, an assessment version of the cognitive training exercise may be performed to determine the psychophysical threshold for the adult with respect to that exercise. For example, determining a threshold for the adult based on the respective final stimulus intensity values for the first track and the second track may include averaging the respective final stimulus intensity values for the first track and the second track to determine the threshold for the adult. In one embodiment, averaging the respective final stimulus intensity values for the first track and the second track to determine the threshold for the adult may include rounding the average of the respective final stimulus intensity values for the first track and the second track to determine the threshold for the adult. Please see the above-described assessments for particular examples of such threshold determination with respect to specific exercises.
In one embodiment, performing the presenting, requiring, and modifying, one or more times in an iterative manner may include performing trials in the first track and the second track in an alternating manner, or, alternatively, performing trials in the first track and the second track randomly with equal probability, the idea being to perform substantially the same number of trials in each track over the duration of the determination process. In some embodiments, the presenting, requiring, and modifying, may be performed (iteratively) until either the stimulus intensity values of the first track and the second track have converged to stimulus intensity values within a specified confidence interval, where the stimulus intensity values are within a specified distance from each other, or a specified number of trials have been conducted for each track. In other words, in some embodiments, trials may be performed until one of these conditions obtains.
In one embodiment, the presenting, requiring, and modifying described above may compose performing a trial (in the determination process or assessment exercise of 4602). Moreover, the method may include saving various parameters or values for each trial performed, including, for example, one or more of: which track was used in the trial; the stimulus intensity value used in the trial; the stimulus presented to the adult; button to response assignments; the series of button selections by the adult; the correctness or incorrectness of the adult's response; the mean of a posterior probability distribution function for the maximum likelihood procedure; and the standard deviation of the posterior probability distribution function for the maximum likelihood procedure, among others.
Additionally, in some embodiments, the method may further include initializing various parameters, such as, for example, one or more of: initial stimulus intensity values of the first and second tracks; the standard deviation of a cumulative Gaussian psychometric function for the maximum likelihood procedure; and the standard deviation of a prior threshold distribution for the maximum likelihood procedure, among others.
In one embodiment, the method may include determining the initial anticipated threshold based on one or more of: the age of the adult, calibration trials performed by the adult, and/or calibration trials performed by other adults.
In 4604, a plurality of trials in the exercise may be performed with stimuli at or near the determined threshold to improve the adult's cognition and memory skills. In other words, the exercise may be performed based on the determined threshold. For example, the adult may be trained through repetitive trials at or near the determined threshold to increase the benefit of performing trials in the cognitive training exercise.
In some embodiments, performing a plurality of trials in the exercise with stimuli at or near the determined threshold may include: providing a set of stimuli for the cognitive training exercise. For each stimulus in the set of stimuli, the stimulus may be presented to the adult at or near the determined threshold via the computing device. A response to the presented stimulus may be received from the adult via the computing device, and a determination made as to whether the response is correct. An indication may then be provided as to whether the response is correct. The presenting, receiving, determining, and indicating may be repeated for each stimulus in the set of stimuli in an iterative manner to improve the adult's cognitive and memory skills. In one embodiment, this presenting, receiving, determining, and indicating for each stimulus in the set of stimuli may include increasing a difficulty level of the stimulus as the adult progresses through the exercise. Note that in preferred embodiments, the stimuli used in determining the psychophysical threshold may be or include a representative subset of the set of stimuli used in performing the plurality of trials in the exercise.
In one embodiment, presenting the stimulus to the adult at or near the determined threshold via the computing device may include presenting aural stimuli, e.g., spoken words phonemes, instructions, etc., and/or visual stimuli, e.g., images, text, etc., to the adult. Various examples of each are provided above with respect to the different exercise assessments described herein.
In preferred embodiments, the method may further include repeating the determining the psychophysical threshold and performing the plurality of trials in the exercise one or more times in an iterative manner to improve the adult's cognitive and memory skills. For example, the repetitions may be performed over a plurality of sessions, e.g., over days, weeks, or even months, as noted above.
It should be noted that any of the techniques, parameters, and aspects disclosed above with respect to any of the various exercises and assessment methods described herein may be used with respect to any others of the exercises and assessment methods, as desired. In other words, any of the particular details described above with respect to any specific method may be used with respect to any of the other methods disclosed herein as desired, the above descriptions being meant to be exemplary only, and not to restrict embodiments of the invention to any particular form, appearance, or function.
Moreover, although the present invention and its objects, features, and advantages have been described in detail, other embodiments are encompassed by the invention. For example, particular advancement/promotion methodology has been thoroughly illustrated and described for each exercise. The methodology for advancement of each exercise is based on studies indicating the need for frequency, intensity, motivation and cross-training. However, the number of skill/complexity levels provided for in each exercise, the number of trials for each level, and the percentage of correct responses required within the methodology are not static. Rather, they may change, based on heuristic information, as more participants utilize the HiFi training and assessment programs. Therefore, modifications to advancement/progression methodology are anticipated. In addition, one skilled in the art will appreciate that the stimuli used for training, as detailed in the Appendices, are merely a subset of stimuli that can be used within a training or assessment environment similar to HiFi. Furthermore, although the characters, and settings of the exercises are entertaining, and therefore motivational to a participant, other storylines can be developed which would utilize the unique training and assessment methodologies described herein.
Finally, those skilled in the art should appreciate that they can readily use the disclosed conception and specific embodiments as a basis for designing or modifying other structures for carrying out the same purposes of the present invention without departing from the spirit and scope of the invention as defined by the appended claims. For example, various embodiments of the methods disclosed herein may be implemented by program instructions stored on a memory medium, or a plurality of memory media.
Description of Target Neurological Deficit:
We believe that that auditory systems in older adults suffer from a degraded ability to respond effectively to rapidly presented successive stimuli. This deficit manifests itself psychophysically in the participant's poor ability to perform temporal order judgments on rapidly presented successive stimuli, and behaviorally in the subject's poor ability to receive speech at normal and high rates.
Goal of the Exercise:
The goal of this exercise is to expose the auditory system to rapidly presented successive stimuli during a behavior in which the participant must extract meaningful stimulus data from both the first and second stimulus.
This can be done efficiently using time order judgment tasks and sequence reconstruction tasks, in which participants must identify each successively present auditory stimulus. Two types of simple, speech-like stimuli will be used in this exercise in order to improve the underlying ability of the brain to process rapid speech stimuli: frequency modulated (FM) sweeps, and structured noise bursts. These stimuli are used because they resemble certain classes of speech. Sweeps resemble stop consonants like /b/ or /d/. Structured noise bursts are based on fricatives like /sh/ or /f/, and vowels like /a/ or /i/.
In general, the FM sweep tasks are the most important for renormalizing the auditory responses of participants. The structured noise burst tasks are provided to allow high-performing participants who complete the FM sweep tasks quickly an additional level of useful stimuli to continue to engage them in time order judgment and sequence reconstruction tasks.
Overview of Exercise Flow and Structure:
High or Low? (Exercise 1) can be divided into two main sections, FM sweeps and structured noise bursts. Both of these sections have a Main Task and an initiation for the Main Task. The Main Task in FM sweeps is Task 1 (Sweep Time Order Judgment). FM Sweeps is the first section presented to the participant. Task 1 of this section must be closed out before the participant can begin the second section of this exercise, structured noise bursts. The Main Task in structured noise bursts is Task 3 (Structured Noise Burst Time Order Judgment). NOTE: Task 2 & 4, the bonus tasks, were removed in HiFi v0.4 s3a.
The Tasks:
Task 1—Main Task: Sweep Time Order Judgment
General Description of Task:
This is a time order judgment task. Participants listen to a sequential pair of FM sweeps, each of which could sweep upwards or downwards. Participants are required to identify each sweep as upwards or downwards in the correct order. The task is made more difficult by changing both the duration of the FM sweeps (shorter sweeps are more difficult) and decreasing the inter-stimulus interval (ISI) between the FM sweeps (shorter ISIs are more difficult).
Trial Description:
Stimuli:
Stimuli consist of upwards and downwards FM sweeps, characterized by their base frequency (the lowest frequency in the FM sweep) and their duration. The other characteristic defining an FM sweep, the sweep rate, is held constant at 16 octaves per second throughout the task. This rate was chosen to match the average FM sweep rate of formants in speech (e.g., ba/da).
A pair of FM sweeps is presented during a trial. The ISI changes based on the participant's performance.
There are three base frequencies:
There are five durations:
Moving through the Task:
An Orientation to Main Task Flow:
A participant will cycle through frequencies, but move in a linear fashion, generally from longer to shorter durations (categories) within each frequency. Adaptive tracking is used to change the ISI based on the participant's performance.
Task 1 Entry Criteria:
Movement within Task 1:
A List of Available Durations (Categories):
The first time in Task 1, a list of available durations (categories) with a current ISI is created within each frequency. At this time, there are only categories in this list that have a duration index of 1 and a current ISI of 600 ms. Other categories (durations) will be added (opened) as the participant progresses through the Task. Categories (durations) are removed from the list (closed) when specific criteria are met.
Choosing a Frequency, Duration (Category), and ISI:
The first time in:
Beginning subsequent sessions:
Switching frequencies:
Switching durations:
Entering a Newly Opened Duration:
Re-Entering an Already Open Duration:
Progression within a Duration (Category )—Changes in ISI:
The amount by which the ISI changes is also varied. This allows participants to move in larger steps when the ISI is longer and then smaller steps when the ISI is shorter. The ISI step size is 10% of the current ISI, but never drops below 5 ms.
An Example of Maximum Reversals within a Durations:
NOTE: The following example uses sequential reversals, however reversals need not occur one after another. There may be several sequential promotions or several demotions before a reversal occurs. A reversal is a “change in direction”.
Opening a Duration (Category)—adding to the List of Available Durations:
A duration index can be added to the list of available durations by:
Exiting a Duration (Category) Weaving a Duration:
The participant exits the current duration:
The participant exits and closes the current duration when:
The participant exits the current duration and drops into initiation step 4 when:
A Summary of when a Duration (Category) is Closed—Removing the Duration from the List of Available Durations:
A duration index is removed from the list of available durations when:
Closing Task 1:
Task 1 is closed when there are no durations (categories) open in any of the frequencies.
Task 1 Initiation—FM Sweep Time Order Judgment
Goal of Initiation:
Many participants find this time order judgment very difficult to perform at first. Yet it is very important that they learn how to perform it, and then perform it intensively. To do so, these participants learn to perform the task through a carefully defined sequence of preliminary steps designed to introduce them to the full task in a step-by-step manner.
General Description of Flow, Trials, and Stimuli:
This portion of the exercise has a linear flow from top to bottom with an “eddy” for remedial instruction. The “eddy” is entered only if the main step is failed. Exiting the “eddy” will bring the participant back to the main step.
Step 3 and its “eddy” (step 3a and step 2) use only one stimulus. An FM sweep is used in step 3. If the participant fails this step, they will be presented with a pure tone (step 3a) and then re-introduced to an FM sweep with no possibility of an incorrect response (step 2) before going back to step 3.
Step 4 and its “eddy” (step 4a) use two stimuli. FM sweeps are used in step 4 and pure tones in step 4a.
The trials in Task 1 initiation and Task 1 are presented in a similar manner (see “Trial Description” under Task 1). Differences are noted in the detailed description, where they occur.
Detailed Description of Task 1 Initiation Steps:
Step 3:
The trial:
Exit criteria:
Step 3a:
The trial:
Exit criteria:
Step 2:
The trial:
Exit criteria:
Step 4 with 700 ms ISI:
The trial:
Exit criteria:
Step 4 with 1000 ms ISI:
The trial (equivalent to block):
Exit criteria:
Step 4a:
The trial:
Exit criteria:
Data Recording:
Data recording for task 1 initiation will be the same as for Task 1, expect that it should be noted the participant was in initiation for these trials.
Task 3—Main Task: Structured Noise Burst Time Order Judgment
General Description of Task:
This task is identical to task 1 (FM sweep time order judgment) except that it uses structured noise burst stimuli rather than FM sweep stimuli. Participants listen to a sequential pair of structured noise bursts, each of which could be perceived as having a higher frequency or a lower pitch. Participants are required to identify each burst as “high” or “low” in the correct order. The task is made more difficult by changing both the duration of the structured noise bursts (shorter bursts are more difficult), decreasing the relative frequency separation between the structured noise bursts (closer in frequency are more difficult), and decreasing the inter-stimulus interval (ISI) between the structured noise bursts (shorter ISIs are more difficult).
Trial Description (Equivalent to a Block):
Trial logic is identical to Task 1 (Sweep Time Order Judgment).
Stimuli:
Stimuli consist of pairs of structured noise bursts, characterized by their base frequency (the center frequency between the pair), their frequency offset (the frequency distance from the base frequency to the peak envelope frequency of each burst), their duration, and their ISI. Other variables defining the structured noise burst (i.e., envelope shape, harmonic structure) are held constant throughout the task.
There are three base frequencies (500 Hz, 1000 Hz, 2000 Hz), identical to those in task 1 (FM sweep time order judgment).
Frequency offset and duration are co-varied with 5 variations:
Moving through the Task:
Moving through the task is identical to task 1 (FM sweep task time order judgment), with the exception that instead of a duration index, there is a frequency offset/duration index.
Closing Task 3:
Task 3 (and the whole exercise) is closed when there are no available frequency offset/durations available.
Replay:
After Task 3 is closed, the entire Task is re-opened beginning with the easiest durations in each frequency.
Data Recording:
Data recorded will be identical to Task 1 with the exception that the frequency/duration index will be used instead of the duration, and there needs to be an indication that the participant is in Task 3.
If the participant is on a reduced schedule, that should be indicated in the trail data.
Task 3 Initiation—Structured Noise Burst Time Order Judgement
In general, initiation occurs in the same way as for Task 1. However, there are no pure tone stimulus fallback trials. Having no pure tone trials caused 2 differences in flow:
All rules for passing and failing found in Task 1 initiation apply to Task 3 Initiation.
Data Recording:
Data recorded will be identical to Task 1 with the exception that the frequency/duration index will be used instead of the duration, and there needs to be an indication that the participant is in Task 3.
Terms and Definitions Unique to High or Low?: (also see HiFi.doc)
3Up—1Down:
Three consecutive correct trials equals advancement. One incorrect equals retreat.
Tracking_Toggle:
The tracking toggle prevents the participant from drifting too far from threshold through inattention. If a participant's responses cause 5 consecutive increases in ISI, the participant pops into Task Initiation. The current ISI is recorded. When the participant passes initiation, they re-enter the duration (or duration/frequency offset) and all re-entry rules apply (an increase in ISI, step size index of 1).
Sweeps_Reversal:
Different from serial reversals, a Sweeps reversal is defined as a “change in direction”. For example, three correct consecutive correct trials move the participant shorten the ISI (increase difficulty). A single incorrect lengthens the ISI (decreases difficulty). The drop to a longer ISI is counted as one reversal. If the participant continues to decrease difficulty (does not respond correctly to 3 consecutive trials), these drops do not count as reversals. A “change in direction” due to 3 consecutive correct responses (difficulty increases) counts as a second reversal. The tally is NOT reset by movement in the same direction.
Threshold:
The threshold is the average ISI of the final 3 Sweeps reversals when the maximum number of reversals has been attained. When exiting a duration (or duration/frequency offset) due to maximum reversals, this average ISI is retained for game initialization instead of the current ISI.
Stable Performance:
Stable performance is based on a comparison of threshold ISIs from 3 different training days. If the difference in ISI between the lowest ISI and the highest ISI is 25 ms or less AND the threshold values do not show a clear trend either upward or downward, stable performance has been reached. For example, sequential threshold values of 15, 23, and 17 would meet the criteria for stable performance. Sequential threshold values of 15, 17, and 23 would not meet the criteria for stable performance since there is a trend toward increasing ISI, nor would sequential threshold values of 23, 17, and 15 (decreasing thresholds).
Description of Target Neurological Deficit:
We believe that auditory systems in older adults suffer from a degraded ability to respond effectively to rapidly presented successive stimuli. This deficit manifests itself psychophysically in the participant's poor ability to perform auditory stimulus discriminations under backward and forward masking conditions. This manifests behaviorally in the participant's poor ability to discriminate both the identity of consonants followed by vowels, and vowels preceded by consonants.
Goal of the Exercise:
The goal of exercise 2 is to force the participant to make consonant and vowel discriminations under conditions of forward and backward masking from adjacent vowels and consonants respectively.
This can be done efficiently using sequential phoneme identification tasks and continuous performance phoneme identification tasks, in which participants must identify successively presented phonemes. We assume that older adults will often find making these discriminations difficult, given their neurological deficits as discussed above. These discriminations are made artificially easy (at first) by using synthetically generated phonemes in which both 1) the relative loudness of the consonants and vowels and/or 2) the gap between the consonants and vowels has been systematically manipulated to increase stimulus discriminability. As the participant improves, these discriminations are made progressively more difficult by making the stimuli more normal.
The Tasks:
Task 1—Main Task: Two Alternative Phoneme Discrimination
General Description of Task:
The task is a sequential phoneme discrimination task. Participants hear a sequence of phonemes from a confusable pair (e.g., da/ga). Participants are required to classify each stimulus as one of the two members of the confusable pair (e.g., as a /da or as a /gal). The difficulty of the task is systematically manipulated by changing the level of emphasis that each stimulus receives, ranging from a high level of emphasis making the discrimination less difficult, to no emphasis making the discrimination more difficult.
Block Description (a Block Consists of 10 Trials):
Stimuli in a Block:
Response Buttons:
Elasped Response Window:
Flow:
Stimuli:
Phoneme Contrasts (Categories):
Two types of phoneme contrasts are used:
The second type covers various contrasts in which the key element is a timing (e.g., silent gap) element. In this type, vowel context is not exhaustively trained because perceptual training is known to generalize adequately across vowel context.
Emphasis Levels (14 Stages):
Both stimuli of each stop consonant contrast pair are synthesized at 14 emphasis levels. The first of the timing contrast pair is only synthesized at the least level of emphasis (14) while the second in the pair is synthesized at 14 emphasis levels. These emphasis levels correspond to stages. At the lowest stage (easiest), the stimuli receive the greatest emphasis. At the highest stage (most difficult), the stimuli receive no emphasis (e.g., the task is to discriminate unemphasized stimuli).
Moving through the Task:
An Orientation to Main Task Flow:
Participants perform blocks of 10 stimuli (trials) with a specific phoneme contrast (category) at a specific emphasis level (stage). Participants make one response per stimulus (trial). The emphasis level (stage) is adaptively tracked within a contrast (category).
Category presentation alternates between stop consonant and timing contrasts. This alternating presentation is maintained over sessions.
Task 1 Entry Criteria:
Movement within Task 1:
A List of Categories and their States:
A list of all 11 categories, their current status, and current stage is maintained. Categories may be either “open” or “closed”. If a category is open, it may either be “active” or “inactive”. Only categories which are both open and active are performed.
The first time in Task 1, all categories are open and only one stop contrast is active. This stop contrast category is chosen randomly. Other categories will be made active as needed.
Choosing a Category and Stage:
The first time in:
Beginning subsequent sessions:
Switching categories:
Entering a Category:
Association:
“Warm-up” trials:
Progression within a Category (Contrast )—Changes in Emphasis Level (Stage):
Exiting a Category (Contrast):
Reclassifying a Category from Active to Inactive (but Still Open):
Reclassifying a Category from Inactive to Active (and Open):
Closing a Category:
Closing Task 1:
Task 1 (and Task 2) is closed when there are no categories (contrasts) open. The exercise terminates at this point.
Task 1 initiation—Two Alternative Phoneme Discrimination
Goal of Initiation:
Participants learn to perform the task through a sequence of preliminary tasks designed to introduce them to the full task in a step-by-step manner.
By the end of initiation, it should be clear to the subject that
General Description of Flow, Blocks, and Stimuli:
Initiation begins with a fairly easy to discriminate phoneme contrast with different vowels sounds (step 2: baa/do). This contrast is presented in unprocessed natural speech. If the participant is unable to perform the sequential classification task with these stimuli, they are presented with stimuli that are even easier to distinguish (step 1: hello/goodbye), presented in unprocessed natural speech. Other wise they are presented again with the (baa/do) contrast, but this time using generated phonemes at the highest emphasis. Once they master this stimuli the participant moves on to the (bo/do) contrast at the highest emphasis.
Stimuli:
Flow:
Resuming Initiation:
If a participant exits the exercise with out completing the initiation, they should resume at part “a” of whichever step they were on when they exited.
If a participant continues the initiation from the Skip/Continue screen, they should resume at part “a” of whichever step they were on when they dropped to the Skip/Continue screen.
Skip/Continue:
If the participant fails at any step four times, they should be taken to the Skip/Continue screen.
This screen notifies the participant that they have reached a point in the training where it is necessary to contact their study coordinator and a number is provided.
Two buttons are active on this screen:
Reward Animations—a Special Case for Tell Us Apart Initiation:
In Tell Us Apart initiation, the participant responds to a block of 2 to 10 trials. In each of the blocks of trials, the next stimulus is presented immediately after the response button is clicked, with the OR only re-appearing after the block is complete. The 10-Pack bonus (and benchmark & mini-series animations) should only play after the block is complete and when the final response in the block is correct. At the end of the block if 6 or more pop-ups are on screen then the bonus will play. If all 9 are on screen and trials remain in the block, there are no additional pop-ups until the final trial of the block. If the final trial is correct, the bonus plays, if it is incorrect the pop-ups are cleared. If multiple benchmark animations are cued, only one will play. The benchmarks only play if the final response of the block is correct. If a benchmark is cued and the final response is not correct, that benchmark will NOT be removed the cue and will have an opportunity to play at the end of each block. Once it is played it is removed from the cue.
Description of Target Neurological Deficit:
We believe that degraded representational fidelity of the auditory system in older adults causes an additional difficulty in the ability of older adults to store and use information in auditory working memory. This deficit manifests itself psychophysically in the participant's poor ability to perform working memory tasks using stimuli presented in the auditory modality.
Goal of the Exercise:
Exercise 3 has two goals:
This can be done efficiently using a spatial match task similar to the game “concentration”, in which participants must remember the auditory information over short periods of time to identify matching syllables across a spatial grid of syllables.
Overview of Exercise Flow and Structure:
Match It! (Exercise 3) has only one Task. The stimuli are identical to those used in Phonic Streams (Exercise 4).
There are 5 speech processing levels. Processing level 1 is the most processed and processing level 5 is normal speech. Participants move through stages within a processing level before moving to a less processed speech level. Stages are characterized by the size of the spatial grid. At each stage, participants complete all the categories.
Task 1—Main Task: Spatial Match
General Description of Task:
The task is a spatial paired match task, based on the children's game “Concentration.” Participants see an array of response buttons. Each response button is associated with a specific syllable (e.g., “big”, “tag”), and each syllable is associated with a pair of response buttons. Upon pressing a button, the participant hears the syllable associated with that response button. If the participant presses two response buttons associated with identical syllables consecutively, those response buttons are removed from the game. The participant completes a trial when they have removed all response buttons from the game. Generally, a participant completes the task by clicking on various response buttons to build a spatial map of which buttons are associated with which syllables, and concurrently begins to click consecutive pairs of responses that they believe, based on their evolving spatial map, are associated with identical syllables. The task is made more difficult by increasing the number of response buttons and manipulating the level of speech processing the syllables receive.
Blocks and Trials:
Block in Match It!: OR button, the grid of response buttons, and all the clicks (trials) the participant makes to find matching response buttons (and clear the grid)
Trial in Match It!: each click on a response button (representing a syllable)
Assigning syllables to a button: Syllables are not assigned to a specific button until the participant clicks on the button for the first time. On the first click, a syllable is chosen randomly from the pool of syllables selected for the grid. The only criterion for choosing a syllable is that it should not match the previously heard syllable (syllable associated with previously clicked button). This approach prevents a participant from receiving credit for clicking a pair of buttons before hearing the syllable associated with each button, which would represent a coincidental match.
A match: Three clicks are generally required to make a match. Two non-sequential clicks on two different buttons assigns the syllable to the buttons. A minimum of one more click (second button to first button) will make the match.
Max clicks: maximum number of clicks the participant can use to successfully clear the grid; this number increases with the size of the grid
Block Description:
Stimuli:
Processing Levels:
Each stimulus is available at 5 levels of speech processing, ranging from level 1 (highly processed) to level 5 (unprocessed natural speech).
Stages:
There are 4 task stages, each associated with a specific number of response buttons in the trial and a maximum number of response clicks allowed:
Categories:
The stimuli are identical to those used in Exercise 4 (Phonic Streams) and consist of consonant-vowel-consonant syllables or single phonemes:
Category 1 consists of easily discriminable CV pairs. Leading consonants are chosen from those used in exercise 2 and trailing vowels are chosen to make confusable leading consonants as easy to discriminate as possible.
Category 2 consists of easily discriminable CVC syllables. Stop, fricative, and nasal consonants are used, and consonants and vowels are placed to minimize the number of confusable CVC pairs.
Categories 3, 4, and 5 consist of difficult to discriminate CVC syllables. All consonants are stop consonants, and consonants and vowels are placed to maximize the number of confusable CVC syllables (e.g., cab/cap).
Moving through the Task:
An Orientation to Main Task flow:
Participants may find either the stimulus discrimination aspects or the memory aspects of the task difficult (or both). Task progression is designed to allow participants with poor stimulus discrimination to advance through the memory challenges of the task without getting stuck because of their poor stimulus discrimination capabilities, and to allow participants with poor memory to advance through the stimulus discrimination aspects of the task without getting stuck because of their poor memory abilities. Participants move through stages (i.e., increasing numbers of response buttons) before increasing processing level (decreasing the level of processing). However, not all stages must be completed before moving on to a new processing level. At each stage, participants complete all stimulus categories.
Task 1 Entry Criteria:
Movement within Task 1:
A List of Categories and their States:
A list of available syllable categories and their current status is maintained.
The first time in Task 1, the list contains all syllable categories. Each syllable category is marked as “open.”
Choosing a Processing Level and Stage:
The first time in:
Beginning subsequent sessions:
Choosing a Category:
Establishing the Grid:
Syllables and response buttons:
Max clicks:
Opening, Closing, and Failing Categories:
Moving between Stages:
Advancing to the Next Processing Level:
NOTE: retreat to earlier processing levels is not possible. When entering a new processing level the stage is set to 1.
Closing Task 1:
After Closing Task 1:
Task 1 Initiation—Spatial Match
Goal of Initiation:
Most participants should be generally familiar with the concept of a spatial match memory game. To ensure that they can perform the task, participants begin the task in a training game with a small number of response buttons and be assisted by visual cues.
By the end of initiation, it should be clear to the participant that
General Description of Flow, Trials, and Stimuli:
Initiation for Task 1 will begin with the use of stimuli that are trivial to distinguish in a small spatial grid (Step 1) and then advance to the use of relevant stimuli (Step 2).
Detailed Description of Task 1 Initiation Steps:
Step 1:
The grid (equivalent to block):
Exit criteria:
Step 1a:
The grid (equivalent to block):
Exit criteria:
Step 2:
The grid (equivalent to block):
Exit criteria:
Step 2a:
The grid (equivalent to block):
Exit criteria:
Skip/Continue:
This screen notifies the participant that they have reached a point in the training where it is necessary to contact their study coordinator and a number is provided.
Two buttons are active on this screen:
This document includes the following sections:
Goals . . . 2
Summary . . . 2
Mechanics . . . 3
User Interface . . . 10
Level Requirements . . . 11
Goals
Description of Target Neurological Deficit
We believe that degraded representational fidelity of the auditory system in older adults causes an additional difficulty in the ability of older adults to store and use information in auditory working memory. This deficit manifests itself psychophysically in the participant's poor ability to perform working memory tasks using stimuli presented in the auditory modality.
Goals of the Exercise
Sound Replay has two goals:
These goals can be met using a temporal match task similar to the neuropsychological tasks digit span and digit span backwards, in which participants must remember the auditory information over short periods of time to identify matching syllables in a temporal stream of syllables.
Summary
Task Description
The task is a temporal paired match task, based on the digit span neuropsychological task. Participants hear a sequence of syllables (e.g., “big”, “tag”, “pat”). Following the presentation of the sequence, the participant sees a number of response buttons, each labeled with a syllable. All syllables in the sequence are shown. In addition there may be buttons labeled with syllables not present in the sequence (distracters). The participant is required to press the response buttons to reconstruct the sequence. The Task is made more difficult by increasing the length of the sequence, decreasing the ISI, and manipulating the level of speech processing the syllables receive.
Task Progression
The task progression is designed to allow participants with poor stimulus discrimination to advance through the memory challenges of the task without getting stuck because of their poor stimulus discrimination capabilities, and to allow participants with poor memory to advance through the stimulus discrimination aspects of the task without getting stuck because of their poor memory abilities. Participants move through stages (that is, increasing sequence length and complexity) before advancing in processing level (decreasing the level of speech processing).
Mechanics
Core Flow: Progression Overview
Sound Replay has an initiation interval (see below) followed by the main task.
NOTE: Version 0. 5 included a bonus task which was removed in version 0.6.
The main task is a nested progression consisting of processing levels, stages, categories, and trials. The following is a brief description of the progression:
Processing Levels
Processing levels correspond to five speech processing levels. Participants move through stages within a processing level before moving to the next processing level.
Stages
There eight stages in the main task, each associated with the following specifications:
the number of response buttons in the trial (including the number of “distracters,” buttons not associated with any stimulus, but included as a distraction)
Within each stage there are five categories. The stage advances or retreats depending on progress through the five categories (this is described in detail below).
Categories
The categories correspond to the five stimulus categories. The stimuli for Sound Replay are identical to those used in Match It! (Exercise 3) and consist of consonant-vowel-consonant syllables or single phonemes. The first two categories are easily discriminable pairs, while the last three categories are difficult to discriminate. See the Stimulus Generation Document, HiFi Stimulus Generation, for details on the stimuli used in Sound Replay.
Main Task Flow
Entry Criteria
Task 1 Initiation passed
Choosing a processing level and stage
The first time in:
Beginning subsequent sessions:
NOTE: Reversals are tracked over sessions (see “advancing to the next processing level”) and the “second chance” rule also applies over sessions (see “opening, closing, and failing categories”).
Warm-Up Trials
Two blocks of two trials using 2 randomly chosen syllables from the starting category are performed. There are no distracters. After the two blocks are performed, the participant begins the session at the same stage as the previous session regardless of their performance on the two trials. There are no warm up trials following initiation.
Categories
Choosing a category
Category status
Trial Sequence
Response Buttons
Opening, Closing, and Failing Categories
NOTE: A maximum of 5 trials is required to meet either the criterion for failing or the criterion for closing the category.
Moving between Stages
Advancing to the Next Processing Level
NOTE: Retreat to earlier processing levels is not possible.
Maximum Serial Reversals
NOTE: The number of reversals between stages is subject to change, so an external file should be created for this information.
Closing Task 1
Initiation
Most participants will be generally familiar with the concept of a temporal match memory game. To ensure that they can perform the task, participants will begin the task in a training game with a small number of response buttons and be assisted by visual cues.
Goal
By the end of initiation, it should be clear to the participant that
Initiation Flow
Initiation for Task 1 will begin with the use of relevant stimuli (Step 2a) and retreat to the use of stimuli that are trivial to distinguish if the participant fails to pass Step 2a. In general, promotion occurs with 3 consecutive correct trials and demotion occurs with 3 incorrect trials.
Coach Voice-Over
Coach is used throughout the initiation. Please see the coach document for Sound Replay, HiFi Sound Replay Coach, HiFi v.6 SoundReplay coach.
Step-by-Step Progression
Step 2a
The trial
Step 2b
The trial
Step 1a
The trial
Step 1b
The trial
Skip/Continue
This screen notifies the participant that they have reached a point in the training where it is necessary to contact their study coordinator and a number is provided.
Two buttons are active on this screen:
Elements
Trial Description (Equivalent to a Block)
User Interface
Flowchart
Art Assets
HiFi Exercise UI Functional Spec, HiFi Exercise UI Func Spec
HiFi Exercise UI Assets, hifi exercise ui assets 100104
Sound and Music
HiFi Coach Functional Spec, HiFi v.6 SoundReplay coach
Level Requirements
Stimuli
HiFi Stimuli Functional Spec, HiFi Stimulus Generation
Points
HiFi Points Functional Spec,
Rewards
HiFi Reward Functional Spec, HiFi v.6 RewardAnimations spec
Description of Target Neurological Deficit:
We believe that the degraded representational fidelity of the auditory system in older adults causes an additional difficulty in the ability of older adults to store and use information in auditory working memory. This deficit manifests itself behaviorally in the subject's poor ability to understand and follow a sequence of verbal instructions to perform a complex behavioral task.
Goal of the Exercise:
Exercise 5 has two goals:
This can be done efficiently using an analog of the token task, a neuropsychological assessment of a participant's ability to follow auditory instructions. In this task, the participant is given auditory instructions of increasing length (e.g., number of steps in the instruction sequence) and complexity (e.g., stimulus discriminations required to perform the task correctly).
Overview of Exercise Flow and Structure:
There are 5 speech processing levels. Processing level 1 is the most processed and processing level 5 is normal speech. Participants move through stages within a processing level before moving to a less processed speech level. Stages are characterized by the number of sequential instructions followed. Participants move through stages independently in four categories. Each category is characterized by instruction type.
General Description of Task:
The task requires the subject to listen to, understand, and then follow an auditory instruction or sequence of instructions by manipulating various objects on the screen. Participants hear a sequence of instructions (e.g., “click on the bank” or “move the girl in the red dress to the toy store and then move the small dog to the tree”). Following the presentation of the instruction sequence, the participant performs the requested actions. The task is made more difficult by making the instruction sequence contain more steps (e.g., “click on the bus and then click on the bus stop”), by increasing the complexity of the object descriptors (i.e., specifying adjectives and prepositions), and manipulating the level of speech processing the instruction sequence receives.
This task may be implemented in any one of a number of different game play styles (e.g., making a machine, following a recipe, manipulating people and objects in a street scene). Much of the exact content of the task (e.g., the exact instruction sequences, the initiation steps) will be dependent on the choice of game play style made.
Blocks and Trials:
Trial Description:
Stimuli:
Processing Levels:
Each stimulus is available at 5 levels of speech processing, ranging from level 1 (highly processed) to level 5 (unprocessed natural speech).
Stages:
There are 6 stages, each associated with a specific number of instructions in the trial.
Categories:
There are four categories of instructions. Each category contains a distinct type of instruction. Categories 1 & 2 are “Click” instructions and categories 3 & 4 are “Drag” instructions.
In Category 4, the prepositional relationship could be: to, next to, to the left of, to the right of, between, above, below, etc.
Category 2 & 4 require that the movable foreground objects on screen appear in confusable sets. If the instruction is to “click on the girl in the red dress,” the character set containing a girl in a red dress, a girl in a green dress, and a girl in a blue dress must appear on screen. Refer to “Screen Requirements” later in this document.
Please reference “Listen and Do Instruction List.doc” for list of recorded instructions.
Moving through the Exercise:
An Orientation to Main Task Flow:
Participants may find either the stimulus discrimination aspects or the memory aspects of the task difficult (or both). Task progression is designed to allow participants with poor stimulus discrimination to advance through the memory challenges of the task without getting stuck because of their poor stimulus discrimination capabilities, and to allow participants with poor memory to advance through the stimulus discrimination aspects of the task without getting stuck because of their poor memory abilities.
Participants move through stages (i.e., increasing numbers of instructions) before increasing processing level (decreasing the level of processing). However, not all stages must be completed before moving on to a new processing level.
Entry Criteria:
Movement within Exercise:
A List of Categories and their States:
A list of available instruction categories and their current status (“open” or “closed” & current Stage) is maintained.
The first time in, the list contains all instruction categories. Each category is marked as “open.”
Choosing a Processing Level and Stage:
The first time in:
Beginning subsequent sessions:
Warm Up Trials:
Two single instruction warm-up trials are performed using a single instruction randomly chosen stimuli from the new session's category. After the two trials are performed, the participant begins the session at the same stage as the previous session regardless of their performance on the two trials.
Choosing a Category:
Choosing Instructions:
Progression within a Category (Instruction Type)—Changes in Stage (Number of Instructions Presented in Sequence):
Exiting and Closing a Category:
Categories are exited and closed when any of the criteria below are met. Once categories are closed, they remain closed and no trials are performed in that category until either the speech processing level changes or the participant returns from initiation (both of which re-open all categories at stage 1).
ALTERNATE 1 Closing Criteria (not used for v0.6):
Categories are closed after they have been exited three times, either within or across sessions, for any of the reasons or combination of reasons described above in the “Exiting a category” section. Once categories are closed, they remain closed and no trials are performed in that category until either the speech processing level changes or the participant returns from initiation (both of which re-open all categories at stage 1).
ALTERNATE 2 Closing Criteria (not used for v0.6):
Categories are closed when any of the criteria below are met. Once categories are closed, they remain closed and no trials are performed in that category until either the speech processing level changes or the participant returns from initiation (both of which re-open all categories at stage 1).
Advancing to the Next Processing Level:
Closing the Exercise:
Schedule After Closing:
Screen Requirements:
Settings:
The current setting for Listen and Do is a street scene with building and characters in the foreground that can be clicked and dragged. Other setting may be developed.
Layers: There are Four Layers in which Objects Appear on Screen
Changing Objects on Screen
The number of foreground elements will vary with the stage as follows:
Object Sets
Please reference “Listen and Do Art List.doc” for list of art elements.
Initiation
Goal of Initiation:
By the end of initiation, it should be clear to the participant that
General Description of Flow, Trials, and Stimuli:
Initiation will begin with the use of single instructions from category 1. There will be fewer objects on screen to manipulate.
Detailed Description of Initiation Steps:
Step 1:
One block (5 trials):
Exit criteria:
Step 2:
One block (5 trials):
Exit criteria:
Step 3:
One block (5 trials):
Exit criteria:
Skip/Continue:
This screen notifies the participant that they have reached a point in the training where it is necessary to contact their study coordinator and a number is provided.
Two buttons are active on this screen:
Data Recording:
Data recording for task 1 initiation will be the same as for Task 1, expect that it should be noted the participant was in initiation.
Listen and Do Instruction List
Instruction Types:
Category 2: CLICK (higher complexity):
Category 3: DRAG (lower complexity):
Category 4: DRAG (higher complexity):
Stages:
When more than one instruction required, use conjunction between instructions: <Instruction> <Conjunction> <Instruction> <Conjunction> <Instruction>
Background: City or town street scene. This background should accommodate four buildings in the background and up to 8 characters in the foreground (see below). Eddie's prototype can be used as a reference. The screen size for the background is 800×275 pixels, but it's constrained at the top by the timer, points display, and dashboard.
Background Objects: These are the buildings in the street scene. All buildings need text labels which can be part of the building design, e.g., on sign of a store or engraved “Bank” in Roman lettering. There will be a maximum of 4 buildings in the background. The exact size of the buildings is flexible. Those in Eddie's prototype range from 120×120 pixels to 150×140 pixels. There should be space between the buildings (40-50 pixels) to accommodate instructions like “move the policeman between the library and the cafe.”
Midground Objects: (These are similar to the buildings in that they can be clicked and foreground objects can be dragged to them, but they cannot be moved. They will appear in front of the buildings, but behind the foreground characters).
Foreground Objects (Characters):
There will be from four to eight foreground objects in a scene depending on the Stage (number of instructions delivered in sequence). The size of the characters is also flexible. It is OK for them to bit a bit oversized compared to the buildings. The should “read” well be easy to identify. The characters in Eddie's prototype range form 40×80 pixels to 100×145 pixels.
Description of Target Neurological Deficit:
We believe that the degraded representational fidelity of the auditory system in older adults causes an additional difficulty in the ability of older adults to store and use information in auditory working memory. This deficit manifests itself behaviorally in the participant's poor ability to remember verbally presented information.
Goal of the Exercise:
Exercise 6 has two goals:
This can be done efficiently using a story recall task, in which the participant must store relevant facts from a verbally presented story and then recall them later. In this task, the participant is presented with auditory stories of increasing length (e.g., number of details in the story) and complexity (e.g., types of details in the story). Following the presentation, the participant must answer specific questions about the content of the story.
General Description of Task:
The task requires the participant to listen to an auditory story segment, and then recall specific details of the story. Following the presentation of a story segment, the participant is asked five questions about the factual content of the story. The participant responds by clicking on response buttons featuring either pictures or words. For example, if the story segment refers to a boy in a blue hat, a question might be: “What color is the boy's hat?” and each response button might feature a boy in a different color hat or words for different colors. The task is made more difficult by 1) increasing the number of story segments heard before responding to questions 2) making the stories more complex (e.g., longer, more key items, more complex descriptive elements, and increased grammatical complexity) and 3) manipulating the level of speech processing of the stories and questions.
Blocks and Trials:
Block Description:
Stimuli:
Stories:
There are five stories, each in 20 story segments from 20-30 seconds long. The five stories are progressively more complex (story 2 is more complex than story 1, story 3 is more complex than story 2, etc.). Complexity is governed by:
Questions:
There are five questions for each story segment. The questions will be presented in a random order
Response Buttons:
For each set of 5 questions, 3 questions will have text response buttons and 2 will have picture response buttons. The responses are presented in a random order. There are four response buttons for each text response question and three (v0.5) response buttons for each picture response question.
Processing Levels:
Each story and associated questions are at a different levels of speech processing, ranging from level 1 (highly processed) to level 5 (unprocessed natural speech). In other words, each story will only have one level of speech processing and all five will be at different levels. The order of the stories is as follows:
Title & End Screens
A title screen is displayed in the reward space at the start of each story. It is displayed during the first segment and remains until it is replaced by the first 10-Pack pop-up after a correct response. An end screen is displayed after the final block of a story is evaluated. If a benchmark animation is triggered, the end screen is displayed after the animation.
Moving through the Task:
An Orientation to Main Task Flow:
Participants listen to a story segment and then respond to five questions about that segment. If they do poorly, they will hear the same segment and associated questions again. If they do fairly well, they will hear the next segment and its questions. If they do very well, they will hear the next two segments together and answer all 10 associated questions. If they continue to do very well, a third segment is added. A maximum of four segments can be heard at once.
Progression through Blocks:
The above progress tracking is maintained across sessions.
Entry Criteria:
Movement within Story Teller:
Choosing a Processing Level and Stage:
The first time in:
Beginning subsequent sessions:
Advancing to the Next Processing Level.
NOTE: retreat to earlier processing levels (stories) is not possible
Closing out Story Teller:
After Closing out Story Teller:
Data Recording:
Data Recording—Trial by Trial Data
In general, all relevant data describing the stimulus and the response should be recorded to allow subsequent analysis to completely reconstruct the trial
The following data should be recorded for each trial:
Story Teller Initiation
Stimuli:
The stimuli includes the first segment of the first story along with the associated five questions and responses.
Trail Presentation:
Following the participant's OR click, the story segment is presented and then the OR button appears again. After the participant clicks the OR, the first question is presented and four response buttons appear. There is a visual cue indicating which button is the correct response. After the first trial, if there is an incorrect response, visual cues are given again until there is a correct response.
Evaluating the Initiation Block:
After the five questions have been answered:
Falling Back to Initiation:
There is no way to fall back into initiation in Story Teller.
Standard Exercise Features:
Interface Elements:
Main Screen buttons:
OR button:
Response Buttons:
User Interface Bar:
Timer and Points Display:
Button States:
Below are the required button states for each exercise:
*Down State: The down state is behaving with slight variations in each exercise. We will standardize the down state (post v0.5?) to be displayed with a short delay (most likely handled as a button animation) that does not interfere with the participants next response (e.g., in High or Low? if the stimulus is two sequential up sweeps the down state of the first click on the up button will
**Inactive State: Instead of using an inactive state the buttons will be only be present when active. The buttons are active only after the stimulus presentation is complete in High or Low? and Sound Replay, they are active at the start of the stimuli presentation in Tell Us Apart and Story Teller. They are never present during the pre-stimulus delay. They are removed following the ding
Keyboard Shortcuts:
Flow:
This diagram shows the events occurring between one observation response (OR button click) and the next. It is meant to show a generic flow of events and each exercise will have it's own variation.
Trial-Related Features:
Limit to Number of Identical Stimuli:
In exercises that use a series of randomized stimuli, the same stimuli in the same order cannot be presented more than 3 times.
Trial is Evaluated as Incorrect after First Incorrect Response:
Trials that require multiple responses will be evaluated as incorrect after the first incorrect response and ended. For example, in High or Low?, if the first response to a 2-sweep sequence does not correspond to the first stimulus, the trial is evaluated as incorrect and no second response can be given.
Saving Trial Data and User Data:
Trial data write out not implemented in v0.5.
XML data and preferences should be written out after every 10 trials in an exercise to minimize the amount of data lost due to computer crashes. Both trial data and user data files should be written out at the same time so there is no discrepancy between them.
Rewards:
No Rewards will be given on an incorrect response. A “thunk” sound will play.
Points-related Rewards (please refer to “Points.doc”):
Animated Rewards (please refer to “RewardAnimations.doc”):
Reward Sequence:
Rewards will play in the following sequence:
Pre-reward delay (100-500 ms)—“Ding” Sound AND Points—Bonus Points—Pre-animation delay (0-200 ms)—10-pack pop-up OR 10-pack bonus—Benchmark Animation (with concurrent text).
Time-Related Features:
OR window: (60 Seconds)
A window of time during which the participant must click the OR button. If this window of time has elapsed, the program enters the paused state.
Response Window: (60 Seconds)
A window of time given for the participant to respond. This window is reset after each response in a series of responses. If the response window has elapsed, the trial (or block in Tell Us Apart) is terminated and evaluated as “timed_out” and treated as an incorrect trial for progression. No incorrect feedback is given and the program enters the paused state. Clicking the Pause button will have the same effect as an elapsed response window. When the participant returns to the exercise from the pause screen the OR is present and the next trial (or block for Tell Us Apart) is cued.
Paused State:
The exercise is suspended and exercise time is not decremented. The Pause Screen is displayed. The paused state is entered by clicking the pause button in the UI bar (or equivalent keyboard shortcut) or through participant inactivity (elapsed OR window or response window).
Audio Prompt:
If ⅔ (40 seconds) of the OR window or response window has elapsed, an audio prompt will be given by the Coach.
Delays that Introduce “jitter”:
These delays will vary the timing of events and make the exercises slightly less predictable. Delays will be chosen at random from a range of milliseconds.
Pre-Stimulus Delay (400-600 ms):
Pre-Reward Delay (0-200 ms):
Pre-Animation Delay (0-200 ms):
Exercise Time-Out:
Each exercise will be assigned a specific number of minutes, the exercise time. When this time has expired the exercise is terminated and the participant is brought to the end of exercise screen as described below:
If the OR is present or the exercise is Match It!, the exercise ends immediately when the exercise time expires.
If the response buttons are present, the exercise terminates after the players response to the trial is complete (and any associated rewards have been presented) or after the response window ends, whichever comes first.
Post v0.5 we may adjust this on an exercise by exercise basis to prevent things like a Story Teller question being presented when there is no time to respond or terminating Match It! when there is only one match left in a large grid.
Terms and Definitions:
Exercise Structure:
Session:
A session begins when the first exercise is entered and ends when exiting the last exercise. It may be possible to have multiple sessions in a single calendar day.
Task:
A fundamental subdivision of the exercise with it's own stimulus presentation and progression rules.
Initiation:
A sequence of preliminary trials designed to introduce the participant to a Task within an Exercise. It may be possible to drop down into initiation for remedial instruction after the main task has been entered.
Entry Criteria:
Conditions that must be met before starting a Task within an Exercise.
Warm-Up:
The warm-up is a series of simpler trials given at the very beginning of a session in some of the exercises. The warm-ups re-orient the participant to the Main Task.
Trial:
The smallest stimulus and response grouping that can be evaluated for progression. For example, in High or Low? Task 1, two stimuli are given and 1 to 2 responses are evaluated to determine if the trial was correct or incorrect. In Tell Us Apart, on the other hand, each stimulus/response pair is individually evaluated and therefore considered a trial.
Trial Evaluation:
A trial may be:
Response:
A single selection representing a single stimulus.
Block:
A group of trials evaluated for progression in some exercises. Tell Us Apart has 10-trial blocks, Listen and Do has 5-trial blocks, and Story Teller blocks range from 5 to 20 trials.
Category:
A group of stimuli sharing predetermined characteristics. This group has significance for progression to other categories or tasks within the exercise. Generally categories can be closed (and never seen again).
Category States:
Processing Level:
A measure of the processing stimuli receive; there are 5 speech processing levels with level 5 being the most processed, and level 1 being normal speech
Stage:
Meaningful for progression between trials of lesser or greater difficulty (typically within a category). Difficulty increases with higher numbered stages.
ISI:
Inter-stimulus interval. The shorter the ISI, the more difficult the trial. The ISI shrinks and expands based on the participant's performance (generally associated with the stage).
Adaptive Tracking:
The means used by each exercise to adapt to the participant's performance. Generally changes are made to the stimulus in order to increase or decrease the difficulty of trials.
Progression:
The following are used to track correct and incorrect trials to determine progression through the exercise. A combination of several may be working at the same time to determine when a participant advances to greater difficulty or retreats to simpler trials.
Steady_state tally:
Typically this is used to track sequential correct or incorrect trials (High or Low? Task 1 initiation, for example).
Trial_tally:
At it's most basic, the trial tally counts the number of trials taken. However, it may be used to count the number of incorrect or correct trials (not necessarily sequential).
Percent_correct:
Stores the number of correct trials and the number of trials taken so a percent correct can be calculated.
Serial_reversal (Sound Replay, Match It!, Listen and Do):
Serial_reversal tallys the number of times there is a series of advancements and retreats with no repetition of movement in the same direction. Distinct from a High or Low? reversal (see below), a reversal in this case is an advancement and retreat between the same two stages (Stage 2-Stage 3-Stage 2). Moving up a stage, down a stage, up a stage, and then back down a stage counts as two 2 serial reversals. As soon as a movement is repeated in the same direction (moving up a stage again), the tally is reset. (3→2→3→2→3→2 is 3 serial reversals; 3→2→3→4 is 0 serial reversals—tally was reset).
3Up—1Down (High or Low?):
Three consecutive correct trials equals advancement. One incorrect equals retreat.
Tracking_toggle (High or Low?):
The tracking toggle prevents the participant from drifting too far from threshold through inattention. The tracking toggle is coupled with 3up—1down (used in High or Low?). If a participant's responses cause 5 consecutive drops in stage (ISI in High or Low?), the participant pops into Task Initiation. The current stage (ISI in High or Low?) is recorded. When the participant passes initiation, they re-enter the task at the same category (duration or duration/frequency offset in High or Low?) and all re-entry rules apply (a drop in difficulty from recorded stage).
High or Low?_reversal (High or Low?):
Different from serial reversals, a High or Low? reversal is defined as a “change in direction”. For example, three correct consecutive correct trials moves the participant shorten the ISI (increase difficulty). A single incorrect lengthens the ISI (decreases difficulty). The drop to a longer ISI is counted as one reversal. If the participant continues to decrease difficulty (does not respond correctly to 3 consecutive trials), these drops do not count as reversals. A “change in direction” due to 3 consecutive correct responses (difficulty increases) counts as a second reversal. The tally is NOT reset by movement in the same direction.
This document includes the following sections:
Links . . . 1
General Specifications . . . 2
High or Low (Exercise 1) . . . 2
Tell Us Apart (Exercise 2) . . . 4
Match It! (Exercise 3) . . . 6
Sound Replay (Exercise 4) . . . 8
Listen & Do (Exercise 5) . . . 9
Story Teller (Exercise 6) . . . 10
Links
Description of the Story Teller Stories° Creation HiFi Stimulus Story Teller Stories Story Teller Scripts:
General Specifications
All sound stimuli in the HiFi exercises have the following parameters:
High or Low (Exercise 1)
Stimuli consist of upward and downward frequency modulated (FM) sweeps characterized by their base frequency (the lowest frequency in the FM sweep) and their duration. In the exercise, a pair of FM sweeps is presented separated by an inter-stimulus interval (ISI).
Sweep Stimuli
The other characteristic defining an FM sweep:
This rate was chosen to match the average FM sweep rate of formants in speech (for example, ba/da).
Generating Sweeps
The sweeps are created according to the following formula:
f(t)=f0*2(16*t)
Example:
fi=f0
ff=f0*2(16*duration)
Upward sweep:
fi→ff
Downward sweep:
ff→fi
There are three base frequencies:
There are five durations:
Burst Stimuli
Stimuli consist of pairs of structured noise bursts, characterized by their base frequency (the center frequency between the pair), their frequency offset (the frequency distance from the base frequency to the peak envelope frequency of each burst), their duration, and their ISI. Other variables defining the structured noise burst (i.e., envelope shape, harmonic structure) are held constant throughout the task.
There are three base frequencies (500 Hz, 1000 Hz, 2000 Hz), identical to those in task 1 (FM sweep time order judgment).
Frequency offset and duration are co-varied with 5 variations:
Generating Bursts
Bursts are single formant structured noise stimuli. They are created by filtering noise stimuli with bandpass filters. The two bursts that must be distinguished differ in their center frequency according to the following formula. The formula used to create the bursts is as follows:
f_highFrequency=f_baseFrequency+f_baseFrequency*multiplier
High burst:
Low burst:
f_baseFrequency
Tell Us Apart (Exercise 2)
Phoneme Contrasts (Categories)
Users are trained on two types of phoneme contrasts:
The Class A phonemes (the b/d/g continuum) are covered semi-exhaustively across vowel contexts because perceptual training of this discrimination does not generalize well across vowel contexts.
The Class B phonemes, the phoneme class in which the key emphasis is timing, are not exhaustively trained across vowel contexts because perceptual training is known to generalize adequately.
Sound Generation
Syllable generation: The syllables in Tell Us Apart are synthetically generated using CV Gen Stim, a speech synthesizer developed by Athanassios Protopapas. Emphasis is added to the phonemes differently depending on whether they are differentiated by formant transitions or timing transitions.
Note: The generated sounds recreate the formant characteristics of the phonemes for which we are training the ear, but do not have all the characteristics of natural speech. For this reason the sounds may not be recognizable speech.
Phoneme Spec Files
Protopapas' program generates synthesized speech by reading in files specifying the parameters of the various syllables. (These speech generation files are called Klatt's formant files after Dennis Klatt who developed MITalk, a research system that converted ordinary printed text into intelligible synthesized speech.) The examples below are Klatt's formant files for the syllable /bi/. The first example is the syllable /bi/ with no lengthening (stretching) added; the second with the highest level of lengthening.
Klatt's files specify the frequency of each formant at the beginning and end of a segment, for example, the first formant in the first segment (200 msec-209.6 msec) begins at 200 Hz and ends at 300 Hz, the second begins at 300 Hz and ends at 310 Hz, and so on. The pitch determines the pitch period (1/pitch) within which stretching may be added. Finally, the amplitude is 85 dB.
NOTE: Because of a glitch in the program, emphasis (change in amplitude) is added later, including ascending and descending ramps (on and off) to make the transition sound more natural.
In the files below, notice that as the consonant is stretched, the vowel shortens, for example, the consonant to vowel ratio in the first example is 36:161 while the ratio in the second example is 92:107. Note that the total time doesn't change. Syllable recognition improves when the ratio between the consonant and vowel is changed in favor of the consonant.
NOTE:
These Klatt's formant files are found in: Neuroscience/Assets/auditory/application/phonic_id/task_1/bi/BBIYO and BBIY13
Format:
number of formants; number of segments
time (msec); pitch (Hz); amplitude (dB); first formant freq (Hz); 2nd formant freq (Hz);
3rd formant freq (Hz)
Class A phonemes: The b/d/g continuum of phonemes is reliably differentiated (that is across accents and vocal ranges) by the sweep of their formants. Emphasis is added by lengthening and amplifying the sweep aspect of both consonants equally. Because the formant is a reliable cue for differentiating these phonemes, the ear is trained to distinguish them by exaggerating this difference only.
Class B phonemes: For the phonemes distinguished by a timing element, the first syllable is not emphasized and the second is emphasized based on the timing element.
For example, the voice onset time (VOT) is emphasized or the aspiration is lengthened depending on the pair.
In the first three voiced pairs, the second syllables /pa(e)/, /to/ and /ki/, are emphasized by lengthening the voice onset time (VOT), the time between when the voiced vowel appears and the consonant.
In the fourth pair, /sta/ is distinguishable from /sa/ by the stop gap between the /s/ and the /a/. Emphasis is added to /sta/ by increasing the gap between the /s/ and the /a/.
In the fifth pair, /che/ is distinguished from /she/ by the abrupt onset produced by erasing part of the /sh/ sound. Emphasis is added by erasing more from the /sh/ to create and more abrupt (and easily distinguishable) /ch/ sound.
Emphasis Levels (14 Stages)
Both stimuli of each contrast pair are synthesized at 14 emphasis levels. These emphasis levels correspond to stages. At the lowest stage (easiest), the stimuli receive the greatest emphasis. At the highest stage (most difficult), the stimuli receive no emphasis (for example, the task is to discriminate un-emphasized stimuli).
NOTE: The difficulty of different levels is currently being tested. It's not necessarily true that the most processed are easiest to understand.
Match It! (Exercise 3)
Processing Levels
Each stimulus is available at 5 levels of speech processing, ranging from level 1 (highly processed) to level 5 (faster than natural speech) according to the table below. The sounds are all pre-recorded in studio and pre-rendered.
Stages
There are 4 task stages, each associated with a specific number of response buttons in the trial and a maximum number of response clicks allowed:
NOTE: Athanassios Protopapas suggested higher maximum clicks for stages three and four, but they were never modified.
Categories
The stimuli are identical to those used in Exercise 4 (Phonic Streams) and consist of consonant-vowel-consonant syllables or single phonemes:
Category 1 consists of easily discriminable consonant-vowel (CV) pairs. Leading consonants are chosen from those used in exercise 2 and trailing vowels are chosen to make confusable leading consonants as easy to discriminate as possible.
Category 2 consists of easily discriminable CVC syllables. Stop, fricative, and nasal consonants are used, and consonants and vowels are placed to minimize the number of confusable CVC pairs.
Categories 3, 4, and 5 consist of difficult to discriminate CVC syllables. All consonants are stop consonants, and consonants and vowels are placed to maximize the number of confusable CVC syllables (for example, cab/cap).
Sound Replay (Exercise 4)
Processing Levels
Each stimulus is available at 5 levels of speech processing, ranging from level 1 (highly processed) to level 5 (faster than natural speech) according to the table below. The sounds are all pre-recorded in studio and pre-rendered.
Stages
There are 8 task stages, each associated with a specific number of syllables in the sequence, a specific sequence length, an ISI, and a specific number of response buttons in the trial (including the number of distracters). See HiFi Sound Replay Functional Specification, HiFi v0.6 SoundReplay spec, for details.
Categories
The stimuli are identical to those used in Exercise 3 (Phonic Match) and consist of consonant-vowel-consonant syllables or single phonemes:
Category 1 consists of easily discriminable CV pairs. Leading consonants are chosen from those used in exercise 2 and trailing vowels are chosen to make confusable leading consonants as easy to discriminate as possible.
Category 2 consists of easily discriminable CVC syllables. Stop, fricative, and nasal consonants are used, and consonants and vowels are placed to minimize the number of confusable CVC pairs.
Categories 3, 4, and 5 consist of difficult to discriminate CVC syllables. All consonants are stop consonants, and consonants and vowels are placed to maximize the number of confusable CVC syllables (for example, cab/cap).
Listen & Do (Exercise 5)
Processing Levels
Each stimulus is available at 5 levels of speech processing, ranging from level 1 (highly processed) to level 5 (faster than natural speech) according to the table below. The sounds are all pre-recorded in studio and pre-rendered.
Stages
These are 6 stages, each associated with a specific number of instructions in the trial.
Categories
There are four categories of instructions. Each category contains a distinct type of instruction. Categories 1 & 2 are “Click” instructions and categories 3 & 4 are “Drag” instructions.
In Category 4, the prepositional relationship could be: to, next to, to the left of, to the right of, between, above, below, etc.
Category 2 & 4 require that the movable foreground objects on screen appear in confusable sets. If the instruction is to “click on the girl in the red dress,” the character set containing a girl in a red dress, a girl in a green dress, and a girl in a blue dress must appear on screen. Refer to “Screen Requirements” later in this document.
Please reference “Listen and Do Instruction List.doc” for list of recorded instructions.
Story Teller (Exercise 6)
Stories
There are five stories, each in 20 story segments from 20-30 seconds long.
Story Complexity
Story complexity has not been implemented: The five stories are progressively more complex (story 2 is more complex than story 1, story 3 is more complex than story 2, etc.). Complexity is governed by:
Questions
There are five questions for each story segment. The questions will be presented in a random order
Response Buttons
For each set of 5 questions, 3 questions will have text response buttons and 2 will have picture response buttons. The responses are presented in a random order. There are four response buttons for each text response question and three (v0.5) response buttons for each picture response question.
Processing Levels
Each story and associated questions are at a different levels of speech processing, ranging from level 1 (highly processed) to level 5 (unprocessed natural speech). In other words, each story will only have one level of speech processing and all five will be at different levels. The order of the stories is as follows:
The Obstacle Course (Level 1) HiFi Stimulus Story Teller Obstacle Course
5-Ton Cake (Level 2) HiFi Stimulus Story Teller 5 Ton Cake
Urban Jungle (Level 3) HiFi Stimulus Story Teller Urban Jungle
Dahlias (Level 4) HiFi Stimulus Story Teller Dahlias
The Spot (Level 5) HiFi Stimulus Story Teller The Spot
The following document details the creation of the Story Teller stories: HiFi Stimulus Story Teller Stories
Title & End Screens
A title screen is displayed in the reward space at the start of each story. It is displayed during the first segment and remains until it is replaced by the first 10-Pack pop-up after a correct response. An end screen is displayed after the final block of a story is evaluated. If a benchmark animation is triggered, the end screen is displayed after the animation.
Triggers and Values . . . 1
High or Low? . . . 1
Tell Us Apart . . . 2
Match It! . . . 3
Sound Replay . . . 3
Listen & Do . . . 4
Story Teller . . . 4
Triggers and Values
The points awarded for a correct response in the HiFi exercises increase as the stimuli become more challenging. This helps to insure the participant continues to earn the same or more points in each session even if their correct response rate drops as expected when the stimuli become more challenging.
Points
Bonus Points
High or Low?
Task 1: Time Order Judgment (Sweeps)
Points Trigger: Correct Response
Points Calculation: Points Awarded=(Duration Index+1)
Bonus Points Trigger: 10 Correct Responses
Bonus Points Calculation: Bonus Points Awarded=(5×Point Increment)
Task 2: Sequence Reconstruction (Sweeps): REMOVED v0.5 s3a
Task 3: Time Order Judgment (Bursts)
Points Trigger: Correct Response
Points Calculation: Points Awarded=(Frequency Offset/Duration Index′1)
Bonus Points Trigger: 10 Correct Responses
Bonus Points Calculation: Bonus Points Awarded=(5×Point Increment)
Task 4: Sequence Reconstruction (Bursts): REMOVED v0.5 s3a
Tell Us Apart
Task 1: Two Alternative Phoneme Discrimination
Points Trigger: Correct Response
Points Calculation: Based on Stage (emphasis level) as shown below
Bonus Points Trigger: 10 Correct Responses
Bonus Points Calculation: Bonus Points Awarded=(5×Point Increment)
Task 2: Two Alternative Categorical Perception: REMOVED v0.5 s3a
Task 3: Three Alternative Phoneme Discrimination: NOT IMPLEMENTED v0.5
Task 4: Three Alternative Categorical Perception: NOT IMPLEMENTED v0.5
Match It!
Points Trigger: Correct Response
Points Calculation: Points Awarded=(Stage #Level #1)
Bonus Points Trigger: Grid Cleared
Bonus Points Calculation: Bonus Points Awarded=(2×(Max Clicks−Clicks))
Sound Replay
Task 1: Forward Temporal Match
Points Trigger: Correct Response
Points Calculation: Points Awarded=(Stage #Level #)
Bonus Points Trigger: Correct Responses
Bonus Points Calculation: Bonus Points Awarded=(5×Point Increment)
Task 2: Backwards Temporal Match: REMOVED v0.5 s3a
Listen & Do
Points Trigger: Correct Response
Points Calculation: Points Awarded=2×(Stage #Level #)
Bonus Points Trigger: Correct Responses
Bonus Points Calculation: Bonus Points Awarded=(5× Point Increment)
Story Teller
Points Trigger: Correct Response
Points Calculation: Points Awarded=2×(#of segments+Level #)
Bonus Points Trigger: On same schedule as 10-Pack bonus animations (see Reward Animation spec)
Bonus Points Calculation: Bonus Points Awarded=(5×Point Increment)
The first response listed is always the correct response. Questions with picture response buttons are always numbers 4 and 5.
5-TON CAKE
Segment 1
It was half past eight in the morning when Molly Lentil stepped out onto the porch of her tidy two-story Tudor home. Birds were chirping, lawnmowers burping, and a beautiful spring day was just getting into full swing. Molly sniffed the air—there was a faint aroma of fertilizer, which always reminded her of her first husband, Oswald. The corners of her eyes crinkled enthusiastically as she smiled. Molly picked up the phone. “Rush delivery,” she said.
Segment 2
A little while later, Professor Bertram Bramble happened by Molly's house accidentally-on-purpose while walking his Scotch terrier, Max. He found Molly hard at work, lining the inside of her empty, kidney-shaped swimming pool with aluminum foil. “What's this?” he asked, twisting his white handlebar moustache in a characteristic gesture. “Radio telescope?” The professor worked at the Mount Molehill Observatory and did, in fact, have a modest radio telescope in his own backyard.
Segment 3
Just then a large, green refrigerated truck with “Dana's Dairy Delivery” printed on the side pulled into Molly's driveway. A round, sweaty man with a nametag reading “Steve” and a ball cap that matched the truck got out and waddled uncertainly across the lawn. He scratched his head, his elbow, then his head again, and looked between Molly and the clipboard in his hand. “Did you order two thousand pounds of butter?” he asked. Molly nodded. “Sign here, please.”
Segment 4
As the butter was being unloaded directly into Molly's swimming pool, three more trucks arrived in rapid succession, carrying two thousand pounds each of sugar, eggs, and flour. The trucks managed to wedge themselves in such a way that each one was blocked by both of the others, and traffic on Juniper Lane came to a standstill. The driver of the sugar truck suggested that they play Rock, Paper, Scissors to decide who would try to move first, and they spent the next thirty minutes arguing over how it could be played with three people.
Segment 5
Professor Bramble coughed politely as he surveyed the contents of the trucks. “Is this your version of a mud bath,” he asked, “or are we making a few thousand cakes?” Molly chuckled. “Cake,” she answered, “but just one. Help me get this sugar into the pool.” They retrieved two snowshovels from Molly's garage and set to work. As they shoveled, a tanker truck slowly and expertly made its way around the traffic and up the sidewalk, demolishing a mailbox and a privet bush but otherwise causing no harm. A sign on the side said simply: “frosting.”
Segment 6
Hovering over Juniper Lane at six hundred feet, Rick Trellis was perplexed. In his four years as helicopter traffic reporter for WOGD radio, he had never seen such a large tie-up on a residential street. He counted five large trucks stopped at various angles in the middle of the block, as well as a construction crane, a limousine and what appeared to be a troop of boy scouts. Cars were backed up for a quarter mile in all directions. Rick felt it was his duty as a reporter to land and find out what was going on.
The mound of butter and sugar was shaped vaguely like Mount Vesuvius, and Molly was considering the problem of how to beat it all together when the tall, red-headed man tapped her on the shoulder. Molly sneezed in surprise. The man introduced himself as Rick Trellis, helicopter traffic reporter. His voice reminded Molly of Mike Wallace. He fairly squirmed with excitement when he found out what was going on. “I'm sorry,” Molly said, “Did you say you have a helicopter?”
Segment 8
A sizeable crowd of curious onlookers had assembled by this time, which was good because it took ten people to tilt the helicopter onto its side, blades in the swimming pool. “Stand back!” shouted an enthusiastic Rick Trellis as he belted himself into the cockpit and started the engine. The crowd murmured appreciatively as the butter and sugar were slowly transformed into a smooth, creamy mass. The lone dissenting voice came from a neighbor wearing a blue bathrobe, who was yelling something about a privet bush.
Segment 9
There are roughly sixteen thousand eggs in a ton. Carton upon carton of them were piled on the lawn in rough towers reminiscent of Stonehenge. Molly looked troubled. Just then, a jowly man wearing a khaki shirt with a red tie introduced himself and said that he had 27 boy scouts who'd like to lend a hand. Molly asked if the boy scouts had a merit badge for cooking. “Since 1911,” said the scoutmaster proudly.
Segment 10
Janet Wilmot had grown up in urban Pittsburg but had been obsessed with horses since the age of six, when her family had visited the canola farm of a relative in Manitoba. Joining the mounted police had been the fulfillment of a lifelong dream. She rode Twyla, a sturdy brown mare, down Juniper Lane into a scene of pandemonium. A group of boy scouts and a little league team seemed to be throwing eggs at a helicopter which had crashed into a swimming pool. A bird-faced man in a bathrobe ran up, shouting about a privet bush.
Segment 11
Janet flashed her badge efficiently at the small, curly-haired woman who appeared to be in charge. “Cake batter,” she noted sternly. “I assume you have a cake permit?” Molly produced an official-looking piece of paper so swiftly that Janet felt as though she were watching a magic show. She studied the angular blue scrawl at the bottom. “That really the mayor's signature?” “Ask him yourself,” said Molly, “he's over there by the limousine.”
Segment 12
Mayor Cyrus Bluebottle watched as the little league bucket brigade dumped sack after sack of flour into the pool. Flour filled the air like snow flurries, momentarily blotting out the sun and coating everything and everyone within forty yards. The mayor idly wrote his own initials in the dust on his limousine and thought how lucky he was that his PR advisor, Louise, happened to live on Juniper Lane. Four news stations were already setting up cameras.
Segment 13
Molly had been so busy dealing with the police, the mayor, the little league, the boy scouts and the television crews that she did not notice that Professor Bramble had disappeared until he reappeared with his son, Ernest, just as the batter was finally ready. They had brought with them an enormous lens, several yards across, balanced precariously in a little red wagon. Ernest extracted a tape measure from one of the many pockets of the overalls he always wore and began sizing up a nearby elm tree.
Segment 14
Ernest and the professor enlisted the aid of several burly construction workers with orange hard hats, who helped them hoist the lens into the elm tree. The lens, the professor explained, was an observatory leftover—it had been replaced by a bigger one twelve years ago and he had kept it in his tool shed ever since. They turned the lens so as to focus the sunlight onto the swimming pool, and the batter slowly began to cook. The air was filled with the pleasing if inexplicable scent of almonds.
Segment 15
Some of Molly's neighbors, led by Frank Murphy, a civil engineer who lived on the corner, temporarily dismantled their picket fences in order to construct an enormous cooling rack atop some cars stuck in the middle of the street. The construction crew brought in a crane that they had been using to move a fifteen thousand pound marble staircase. Fortunately, the aluminum foil lining the swimming pool was of the heavy duty variety. It held, and the cake was lifted free with minimal superficial crumbling. It lay steaming and vulnerable in the afternoon sun, a mouth-watering island in a hungry human sea.
Segment 16
The pumps on the frosting tanker were started and swiftly built to a jet-engine whine. Three men took hold of what appeared to be a fire hose attached to the left side of the truck and began spraying the cake with long, even strokes of sugary pink. With consummate skill they camouflaged an angular irregularity on the north end of the cake, where the pool steps had been, until it was a pleasing, gentle ripple. The crowd fell silent in admiration. A single voice called out: “Who's going to pay for my privet bush?”
Segment 17
Mayor Bluebottle was incredulous. “You mean you brought the silver scissors but no ribbon? Louise, how could you forget the ribbon? I've got to cut something for the TV cameras! What about that garden hose? I could cut that! Whose hose is this?” A bird-faced man who had been saying something about a privet bush spoke up. “It's mine.” The mayor thought about the television crews. “I'll give you a thousand dollars for that hose,” he said.
Segment 18
Although Mayor Bluebottle was the most notorious windbag in five counties, the cake dedication speech he gave before cutting the garden hose was his shortest on record. He had barely started talking about his father's uncle Wilbur, who had allegedly been a baker, when the hungry crowd began waving plastic forks in his direction. Unnerved, he jumped directly to his closer, quoting Marie Antoinette, and turned things over to Molly. She waved a paisley handkerchief in the air and simply said, “Dig in!”
Segment 19
It was the biggest thing that had happened in town since the Wallabies won their minor league division. The owner of the local supermarket passed out paper plates and napkins. Milk was poured into a dozen rented canoes. The drivers of the supply trucks discovered a common interest in music, formed a barbershop quartet on the spot, and sang a modified version of Sweet Adeline they dubbed “Sweet Cake-O-Mine.” Molly got a congratulatory phone call from the governor, and the cake was universally declared to be delicious.
Segment 20
When it was all over, and the people had gone home, and the fences had been put back, and the construction crane had lifted the supply trucks out of their mutual tangle, Molly, hands on her hips, surveyed her lawn. Crumbs covered it like a carpet of cherry blossoms. For a moment she felt as though she were standing in an Andrew Wyeth painting. Professor Bramble nudged her gently. “Why?” he asked. “Nice day for it,” replied Molly with a subtle smile. The professor twisted his moustache. “What are you doing tomorrow?”
The first response listed is always the correct response.
Questions with picture response buttons are always numbers 4 and
Dahlias
Segment 01—WC 95
In every aspect but one it was a usual day: Wyndham would awake at 5:30 with no alarm clock, do her horizontal stretches in bed with the morning light filtering across the north wall, and pick up her breakfast tray and reading material outside the bedroom door at 5:45. In either a navy or a taupe satin robe, with pockets, she would sit at a small table made of walnut and ash, its corners carved expertly into trailing vines with inlaid rose buds of cherry wood, and take her tea and egg and fresh fruit.
Segment 02—WC 87
She would pour a second cup, this time with sugar, two lumps, and peruse, from back to front, a gardening journal or newsletter, looking out now and then at the southeast garden below her window to check the bird bath or the light on the roses. Then she would dress and go about her day, which, if it were Wednesday, would include a bridge game, and if it were Monday or Thursday, pastries and coffee at the Green Door Café with her old friends Anne and Charles.
Segment 03—WC 86
What was different about this day, which happened to be a sunny Monday, was a small thing. It didn't take up any room, or much time, and it didn't interfere with her daily activities. But it changed everything. From this day on, her eyes had a reaching quality about them. This felt youthful, and pleasurable, and a bit annoying in its constancy. For now, she had a mystery. She had her own private who-done-it, the culprit yet to be discovered, the victim her own treasured tedium.
Segment 04—WC 95
It all began when she stepped out for her morning stroll. There, about six steps from the door, on the stone patio leading to the shady west path, sat a vase. A plain vase—nearly and intentionally invisible in its plainness, for in it was a stunning Nellie Broomhead, a rare dahlia, and not from her own garden—though it would certainly have been a worthy competitor, so perfect were its tubular petals, so strong its stem, so even the transition of shading from the pure white base to the neon mauve edge.
Segment 05—WC 123
Behind the exuberant heirloom flower, a small paper, rolled up like a miniature scroll, was balanced among broad leaves. Wyndham unrolled it at once.
It was at this instant her eyes changed, reaching for an answer before the question had formed—the way a nerve ending reaches for something electric before it arrives.
It was a poem, typewritten, in italics, unsigned. And she could not imagine its author.
She read it three times, possibly four:
For lovely Wyndham:
Against the painted garden wall
Where shadow kisses white
The light reveals the reaching bloom
As the bloom reveals the light
Wyndham carefully re-rolled the poem and tucked it where a woman tucks small treasures, a hiding place she had not made use of in a very long time.
An hour later over pastries—today it was croissant with almond and apricot filling—she did not mention the mysterious note to Anne and Charles. Nonetheless, Charles, being his usual observant self, commented as he stirred his coffee, “You're looking bright, today, Wyn. Such good color.” Anne agreed.
Wyndham knew they were referring to her cheeks, but she deflected, smoothing the cuff of her lavendar cardigan, “Thank you. This sweater often draws compliments—I must wear it more often.” She would save the story for the next time, when she had more information.
Segment 07—WC 136
But more information, as it turned out, was not forthcoming. She received no phone calls or visitors revealing the poet's identity. Her housekeeper Ivy claimed to know nothing about it, and before Wyndham had had time to gather any clues, she found another dahlia on Tuesday, the very next morning, on the stone bench along the herb garden.
This beauty, the elegant white Alpine Snowbird, was less rare but more playful, with eight simple petals curled like small tongues from a lemon-colored center. It was accompanied by the poem, with a second verse added:
Against the painted garden wall
Where shadow kisses white
The light reveals the reaching bloom
As the bloom reveals the light
And once revealed, this lustrous pair
a destiny doth hold
compelled by such a simple task:
to let the thing unfold
Segment 08—WC 104
On Wednesday, Wyndham awoke, did her horizontal stretches, and picked up her tray, as always. She poured her second cup of tea, plunked in two lumps, and looked up from her reading to watch a family of finches in their comical bathing routine, fluffing and preening, until the gardener scared them away with a wheelbarrow of compost on his way to the delphinium.
Mid-swallow, she suddenly gave a surprised cough, almost dropping her cup. Robert? she thought, then immediately said aloud, “Oh, that's absurd.” But she watched him, his broad graceful shoulders leaning into the tall blue stems, until he had emptied the wheelbarrow.
Segment 09—WC 109
Robert, she realized, was the only other one who would know her walking routine: the herb garden and rosebeds on Mondays and Wednesdays, the west path on Tuesdays and Thursdays, and on Fridays, saving the best for last, her prize-winning dahlia beds. No one but he and Ivy would have known where to place the vases where she would see them, and Ivy had already sworn innocence.
But Robert? He had been with her for years, and although they both knew flowers intimately, and had shared everything two people could share about loams and fertilizers, hybrids and tubers, he was private in other respects. She knew little about him.
Segment 10—WC 91
He does have deep, intelligent eyes, she thought, and uses his horticultural Latin quite well. She could picture him a poet, ensconced in a small private library, tending words the way he tends new shoots.
But her gardener? What a scandal that would be among certain of her acquaintances. And socially, how would that work? Would she have to stop employing him? Perhaps she would just marry him, and his livelihood would simply become his hobby. “Stop that,” she chided herself aloud, and pushed her chair back and rose to dress.
Segment 11—WC 123
At The Green Door Cafe later that morning, enjoying an assortment of tiny strawberry tartlets—creamed, brandied, adorned with chocolate—Wyndham described the situation as casually as possible to Anne and Charles.
“It appears I have a suitor,” Wyndham announced. Anne's fork, Charles' cup, and Wyndham's words all hung in mid-air.
“A suitor!” Anne exclaimed, setting her fork back down with a wedge of pink still balanced on it. “Wyn, tell us!”
“There's not much to tell,” Wyndham said, “he's anonymous. I've been receiving notes.”
“Notes?” Charles said.
“Yes,” she replied, not mentioning the flowers or poems, “I suspect I know who it is, but I'm not quite sure how to respond, and I thought you might be able to offer some advice.”
Segment 12—WC 109
They eagerly agreed to advise her as best they could, and she went on. “Imagine being alone as long as I have. I have everything I need. I'm satisfied, and with my dahlias winning more prizes every year I always have something to strive for, which is important. Besides, . . . ”
Anne interrupted her, “Are you trying to convince yourself that you're not interested in a possible romance?”
“Not a complicated one.” Wyndham said. She waited for the waiter to refill their coffee cups, and then continued. “And this one—he is handsome and intelligent, a poet, but . . . well, perhaps a bit below my station, if that doesn't sound too crude.”
Segment 13—WC 152
Charles wiped his mouth with his napkin. “Below your station? How do you mean?”
Wyndham skirted the question and went on to explain that her life was going smoothly, and she wasn't sure she wanted to stir the waters.
“For Pete's sake, Wyn, stir the waters! You're bored, face it,” Charles laughed, and Wyndham gave him a playful glare to show she knew he was right.
The three of them talked for two hours, with too many refills, and finally they ordered lunch just to counteract the caffeine. Anne played the devil's advocate, while Charles, being Anne's older brother, took the opposite role, and goaded Wyndham to take a chance, get bold, let her hair down. Finally he looked sincerely into her eyes, and said, “If you think there is any chance you could love this mystery poet, then give it your all.” By the time they parted, Wyndham was convinced.
Segment 14—WC 147
Friday Wyndham awoke earlier than usual and went straight to her desk. “Dear Robert,” she wrote on plain stationery, “Please meet me in the dahlia bed . . . ” She stopped, thinking better of her wording and began again on a fresh piece of paper. “Please meet me in the dahlia garden at 10:00. We have flowers to discuss.” Then, feeling bold, she added, “And if you would, please check the rose garden below my window first thing this morning for . . . ” for what, she wondered, “for flaws,” she decided ridiculously, adding, “I'm sure I will enjoy the view . . . knowing it is perfect.”
She sealed her note in an envelope and wrote another to Ivy. “Please post this memo for Robert on the garden shed so he'll see it when he arrives. Thank you.” She leaned the note and the envelope outside her door and awaited the arrival of her breakfast tray.
Segment 15—WC 95
She watched Robert among the roses, all the way through her first cup and her second, and she barely touched her fruit. He inspected them tenderly, his tan arms moving gracefully among the thorns. He moved below her, never glancing up, and she imagined herself loving him for that quality of restraint, and even more for the subtle seduction of poses struck for her benefit—the slow arching of his back as he stretched in the morning sun; the pause for a deep breath of the fragrant Damask rose; the way he removed his vest.
Segment 16—WC 99
When he moved on to the herbs she moved with him, opening up the west bedroom for the first time in years to stand in her robe, unseen at the window. She had never appreciated the poetry in his routine—the rhythm of his movement as he trimmed back the rosemary, shaped the edge of thyme along the curving stone where it trailed, clipped the lavendar to hang upside down on the trellis to dry for flower arrangements. She imagined how his hands must be scented now, with the herbs and the sun and her eyes all on them.
Segment 17—WC 97
Back in her own room, Wyndham pondered her closet. It seemed unusually drab, and what could possibly be suitable for a casual-but-not-casual meeting among dahlias?
She wished she had something common and loose, in tangerine, three light layers with two of them sheer—but she did not, and settled on a comfortable light green garden smock. She bathed and dressed without looking in the mirror, so as not to risk disappointment, relying on the feeling in her skin and eyes to give her confidence.
At one minute after ten, she walked into the dahlia garden toward Robert.
Segment 18—WC 113
Robert was on his knees with his head hidden in a riot of color, tossing handfuls of young weeds into a pile behind him. She watched the movement of his hips a moment while wondering how best to get his attention.
She picked up a wooden garden stake nearby and tapped him smartly on his bottom.
He bolted upright and saw her standing there, smiling. “S-sorry, just . . . just cleaning the beds,” he sputtered, glancing at the stake still in her hand, “I didn't hear you coming.”
“That's quite alright,” Wyndham smiled, “it seems surprise is the theme, of late.”
They stood there looking at one another for a long moment. “Well.” Wyndham finally said.
Segment 19—WC 113
Made nervous by her strange smile, and his backside still remembering if not stinging, Robert simply replied, “The flowers?” referring to the note on the garden shed.
“Yes,” Wyndham said. “Let's start with the Nellie Bloomhead, shall we?”
She expected him to say something, but instead he led her down the path, and she followed, until they came to her bright prized Nellie Bloomhead. He adjusted the stakes and twine, and then before he could say a word about the health of the plant or what he'd been feeding it, he felt her hand on his neck. There he stood, between Nellie and Wyndham, frozen as the sun itself on the summer solstice.
Segment 20—WC 128
“Against the painted garden wall . . . ” Wyndham recited the first line of his poem, certain he would join in.
“Ma'am,” Robert whispered shakily.
Wyndham took a step back. “Do you know anything about a Nellie Broomhead left for me on the patio, or an Alpen Snowbird on the stone bench?”
“Yes, ma'am. Your friend Charles left them. He's taken an interest in dahlias.”
Like a fallen skater bouncing up from the ice, like a chef catching a descending tomato before it hits the floor, Wyndham's recovery was flawless.
“Tell him, please,” she said, “next time you see him, that his taste is impeccable.”
She dismissed Robert from their 10:00 meeting, and strolled down the long rows of exploding color, thinking about Charles, his long graceful hands and kind heart.
The first response listed is always the correct response.
Questions with picture response buttons are always numbers 4 and
The Obstacle Course
Segment 01—WC 108
Rita's hands trembled. “Officer, there's been an accident,” she said. She held the phone in one hand and jotted notes with the other in pink highlighter, the only thing she could find after her purse had spilled its contents on the floor of her car.
“Goodwin,” she said, spelling out her last name.
Rita regained her bearings as she answered the officer's questions. She didn't seem to be hurt, just rattled. Her car, on the other hand, was badly dented and scraped—that beautiful cobalt blue custom paint. It was a birthday present that her late husband had been so proud to unveil for her in their driveway.
Segment 02—WC 93
After a long afternoon dealing with a ruined fender, phone calls, and the tow truck that hauled her poor car off for repairs, Rita finally took a taxi home. She arrived to find that her daughter-in-law was preparing her favorite meal of stuffed game hens and asparagus salad. She thought such doting was pleasantly uncharacteristic until the meal was done and dessert was served. Then, as her son cut into the lime pie, he turned the conversation to an old sore spot.
“Mom,” he said, “we really think it's time you stopped driving.”
Segment 03—WC 109
Rita bristled. “It wasn't my fault,” she said firmly. “There was a chunk of metal in the road. I had to swerve to miss it and scraped a fire hydrant. My driving is fine. My eyesight is fine. My judgment is fine. Now goodnight. I've had a long day.”
Her son poured more tea and said he knew how strongly she felt about it, but he wasn't convinced that at age 75 her reflexes were what they used to be. He assured her he was just worried for her safety.
Rita looked at him knowingly and simply said, “What would you do with my car?”
Her son didn't answer.
Segment 04—WC 89
A few days later Rita sat patiently in the waiting area of Sammy's Auto Body Shop.
She studied the coffee table in front of her made of stacked tires and a top of thick green glass. Among the magazines and brochures scattered on it was a small pile of yellow flyers. “Drive Like A Pro!” the heading said. “Starting October 16th—Safety Driving Course.”
She picked up a flyer and read further. “Your Car or Ours and a Proven Obstacle Course!”
Rita wondered how one proves an obstacle course.
Segment 05—WC 114
As Rita tucked the flyer into her purse, a workman stepped in from the back. He held the swinging door open with his foot while he wiped his hands with a rag. Through the opening Rita saw a teenage boy bending with great concentration over a piece of painted metal, rubbing intently with a cloth. One hand moved in fast circles and the other occasionally wiped something white on the leg of his excessively roomy pants. He seemed to badly need a belt. The way these kids dress, Rita thought, the girls are too skimpy and the boys too baggy.
The man interrupted her thoughts. “Your car's ready, Mrs. Goodwin. It's out front. Sorry for the wait.”
Segment 06—WC 128
The following Saturday, as part of a deal with her family to prove that she was not a risk on the road, Rita was seated in the Safety Driving Course. The room smelled of onion bagels and auto grease. The October sun sliced across the middle of the whiteboard on the wall. On the board was a diagram of lines and triangles in green marker, punctuated by little car-shaped magnets.
There were 10 people in the class, of different ages and types. She noticed with some pride that she was clearly the most senior person there. Directly in front of her was a man with a shiny bald head. To his right was a woman in supposedly casual clothes that looked like they'd been prepped by a butler.
Segment 07—WC 87
On Rita's right, a teenage boy with rather frightening orange hair was bouncing his leg nervously. Suddenly she recognized the pants and their white stains. “Pardon me, young man,” she said, “what's your name?” The boy looked at her without answering. “Your name?” she repeated.
“Marco.”
“Hello Marco. I'm Rita Goodwin. Do you work at an auto body shop?” He nodded, barely. She tried not to stare at the small silver ring in his eyebrow. “I saw you there last week. You were working like a fiend.”
Segment 08—WC 108
Marco looked cautiously at Rita. His eyes changed almost imperceptibly. “Are you the lady with the blue Cadillac?”
They were interrupted by the instructor who began his lecture.
Marco pulled a purple pencil with a big black eraser on it out of some invisible compartment in his hair and began taking notes furiously. Rita studied him from the corner of her eye. She felt repelled by the chaotic hair and dirty pants, but she was curious too. She liked contradiction, and here was a live one: an industrious teenager; a shy rebel. She noted the bouncing leg again and wondered if he was hyped up on drugs.
Segment 09—WC 99
When they were dismissed for a mid-morning break, Rita turned to Marco again and said, “Yes, to answer your question, I'm the lady with the blue Cadillac. It was a gift from my husband the same year he died.”
Marco didn't know how to reply to this, and excused himself to find the vending machines. When he returned, Rita had another question for him. “I don't mean to be rude, but what is the point of this . . . ” she waved her hand at him, “ . . . this outfit you have? Are you trying to scare people on purpose?” Marco sat down without answering.
Segment 10—WC 96
That afternoon the class moved outside and took turns practicing on the course—skids, swerves, and sudden braking. On the bleachers, watching the woman with the pressed jeans show off in her black Lamborghini, Marco finally spoke. “What are you doing in this class, anyway?” he asked Rita.
“You mean at my age?” she replied. Rita explained about the accident, and her family's request that she stop driving. She added, “They won't admit it, but they want my car for my granddaughter who just started college. I'm sure she'd trade it in for a tin can.”
Segment 11—WC 119
“And you?” Rita said. “Why are you in this class?”
Marco blushed. He removed and redocked his pencil in his rough sea of hair. “My parents don't drive, and I need to get really good.”
“For a girl?” Rita guessed. Marco blushed, and just shrugged.
“I'm working after school and weekends for a car. My parents don't think I can save enough and afford insurance too. But I really want it, and I hope I can make enough to buy it before April.”
Rita guessed again. “For the spring prom?”
“Who are you, lady?” Marco said,-annoyed, and blushed again.
The instructor promised wet roads and oil slicks for the next class, and dismissed them until the following week.
Segment 12—WC 72
The next week, Rita showed up with blue leather driving gloves, ready for action. Marco wore the same pants, which looked just fine with the slightly dented gray hatchback provided for him by the driving school, and Miss Lamborghini showed up in shiny black capris, a tight black sweater, and stark white running shoes. They all wore sunglasses, as recommended by the instructor who'd said to prepare for glare and reduced visibility.
Segment 13—WC 101
On the bleachers, waiting for her turn to maneuver her way through an oil slick, Rita apologized to Marco for prying the week before.
“That's ok,” Marco said. “I guess I forget you were my age once.”
“I certainly was,” Rita said, securing a hair pin at the back of her head. “And what a great age it was. I remember the proms, and the dancing-but then, I went dancing every Saturday.”
Marco raised his eyebrows. The skin under the silver ring puckered, and Rita tried not to look at it.
“Oh yes,” she said. “Every Saturday at Sweet's Ballroom.”
Segment 14—WC 103
Her eyes sparkled. She couldn't believe Marco had never heard of the musicians and bands that had been so famous—Benny Goodman, Glenn Miller, Tommy and Jimmy Dorsey.
“I remember the mirror balls, and being right below the stage, watching the Andrews sisters—those turquoise gowns.” Rita paused.” And, oh—what was that drummer's name—oh, Gene Krupa! He got arrested for marijuana the night after I saw him, you know! At the Golden Gate Theatre.”
Marco asked if she'd had a boyfriend. “Oh yes,” she said. “Clarence. He was a true gentleman. He bought me a gardenia corsage every Saturday. 25 cents a piece.”
Segment 15—WC 77
The instructor called Marco up next to run the obstacle course; Rita watched him. Sudden stop for the rolling basketball. Slow correction for the oil slick. Steady soft brake on the simulated ice patch. Six orange cones, unscathed.
I hope he gains as much confidence with the girls, Rita thought. He'd be a handsome young man if he'd fix that hair and get some new pants.
The instructor's voice broke her thoughts. “Rita Goodwin. You're up next.”
Segment 16—WC 116
At the end of the afternoon, certificates were handed out to eight of the ten students. Rita fished a roll of tape from her purse and mounted hers to the inside of her back windshield, with every intention, she said, of backing into the driveway when she got home so her family could see it from the dining room table. Marco tucked his in his notebook. He waved to Rita as she got in her car, and started walking toward his bus stop.
Suddenly he heard an unmistakable scraping sound, and turned around just in time to see the cobalt blue paint peeling off the back right fender, and Rita's face turn about three shades whiter.
Segment 17—WC 93
“It could happen to anybody,” Marco said, trying to quiet Rita who was now sobbing into her blue gloves. “That railing was right in your blind spot.”
“I don't know,” Rita shook her head. “Maybe my son is right. Maybe it's time to let go. Perhaps I'm not as quick as I thought.”
“What are you going to tell your family?” Marco asked.
“I think I'll just park it next to the hedge. They won't see the dent tonight,” she said, removing the certificate from the windshield. “I'll sleep on it. Maybe they're right.”
Segment 18—WC 108
Rita didn't sleep well at all, in fact. The next morning, she started her blue Cadillac for the last time. She pulled out away from the hedge and into the driveway, in clear view of the front door. There she listened to the car radio and waited for her son to come out of the house on his way to work. She had her title in hand, a business card for Sammy's Auto Body Shop, and her words ready.
When her son saw her sitting there in the car, he stopped and looked a moment. Finally he came around and opened her door. “What are you doing, Mom?”
Segment 19—WC 115
Rita got out and walked around to the other side of the car. “Well, this . . . ” she started to say, indicating the place where she expected a long ugly scrape. But there was nothing—nothing but a beautiful blue fender. Not a mark. In fact, it looked especially shiny—a little shinier than the rest.
“Mom, what's up?” he said.
“Um, oh . . . nothing,” Rita said, confused. “I mean, I . . . I wanted to show you my certificate.” She opened the door to get the certificate off the front seat, and there, underneath it, was Marco's purple pencil with its big black eraser.
“Congratulations, Mom,” her son said unconvincingly, while Rita's mind reeled. Marco, she thought, crossing her arms, how in the world . . . .
Segment 20—WC 91
Early the following April, Marco returned to the shop to find an envelope addressed to him. Inside, a note said, “Hello, Marco. I hear you're still working hard for that car. I'm going on vacation for three weeks, and thought you might enjoy a loaner for the prom.” Marco stared at two silver Cadillac keys taped to a blue card.
The note continued, “No horseplay or smoking please. Otherwise, have fun and be a gentleman. Everyone loves a true gentleman. Rita.”
Marco broke into a smile.
The first response listed is always the correct response.
Questions with picture response buttons are always numbers 4 and
The Spot
Segment 1
Jeremy was an open-hearted and gregarious man. He always took the time to stop and chat with those he encountered. His three brothers often joked that you couldn't tell Jeremy anything you didn't want the whole town to know. But on one subject, Jeremy was uncharacteristically mute. trout fishing. In another family, his silence on the matter might have gone unnoticed, but the Madden family was mad for trout. As boys, Jeremy and his brothers had been avid fishermen, often skipping school to sneak away to a quiet creek. Their findings were usually unimpressive—maybe a couple of fish for supper. But when Jeremy became an adult, he stopped going out with his brothers. And at their yearly Thanksgiving gathering, where conversations centered around bait, fishing techniques, and good fishing spots, Jeremy kept quiet.
Segment 2
Jeremy wasn't the only Madden who didn't talk about fishing. The other was Jeremy's father, Martin, a soft-spoken engineer who privately felt fishing was cruel and unfair. Born and raised in Manhattan, he did not understand his sons' fascination with fishing and was deeply relieved they did not hunt. But while Martin's silence was forgiven, Jeremy's was not, for when Jeremy disappeared in his orange fishing vest he always came back within a few hours lugging a big cooler full of the fattest trout they'd ever seen. The smallest of his catch were twice as large as the largest ones his brothers had landed, and his mother always said the pink flesh tasted absolutely perfect buttery with a hint of almond and grass.
Segment 3
Over the years, Jeremy's brothers had tried everything to find out where Jeremy fished, from attaching a radio tracker to his coat to breaking into his truck to check the odometer. Many an afternoon had been spent on a wild goose chase. Finally, they'd given up. Jeremy's secret spot was declared the unsolvable family mystery. He wouldn't even share it with his son, Lucas, who tagged along with his uncles and cousins instead. Still, whenever his family planned an early-morning fishing trip, Jeremy was always awake to hand his sleepy son a sandwich and a thermos of hot tea. He watched from the veranda as his brothers and son packed up their rods in the bluish dawn. As they drove away, he waved to his son, and Lucas always waved back. While he waited for their return, Jeremy played chess with his father.
Segment 4
After they waved to the departing party, Martin and Jeremy sat down on the veranda steps like two mountain men, a glass of whisky lemonade in one hand, a bowl of hot porridge in the other, a thick wool blanket on each of their laps. Their chess games would often last for hours. Much of the time, they sat in a companionable silence, though they occasionally talked about their families and lives. In this way, Martin and Jeremy built a strong relationship. Jeremy considered Martin to be one of the finest men he knew. Martin felt the same way about Jeremy.
Segment 5
Once every Thanksgiving week, Jeremy vanished like a tomcat on an adventure of his own. A few hours later, he'd reappear with a cooler full of giant, glistening trout and Lucas would feel quite proud of his father no matter how secretive he was. To Lucas, there was nothing as beautiful or mysterious as these trout from Jeremy's secret fishing spot. The small trout he'd caught were a muddy grey, but Jeremy's monsters flashed pink and the gold of their eyes was a pure, wet gold. They were so much more alive than anything he'd seen before and, although he loved trout dinner, Lucas wanted to throw them back into their depths so they could go on being immortal.
Segment 6
Strangely enough, there was no special trout clock for Jeremy. Sometimes he'd be gone as early as five in the morning. Other times, he slept in, loitered, then vanished late in the afternoon. This year at Thanksgiving his brothers had taken to teasing Jeremy's wife, Mary, and offering outrageous bribes if she would tell them Jeremy's secret. Mary shrugged and said she preferred turkey anyhow. She guessed that her husband simply went out and bought them at the local fish stall. Maybe the farmer next door stocked his pond with trout. She was certain that he didn't catch them himself. One time, he'd left the fishing pole behind. And he'd still come back with lots of trout.
Segment 7
Jeremy's brothers were frustrated by the fact that Mary would not tell them anything. Mary, in turn, felt a particular gleam of revenge when one afternoon her husband brought home fifteen of the most beautiful trout the family had ever seen. That evening, she rolled the trout in hazelnuts and fried them in butter. Everyone ate in complete silence and not a scrap of trout was left. Afterwards, as each of them sat in quiet satisfaction, Jeremy's father smiled and asked the magic question: “Jeremy, just where is this trout from? Do you sing to them?” Jeremy smiled back and said, “No, I don't sing to them.”
“Well, I'm curious, that's all.”
“Are you?” asked Jeremy. His father nodded.
Segment 8
The next morning, just before dawn, Martin was woken by someone gently shaking him by the shoulder. “Get up, Dad. We're going.”
Martin opened his eyes. Through the blackness he could see the silhouette of his son. Jeremy, fully dressed, was holding out a thick grey sweater. Martin flicked on a lamp.
“Wear this, Dad,” Jeremy whispered.
Looking closer, Martin saw that Jeremy had a toolbox in one hand. He was wearing thigh-high wading boots. Martin glanced at his clock. It was four in the morning. The stars were still out.
Segment 9
“I don't usually get up at four. Why are you up?”
“This is the only way we're going to give them the slip.”
Jeremy's father blinked. “Give who the slip? Why?”
“We're going to the trout. Just the two of us.”
“Now?”
“Yes, now. I've packed breakfast. We'll be back by noon.”
“Don't fish sleep?”
“They do and they don't. Are you coming?”
“I wouldn't miss it,” Martin said. He sat up and put on his glasses.
While Martin dressed, pulling on his favorite green pants and a heavy plaid shirt, Jeremy searched for Martin's rain boots. He found them in a corner of the room, shook out a spider, and handed them to Martin. Martin stepped into them and started walking toward the garage.
Segment 10
As Jeremy and Martin got in the pickup truck, Martin asked, “Are we going to the famous secret Fishing Spot?”
“Yup.” Jeremy responded.
Jeremy began to drive. Martin sat silently in the passenger seat, thinking about Jeremy and his secret Spot. “You know your brothers would do anything to go with you,” he pointed out. “And Lucas. Why not take him?”
“I don't think he's ready for it,” Jeremy replied.
“But I don't even fish,” Martin pointed out.
“I know, but there's something I want to show you,” explained Jeremy.
“So we are going to the Spot. Your spot,” Martin repeated.
“Well, I'm going to have to blindfold you,” Jeremy joked.
Segment 11
Martin looked out the window, and saw they were passing the gas station. It was open. He hadn't realized it was open 24 hours a day.
Martin thought about asking Jeremy to stop for a cup of coffee. “How far away is it?” he asked.
“Not far,” Jeremy responded.
They drove for another minute or so then took a bend in the road. This was the part of the creek that was closest to their house, a fifteen minute walk at most. It was a nice place for a cool swim but Martin had never seen trout there. They stopped the truck.
Jeremy laid out a blanket for his father to sit on. He poured coffee from a thermos, and handed his father a cup.
“Thanks, that's just what I needed,” his father said. In the dark, Martin heard a frog croaking.
Segment 12
Martin waited quietly for Jeremy to tell him what was going on, and why they were there. But Jeremy seemed to be in no hurry to explain. Instead, he sat down beside his father. He took English muffins and homemade sausages out of a bag, and handed one of each to his father. As the two men ate, the first sign of the morning light began to drive the darkness away.
Only when he had finished breakfast did Jeremy stand up. “Watch,” he said. Martin watched. Jeremy waded into the creek and stood among the green crayfish and schools of translucent minnows. Then an enormous trout swam up to him. It swam between his legs, doing a figure eight and caressing his ankles.
Segment 13
Martin's first reaction was surprise. “I've never seen a trout out here before,” he said. Then he laughed. “That fish thinks it's a cat!”
The fish swam around until Jeremy reached into the cold water. As Jeremy pulled the fish out, Martin saw just how large the trout was.
“That trout is the mother of all trouts,” said Martin admiringly. It was as thick as Martin's leg and it had a prehistoric look to it. Several hooks were tangled in its mouth, signs of the trout's victory over more than one fisherman. “It would be a pity to kill something that's lived so long.”
“I agree. We'll let it go. There are plenty of others,” Jeremy replied, as he gently lowered the trout back into the water.
Segment 14
Jeremy came back onto shore and sat down on the blanket. “I have always liked to come here when something's on my mind, or just to get some time by myself,” he began quietly. “It's a place where I can work through problems. One night about ten years ago, I was out here swimming. I remember the moon was full. Suddenly, trout started coming at me from every direction. They kept coming until I could hardly see the water through the trout. But instead of feeling panicky or nervous, I felt more relaxed than I had in a long time. Since then, I come back here whenever I can.” Jeremy looked pensive. “I don't like to keep any secrets from anyone,” he said, “but somehow this feels very private. Not to mention a little weird—I mean, I like to hang out with mystical trout.”
“I guess that is pretty unusual,” Martin agreed.
Segment 15
Jeremy looked at the sky. It was still early morning, but the sun was on its way up. “Well,” he said, “I guess it's time we caught some trout before everyone starts to wonder where we are.” He walked back into the creek. Trout came swimming toward him from who knows where. He dipped a net in the water, and four trout obligingly swam right in. Within a matter of minutes, he had bagged sixteen beautiful fish. He put the fish in the cooler, and hauled the cooler to the back of the pickup.
“Dad?” he said, as he picked up the blanket and threw it in next to the cooler.
“Yes?”
“Part of the reason that I brought you out here was because I wanted to share this secret—of the Spot—with someone I thought would understand. But I also wanted to ask your advice about Mary.”
Segment 16
“What's happening with Mary?” Martin said. “Don't tell me you aren't happy with her. She's so lovely.” “No, no, nothing like that,” Jeremy replied quickly. “But she's acting a little strangely lately, and it's got me worried.”
“What's she doing?”
“She locks herself in our bathroom every night, sometimes for hours at a time.”
“Well, maybe she's taking a bath. It's hard being a mother and a wife. She may just need a break.”
“Maybe . . . But while she's in there, I can hear her typing and giggling. And she always brings the newspaper with her. She says she's reading about the Senate elections, but they've been over for two weeks and she's still reading the paper. And giggling.”
“At least you know she's not depressed, what with all that giggling.”
“That's true. I just wish she'd let me in on whatever it is that she's doing.”
Segment 17
“Have you tried asking her?” asked Martin. “Sometimes it pays to be straightforward.”“I asked. She smiled and said ‘A rose is a rose is a rose’.”
“Did you ask her what that meant?”
“Yes. And she said that ‘A rose by any other name would smell as sweet.’”
“That's Shakespeare.”
“I thought so. Anyway, I asked her what that one meant and she just gave me another riddle. She told me to look under the rose. So I dug up the rose bush, but there was nothing there.”
Martin smiled. “Mary has always been clever. I guess you'll have to do some detective work to figure out her secret.”
“You don't think I'm invading her privacy?”“I don't think she'd be giving you clues if she didn't want you to find out.”
Segment 18
A few days later, Jeremy and his father were sitting on the veranda steps. “I read yesterday's newspaper very carefully,” Jeremy said, “to see if I could figure out what interests Mary so much.”
“That was a good idea.” Martin replied.
“Rose came up a couple of times. There was an article about fertilizing rose bushes, and an advice column by somebody called “The Rambling Rose,” and a marriage announcement of Beneford Rose to Jennifer Garner.”
Martin mulled this over for a few moments. “You know,” he said, “‘under the rose’ is the Latin translation of sub rosa. It means something that is secret or confidential.”
“Hey, that's interesting,” said Jeremy.
Segment 19
When Jeremy got home, he kissed Mary hello and took off his jacket. Mary was in the kitchen, stirring a pot on the stove. “Smells delicious,” Jeremy said. “New recipe,” said Mary. “I got it out of the paper.” Jeremy looked down at the recipe. Beef stew with dumplings. His mouth was almost drooling in anticipation. Then he saw that peeking out from under the recipe was another section of the paper the Rambling Rose advice column. He began to understand.
“What's sub rosa?” he asked her.
“Haven't you guessed yet? I'm the Rambling Rose.” Mary laughed as she said it.
Segment 20
“Hasn't that column been around for awhile?” he asked.
“Yup. Sue Repko's been writing it for twelve years. Now that she retired, I'm the new gal,” Mary explained, as she dished the stew into three bowls.
“Why did you keep that secret from me?” Jeremy asked.
“I ladle out advice to the lovelorn. You sing to addled trout—or whatever you do. I don't ask. Everyone has a secret spot, Jeremy. In fact, there are hundreds of secret spots in each of us—each a cool dark watering hole in speckled shade. That's where we go to replenish our own reserves when we've given all we have to the people around us,” Mary explained as she set the bowls of stew on the table.
Jeremy understood perfectly.
The first response listed is always the correct response
Questions with picture response buttons are always numbers 4 and 5.
Urban Jungle
Segment 1
Joe licked some mustard off his hot dog and grinned at his friend. “Beautiful!” he said. Marv beamed back. He knew what Joe meant. The sun was shining in Central Park on this crystal spring day. They were sitting on the steps of the seal enclosure in the zoo. It was good to be alive, still friends after more than half a century. It was good to feel the sun warming their scalps, through Joe's white curly hair and on Marv's gleaming bald pate. It was good to stretch their legs and to see the excitement of the children gathering around for the seal feeding.
Segment 2
Joe lived near Central Park, in a fancy apartment on the Upper West Side of Manhattan. Marv lived across the East River in Queens, near his successful auto body shop. They had been friends since World War II, when Joe was a medic and Marv a gunner in the Marines. They had lost touch after the war, and then met again in strange circumstances.
Once a month, rain or shine, they met at the zoo. They were both nearly 80. Now they watched the seals jumping for fish in their pool. Suddenly a group of young zoo workers in green shirts came running towards them and pulled out bullhorns. “Attention, people! Don't be alarmed!”
Segment 3
The seal feeders stopped throwing fish, and everyone turned toward the bullhorns.
“We have a situation here and we could use your help, ladies and gentlemen, and especially kids. One of our Japanese snow monkeys, a small baby, has somehow gotten out of her enclosure. We are cordoning off the zoo. We are very sorry, but nobody is allowed to leave or come in until we find our monkey. Kids, please look out for this baby. If you see her, tell someone with a green zoo shirt immediately. She has light brown fur and a pink face. Don't be scared, but do NOT try to approach or touch the monkey. She is not a pet. She is a wild animal.”
Segment 4
The crowd buzzed with excitement. Some of the mothers snatched up their babies and looked around in panic. “Oh my God!” said a young woman in a yellow shirt and bright green pants. “A wild monkey, ohmigod!”
“Cool!” said an eight-year-old boy in baggy shorts to his friend, another eight-year-old boy wearing a T-shirt with two cartoon fish on it. “A wild monkey hunt! Let's find it and capture it!”
Segment 5
Marv and Joe went over to the monkey enclosure, and made their way through a crowd that was leaning over the parapet and staring at the distraught mother monkey, who was sitting on a rock and shrieking for her missing baby. She had thick light brown futr and a bright pink face. She jumped from the rock onto a tree and looked around frantically. In the moat at her feet, a pair of black-necked swans floated peacefully by, followed by tiny cygnets.
Segment 6
“Mama, look! Piggyback!” squealed a small girl next to Marv. He looked to see where she was pointing, and sure enough, a fluffy grey baby swan was riding on one of its parent's backs, looking perfectly content and oblivious to the commotion around it.
“I remember taking my babies piggyback in this park,” said Joe, smiling sadly. Marv looked at his friend, then back to the South American black-necked swan. He felt sorry for his friend. Although his friend was wealthy after a lifetime of being a successful cosmetic and reconstructive surgeon on Park Avenue, his children lived far away and he and his wife were quite lonely. He didn't have any grandchildren; Marv had six.
Segment 7
They moved onto the polar bear section. There, swimming in his tank, was Gus, the legendary Central Park polar bear. Gus is a famous personality, written about in magazines and newspapers, and even making the occasional TV appearance.
“Remember the psychiatrist?” Joe asked Marv, and they both chuckle
Segment 8
“Yeah, I remember,” said Marv. He leaned against the wall and raised his foot to retie his shoelace. “Let's face it, the bear might look happy now, but I bet he would rather be in the snow somewhere, free to hunt and roam. A polar bear in New York—does that sound right to you?”
Joe moved aside to let a zoo worker squeeze up to the edge and lean in to look for the missing monkey, in case it had somehow got into the polar bear tank. The young man took off his sunglasses and peered into the depths of the tank.
“I know how Gus must feel,” Joe said. “I used to feel like a polar bear in New York, when I first opened my practice.”
Segment 9
Marv clapped his friend on the shoulder. “I bet you did!” he said. Joe was African-American, and when he opened his own practice on East 79th Street, he was a rare specimen. He had dealt with racism and bigotry. He remembered one woman who had literally run out of his office when she met him, and that was after desegregation. By the time he had retired ten years ago, things had changed a lot, but he had never forgotten his first years in the business. He had risen to the top of his profession purely because of his skill and what Marv liked to call his “chutzpah.”
“Ma, I see the monkey!” yelled a girl in a polka-dotted dress and pigtails. “There, there!” A zoo worker came running up and a crowd quickly gathered. But the little girl had only seen a common grey squirrel scurrying around the bushes. The crowd dispersed. Ida, Gus' polar bear companion, swam up to the edge and stuck her head out of the water.
Segment 10
Joe and Marv moved to the rain forest house, keeping an eye out on the way for the lost snow monkey. Around them, kids raced, strollers collided, and walkie-talkies hissed and crackled as zoo employees checked in with each other about the monkey.
“I remember . . . ” began Joe. The two friends began a lot of their sentences this way. “I remember my grandma Bessie had a pet monkey in her house. She was a maid in someone's house and they had this monkey. They got sick of it and gave it to her.”
Segment 11
“A pet monkey in North Carolina?” Marv wanted to know. He was a New York City boy and his most exotic pet had been a white rat called Monty.
“Yes,” reminisced Joe, as they walked into the moist green tropical rain forest enclosure. “That was one naughty monkey. It was some kind of Indian monkey, called a rhesus. He used to get into the bathroom and squeeze the toothpaste out of its tube. Grandma Bessie used to get so mad. She would take that monkey—Sammy, he was called—and smack his bottom, just gently, to show him that squeezing the tube was wrong. Then she would open the window and toss him out into the hickory trees. One day she walked into the bathroom and there was Sammy, standing with the tube in his paw, toothpaste all over the floor. He looked up and saw Grandma. And then, what do you know, that monkey smacked himself on the bottom and jumped out of the window!”
Segment 12
Marv laughed. They both turned their attention to the simulated rain forest all around them. They looked at the keel-billed toucan with its huge colorful beak. There were scarlet ibises and yellow and blue macaws. It was impossible not to get caught up in the excitement of the monkey chase, and all the time they were looking at the other exotic creatures, they were also looking for a tiny pink-faced monkey. A brilliant blue bird flew right by them.
Segment 13
“Fairy bluebird!” exclaimed Marv. “Sita talks about them. She says she would see them when she was young.” Sita was his Indian daughter-in-law, who had grown up on a tea plantation in Assam. She often spoke of the iridescent blue and black fairy bluebirds that used to swoop down and eat the wild figs near her house. She had had a swing hanging from the fig tree.
Segment 14
They walked near the frog exhibit. Brilliant blue, yellow, green, orange and purple frogs hopped around like animated jewels. Joe and Marv both instantly thought about the frogs they had hunted as boys. Joe had chased green tree frogs and bull frogs in the marshes near his home. Marv had grown up listening to the “gick, gick, gick” sound of the cricket frogs in his garden.
“That's a blue poison frog,” a woman next to them explained to her son, who was clutching a skateboard under his arm. “No one messes with them. I read that people have seen snakes bite one, and then quickly spit it out.”
Segment 15
“Pthoooey!” said her son, dramatically spitting in Joe and Marv's direction. They hastily moved back.
“Yes, like that,” said the mother. “Then the snake rubs its mouth against the ground and writhes around in pain. Sometimes the poison gets to it so badly that it goes into a coma!” Her son obligingly threw himself on the ground and rolled around in pretend agony before he rolled his eyes up, stuck his tongue out, and went rigid. His mother laughed, and he got up. She picked up his cap.
Segment 16
“Let's go hunt the monkey!” she said, and put her arm around her son's shoulder as they walked on.
Joe and Marv looked at the giant fake trees and spotted some more creatures. There were turquoise tanagers, saffron finches and violet turacos. There was a cage with a family of colobus monkeys. The two men peered in, checking to see if the snow monkey had somehow come to visit its tropical cousins. A fruit bat whirred in front of Marv's face and he flinched.
A zoo worker with a bullhorn came and stood in front of the alligator cage and announced the missing monkey again. “Remember,” she cautioned, “it is a wild animal, it is NOT a pet.”
Segment 17
Joe and Marv decided to stop at the Leaping Frog cafe for lunch. They usually went for a walk after the zoo, but today there was a boisterous carnival atmosphere because of the missing monkey. The zoo people were worried but many of the visitors didn't really understand that the young monkey was in danger. They were just entertained and interested. They also weren't allowed to leave, so Joe and Marv went inside and ordered some food.
“I'll have a cheeseburger and fries, with a large coke,” said Marv. The woman behind the counter took his order, all the while scanning over his shoulder in case the monkey decided to come in for a bite. Joe ordered a grilled chicken sandwich and glass of lemonade. A child screamed behind them, “I want! My! Milk! Now!” and its mother shushed it. Joe and-Marv talked while they waited for their food.
Segment 18
“Yeah, the shop's doing well,” said Marv. He had retired ten years ago and his son, Marv Junior, was managing the auto body shop now. The two friends often joked about how they actually had done the same thing all their lives: Joe had fixed people's bodies, and Marv had done the same for their cars. Marv stayed in touch with his business because it was still in the family. Joe, however, had sold his practice when he retire
“How's Sue?” he asked Marv.
Segment 19
“She's great!” said Marv. Sue was his oldest daughter. When she was eight years old, Marv had taken her to the boardwalk at Coney Island. She had fallen off one of the amusement park rides and lain screaming on the ground. Out of the crowd had appeared a calm man with a black bag, saying, “Excuse me, I'm a doctor.” It had been Joe.
Segment 20
Now Marv and Joe walked out under the ginkgo trees and looked for a table. They passed a cluster of strollers and Marv glanced down as he maneuvered his way around them. He stopped suddenly and did a double-take. “Hey, Joe!” he said, softly. “Check out this wild and dangerous animal!”
There, curled up and fast asleep in a black checked stroller, clutching a yellow blanket and a blue plastic teething rattle, was a small pink-faced Japanese snow monkey.
This application claims the benefit of the following US Provisional Patent Application, which is incorporated herein in its entirety for all purposes: PS.01 17 60/749979 Dec. 13, 2005 ZEST PROGRESSIONS IN HiFi ASSESSMENTS
Number | Date | Country | |
---|---|---|---|
60749979 | Dec 2005 | US |