Patent Application
20150302436
A computer program listing appendix containing the source code of a computer program that may be used with the present invention is incorporated by reference in its entirety and appended to this application as one (1) original compact disc, and one (1) identical copy thereof, containing a total of three (3) files as follows:
The present invention relates to a method and system for performing market research via interviewing and analysis of the resulting interview data on a communications network, and in particular, for determining customer decision-making factors that can be used to increase customer loyalty and/or market share.
There are at least two important categories of object loyalty definitions (wherein “object” may be a brand, company, organization, product or service). The first category is “operational” object loyalty definitions, wherein such loyalty is defined and measured by analysis of, e.g., customer purchasing behaviors. That is, since one cannot look into a customer's mind, one looks instead into the customer's shopping cart, a parts bin, or an order history. Thus, customer loyalty behavior toward an object is analyzed, according to at least one of the following operational definitions of loyalty: (a) “choosing the object on k of n opportunities or purchase occasions,” (b) “choosing the object k times in a row,” or (c) “choosing the object more often than any other.”
A second category of object loyalty definitions include definitions that provide a description of a “psychological” state of: (a) a predisposition to buy, or (b) a conditional preference, e.g., an attitude, which may be favorable or unfavorable to the object. That is, the definitions of this second category provide descriptions of the mental state(s) of a customer(s) so that one can hypothesize a framework for assessing object loyalty. A customer's attitudes, however, are based in their beliefs, wherein beliefs are descriptive thoughts about things that drive customer choice behavior. Said another way, belief connotes conviction, whereas attitude connotes action.
However, neither of the above definitions of object loyalty are satisfactory for customer loyalty, and at least as importantly, for determining how customer loyalty can be cost effectively increased. For example, for an “operationally” identified loyal customer who buys over and over again, there is no certainty that this customer is actually loyal. Not unless one knows that the purchasing choice was: (a) at least relatively unconstrained, for example, that the customer did not face costs to switch to a competing product, and (b) made in congruence with the customer's preferences. In fact, it may be that the customer is uninformed regarding the market, and/or indifferent to competitive offerings. Moreover, for a “psychologically” identified loyal customer who has a predisposition to perform a transaction with or for an object (as defined hereinabove), there is also no certainty that this customer is actually loyal. In particular, it does not mean that the customer will be more likely to perform such a transaction. To illustrate, an individual may admire a Mercedes, and say it is the best of cars, but cannot afford one. Is he/she loyal? At least from a marketing perspective probably he/she is not.
With belief and behavior comes experience. Experience, over time, creates in customers' minds a set of ideas (i.e., perceptions) about an object. Thus, the term loyalty as used herein may be described as including: (1) favorable customer perceptions built up over time, as evidenced by both belief and behavior, that induce customers to perform transactions (e.g., purchases) of, from or with the object, and (2) such favorable customer perceptions are a barrier for the customers to switch to a competing object (e.g., a competing brand, company, organization, product or service). Evaluation of such object loyalty is desirable for making informed marketing decisions regarding the object, particularly, if such evaluations can be performed cost effectively.
The equity of an object (e.g., a brand, company, organization, product or service), may be described as the aggregate loyalty of the object's customers to continue acquiring or using (e.g., service(s) and/or product(s) from) the object. Equity, then, may be considered a function of: (f1) the “likelihood of repeat purchase,” which is a function of (f2) loyalty, which in turn is a function of (f3) customer satisfaction, which following from the standard satisfaction attitude research framework, is a function of (f4) the belief and importances of attribute descriptors. Said another way,
Equity=f1(likelihood of repeat purchase)=f2(loyalty)=f3(satisfaction)=f4(beliefs,importances)
A company that has built substantial customer equity can do things that other companies cannot. In particular, the greater number of loyal customers, the greater degree of protection from competitive moves and from the vagaries of the marketplace.
Thus, evaluation of such object equity is desirable so that informed marketing and business decisions regarding the object can be made, particularly, if such evaluations can be performed cost effectively.
The primary focus of a marketing manager, when framing a marketing strategy for an object, in order of importance, is: (a) maintaining the object's loyal customer base, and (b) increasing the number of “new loyals.” Increasing sales can be seen as a direct result of these two strategic marketing focuses. For the first “maintenance of loyals” group, two questions arise: (1) why do such loyal customers decide to, e.g., purchase our product instead of the competition's product, and (2) what barriers exist for loyal light users to becoming heavier users. The answer to the first question defines the equity of the business. The answer to the second question gives management insight into how directly to increase sales—by minimizing the barriers for increasing customer loyalty. In particular, the techniques and/or features for attracting non-loyal customers, heavy users and light users, respectively, to become more loyal to an object is the input that a marketing manager needs for developing a strategy that increases sales. Also, attracting loyal customers away from a competitive object represents yet another separate strategic issue. These key inputs, which are grounded in the ability to understand (summarize, quantify and contrast) the customer decision processes of target customer populations, provides the marketer with the insight required to optimally develop effective marketing strategy. Thus, a method and system for cost effectively answering the above two questions (1) and (2) is desirable so that informed marketing and business decisions regarding the object can be made.
Many marketers have made the realization that loyalty is key to a successful business strategy, and they have operationalized the research of loyalty in terms of customer satisfaction. In fact, customer satisfaction research is one of the largest and fastest growing areas of market research. There exist numerous specialty customer satisfaction assessment research organizations, e.g., (i) for universities: Noel-Levitz, Inc.; an example of such assessment research is Lana Low (2000). Are College Students Satisfied? A National Analysis of Changing Expectations. (http://www.noellevitz.com/NR/rdonlyres/DB91046E-59FE-4AB0-AB49-9CAF8EE84D73/0/Report.pdf), Noel-Levitz, Inc. incorporated herein by reference; (ii) for healthcare: Press Ganey Associates Inc. (www.pressganey.com); (iii) for government services: (Opinion Research Corporation incorporated herein by reference) and (iv) for brand satisfaction: Burke Inc. (www.burke.com). These marketing research organizations use methodologies (referred to herein as “attitudinal methodologies”) based upon a traditional attitudinal research framework directed to assessing customer attitudes. That is, they ask questions of customers regarding their beliefs as to what degree a company's product, and competitive products, possess a given set of brand and/or service descriptors (e.g., attributes) and the relative importance of these descriptors to the company's customers (and/or the competitor's customers). The analysis output by such market research, as one skilled in the art will understand, is a set of mean belief ratings for the descriptor attributes, as well as mean importance ratings which can be broken down, if desired, for the various customer segments. Moreover, the analysis output provided by these marketing research organizations provides ongoing customer tracking to assess customer attitude changes over time, so that interpretation of the mean statement customer response scores serves as a basis for strategic decision-making by the object being evaluated. As will be detailed hereinbelow, the approaches and methodologies used by these market research organizations are believed to be sub-optimal for a variety of reasons. However, before describing perceived problems with these prior art market research approaches and methodologies, examples of various marketing challenges are first provided as follows.
The examples of marketing situations outlined above serve to illustrate the fact that the primary function of a market-driven strategy is to maximize the equity of an object, which translates to maximizing customer loyalty, which requires gaining an understanding of what customer (and employee) perceptions are that drive satisfaction. As the above examples of marketing challenges illustrate, maximizing or increasing the equity of an object is desirable for virtually all business enterprises. Thus, it would be desirable to have a method and system for performing market research that determines the relative weights of components or aspects of an object that will maintain and increase customer satisfaction (with respect to, e.g., predetermined target customer groups). These components or aspects, when communicated and delivered by the object, will likely increase the satisfaction level, thereby increasing loyalty, likelihood of repeat purchase, and result in increasing equity (as this term is used herein).
Attitude models (Allport, 1935, Ref. 2. of the “References” section incorporated herein by reference) represent the prototypical, most frequently used research framework utilized in the domain of marketing research. The tripartite social psychological orientations of cognitive (awareness, comprehension, knowledge), affective (evaluation, liking) and conative (action tendency) serve as the research basis of gaining insight into the marketplace by understanding the attitudes of its customers.
Questions regarding any component, or combinations thereof, of the attitude model are regarded as attitude research. Conative, for example, refers to behavioral intention, such as a likelihood to purchase, which is prototypically asked in the following scale format for a specific product/service format (Zigmund, 1982, p. 325, Ref. 11 of the “References” section incorporated herein by reference).
Therefore, if the past purchase or consumption behavior for each individual in the sample of respondents were known from another question in the survey (or consumer diary), the behavioral intention question would be used to compute the likelihood of repeat purchase.
Satisfaction (affect for the consumption and/or use experience) is typically measured using a scale such as the following for a specific product or service (Zigmund, 1982, p. 314-315, Ref. 11 of the “References” section incorporated herein by reference).
Attitude research is based on a theoretical model (Fishbein, 1967, Ref. 8 of the “References” section incorporated herein by reference) containing two components: one, beliefs about the product attributes of the object, and two, an evaluation of the importances of beliefs (descriptors). This theoretical relationship may be represented as:
where, A0 =attitude toward the object
Attitude toward the object (Ao), then, is a theoretical function of a summative score of beliefs (i.e., “bi” descriptors or characteristics) multiplied by their respective importances (“ei”). Assuming this theory to hold, market researchers construct statements to obtain beliefs specific to product and/or services, such as (Peter and Olson, 1993, p. 189, Ref. 16 of the “References” section incorporated herein by reference):
Additionally, market researchers obtain importances using scales that generally appear in the following format (Peter and Olson, 1993, p. 191, Ref. 16 of the “References” section incorporated herein by reference):
For the three standard types of attitude scales noted above, the researcher assigns numbers (integers) to the response categories. In the cases of the behavioral intention scales and satisfaction (affect), successive integers are used such as (+2 to −2, and +3 to −3, respectively). Analysis of the data then involves computing summary statistics for each item, for the customer groups of interest.
In sum, from the perspective of marketing research, customer understanding is derived from studying the tables of summary statistics indicative of customer responses related to a combination of product and/or service customer beliefs (cognitive), corresponding customer importances (affective) with regard to key attribute descriptors, and the likelihood of acting (conative).
Difficulties with the above attitude research methodology for measurement of attitudes include individual differences in interpretation of questions, which result in a compounding of error of measurement. Detailed below are the assumptions that underlie the use of attitude models, along with examples of how error is introduced into the resulting measures.
The above problematic assumptions have been individually discussed in virtually all psychology and marketing research textbooks. However, in reality, these issues have never been adequately addressed, especially in light of the compounding effect caused by multiple violations of the assumptions. Understanding the potential confounding effect of the above five assumption violations can be even more problematic to obtaining valid measures when the following not-previously-identified further assumption of attitude research models is also considered.
That is, in attitude research models importances are assumed to be distributed equally across belief scales. Such an assumption is denoted herein as the “uniform importances assumption”.
For example, if a person has a given belief level or position on an attitude scale, e.g., an attitude of “not satisfied,” what is assumed important to him/her is both: (a) some weighted composite of the importance scores across all the attribute dimensions, and (b) that these importances are somehow independent of his/her belief level. That is to say, if one asks how to increase a respondent's attitude score/satisfaction level one A (i.e., one scale point), the assumption that has heretofore been made is that a weighted composite of attribute scores would be needed, and regardless of the level (higher or lower) on the attitude scale, the same weighted composite is used by the person.
Asking three questions can test this uniform importances assumption. First, an “anchor” question for establishing a position on the attitude scale of interest is presented to a respondent. In the example anchor question [1] immediately below, a “satisfaction” question is presented to the respondent. Following this anchor question hereinbelow are second and third questions which simply ask for the key attribute or reason that is the basis for the person's rating in question [1].
If the assumption holds that importances are equally distributed across the scale points of the attitude scale, the most likely outcomes would be that the most important attribute would be mentioned for both questions [2] and [3] above, or alternatively, that the first and second most important attributes would be mentioned in the response for questions [2] and [3].
The conclusions from customer research using the above questioning format do not confirm the uniform importances assumption. In fact, importances are not equally distributed across such an attitude response scale. To empirically test this assumption, the above three questions [1], [2], and [3] were asked of independent samples of respondents (total of 750) across the five product/service categories mentioned previously, from durable goods to nonprofits. Analysis of the responses to questions [2] and [3] revealed that: (a) in less than two percent (2%) of the cases was the same attribute mentioned for both questions [2] and [3], and (b) the first and second most important attributes (determined by a traditional market research importance scale) combined were mentioned less than 50% of the time.
Thus, if the market research question(s) is how best to improve respondents' attitudes (e.g., satisfaction level underlying loyalty), then the prior art attitude research methodologies are believed to be flawed due to the implicit acceptance of the uniform importances assumption as well as the acceptance of the other five erroneous assumptions recited hereinabove.
Yet another newly discovered assumption of the attitude research methodologies that is also suspect in attitude is as follows:
This assumption is held by all traditional attitude models and has been empirically demonstrated to be false. Research has shown (Reynolds, 1985, Ref. 18 of the “References” section incorporated herein by reference; Reynolds, 1988, Ref. 19 of the “References” section incorporated herein by reference; Jolly, Reynolds and Slocum, 1988, Ref. 14 of the “References” section incorporated herein by reference) that higher levels of abstraction beyond attributes (e.g., consequences and personal values) contribute more to understanding preferences and performance ratings than do lower-level descriptor attributes. Therefore, to gain a more accurate knowledge of the basis of customer decision-making, one must understand the underlying, personally relevant reasons beyond the descriptor attributes provided by respondents.
Accordingly, it is desirable to have a market research method and system that provides accurate assessments of, e.g., customer loyalty, and accurate assessments of the attributes of an object that will influence customers most if changed. In particular, it is desirable that such a market research method and system is not dependent upon the above identified flawed assumptions.
The invention disclosed hereinbelow addresses the above identified shortcomings of prior art market research methods and systems, and in particular, the invention as disclosed hereinbelow provides a market research method and system that provides the desirable features and aspects recited hereinabove.
The following references are fully incorporated herein by reference as additional information related to the prior art and/or background information related to the present disclosure.
Cognitive Abstraction Utilized by Consumers in Product Differentiation. In J. Eighmey (Ed.), Attitude Research Under the Sun. Chicago, Ill.: American Marketing Association, 128-150.
Framework. In J. Jacoby and J. Olson (Eds.), Perceived Quality of Products, Services, and Stores. Lexington, Mass.: Lexington Books.
Between the MECCAS Model and Advertising Affect. In A. Tybout and P. Cafferata (Eds.), Advertising and Consumer Psychology (Vol. IV). Lexington, Mass.: Lexington Books.
The “means-end approach” (as also described in Ref. 9 of the References Section hereinabove) has at its foundation the notion that decision makers choose courses of action (purchase behavior) that will achieve their desired outcomes or end-states (Gutman 1982, Ref. 9 above). Means-end research methods focus on deriving chains that represent an association network of meaning, from attributes to consequences to personal values. Values are generally defined as the important beliefs people hold about themselves and their feelings regarding others' beliefs about them. According to means-end theory, values (V) provide the overall direction and give meaning to desired consequences (C). A desirable consequence (i.e., that satisfies a higher order value) determines what attributes (A) of the choice option are salient, which define the competitive behavioral options. By uncovering the important network of meanings for a category in this way, a market researcher is provided with an in-depth understanding of how customers perceive an object and/or its marketplace.
A ladder is a chain that is obtained from the laddering methodology described hereinabove. In particular, a ladder as used herein will generally have four or more levels for describing a group's perceptions of an object, wherein the levels extend from a lowest level reciting substantially objective facts that are perceived as important (enough to recite) by the group, to a highest level of persistent and personal values or beliefs.
A market research analysis method and apparatus (collectively, also referred to as a market analysis system, and StrEAM™ herein) is disclosed for performing market research and developing marketing strategies, wherein at least the following features are disclosed:
The market analysis system includes a method and apparatus for obtaining and evaluating interview information regarding a particular topic, (e.g., object) thereby determining significant factors that, if changed, are more likely to persuade the interviewees (and others with similar perceptions) to change their opinions or perceptions of the topic. The market analysis system includes four subsystems. A first such subsystem is an interactive interview subsystem (also identified by the product name StrEAMInterview and StrEAM*Interview herein) which includes a set of computer-based tools used to conduct rigorous interviews and capture results therefrom about topics and/or objects related to areas such as consumer market research for a particular product or service, voter analysis, opinion polls, et cetera. The second subsystem is an interview data analysis subsystem (also referred to as StrEAMAnalysis and StrEAM*Analysis herein) that includes an integrated set of software components for analyzing interview data obtained from, e.g., the interactive interview subsystem. The interview data analysis subsystem includes interactive software tools that allow a market research analyst to: (a) categorize the interview data in terms of meaningful categories of responses, such as Means-End chains as described in the Means-End Theory description of the Definitions and Descriptions of Terms above. Note that in at least some embodiments, such chains are of at least four in length; however, longer chains are within the scope of embodiments of the market analysis system disclosed herein). The third subsystem is an administrative subsystem (also referred to herein as StrEAMAdministration and StrEAM*Administration herein) which includes market research project planning, interview scheduling, and tracking the status of market research projects. The fourth subsystem is a user support subsystem (also referred to herein as StrEAMRobot and StrEAM*Robot) which includes functionality for automating the market research method and system disclosed herein such that an interviewer for conducting interviews is substantially (if not entirely) eliminated.
The interactive interview subsystem StrEAM*Interview is, in some embodiments, network-based such that the interviews can be conducted remotely via a telecommunications network (e.g., the Internet) in an interviewee convenient setting. The interactive interview subsystem provides automated assistance to an interviewer when conducting an interview. For example, not only are interview presentations (e.g., interview questions) provided to the interviewer, but also information for interpreting and/or classifying responses by an interviewee is provided substantially while such responses are being obtained by the interviewer. In particular, the interactive interview subsystem may assist in obtaining various hierarchical views of each interviewee's reason for having a particular opinion or perception of an interview topic/object. The interview presentations presented to each interviewee (also referred to as a “respondent” herein) are designed to elicit interviewee responses that allow one or more models to be developed of the interviewee's perceptual/decision framework as it relates to the object or topic that is the subject of the interview. In particular, open-ended questions may be presented to the interviewee, thereby allowing the interviewee greater flexibility of expression in providing insight into his/her perceptions of the object.
The interactive interview subsystem includes built-in quality control features which focus the interview on obtaining both complete and detailed levels of information about interviewee perceptions, and in particular, quality control features for substantially ensuring that all levels of individual ladders and/or chains (as these term are described in the “laddering” and Means-End Theory descriptions of the Definitions and Descriptions of Terms sections hereinabove) are addressed in the interviews. Accordingly, the result of these quality control features, when used in conducting an interview via a communications network (e.g., Internet), is significantly higher quality interview data as compared to traditional face to face market research interviews.
It is also an aspect of the interactive interview subsystem disclosed herein that it can be administered and analyzed, via Internet communications, wherein such communications may include: (a) real-time interactions with a trained interviewer, and/or (b) a substantially automated interviewing process (or portions thereof) wherein the process is conducted totally or substantially by networked computational devices. In particular, an embodiment of the market analysis system may be provided wherein an interviewer is substantially only required to communicate with an interviewee when interviewee responses are detected that indicate the interviewee is confused, and/or the interviewee's responses are inappropriate.
The market analysis system (i.e., StrEAM) disclosed herein may be applied to a wide range of research topics of interest beyond determining, e.g., key object factors and/or key perceptions of an object. For instance, embodiments of the market analysis system may be used to analyze or identify perceptions and/or beliefs of: employees, product distributors, investors (or potential investors), voters, competitors, and even parties that are generally considered to be disinterested. In fact, the market analysis system disclosed herein can be used to assess and/or identify beliefs, behavior, attitude, voting intent, and/or loyalty in substantially any area where human decision making is heavily dependent upon beliefs, behavior, attitude, and/or loyalty. For example, the market analysis system may be used to assess and/or identify beliefs, behavior, attitude, and/or loyalty of customers and/or employees of such diverse organizations as political parties or candidates, cosmetics companies, automobile manufacturers, direct selling companies, service stations, insurance agencies, automobile dealerships, and electrical and industrial distributors.
The market analysis system disclosed herein may also provide better direction in determining advertising for an object. In particular, the market analysis system can be used to derive or identify advertising that is more effective (and cost-effective) than heretofore has been possible. For example, the market analysis system can be used to develop or identify adverting having messages to which a particular targeted population is positively disposed. Additionally, the market analysis system disclosed herein can be beneficial in identifying public relation messages that can be used for: (i) retaining and/or hiring employees with desired attitudes or perceptions, (ii) retaining or attracting distributors, and (iii) retaining or attracting investors. In particular, such public relation messages may be directed to insights resulting from the use of various embodiments of the market analysis system disclosed herein.
The market analysis system disclosed herein is effective for the assessment of customer loyalty and satisfaction for an object whose market is being evaluated. The market analysis system includes techniques or methods for performing such assessments, and also includes various computational components for embodying the methods, and in particular, providing such components for performing customer loyalty and satisfaction assessments using the Internet and/or another ubiquitous communications network.
The market analysis system may determine the substantially loyal customer groups for an object being marketed, and contrast these loyal customer groups with less loyal customer groups. In particular, the market analysis system disclosed herein facilitates understanding what drives decision making in a customer population (i.e., the aggregate population of both customers and/or possible customers) when it comes to purchasing a particular object being marketed to members of the population. Typically, object purchase price sensitivity by the population and the beliefs of such population members about the object (e.g., quality, reliability, etc.) are the important factors for such decision making. For the present disclosure, price sensitivity and customer beliefs can be described as follows:
One aspect of the market analysis system is that for a given market research issue/problem, a joint distribution (as in
The present market analysis system also provides a framework for a detailed Laddering interview process, wherein there are four (4) levels to the laddering interview process. That is, the level (or rung) denoted “consequences”, in the means-end theory description in the Definitions and Descriptions of Term section above, is divided into two distinct categories of “Functional Consequences” and “Psychosocial Consequences” (as also described in the Definitions Description of Terms section). Symbolically this enhanced ladder or chain can be represented as follows:
It is a further aspect of the market analysis system disclosed herein to enhance the laddering interview technique with an additional interviewing methodology. In some laddering interviews, beginning at the object attribute level and moving up the “levels of abstraction” to personal values, such interviews may not appropriately capture a respondent's decision making structure related to the object. For example, for market assessments of objects, such as cars, wherein many prospective customers are interested in the image projected by driving or owning certain car models, an additional/alternative interview methodology may be used, known as “chutes” herein. In the chutes interview methodology, one or more questions (i.e., Egosodic Valenced Decision questions as described in the Definitions and Descriptions of Terms section above) are directed to the interviewee for thereby obtaining a “top of mind” (TOM) response(s) related to the object being researched (or competitive object(s)). Once such a TOM response(s) is obtained, additional questions are posed to the interviewee, wherein these additional questions are intended to obtain interviewee responses that identify what features of the object (or competitive object(s)) that typically serve as the primary determinants of object choice, or choice of a competing object. By initializing the laddering process through Egosodic Valenced Decision Structure (EVDS) questions, the interviewee's general decision making process can be determined. Then, by going “down” the rungs of the laddering interview process for the object (e.g., product/service), more specific features of the object are identified that the interviewee associates with the TOM response. Additionally, by going “up” the rungs of the laddering interview process, a complete ladder of the interviewee's perception of the object can be developed. These decision networks can be developed individually for common TOM descriptors, yielding specific CDMs (i.e., customer decision map, see the Definitions and Descriptions of Terms section hereinabove), which represent decision segments.
Other benefits and features of the present invention will become apparent from the accompanying drawings and the description hereinbelow.
The present disclosure is substantially based on a market research theory termed means-end theory (as described, e.g., in the following references incorporated herein by reference: Howard, 1977, Ref. 12 of the “References” section hereinabove; Gutman and Reynolds, 1978, Ref. 10 of the “References” section hereinabove; Gutman, 1982, Ref. 9 of the “References” section hereinabove). Means-end theory hypothesizes that end-states or goal-states (defined as personal values) serve as the basis for the relative importance of attributes, e.g., of a product or service. For instance, attributes of a product or service are hypothesized to derive their importance by satisfying a higher-level consumer need or goal. Said another way, such attributes have no intrinsic value other than providing the basis for a consumer to achieve a higher-level need or goal. For example, “miles per gallon” is an attribute of automobiles, but the importance of this attribute to a particular consumer may derive from a higher-level consumer need or goal of “saving money” which, in turn, may be personally relevant to the consumer because it enables the consumer to “have money to purchase other consumer items” or perhaps “invest money.” That is, a hierarchy of progressively more personally important goals and needs (and ultimately personal values) can be identified for the consumer, wherein such a hierarchy (also referred to as a “decision-making hierarchy” herein) can be used by an embodiment of the market analysis system for modifying consumer's perception of a particular object, or the perception of other consumer's having a similar decision-making hierarchy.
Accordingly, means-end theory postulates that it is the strength of a person's desire to satisfy these higher-level goals or needs (and ultimately values) that determines the relative importance of product/service attributes (more generally, object attributes). Thus, identification of such higher-level goals or needs can translate directly into understanding the basis of customer decision-making (a more detailed discussion of means-end theory can be also found in Reynolds and Olson, 2001, Ref. 36 in the “References” section hereinabove.
One methodology used to uncover such means-end higher-level goal or value hierarchies is termed laddering as described in the Definitions and Descriptions Terms section hereinabove. The laddering methodology models both the structure and content of a person's mental associative network of cognitive meanings, and thus, models a basis of decision-making. The present market analysis system provides an effective way to identify such personal hierarchies, e.g., interviews of a target customer population are conducted for: (a) obtaining, for those interviewed, the most important (object preference discriminating) attribute(s) that underlie object selection, and then (b) laddering such attributes to higher levels of personal importance by asking alternative forms of a question such as: “Why is that important to you?”. Thus, in performing [the] steps (a) and (b) immediately above for each interviewee (also denoted “respondent” herein), the interviewee's personal cognitive decision-making structure can be modeled by the market analysis system disclosed herein. In particular, a four-level goal/value hierarchy, as shown in
An illustrative embodiment of the laddering process is represented in
Additionally, the market analysis method and system disclosed herein is useful for understanding the motives of why a consumer purchases (or does not purchase) a particular product or service. In particular, the market analysis system may be used for identifying consumer perceptions related to price versus quality tradeoffs for a given object. For example, as shown in
In
Before describing the computational and network features of the market analysis system, a description of the methodologies used by this system, as well as a number of market research examples, will be provided, wherein the methodologies and examples are illustrative of the use of the market analysis system. In particular, these methodologies and steps are illustrated in various market research study examples hereinbelow. Note that each of the market research study example hereinbelow may in performed by the market analysis system embodiment as shown in
At a high level, the market research method (more generally, “perception” research method) upon which the market analysis system disclosed herein is based performs at least the first four of the five steps of
An important aspect provided herein is that the answers to only four market issue/problem “framing questions” in step 1000 provide substantially all the marketing information needed to develop a sufficiently clear understanding of the market issues to be investigated so that appropriate market research interview questions can be constructed. Accordingly, it is an important aspect disclosed herein that only answers to the four framing questions are required to address a marketing issue/problem, if the issue/problem is framed in terms of the customer decision-making that underlies satisfaction, and ultimately, loyalty. In one embodiment, these framing questions are:
Once a concise statement of the issue/problem to be researched is generated from the answers to such framing questions, interview questions then can be generated in step 1004. That is, research (i.e., interview) questions are developed according to the framing of the research problem/issue. Note, it is an aspect of the present market research method that the interview questions developed include substantially different questions from the types of questions asked in most prior art market research systems and methodologies. In particular, various “equity” questions may be constructed that are intended to elicit interviewee responses in order to identify aspects of the particular object and/or the problem/issue that could change interviewee perception of the object (positively and/or negatively). Additionally or alternatively, various “laddering” questions may be constructed for obtaining means-end chains of interviewee perceptions related to the object and/or the problem/issue, wherein collections of such chains or ladders can provide insight into the perceptual framework of the group.
It is an important aspect of the market research method disclosed herein that a substantially reduced number of interview questions are generated for presentation to members of the target group, in comparison to the number of questions likely required if a standard attitudinal market research survey were conducted, wherein 50 to 100 or more questions are likely to be generated in order to assess the beliefs and importances of a predetermined set of attribute descriptors. In particular, the market research method (and corresponding market analysis system) disclosed herein may present approximately 15 to 30 questions to interviewees including at least some of the following questions (or their equivalents):
Accordingly, in step 1004, interview questions are constructed that are intended to elicit from each interviewee at least some (if not most of the following):
Once the interview structure and content is determined (including constructing the questions of step 1004), in step 1008, interviews are conducted with individuals of the target customer population, wherein responses to the questions developed in step 1004 are obtained. In one embodiment, +equity and −equity questions are asked the interviewees. Alternatively/additionally, laddering questions may be asked interviewees. Then, in step 1012, the question responses are analyzed according to the novel techniques and methodologies described hereinbelow (and in
Finally, in step 1016, strategic decisions can be made by those responsible for proposing how to address the problem/issue.
Thus, by determining the values of a target population group, marketing and/or advertising presentations may be developed that take existing features of the object and present them in a way that emphasizes their positive relationship to the values of this target group. Optionally, the perceptual framework of the target population group also may be used to determine how to most cost effectively enhance or modify the object (or alleviate the problem/issue) so that it appeals more to the target population group (i.e., is more consistent with the decision chains of the target population group).
The six market research examples hereinbelow illustrate (a) how to develop interview questions for use in interviewing members of a target population group according to various embodiments of the market research system disclosed herein, and (b) how to analyze the interview responses therefrom according to the steps of the flowchart from FIGS. 10 and 11A-B. Note that the following first two examples (i.e., a resort market research example, and a museum market research example) illustrate how the market research method and system of the market research method (and corresponding market analysis system) can be used to determine the key underlying decision elements within the decision structures of customers/clients that have the highest potential to increase customer/client satisfaction and thereby increase loyalty.
A marketing manager at a golf and country club in an exclusive mountain resort area with little or no competition is confronted with the situation that several new private and semi-private golf courses are in the planning stages, with two already under construction. The manager is worried about the competitive forces in the relatively small marketplace, that will be created by the new competitions' price points, both below and above his current pricing level (recall the initial market evolution diagram in
Problem framing.
The manager first defines the business problem in terms of answering the four framing questions:
From the answers to the above framing questions, the market research problem is stated as follows:
Note: The phrases in italics within the above problem statement are taken directly from the answers to the framing questions above.
The specificity of the problem statement above provides the manager (and/or an interview question designer) with the needed subject matter and focus to generate interview questions accordingly to the present invention. In particular, the following questions are representative of interview questions according to the present invention.
By asking questions such as the five listed above, the members provide direct insight into what specifically is important to them for increasing their level of satisfaction, which is the essence of the management question. The member's answers, when summarized, reflect the most leverageable aspects of the club, in terms of increasing the overall member satisfaction level.
Once a statistically significant number of club member responses are obtained, the following data analysis steps are performed.
Thus, review of the output from ELA permits management to see how well each functional area is perceived by the respondents via the Equity Attitude values, and to focus upon the key tactical and strategic issues that will raise the average level of satisfaction, for example, one level (ΔL).
If importances were asked directly, general member activities would appear as the highest scoring reason, with golf being second (
It is important to note that although the present example refers to the subcategories that make up each content code as a “functional area”, it is within the scope of the present disclosure that such subcategories can be determined by other criteria than function. For instance, the description and steps for performing the ELA are equally well suited to identifying the characteristics of a political candidate that if changed would yield the most favorable response from voters.
The management problem is determining which areas to focus upon in order to create more loyalty with the membership, thereby minimizing the likelihood of switching. Based upon the Leverage Analysis (e.g.,
A marketing manager for a national museum is concerned about reduced member participation over the last year (−15%) in sponsored events and exhibitions. The manager knows how vital membership “donations” are to the museum. In fact, such donations account for 50% of the gross operating budget with the remaining monies coming primarily from admission fees. As member participation falls, the manager fears donations will also fall, resulting in severe financial problems. A related concern of the manager is: What is the most effective manner in which to communicate with the membership?
Problem Framing.
The Management Problem is Stated as Follows:
Research Questions.
Summary Charts.
The management question, then, is why people are reducing their participation, and, secondarily, what can be done to better satisfy the members, thereby increasing their participation level. The summary results provided by the present invention are presented in
Review of the ELA results suggests focusing on two key areas to improve member satisfaction: Tutorials, which is consistent with the earlier reasons for joining (Programs and education), and improving the art works in the Collection.
Management Direction.
By framing the business problem in terms of satisfaction with key customer groups (defined by Loyalty and Usage), a research framework methodology disclosed herein identifies the most leverageable equities and disequities. Computation of the Leverage index provides a direct measure of the areas of potential changes that will have the largest effect on improving satisfaction with customer segments. Accordingly, the Leverage index may be used to take cost effective steps for increasing positive perceptions in a target population of the object (e.g., museum) whose market is being analyzed, i.e., increasing the object's “strategic equity” as described in the Definitions and Description of Terms section hereinabove.
The following healthcare example illustrates how the market research method and system of the present disclosure can be used to provide a research methodology that permits computation of statistical summary indices, which can be used to track the changes in satisfaction by sub-units within a business organization over time.
A hospital administrator for a healthcare provider in a relatively geographically isolated city, with few competitors, has noticed a decrease over time in the number of patients served, in particular, those undergoing surgical procedures. From her interactions with competitive healthcare administrators in the area, it is her understanding that the number of patients and procedures at the competitive hospitals is increasing. She wants to design a “satisfaction barometer” that:
Problem Framing.
The administrator organizes a meeting with her staff, with the goal of defining the business problem, and they answer the four general framing questions.
The Management Problem is Stated as Follows:
Various options can be considered as to the timing of the research administration of the “satisfaction barometer.” At “time of check out” was considered to be the most appropriate due to its immediacy with regard to the hospital stay experience. This decision necessarily requires the questions to be administered to be few in number and easily presented by the current hospital staff.
After constructing a data file merging in relevant patient background information:
T
s=[((n+)−(n−))+½*(n0)]/N
where, “n+” is the number of Above Average (7-9) ratings, “n0” is the number of middle or Average ratings (4-6), and “n-” is the number of Below Average ratings, and N represents the number of total ratings.
Summary Charts.
The ANCHOR scale numbers serve as the basis to elicit specific reasons as to the positive and negative equities with regard to satisfaction. The first step of the analysis determines the major code categories (see Definitions and Descriptions of Terms section hereinabove). In the hospital example scenario, the categories are Nurses (attending), Staff (departments), Personal (pain, stress), MD's, and Facilities (environment). The (I) importances for each category of customer responses are computed. The nine-point satisfaction scale is recoded into three classifications: [−] (1-4), [0] (5-6) and [+] (7-9).
Review of the equity data reflects significant differences in what is important by level of satisfaction. For example, the [−] satisfaction group focuses on Personal (pain, stress) as the dominant negative that, if addressed, would increase their level of satisfaction. At the upper end of the satisfaction, i.e., the [+] satisfaction group, one of the most significant barriers to satisfaction is MD's. The difference in importance by level of satisfaction detailed here corresponds to the violation of Assumption 6: Importances are assumed to be independent of beliefs (the “Attitudinal Research Framework Descriptions” section hereinabove). Therefore, without the methodology of the present invention, one would not be able to identify what the equities are that should be focused upon.
Management Tool.
Review of the response data for Nurses provides a framework for management to prepare nurse training material. Moreover, by detailing the qualitative input that comprises each code, specific areas of focus that translate into customer satisfaction can be highlighted for use in periodic performance-improvement meetings. This, of course, should be done within each organizational unit.
Beyond the qualitative input and the summary statistics of importances and beliefs, a single measure, Ts, for each major code can be computed for purposes of tracking “satisfaction performance” over time. Note that this measure is relative, in the sense that the sum of all Ts's across the operational units will be about zero. As the dynamics of the service component of the hospital improve (leading to increased customer satisfaction), the relative importance of the sub-codes will change, providing a management framework for focusing on constant improvement.
The following two examples illustrate how the market research method and system of the present invention can be used to identify the differentiating decision “equity” elements within and between Loyal×Usage customer population segments that have the most potential to drive loyalty and/or increase consumption.
A preeminent direct selling company of cosmetics, which has experienced steady sales growth for 20 years, sees its sales significantly decline over the course of a year, greatly reducing its market value (reduction in stock price of 80%). Because the perception of growth and financial opportunity is critical to maintaining and recruiting new sales associates, the decline in stock price causes the company to begin the “death spiral” to financial ruin. The fact that direct selling organizations commonly experience turnover of 100% of their sales force per year exacerbates this problem. Market research reports that the beauty products and their packaging sold by the company have an older, out of date look that is not appealing to either their existing, as well as potential, end-users. Management must make a decision immediately as to which strategic issues to address, before the company loses critical mass necessary to fund the overhead cost of operations and its debt load, and cannot continue to operate.
Problem Framing.
The research problem, then, can be defined as delineating the common decision structures underlying the three decisions central to the direct selling business, and determining the equities and disequities associated with this specific type of direct sales experience. The development of an optimal strategy, with regard to recruiting and retention, involves leveraging equities and supplanting disequities.
Research Questions.
To illustrate the equity analysis framework, consider
Still referring to
Summary Charts.
The contrast is between those who stayed with the company (loyal), and those who had just left (non-loyal). The reasons that people joined the direct selling company were, in fact, the reasons that the company talked about in its current communications: Make money, Contribute to household, Be your own boss, and Work your own hours.
It is now possible to see that, if the goal is not only to get people to join, but also to stay with the company, one cannot put emphasis solely on the message “you can make lots of money.” A more powerful pathway makes use of the teaching and learning component of the direct selling experience, explicitly highlighting the opportunity for personal growth and development that many loyal sales associates have found appealing over time. People work for money, and that is a given. What is the “value-added”, in the present example, is the personal growth component offered by this direct selling company.
The StrEAM™ methods that led to this strategic insight, in this example scenario, were twofold. First, framing the marketing problem in terms of understanding human decision-making with regard to specific customer groups provided a research framework to focus precisely on the key issue at hand. Second, by using the classification of
Management Decisions.
Summary.
The direct selling example scenario illustrates the value of being able to contrast decision structures of different segments within a customer population in order to develop a marketing strategy (in this case, recruiting) using a computed Equity/Disequity grid based on the decision structures presented in the CDM.
The management of an American automobile nameplate (i.e., manufacturer) is very troubled by their declining sales figures. Increased advertising expenditures and promotional events are not driving sales. Management concludes their positioning strategy is not effective. Market research using, e.g., the present invention, to determine joint distribution of price sensitivity and conditional beliefs framework (e.g., as shown in
Problem Framing.
The Management Problem is Stated as Follows:
Research Questions.
Once the attribute descriptor (i.e., “oversize instrument gauges”) is obtained, the data for the entire means-end chain is linked together in the next set of questions that move toward related personal values of the respondent. That is, continuing with this example, the respondent could be asked, “Considering that ‘oversize instrument gauges’ are important because they help define your idea of ‘superior interior design’ and that translates to ‘cool image,’ why is this important to you?” Moving up the ladder in this way, using laddering probes, could yield responses such as “impress others” and then “enhanced social status” as indicated in
Summary Charts.
For example, the COOL IMAGE orientation discussed earlier is a function of three possible decision pathways, namely: convertible, interior and exterior styling, each representing a segment. The common higher-level reason COOL IMAGE is important for customers is because of their perception of the automobile's ability to “impress others,” which leads to “social status.”
The management question, then, is “what do current customers believe that potential customers do not?” The research to answer this question, as noted, involves contrasting buyer segments to determine their respective equities and disequities. To illustrate,
The representation produced, using the TOM-derived segments as the basis, provides significant advantages over standard multi-dimensional representation methods. Standard analytical procedures place the characteristics in the space, assuming no connection or structural relationships between them (independence). The methods presented here have two significant advantages. First, by virtue of the sampling frame, key equity contrasts can be made, which leads directly into the strategy development process. Second, the a priori knowledge of the underlying decision structures allows for a more comprehensive interpretation, by providing a clustering or grouping basis for connecting the defining decision elements.
Management Interpretation.
The dominant reason “First time buyers” decide to choose the nameplate of interest is because it HOLDS VALUE. As can be seen in
The challenge management faces is how to position their nameplate so as to appeal to this modern, style-driven decision segment. Two options emerge. One, change the design features. This is obviously too costly, takes many years to implement, and therefore is not practical in the short term. Second, change the perceptions of the target customer population regarding the social status that can be gained from being secure in one's (investment) decision to buy the nameplate of interest. The decision orientation to be developed is:
The underlying premise of this redefining of social status is that social status drives the importance of the lower-level elements in the current COOL IMAGE decision orientation. And, if it can be communicated to the potential customers that there is another facet of status, one that is defined in terms of recognizing the value of a good investment, there is an increased likelihood of purchase by this target segment.
It is worth noting that different population groups have different equities and disequities. Thus, such population groups can be prioritized according to which are believed most important for developing a marketing strategy. Additionally/alternatively, a marketing strategy may be developed based on common or shared equities and/or disequities between different potential customer population groups. So, if it were determined that single men age 50 to 55 who “Considered, but rejected” the automobile nameplate also did so because of a desire for a more COOL IMAGE, then a common marketing strategy that appeals to both the above-identified COOL IMAGE first time buyers and single men age 50 to 55 may be determined.
Summary.
The Equity/Disequity grid methodology, for identifying which decision data provide the most potential leverage to be incorporated into a positioning strategy, is detailed.
A second methodology, which avoids some of the limitations of laddering, is developed. Traditional laddering, beginning at the attribute level and moving up the “levels of abstraction” to personal values, does not necessarily capture the decision constructs that typically serve as the centerpiece of choice for more high-image categories. By initializing the laddering process through Egosodic Valenced Decision Structure (EVDS) questions, the general decision construct can be obtained. Then, by going “down” to what features of the product/service are used to define the presence of the construct (“chutes”) and then going back up to values, a complete ladder can be developed. These decision networks can be developed individually for common TOM descriptors yielding specific CDMs, which represent decision segments.
The application of the StrEAM Equity Grid™ methodology “contrasts” to relevant customer groups provides the ability to identify differentiating decision “equity” elements that have the most potential to drive purchase for these high-image categories. Management prioritization of these contrasts leads to the development of optimal strategy.
The general management problem common to substantially all businesses is to develop a market research tracking framework: (i) to identify features of the business' marketplace that affect the business' competitiveness therein, (ii) to provide for ongoing measurements on a periodic basis of such features, (iii) to identify, and quantify the changes in the marketplace, and (iv) adjust their marketing activities to cost effectively address the features.
Examples of the types of strategic questions that a tracking framework must address are:
A central feature of a market research tracking process is the identification of the key differentiating and leverageable decision elements (e.g., Equity/Disequity grid methods) that define the “equity” segments by the StrEAM™ joint distribution of price sensitivity and conditional beliefs classifications (e.g., as in
There are four primary tasks of a process for tracking strategic equity of a particular object: (1) identify the competitors for the particular object, (2) identify the drivers of consumer choice and/or consumption (including corporate image) related to the object, (3) evaluate current marketing activities related to the object, and (4) identify marketing trends and their underlying causes related to the object.
Each of the above-identified four tasks for tracking strategic equity is further described hereinbelow.
(1.2.1) Identify the competition.
Competitive alternatives to using or preferring a particular object may not be readily apparent without investigation into possible competitors at a high level of abstraction. For example, to identify competitors to a particular object, an investigation into what consumers or customers view as such competitive alternatives may identify alternatives that are beyond the category of competitors that, e.g., provide a competing product or service that is substantially similar to the particular object. That is, such an investigation must at least initially broaden to encompass higher level categories (i.e., meta-categories).
Choice is context-dependent, so the meta-category definition depends on context. This means the choice context drives who is defined as the competition, in particular for frequently purchased consumer goods. For example, in 11 of 12 countries in the Eastern hemisphere, the number one competitor for a certain carbonated soft drink brand is not a carbonated soft drink—but “CSDs (Carbonated Soft Drinks)” are only what the company tracks that distributes the carbonated soft drink brand. Brand usage information, correspondingly, is gathered by consumption occasion where relevant, along with demographic information. Brand share should first be thought of in a consumption occasion context. Of course, for consumer durables this distinction is not nearly as relevant. However, for most consumer goods the concept of occasion-specific decision-making is critical to understanding the equities in the marketplace.
The central point here is that one gains a complete strategic picture only by examining the buyer beliefs that drive choice (brand usage) in different contexts, where context can be defined by the behavior of interest (purchase or consumption, for example), or by time of day, location, or significant others present. Not taking into account context differences, and not grounding the respondent in this way, results in ambiguity and error in terms of each individual respondent's interpretation of the research questions. People do not behave or think in general terms; they seek satisfaction, think and behave in specific situations. And these situations determine the decision structures that will be utilized by the consumer.
From the product usage information obtained, brand loyalty for all brands can be computed in various ways, which serves as a primary classification for the StrEAM Equity Grid™ contrasting analysis (depending on the management issue).
(1.2.2) Identify the Elements that Drive Consumer Choice.
Which strategic elements drive choice? Again, in the case of frequently purchased consumer goods, one needs to measure the relevance of the strategic decision elements (product features or attributes, consequences of consumption, and psycho-social imagery) in each context to determine which specific strategic elements are the key drivers of equity for each consumption occasion. Again, consumption contexts need to be analyzed individually to identify the decision structures that drive the equities and disequities of the respective competition.
In addition, elements of corporate image, defined as leadership traits, should also be measured. Note that image research (Reynolds, Westberg and Olson, 1997, Ref. 33 of the “References” section) indicates that characteristics comprising the concept of a “leader” parallel the psycho-social consequences for consumer brands, which also holds for political candidates. These key leadership traits, and their respective definitions, that define corporate (and political) image are: Trustworthy: Honest and worthy of trust; Effective: Capable, Gets things done; Popular: Number one; Lots of people like it; Traditional: Has strong heritage and tradition; Caring: Cares and concerned about people; Efficient: Uses resources wisely; and Innovative: Comes up with creative new ideas. The measurement of corporate image is important because many marketing activities, as detailed below, are intended to drive corporate image. Therefore, the ability to measure their effect on these key dimensions must be provided. Note that corporations can be considered leaders in society because they fit key leadership-role criteria: They can exert influence in order to affect the performance of society. Because one needs to measure the linking of elements of strategic equity with marketing mix elements, one must also be sure to examine the relationship between the kind and degree of sponsorship participation and the strategic elements, particularly those that comprise the leadership/corporate image dimensions. Companies' ability to profit, in the sense of increasing strategic equity from sponsorship of events or causes, varies greatly. The reason is that some of their sponsorship efforts are “on strategy,” and some are not. If the corporate philanthropy efforts are focused not only on being a leading corporate citizen, but also on building the image of a leading corporate citizen, then the community, but as well as the employees, customers, and other stakeholders, will benefit.
Which marketing elements are working and what are they affecting? One should measure awareness and recall by key demographic and behavioral variables. And, one should be able to measure the effects of company messages on the beliefs and salience of the strategic elements (the attributes, functional consequences, and psycho-social consequences) that are the decision elements of one's target equity segments of customers. And, as mentioned, some types of promotional activities are intended to affect corporate image, so these measures should also be analyzed for differences resulting from exposure or participation in sponsored events. Perhaps most telling is the longitudinal aspects of measuring pre- and post-differences corresponding to before and after a marketing activity. And, to carry this a bit further, the possibility of correlating the co-relation of gains in equity directly to these marketing activities becomes possible.
(1.2.4) Identify Trends and their Underlying Causes.
By asking a panel of consumers to explain trends in their consumption behaviors (i.e., FUTURE TREND ANCHOR), one can get insight into the reasons that changes have occurred in a particular market, as well as insights into the likely future competitive environment in which an object competes. This is accomplished by asking the consumers how their behavior is different today as compared to some product-relevant time frame (e.g. one year ago), and how it will likely change, for example, in the next year. Understanding the “Why?” of these customer-perceived changes provides management with the ability to substantiate the reasons for changes in sales, as well as the ability to understand future trends that are likely to influence their sales. Tracking changes in sales, share, entry, or exit data will give an after-the-fact trend line, whereas the StrEAM™ methodology will give another, superior one that explains trends from a customer's point of view. The value of an “early warning system” such as this for management cannot be overstated.
The steps of such a market research tracking process involves computer-aided interviewing software for adaptively asking relevant questions to individual interview respondents. The computer-aided interviewing software tailors questions to each particular dialog during an interview thereby greatly reducing the number of questions asked of each respondent, and thus providing greater overall efficiency to the market research process. To illustrate, consider the following steps of a computer-driven interview. (This research platform assumes, like all such market research tracking models, that an appropriate sampling of a target population group is identified.) The categories of questions are:
Consider the case where management of a carbonated soft drink company, with several products in their portfolio, including non-carbonated beverages such as juices and water, would like to understand the interactions across their products and their respective competitors. Only by defining the competitive set in the broadest possible terms, i.e., a meta-category for their products, can these interactions be understood.
The meta-category competition framing question for this example is, “What is your share of the commercial non-alcoholic beverage market?” which necessarily includes defining competition across non-alcoholic beverages.
For the market analysis of the non-alcoholic beverage company's product portfolio, the inputs that are required for the market research computer-aided interviewing system disclosed herein to implement the market research tracking process (i.e., also denoted “StrEAM™ STRATEGIC EQUITY TRACKING”) are:
When a representative sample of consumers is obtained, a decision as to the definition of loyalty is required. This can be done in several ways, including overall percentage of consumption by occasion (time of day) and/or by functional subcategory (e.g. diet colas). Once this decision is made, the types of analyses used to understand equity is almost limitless. The framing of these analyses, however, is centered on understanding the (loyal) classification categories output in a StrEAM™ joint distribution graph (e.g.,
The StrEAM™ ADVERTISING STRATEGY ASSESSMENT provides the fifth of the “Five Aspects” briefly described in the Summary section hereinabove. In particular, this aspect of the present disclosure provides a methodology to quantify the contribution of key perceptual associations, corresponding to customer decision structures, caused by communications that drive affect for the product/service.
Communication or positioning strategy is the process of specifying how consumers in a target population will meaningfully differentiate an object (e.g., a brand, company, idea, or candidate) from its competitors (Reynolds and Rochon, 1991, Ref. 27 of the “References” section). The phrases “specifying” and “meaningfully differentiate” are noteworthy.
Several benefits accrue when management clearly articulates and specifies the basis for positioning strategy. First, company management retains control of the process. Strategy is, after all, the responsibility of the company, not the copywriter or agency account manager. Next, the strategy articulation provides a basis for the discussion of alternative executions, based in a common lexicon. Finally, managers and agency personnel can assess advertising executions and their delivery against desired product positions objectively. This benefits both the agency and the manager, since it keeps the agency on strategy and protects the agency from arbitrary second-guessing.
The phrase “meaningfully differentiate” refers to the goal that advertising strategy must be in the consumer's own language and follow decision pathways that ensure that the message is personally relevant. Thus, by understanding what are the leverageable strategic elements, through the StrEAM™ family of research methodologies, that drive satisfaction and/or loyalty, that in turn define strategy, the MECCAS framework (Reynolds and Gutman, 1984, Ref. 23 of the References section hereinabove), to be defined below, permits a direct translation to advertising strategy specification.
To facilitate the specification process, a manager can use the MECCAS strategy model, where the components of the model are isomorphic to the decision structures developed through means-end theory. MECCAS is an acronym for Means-End Conceptualization of the Components of Advertising Strategy. This framework helps, e.g., a manager translate the understanding of consumer decision making into advertising language. The MECCAS framework is usually presented as a hierarchical sequence of levels of object evaluation, wherein “Message Elements” are at the bottom or lowest level of the hierarchy, and “Driving Forces” are at the top level of the hierarchy. In particular, these levels directly correspond to the means-end decision structure generated from means-end data. That is, the correspondence is as follows:
Once management decides what is to be communicated or linked (i.e., the positioning strategy), it is the job of the creative team to create three “bridges” linking the product to the self. The product bridge, linking message elements and functional or performance benefits, the personal relevance bridge, linking consumer benefits with the leverage point; and the values bridge, linking the leverage point to the driving force. An illustration may make the process easier to understand.
In most product classes, decisions are made, not at the level of values, but at a lower, psycho-social consequence level, e.g., the level corresponding to concepts such as “coping” or “caring.” Note that Reynolds and Trivedi 1989, Ref. 31 of the “References” section, found that the highest correlations for product (more generally, object) preference were obtained-from marketing statements directed to the “Leverage Point”, which corresponds to the psycho-social consequence level of means-end decision structures rather than the value level. Moreover, within the concept of “coping,” one can imagine: (i) people who are coping by hanging-on-by-their-fingernails-and-hoping-to-get-through-unscarred (i.e., need for Peace of Mind), and (ii) people who are coping by I-have-lots-to-do-and-I-can-do-more-and-get-that-corner-office (i.e., need for Accomplishment). These two types of coping are defined by their respective higher-level goals or end-states, represented by their personal values. But, it is coping that is the “leverage” to activate corresponding end values. Thus, a marketing an image of a product/service as providing or facilitating a better way to cope with a potential customer's circumstances can be the most meaningful driver for affecting a favorable response to the product/service. To illustrate this point,
Indeed, the message to such a decision segment (Accomplishment driven) is different than the message directed at a target segment motivated by “holding on,” with an orientation to just get through the day (Peace of Mind driven). Understanding this difference that is grounded in meanings, which are defined by the connections between the respective levels, is the focus for the new research methods that will be introduced in the following section.
The research problems addressed by the analysis model include identifying, which decision networks best predict Affect. This can be accomplished by a stepwise regression analysis optimizing the selection of pairwise connections for each of the three types. This analysis requires that equal weights be applied to the three sets of predictor connections, thereby not capitalizing on the bias often created by least squares optimization (Cliff, 1987, p. 182, Ref. 6 of the “References” section). This means a simple summary composite index can be computed for each combination of the three bridges between decision elements. Note that the independent measures for each decision network range from 0 to 8, which is computed from adding the connection scores, which has a maximum of two for each. In this regression analysis, the summaries of the three-way combinations (across four decision element levels in MECCAS), representing the three connections, are evaluated as to how well the combination predicts Affect (resulting R2). Note that the dependent measure in the regression has five integer scores, 0-4, representing the sum of the two Affect statements. The statistical significance of the multiple correlations for the decision networks provides the order of contribution and thereby identifies what possible other decision structures, representing positioning strategies, are activated by the communication. To obtain a measure of overall fit, or predictability accounted for by the respective decision networks included, another regression analysis, permitting least squares weights to be computed, can be done. The R2 output provides an upper bound estimate of how much affect is explained.
There are two sub-models of this analysis, which result from the assumption regarding common elements in the decision network. Model I does not allow any common elements in any levels, essentially yielding statistically independent dimensions. Of course, true independence is unlikely, due to the commonality of meanings (which translates to dependence) between and across of the decision elements. Model II permits common elements to be used in the different decision networks.
There are two primary inputs to the computer program that administers the communication strategy assessment: affect questions, for both product and the advertising, and statements that correspond to the decision elements by means-end level. The program has flexibility to accommodate statements for the Executional Framework and qualitative responses, as well, but these are optional and as such are not an integral part of the strategic analysis.
There are three types of strategic questions presented. For the first type, the computer program presents Affect statements using a standard scale format, anchored by the degree of agreement with the specific statement. Note that Affect statements are a combination of two statements. For example, Affect for the Product/Service is a composite summary score of “increase liking” and “more likely to buy (intent).” The second type, the decision element statements, phrased appropriately to the level they represent, are presented in a two-step process. The first question asks if it the concept is “CLEARLY” communicated (YES or NO). The second question is asked only in the case of a YES response, and it focuses on the strength, “CLEARLY” or “PERFECTLY.” This two-step process is key in that it permits the program to adaptively only ask the relevant questions of the third type. This final type focuses on the degree of connection or association between the decision elements caused by the advertisement. The three-point scale used to represent connectivity is presented in a Venn diagram format, with approximately 15%, 50% and 85% overlaps, respectively.
The weighting system utilized to assign weights to the responses for the NOT CLEARLY, CLEARLY and PERFECTLY response categories for the statements are 0, 62.5 and 100, respectively. Note that these weights were derived from a series of studies contrasting different scale markers on 100-point scales. The strength of connections is scored 0, 1 and 2, respectively. A multiplicative composite score for a connection is computed using the relevant ends (the statements scored 0, 1 and 2) multiplied by the connection strength between them (0, 1 and 2), which yields a range of scores from 0 to 8 (2×2×2). The resulting product is then assigned a number ranging from 0 to 9. These equal distance ranges of outcomes for each assigned number are defined by the probabilities of random occurrence of the possible combinations of connection product scores (0, 1, 2, 4 and 8).
The resulting numbers output by the computer program reflect the (mean) strength of communication of a statement (decision element) caused by the advertisement on a 0-100 scale, and the (mean) connection strength for all pairs of statements on a 0-9 scale.
Conceptually, the strategy assessment process mirrors the strategic goal of advertising, namely, linking the product (defined by its attributes) to the person (defined by personal values) using the (differentiating) decision structure that drives choice. The interpretation of the strength of a decision network created by the advertisement is the evaluative criteria as to how effective it is in communicating the positioning strategy represented by the entire network of meanings.
Analysis of over 100 advertisements using the StrEAM™ ADVERTISING STRATEGY ASSESSMENT methodology across product classes using advertisements from different countries reveals that a single composite score of the three levels of connections (composite summative scores range from 0-27) correlates 0.71 (r2=0.50) with Affect for the respective product/service. This one-dimensional solution strongly supports the theory that creating connections between decision elements drives the creation of Affect for the product/service. Note that contrasting structural models comprised of the strategic elements to models comprised of only the connections across LOYALS and COMPETITIVE LOYALS reveals significant differences in the basis of how Affect is created and reinforced (see Reynolds, Gengler and Howard, 1995, Ref. 22 of the “References” section).
The application of the StrEAM™ ADVERTISING STRATEGY ASSESSMENT methodology to an a priori defined marketing strategy provides a common framework to assess how well advertising of the marketing strategy delivers the desired positioning. That is, what is needed is the development of an analysis frame that permits additional learning by quantifying the correlational relationship of both the strategic elements and their connections to both Affect for the product and Affect for the advertisement. This new analysis should provide management additional insight, beyond simply assessing their one predetermined strategy, by identifying other strategic elements that have the potential to drive product/service affect, which is the basis of the superiority belief. This application will be of particular value in assessing the competition's advertising communications, as well as gaining a better understanding of their own current and past advertising (which could be related to sales trends at the time it was on air).
The translation of understanding the decision networks that drive satisfaction and loyalty into positioning strategy can be readily accomplished using the MECCAS model. This evaluation of how well a pre-specified strategy is communicated by a given advertising execution can be assessed by the strength of the levels of the key statements corresponding to the decision elements and their respective levels of connectivity.
By developing a research methodology to investigate advertisements where there is a general understanding that there is no a priori knowledge as to strategy, or one assumes no a priori knowledge, management has the ability to determine which driving elements are creating Affect. This understanding is particularly useful when studying the competitive communications environment. When results from studying the competitive communications environment is combined with the equity analysis derived from the Equity/Disequity grid (e.g., as in
Strategic equity serves to insulate a brand, company, or service. It provides protection from the competitive forces in the marketplace. Conversely, a store of strategic equity makes one's marketing programs more effective, precisely because one has a base upon which to attract competitive customers (shift their beliefs underlying brand choice).
The logic equation that underlies the StrEAM™ research framework for identifying and quantifying the basis of strategic equity is as follows:
Equity=f1(likelihood of repeat purchase)=f2(loyalty)=f3(satisfaction)=f4(beliefs,importances)
This general equation can be applied to frame marketing problems into research problems that focus on defining the relationships between and across these key functional relationships.
The fundamental grounding of the research process requires gaining an understanding of the customers' decision elements that drive choice. This understanding provides the foundation for the development of optimal strategic options.
There are five interrelated components of the StrEAM™ Process Model, each with their own combinations of research methodologies that define the management problem framing task specific to optimizing strategic equity. These requirements, along with their respective research solutions, are (1) through (5) following:
The MECCAS translation (Reynolds and Gutman, 1984, Ref. 23 of the “References” section, Reynolds and Craddock, 1988, Ref. 20 of the “References” section of communication and advertising strategy to customer decision elements, reflecting the means-end network, is used as a framework to assess communications. The StrEAM™ assessment framework obtains measures of the strength of the strategic elements (decision nodes) and the strength of their respective connections between elements at different levels of the model. Management review of these communication measures reveals the extent to which the communication is “on strategy,” meaning the degree to which it communicates the a priori positioning strategy.
StrEAM™ also presents a methodology to assess advertising communications without an a priori strategy specification. Using Affect as a dependent criterion variable, the optimal predictive set of decision structures (using the three connection bridges as a composite independent variable) can be identified and ordered by degree of explanatory contribution. This methodology provides management with the ability to specify what decision networks are being developed or impacted by advertising, which is relevant to analysis of competitive advertising.
The StrEAM™ family of research methodologies is applicable to solving a wide variety of marketing problems, both tactical and strategic in nature. The basic key to their successful implementation is the framing of the marketing problem in customer satisfaction and/or loyalty terms. This is critical because these constructs represent the operational components of strategic equity of the product/service, which management uses as the guiding metric to their decision-making.
The entire set of StrEAM™ research methodologies is designed to be implemented via computer interfaces with electronic communications. In many cases, the adaptive questioning procedures embedded in the programming are necessitated due to the branching required to select the most appropriate question for the individual respondent. That is, questions are asked (i.e., presented) using the respondent's prior answer as a basis to frame the subsequent question, or using relevant criteria obtained for a respondent. Additionally, because graphical scales and other stimuli are standard to the research methods of the market research system disclosed herein (and referred to hereinbelow as the market research analysis method and system 2902), the ability to present these images and work with them in real time, to focus the respondent on the distinctions of interest, is required.
A more detailed block diagram of one embodiment of the market analysis method and system 2902 (denoted StrEAM herein) is presented in
Multiple, simultaneous, one-on-one interviews between interviewers and respondents (i.e., interviewees) are supported via the web server 2904. These interviews are conducted via Internet communications using standard (Internet) browser-based components on both the client and server sides of such communications. Thus, such interviews can be conducted while providing flexible, geography-independent participation (by both respondents and interviewers).
An interview administrative database 2939 (
In particular, the interview administration database 2939 stores information about the status of market research projects (e.g., data indicative of: (1) the status of an interview design, the status of interviews, e.g., how many interviews have been performed, how many candidate interviewee's have been identified, whether interviewers reviewed the interview, etc., and (2) the status of respondent information, e.g., which respondents have completed which interviews, which respondents need to be (re)contacted, etc.).
In one embodiment, the StrEAM*Interview subsystem 2908 provides a web-based framework in which an interviewer and an interview respondent can interact over the Internet (or another network as described hereinabove) to conduct a structured market research interview. The StrEAM*Interview subsystem 2908 serves up predefined presentations to each interview respondent and provides for open dialog between the interviewer and respondent during the interview. The results of these interviews are captured in a form facilitating both downstream analysis and preservation of the original verbatim dialog between the respondent and the interviewer.
The StrEAM*Interview subsystem 2908 is designed to support a Means-End analysis (cf. Definitions and Descriptions of Terms section hereinabove) to understanding consumer decision-making. This is achieved, not by gathering input from an exhaustive questionnaire, but rather by engaging in a multi-level dialog with individuals (i.e., respondents) about their decision-making process. Of interest are the relationships between, e.g., one or more product (or service) attributes and the perceptions of the respondent(s) regarding the product or service (more generally, object). In particular, such relationships are discovered and explored through an interview technique known as “Laddering” described further hereinabove (cf. Definitions and Descriptions of Terms section hereinabove).
The StrEAM*Interview subsystem 2908 (
The following table provides descriptions of the types of interview questions provided by an embodiment of interview subsystem 2908.
As noted for a given interview, providing probe questions during the course of an interview session can be repetitive from interview session to interview session. Therefore it is possible to create a substantially uniform list of probe questions that might reasonably be used in most of the sessions for the interview. Ladder probe questions in a StrEAM interview take the form of “moving” through a respondent's thought process “from” one level of the ladder “to” another. For instance when a respondent states that “high price” (an attribute) is an issue, the interviewer might ask: “What is the biggest problem that this causes for you?” in order to probe for a functional consequence. A response pointing out “difficulty staying within monthly budget” might cause the interviewer to next ask: “How does that make you feel?” in order probe for a psychosocial consequence.
The market research analysis method and system 2902 includes a StrEAM Ladder Probe Question service 2971 (
Referring to the above PROBE SCHEMA EXAMPLE, instances of probe questions 4504 are asked when particular interview session status conditions are satisfied. For example, each of the first three lines of the above example is illustrative of an “if-then” rule for presenting a corresponding probe question 4504. Thus, if a particular interview session status occurs (e.g., an interviewee response has been categorized as belonging to a certain ladder level, and it is determined that a response for a lower or higher level of the ladder is needed), then the text of the “probe question” is presented to the interviewee. Alternatively, each of the fourth through sixth lines of the above example is illustrative of an “if-then” rule for presenting a probe question, wherein if the interview session status is such that a response from the interviewee has been identified as identifying a particular predetermined code (e.g., a word or a phrase), and it is desirable to obtain a response from the interviewee about a ladder level above or below the identified code, then a corresponding probe question may be presented to the interviewee. Note, such a circumstance may occur when two different ladders have a level identified (by an interview composer) for a common predetermined code.
It should be noted that while the syntax of the above probe question schemas does not require the corresponding probe questions 4504 to be only one level above or below a currently identified ladder level, each probe question 4504, e.g., stored in the ladder probe database 2975 (
Note that a same set of interview session circumstances may satisfy the conditions for more than one probe question presentation rule. Accordingly, if an interviewer is conducting an interview, the interviewer can select an appropriate probe question 4504 to present to the interviewee. However, during automated interview sessions if multiple probe questions are identified as candidates for presentation, the one to be presented may be chosen according to one of the following criteria: (i) chosen randomly, (ii) chosen to complete a particular ladder, and/or (iii) chosen to obtain a particularly important (high priority) ladder level (e.g., a ladder level that is common to many ladders, but whose response can affect the efficiency with which an automated interview session is conducted). Note, such strategies may correspond to the manual interviewing process disclosed hereinabove in that probe question suggestions may be presented to an interviewer during manual interviews, and the interviewer may choose from among the suggested questions to present to the interviewee.
A representative user interface display of the respondent application 2934 is shown in
The interview display area window 3206 is generally for presenting formal stimuli (e.g., a question and/or scenario) to the respondent and receiving a response from the respondent. Such formal stimuli may be presented as a series of “slides” (some of which can be animated) that are controlled by the interviewer conducting the interview. In certain cases, the respondent will interact with the interview display area window 3206, such as answering a multiple-choice question and/or inputting a rating of an object or characteristic thereof. In such cases, the selection among the presented alternatives may be performed with a mouse, trackball or another computational selection device. However, it is within the scope of the StrEAM*Interview subsystem 2908 to obtain such respondent selection via voice input and/or use of a touch screen.
The area interviewer instant message window 3212 is generally for presenting unstructured text entered by the interviewer (e.g., feedback, comments, and/or further information such as explanation or clarification) to the respondent. In
The display area 3218 (also identified as respondent instant message window 3218) is where a respondent can input unstructured text at any time during an interview session with the respondent. Previously sent messages can be displayed (and scrolled if necessary with window 3218). Note that on the interviewer's display (e.g., computer “desktop”), the corresponding window 3218 is typically display only, and responsive text from the interviewer is entered in the interviewer instant message window 3212.
The fourth area, denoted the notes and playback area window 3224, is used for presenting formal responses (i.e., responses recorded by the interviewer) to the respondent for his/her approval. The content of this window is built by the interviewer and when appropriate (e.g., approved by the respondent), is recorded as the formal response to a currently presented interview question or scenario. In particular, the window 3224 is used, for instance, in building a ladder, one such ladder being shown in this window in
The respondent application 2938 also includes several items that display information (e.g.,
In addition, there are several items on the respondent's computer that provide some control over the respondent application 2934 or can be used to help respond to the interviewer.
Note that the table above does not include the user interface controls that are provided to the respondent for interactive interview questions. Such controls are specific to each form of interview question and are provided within the interview display area window 3206.
An annotated sample of the StrEAM*Interview interviewer desktop application 2934 is shown in
Below is a list of the display items provided by an embodiment of the interviewer application 2934 (
In addition, the interviewer application 2934 provides several items that provide user control over various features. These items are given in the table below.
Most of these items are buttons that appear in the location indicated by interview control buttons in
In one embodiment, there are context-dependent pop-up (right-click) browser menus for assisting the interviewer by, e.g., providing “hints”. Basically, the function of any of these Interviewer application context menu pop-ups is to offer a piece of text to be input to the applicable text entry buffer for transmitting to the respondent application. The text will not be automatically sent; the Interviewer must activate the sending (by hitting return). This way the Interviewer is able to edit (if desired) such text prior to it being transmitted to the Respondent.
Note that if there is any text in the chosen buffer prior to the menu choice it will be overwritten by the menu choice.
If a right-click is detected over the interviewer's text box (or in one embodiment, the whole Interviewer dialog window) any interviewer hints (as described in the section “Interviewer Hints” hereinbelow) that are available are displayed.
The options that are available for use with the interviewer hints may be:
There are three different Notes area 3224 modes depending on what kind of input is being constructed by the interviewer. These modes are represented by: (1) ladder building result boxes, (2) set building result boxes, and (3) simple response boxes. Each of these is described immediately below.
When a ladder is being constructed as the official interviewee response, there are four text boxes in the Notes area 3224 (i.e., one for each of: a value response, a psychosocial consequence response, a functional consequence response, and an attribute response).
Right-clicking over any of these boxes will offer the following options:
When building a set as a result (this is both for a set generator and for a set elaborator), there will be several text boxes in the Notes Area 3224 (the number specified by the IDefML set-maximum attribute for the set producing topic, or the number of actual set members when doing a set elaboration). Each box will correspond to an answer. There is an unselected box border (thin white) for boxes not selected, and a selected box border (thick yellow) for selected boxes.
The right-click/pop up menu available is the same for all boxes. The box that is right-clicked over is the box that will be the target for whatever is pasted. The options are:
This is a text box in the Notes Area 3224 that will be used to construct and/or replay any other kind of response. When a user right-clicks over this box the options available are:
StrEAM Interview data includes the data structures, data content for defining the behavior of interviews performed by the StrEAM*Interview subsystem 2908. The StrEAM Interview data may be specified in data repositories such as files (e.g., IDefML data definition 3110 files and resource data 3114 files,
In at least one embodiment of the market research analysis method and system 2902, interview sessions are defined (and controlled) by above mentioned two types of data; i.e., IDefMS data 3110 (e.g., provided in a plain text file) for defining the structure of each interview, and resource data 3114 (e.g., provided by a Macromedia® Flash® player movie file that defines the graphics and interactive behaviors for the interviewer). Note that the interview definition data 3110 is also referred to herein as the “StrEAM*Interview Definition file”, and the resource data 3114 is also referred to herein as the “Flash® Interview Resource file”.
StrEAM interview sessions may be composed of a series of “slides” that are informational or ask for a response from the respondent. The sequence is ordered according to the interview definition data (file) 3110. However, note that such an ordering may include a random rotation of questions and groups of questions. Various branching and conditional interview session controls are also available based on respondent answers to previous questions. References and/or statements can be incorporated into the interview definition data 3110 between interview questions provided therein to control interview session flow, and also to amend the display. In addition to interview questions directed to completing various ladders, the interview definition data 3110 may additionally include interview questions (or imperative statements) directed to non-laddering questions such as the question types described in the examples of section (1.1) hereinabove. In particular, the following types of questions (or imperative statements) may be provided in the interview definition data 3110: anchor questions, +Equity questions, −Equity questions, top of mind questions (section 1.15), expectation questions (section 1.14), usage questions, trend anchor questions, valence questions (section 1.15), etc.
Practice questions can also be included in a StrEAM*Interview session in order for the respondent to become acquainted with the various mechanisms.
StrEAM*Interview subsystem 2908 interview data may be comprised of a series of “topics”. Each topic presents its own graphical display on the interview desktop (for both the interviewer and respondent). In some cases, a topic graphical display can be static, and in others such a display supports interaction on the part of the respondent. Each interview topic may also cause messages to be sent from the interviewer to the respondent by way of the interviewer's instant messaging capability.
Some interview topics are only for informational purposes whereas others include one or more questions related, e.g., to an object being researched (such interview topics are referred to herein as “question topics”). In the case of question topics, the both the interviewer computer and the respondent computer enter a corresponding mode such that the respondent is substantially required to answer the question presented, and the interviewer application 2934 records the answer. The actual behavior of a question topic depends on its type.
The interviewer controls the pace of an interview session, and the interviewer is responsible for advancing from one topic to a next. In the case of question topics, the interviewer cannot generally proceed to a next topic until a satisfactory response has been gathered from the respondent for a present topic, and recorded. The following is a list of the form of questions that may be presented in an interview session:
In at least one embodiment, the data 3110 and 3114 are further described as follows:
During the course of execution of a StrEAM*Interview for an interview being conducted, there is an output (e.g., a file) which is stored respectively in the interview archive database 3130 (e.g., as shown in
Interview communication interview dialogs conducted via the market analysis system 2902 may utilize multiple forms of network communication. In particular, substantially any combination of the audio, visual (video and/or graphs), and textual forms of communication may be used during an interview session for communicating between an interviewer and a respondent, depending upon the hardware communication capabilities of the interviewer and respondent computers. Note, however, that typically only a subset of such communication combinations will be utilized, particularly on the respondent side, in order to minimize the interview set up effort required on the respondent end.
More details of one embodiment of an XML language used for defining interviews are provided in Appendix A hereinbelow.
As described hereinabove, one embodiment of the StrEAM*Interview subsystem 2908 uses interview resource data 3114 as part of the data for defining an interview. The interview resource data 3114 contains the information for specifying the behavior of, e.g., the interview questions/topics presented to an interviewee. For instance, the manner in which the StrEAM*Interview subsystem 2908 asks a multiple-choice question can be changed via the interview resource data 3114. Such resource data 3114 contains the graphical user interface resources for a corresponding interview. Both the interviewer application 2934 and the respondent applications 2938 load the resource data 3114 corresponding to an interview session. The resource data 3114 contains default presentation techniques for all of the different interview topic types. Such resource data 3114 also can contain custom developed presentations to be used for specific topics, as would be indicated in the corresponding interview definition data 3110. Such resource data 3114 may be provided in a format for execution by a Flash® Player from Macromedia Inc. as one skilled in the will understand. Hence, the resource data 3114 may be embodied as a Flash® interview resource file, as one skilled in the art will understand. However, other types of resource data 3114 are within the scope of the present disclosure. For example, resources (e.g., movies, animations, graphics, audio clips, etc.) may be presented in interview presentations according to information in the interview resource data 3114.
Each instance of resource data 3114 is typically specific to a particular interview definition data 3110 (though a generic default resource may be used where no custom interview presentations are to be presented during an interview session). As with the interview definition data (file) 3110, the path to the flash interview resource data (file) 3114 is specified to the interviewer and respondent applications (2934 and 2938 respectively) at commencement of an interview session.
The resource data 3114 may include labeled Flash® movie “frames”, as one skilled in the art will understand. The StrEAM*Interview applications 2934 and 2938 use the “goToAndPlay(label)” directive to cause the Flash® Player invoke one of these resources. Those frames can contain any combination of graphics and ActionScript. Multiple frames can be used if desired so long as the resource concludes with a “stop( )”. For very complex effects, a resource may in turn load and play other Flash® movies. Through this mechanism, this embodiment of StrEAM*Interview can support interviews of unlimited complexity with respect to user interface.
As implied above, it is possible to define an interview that makes no explicit references to custom interview resources. This is because the basic behavior is provided by a set of default interview resources that will always be available in each interview resource data instance 3114. These resources support the default functionality of all of the current interview topic types. The default resources are as follows:
Custom variations of the display slides may be created, typically to provide richer graphics for the display area. Slides can be defined with different and more complex text formatting than provided by the default mechanisms. Or they may be built to display more advanced graphics as part of the stimuli for an interview. This includes the delivery of full Flash movies and/or video.
If a custom resource is to be used in a context expecting some form of animation, care must be taken to make sure that the custom resources comply with the input/output requirements. Generally it is best to start with a copy of the applicable default resource and customize from there.
The results of an interview session with an interviewee may be written to a plain text file (e.g., XML file 3118,
Below is a summary of the interview question types from the perspective of how the “results” are formed and captured. Note that in cases where additional questions are asked for each member of a multi-valued answer (“set” or “ladder” elaboration), the result format for each responses to the additional questions is independent of the fact that the additional questions were generated as the result of an elaboration.
The result is that there are four (4) basic forms of result construction:
Each of these is described in more detail below:
Along with the responses to interview questions, information about the overall interview session itself is recorded in a corresponding StrEAM*Interview result file 3118. This file includes, e.g., the identifiers for identifying the interviewer and the respondent, the date and time the interview session began and finished, and other such information. Additionally, respondent responses to all questions are retained in the interview result file, and in the order such responses were provided by the respondent.
Note that for chip allocation questions, each allocation option is recorded (even those with no chips allocated) along with the number of chips that were allocated to it by the respondent. It should be noted that chip allocation options may be randomized for a given interview session.
For ladder questions, the results therefrom are recorded in the form of ordered ladder elements (cf. the Definitions and Descriptions of Terms section hereinabove).
More details of one embodiment of an XML language used for defining interview result data is provided in Appendix B hereinbelow.
In one embodiment, the StrEAM*Interview subsystem 2908 is intended to support an interactive dialog between an interviewer and a respondent. The subsystem 2908 allows for unstructured dialog between the interviewer and the respondent. However, as an optimization, the StrEAM*Interview subsystem 2908 may provide some automated assistance to the interviewer for inputting dialog to be communicated to the respondent. The availability of such assistance, as well as some of its content, is controlled by entries in the interview definition data (IDefML) entities 3110 (
Interviewer assistance is provided in the form of context-specific pop-up menus (e.g.,
Additionally, convenient paste options are provided that allow the interviewer to select and copy, e.g., a display of respondent text input (if available) to another area for the interviewer display. Moreover, an interviewer may have access to various interview information agents (e.g., software programs reviewing the interview process) that can provide the interviewer with “hints” regarding how to proceed with the interview. In general, such hints may be pre-formed questions or statements (whole or partial) that can be used when probing a respondent or capturing a desired respondent response.
Interviewer hints are aids for the interviewer during the interviewing process. For an instance of an interview question, an interview composer (also referred to as a designer herein) may include a set of interviewer hints. If such hints are provided, then during, e.g., a Question state or other appropriate context, a pop-up menu is displayed in (or near) an Interviewer Dialog box 3500 (
An interviewer can either send selected hint text verbatim to the interviewee, or edit it to form, e.g., a more specific probe question for the interview.
An interviewer-hints element in the interview definition (IDefML) file 3110 has the following form:
Accordingly, activation of an IDefML interviewer-hints element by the interviewer clicking his/her mouse button in the Interviewer Dialog box 3500 results in a pop-up menu with the content of menu 3504 of
An interview composer (
Representative examples of the pop-up menus for hints are also shown in
An example of a data set for defining Ladder Hints is given immediately below:
An interviewer proceeds sequentially through a series of presentations, continuing from one step to the next only as allowed by a predetermined interview framework as defined in a corresponding interview definition data 3110.
Since the interviewer application 2934 controls what happens on the respondent application 2938, the interview workflow may be described in terms of the state of the interviewer application.
Interviewer application states may be described in terms of interview “presentations”, wherein the term “presentation” refers herein to a semantically meaningful segment of the interview. Said another way, “presentation” refers to a collection of program elements for presenting interview information to the interviewer (and likely to the respondent as well), wherein the collection is either executed until a predetermined termination is reached, or, the collection is not activated at all. Accordingly, each such “presentation” corresponds with what is commonly referred to as a database transaction. There are four general types of presentations: OPENING, CLOSING, QUESTION, and INFO. The QUESTION type is where the interviewer causes a presentation, requiring a response from the respondent, to be presented to the respondent. In one embodiment, the interviewer may select such a presentation from thumbnail displays provided to the interviewer by the interviewer application. However, at least one preferred sequence of presentations is available to the interviewer for conducting the interview session. The OPENING and CLOSING presentations are special placeholder presentations at the beginning and end of an interview session, respectively. The collection of program elements for these states, respectively, initiates and terminates the capture of interview information. The INFO presentations are for presenting introductory information to the respondent, or help information to assist a respondent during an interview. No interviewer or respondent action may be required by an INFO presentation.
Given the above discussion of interview presentations, there are four basic states or modes that the interviewer application may be in during an interview session. They are:
An interview, therefore, progresses from an OPENING presentation through any number of QUESTION and INFO presentations until reaching a CLOSING presentation.
Note, for each such hierarchy for which the interview is designed to elicit at least one of the levels (D)(1) through (D)(3), such a hierarchy is typically obtained from interviewee responses identifying the reason(s) the interviewee provided a rating in an equity question according to (C)(1) or (C)(2) above.
Subsequently, in step 2722 (following step 2714), the interview design is used to create the data structures and data files that define the interview. In particular, the interview composer tool 2940 (
Returning to step 2718 mentioned above, the relevant characteristics of the group or population of individuals to be studied are used to identify and recruit a representative sampling of the group for being interviewed and capturing their responses to the research interview questions designed in steps 2714. Techniques and commercial enterprises for identifying such a population sampling are well known in the art. Subsequently, steps 2730, 2734, and 2738 may be performed in substantially any order. In step 2730, interviewers may be scheduled for conducting and/or assisting with conducting interviews of members of the representative sampling. However, in one embodiment, the scheduling of interviewers may be unnecessary since the interview process may be automated so that interviews are conducted substantially without an interviewer being involved.
In step 2734, interviews are scheduled with members of the representative sampling. In particular, each such member is provided with an Internet uniform resource locator (URL) of a chat room 2972 (
Referring to step 2738, the Internet chat room 2972 is configured (if necessary) to appropriately communicate with the prospective interviewees from the sample. In particular, the chat room 2972 may be configured to respond to a sample member's initial contact with a welcome message identifying the interview that the sample member is going to take, and when the interview is estimated to actually commence. Moreover, the message (and subsequent chat room communications) may be in a language previously designated by the sample member, e.g., in a language identified by the sample member in step 2734 as being preferred when the interview is presented to the sample member. Additionally, communications in the chat room 2972 may be used to assure that the sample member's computer 2937 (
Subsequently, once all of the above described steps of
In step 2754 a determination is made as to whether the sample member has qualified to be interviewed. Note that such qualification includes each of the tasks (A) through (C) immediately above are sufficiently satisfied so that appropriate and relevant interview data results. Assuming the sample member qualifies to be interviewed, in step 2758 the sample member (who now can be referred to as an “interviewee” or “respondent”) waits for an interviewer to contact him/her. Alternatively, the interviewee may be instructed to contact a designated interviewer computer 2936 via the Internet. Alternatively, the interview manager 3126 (
Assuming that the interviewee waits for an interviewer to contact him/her, and that the interviewer contacts the interviewee (in step 2762), in step 2768 the interview is conducted as is described further herein.
The StrEAM*analysis subsystem 2912 (
In
As with the StrEAM*Interview subsystem 2908, the StrEAM*analysis subsystem 2912 implements a document metaphor for persistent data. That is, the primary data unit for access, storage, and processing are collections of a single data organization referred to herein as a “document”. By employing this document-centric view rather than a database-centric view of data, StrEAM*analysis subsystem 2912 provides enhanced support for iterative and team-based operation of the object research market research process disclosed herein.
Analysis of interview data (i.e., data obtained from interviewee responses) is accomplished by developing and applying a meaningful system of codes to interview data collected during a market research interview process.
Since each StrEAM*analysis configuration database 2980 includes information for a corresponding market research project conducted, there may be a plurality of such StrEAM*analysis configuration databases 2980, one for each distinct market research project conducted. To simplify the description herein, a single configuration database 2980 is shown in the figures and described hereinbelow. However, this simplification is not to be considered as a limitation of the present disclosure in that it is to be understood that the processing provided by the analysis subsystem 2912 described herein can be applied to each of a plurality of configuration databases 2980.
Each StrEAM*analysis configuration database 2980 contains analysis configuration data for supporting analysis of the corresponding interview data. Each StrEAM*analysis configuration database 2980 contains a variety of elements that are used to code and manipulate the corresponding interview data provided in a corresponding analysis model database 2950 (
Central to the analysis of such interview data is a process of grouping the textual responses given in response to, e.g., laddering questions and equity leverage questions. This grouping process is iterative and, in one embodiment is performed by an analyst who quantifies the qualitative (and subjective) interviewee responses in a meaningful way. To “discover” the (or, at least one) useful way to categorize such interview session responses, an analyst may need to study the results obtained from classifying (i.e., “coding”) such responses according to various collections of classifications (i.e., codes). Accordingly, the StrEAM*analysis subsystem 2912 provides both convenient graphical visual tools for an analyst, and also tools for generating a data model that supports an iterative coding and analysis process for interview data.
In particular, the StrEAM*analysis subsystem 2912 allows for sets of codes (i.e., code sets 3942 in the analysis configuration database 2980) to be used to: (a) identify or categorize individual elements of interviewee responses to ladder questions, and (b) identify or categorize (any) open-ended, qualitative interview questions. Each such code set 3942 contains a list of codes (i.e., identifiers or descriptors, identified in
Code sets 3942 for categorizing individual elements of interviewee responses, and in particular, responses to ladder questions, are provided. In one embodiment, each such ladder related code set 3942 must be identified with one of the four ladder levels: value, psychosocial consequence, functional consequence, or attribute. In one embodiment, the code sets 3942 for particular ladder questions may be provided may be provided in the analysis model database 2950.
For code sets 3942 that categorize open-ended interview questions, each of these “question” code sets 3942 may be, in one embodiment, provided in the StrEAM*analysis configuration database 2980 (however, this embodiment is not shown in the figures). In one embodiment, each such question code set 3942 is identified by (or associated with) an identifier of a corresponding open-ended question from the corresponding StrEAM*Interview definition data 3110 (
In one embodiment, the StrEAM*analysis configuration database 2980 includes the code sets 3942 that are the derived from code sets in the model database 2950. Note that the configuration database 2980 may include only one code set 3942 for each of the four ladder levels, and only one code set 3942 may be defined for each open-ended question. In order to have different code sets 3942 for the same ladder (or open-ended question), multiple StrEAM*analysis configuration databases 2980 can be created.
Additional description of the data in the configuration database 2980 follows.
Qualitative interview data is coded to enable its analysis. Sets of codes are defined as part of the process to be used with the individual elements of Ladder answers, as well as any other open-ended, qualitative interview questions. Each code set 3942 contains a list of codes. A code is an arbitrary string and must be unique within the whole code model (not just a code set 3942). For each code, there is a long description and a short descriptive title for display purposes. A code-set may contain an arbitrary number of codes (including none).
Code sets 3942 can be one of two basic types (as indicated in the type attribute). They may either be for “ladder” coding, or for the coding of other unstructured, open-ended “questions”. In the case of “ladder” code sets 3942, they must be targeted at one of the four ladder levels: Value, Psychosocial Consequence, Functional Consequence, or Attribute. That is indicated in the target attribute. In the case of “question” code sets 3942, the target specifies the question id from the StrEAM*Interview definition file 3110 that the codes in that set will be used for.
Note that in a given configuration database file 2980 only one set of codes 3942 may be defined for each of the four ladder levels and only one set may be defined for a question-id. In order to have different code sets 3942 for the same ladder (or question), multiple configuration databases/files 2980 can be created.
Code definitions 3950 are created and maintained using the Define Codes tool 3972, an screenshot of the user interface for this tool is shown in
It should also be noted that in order to support the iterative nature of code development and data coding, the tools for coding data (for instance Code Ladders 3988) can also be used to create/edit code definitions 3950 on the fly.
Analysis of decision making often requires analysis of responses to more than one ladder question at the same time. This is supported by StrEAM*analysis subsystem 2912 with question groups 3954. Each question group is a named set of ladder questions, the responses to which will be considered together. Each question group 3954 can contain one or more of the ladder questions in the interview. Each ladder question can appear in any number of question groups 3954, but must appear in at least one in order to be considered during interview analysis.
Question groups 3954 are defined in the StrEAM*analysis configuration database 2980, and are maintained using the Configure Analysis tool 3968. A screenshot of the user interface for this tool is shown in
Question groups 3954 provide one mechanism for partitioning interview data for analysis. Another mechanism is provided by data filters 3958. The data filters 3958 specify criteria by which portions of the interview data are selected from the corresponding StrEAM*analysis model database 2950. Each data filter 3958 identifies one or more interview questions that may be used by an analyst in selecting portions of the interview session data 3932. The specification of such data filters 3958 is similar to specifying database queries, e.g., for a relational database. However, an analyst may be provided with a graphical interface for creating such filters as one of ordinary skill in the art will understand. Moreover, at least some data filters 3932 may be specified when the interview is being composed.
Once a data filter 3958 has been specified, input to the filter includes one or more interviewee responses (or codes therefor) for an interview question identified with the filter. Accordingly, when the filter 3958 receives such input, the filter selects, from the interview session data 3932, the interview session(s) having such input as the response to the interview question(s) identified with the filter.
When multiple answers are listed for an interview question, there is a Boolean OR relationship between them (i.e., a selected interview session must have one of the OR'ed answers for that question). When there are multiple questions in a data filter 3958, there is a Boolean AND relationship between the questions. In such cases, a selected interview session must have one of the included answers to each of the questions in the data filter 3958.
Note that the combination of a question group 3954 and a data filter 3958 are used to select a set of data for study in the StrEAM*analysis tool set.
Data filters 3958 are specified and modified using the Configure Analysis 3968 tool as shown in
A common task during interview data analysis is to review the usage of ladder element codes (cf. the Definitions and Description of Terms section hereinabove for a description of ladder element codes) in various subsets of the interview data being analyzed. Although there is a standard report that provides a view of the overall assignment of ladder codes to ladder elements, the StrEAM*analysis subsystem 2912 also provides for the specification and generation of customized reports that can be used to examine ladder code assignment at a more detailed level. Code mention report definitions 3986 defined in the StrEAM*analysis configuration database (or file) 2980 are used for this purpose; i.e., each code mention report definition defines a corresponding code mention report 3987. In one embodiment, a code mention report 3987 provides a report row for each ladder code, wherein these rows are grouped together according to ladder level (e.g., the ladder levels: attributes, functional consequences, psychosocial consequences, and values). Each column may contain the frequency that each ladder code is used within the data selected by a given data filter 3958 (in addition to that used for the current analysis data set).
Below (in the “Example of a Code Mention Report” table) is a partial listing of a StrEAM code mention report 3987. This partial listing provides statistics for various ladder codes identified in the first column. In particular, the ladder codes shown are only for the “attributes” ladder level of ladders whose corresponding interview ladder question(s) relate to the U.S. presidential election of 2004. The full code mention report includes statistics for ladder codes for all ladder levels, e.g., attributes, functional consequences, psychosocial consequences, and values. The information in each of the partial listing's four boxed columns is defined by a corresponding data filter 3958. That is, there is a data filter 3958 for selecting the ladder codes for the attribute level of ladders whose interviewee responses were obtained from one of: (i) male respondents with ages 40 and under, (ii) male respondents over 40, (iii) females respondents 40 and under, and (iv) females respondents over 40. Each of the boxed columns includes two values per row. The first (leftmost) value of each row gives the number of times the corresponding ladder code (in the same row) identifies an interviewee response (a “mention”) for the ladders selected by the corresponding data filter. The second (rightmost) value is the percentage of the first value relative to all the ladders selected by the corresponding data filter. For example, the ladder code “Average Citizen Orientation (108)” is mentioned in 2 of the 23 ladders completed by male respondents 40 years old and under (i.e., 8.7% of all selected ladders). Note that in cases where there is more than one ladder element at the “attribute” ladder level of a ladder, the number of mentions can exceed the number of actual ladders (though this is not the case in partial listing below).
Multiple code mention report definitions may be specified in the StrEAM*analysis configuration database 2980. Each such definition can have any number of columns (each column corresponding to a Data Filter 3958). Code mention report definitions are maintained using the Configure Analysis tool 3968. A screenshot of the user interface for this tool as it applies to code mention reports 3986 is shown in
The decision analysis tool 3996 performs decision segmentation analysis (DSA, cf. Definitions and Descriptions of Terms section above) of the StrEAM*analysis subsystem 2912 for automating the discovery of major decision pathways contained within a set of ladder data. All of the parameters that control the automated decision segmentation analysis can have defaults specified in the StrEAM*analysis configuration database 2980. The parameters for the decision segmentation analysis process (via the decision analysis tool 3996), and their effect on decision analysis tool 3996 behavior are described in the section covering decision segmentation analysis (titled: StrEAM*analysis-Decision Segmentation) further below.
The StrEAM*analysis subsystem 2912 has the ability to export interview analysis data to target applications such as the statistics package, SPSS® (from SPSS, Inc.) and Microsoft® Office Excel 2003. To control the data which gets exported, each StrEAM*analysis configuration database 2980 can include the definition(s) of named export lists 3962. Each such list contains a detailed list of the data items to export, as well as the sequence in which to export them.
Export lists 3962 include specifications of both interview question responses as well as general information about the interview session. Any combination of data can be listed in any order within an export list 3962.
Further description of the data residing in the analysis configuration database 2980 is provided in Appendix C.
As shown in
In addition to interview data 3932 resulting from interviewee responses, each StrEAM*analysis model database 2950 may also contain solution data 3936 output from the decision segmentation analysis tool 3996, wherein the solution data includes: ladder mappings 3940 and decision models 3944 as described in the Definitions and Descriptions of Terms section preceding the Summary section hereinabove (in particular, cf. “Decision Segmentation Analysis” description for “ladder mapping” description). Additionally, the analysis model database 2950 includes proposed and/or temporary definitions of codes (identified as code definitions 3951 in
Each StrEAM*Analysis model database 2950, in one embodiment, may be implemented as a structured, plain text file that contains all of the desired interview response data and information identifying how a particular set of ladder codes has been applied. However, other data repository components are also within the scope of the analysis subsystem server 2914, such as relational and object-oriented databases as one skilled in the art will understand.
Multiple versions of the analysis model database 2950 can exist for the same set of interview data. In one embodiment, more than one analyst may code the same interview data wherein the results from one analyst may be compared with the results from another analyst (via, e.g., the quality assessment tool(s) 2994, and more precisely the compare models tool 3992 of
The configure analysis program 3968 is used by StrEAM project administrators (e.g., via an administrator computer 3964) and analysts (e.g., via an analyst computer 2948) to define several configuration settings of an analysis configuration database 2980 (
The define code tool 3972 (
The exports tool 3976 (
Export lists 3962 include specifications of both interview question responses as well as general information about the interview session. Any combination of data can be listed in any order within an export list 3962. Export tools (in define exports tool 3976,
The top of the screen in
On the lower right portion of the screen is a window that lists all of the data items that are currently part of the selected export list 3962. When one or more of the items in the “ExportListContent” are selected, the REMOVE button in the center can be activated to: (i) remove the selected item(s) from the selected export list 3962, (ii) move them out of the list on the right of the screen, and (iii) add them back to the window on the left.
Alternatively/additionally, the user/analyst may use the MOVE UP or MOVE DOWN buttons on the far right to rearrange the order of the Export List items.
The build model tool 3978 (
Accordingly, the build model tool 3978:
Using the build model tool 3978 an operator (or analyst) may open an existing StrEAM*analysis model database 2950 or create a new model database 2950, e.g., for analyzing recently obtained interview results (e.g., interview session data 3932). StrEAM*Interview result files 3118 in the interview archive database 3130 (
The ladder coding tool 3988 (
A user interface 5404 for the ladder coding tool 3988 is shown in
In
In order to aid the analyst in keeping track of how much coding work remains for the identified analysis model 2950, certain additional statistics are displayed. At 5412, the number of ladders obtained from the interviewee response data are displayed, and at 5416, the number ladder elements obtained from the interviewee response data are displayed. More precisely, for 5412 the following values are provided:
In the upper right of the user interface 5404, there is a scrolling list 5420 that displays a summary of each instance of ladder data from interviewee responses to ladder questions in the selected question group 3954. Each instance of such ladder data is represented by one row in the scrolling list. Each of the rows includes the following items:
When a row of ladder data is selected in the scrolling list 5420, the corresponding details for the selected row are displayed in the fields in the “Current Ladder” window 5424. The first row of data in the window 5424 provides the ladder data ID (e.g., “NZ707C”), the description of the ladder (e.g., “bush-image-ladder”), and the initial ladder question for the selected ladder data (which in
Across the lower part of the user interface 5404, there are four sections titled “Attributes”, “Functional Consequences”, “Psychosocial Consequences”, and “Values”. Each of these sections display a scrolling list 5428 of the current codes for the corresponding ladder level (e.g., attributes, functional consequences, psychosocial consequences, and values). For each of the corresponding ladder levels and its corresponding scrolling list 5428, there is a collection of corresponding statistics shown that apply thereto. Each collection of statistics has a field identifier and a corresponding value field. The field identifier and the corresponding value field are described as follows:
The decision analysis tool 3996 (
The decision analysis tool 3996:
The interview report program 3984 (
Note that, in one embodiment, the reports produced by this tool are written in the SpreadsheetML (XML) language for formatting and presentation with Microsoft® Office Excel 2003. Accordingly, the interview reports tool 3984:
The compare models program 3992 (
In the current embodiment, each StrEAM*analysis subsystem 2912 program listed in hereinabove may be implemented as a Microsoft® Windows application (using VB.NET, as one skilled in the art will understand). Such an implementation provides an analyst with a rich user interface to enhance several of the analysis-related tasks, such as coding data. Note that such user interface analysis tools 2954 (
Bulk coding tools 3994 (denoted herein as the tools Level Elements and Code Elements, not individually shown in the figures) are available to provide alternative views of interview data in a StrEAM*analysis model database 2950, and to provide an alternative mechanism for assigning ladder levels and codes to ladder elements. The tools Level Elements and Code Elements provide views of interviewee responses as independent verbatim quotes, linked only by the question they to which were in response. A convenient graphical user interface then allows the analyst to “drag-and-drop” interviewee responses to ladder questions into appropriate lists according to “ladder level” (via the Level Elements tool), and “code” (via the Code Elements tool).
These tools are designed for the rapid assignment of levels and codes to phrases simply on their own merit. Their typical use is to provide a cross-check of the levels and codes assigned to ladder elements through the use of the standard ladder coding tool 3988 (
Note that as with the Code Ladders tool 3988, in support the iterative nature of the code development/data coding process, the Code Elements tool is also capable of modifying a code definitions (in the StrEAM*analysis configuration database (file) 2980) as well. Codes may be created, modified, deleted, and collapsed into one another.
Regarding step 1012 (
Accordingly, for an object being researched, such a coding process (as in step 1114) attempts to group interview responses from a plurality of interviewees into meaningful categories relative to the research being performed. For example, for an interview question requesting interviewees to describe a least desirable attribute of a particular beverage, one interviewee might reply that the beverage is too foamy, while another interviewee might reply that the beverage froths too easily. Such replies may be categorized into the same category identified by the content code “too easily foams”.
In step 1118, the evaluators 2998 may be activated for each of the qualitative top-of-mind, and equity question responses, wherein there is a corresponding quantitative question whose response is associated with a rating of the qualitative question, generate summary data that classifies each interviewee's qualitative response according to the associated quantitative response. For example, an interviewee might respond to the question: “what comes to mind when you think of General Motors?” with the reply: “Big cars”. Subsequently, the interviewee may be asked “Is that a positive or negative for you?”. Note that an answer to this last question can be presented so that a quantitative response is requested (e.g., discrete values corresponding to a range from “very negative” to “very positive” on a scale of, e.g., 1 to 10). Accordingly, such quantitative responses from all interviewees responding to the interview questions may be summarized using codings of the responses to the first question. For example, categories may be created that are identified by the following code contents: “larger than average cars”, “fast cars”, “economical cars”, “reliable cars”, etc. Thus, the “Big cars” interviewee response above would likely be categorized or coded into the “larger than average cars” category, and for all similarly coded interviewee responses, the total number of interviewees indicating their response is a positive for them can be obtained, as well as the total number of interviewees indicating their response is a negative for them. Accordingly, such totals can be provided as part of the summary data.
Subsequently, in step 1122, a determination is made as to how to analyze the interview responses from the interviewees, i.e., the interview session data 3932 (
Subsequently (in steps 3424 and 3428,
Alternatively, for interviewee responses (i.e., interview session data 3932) that were obtained from equity questions, steps 1130 through 1150 are performed using the evaluators 2998. In step 1130, for each category (C) of non-quantitative responses determined in step 1114 (wherein such responses are +Equity or −Equity responses), determine the importance (I) of the category according to the number of times that interviewee mentions (in the category C) were provided as responses to the equity questions (positive equity and negative equity question). Note that such categories may be functional and/or organizational units of a business enterprise as in the resort example of (1.1.1) above. Additionally/alternatively, such categories may correspond to more general attributes of the object being analyzed. For example, in the museum example (1.1.2) above there are categories identified as “variety” and “presentation”. In general, such categories may be substantially any relevant attributes/features of the research object identified, e.g., by the responses to the framing questions as discussed in the examples of section (1).
In one embodiment, each such importance value may be computed as a percentage of the total number of mentions in responses to equity questions. However, it is within the scope of the present disclosure that other measurements indicative of importance may also be provided, such as for a term/phrase in the mentions, its importance may be determined relative to a particular subcollection of all the terms/phrases in the mentions. Thus, such an importance value may be a percentage (or other value, e.g., a fraction) indicative of the relative frequency of the term/phrase in comparison to other terms/phrases in the subcollection. Alternatively, such importances may be based on interviewee responses to only certain questions (e.g., only positive equity questions, or, only negative equity questions). However, a term/phrase may then have multiple importances; e.g., there may be different importances for different interview contexts. For simplicity below, it is assumed that there is a single importance for each term/phrase mention.
Subsequently, in step 1134, the categories are classified according to: the (higher level) organizational/functional units of the research object that are being analyzed, and/or the features of the research object that are being analyzed. Examples of such organizational/functional units being analyzed, and/or the features being analyzed are provided in the market research examples hereinabove (e.g., the higher level organizational/functional units shown in
In step 1142, a value referred to herein as the “equity attitude” is computed for each of one or more aspects or subentities that are organizational/functional units or features of the object being researched. In particular, such an equity attitude value may be computed for each of the categories and/or the classifications of categories. Each such equity attitude value is a measurement indicative of the importance of a favorably perceived corresponding aspect (e.g., functional unit, feature or attribute) of the object being researched. In one embodiment, for each such object aspect, the corresponding equity attitude value is computed by: (1) multiplying together the importance (I) of the aspect and the belief (B) of the aspect to obtain what is denoted herein as a “non-normalized equity attitude”; and then (2) determining percentage of non-normalized equity attitude relative to the total of all non-normalized equity attitudes for all aspects of the object that are being analyzed.
Other examples of various are organizational/functional units or features of the object being researched follows. In a first example, if the object being researched is a hospital, then an organizational unit of the hospital may be its emergency care division. As another example, if the object being market researched is a particular automobile make (more precisely, the market therefor), then a functional unit that may be important for a target automobile buying population may be the maneuverability of the automobile make. Alternatively, if the object being researched is home exercise equipment, then a feature of such equipment might be its portability (or lack thereof).
Subsequently, in step 1146, a value referred to herein as the “equity leverage” is computed for each of one or more aspects or subentities that are organizational/functional units or features/attributes of the object being researched. Each such equity leverage value is a measurement indicative of a potential gain in favorable perception within a target population that can be obtained by changing the object's corresponding aspect or subentity to which the equity leverage value applies. In one embodiment, the equity leverage for an aspect or subentity (ENT herein) may be computed as:
I
ENT*(10−BENT)/2
where IENT is the importance value for ENT, BENT is the belief value for ENT, and 10 is assumed to be the highest possible value for belief. Note that the rationale for the term (10−BENT)/2 is based upon the assumption that if management for the object focuses on one specific aspect/subentity of the object being researched, such an increase is achievable. Said another way, the units of incremental gain across aspects/subentities of the object are assumed to be defined as one half of the difference to 10 (i.e., to the maximum belief value). However, it is within the scope of the present disclosure that other measurements indicative of equity leverage may also be provided, such as:
I
ENT*(MaxBeliefVal−BENT)/MngmtInertiaENT,
Where MaxBeliefVal=the maximum belief value (10 hereinabove),
It is believed that the aspect(s) or subentity(ies) having highest corresponding equity leverage values are the aspect(s) or subentity(ies) that should be focused on for changing the object's perception in the minds of the target population to a more favorable view of the object (e.g., greater loyalty to the object). So, in step 1150, one or more of the aspect(s) or subentity(ies) having the one or more highest corresponding equity leverage values are identified so that those responsible for modifying the object can focus resources on these aspect(s) or subentity(ies) rather than in other areas.
Moreover, the following additional data is provided in the initialization of the configuration database 2980 via the configuration tools 2990:
Subsequently, in steps 3408 and 3412, codes are iteratively determined for classifying elements of interviewee ladder responses, and applying such codes to the interview data 3932 in the analysis model database 2950 to thereby generate ladders 3995, as one of ordinary skill in the art will understand. In particular, an analyst uses the define codes tool 3972, and the ladder coding tool 3988 (
One result from the steps 3408 and 3412 is the generation of statistics related to the coded ladders and interview data 3932 in the analysis model database 2950. In particular, the statistics generated are described in various sections hereinbelow in the context of the description of the following figures:
In step 3416, the statistics related to the coded ladders and the interview session data 3932 provided by interviewees (for a given object being researched) are explored for determining an appropriate partitioning of the interview data into collections or subsets, wherein for each such collection, the coded ladders and the interview session data 3932 therein relate to and/or identify a single feature or characteristic of the object being researched. In particular, data filters 3958 are the primary mechanisms for partitioning the coded ladders and the interview session data 3932. An analyst may create data filters 3958 that can combine interview questions and their answers in arbitrary ways in order to partition the interview data. An example of such partitioning is described in the Code Mention Report Definition subsection of section (4.1) above. In particular, the Example of a Code Mention Report provided in the above Code Mention Report Definition subsection shows the partitioning of the interview session data 3932 into a combination of various age and gender partition, wherein the age and gender information was provided by interviewees in response to interview questions.
Note that partitioning of the coded ladders and the interview session data 3932 is also provided by allowing an analyst to specify question groups 3954, wherein such groups can be used for combining the answers to questions for different ladders as described in the Question Groups subsection of section (4.1) above. Note that the interview data may be also partitioned in ways that are significant to the decisions about the object being researched.
In step 3424, the most significant decision pathways (i.e., the ladders 3995 that are determined to be most important) are identified from the interview ladder data 3995 for obtaining instances of the decision models 3944. In particular, these pathways are determined by decision analysis tool 3996 (
In some cases, e.g., the decision pathway determination step (i.e., step 3424) and the segmentation step (i.e., step 3428) are performed substantially without human intervention, e.g., various components of the StrEAM*analysis subsystem 2912 may be activated substantially (if not completely) automatically. In the steps 3404 through 3420, where investigation by an analyst is required, the StrEAM*analysis subsystem 2912 provides intelligent automated assistance. Note that such assistance streamlines the interview data analysis process and facilitates rigorous adherence to a predetermined interview data analysis methodology. Of particular note is the support StrEAM*analysis subsystem 2912 provides for the inherently iterative steps 3408 and 3412 of data coding, and the iterative steps 3416 and 3420 for partitioning.
The Decision Segmentation Analysis (DSA) process examines a set of ladder answers 3995 in the model database 2950 that have previously been coded and finds the primary decision paths (known as “cluster chains” in the art, cf. Definitions and Descriptions of Terms prior to the Summary section above) contained in that data. That is, DSA is a process for assigning (or mapping)) each coded ladder (cf. the “Coded Ladder” description in the Definitions and Descriptions of Terms section prior to the Summary section) to a cluster chain that best represents the coded ladder. The assignment of coded ladders to cluster chains is done in the context of what is known as a “solution map” (also denoted as a ladder mapping 3940,
As described in Ref. 24 of the References Section hereinabove, once the a solution map 3940 is generated, one or more decision models 3936 can be derived therefrom. In particular, each such decision model 3936 (referred as a “Hierarchical Value Map” or “HVM” in Ref. 24) is derived by connecting all the cluster chains by the following steps (A) through (C):
Further description of StrEAM*analysis Decision Segmentation Analysis is provided hereinbelow. Note that, as stated above, decision segmentation analysis is performed by an analyst interacting with the decision analysis tool 3996 (
Prior to further description of the decision analysis tool 3996 some clarifying definitions are provided as follows.
Some important terminology for StrEAM*analysis Decision Segmentation is defined here:
An illustrative display provided to an analyst by the decision analysis tool 3966 is shown in
In the middle section (at 5516) of
At the right of the display (at 5528) is the third section providing information on an analyst chosen data filter 3958. A scrollable list at the top of this section is used by the analyst to choose a data filter 3958 to apply to the interview session data 3932 corresponding to the chosen question group 3954. The interview data resulting from using the chosen question group and data filter is the subset of the interview data 3932 to be used by the decision analysis tool 3996 to perform decision segmentation analysis. Note that for the present example (
After a data filter 3958 is chosen, the DSA tool 3996 examines the ladders and ladder elements in the resulting data set. If a ladder does not have at least two valid, coded ladder elements, then it is not useful for DSA processing, and accordingly is eliminated from further DSA processing. In addition, specific codes may be marked as ‘Not for Analysis’ (e.g., typically reserved for ladder element text that cannot be classified). Such codes are also not useful for further treatment by the DSA tool 3996, and accordingly are also eliminated. As a result, it is possible that after such elimination, the DSA tool 3996 may have eliminated some of the potential ladders (and their corresponding ladder elements) from further analysis. The results of this ‘weeding’ process are shown in the sub-section titled “Valid (Included for Analysis)” (at 5532). This subsection gives the number of various ladder classifications and ladder element classifications that will actually be used for (DSA) analysis along with the percentage each number represents out of the total for the corresponding ladder classification or ladder element classification, wherein each total corresponds to the corresponding interview data 3932 that satisfies the chosen data filter 3958. The statistics at 5532 are defined as above for the field at 5508, except—of course—that they represent the count and percentages relative to the valid items in the selected data set.
Finally the section labeled “Analysis Statistics” (at 5536), displays statistics about the interview data to be analyzed by the DSA tool 3996, in the terminology used by DSA. The statistics displayed are:
An illustrative display provided to an analyst by the decision analysis tool 3966 is shown in
Note also that whenever a ladder row (chain instance) is selected in from the display of
An illustrative display provided to an analyst by the decision analysis tool 3966 is shown in
Note that when a unique chain is selected from
An illustrative display provided to an analyst by the decision analysis tool 3966 is shown in
Note that when an implication is selected, the details of the code descriptions can be displayed in a pop-up form (5616) as shown by 5616 in
A concept central to Decision Segmentation Analysis (DSA) is that of assigning (or mapping) each of one or more coded ladders to a cluster chain (and subsequently determining one or more decision models 3944) that best represents the coded ladder. The assignments of coded ladders is done in the context of a solution map 3940 containing multiple cluster chains. Therefore assignment involves deciding on whether a coded ladder is: (a) a good enough fit to one or more of the cluster chains in the solution map 3940 such that the ladder can be assigned, and then (b) determining which cluster chain is the best fit for the coded ladder.
In order for a coded ladder to be considered potentially assignable to a cluster chain, the coded ladder needs to satisfy one of the following two (2) conditions:
In the event that the coded ladder (L) being inspected is assignable to more than one cluster chain under consideration (for a given solution map 3940), then the ladder L is assigned to the best cluster chain fit based on the following steps (applied in the sequence given):
Accordingly, each of the steps 1 through 6 immediately above are performed in order until a single (or no) cluster chain is identified for assigning the ladder L. If after the steps 1 through 6 above have been performed, multiple cluster chains (for a single solution map 3940) still remain as possible candidates for assignment of the ladder L, the algorithm will examine the places where the ladder L, and the cluster chains do not match, and award the ladder assignment to the cluster chain that represents more data in the interview session data 3932.
It should be noted that it is possible for a ladder 3995 to not be assignable to any cluster chain in a given solution map 3940. In that case the ladder 3995 is identified as unassigned.
During the course of Decision Segmentation Analysis various statistics are computed for display and to direct the analysis computations. These statistics are computed for cluster chains as well as for solution maps 3940 as one of ordinary skill in the will understand.
The statistics below apply to cluster chains.
The statistics in the table below apply to solution maps 3940.
Decision segmentation solution maps 3940 are generated through a series of automated steps where generated code sequences (referred to as “potential seed cluster chains” herein) are: (i) created for ultimately generating a decision model 3944, (ii) the potential seed cluster chains are tested against the interview session data 3932 being analyzed, and then (iii) chosen as part of a solution map 3940. The behavior of the StrEAM*analysis DSA process as provided by the decision analysis tool 3996 is highly configurable, and it assists an analyst in performing the following steps:
Each of these steps is described in more detail below.
The creation of a collection of code sequences to be considered as potential seed cluster chains. is determined, in one embodiment, by the following steps:
The collection of potential seed cluster chains is the collection E; i.e., each of the initial code sequences in the collection E is referred to as a potential seed cluster chain hereinbelow. Note that each potential seed cluster chain of the collection E includes at most one code per ladder level, and no codes or levels are repeated in the initial code sequence. Further note that the resulting collection E of potential seed cluster chains has no duplicate potential seed cluster chains therein.
The collection E may be quite large. However many of the potential seed cluster chains therein may be combinations that do not represent interviewee decision paths that actually occur in the data set chosen from the interview session data 3932.
To start the identification process of step (C) above, a potential seed cluster chain from the collection E is selected having the highest significance in the chosen interview data set. In one embodiment, the selected member of E is becomes the first member of a collection referred to herein as a “pseudo-solution map”. This pseudo-solution map is now applied to the interview data set (chosen from the interview session data 3932) in order to determine the remaining-significance (as defined in section (4.4.3) above) of the remaining potential cluster chains. The next potential cluster chain chosen is the one with the highest remaining-significance that does not exceed the limit on overlapping codes with the seed cluster chain already in the pseudo-solution map. This process is repeated until no more potential cluster chains can be chosen.
The next step is to review the collection E of generated potential seed cluster chains and pick out those most worthy of being considered as seed cluster chains. This is done by building a pseudo-solution map of the seed cluster chains, using the significance and remaining-significance metrics for the potential seed cluster chains. Also used is a configuration parameter (in the analysis configuration database 2980) that specifies a limit on the amount of overlap the seed cluster chains may have.
In some cases, particularly when there is substantial overlap of codes between the seed cluster chains identified in Step B (e.g., when an overlap limit parameter is set to a high value), an excessive number of seed cluster chains may be produced by Step B. Configuration parameters (in the analysis configuration database 2980) can define the maximum (and minimum) number of seed cluster chains allowed, e.g., such parameters may constrain the number of seed cluster chains. If the number of seed cluster chains is greater than the maximum, then the DSA algorithm can apply various filters on the seed cluster chains to eliminate those least likely to be important in modeling interviewee perceptions of the object being researched (e.g., via a derived solution map 3940).
The filtering metrics/parameters are described in the cluster chain statistic definitions of section (4.4.5) hereinabove. That is, for each of the cluster chain statistics described in section (4.4.3) there may be a corresponding parameter whose value can be set by, e.g., an analyst. For example, an analyst may assign a value to chain strength parameter so that only seed cluster chains having at least this value are given further consideration. Thus, there may be parameters for the following metrics, wherein values for these parameters can be used to trim the list of seed cluster chains until it reaches a manageable length:
Actual solution maps 3940 are now chosen by examination of each possible combination of the seed cluster chains, e.g., up to a predetermined maximum number of seed cluster chains in each combination. For example, the DSA tool 3996 may produce solution maps 3940 that have anywhere from 2 to 9 seed cluster chains. More particularly, when generating, the 4-dimension solution map 3940 (i.e., a solution map 3940 generated from a combination of four of the seed cluster chains), the DSA tool 3996 determines which combination of 4 seed cluster chains from the seed cluster chain list represent the most ladders (or ladder instances) from the current data set.
When this step is finished, each requested solution map 3940, of a given dimension, is populated by the corresponding number of 4-code seed cluster chains that provide the best solution for the given dimension. Note that production of an n-dimension solution map depends on there being at least n seed cluster chains. Note that the seed cluster chains of the resulting solution maps 3940 are referred to as “cluster chains”.
The final step taken during the Decision Segmentation Analysis process is the review of each cluster chain in each solution map 3940 for the possible addition of another code to the cluster chain. All codes not already in the cluster chain are considered with regards to the impact on chain strength of the cluster chain. If the incremental contribution to chain strength of any code exceeds a given threshold (specified as a configuration parameter in the analysis configuration database 2980), then that code may be added to the cluster chain. If no code exceeds the threshold, then no code is added.
This elaboration of cluster chains is subject to a DSA tool 3996 configuration parameter that constrains cluster chain length (to a predetermined number, e.g., 4, 5, or 6 codes). If constrained to 4 codes, no elaboration takes place. If constrained to 5 codes, then only one code will be added (if any). In one embodiment, two codes are the maximum that can be added, and then only if 6 code chains are allowed.
Each resulting solution map 3940 (output by the DSA tool 3996) is a representation of its collection of cluster chains that may be displayed as, e.g., a directed acyclic graph similar to those of
As noted above, the behavior of the Decision Segmentation Analysis algorithms is determined by settings included in the StrEAM*analysis configuration database (file) 2980, wherein these settings are values for the DSA parameters described in the following table.
Each analysis configuration database file 2980 contains a section of modeling parameters as in the following example:
As this example demonstrates, certain actions are specified to be taken during the Decision Segmentation Analysis and provides parameters to be used.
Various reports are available to the analyst using the StrEAM*analysis system 2914 when using the analyze decisions tool 3996. Some of these reports are generated from specific data sets (where a specific question group 3954 and data filter 3958 are in use). Other reports operate on decision models 3944, and select data based on the decision model specifications. All reports require the availability of an appropriate StrEAM*analysis configuration database (file) 2980.
In one embodiment, all reports produce XML files targeted specifically at Microsoft® Office Excel, using the “SpreadsheetML” language.
Each of the data set statistics reports of
The implication distribution report presents a view of the effect of using alternative values for the implication threshold during decision segmentation analysis. This report summarizes how many implication instances will be utilized for each possible setting of the implication threshold analysis parameter.
The report of
A code usage summary report gives a breakdown of how often codes appear in the ladders 3995 in a chosen interview data set. An example of a code usage summary report is shown in
Each decision model detail report contains the details regarding a decision segmentation analysis decision model 3944 (i.e., “decision model” as described in the Definitions and Descriptions of Terms section hereinabove) output by the DSA tool 3996. Since each decision model 3944 includes of one or more solution maps 3940, each solution map is listed in this report along with its constituent cluster chains and associated statistics.
Following the solution map over all statistics, the four cluster chains identified, each such cluster chain is identified by one of the following labels: “Seed-1”, “Seed-2”, “Seed-3”, and “Seed-4”. Thus, the cluster chain identified by “Seed-1” has: (i) a “value” ladder level of “Peace of Mind”, (ii) a “psychosocial consequences” ladder level of “Confidence”, (iii) a “functional consequences” ladder level of “Trustworthy”, and (iv) an “attributes” ladder level of “Candidate Image”. To the right of each cluster chain, there are corresponding statistics. In particular, the following statistics are shown in
The decision model matrix report presents an alternative view of the details of solution maps 3940. For a given analysis model database 2950, this report lists each solution map 3940 and details about the implications defined by the constituent cluster chains within the solution map.
To the right of each collection of cluster chain statistics is a list of the code titles and corresponding code that comprise the cluster chain. For example, the first code title for the Seed-1 cluster chain is “Candidate Image”, and its corresponding code is “139”. The codes are also duplicated as the heading of columns of a 4-by-4 matrix having entries in only above the main diagonal of the matrix. Both rows and columns of what resembles a miniature “implication matrix”. Each cell of such a matrix contains the number of implication instances (for the corresponding row/column code pair) that have been assigned to the cluster chain (by virtue of ladders 3995 that have been assigned to that cluster chain). As with a normal implication matrix report (e.g.,
Interview session data for StrEAM research studies can be collected from respondent interview sessions through both manual interviews (i.e., having a human interviewer), and automated interviews (i.e., computerized interviews having substantially reduced or no human intervention during an interview session). The StrEAM*Interview subsystem 2908 described hereinabove, provides the tools for conducting manual interviews between human interviewers and respondents. The StrEAM*Robot subsystem 2913 (
To automate the interviewing process, an essential capability is the automation of the actions that would otherwise be performed by a human interviewer when attempting to solicit ladder responses from interviewees. For example, appropriately obtaining, in an open-ended manner (e.g., not having respondents choose from pre-defined ladder answers), ladder responses without a human interviewer, is a key aspect of the present disclosure.
It is important to obtain from each interview session, an interviewee ladder response to each level of each ladder of the interview. Although laddering interviews are, by definition, intimate and one-on-one, the interview interactions by the interviewer, when obtaining a ladder response, may be highly repetitive. In practice, there may be a fairly small set of follow-up probe questions that are presented to an interviewee for obtaining appropriate interviewee responses to all the levels of a ladder. For a given response from an interviewee, choosing the appropriate follow-up probe question to present requires the interviewer to do several tasks, including:
For a human StrEAM interviewer, the choice and wording of follow-up probe questions becomes more and more well-defined as more and more interview sessions (for a given interview) are conducted during a given market research project. As interview response data is collected, the responses fall into the categories that will later (in StrEAM*analysis via the analysis subsystem 2912) be used for coding.
The probe question to ask an interviewee when “moving” the interviewee from a given category (or ladder level) of response to another ladder level is generally straightforward for an interviewer (with experience in conducting the interview session) to determine. For example, such probe questions may be determined according to patterns learned during an initial one or more interview sessions. Moreover, such patterns can be learned and/or provided to an automated interviewing system 2913 (
An embodiment of the StrEAM*Robot subsystem 2913 is depicted in both
Regarding the embodiment of the StrEAM*Robot subsystem in
The sections that follow describe, in more detail, the process of ladder element classification and the ladder probe question look-up for enabling the automation of ladder interviews by the StrEAM*Robot Subsystem 2913
In order to computationally understand enough about an interviewee's response during a ladder question dialog to be able: (a) to classify the response as identifying (or not) a particular ladder level of the ladder being constructed, and (b) to subsequently generate an appropriate follow-up probe question (if needed), the StrEAM*Robot subsystem 2913 uses one or more text classification processes. In particular, the ladder element classification service 2966 described above may use one or more such text classification processes to classify each interviewee response in terms of:
The categorization of text documents is a well-studied topic in computer science, and has been applied in various domains, e.g., the filtering and organization of email, and medical diagnostics. However, the inventors of the present disclosure have applied processes for performing such categorization to the analysis of market research interview data.
In the most general case, a classifier is a function that maps an input attribute vector:
{right arrow over (x)}=(x1,x2,x3, . . . ,xn)
to some measure of confidence that the input belongs to a class:
ƒ({right arrow over (x)})=confidence(class)
For the classification of text, the attributes of the input vector are words from the text. A text classifier, therefore, is a module (e.g., software component) that determines the likelihood that a text string (or document) should be assigned to a predefined category.
In a system to categorize text (or documents), categories are defined, then a text classifier is implemented for each of those categories. As text strings are processed by such a categorization system, each (or one or more) text classifier(s) may be asked to assess the likelihood that the input belongs to its associated category.
It should be noted that classifiers come in two basic forms: binary and multi-class. A binary classifier decides whether an input should (or should not) be assigned to a particular category, whereas a multi-class classifier chooses among several categories for the best assignment. However a series of binary classifiers (for multiple categories) can be used to the same effect as a multi-class classifier, as one skilled in the will understand.
At least some of the text classifiers (e.g., identified in
The training of a text classifier used in StrEAM*Robot 2913 (and more particularly used by the element classifier training tool 2969 shown in
The training of each text classifier is accomplished by supplying examples of text that are appropriate for the corresponding one or more categories that the classifier can classify text into. Further training may also be accomplished by supplying examples of text that should not be assigned to a particular category or categories, as one skilled in the art will understand.
Once trained, each text classifier is activated by supplying an input text string for which the classifier determines a likelihood (e.g., a probability value in the range 0.0 to 1.0) that the input text string should be assigned to each of one or more categories. An example of this is given in the fragment of pseudo code as follows:
Potential assignment of a text string to a category is determined by determining whether a classification likelihood value is greater than some threshold value (predetermined or dynamically determined). Typically such threshold values are established through the by use of some additional test data (of known categories) after the initial training classifiers. Note that an input text string may qualify for membership in more than one category.
Numerous methods exist to provide automated text categorization using text classifiers as described above. As one skilled in the art will understand, below is a sampling of some well-known text classification approaches that can be applied by the StrEAM*Robot 2913 (and more particularly by the ladder element classification service 2966) in processing interviewee responses. The list of classification techniques following is meant to be illustrative, and by no means meant to be exhaustive of the techniques that can be used with various embodiments of the StrEAM*Robot 2913. Note that each classification “technique” hereinbelow includes an inductive learning methodology combined with a classification model, as one skilled in the art will understand.
Note that examples of available implementations of some of the above listed technologies are given in Appendix F herein.
The StrEAM*Robot Subsystem 2913 (and in particular, the element classifier training tool 2969) applies test classification methods such as those listed above (individually or in combination) to create ladder element classifiers 4510 and 4514 (
The StrEAM*Robot Ladder Element Classification Service 2966 uses text classifiers 4510 and 4514 to determine the ladder level and analysis code of a piece of interview response text that is intended or targeted as a ladder element. That is, potential ladder element text is classified for both ladder level and the analysis code. In one embodiment, there can be two sets of “binary classifiers”. One set includes one or more element code classifiers 4510 (
In the pseudo-code following, each of the binary classifiers 4510 and 4514 (identified by the identifier “textClassifier” in the pseudo-code below) determines whether an input text element (“element-text” in the pseudo-code below) should be classified as belonging to the corresponding category for the binary classifier. In particular, when one or more of the binary classifiers scores the input text element high enough, it is assumed that the input element should be categorized in the category corresponding to the binary classifier. Note, there are two configuration parameters (“thresholdScore” and “scoreMargin”) which control behavior regarding what constitutes a sufficiently high score for a text input.
If the list of “best” classifiers that result from the above pseudo-code contains only one classifier, then the assignment is clear (either for ladder level or for analysis code). In the case of ladder level classification, if there is no “best” classifier—or multiple level classifiers are deemed “best”, then in one embodiment, the input will be rejected as invalid, and the automated interview subsystem 2913 may ask the interviewee to restate or clarify the response. On the other hand, in the case of analysis code classification ambiguity, the respondent may be asked to choose the code that is best suited from among those in the “best” list. If no such code is judged “best”, then a new code category may be created as is described hereinbelow in reference to
The Ladder Element Classifiers 4514 for StrEAM*Robot 2913 are trained by first conducting standard manual StrEAM*Interview interviews (with human interviewers using the interview subsystem 2908) with an appropriate (e.g., statistically significant) sample of respondents. The ladder data collected from these initial or pilot interviews is used to: (i) determine and populate ladder levels, and (ii) manually determine codes (i.e., semantic categories) that group together interview responses that appear to have been the result of substantially common perceptions by the interviewees. In particular, the processes (i) and (ii) are performed by market research analysts using the market research analysis capabilities of the StrEAM*analysis subsystem 2912. Accordingly, the resulting ladder leveled and coded data is then provided as input to “train” the classifiers 4510 and 4514 (
As discussed earlier, for a given market research interview, generating probe questions to solicit appropriate interviewee ladder responses is substantially similar from interview session to interview session. Therefore it is possible to create a substantially uniform collection of probe questions and related data structures so that an automated process of ladder probe question selection can be performed. The list of probe questions for an automated interview is stored in the ladder probe database 2975 as depicted in
An effective way to prompt a respondent to provide responses that fill in each of the four levels of a ladder (attribute, functional consequence, psychosocial consequence, and value) is to ask probing, follow-up questions that “move” the respondent through his/her thought process “from” one level of abstraction (i.e., ladder level) that has been elicited, “to” another level of abstraction (i.e., ladder level) that has not yet been elicited. For instance, when a respondent states that “high price” (an attribute) is an issue, the interviewer might ask: “What is the biggest problem that this causes for you?” in order to probe for a functional consequence. A response pointing out “difficulty staying within monthly budget” might cause the interviewer to next ask: “How does that make you feel?” in order probe for a psychosocial consequence.
The ladder probe question service 2971 maintains information regarding the state of an interview session such as which ladders have been completed, which ladder level(s) of which ladder(s) remain incomplete, which ladder (the “current ladder”) is the interview session currently attempting to complete, which level of the current ladder is targeted for completion, and which (if any) transition probe question has been selected for “moving” the interviewee from a ladder level for which an appropriate response has been obtained to another ladder level that requires a response to be entered. Such interview state information determines the ladder probe database 2975 access parameters for accessing this database and retrieving an appropriate ladder probe question for the current state of the interview session.
More particularly, the ladder probe question service 2971 (
As this Ladder Probe Definition Example above indicates, probe question rules include a “from” field (i.e., a “from-level” field, or a “from-code” field) which is either a ladder level, or a ladder element code. Accordingly, for a particular interviewee response (e.g., a most recent response), one or more codes may be determined from the response, and additionally, for a current ladder (if any), a ladder level (if any) of the current ladder having an interviewee response therein may be determined The results of such determinations are used to identify the next probe question rule to select so that its corresponding value for the text of the next probe question (“text of probe question” in the Ladder Probe Definition Example above) can be presented to the interviewee.
Since the probe questions having ladder element codes in the “from” field of their probe question rules may correspond to very specific probe questions (identified by the “text of probe question” field in each such probe), such probe question rules are examined first for determining a next probe question. For a particular code (C), if there is no probe question rule, wherein C is the value for the from-code field, then a probe question rule may be selected having a “from-level” field for the current ladder instead.
Note that the intended classification for an interviewee response is also defined in each probe question rule. That is, in response to the presentation of the text of the probe question (for a selected one of the probe question rules), the “to” field identifies the ladder level to which the interviewee response may be assigned. That is, the “to=” designation for a probe question is always a ladder level. Note that the interview session transition defined by a probe question may be “up” (e.g., from attribute to functional, or from functional to psychosocial consequences, or from psychosocial consequences to values), or “down” a ladder (e.g., from values to psychosocial consequences, or from psychosocial consequences to functional, or from functional to attribute). Thus, associated with each probe question in the ladder probe database 2975 is a direction field indicating whether the associated probe question is for going up a ladder level(s), or going down a ladder level(s). It should also be noted that while the syntax of these probe question definitions does not require such interview transitions to be only one level up or down a ladder, in one embodiment, the ladder probe question service 2971 will typically seek to retrieve a ladder probe question that involves a single level of transition in ladder levels.
Each potential probe question is explicitly associated in the ladder probe database 2975 with the data identifying the interview session circumstances in which it might be appropriate for the probe question to be presented to the interviewee. For example, if an interviewee indicates that expensive electricity was a primary (negative) attribute when discussing satisfaction regarding a local utility, then an appropriate probe question to elicit the functional consequence of that attribute might be:
Note that the same set of circumstances may yield more than one potential latter probe question from the database 2975. During automated interviews if multiple probe questions are identified as candidates, the one to be used may be chosen by various techniques, including: (i) randomly, (ii) a confidence value indicative of past performance in the probe question eliciting the desired interviewee response, (iii) a probe question that is most dissimilar from (any) other ladder probe questions previously presented, (iv) a probe question for a ladder or ladder level that is identified as more important to the completed than ladders or ladder levels of other candidate probe questions.
There is no guarantee that the respondent will provide a response corresponding to the ladder level to which a corresponding probe question (or even an initial ladder question) is directed. Accordingly, to conduct an effective interview laddering session, the automated interview subsystem 2913 (more specifically, the ladder probe question service 2971) must review the state of ladder completion each time an element (i.e., interviewee response) is added to a level of the ladder, and then choose which level needs to be filled in next (e.g., the “to” field in the Ladder Probe Definition Example hereinabove, and also referred as the “target level”). More specifically, the ladder probe question service 2971 determines:
The ladder probe question service 2971 takes into account a preferred transition to fill in a designated target (ladder) level. For example, if both the attribute level of a particular ladder has been filled in during an interview session, and the psychosocial consequence level has also been filled in, then the ladder probe question service 2971 will, in most such circumstances, select a ladder probe question that is directed from the known attribute interviewee response to the unknown functional consequence level of the ladder. Thus, a probe question such as “Given that you like the prompt delivery service of company X, what functional benefit is that for you?” (i.e., a probe question from a previously identified object attribute ladder level to a functional consequence ladder level) is generally preferred over a question such as “Given that you feel less pressure from your boss when packages are not sitting in your office, what benefit does company X provide in reducing this pressure?” (i.e., a probe question from a previously identified object psychosocial consequence level to a functional consequence level).
A representative flowchart for selecting the target level for presenting the next probe question is shown in
Alternatively, if in step 4808, the “target” level is the attribute level, then in step 4820, a determination is made as to whether there is at least one interviewee response element that has been filled in for the ladder L immediately above the attribute level. If not, then step 4824 is performed wherein the identifier “target” is assigned ladder level data for the next higher ladder level. Subsequently, step 4820 is again performed. However, since at least one of the levels of the ladder L is filed in, a performance of step 4820 will eventually yield a positive result, and accordingly, step 4828 is performed, wherein the “from” ladder level data is designated as the next higher ladder level data (from that of the “target” level), and the “to” ladder level data is designated as the “target” level. Subsequently, the identified “from” and “to” data are output (step 4816) for generation of the next ladder probe question.
It should be noted that in some cases ladder probe questions are considered “chutes”, wherein the preferred direction of discussions is moving a respondent “down” through the ladder levels, starting at the value level. In these cases, the algorithm for determining the context or interview state for the next ladder probe question is substantially the inverse of the steps illustrated in
Through the use of the ladder element classification service 2966 and the ladder probe question service 2971, the automated interviewer server 2965 (
As indicated above the ladder probe question service 2971 will word a probe question so that a prior interviewee response is used to generate a probe question for obtaining an interviewee response for an adjacent as yet unfilled in ladder level. In addition, as also indicated above, for most interview session states, the preferred direction is to move “up” the ladder (i.e., from attribute to functional consequence, or from functional consequence to psychosocial consequence, or from psychosocial consequence to value). A flowchart showing the steps performed by the StrEAM automated interview subsystem (StrEAM*Robot) 2913 when generating questions for a given ladder is shown in
It should be noted that during an automated ladder question dialog, the robot interviewer server 2965 prevents respondent interactions from going on indefinitely through the use of counters and elapsed timers. For the sake of clarity only one instance of this type of safeguard is depicted in
Note that prior to classifying an interviewee's response to an interview question, the robot interviewer server 2965 will first examine the input from the respondent to see if it is a request for help—in which case appropriate messages are sent and the ladder dialog is resumed.
The StrEAM*Robot subsystem 2913 components can also provide assistance to human interviewers when conducting manual StrEAM*Interview interviews. In such a case, the interviewer desktop web application 2934 can be configured to connect to the ladder element classification service 2966 and the ladder probe question service 2971 as shown in
In the context of a manual StrEAM*Interview, the StrEAM*Robot components only provide recommendations to the human interviewer. Therefore the logic utilizing the StrEAM*Robot components differs from that during automated interviews. A flowchart showing the steps performed when using the StrEAM*Robot components during a manual interview is depicted in
It should be noted that since the automated interview subsystem 2913 components may, in one embodiment, only provide recommendations, such components can be used with manual StrEAM*Interviews whether or not the ladder element classifiers 4514 are fully trained. This allows the ladder element classifiers 4514 to be used even during pilot interviews that are performed by an interviewer.
The StrEAM*Robot subsystem 2913 as described hereinabove is readily extended to make use of text-to-voice technologies that can mimic a human interviewer further, e.g., via synthesized speech. Accordingly, a speech synthesizer may be provided with input text to “verbalize” by the robot interviewer server 2965 based on the text contained in the interview definition data 3110 (
Note that in one alternative embodiment, the automated interviewing tool 4514 may be provided at (e.g., downloaded to) an interviewee's computer. Thus, the automated interviewing tool 4514 may be incorporated into the respondent application 2938 (
Note that the automated interview subsystem 2913 can be incorporated into the market research network server 2904 (
The StrEAM*administration subsystem 2916 provides the basic capabilities to administer a StrEAM research study (one that utilizes StrEAM*Interview subsystem 2908, and the StrEAM*analysis subsystem 2912).
The primary focus of the administrative subsystem are the processes involved in organizing the desired set of study participants (respondents) and coordinating them to be interviewed by StrEAM interviewers. In the general case, these processes are depicted in
An important characteristic of the StrEAM*Administration system is its flexibility in supporting these various workflows which might be used to get to the point of conducting interviews. Study-specific configuration information determines how the workflow for that study will differ (if at all) from the generic process flow shown in
The automation contained in the StrEAM*Administration subsystem 2916 is supported by a special directory structure on the central StrEAM Web Server 2904 (
As
Configuration information, respondent data, interview definitions, etc. that are specific to a market research study are then contained in XML document files in each study directory. Standard names (and naming conventions) are used across StrEAM studies. A summary of the XML documents involved is given in Appendix I hereinbelow.
Referring to
In an embodiment, an AI-driven platform may be used to obtain an in-depth understanding of the decision-making processes for key target segments in the competitive marketplace for the purpose of strategy optimization. The novelty of this concept includes an integrated platform for gathering data (e.g., individual's decision-making data) directly from the individuals (e.g., consumers, voters) in order to understand and quantify the bases of their decision making (e.g., in the post-data gathering analysis). For example, the decision platform framework may involve questioning respondents (e.g., the individuals) to uncover the reasons they hold the views/perceptions they do via an Internet platform (e.g., the market research network server 2904 and/or other embodiments of the StrEAM platform).
A goal of decision strategy analytics is to uncover, classify and quantify the underlying decision structures from a sample of potentially strategy-determining (potential) “customers” that comprise a given market for the purpose of optimizing management decision making Given the efficiency of an embodiment of this artificial intelligence (AI) interviewing platform, very substantial savings in terms of both time and cost result.
In an embodiment, the decision analytics solution may be a computer-administered interview combining (a) a general decision framework based upon means-end theory and (b) strategy-problem-specific research design models that uncover the underlying distinctions that uncover the relevant decision-making processes, with (c) a critically necessary instructional grounding that provides the understanding of the common decision framework so an individual respondent can self-question themselves as to the personal, motivating reasons (functional and psycho-social consequences, linked with value-based insights) underlying their decision making.
In combination with a formal view of the goals of embodiments of this software platform as discussed above, there are academic literatures that serve as the foundation of this solution. The following references are fully incorporated by reference as additional information related to the present disclosure.
In essence, the AI-driven decision strategy analytics platform goal can be viewed as an extension of 35 years of work in this research discipline, which has been pre-tested in the political space over the last few election cycles. Analysis of the decision structures initially involves a combination of structured text analytics as well as multiple internal reliability data assessment methodologies. The strategic insights are derived from a combination of decision-based customer syntax and statistical summaries of decision equities and disequities underlying a decision segmentation analysis, which is contrasted across multiple strategy-framing constructs, such as “brand” usage and loyalty, for the competitive set. This level of in-depth understanding of the competitive marketplace in a large scale context provides the foundation for optimizing “brand” positioning and communication strategy when combined with management insight.
The developmental stages of the AI-driven decision strategy analytics platform are as follows:
1. Self-coding format for the decision structures forthcoming from the laddering methodology that can serve as basis to assess and contrast the AI vs. self coding.
2. Implementation of respondent-based video used to (a) summarize decision structure, (b) explain highest level motive (personal value—and permit self coding) and (c) explore associations and examples with a focus on potential illustrative metaphors.
3. After the foundation of a lexicon for a given category is firmly in place, and the correspondence of the AI coding to self-coding passes a given threshold, movement to complete AI with options for self-coding resolution if needed.
Further, this computer-based interviewing may be conducted in a very engaging, highly involving context, including the use of video graphics-based “interviewing interactions.”
In an exemplary embodiment of an AI-driven decision strategy analytics platform, the following standards would be preferred. The respondents are prescreened for their attentiveness and thoughtfulness, and a database would be created for future research (with the possibility of developing a consumer or voter panel). Within the interview interface, avatar(s) would be used to involve the respondent in the in-depth questions being asked. AI will be utilized to classify the respondent's verbatim responses; this would lead to optimal framing of the next questions, as well as serving as a basis to compute reliability of self-coded responses. For time and/or effort efficiency, the interviewing set-up (design) would be streamlined, as will the analysis of the resulting data, so as a complete project could be completed within a certain timeframe (e.g., a week for a sample size of around 1000 respondents). This may include using several analytical shortcuts to identify non-qualifying respondents due to their inconsistencies (in terms of their responses and coding).
In an embodiment, the decision strategy analytics platform 6701 may include the study design subsystem 6710, the study interview subsystem 6720, and the study analysis subsystem 6730. The study design subsystem 6710 may includes the study management module 6711, questions file management module 6712, and the pre-test module 6713. The study interview subsystem 6720 may include the avatar module 6721, the interview management module 6722, the self-coding module 6723, and the AI analysis module 6724. The study analysis subsystem 6730 may include the analysis management module 6731, the segmentation module 6733, and the analysis review/editing module 6732. The decision strategy analytics platform 6701 may further includes the design component database 6793, the studies database 6791, the questions database 6792, the interview database 6793, and the analysis database 6794. The decision strategy analytics platform 6701 may be accessible by an access interface 6750 (e.g., a local terminal or through a network).
In a preferred embodiment, the AI-driven decision strategy analytics process 6800 includes three phases (components): the study design setup phase 6820, the study response phase 6840, and the study analysis phase 6860.
The study design setup phase 6820 is configured for a study designer (e.g., by a study designer or administrator through the study design setup interface 6751) to design and test a complete research protocol (e.g., by study design subsystem 6710 starting with the study setup step 6821 through the study management module 6711) to load onto the website where the self-administered, AI-based interviewing is operationalized (e.g., by the study interview subsystem 6720). This means a data file of types of questions that can be edited and inserted into a new interview (e.g., in the develop question file step 6822). And, this is a file of old interviews as well (e.g., in the study database 6791 and may be edited by the edit study step 6823). The ability to detail branching and termination rules is also provided (e.g., by the order questions step 6824 through the questions file management module 6712). The ability to pre-test all combinations of the rules (with viewing) is included, as well as which avatar is used (e.g., in the pre-test step 6825 through the pre-test module 6713). The resulting designed study may be stored in the studies database 6791 (e.g., through the output setup file step 6826). Basically, this is a self-contained module that when completed and tested can be downloaded into the website for the access by respondent. Also noteworthy is the specification of security rules and methods.
The study interview phase 6840 is where the respondent goes for the interview (e.g., through the respondent interface 6752). The ability to check on the status by key variable may be included so the researcher can evaluate progress against specific sampling criteria (e.g., by the studying administrator or analyst accessing the interview subsystem 6720 during an interview of the respondent or through stored session of the interview stored in the interview database 6793).
The third component is the study analysis phase 6860. A limited number of standard analyses may be pre-coded for expediency, including the 3- and/or 4-mode combination of codes model (for identifying and quantifying decision segments) (e.g., in the multi-dimensional segmentation step 6863).
In an embodiment, each study may be setup by the study design or administrator (e.g., of an organization for certain study interviewing respondents). Through the study design setup interface 6751, the study management 6711 may be accessed to setup the study using the study setup step 6821 or the edit study step 6823.
A typical study may be designed with the intent to not exceed 30 minutes for a respondent of the interview of the study, and includes some or all of the following sections.
Under the study setup step 6821, the study design may further define security protocols on how to restrict access to the interview to the selected respondent (e.g., password protection, unique hyperlink activation, security questions) and timer feature (limiting the length of the interview. Under the edit study step 6823, the study design has the further ability to use a prior study as a base for the new design (e.g., stored in the studies database 6791).
In an embodiment, the study management module 6711 used in the study setup step 6821 includes options and functions for creating a new study, as outlined below:
In an embodiment, the study management module 6711 used in the edit study step 6823 includes options and functions for reviewing and/or editing a prior “named” study (e.g., choosing a section of the study (or reviewing, editing, or printing) with drag and drop support, having the ability to review questions by file or section and the ability to edit and save or edit and create new ones).
Through the question file management module 6712 (which may be accessed through the studying management module 6711 in the study setup step 6821 or the edit study step 6823), the various questions (e.g., demographics/usage 6902, decisions examples 6903 or 6904, ladders 6905, 6905, or 6909, and wafer/reliability 6906 or 6908) may be created, developed, edited, or selected (e.g., in the develop question file step 6822 or order questions step 6824) and are stored in and accessed from the questions database 6792.
In an embodiment, questions may be pre-designed and selected for the study or may be specifically developed for the study. When a question that will be used is located, the question may be further edited in that format (e.g., copy and rename, save new and old). The question may then be copied as a new question and move to new study design. In an embodiment, the drag and drop method may be used as the user interface.
In the order questions step 6824, the order of one or more questions in the new study may be ordered (within sections of the study or in the overall study) by randomization, branch to separate questions, skip questions depending upon response, have the order stay fixed, a combination of the above, or ordered by other arrangements. In an exemplary ordering in a section, a section may contain one or more questions in a specific order. Depending on the answer (answer code) given by the respondent, the next question that is presented is based on the designed ordering (e.g., branching to another question or possibility to skip to another place, if an answer code is out of bounds of an accepted answer, the questioning for this section may terminate).
In the pre-test step 6825, various and all possible combinations of the ordering of the questions may be reviewed. Here, the study design subsystem 6710 may run through the various and all possible sequences. The pre-test may also be run with each screen viewable by the study designer for a given period of time (e.g., 3-6 seconds—including a section code #) to allow the study designer to review in case some changes need to be made. Accordingly, the pre-testing may uncover errors which may be resolved. This involves a system to accomplish this in a very timely basis—including fixing program errors and the ability to reset and test a specific fix or set of fixes.
The ability to set the question presentation order and the ability to randomize within a section is needed. For example, sections of questions can be coded with an ordinal designation (e.g. 1, 2, 3) and within so the question codes could be (1, 2, 2, 2, 3, 3), and within that are separate branching options, e.g., [Q1] 1; [Q2] 2 (if ‘c’ go to Q5); etc.
In an embodiment, a study may be designed to allow administrative access to an administrator or other person to monitor the progress of the respondent as they being and complete the study.
The questions database 6792 is configured to store an inventory of questions (pre-designed as general question or specifically developed for a study) and may include the following:
Brand usage, occasion usage, and brand switching types of questions may be typically used for “framing of” the decision-based ladders, as well as other questions. The ability to branch to other questions dependent on a pre-coded response and skip subsequent questions dependent on a given response.
Reliability (internal consistency) questions may typically involve asking the same questions but using different response formats. A purpose of the reliability questions is to have a separate evaluation screening analysis with to determine “thinking ability” of the potential respondents. The idea is to insert in two places in the design, either (a) the same question with different response formats or (b) the reversal of questions formats. Within each study there will be reliability components (e.g., wafer/reliability questions 6906 and 6908) intended to confirm that those respondents who made it through the screening analysis are “paying attention” to their current task. In an embodiment, if the respondents do not pass the reliability component (pick comparable answers), they would not pass the pre-determined elimination rule and will be directed to the security page stating what they agreed to and the possible reasons they were eliminated. There may be possibly that the respondent would be allowed a chance to continue (e.g., that they agreed to abide by the study and “pay attention” and ended up passing the reliability questions.
Respondent screening will be the focus on the recruiting in terms of the involvement and attention to the task. Even though a respondent has been selected before they get to participate, the design will include several reliability checks. One of the checks is the use of this reliability question. Internal consistency reflecting “respondent quality” translating to “data quality” will be integrated into the design as a basis to eliminate bad data.
For example, consider that the respondent is asked these repeated questions: What brand of laundry detergent do you buy most often? (Most Often Brand); About what percentage of the time do you purchase (Most Often Brand) of laundry detergent?
In another example, the respondent may be given the same percentage question but using different response categories (e.g., the different choices of a and b below)
The respondent should select the range that represents the exact percentage of the answer in order to pass the reliability question. For example, if the answer is 45%, the respondent should select the responses 45%-64% for a and 35-54% for b.
The study design is configured to accommodate at least four types of laddering distinction types (that underlie a basis of a decision process).
The preference type asks why one thing is preferable to another, which means the two things will have to be obtained in an earlier section of the study (e.g., “After getting Most Often ‘brands’: Why do you ‘buy/prefer’ #1 (sss) over #2 (fff)?).
The on-the-margin type needs a graphical scale, which codes for some or all of the respective points. In a unipolar on-the-margin question (e.g., “How satisfied are you with Brand X?”), the respondent selects a rating on a scale. After getting the rating, the respondent may be asked follow-up on-the-margin questions (e.g., “Why not one point lower or higher?”). In a bipolar on-the-margin question (e.g., “What is the likelihood of voting for candidates ‘left’ and ‘right’?”), the respondent selects from a spectrum of one extreme (e.g., “definitely ‘left’”) to the other extreme (e.g., “definitely ‘right’”) with the middle being “undecided.” The follow-up on-the-margin question would reflect the “barrier to movement” (e.g., “Why not one degree more ‘left’ or ‘right’).
The top-of-mind type asks the respondent to provide a “free association” to the topic (e.g., “What is the first thing that comes to mind when I say (brand ggg)”?). The follow-up includes the valence “reason” ladder question which asks the respondent's perception of the “thing that comes to mind” (e.g., “Is that a positive or negative to you?”).
The most important type determines the concept or idea that is the most important to the respondent (e.g., “What is the key ‘aspect’?”). In one embodiment, the concept or chip allocation may come from a pre-determined list. In one implementation, the respondent may (a) be presented with a short paragraph for 2-3 concept statement, (b) rate the concepts on a scale, and (c) if a concept is selected having the most rating, rate the key feature of that concept from a list of a number of elements (e.g., 3-5 elements); if a tie between two concepts having the same rating, the tie must be broken first. From here, the laddering “of why most important” is conducted.
In an embodiment, the study may include an avatar or other human interface elements to help the respondent when conducting the interview (through the avatar module 6721). The avatar is configured to be displayed on the interview and may include interview functions such as introductory instructions, question explanations, engagement of the respondent, and question asking (e.g., synchronized to lip movement and word presentation on the screen with an audible voice).
Other interactive functions include the avatar being integrated to the interactive interface (e.g., respondent interface 6752) and the interview subsystem (e.g., interview subsystem 6720) for presenting questions based on the previous answers (or based on questions ordering as discussed above), training the respondent in decision making, and timing the verbal reinforcements by the avatar (e.g., only say “good job” if the respondent is answering at the correct ladder level or provide a worthy answer).
Under instructing and training the respondent, the avatar may also be used for summarizing overview of the decision steps (repeating intent). For example, several pre-recorded example demos and usage questions may be chosen from. In another example, the avatar may provide the level of thinking tutorial (e.g., providing the respondent of what not to do and notice that the respondent can review the tutorial at any time). In yet another example, the respondent may define the fitness of the clients to the respondents (e.g. letting the respondent know that the system can determine if the respondent is paying attention); in an embodiment, this may be done by tracking eye movements or other biological signals (e.g., heart rate, breath rate, or other signals such as used in a lie detector) in more sensitive applications of a study.
In the design of the avatar under the study setup step 6821, the study designer may choose the instructional video and/or avatar (with options for both examples and the actual study). The avatar form may be chosen (e.g., the preloaded avatar images and voices, the avatar reminders and verbal reward messages).
In an embodiment, third-party avatar software may be used (e.g., through the avatar module 6721) with one or more of the following features: 3D face profile & orientation, Custom eyes & teeth, Background removal, Auto Motion, Auto audio lip-sync or TTS with dialect, Auto audio-driven animation: Talk/Listen, Basic auto motion adjustment, Advanced auto motion, Muscle control/time offset/ping pong/curve & spring/motion blend, and Multiple auto motion. In a preferred embodiment, Avatar software by Reallusion may be used.
In another embodiment, video recording may be used to record (e.g. screen recording of the respondent interface 6752 for the duration of the study and/or recording) for the respondent to read back their ladder.
The interview management module 6722 of the study interview subsystem 6720 is configured to serve the AI-driven decision strategy analytics interview to the respondent through the respondent interface 6752.
At the respondent sign-in step 6841, the respondent signs in to the study interview through the respondent interface 6752 and passes various security checks and/or logins to ensure the respondent has access to the study interview. For example, the respondent may be given a hyperlink to invited survey, which may include deadline date with the hyperlink expiring on the deadline date so that the respondent will not be able to access the survey after the deadline date. In another example, the system may check email, phone number, or other identification information of the respondent that signed in against files or lists of the possible respondent. In yet another example, security password or code may be required by the respondent in order to gain access to the survey, the system may also interrupt or prevent access to the survey after the respondent has incorrectly entered the security password or code (e.g., after access is attempted by the same computer or network address within a certain period of time).
Other information regarding the survey may be displayed to the respondent in the respondent sign-in process 6841, such as the logo of the organization, intellectual property warning (e.g., copyright and patent warnings), information about the system (e.g., survey website) and privacy policy (e.g., regarding watermark on the site if the respondent tries to copy or print the site), and payment information (as needed and verifiable at, e.g., the end of the survey, for payment to the respondent by the organization). In an embodiment, the respondent may have the option to select or unselect one or more options regarding the survey, such as not allowing the respondent's taking of the survey to be “record” (e.g., using screen recording software).
In an embodiment, verbal or video introduction or instructions to the survey may be present by an avatar (e.g., an avatar whose “role is to guide you through the interview, brief interview of kinds of questions, first some background information) through the avatar module 6721 in the avatar and questioning introduction step 6842. The introduction and instructions may further make the respondent feel important in that he has been chosen because of his “thinking ability” and may help to promote a higher quality of response from the respondent.
In a further embodiment, examples may be presented to the respondent (e.g., through the examples step 6843) for examples such as decisions examples 6903 and 6904.
A goal of the AI-driven decision strategy analytics platform is to provide the respondent with self-interview laddering and develop a “decision ladder” for the respective distinction upon which the decision ladder is based. In a general embodiment, the respondent is given a number of interview questions that are laddering distinction types as discussed above. Each answer to a present question may be based upon prior responses given (e.g., answers to questions at previous levels). Each answer is an open-end response, which the respondent self-codes. The AI is used to calculate the probability of each response (e.g., as an accepted response for the given level or other levels) AND that is compared to what the respondent self-codes (e.g., for correctness of the self-code); that is, two checks at each level). A code is put into the file indicating the relative “fit” of the response. Multiple options emerge, including (a) there is a “fit” for the same level and same code; (b) there is “no fit” for the response to the code at the same level; (c) there is a fit for the code at a different level but “no fit” for the same level; and (d) there is “no fit” at all, either the code or the level.
With regards to calculating the probability of each of the responses, the AI may check both the acceptance of the response within the questioned level as well as other levels for self-interviewing laddering (means-end) levels. In an exemplary decision ladder, the respondent may be asked at an attribute level to identify one basic “tangible product” descriptor characteristics (from a list including a complete range of examples of attribute chacteristics). In the next consequences level of the ladder, the respondent may be asked for the primary product delivery reason why the descriptor characteristic (e.g., from the attribute level) is e.g., important (from a list including a complete range of examples for positive (+) benefits and negative (−) avoidance. In the last values (combined psychological and social) level, the respondent may be asked for the personal reason of the two attribute characteristics and delivery-benefit given in the previous two levels (from a list including a complete range of examples of psychological and social values). Here, the respondent who may be unfamiliar with the laddering framework may misinterpret the question and give a response for the incorrect level (e.g., the consequences level).
It is noted that if multiple codes contained in the response or self-coding, the AI should compute a “fit” with each code. For example, it could be that two words from different levels are given for a response (e.g., as discussed in the example above). In such a case, the system may either opt to ask the respondent a tie-breaking question (between the two codes) or to ask the respondent to stay within level. In a further specific example, if an attribute and a functional response are given at the attribute level, the avatar may inform the respondent: “You mentioned 2 levels; let's talk about this level now (attribute)” and the AI presents codes from attribute level. Alternatively, the level lexicon may be trained and reviewed for further definition of unrecognized codes (e.g., on “blank” level as in used in training) In another note, it is usually preferable that the ladder is built by going from a lower level to a higher level; however, it is also available if the ladder is built by going from a higher level to a lower level.
In an embodiment, the respondent may be given timely verbal reinforcement (or negative reinforcement by the avatar (e.g., do not want to say good job if they are answering at wrong levels or answers are unworthy).
The self-interviewing laddering process 7000 starts with the presenting the next ladder question (e.g., a low (attribute) level ladder question) to the respondent step 7001 (in the ladder study step 6844). The respondent answers the presented question with a response in step 7002 (as presented and through the respondent interface 6752).
As discussed above, the laddering questions may include 4 types of laddering questions/judgment types: preference (brands known prior to the questioning), on-the-margin, top-of-mind (with valence), and “most important” preference concept (and “most important” from a list known prior to the questioning). Because “most important” utilizes prior rating, this type of question may be used to break ties for the most important, like chip allocation representing importance, if needed.
In an embodiment, the avatar 7110 overviews the task at hand to the respondent (as discussed above). The distinction (laddering) question 7130 is presented, and the respondent would enter the response 7120. If a graphic scale is used (e.g., for an on-the-margin question), the respondent would provide a subsequent response (e.g., a rating) at the question 7130 (see e.g.,
Once the respondent has satisfactorily input a response to the question, the respondent may select “continue” 7140 to move to the next screen (e.g., the next question) with no option to go backwards. The respondent may also select to view the tutorial again or to view a pop up with various definitions 7160.
It is noted that bipolar scales can also follow this format (e.g., the graphical scale at question 7130 would be a gradation instead of a numeric rating scale (e.g., the middle being neutral between the bipolar extremes, with greater leaning towards one extreme as the chosen gradation leans further away from neutral).
Here, it is also shown that the self coding choices 7150 provided by the AI (e.g., self-coding module 6723) in the self-coding step 6845 (e.g., where the respondent selects the code from a list that best matches the response 7003).
A goal of laddering is to uncover the “higher level” reasons as to why that distinction is important to the respondent. This may be done by asking some form of the “why is that important to you” question. In an exemplary ladder:
Why is “portability” important?→Because “it's easy to carry with me.”
Why is “easy to carry with me” important?→Because “it's always available.”
Why is “always available” important?→Because “you never know when you will need one/feel more secure.”
The laddering questions as discussed above would elicit the laddering response from the respondent, leading to uncovering the “higher level” reasons.
In an embodiment, the self-interview laddering (e.g., steps 6844-6846) follows a general order of events as follows: 1) presenting a judgment question (e.g., a laddering question) to the respondent; 2) receiving the answer from the respondent of the judgment question; 3) matching the answer from the respondent to possible code matches (e.g., by the AI in “AI coding”); 4) presenting a list of the codes (e.g., either a list of the possible code matches found by the AI or other lists of codes or code matches) to the respondent; 5) receiving an answer of a code match selected by the respondent from the list of codes (e.g., “self-coding” by the respondent); and 6) moving to the next question (e.g., judgment question) or completing this section of the self-interview laddering.
As such, in building the self-interviewing ladder 7200, in an embodiment, the respondent is asked the relevant distinction question 7230 for each level, and the respondent provides the response in one of the response 7220A-7220C corresponding to the correct level. For example, in a typical upward ladder, the ladder questioning starting at the lowest level (e.g., response 7220A) and is built upward (e.g., towards response 7220C). In the self-coding step 6845, the respondent is presented a list of probable codes for that level (e.g., self coding choices 7150) and match the response to the summary code for the code levels 7260A-7260C.
Self coding choices 7250A-7250D shows lists of the probable codes for each of the respective code levels 7260A-7260C. In an embodiment, the AI has matched codes based on the respondent's response and has provided a list of coding choices (e.g., self coding choices 7250A) to be used for self coding, based on the respondents answer. In a preferred embodiment, every level's screen should have a definition of that level easily visible.
In an embodiment, after the respondent provides the answer 7220A, and “continue” button 7240 may be pressed to record the answer 7220A. The self-code options (e.g., self coding choices 7250A) may subsequently appear for the respondent to choose the suitable code from the self-code options that the respondent deems suitably matches the answer 7220A that he has provided. In an embodiment, the respondent may also have an option to select an “other” choice as the self-code, if the respondent deems none of the options in the self-code options would match the answer 7220A that he has provided. In the case that “other” is selected, the respondent may be asked to provide a summary word or short code phase as the self-code (e.g., provided in a “Code Assigned” box on the interface). In an embodiment, the question 7230, answers 7220A-7220D, lists of self coding choices 7250A-7250D, and the “continue” button 7240 may be of one or more different colors as displayed on the interface.
In an embodiment, each level (e.g., each of the first to the fourth levels of the self-interviewing laddering example as discussed above with respect to
One example where multiple sub-levels (e.g., 2 levels) are needed at the attribute level is politics. Politics most likely includes more than 15 single attributes (codes). For some applications, it may be that any number over 8 or 9 attributes (codes) is too many (e.g., to be displayed on the lists of self coding choices 7250A-7250D on the respondent interface 6752). As such, the attributes (codes) may be further sub-grouped into various topics. For example, for politics, this type of subject matter may be grouped into the topics of generalized attributes such as Social Issues, Economic Issues, Foreign Policy, Leadership. Each of these have assigned codes that are then used to choose from.
Further regarding the self-interviewing laddering process 7000 with respect to
In an implementation, in the AI analysis step 6846 (through the AI analysis module 6724), the system performs matching of the response from the respondent to all codes across level to obtain the probability of the match both by level and across all. Here, after the respondent has answered prompt from the ladder question, the AI reads and assigns a code based on codes and descriptions entered and presents a list of possible codes for the level. Here, the AI may select the codes on the list based on a preliminary assessment (or “fit” as discussed below) of the answer to the possible codes. Alternatively, the AI may include other codes (e.g., frequently used code for the level) in the list of possible codes. The respondent then self-codes from the list of codes provided by the AI on which one of the codes best represents the meaning of the respondent's response (e.g., step 7003).
In a preferred case, the respondent states the response at the correct level, and the AI agrees. The response will then see a box with all of the codes from that level and pick the correct code again. This means the self-coding is correct. For the AI, in the compare response with the self-code decision 7010, this is the “fit” case and the self-interviewing laddering process 7000 advances to the code level as coded and move to the next ladder level step 7011 (or to end 7099 the self-interviewing laddering process 7000 if the ladder level is the last ladder level).
In a specific case where “other” is chosen (e.g., the “other” input box in the lists of self-coding choices 7250A-7250D), the respondent inputs its own description of the code. The “code” entered at the “other” input box may then be “fitted” by the AI similar to the comparison at the self-code decision 7010 (e.g., for the pre-defined codes), and two or more of the same answer at a level entered on the “other” input box gets added into the level lexicon if “fitted.”
In an embodiment, the AI may use one or more methods of “fitting” the response to the self-code as known now or may be later derived. For example, in one implementation, the one or more text classification approaches as discussed with section 6.3.3 (Text Classification Approaches) or Appendix F (Text Classification Implementation) may be used. In a further implementation, one or more of the self-interviewing laddering process 7000, the decision strategy analytics platform 6701, and the AI-driven decision strategy analytics process 6800 may be implemented by a modification of the StrEAM*Robot Subsystem 2913 as discussed with respect to
In the not so preferred case, the answer and/or code provided by the respondent may contain one or more potential problems. Some possible problems may include: the provided answer not in the list of codes for the current level, the provided answer is not in the list of codes for any level, two or more answers are given, two or more answers are given and one or more is for the current level but one or more is for another level, answer provided by the respondent is invalid (e.g., respondent provided blank answer, no clear answer, or other invalid answers), or other problems. In an embodiment, the potential problems may be generalized as having a “fit” (at the current level) but require clarification as to the best code (e.g., a “fit” or two or more codes) or no “fit” (at the current level) but may have a “fit” for another level.
In the case that the response that the respondent has provided matches two or more possible codes (e.g., the AI found a “fit” for two codes), clarification by the respondent may be needed to select one of the code for the ladder. As such the process may present clarification question to the respondent for the best code 7012, (e.g., asking the respondent: “you said multiple ideas, from this list, which one is the most important to you?’).
In another embodiment, the AI may perform further analysis of the answer as an open-ended response (at the current or another level), taking into account both the level of detail and meanings. Every code may be assigned with a probability (for a level) based upon its probable match to an aspect of the lexicon sub-codes (for that level). As such, a set of rules may be written to match the possible answers, and the AI can build the rules over time. In an embodiment, these probabilities and rules may work in conjunction with the text classification approaches or other “fitting” methods as discussed above (e.g., with respect to section 6.3.3 (Text Classification Approaches) or Appendix F (Text Classification Implementation)). It is noted that various synonyms (e.g., 3 to 4 synonyms) may be needed to be grouped with every code. However, generally no overlapping meaning between code statements may exist.
In one example, if one of two codes matches the respondent's answer with a clearly higher probability than the other, the AI may code the answer as a “fit” for the code with the higher probability (e.g., in step 7011). In another example, if two codes match the respondent's answer with a high probability, the AI may then ask the respondent which is the most important (if they are both appropriate for the level) (e.g., step 7012). For example, if the highest probability for the level matches the respondent's open-end answer and also matches another code option, the AI may present to the respondent the question: “You said multiple things but at this level, which one best fits?”
However, if the respondent's answer does not “fit” a code at the current level, the process 7000 may further evaluate the probability that the response “fit” codes of other levels (e.g., the AI cannot match the response with any significant probability for codes for the current level). One rationale for this may be that the respondent is unfamiliar with the coding process.
In an embodiment, the AI may perform the evaluation for the response matching the code in another level similar to the comparison of the response with the self-code for the present level (e.g., step 7010). If it is determined that there is a high probability of a match (e.g., the response matching a code for another level), it is likely that the respondent has incorrectly coded the response and the respondent is asked to stay on the currently level 7022. If it is determined that there is a low probability of a match (e.g., the response does not match any code at any level for this study), it may be that the response has not been considered by the study and the code may be added to the level lexicon 7021 or may be otherwise recorded (“flagged”) for review (e.g., by analyst 6753).
It is noted that the respondent may have the ability to review the tutorial during the self-interviewing coding process 7000 to facilitate the respondent's knowledge in the coding (e.g., if the respondent is having difficulty to successfully self-code).
In an embodiment, summary statistics may be recorded on the number of correctly coded responses and/or ladders (e.g., for review by the analyst 6753). The summary statistics may include a matrix for each ladder: Predicted (for rows) x Coded (columns).
In the status check 6847, the respondent may be asked to perform a task assessment (e.g., task assessment 6910). It is noted that this task assessment may further be a self-interviewing ladder as discussed above. In an embodiment, the respondent may be asked:
In an embodiment, the respondent may also be asked payment information (e.g., for a paid survey), such as for electronic payment (e.g., Paypal) or through other payment methods.
In an embodiment, the studying analysis 6860 may be performed by the studying analysis subsystem 6730 (e.g., by the analyst through the analysis interface 6753). The study analysis 6860 may start with opening the data file 6861 (e.g., the files in the interview database 6793 containing an interview or an aggregate of the interviews, through the analysis management module 6731).
In the coding review or AI-summary analysis/editing 6862, various quality assessment of the data may be performed through the analysis review/editing module 6732. For the reliability question, it may be determined the rule for termination and scoring summary code by assessing different levels' reliability. For example, if there are two questions, is it 1 or 2 that is consistent to be included in the sample.
For the quality of the ladders (e.g., the self-coded ladders), the consistency of the self-coding (e.g., the “correctness” of the self-coding of the respondent) may be summarized (e.g., the “same” vs. “different” of the self-coding and the AI analysis may be summarized by each level of each ladder). For example, the number of “correct” self-coding may be summarized for each individual respondent by each ladder (e.g., number of “same”). In another example, sample changes on a given differing levels may be assessed for data exclusion on the basis of data quality. In another example, a resulting sample may be determined in the case that ladders are deleted if “not correct” (e.g., off by one or more errors). In yet another example, certain “data-cleaning” rules may be determined (e.g., based on the assessment of the samples of the AI analysis).
In a further embodiment, a summary of the other codes (e.g., unselected codes by the respondents) may be further included for review and verification.
The data summary may be arranged in the form of a pivot table with multiple labeled cross-tab. The code summary may be arranged by questions, by other key combinations of variables such as brands most often used, age of respondent, or by other combinations, by ladder, and/or by code. The ordered data summary may further include percentages of multi-way combinations (e.g., three-way combinations) and by clustering (grouping).
The segmentation module 6733 may further performing multi-dimensional segmentation on the data summary (e.g., multi-dimensional segmentation 6863).
For example, one decision segmentation question may ask: how many people (e.g., respondent) are following the same pathway (e.g., ladder). One basis is that if the AI computes all possible pathways that respondents may take to make a decision, the numbers may be ordered on the basis of frequencies (e.g., how often a pathway occurs). One goal of segmentation analysis is to determine the segments (e.g., what are the pathways people choose when they go through the laddering process).
The output of the analysis may be stored in the analysis database 6794 in step 6864 for further processing.
While various embodiments of the present invention have been described in detail, it is apparent that modifications and adaptations of those embodiments will occur to those skilled in the art. It is to be expressly understood, however, that modifications and adaptations are within the scope of the present invention, as set forth in the following claims.
A description of the language used for defining an interview is given below. Overall, the definition of a StrEAM interview is contained within an interview definition element which may be described as follows:
A StrEAM-Interview-Definition in turn contains three (3) container sub-elements:
Each of the above three XML interview definition elements is described hereinbelow.
There are seven possible elements in the header section of a StrEAM*Interview Definition data 3110:
A StrEAM*Interview session includes a series of “topics”. Each topic may include of some form of survey question or just some information to be displayed to the interview respondent. There are twenty-one different types of “topics” that may be included in a interview definition. Generally these may appear in any order, and with any desired frequency. The exceptions are the <opening-information> and <closing-information> elements which are used at the beginning and end (respectively) of every interview (only one occurrence of each).
The available types of StrEAM*Interview topics are defined by the following XML elements:
There are several sub-elements that are common to all StrEAM*Interview topics. These are:
Several XML elements of the interview definition language exist to define interview topics that are not questions, but just display information for the respondent. No response is required from the respondent (though a conversation may take place through the instant messaging windows). These elements are defined below:
All other interview definition topic elements define interview questions that have a built-in expectation of a response from the respondent. As such, each of these puts the interviewer's interview application 2934 into a mode expecting a response from the respondent, such that the response can be recorded when it is received.
The interview question tonics break down into four (4) basic categories:
Several types of “simple”, open-ended questions are currently allowed. Each simply elicits an open-ended response from the respondent, which is typically saved verbatim by the interviewer. Each of the simple question elements has an identical structure:
All simple questions may include the following sub-elements:
A StrEAM Radio Question is a multiple choice question, where the respondent is presented with several options and is required to select one (and only one).
Each Radio Question element can contain the following sub-elements:
Chip Allocation questions enable a StrEAM interviewer to present a respondent with a series of options to elicit a response about the relative weighting of those options in response to some question. Currently this is done by presenting a set of 10 chips and the respondent distributes those 10 chips across the options presented.
Chip Allocation elements contain the following sub-elements
The interview definition language supports several forms of Ladder Questions. They all define an interaction between the respondent and interviewer in which an answer in the form of a multi-level “ladder” is elicited in response to some question.
All of these ladder type question elements can have sub-elements as follows:
The following is a description of the data format for the interview result files 3118. The results of a single interview session are contained within a <StrEAM-Interview-Session> element that can be described as follows:
Note that when interview results are approved and promoted for subsequent analysis (e.g., via the StrEAM analysis subsystem 2912), multiple <StrEAM-Interview-Session> elements are, in one embodiment, combined together into a single StrEAM*analysis model database 2950 (
An <StrEAM-Interview-Session> element in turn contains three main container elements:
In one embodiment, there are eleven possible sub-elements elements in an interview result <header> element.
The results of each interview session conclude with a <footer> element that reports information available at the end of the interview session. The <footer> element includes the following elements.
The <result> element includes all of the answers recorded for questions in a corresponding interview session. Each <result> element includes a series of <answer> sub-elements as described below:
Each <answer> element may contain the following sub-elements.
As befits the name, a <simple-response> element contains one (and only one) sub-element:
A<radio-response> element contains simply one (and only one) sub-element that indicates the choice made by the respondent:
A<chip-allocation> element will contain one or more <allocation> sub-elements. Where there is an <allocation> sub-element for each possible option that the respondent may allocate chips to.
The result for a ladder question consists of a <ladder-response> element that contains individual <ladder-element> sub-elements.
The StrEAM*analysis configuration database 2980 may be a plain text file, in one embodiment, using an XML-based syntax. This syntax defines each of the StrEAM*analysis configuration items described in the Analysis Configuration Database section hereinabove. The format of such a configuration database (file) 2980 is described below.
The <StrEAM-Analysis-Configuration> element in turn can contain the following sub-elements:
There are six (6) possible sub-elements elements in a StrEAM*Analysis Configuration <header> element.
Code Sets 3942 (also denoted Code Set Elements)
A<code-set> element is a container for a series of codes to be used to quantify the responses to a qualitative StrEAM*Interview question. There must be one <code-set> defined for each ladder level (attribute, functional consequence, psychosocial consequence, and value). There can also be a <code-set> defined for any other qualitative (open-ended) questions as well.
Within a <code-set> element, there may be any number of <code> sub-elements. There always should be at least one.
Then each <code> definition will include the following sub-elements:
There may be any number of <question-group> elements 3954 in an analysis configuration database (file) 2980, as defined below. Note that question groups 3954 are not mutually exclusive. Interview questions may appear in any number of question groups 3954. In order for a question to be accessed during analysis, it must appear in at least one question group 3954.
Within each <question-group> 3954 is the following sub-elements:
Data filters 3958 are used to select a subset of an analysis model database 2950 for examination. Such data filters 3958 identify certain questions (and their answers) that will be used to select interviews from the analysis model database 2950.
Each data filter 3958 definition (except for the special “All Interviews” filter) will include the following sub-elements:
Each <question> element in turn will contain the following sub-elements:
Each <mention-report> element hereinbelow defines a StrEAM*analysis Code Mention report 3986 as follows:
A<mention-report> definition will then include the following sub-elements:
Each <column> sub-element, in turn, contains the following elements:
The <decision-modeling> section may contain any of the elements listed below.
The <export-list> element defines a single, named, “Export List” item. An Export List 3962 is used to determine the data to be exported in tools like the export to the statistical package, SPSS®. Any number of Export List items can be defined.
Each <export-list> element will contain the following sub-elements:
Part of the data used during StrEAM*analysis subsystem 2912 processing is contained in an analysis model database 2950 (
The result of each StrEAM*analysis decision segmentation analysis is stored in a StrEAM*analysis model database 2950 in the form of a decision model 3944 (
Below is a detailed description of the XML-based syntax contained in a StrEAM*analysis Model database 2950.
An analysis model element in turn contains the following sub-elements:
There are nine (9) possible sub-elements elements in a StrEAM*analysis model <header> element.
The results of StrEAM*analysis decision segmentation analysis (DSA, cf. Definitions and Descriptions of Terms section above) are stored in a StrEAM*analysis model database 2950 in the form of a decision model(s) 3944 (
The <decision-model> element contains the following sub-elements:
The <solution-map> sub element contains information detailing a specific solution (map of cluster chains). The <solution-map> element is described below. Note that the <initial-chain-map> is of the same form as the <solution-map> element, however it does not represent an actual solution, but rather some important interim results that might be of interest later.
Each solution map XML element (of either type) in turn contains a series of <cluster-chain> elements. These are the ladder code sequences (pseudo-ladders) that represent a particular solution to the Decision Segmentation Analysis. In the case of an <initial-chain-map> element, the cluster chains are not a solution, but list of “seed” chains used to find the solutions maps 3940 in the decision model 3944.
A<cluster-chain> element is made up of a list of codes. These are defined in <code> sub-elements. There can be up to six (6)<code> elements in a <cluster-chain>. For solutions generated by StrEAM*analysis Decision Segmentation, there will always be at least four (4) codes (one for each ladder level).
An analysis model 2950 contains <interview-session> elements which hold the interview results used for analysis (i.e., the interview session data 3932). As noted earlier, most of interview session data 3932 is exactly as described in the StrEAM*Interview Result XML documentation.
The difference, in an Analysis Model, is that the ladder results contained in Interview Sessions may also contain the results of Decision Segmentation Analysis. Such results are represented in the form of <ladder-mapping> elements. There will be one <ladder-mapping> element for each decision model 3944 that a ladder result participates in1.
Therefore, each <ladder-response> element may contain one or more <ladder-mapping> elements as defined below:
Within each <ladder-mapping> element is a series of individual <mapping> sub-elements. These specify how the ladder was mapped in the Solution Maps for the Decision Model 3944. There are as many <mapping> sub-elements as there are Solution Maps in the Decision Model (note that this does NOT include the Initial Chain Map).
1Participation is a result of a ladder response qualifying under the Question Group used and the Interview Session itself qualifying under the Data Filter used.
2The notion of “assigning” a ladder to a Cluster Chain is defined elsewhere in the StrEAM*Analysis documentation.
Abstractly, the probability model for a classifier is a conditional model
p(C|F1, . . . ,Fn)
over a dependent class variable C with a small number of outcomes or classes, conditional on several feature variables F1 through Fn. The problem is that if the number of features n is large or when a feature can take on a large number of values, then basing such a model on probability tables is infeasible. We therefore reformulate the model to make it more tractable.
Using Bayes' theorem, we write
In practice we are only interested in the numerator of that fraction, since the denominator does not depend on C and the values of the features Fi are given, so that the denominator is effectively constant. The numerator is equivalent to the joint probability model
p(C,F1, . . . ,Fn)
which can be rewritten as follows, using repeated applications of the definition of conditional probability:
and so forth. Now the “naive” conditional independence assumptions come into play: assume that each feature Fi is conditionally independent of every other feature Fj for j≠i. This means that
p(Fi|C,Fj)=p(Fi|C)
and so the joint model can be expressed as
This means that under the above independence assumptions, the conditional distribution over the class variable C can be expressed like this:
where Z is a scaling factor dependent only on F1, . . . , Fn, i.e., a constant if the values of the feature variables are known.
Models of this form are much more manageable, since they factor into a so-called class prior p(C) and independent probability distributions p(Fi|C). If there are k classes and if a model for p(Fi) can be expressed in terms of r parameters, then the corresponding naive Bayes model has (k−1)+n r k parameters. In practice, often k=2 (binary classification) and r=1 (Bernoulli variables as features) are common, and so the total number of parameters of the naive Bayes model is 2n+1, where n is the number of binary features used for prediction.
In a supervised learning setting, one wants to estimate the parameters of the probability model. Because of the independent feature assumption, it suffices to estimate the class prior and the conditional feature models independently, using the method of maximum likelihood, Bayesian inference or other parameter estimation procedures.
Constructing a Classifier from the Probability Model
The discussion so far has derived the independent feature model, that is, the naive Bayes probability model. The naive Bayes classifier combines this model with a decision rule. One common rule is to pick the hypothesis that is most probable; this is known as the maximum a posteriori or MAP decision rule. The corresponding classifier is the function classify defined as follows:
The naive Bayes classifier has several properties that make it surprisingly useful in practice, despite the fact that the far-reaching independence assumptions are often violated. Like all probabilistic classifiers under the MAP decision rule, it arrives at the correct classification as long as the correct class is more probable than any other class; class probabilities do not have to be estimated very well. In other words, the overall classifier is robust enough to ignore serious deficiencies in its underlying naive probability model. Other reasons for the observed success of the naive Bayes classifier are discussed in the literature cited below.
Here is a worked example of naive Bayesian classification to the document classification problem. Consider the problem of classifying documents by their content, for example into spam and non-spam E-mails. Imagine that documents are drawn from a number of classes of documents which can be modeled as sets of words where the (independent) probability that the ith word of a given document occurs in a document from class C can be written as
p(wi|C)
(For this treatment, we simplify things further by assuming that the probability of a word in a document is independent of the length of a document, or that all documents are of the same length).
Then the probability of a given document D, given a class C, is
The question that we desire to answer is: “what is the probability that a given document D belongs to a given class C?”
Now, by their definition,
Bayes' theorem manipulates these into a statement of probability in terms of likelihood.
Assume for the moment that there are only two classes, S and S.
Using the Bayesian result above, we can write:
Dividing one by the other gives:
Which can be re-factored as:
Thus, the probability ratio p(S|D)/p(S|D) can be expressed in terms of a series of likelihood ratios. The actual probability p(S|D) can be easily computed from log (p(S|D)/p(
S|D)) based on the observation that p(S|D)+p(
S|D)=1.
Taking the logarithm of all these ratios, we have:
This technique of “log-likelihood ratios” is a common technique in statistics. In the case of two mutually exclusive alternatives (such as this example), the conversion of a log-likelihood ratio to a probability takes the form of a sigmoid curve: see logit for details.
The classification of natural language text has been studied extensively by academic and commercial researchers. As a result there are numerous software packages that can be used to provide the underlying mechanics for the StrEAM*Robot 2913 text classification processing. These range from generic toolkits for text modeling and manipulation to complete document categorization applications.
A sampling of currently available software (some free, some licensable) is given below. Any of these—or others like these—may be used by one skilled in the art to implement the text classification mechanisms required for the StrEAM*Robot Subsystem 2913 components. The order of their presentation below is not significant.
All data for the administration of StrEAM studies is contained in XML documents, as is mentioned earlier. This form provides the flexibility needed for handling the relatively unstructured nature of much of the information used to administer a market research study. The sections below describe these XML documents in detail.
All (7) Screening Questionnaire Topic types (<information-topic>, <text-question>, <date-question>, <radio-question>, <droplist-question>, <checkbox-question>, and <combobox-question>) may have the following attributes (none of these are required).
All (7) Screening Questionnaire Topic types (<information-topic>, <text-question>, <date-question>, <radio-question>, <droplist-question>, <checkbox-question>, and <combobox-question>) may have the following sub-elements.
The (6) Question Topic types (<text-question>, <date-question>, <radio-question>, <droplist-question>, <checkbox-question>, and <combobox-question>) may have the following attributes.
For the four (4) Question Topic types that present the respondent with a list of options, the following attribute may be used:
The (6) Question Topic types (<text-question>, <date-question>, <radio-question>, <droplist-question>, <checkbox-question>, and <combobox-question>) may have the following sub-elements.
Both the Text Question Topic type (<text-question>) and the Combobox Question Topic type may include the following sub-elements:
Date Question Topics (<date-question>) may include the following sub-elements
Each of the four (4) List Question Topic types (<radio-question>, <droplist-question>, <checkbox-question>, and <combobox-question>) MUST have a “choices” sub-element as defined below, which will contain the possible answers for the question. In addition, these topics may include one or more “drop-choice” sub-elements that enable the answers to previous questions to constrain the available choices
Configuration information, respondent data, interview definitions, etc. that are specific to a market research study are then contained in XML document files in each study directory. Standard names (and naming conventions) are used across StrEAM studies. A summary of the XML documents involved is given in the table below.
The present application is a continuation-in-part (CIP) of U.S. patent application Ser. No. 14/052,677, filed Oct. 11, 2013, which is a continuation of U.S. patent application Ser. No. 13/663,407, filed Oct. 29, 2012, which is a divisional of U.S. patent application Ser. No. 11/925,663, filed Oct. 26, 2007, now U.S. Pat. No. 8,301,482, which is a continuation-in-part of U.S. patent application Ser. No. 10/927,222, filed Aug. 25, 2004, now U.S. Pat. No. 7,769,626, which claims priority from U.S. Provisional Patent Application No. 60/497,882, filed Aug. 25, 2003; each of the above-identified applications is fully incorporated herein by reference.
| Number | Date | Country | |
|---|---|---|---|
| 60497882 | Aug 2003 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 11925663 | Oct 2007 | US |
| Child | 13663407 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 13663407 | Oct 2012 | US |
| Child | 14052677 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 14052677 | Oct 2013 | US |
| Child | 14743418 | US | |
| Parent | 10927222 | Aug 2004 | US |
| Child | 11925663 | US |
