You are on page 1of 19

International Journal of Market Research Vol.

54 Issue 5

Choosing the right baskets for your eggs


Deriving actionable customer segments using supervised genetic algorithms
Sam Davis
TNS

In the context of key driver analysis in applied customer satisfaction research, the assumption of sample homogeneity (that single models perform adequately over the entirety of a survey sample) can be shown to restrict the value of the insights derived. While latent class regression has been used as a method of circumventing some of these issues, it is proposed that there are major barriers to both uptake and successful practical usage of the technique. Several of these issues are common to any multivariate technique, while others are specific to latent class regression. Following an examination of these issues, we introduce an alternative technique for deriving discrete latent classes, using a combination of genetic algorithms and (bivariate) correlations. This paper concludes that the proposed approach outperforms latent class regression in its ability to deliver action-orientated insights, and is better placed to assist marketers facing real-world research questions and datasets.

Introduction and theoretical perspectives A common issue facing market researchers is determining the importance of a particular variable, or set of variables, in influencing another: 'What drives satisfaction?' or 'How do I improve my brand image?' are particularly common questions. The analytic techniques used to answer these questions can be grouped under the umbrella term of key driver analysis (hereafter, KDA). KDA can range from simply asking people directly what they think is important, to a range of more complex statistical methods. Direct methods are generally not commonplace in contemporary market research largely

Received (in revised form); 18 April 2012

@ 2012 The Market Research Society DOl: 10.2501/UMR-54-5-689-706

689

Choosing the right baskets for your eggs

because of respondents' propensity to focus on more obvious areas such as price. It is also commonplace for respondents to claim that everything is important (among others. Bacon 2003). There is almost continual debate over the best indirect analytic techniques to answer these sorts of questions, but most arguments do not consider that different people tend to be motivated by different things or, more formally, they take the view that the sample is homogenous. This assumption of sample homogeneity can be limiting. A simple yet powerful illustration is given below. Imagine that we have been asked to determine the factors that most influence the purchasing of digital cameras. We may develop a set of scales designed to measure performance on areas that are thought to influence purchasing, and then attempt to determine their influence statistically. Three areas are tested: (1) price; (2) user-friendliness; and (3) the ability to customise camera components. We could then use multiple regression (hereafter, MR) to determine both the combined effect of these three areas on purchasing as well as the relative importance of each area. With such a methodology we may well find that we achieve only a poor ability to predict purchasing and that each of the three areas is a relatively weak driver. It is not uncommon to report these results as-is, and a fairly literal translation of this for the end-user of the research would be that they can afford to under-deliver in these areas. Outside of the narrow context of these statistics this is highly unlikely to be the case; the result may have come about at least partly because of a violation of the assumption of sample homogeneity. In reality, it is likely that there are several underlying groups with differing drivers of purchase. Grouping these together as a single analysis sample cancels out these effects, and weakens the analysis and the conclusions we may draw. Latent class regression (hereafter, LCR) is a variation of MR that does not assume sample homogeneity. It works by defining segments according to respondents' motivations, and runs regression models for each of these groups separately (Pinnell 2003). In the presence of underlying sample heterogeneity, LCR offers advantages over single-sample models, yet seems not to have come into widespread use. This paper asserts that this is likely to be a result of several barriers: budgetary constraints - data analysis can be a costly business

690

International Journal of Market Research Vol. 54 Issue 5

relatively low awareness of these techniques compared to single-sample methods LCR tends to perform poorly in delivering a practically useful framework a disconnect between the textbook data we often see in theoretical literature on this topic and the realities of live survey data.

While we argue that tbe objective of LCR is indeed conceptually imperative in many instances, we also contend tbat the application of LCR raises some concerns. This paper examines tbe issues associated with applied usage of LCR, and sbows how it may potentially lead to sub-optimal conclusions. Two further assertions are made, as follows. 1. Subjectivity is critical in this area - the expertise of the researcher counts a great deal. 2. In tbis field, multivariate analysis methods themselves may not be best suited to the research problem and/or data. Once the case for these claims has been established, an alternative method tbat upbolds the principles of LCR yet seems to offer more actionable results will be introduced. Before continuing, it is important to note that this paper is not intended as a statistical discussion of this area, but is focused on applied usage for research practitioners and users. A basic understanding of the principles of correlation and regression-based metbods is assumed. Multiple regression in customer satisfaction research
The dual positioning of multiple regression

Tbe objective of MR is to understand bow well a set of predictors influences a single variable (usually one that is linked to a critical commercial outcome), and in particular how important tbese predictors are in relation to one another. Any reader of introductory, or not so introductory, texts discussing analytics in market research would assume tbat MR is not only among our most powerful tools, but is also used with a great deal of frequency. Tbis paper holds that this is not the case. MR suffers from some well-establisbed shortcomings when brought to bear on real-world

691

Choosing the right baskets for your eggs

research problems and, in our experience, its use is on the decline as a result. Consequently it has given way to a range of more sophisticated techniques, most of which remain related to MR, however. We shall now turn to a brief discussion of these shortcomings and describe how commonly applied statistical remedies to counteract them fail to do so because these are not truly statistical in nature. Instead, these tend to be a result of the divide between textbook statistics and the real-world properties of customer satisfaction survey data. Grapentine (1997) notes that textbook examples in the area tend to offer very little in the way of a meaningful discussion of applied issues, leaving practitioners ill-equipped to handle these. Multicollinearity and the particular dangers of spurious multicollinearity Multicollinearity is defined as the scenario where the variables we are using to predict an outcome are, themselves, related to each other (among many others, Tabachnik & Fidell 2001). Let's imagine we were testing the combined effect of age and length of time in tertiary education on intelligence test performance among university students. MR will not work adequately here if our objective is to understand whether age or duration is the more important factor. This is simply because these are so interrelated that the influence of each cannot be accurately divided between them. This specific example is in contrast to many of the multicollinearity problems we face in applied customer satisfaction research because the relationship between these predictors is logically necessary, albeit imperfect. Necessary relationships seldom exist in customer satisfaction data: it is typical to observe very high levels of multicollinearity even where there is not a logical relationship. This is often the result of the 'halo effect' of a product or brand - i.e. these are generally just good or bad across a wide range of areas we are measuring. This issue is compounded by research users' need for a level of granularity that a survey respondent struggles to provide. This paper will refer to this as spurious multicollinearity and contends that this is a special (applied) case of the problem that statistical interventions cannot solve. The example in Table 1 shows a correlation matrix between variables from a real-life study examining customer satisfaction with an organisation's website. The high level of interrelatedness between these four variables essentially means that there is effectively one statistical construct underpinning these

692

International Journal of Market Research Vol. 54 Issue 5

Table 1

Illustration of spurious multicollinearity - website satisfaction: Pearson correlation Ease of navigating Functionality website of website 1 0.77 0.77 1 0.72 0.48 0.72 0.59 Speed of website 0.72 0.72 1 0.67 Reliability of website 0.48 0.59 0.67 1

Ease of navigating website Functionality of website Speed of website Reliability of website

four variables, and that all we have really been able to measure is that a website is perceived by respondents as either good or bad. This does not mean, of course, tbat these factors are unimportant in defining wbat good or bad means. We must also note here that, wbile tbese relationships make sense, they are certainly not necessary. 'Reliability' as it is defined here is intended to mean that the website does not crash. Clearly tbis has no necessary relationship with tbe ease of navigating the website in the way that the research user was intending - essentially tbat various components of the website were easy to find while the website was functioning. It follows, tben, tbat tbe end user will require a separate understanding of these issues. When we correlate these individual variables against an overall satisfaction metric we would expect to find little to choose between them, as is indeed tbe case (Table 2).
Table 2 Flat correlations emerging from multicollinear predictors: Pearson correlation Overall company satisfaction measure Ease of navigating website Functionality of website Speed of website Reliability of website 0.40 0.44 0.53 0.42

These correlations suggest tbat all tbe variables in tbe analysis have a moderate influence on satisfaction. Of course it would be a mistake to presume a causal relationsbip. A regression analysis paints a very different picture. Tbe standardised regression coefficients are given in Table 3. It is clear from tbe regression coefficients tbat Speed of Website is far and away tbe most important factor. We know, however, that tbis extremity is not the case if we simply consider the correlations. In addition, a very small change in tbe correlations between any of tbe predictors and tbe criterion variable may lead to a very different regression result.

693

Choosing the right baskets for your eggs

Table 3

Multiple regression analysis of factors influencing Vkiebsite satisfaction: standardised regression coefficients
Standardised regression coefficients -0.03 0.12 0.40 0.10

(Constant) Ease of navigating v>/ebsite Functionality of v\/ebsite Speed of website Reliability of website

The presence of a negative sign when we know the bivariate relationship is positive is a well-established symptom of a regression model that is affected by multicollinearity (Zhang &c Mahmud 2005). It is also important to note that similarly suffering models do not always exhibit such obvious warning signs. These regression results do seem problematic, as to report these literally would yield conclusions that are misleading (Speed of Website is not the only important factor) and of hmited use to an end-user or client. Various statistical solutions have attempted to address this. Kruskal's Relative Importance (Kruskal 1987) and Shapley-Value Regression have received a great deal of press in recent years, and are seemingly being afforded a level of 'holy grail' status. A full treatment of these techniques is beyond the scope of this paper, but in essence they attempt to partition importance scores more accurately across the predictors in the presence of multicollinearity. This does, in itself, beg the question as to how 'accurately' can be defined in this instance. When using these techniques, the variable with the highest correlation with the criterion variable tends to remain inflated - so in essence these methods do not offer an improvement over MR from this perspective. Although coming at this from a slightly different perspective, Grapentine (2010) offers a critique of these methods concluding they have become an 'easy option' and are fundamentally ill-considered. There may, arguably, be a temptation for some clients (and agency-side researchers) to prefer inflated importance scores as these provide more interesting results. This paper contends that this can be problematic, especially when these inflated factors are privileged to the detriment of others that remain clearly important in bivariate terms (and with face-validity). Many traditional texts (even those with an applied stance) have recommended the removal of one or more variables to reduce multicollinearity and improve the accuracy of the remaining importance estimates (for example, Tull &c Hawkins 1990).

694

International Journal of Market Research Vol. 54 Issue 5

While there are obviously good statistical reasons for removing variables, this tends to cause concern for clients and end-users. It is difficult to recommend this approach as clients simply need a method that allows them a view of all the areas included in the questionnaire. Another frequently proposed remedy is to use factor analysis to reduce the predictors into a smaller number of independent factors. This will solve the multicollinearity problem statistically but does not fare any better as clients tend not to be able to do much with 'factors'. This is particularly true in the event of spurious multicollinearity as we will be grouping together areas with very little intuitive or practical connection. Again, we should note that the same applied text (TuU & Hawkins 1990) is one of many that advocate this approach. Given the frequency of detecting spurious multicollinearity in customer satisfaction studies, it is clear that practical solutions are required. It seems that proposed solutions are either not sufficiently cognisant of the realities of the practical needs of clients or simply do not offer a superior alternative to conventional MR.
Systematically missing data

A further barrier to the use of MR is that multivariate analysis requires that all cases (usually respondents) contain usable data for all the variables used in a model. This is seldom the case in customer satisfaction studies. A common study design will ask a set of satisfaction or performance questions (generally in the form of Likert scales) across a wide range of respondents' interactions with a product or brand. A single study, for example, may ask respondents to comment on their levels of satisfaction with both the taste of a breakfast cereal as well as how satisfied they were with the customer helpline in the event they needed to raise a complaint. Most customers will have tasted the cereal but few will have called the helpline. As a result we do not have the complete dataset required to perform the analysis. It is notable that, when the helpline has been called, it is likely to have been in order to complain. Again, many proposed solutions do not fully reflect the reality of applied research. Predictably, perhaps, one of these is the removal of variables that contain the most missing values in order to reduce the amount of cases/respondents this affects (Hair et al. 2010). This minimises the amount of sample that needs to be excluded from the analysis in order to have a complete dataset. Given the arguments already raised above, it seems unnecessary to spend too much time on this solution, given that

695

Choosing the right baskets for your eggs

most clients are likely to view this as both unpalatable and simply not useful. An alternative solution is data imputation, i.e. estimating data values for missing cases based on the data provided by respondents with non-missing values. This may work to an adequate degree for missing data problems that are not systematic. This may include, for example, illegible handwriting on the part of a face-to-face interviewer, which may cause gaps across a random range of variables. The common customer satisfaction study design described above is clearly different in kind: we cannot ascribe satisfaction levels with an experience to people who have not had an experience from those who have. In addition, it is likely that the proportion of respondents who have called the customer helpline is so low that we'd be populating the data with more imputations than measured responses. This is obviously bad practice. There is a wealth of literature on the advantages and disadvantages of various imputation methods. These seem to focus on statistical considerations where the objective of various sophisticated imputation techniques is to preserve the structure of the data prior to imputation (Allen &C Rao 2000). Even literature on this topic that has an overtly applied tone suggests that sophisticated data imputation methods are a feasible remedy for non-random missing values (for example. Hair et al. 2010). Our view is that both strategies aimed at solving the issues caused by systematically missing data are unsatisfactory in the context of the realities of applied customer satisfaction research: statistical/analytic methods attempt to solve a problem they cannot be equipped for - simply because the problem itself is not statistical in nature, but one of logic and related more to sampling than statistics.
The status of multivariate versus bivariate methods

It seems likely that the problems caused by spurious multicollinearity and systematically missing data are made worse by both the prevalence and status of multivariate analysis (relative to bivariate methods). It has been noted for some time that researchers are adopting more complex multivariate methods with greater and greater frequency - simply because they are now able to as a result of the computing power that is available (Hair et al. 1998; Zuccaro 2010). One would imagine this trend is likely to continue, if not accelerate, as data analysis software becomes more and more user-friendly. As

696

International Journal of Market Research Vol. 54 Issue 5

multivariate techniques grow in usage, their status becomes increasingly difficult to challenge. While it is undoubtedly true that multivariate analysis offers a wealth of power that bivariate techniques cannot hope to emulate in certain settings, this is certainly not true in all applications. Grapentine (1997) argues that researchers should exercise caution when using bivariate methods to assess the importance of variables in KDA because of the advantage of MR holding additional predictors constant when determining importance. This paper has already shown that there are other practical concerns to consider when using MR that may outweigh the advantages it offers in some circumstances.
Latent class regression

LCR works by finding a set of regression models that maximises the explanatory power of the relationships between the predictors and the criterion variable by finding groups with differing regression coefficients (or importance scores). The end result is that the analysis produces an otherwise conventional MR analysis on these groups. In effect, then, it is a combination of regression modelling and segmentation analysis (Vermunt &C Magidson 2002; Pinnell 2003) with the result that we are left with groups that are motivated in different ways. Generally we find that each of the latent classes/segments identified achieves substantially improved model fit compared to a single-sample model. This is generally considered an indication that the analysis has been successful in uncovering and accounting for heterogeneous samples. Support of this basic premise of LCR has already been noted, but it is now important to discuss potential weaknesses that emerge in the course of the shift from theory to practice. Given that LCR is a multivariate technique, the problems relating to spurious multicollinearity and systematic missing data already raise some serious challenges. In addition two further, more specific, issues raise further questions over the usefulness of the method. These are discussed in more detail in the following sections.
The dangers of over-fitting

In the context of MR the concept of over-fitting usually refers to a scenario where the explanatory power of a model is inflated because of a high number of predictor variables. This is particularly the case with a high ratio of variables relative to the number of cases analysed (Tabachnik &c Fidell 2001).

697

Choosing the right baskets for your eggs

Using 12 random predictor variables in a single-sample MR yielded low levels of model fit (as we would expect). We tben performed LCR using the same variables and were able to detect four latent classes witb very strong model fit: Class 1:R2 = 0.83 Class 2: K" = 0.84 Class 3: ^ = 0.87 Class 4: R^ = 0.84 In effect, we are able to use LCR to create a sense of order and predictability where none can exist in reality. The fact is that we simply will find statistically differentiated groups (witb high R} statistics) because tbe analysis cannot fail to do so. To expect anytbing different would be akin to asking a computer to produce a sbape otber than a triangle out of three adjoined sides. If a similar result occurred in practice it is highly likely that tbe LCR approach is modelling as much noise as it is signal in its effort to increase the explanatory power achieved by a single-sample model. In addition, while excellent levels of explanatory power may be acbieved in tbe single dataset in question, it is almost unfeasible tbat sucb an over-fitted model could be repeated in another sample. Surely, then, tbe underlying reality of the scenario we may infer from these results can be questioned. With substantial marketing and product development budgets on tbe line, how much of the researcher's power should really be surrendered to tbe results produced by tbe analytic software? We sball return to tbis in more detail wben looking at a practical example below.
Differentiation of raw input variables across latent classes

Wben profiling the segments detected using LCR we tend to find very little differentiation between classes wben looking at tbe raw responses to tbe variables used in the model (as opposed to a different pattern of regression coefficients). In otber words, tbere is often limited ability to describe the resulting segments using criteria external to tbe regression model itself. It is accepted tbat not providing for differentiation in the original model variables is not a self-evident disadvantage of LCR, given tbis is not an objective of the analysis. However, from a practitioner's perspective, it is important to note that the author has had to abandon the use of

698

International Journal of Market Research Vol. 54 Issue 5

the technique as a segmentation tool for that very reason. Analysts can (correctly, from a statistical perspective) explain the lack of differentiation in the original variables, but it takes an unusually statistics-driven client to accept a solution with these characteristics. So while we concede that achieving differentiation on additional profiling variables or the raw inputs themselves is not top priority, we would certainly favour a technique that would enable us to achieve this to some degree. Offering an alternative: supervised methods using genetic algorithms Up to this point we have raised several issues regarding the use of LCR in practice. We will now show that a technique using supervised genetic algorithms can account for sample heterogeneity while achieving more practically useful results. The following section will briefly introduce the theory behind genetic algorithms before describing how these can be brought to bear on the problem. Finally, it will be demonstrated, through comparative analysis between methods, that this approach offers some tangible practical advantages over LCR.
A brief introduction to genetic algorithms

Genetic algorithms (hereafter, GAs) were proposed (Holland 1975) as a method of solving optimisation problems based on the principles of natural selection. The GA process is outlined more fully in other texts, (among others Steiner & Hruschka 2003), but a basic overview is given here. A set (or 'population', in GA terminology) of potential solutions to a given problem is generated, and the performance of each is assessed with respect to how well the problem is solved. Following a series of trials (the number of which is specified by the researcher) the 'fittest' solutions are selected to remain in the solution population. These potential solutions then give rise to a new generation of solutions by passing on characteristics that are useful in solving the problem. The process is then repeated until a satisfactory level of performance is reached. In addition to this process of 'adaptation' and 'breeding', solutions may also be allowed to randomly mutate. This ensures that the algorithm does not evolve along a single narrow path that will effectively lead to a sub-optimal solution, given all the potentially better options that may exist.

699

Choosing the right baskets for your eggs

The role of supervision in the process A useful characteristic of GAs is tbat a solution can be aimed not only at improving a particular objective, but constraints can also be placed on specific parameters of the solution. The importance of tbis will be elaborated on more fully below, but it sbould be clear tbat it is tbis ability to constrain tbe results tbat adds a great deal of power to tbe analysis, given tbe concerns raised witb regard to over-fitting in LCR. We should consider a potential objection tbat may stem from tbis point: tbat a researcber should be in the business of finding answers, not co-creating tbem. In tbe experience of tbe author, there are two types of niarket research client: tbe first views statistical analysis as a tool to assist in answering a business question, and the second believes in the objective trutb of 'clean statistics' - or statistical truths that emerge from tbe data that sbould not be tinkered witb at any cost. Tbis paper argues that a balance between tbese two poles is optimal. We sbould neither blindly accept the results generated by analytic software, nor sbould we take so much of a lead role in shaping tbe analysis tbat the results take on an overly 'manufactured' form, for botb practical and ethical reasons. Contemporary literature on tbe topic of market segmentation is particularly sensitive to this spectrum (Bottomley &C Nairn 2004; Stewart-Hunter & Bergentbal 2004). In tbis field, tbere is a sbift towards allowing a researcber to generate tbe most useful results by employing a degree of subjectivity. Otber applications of statistics in market researcb (KDA included) seem to have fallen bebind a little and remain more witbin an 'emergent truth' school of thought. We believe that allowing the researcher to take a supervisory role minimises tbe risk of over-fitted results tbat we have already demonstrated and allows for more usable results. Applying genetic algorithms to account for sample heterogeneity in key driver analysis We bave set up several experiments to assess bow we can combine correlation analysis witb GAs to allow us to identify discrete 'latent classes' while overcoming the issues illustrated above. Due to constraints of space, bowever, we are able to discuss only one example in detail (given further below in tbis paper). Tbe key characteristics of the approacb are as follows: The analysis attempts to divide tbe sample into latent classes where each is differentiated with respect to the patterns of bivariate

700

International Journal of Market Research Vol. 54 Issue 5

relationship (Pearson Product Moment Correlations) between a set of predictors and a criterion variable. This approach, then, is not seeking to maximise the explanatory power of the model. The resulting correlations are constrained such that 'unrealistic' results that may stem from over-fitting in LCR are not tolerated. Specifically, the approach will not allow strong correlations in a group where it seems unreasonable that these will exist in practice or where these are simply unusable for the research client. The model will attempt (as a secondary objective) to maximise differentiation between classes with respect to the raw variables inputted into the model. This allows for a better descriptive understanding of each of the latent classes. The solution is also constrained to ensure that the resulting latent classes are of sufficient size to merit consideration as meaningful groups.

We have also experimented with maximising differentiation on areas of known commercial importance. In this way we can structure the analysis such that variables can take on more or less of a role in shaping the solution. This step is not necessary to the analysis, and the choice to take it depends on where the researcher wishes the analysis to be positioned on the emergent/supervised spectrum.
A practical example using supervised genetic algorithms

We conducted this analysis on data from a customer satisfaction survey concerning drivers of car park satisfaction in the UK with a sample size of = 325. This analysis contained nine predictor variables and each was correlated against value for money as the dependent variable: 1. 2. 3. 4. 5. 6. 7. 8. 9. Sufficient parking spaces available Ability to find a parking space Ease of manoeuvring into a parking space Ease of entry/exit in and out of the car park Navigation around the car park Ease of paying for parking The cleanliness of the car park The level of lighting in the car park Feelings of security while in the car park

701

Choosing the right baskets for your eggs

Tbe model fit yielded by a multiple regression analysis conducted across tbe total sample is weak {R^ = 0.14). A comparison between tbe model fit scores achieved by tbe LCR and tbe GA metbods for defining latent classes are given in Table 4. Note that four classes were used for tbis analysis. Both metbods acbieve groups witb substantial improvement in model fit. Tbis is particularly true for tbe LCR approacb, but recall our earlier argument that a particularly strong model fit may indicate over-fitting. A comparison of tbe two approaches with respect to the correlations between the predictor and criterion variables is given in Table 5. The LCR method yields some results that are immediately counter-intuitive. Class 1 in particular exhibits strong negative correlations for Sufficient Parking Spaces and Ease of Paying. One may plausibly understand tbe latter as a lack of value for money caused by having to pay at all, but we would argue tbat a researcber would have a difficult task convincing tbe client that there is a group of people who would prefer to have fewer parking spaces to choose from. LaLonde (1996) has also detected (and raised concerns over) the presence of strong negative importance scores in bis treatment of LCR. In our view, the strong negative correlation indicates that tbe LCR approach may be over-fitted and seems to offer less actionable groups as a result. Class 3 of the LCR approach has similarities to Class 3 of the GA approacb and is not seen as weak (detailed below), altbougb tbere are some moderate negative correlations. Class 2 tends to bave higber importance scores tban other classes almost across tbe board, wbile Class 4 is moderately positive on environmental factors: cleanliness, lighting and security in addition to ease of paying. Tbe GA approacb has been constrained so as not to tolerate an 'unrealistic' level of negative drivers, so we do not see a scenario similar to Class 1 of the LCR approach. Class 1 (GA) may be understood as interested in being able to park, and while availability of spaces and ease of finding a space are
Table 4 Model fit comparison of latent class regression versus the genetic algorithm approach
Latent class regression 0.14 0.70 0.94 0.66 0.95 Genetic algorithm method 0.14 0.42 0.42 0.50 0.35

Class model fit (R') Total sample 1 2 3 4

702

International Journal of Market Research Vol. 54 Issue 5

Table 5

Bivariate importance scores (Pearson correlations): latent class regression versus genetic algorithm approach Genetic algorithm approach 1 70 0.35 0.35 2 81 0.40 3 98 4 76

Latent class regression Correlation with value for money Class size Sufficient parking spaces available Ability to find a space Ease of manoeuvring into a space Ease of entry/exit into and out of the car park Navigation around the car park Ease of paying The cleanliness of the car park The level of lighting in the car park Feelings of security whilst in the car park 1 81 2 84 3 66 0.07 0.17 4 94 0.06 0.07 0.04 0.27 0.43 0.49

-0.35 0.84 0.15 0.47 0.02 0.15

0.61 -0.05 0.22

0.31 -0.11

0.03 0.17 -0.08 -0.04 0.19 -0.05 -0.49 0.47 -0.02 0.12 0.80 0.02

0.03 0.29 0.60 -0.07 -0.09 0.19 0.22 0.02 -0.04 0.12 0.27 0.09 0.02 0.12 0.14 -0.03 0.04 0.07 0.66 0.29 0.10 0.20 0.03 0 34 0.00 -0.10 0.52

-0.01 0.23 -0.15 0.25 0.01 -0.20

0.26 -0.03

0.25 -0.04 -0.05

Note: Cells with absolute correlations greater than 0.3 are highlighted for ease of illustration.

certainly influencing factors, ease of getting into and out of spaces is the critical single factor. This has clear implications, specifically, for the way that spaces are designed and potentially fitted into existing architecture (e.g. positioning pillars in relation to spaces, size of spaces). While the input variables do not offer this level of interpretation, it is certainly sufficient to warrant greater investigation of the issue. It is also important to note that satisfaction with the ease of getting in and out of spaces is relatively low, so we have the additional benefit of being able to note that there is room for improvement in this area. This level of differentiation with respect to the raw variables is not achieved by the LCR approach. Class 2 (GA) are unconcerned about anything other than having spaces to park in. While Class 2 in the LCR model is also influenced by availability, it is not as distinct in this regard as Class 2 in the GA solution. The action for this segment is clear, of course, but there are likely to be barriers to increasing space in many car parks. This is likely to be a group that it is simply going to be too expensive to please in the short-term, so meeting their needs may be delayed while easier wins are found. Class 3 (GA) are focused on ease of paying, and it is worth noting that this is a group with high levels of satisfaction. While this is an already-satisfied group, they could well be responsive to further innovation in this area. No group is as single-minded regarding this issue in the LCR solution, although Class 3 of the LCR approach is similar.

703

Choosing the right baskets for your eggs

Class 4 of tbe GA approacb (like Class 4 generated by LCR) is most driven by environmental factors of cleanliness, lighting and security. Wbile we have noted that tbe LCR method yields a class where these issues are also important it seems to be to the detriment of tbe importance of other aspects of the car park. We believe that the moderate importance of tbe (potential) hygiene factors of being able to find a parking space adds credibility to the GA version of this group. Our conclusion based on this comparative exercise is that using the GA approacb offers clear interpretive benefits and greater ability to assist decision making.
Applying the method to data containing systematically missing values

As noted above, systematically missing values cause major problems in deriving importance measures in customer satisfaction studies. This is most critically so in the specific context of multivariate analysis. LCR simply cannot handle missing values without eitber removing some combination of variables and/or cases from the analysis, or imputing data from an unrepresentative sub-sample. We have argued that adopting either of tbese approacbes is a cause for concern. Bivariate metbods are not so hampered by missing data. By extension, therefore, neither is the GA methodology described above. We do anticipate an objection to tbe application of tbe GA method to datasets witb systematically missing values: can tbe method guarantee that a latent class placing high importance on a specific area bas a sufficiently low level of missing values witbin tbat variable? While our position is that it must be a strength of tbe approacb tbat tbe analysis is even possible, we shall give this some consideration. The analysis above details a latent class wbere ease of paying was of critical importance. There may, however, be people in tbe sample wbo did not have to pay for a range of possible reasons. As a result, tbese cases would not bave valid data recorded for tbis variable. These people can still, statistically speaking, be represented in tbis latent class as tbeir otber responses will still contribute to tbe overall sbape of the solution. Our grouping still only formally means, 'among those in tbe group wbere X is relevant, X is important'. We appreciate that any substantial number of instances wbere tbis takes place may reduce end-user confidence in tbe resulting solution. Consequently, we have successfully experimented with an additional parameter constraint in tbe GA solution such that a latent class cannot have

704

International Journal of Market Research Vol. 54 Issue 5

a large number of missing cases for those variables where importance is high. While this will decrease the model fit and the levels of differentiation between latent classes, the ability to apply this constraint does seem to be a further benefit of the GA approach.
Conclusion

While this paper supports the principle of considering sample heterogeneity in the context of key driver analysis, it also argues that the conventional method used to overcome this - latent class regression - has some drawbacks when used in applied customer satisfaction research. This paper has focused on four issues concerning the applied usage of latent class regression: 1. 2. 3. 4. spurious multicollinearity occurs in many customer satisfaction datasets systematically missing data are a common occurrence over-fitted results that are, at best, difficult to use a tendency for latent classes to offer little differentiation with respect to the raw variables used in the analysis.

Most of these stem from a fundamental disconnect between (sound) theory and the practical realities that tend to constrain the data generated in research of this kind. Given these concerns we have proposed, and trialled, an alternative approach using supervised genetic algorithms to assess bivariate relationships in order to achieve the primary objective of finding distinct latent classes. Our results showed that the genetic algorithm approach was superior and, critically, provided us with a way of drawing better commercially focused conclusions than a latent class regression solution. Acknowledgement The author would like to thank Ian Brace, Director of Research Methods at TNS, for his invaluable guidance throughout the preparation of this paper.
References
Allen, D. & Rao, T. (2000) Analysis of Customer Satisfaction Data. Wisconsin: Quality Press. Bacon, D. (2003) A comparison of approaches to importance-performance analysis. International Journal of Market Research, 45, 1, pp. 55-71.

705

Choosing the right baskets for your eggs

Bottomley, P. &C Nairn, A. (2004) Blinded by science: the managerial consequences of inadequately validated cluster analysis solutions. International Journal of Market Research, 46, 2, pp. 171-187. Grapentine, T. (1997) Managing mulricollinearity: real-world survey data can create surrealistic analytic models. Marketing Research, 9, 3, pp. 10-21. Grapentine, T. (2010) Taming the troublemaker: a critique of averaging over orderings as an approach for managing multicollinearity. Marketing Research, 22, 4, pp. 14-19. Hair, J., Anderson, R., Tatham, R. & Black, W. (1998) Multivariate Data Analysis. New Jersey: Prentice Hall. Hair, J., Black, W, Babin, B. & Anderson, R. (2010) Multivariate Data Analysis: A Global . Perspective. New Jersey: Pearson. Holland, J.H. (1975) Adaptation in Natural and Artificial Systems. Ann Arbor: University of Michigan Press. Kruskal, W. (1987) Relative importance by averaging over orderings. The American Statistician, 41, 1, pp. 610. LaLonde, S. (1996) Key driver analysis using latent class regression. Proceedings of the Survey Research Methods Section, American Statistical Association. Online at: http://www.amstat. org/sections/srms/proceedings/papers/1996_079.pdf (accessed 31 July 2012). Pinnell, J. (2003) Improving healthcare markering through market segmentation and targeting. ESOMAR Healthcare Conference, New York. Steiner, W & Hruschka, H. (2003) Genetic algorithms for product design: how well do they really work? International Journal of Market Research, 45, 2, pp. 229-240. Stewart-Hunter, M. Sc Bergenthal, M. (2004) The art and science of consumer segmentation. Admap, 454, pp. 65-68. Tabachnik, B. & Fidell, L. (2001) Using Multivariate Statistics. Needham Heights: Allyn & Bacon. Tull, D. & Hawkins, D. (1990) Marketing Research: Measurement and Method (5th edn). New York: Macmillan. Vermunt, J. Sc Magidson, J. (2002) Latent class analysis, in J. Hagenaars &C A. McCutcheon (eds) Applied Latent Class Analysis. Cambridge, England: Cambridge University Press, pp. 89-106. Zhang, J. & Mahmud, L (2005) A simulation study on SPSS ridge regression and ordinary least squares regression procedures for multicollinearity data. Journal of Applied Statistics, 32, 6, pp. 571-588. Zucarro, C. (2010) Statistical alchemy - the misuse of factor scores in linear regression. International Journal of Market Research, 52, 4, pp. 511-531.

About the author Sam Davis has been working in market researcb for ten years in operations, client-service and analytic roles - most recently as a director at TNS's Marketing Science Centre in London. In this role he bas worked witb a wide spectrum of clients across many sectors, ranging from government agencies to EMCG products. Sam's main researcb interests are market segmentation and customer satisfaction/loyalty researcb. Address correspondence to: Sam Davis, Email: samadavis@googlemail.com

706

Copyright of International Journal of Market Research is the property of Warc LTD and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.

You might also like