You are on page 1of 21

International Journal of Selection and Assessment

Volume 21 Number 1 March 2013

Individual Differences in the Effectiveness of Error Management Training for Developing Negotiation Skills
Michael J. Cullen*, John P. Muros**, Rena Rasch*** and Paul R. Sackett****
*Human Resources Research Organization, 100 Washington Avenue South, Suite 1660, Minneapolis, MN 55401, USA. mcullen@humrro.org **AT&T, Dallas, TX 75202, USA ***Kenexa High Performance Institute (KHPI), Minneapolis, MN 55402, USA ****Department of Psychology, University of Minnesota, Minneapolis, MN 55455, USA

This study investigated whether the effectiveness of an error management approach to training negotiation knowledge and skill depended on individual differences in conscientiousness, extraversion, and openness to experience. Participants were randomly assigned to two training programs that incorporated key elements of an error management and behavioral modeling approach to training, and were trained in the complex interpersonal skill of negotiation. At the end of training, declarative knowledge acquisition, procedural knowledge acquisition, declarative knowledge retention, and transfer performance were assessed at different points in time. Results suggested that the effectiveness of the error management training program was dependent on individual levels of conscientiousness and extraversion. For several learning outcomes, the performance of highly conscientious and extraverted individuals was superior in the error management condition, while the performance of less conscientious and introverted individuals was superior in the behavioral modeling condition. The implications of these ndings, and suggestions for future research, are discussed.

1. Introduction
esearchers have recently become interested in investigating the effectiveness of error management training (EMT) for a variety of learning and transfer outcomes. EMT is a relatively new approach to skill acquisition pioneered by cognitive psychologists, primarily in the educational domain (e.g., Brooks, 1990; Fosnot, 1996). The central premise of EMT is that the learning of complex, cognitively laden skills is best accomplished in an environment in which trainees actively engage in exploration, problem solving, hypothesis testing, making mistakes, and learning how to recover from mistakes (Ivancic & Hesketh, 1995/1996). The focus in an errorbased approach is on reducing training content, increasing participant involvement, and encouraging errors. Theoretically, EMT is believed to lead to increased depth of processing of information in training and im-

proved encoding, retention, and recall of information later on (Chandler & Sweller, 1991; Craik & Lockhart, 1972). By forcing trainees to interrupt training and reect, errors are hypothesized to lead to better mental models of the material to be learned, and to increased practice of behaviors that are the source of the error. Finally, EMT may lead trainees to practice errorrecovery strategies, which may increase performance (Frese & Zapf, 1994; Heimbeck, Frese, Sonnentag, & Keith, 2003; Ivancic & Hesketh, 1995/1996). Recent studies have supported the effectiveness of EMT in fostering learning. Several studies have reported medium to large effect sizes for EMT relative to other proceduralized training methods that advocate step-bystep instructions and the avoidance of errors (Frese, 1995; Nordstrom, Wendland, & Williams, 1998; Wood, Kakebeeke, Debowski, & Frese, 2000). In addition, a recent meta-analysis of EMT by Keith and Frese (2008)

2013 Blackwell Publishing Ltd., 9600 Garsington Road, Oxford, OX4 2DQ, UK and 350 Main St., Malden, MA, 02148, USA

bs_bs_banner

2 found that deliberately incorporating errors into training can be an effective means of promoting learning. Across 24 studies that investigated the effectiveness of EMT, they found that EMT led to superior overall learning outcomes compared to proceduralized methods that discouraged errors or exploratory training methods that did not involve explicit encouragement of errors (d = 0.44). Importantly, however, the training tasks included in these earlier studies, and in the meta-analysis, were conned to a very narrow range of tasks. In most cases, the training involved using a new computer software package. None of the studies in the Keith and Frese (2008) meta-analysis examined the effectiveness of EMT for training a complex interpersonal skill. The purpose of this study is to investigate whether the effectiveness of EMT in training the complex interpersonal skill of negotiation is moderated by individual differences in conscientiousness, extraversion, and openness to experience. To examine this issue, we develop two training programs that incorporate key elements of an EMT approach to training interpersonal skills, and a behavioral modeling approach (BMT). Because several different viable EMT and BMT negotiation training programs could be developed using fewer or more of the key elements of an EMT and BMT approach thus potentially leading to different learning outcomes depending on which features are included in the training programs our goal is not to compare the effectiveness of these training programs themselves. Rather, we focus on the role of conscientiousness, extraversion, and openness as moderators of the effectiveness of an EMT approach to training interpersonal skills. This research therefore responds to Keith and Freses (2005) call to continue the effort to understand the individual differences potentially affecting the effectiveness of EMT training. Although some efforts have been made to investigate this issue (e.g., Carter & Beier, 2010; Gully, Payne, Kiechel Koles, & Whiteman, 2002; Keith, Richter, & Naumann, 2010), no studies to date have examined the moderating role of important individual difference variables for a complex interpersonal skill such as negotiation.

Michael J. Cullen, John P. Muros, Rena Rasch and Paul R. Sackett oral modeling instruction are taught by a trainer in a group setting. In that group, trainees (1) receive an introduction to a topic, (2) watch a model perform the desired behaviors, (3) discuss what the model did right and wrong, (4) practice the desired behaviors via role playing, and (5) receive feedback about their performance. Learning is hypothesized to take place as a result of attentional, retentional, and motivational processes invoked by the procedure. Introducing an individual to a topic invokes attentional processes. Similarly, watching, discussing, and practicing the relevant behaviors invokes retentional processes. Finally, the provision of feedback invokes a motivational process (Bandura, 1969). Meta-analytic reviews have generally supported the efcacy of behavioral modeling as a method of training interpersonal skills. In their recent meta-analysis of BMT, Taylor, Russ-Eft, and Chan (2005) found large effect sizes for BMT on interpersonal knowledge and skill outcomes. Among studies employing control groups, the mean population effect size estimate was slightly more than 1.0 standard deviation (SD), which is comparable to those found in earlier meta-analyses for similar criteria (Arthur, Bennett, Edens, & Bell, 2003; Burke & Day, 1986). The mean population effect size estimate for attitudes was smaller, but still substantial, with an average change of one third of a standard deviation. Finally, the estimated population effect size for on-the-job performance behavior was approximately one quarter of a SD. This effect is smaller than the results reported in other meta-analyses for this criterion (Arthur et al., 2003; Burke & Day, 1986; Guzzo, Jette, & Katzell, 1985), which the authors attribute to the inclusion of more recent studies showing smaller BMT effects on behavioral outcomes (May & Kahnweiler, 2000; Russell, Wexley, & Hunter, 1984; Werner, OLeary-Kelly, Baldwin, & Wexley, 1994), and the inclusion of a greater number of unpublished studies than previous meta-analyses. Although the Taylor et al. (2005) effect size estimates were positive for all criteria, there was sufcient remaining variance in training effects, after removing sampling error variance, to warrant a search for moderators. On theoretical grounds and on the basis of prior empirical ndings, Taylor et al. decided to investigate four training design characteristics that may moderate the validity of BMT: (1) learning points (use of retention aids; display of learning points during modeling), (2) characteristics of the modeling display (presentation of positive-only models vs. mixed models), (3) characteristics of the behavioral rehearsal (instruction to symbolically rehearse newly learned skills; coaching prior to behavioral rehearsal; use of trainee-generated vs. trainer-provided practice scenarios), and (4) the number of hours of training. Results supported the benecial effects of many of the training design characteristics that had been advocated in the literature, including using learning points presented as rule codes,

1.1. Training interpersonal skills using behavioral modeling and error management approaches
Despite the availability of many different methods for training interpersonal skills (e.g., lecture, modeling, tutorial, simulation, discovery, error-based learning), BMT has long been the most popular method. Literally, millions of managers have been taught supervisory and teamwork communication skills using BMT (Wexley & Latham, 2002). Behavioral modeling has its roots in social learning theory, which posits that learning can take place by virtue of watching others perform a behavior (Bandura, 1965). Typically, individuals receiving behavi-

International Journal of Selection and Assessment Volume 21 Number 1 March 2013

2013 Blackwell Publishing Ltd.

Individual Differences and Error Management Training using mixed rather than positive-only models, and having trainees generate some of the scenarios they practice. Not unexpectedly, more hours of training was associated with greater development of procedural knowledge. In our study, we take these moderator ndings into account by attempting to include in our behavioral modeling program the key features found in the Taylor et al. meta-analysis to enhance learning and transfer. Although BMT has predominated as a method for training interpersonal skills, other methods have been used. For instance, interpersonal skills have been taught using the lecture method, programmed instruction, discussion, and various combinations of these methods. In their meta-analysis of the effectiveness of different training methods for teaching different categories of skills, Arthur et al. (2003) found a wide range of effect sizes for these training methods and combinations of methods for teaching interpersonal skills. The effect sizes varied considerably (i.e., from d = .22 to d = 1.44) depending on whether reaction, learning, or behavioral criteria were employed. Another method for training interpersonal skills that holds promise is EMT. In contrast to a BMT, in which the focus is on guiding learners in an errorless, stepby-step fashion through a preestablished set of training material, the focus in an error-based approach is on reducing training content, increasing participant involvement, and encouraging errors. In contrast to BMT, the learning processes underlying EMT are only just beginning to be understood. As mentioned earlier, the chief benet of an error-based approach is that it forces individuals to generate and test hypotheses about a learning situation, thus increasing the cognitive load and depth of processing (Chandler & Sweller, 1991). Greater depth of processing of information, in turn, is anticipated to lead to better encoding of information, and to better recall (Craik & Lockhart, 1972). It also may promote more controlled, as opposed to automatic, processing of information (Ivancic & Hesketh, 1995/1996) because the errors that occur as a result of the training increase the attentional resources that must be devoted to understanding the error. More controlled processing may in turn play a role in how well principles are learned (Sweller, 1988). A third argument for EMT is that errors are a source of negative feedback, and such negative feedback may be necessary for learning (Frese & Zapf, 1994; Heimbeck et al., 2003). Advocates of EMT acknowledge that there are potential downsides to an error-based approach to learning. EMT may create negative emotions and reduced selfefcacy, which could potentially decrease learning. To address this, advocates of EMT promote the use of error management instructions (Keith & Frese, 2005; Keith & Frese, 2008), which point out that errors are an expected part of the learning process and are important,

3 if not essential, to learning. In this manner, the error management instructions aim to reduce the potentially negative emotions associated with making errors. Another way to reduce the anxiety of EMT is to correct errors that do occur. Once errors occur, there are potentially two routes the training designer might take. One route is to allow trainees to correct the error on their own, unaided by the trainer (Nordstrom et al., 1998). A second approach is to assist trainees in the resolution of errors (Carlson, Lundy, & Schneider, 1992; Debowski, Wood, & Bandura, 2001), thus potentially alleviating some of the negative feelings associated with errors. In this study, we aim to reduce the negative emotions associated with making errors by incorporating the kind of error management instructions suggested by other researchers. Once errors have occurred, we follow the second approach to error correction; we provide a set of hints to help trainees get back on track during the training.

1.2. The role of individual differences


As mentioned earlier, because EMT and BMT programs can take a variety of specic forms, it is difcult to test directly whether an EMT or BMT approach to training interpersonal skills is more effective. Because the results obtained will be limited to the specic features of the BMT and EMT training programs developed, any broad generalizations about the relative effectiveness of either training program for training negotiation knowledge and skill would not be appropriate. Accordingly, our goal in this study is to not to compare the relative effectiveness of these two viable approaches to training negotiation knowledge and skill. Rather, we focus on the important issue of whether the effectiveness of these broad approaches for training a complex interpersonal skill might be moderated by key individual differences. The investigation of whether the effectiveness of a training method is moderated by individual differences is known as the study of aptitude treatment interactions (ATI). ATI have been a fertile source of research activity among educational psychologists, but until recently, have been less actively pursued by training researchers (Warr & Allen, 1998). Nonetheless, ATI are important in training research because, when substantiated, they warrant the placement of trainees to different training methods. The individual and organizational benets of such differential placement of trainees when ATI exist were documented several decades ago by Cronbach and Gleser (1965). We believe there are strong theoretical reasons for expecting interactions between the EMT and BMT training methods and several individual difference characteristics. As Gully et al. (2002) point out, an EMT program is unlikely to work equally well for everyone because learners may differ in the extent to which they prefer to

2013 Blackwell Publishing Ltd.

International Journal of Selection and Assessment Volume 21 Number 1 March 2013

4 engage in, or are able to engage in, active experimentation and exploration of material. Put another way, learners may vary in either their preference for dealing with, or ability to handle, different levels of structure in a training program. Within the educational domain, the idea that individuals may learn better in environments that suit their preferences is well reected in the learning styles literature (Pasher, McDaniel, Rohrer, & Bjork, 2008). A number of different learning styles models have been proposed and several assessments based on these models have been created (e.g., Dunn, 1990; Kolb, 1984). Although there is little evidence that any of the learning styles reected in these models reliably interacts with an educational treatment, there is evidence for an ATI between level of structure in a training program and two important individual difference variables: cognitive ability and locus of control. One of the most consistent ndings in the ATI literature is that individuals higher on cognitive ability learn more when the level of structure during training is low, and those lower on cognitive ability learn more when the level of structure is high (Snow, 1989; Snow & Lohman, 1984). Methods that are low in structure present material at a relatively rapid pace, under incomplete learning conditions where the students must infer much of the instructional message for themselves (Snow, 1989; Snow & Lohman, 1984). In contrast, highly structured methods allow the instructor to maintain a high level of control over learning activities. One explanation for this consistent ATI is that high-ability learners may prefer to develop their own strategies for learning material, and a low-structure condition best facilitates such self-directed learning (Snow & Lohman, 1984). In contrast, low-ability learners may prefer to have the trainer organize and present the material for them (Snow & Lohman, 1984). These differences in learner preferences may in turn affect the motivation of high- and low-ability learners in high- and low-structure training environments, thereby leading to differential learning for each group in these two conditions. Similar ATI ndings have been made for locus of control, which refers to an individuals belief about whether his or her successes or failures are a consequence of internal or external factors (Rotter, 1966). A number of studies have found that students with an external locus of control demonstrated better understanding of material taught in highly guided instruction conditions, whereas students with an internal locus of control demonstrated better understanding in less highly guided conditions (Daniels & Stevens, 1976; Horak & Horak, 1982; Horak & Slobodzian, 1980; McLeod & Adams, 1980/1981). Like the ATI studies centering on cognitive ability, these studies suggest that differences in preferences for structure may differentially affect learning in training conditions high and low in structure.

Michael J. Cullen, John P. Muros, Rena Rasch and Paul R. Sackett In our study, the BMT and EMT conditions were both highly structured throughout the majority of the training. In both conditions, instructional tasks were broken down into small units, the trainer claried material frequently, and maintained control over the content and pace of learning. However, during the key study manipulation, when the negotiation strategies were taught, the level of structure was quite different between the programs. In the BMT condition, the negotiation strategies were taught in a highly structured way. The trainer introduced the negotiation strategies and associated key behaviors, reviewed them one by one with trainees, and presented trainees with video models enacting the strategies and key behaviors. In contrast, in the EMT condition, the trainer did not inform participants what the effective negotiation strategies and behaviors were, and no modeling videos were employed. Instead, participants endeavored to learn the strategies and behaviors on their own by participating in a novel group brainstorming exercise. Accordingly, the level of structure was much lower in the EMT condition during this training component. We believe this difference in the amount of exploration and experimentation encouraged by the two programs during this key training component may have differentially affected learning for trainees with differential standing on three important individual difference variables: conscientiousness, extraversion, and openness to experience. 1.2.1. Conscientiousness Conscientious individuals are described as efcient, organized, dutiful, thorough, self-disciplined, and deliberate (Costa & McCrae, 1992). The Barrick and Mount (1991), Hough, Eaton, Dunnette, Kamp, and McCloy (1990), and Salgado (1997) meta-analyses indicate that conscientiousness is positively related to training outcomes. Individuals high on conscientiousness are likely to pay attention to the training material, organize training content, and use the meta-cognitive strategies of planning, monitoring and evaluating to enhance their learning. Conscientiousness may also affect learning by affecting the motivation to learn via its effect on selfefcacy (Colquitt, Lepine, & Noe, 2000). More specic facets of conscientiousness, such as achievement orientation, are also positively related to the motivation to learn (Mathieu, Martineau, & Tannenbaum, 1993). Consideration of the preferences of trainees with higher and lower levels of conscientiousness suggests a moderator hypothesis for this study. Just as less highly guided classroom conditions may be preferred by individuals with a greater sense of control over their ability to learn (e.g., Horak & Horak, 1982), a training environment with more student-oriented learning, such as the key training component in the EMT condition, may allow highly conscientious trainees to learn the training material the way they prefer, by organizing the material them-

International Journal of Selection and Assessment Volume 21 Number 1 March 2013

2013 Blackwell Publishing Ltd.

Individual Differences and Error Management Training selves and being self-disciplined and effortful in their approach to learning. This should motivate highly conscientious trainees, leading to greater learning for them than in the more highly structured BMT component dealing with the negotiation strategy training. In contrast, the BMT condition may appeal to trainees low in conscientiousness since it allows them to be less industrious by letting the trainer organize the key negotiation strategy content for them. Additionally, relative to the more structured BMT condition that requires less organization of material and self-discipline to learn the negotiation strategies, conscientious trainees organized, self-disciplined and deliberate approach to tasks may make them much better equipped than those lower in conscientiousness to formulate and execute effective learning strategies during the key low-structured EMT training segment. Thus we hypothesize H1: Conscientiousness will interact with training condition to inuence learning and performance of negotiation skill (EMT > BMT for trainees high in conscientiousness; BMT > EMT for trainees low in conscientiousness). 1.2.2. Extraversion Extraversion is the tendency to be sociable, assertive, active, and to experience positive emotions such as energy and zeal (Costa & McCrae, 1992). In the Barrick and Mount (1991) meta-analysis, the estimated population validity of extraversion was higher than for any other Big Five personality variables (r = .26). One reason extraverted trainees may excel in training is the active stance they take toward learning. Taking an active stance toward learning may improve learning and transfer by increasing higher order metacognitive activities such as planning, monitoring, and evaluating, which are in turn related to learning (Bell & Kozlowski, 2008; Keith & Frese, 2008). Consideration of the preference of extraverts for an active training program suggests a moderator hypothesis for this study. As is typical of all EMT programs, our training study required active exploration of alternative hypotheses regarding the training material to be learned. In particular, in the EMT condition, trainees learned the negotiation strategies by brainstorming with other trainees, and testing hypotheses about what the effective strategies might be. In contrast, in the BMT condition, those strategies were taught directly by the trainer. Thus, during the key manipulation in the study, trainees were much more actively involved in determining the training content in the EMT condition than in the BMT condition. Extraverts may have preferred this more active approach to learning the strategies than introverts, thereby increasing motivation to learn, and thus learning itself, in extraverts in the EMT condition. Another reason one might expect an interaction between our training conditions and ex-

5 traversion has to do with (1) the differing levels of activity in the two training conditions, and (2) the fact that introverts have a higher resting state of arousal than introverts (Eysenck & Eysenck, 1985). In line with the YerkesDodson law, which predicts a curvilinear relationship between arousal level and performance (Yerkes & Dodson, 1908), the less-active, and presumably lessstimulating or arousing BMT condition may be optimal for introverts, who already have a high resting state of arousal. In like fashion, the more active EMT condition may raise extraverts lower resting state of arousal to more optimal levels for learning. These considerations suggest the following hypothesis: H2: Extraversion will interact with training condition to inuence learning and performance of negotiation skill (EMT > BMT for trainees high in extraversion; BMT > EMT for trainees low in extraversion). 1.2.3. Openness to experience Openness to experience is the tendency to be imaginative, nonconforming, unconventional, autonomous, inventive, curious, and innovative (Costa & McCrae, 1992). At least three meta-analysis indicate that openness to experience is positively related to learning (Barrick & Mount, 1991; Hough et al., 1990; Salgado, 1997). A high capacity for deep processing of information seems likely for learners high in openness in light of their active approach to learning and willingness to try new things (Barrick & Mount, 1991). Since individuals higher in openness are imaginative, curious, and enjoy learning things in different ways (Costa & McCrae, 1992), they may prefer to learn the negotiation strategies in the less-structured EMT component dealing with this content than in the BMT component dealing with this content. In contrast, individuals lower in openness may prefer the traditional teacher-centered BMT approach for learning the strategies. As with conscientiousness, these differences in learner preferences may lead to differential motivation to learn, and thus learning itself, for trainees higher and lower in openness in the two conditions. In addition, relative to the more structured BMT condition in which the negotiation strategy content is fairly easily processed by everyone, trainees higher in openness may be much better able than trainees lower in openness to deeply process the key content in the lower structured component of the EMT condition (Gully et al., 2002). Thus we hypothesize H3: Openness to experience will interact with training condition to inuence learning and performance of negotiation skill (EMT > BMT for trainees high in openness; BMT > EMT for trainees low in openness). To test these hypotheses, we created comprehensive BMT and EMT programs to train negotiation skills. Our

2013 Blackwell Publishing Ltd.

International Journal of Selection and Assessment Volume 21 Number 1 March 2013

6 central goal in creating these training programs was to create fair representations of an EMT and BMT approach to training an interpersonal skill such as negotiation. Accordingly, for each training method, we endeavored to incorporate as many BMT or EMT features as we could that had been advocated by research or theory. In past studies, hypotheses concerning the effectiveness of EMT compared to other training methods have differed depending on the learning outcome. For instance, in their meta-analysis of the effectiveness of EMT, Keith and Frese (2008) hypothesized that shortterm learning and performance may not be superior for trainees in an EMT condition than those in a more proceduralized, errorless, training condition. With regard to transfer, Keith and Frese (2008) hypothesized that EMT may be as effective as other methods that avoid errors on measures of analogical transfer, but more effective than such methods on measures of adaptive transfer. Because the relative efcacy of EMT and BMT may vary by type of learning outcome, we included four different measures of learning in this study, spread out over time: (1) declarative knowledge acquisition, (2) procedural knowledge acquisition, (3) declarative knowledge retention, and (4) transfer performance. Measures of declarative and procedural knowledge acquisition were administered immediately following training; measures of declarative knowledge retention and transfer performance were administered 3 weeks after training. While declarative knowledge measures were traditional paper-and-pencil measures, procedural knowledge measures were actual negotiation simulations that required trainees to display their knowledge. In sum, this study contributes to training research in the following ways. First, this study responds to the call of Keith and Frese (2005) to continue to investigate the moderating role of individual difference variables for EMT programs. Second, just as importantly, it investigates the moderating role of important variables in relation to an interpersonal skill. Almost all prior studies investigating either the effectiveness of EMT for learning, or possible mediators or moderators of EMT have done so using a fairly narrow range of computer tasks. Thus, as advocated by Keith and Frese (2005), this study begins to look at the effectiveness of EMT programs for a broader class of skills. Finally, this study examines the moderating effect of individual differences for an especially wide range of learning and transfer outcomes spread out over time.

Michael J. Cullen, John P. Muros, Rena Rasch and Paul R. Sackett ing was a relevant instructional experience, such as courses in communication studies, persuasion, sales, and marketing. Participants were paid for their participation in the study, and the top performers received additional cash prizes. The sample was 53.7% female, and the mean age was 24.16 (SD = 7.68). Approximately 84% of participants were White, 5.9% were African American, 7.4% were Asian, and 2.2% were Hispanic. To participate in the study, participants were required to speak English uently. Only participants with no prior negotiation training were eligible for the study.

2.2. Experimental design and procedure


Participants received training in negotiation skills. The training program was developed for use in mixedmotive negotiations involving both distributive and integrative elements. In distributive negotiations, the resource pie is xed and the negotiators chief interest is obtaining as big a piece of this xed pie as possible. In integrative negotiations, the negotiator may be interested in expanding the available pie to satisfy a variety of important interests (Pruitt & Rubin, 1986; Raiffa, 1982; Walton & McKersie, 1965). We chose to train a key set of distributive and integrative strategies, based on the pioneering work of Fisher, Ury, and Patton (1999), and chose to train two key integrative strategies and six key distributive strategies. The eight strategies, and the key behaviors associated with those strategies, are described in Table 1. 2.2.1. Training program The training program contained six parts. All trainees participated in Parts I, II, IV, V, and VI, and all of these training segments were identical for both groups. In Part I, the trainer welcomed trainees and reviewed the training agenda and learning objectives. In Part II, the trainer introduced trainees to basic negotiation principles, including key terms and distinctions. In Part IV, the trainer led participants through a mock negotiation role play exercise in which they practiced using the negotiation strategies. In Part V, the trainer reviewed the negotiation strategies taught. In Part VI, the trainer led participants through a goal setting exercise to help motivate them to use the strategies outside of training. The key manipulation took place in Part III, when the trainer taught the eight negotiation strategies using either the BMT or EMT training method. The training programs in both conditions were all taught by the same trainer, an advanced female graduate student in industrial organizational psychology with prior teaching experience. Prior to the training, the trainer underwent several hours of training regarding how to deliver the BMT and EMT training content. The training program in both conditions lasted approximately 3 hours.

2. Method 2.1. Sample


One hundred and thirty-two undergraduates were recruited from three Midwestern universities. Students were recruited from classes for which negotiation train-

International Journal of Selection and Assessment Volume 21 Number 1 March 2013

2013 Blackwell Publishing Ltd.

Individual Differences and Error Management Training


Table 1. Negotiation strategies and key behaviors
Integrative strategies Strategy 1: Focus the discussion on interests Focus the discussion on interests, not positions Indicate what your own interests are in the situation Inquire what the other partys interests are in the situation Strategy 2: Appealing to interests Propose win-win solutions Discuss how your proposal satises the other partys interests Highlight the mutual benets of the proposal Distributive strategies Combating stubbornness Strategy 3: Using silence If you dont like the other partys proposal, dont say anything Strategy 4: Broken record reiteration of main points Repeat the same or similar phrases over and over Strategy 5: Display knowledge of similar situations Bring up a similar negotiation situation Ask the other party what they have done in similar circumstances Combating attacks and tricks Strategy 6: Responding to tricks or ploys using direct counters Question the tactic directly Refrain from attacking the other partys personal integrity Discuss how the tactic makes you feel Strategy 7: Responding to criticisms of your proposal If the other party criticizes your proposal, do not attack their position Dont defend your ideas invite criticism and advice Combating the strategic use of emotion Strategy 8: Responding to displays of emotion using attitudinal bargaining Allow the other side to let off steam Acknowledge the emotions and make them legitimate Control your own emotions

7 scenarios depicted interactions with which participants would be familiar, such as bargaining for the purchase of an automobile, or negotiating with a partner about where to spend a vacation. The scenarios contained examples of both effective and ineffective behaviors. Following each role play, the trainer led a group discussion about key behaviors used, providing corrective and reinforcing feedback as appropriate. Participants then watched two videos to help them model the behaviors they had learned. The rst was a negative display and showed the actor failing to use any of the correct strategies. The second showed the actor using all of the strategies correctly. In the second video, as each strategy was used, it was ashed on the video. According to Taylor et al.s (2005) BMT metaanalysis, skill development is greatest when learning points are presented as rule codes rather than descriptions or summaries of what the actor did. Accordingly, in the video displays, the strategies were presented as rule codes. As each video was played, participants answered questions in their workbook about which strategies were used effectively and ineffectively. At the conclusion of each video presentation, the trainer led a discussion about which strategies were used effectively and ineffectively, again providing corrective and reinforcing feedback as appropriate.

2.2.2. Strategy training in the BMT condition The typical sequence of learning activities in BMT is provision of the skills behaviors to be learned, prior to, or along with modeling, and then practice with feedback from either trainees and/or the trainer (Taylor et al., 2005). Feedback serves a correctional function when trainees do not use the behaviors correctly, and a motivational function, through a focus on reinforcement (i.e., praise) when a trainee performs behaviors correctly (Goldstein & Sorcher, 1974). We made a concerted effort to include all of these typical features in the BMT program. In the BMT condition, the trainer rst reviewed the declarative knowledge to be learned: the eight negotiation strategies and associated key behaviors. For each strategy, the trainer rst presented a denition of the strategy, presented a rationale for its use, and explained the key behaviors associated with the strategy. Next, to assist in knowledge compilation, the trainer asked two volunteers to role play eight scenarios in which each of the eight strategies and its associated key behaviors were implemented. Each scenario dealt with one strategy and its associated behaviors. The

2.2.3. Strategy training in the EMT condition EMT contains three characteristics that distinguish it from more proceduralized methods such as BMT. First, participants are given minimal guidance and are encouraged to explore material on their own (Keith & Frese, 2008). Second, EMT creates a learning situation in which errors are likely to occur (Keith & Frese, 2008). Finally, EMT participants are explicitly informed of the positive value of errors and are provided with error management instructions to reduce the frustration and anxiety that may be induced by errors (Heimbeck et al., 2003; Keith & Frese, 2005). In designing the strategy training for the EMT condition, we made a concerted effort to include these typical design elements. In the EMT condition, the trainer did not review the eight strategies and associated behaviors with participants. Instead, she informed participants that they would learn the strategies on their own by participating in a group exercise. Prior to commencing the exercise, the trainer issued a variety of error management instructions to participants that were similar to those used in past EMT studies. For instance, she told participants Dont worry about making wrong guesses, We expect you to make errors during this process, and that errors will help you in the learning process (Debowski et al., 2001; Heimbeck et al., 2003; Keith & Frese, 2005). The trainer then began the group exercise. First, participants were given transcripts of the eight scenarios that were

2013 Blackwell Publishing Ltd.

International Journal of Selection and Assessment Volume 21 Number 1 March 2013

8 role played in the BMT condition. Unlike the BMT scenarios, however, these scenarios contained correct implementations of the key behaviors only. Second, participants were asked to read the scenarios on their own and take notes on which strategy and key behaviors they believed were being used. Third, two volunteers role played each scenario, after which the trainer led a group discussion about which key behaviors had been used in that scenario. Feedback similar to that provided in the BMT condition was then provided. When participants generated an incorrect key behavior, the trainer told participants it was not the behavior she was looking for. When participants generated a correct behavior, reinforcing feedback was provided and the group had a discussion about why that negotiation behavior was useful in that situation. If, after three successive tries, the group failed to come up with a correct key behavior for a scenario, the trainer took the class to the narrow passage in the scenario where the key behavior was demonstrated, and asked them a series of questions about the passage. Once all the key behaviors for a given scenario had been identied, the trainer asked for volunteers to identify the overall strategy being employed in the scenario. When participants identied the correct content for the strategy, the trainer revealed this strategy on a PowerPoint slide. The entire process was repeated for all eight scenarios. In both training conditions, we created a mock negotiation role play to facilitate the development of procedural knowledge. The purpose of the role play was to give trainees practice in using the strategies and effective behaviors, and to provide informational feedback about their use of the strategies. The role play contained two roles: a sales representative for a company seeking to purchase photocopiers, and an agent for the vendor. To facilitate the role play, each role was provided with a set of background materials. Sales representatives were presented with a tentative offer by the vendor. They were instructed that their negotiation objective was to satisfy as many of their nancial and nonnancial interests as possible by using as many of the key negotiation behaviors they could. To that end, they were provided with a list of their nonnancial interests. The agent for the vendor was provided with background information about the tentative offer that had already been made, and a list of her own nonprice-based interests. The agent for the vendor was also provided with a role play script to guide the mock negotiation. The script was designed to loosely structure the dialogue so that the sales representative would have the opportunity to use all of the key negotiation strategies and behaviors. At the conclusion of the role plays, the trainer led a group discussion about (1) how the negotiation went, (2) which aspects of the negotiation were challenging, and (3) which specic strategies participants had struggled to use.

Michael J. Cullen, John P. Muros, Rena Rasch and Paul R. Sackett Following the practice role plays, trainees were asked to consider how setting goals for using the strategies and associated behaviors could be a useful tool for remembering the strategies, and motivating them to use the strategies, in the weeks to come. The study utilized a between-subjects design. In both training conditions, participants began the training session by completing a set of premeasures. Subsequently, they completed the training program, a set of postmeasures, a knowledge acquisition measure, and participated in a mock negotiation role play to assess procedural knowledge acquisition. Three weeks later, participants returned to complete a measure of declarative knowledge retention and to participate in a transfer role play assessment. Before the transfer assessment commenced, participants also completed a short questionnaire about whether, and how often, they had attempted to use the negotiation strategies since the training program. The long duration of this training program made it efcient to administer the training to groups of students. Accordingly, small groups, rather than individuals, were randomly assigned to one of the two experimental conditions. Groups were roughly equally heterogeneous in terms of their ethnic, gender, and age composition. In addition, none of the personality variables investigated in this study were signicantly correlated with training condition (see Table 2), providing further evidence of the success of random assignment. In order to increase trainee motivation to learn the negotiation strategies, participants were told in Part I of the training that the top three performers on the assessments would receive cash bonuses of $100, $50, and $25, respectively, and that if they scored in the top one third of participants on the assessments, they would be entered into a lottery for three more cash bonuses in the same amount. We asked participants in a posttraining questionnaire whether they were motivated by those potential bonuses to learn the negotiation strategies. The mean response to these questions on a 7-point Likert scale (1 = strongly disagree and 7 = strongly agree) were 4.90 (SD = 1.01) for the effect of the bonuses for the three top performers, and 4.60 (SD = 1.01) for the effect of the lottery bonuses, indicating that both bonuses had a moderately positive effect on motivation to learn the negotiation strategies.

2.3. Measures
2.3.1. Performance measures 2.3.1.1. Declarative knowledge acquisition. Declarative knowledge acquisition was assessed with a 30-item test of the extent to which participants had learned the eight negotiation strategies and the key behaviors associated with each of the strategies. Participants were asked to

International Journal of Selection and Assessment Volume 21 Number 1 March 2013

2013 Blackwell Publishing Ltd.

Individual Differences and Error Management Training


Table 2. Descriptive statistics and intercorrelations Variable 1. Training condition Performance 2. Knowledge acquisition 3. Knowledge retention 4. Procedural knowledge 5. Transfer performance Premeasures 6. Pretraining self-efcacy 7. Pretraining desire to learn Moderators 8. Openness to experience 9. Conscientiousness 10. Extraversion M .51 28.82 25.22 15.10 15.12 0.00 0.00 3.70 3.56 3.55 SD .50 11.60 9.99 5.61 6.54 1.83 1.93 0.56 0.65 0.80 1 .23 .01 .00 .08 .11 -.14 .14 -.13 .10 .74 .72 .54 .53 .05 .01 .19 .01 .06 2 3 4 5 6 7 8 9

10

.67 .51 .51 -.04 .02 .04 .17 -.07

.75 .71 -.07 .11 .29 -.06 .07

.70 -.07 .02 .21 -.10 .09 .95 .40 .17 .06 .36

.96 .23 .12 .14 .78 -.11 .30

.81 .06

.89

Note: N = 127. Training condition was dummy-coded 0 for the BMT condition and 1 for the EMT condition. Positive correlations with training condition indicate the EMT condition scored higher. Internal consistency reliability coefcients are on the diagonal where appropriate. Correlations greater than .18 are signicant at p < .05.

list the eight negotiation strategies and key behaviors associated with each (1 point for each strategy and behavior, for 26 possible points), to identify which strategies and key behaviors they would use in specic negotiation settings (12 total points), and to identify strategy use in a set of scenarios (13 total points). The total score for this test was the sum of scores for these questions, for a total possible score of 51. Scores ranged from 5 to 51, with a mean of 28.31 (SD = 11.61). Coefcient alpha for this test was .74. 2.3.1.2. Declarative knowledge retention. Declarative knowledge retention was assessed with a 36-item test assessing the same elements as those measured on the test of declarative knowledge acquisition, but using different items to avoid confounding retention of knowledge and memory for specic test items. The total possible test score on this test was 53. Scores ranged from 5 to 44, with a mean of 24.88 (SD = 9.91). Coefcient alpha for this test was .67. 2.3.1.3. Procedural knowledge acquisition. Procedural knowledge acquisition was assessed using a mock negotiation role play exercise that took place immediately following training. The role play was structurally identical to the one participants had practiced during training, but involved the purchase and sale of computers instead of photocopiers. Role plays were conducted with one of four confederates, who recorded whether each of the 18 key behaviors was used and whether the behaviors were used in the proper sequence. The role play commenced with one of the four confederates playing the role of the vendor. The confederates had either recently attained a PhD in psychology or were advanced industrial organizational psychology graduate students. As in the negotiation scenario practiced during training,

the vendor led participants through a scripted role play designed to give participants the opportunity to use all of the negotiation strategies and associated behaviors. To make the negotiation realistic, the vendor occasionally lowered the offer price in response to the correct use of behaviors during the negotiation. However, the vendor did not lower the price each time a correct behavior was used. Given the large number of key behaviors to be displayed in the negotiation, we reasoned that it would be too taxing on the vendor, and too unrealistic, for the vendor to lower the price whenever one of the 18 available key behaviors was displayed. Accordingly, rather than assigning a score based on the nal offer price negotiated, as has been done in some previous studies (e.g., Gist, Stevens, & Bavetta, 1991), participants were scored on their actual use of the key behaviors. The mock role plays were not videotaped. Rather, confederates kept track of the use of key behaviors on a scoring protocol in real time as the negotiation proceeded. The scoring protocol was divided into six separate sections, as follows: (1) opening strategy: focusing the discussion on interests (three points), (2) second strategy: appealing to interests (11 points), (3) attack 1: threat to end negotiation (three points), (4) third set of strategies: moving past stubbornness (three points), (5) attack 2: expression of frustration (three points), and (6) attack 3: criticism of position (two points). In general, one point was awarded for each key behavior used. However, for the second strategy, appealing to interests, we granted one point for each win-win solution proposed. Because there were nine possible nancial and nonnancial interests capable of being exchanged, nine total points were available for proposing exchanges of these interests. Finally, four points were available for using behaviors within the rst two strategies in the right order, as

2013 Blackwell Publishing Ltd.

International Journal of Selection and Assessment Volume 21 Number 1 March 2013

10 taught in training. Additional points were not awarded for repeated use of key behaviors. Ideally, to eliminate the possibility of rater effects, procedural knowledge acquisition would have been assessed by the same person. From a practical and logistical standpoint, however, this was not possible, so we used four different raters. We employed two strategies to control for rater effects. First, to eliminate the possibility that raters judgments of transfer performance might be affected by prior knowledge of performance on the measure of task performance, we ensured that for any one participant, different raters rated task performance and transfer performance. Second, all four raters underwent rater training. The total possible score for the test was 29. Scores ranged from 1 to 25, with a mean of 14.79 (SD = 5.80). Four confederates were used, and each one received training from the lead confederate on how to score the behaviors using videotaped sessions of practice role plays. Ratings were standardized within rater to control for any rater differences in scale use. Coefcient alpha for this test was .75. 2.3.1.4. Transfer. For the transfer assessment, participants negotiated a compensation package for a new job. Participants attempted to use as many key negotiation behaviors as possible to help them achieve all of their nancial and nonnancial objectives, and behaviors were scored by the confederate using a scoring document identical in its structure to the one used for scoring procedural knowledge acquisition. Because, structurally, the transfer task was similar to the negotiation task practiced during training, the transfer task was an analogical rather than an adaptive transfer task (Ivancic & Hesketh, 1995/1996; Keith & Frese, 2008). The total possible score for this test was also 29. Scores ranged from 1 to 28, with a mean of 15.12 (SD = 6.54). Once again, four confederates trained in scoring the relevant behaviors were used. Ratings were standardized within rater to control for any rater differences in scale use. Coefcient alpha for this test was .70.

Michael J. Cullen, John P. Muros, Rena Rasch and Paul R. Sackett (Gist et al., 1991; Martocchio & Webster, 1992; Quiones, 1995). Accordingly, we created two specic negotiation-related measures of self-efcacy. The rst measure was a six-item measure of pretraining selfefcacy for learning negotiation skills. Sample items included I believe that I can meet the challenges of this negotiation training program and I am certain that I can manage the requirements of this negotiation training. Coefcient alpha for this scale was .93. 2.4.2. Pretraining negotiation self-efcacy We used a six-item measure adapted from a similar measure created by Bell and Ford (2007). Sample items included I can meet the challenge of negotiating in everyday conditions and I can negotiate effectively without help from others. Coefcient alpha for this scale was .94. 2.4.3. Pretraining desire to learn negotiation strategies Interest in a domain has also been demonstrated to be positively related to learning. For instance, Schiefele, Krapp, and Winteler (1992) investigated over 121 interestachievement correlations and found a mean r of .32. Accordingly, we created two interest-related premeasures. The rst measure was a six-item measure of desire to learn negotiation strategies adapted from a negotiation study conducted by Weissbein (2000). Sample items included I am motivated to learn the skills emphasized in the training program and I am interested in learning the training material. On a 7-point Likert scale (1 = strongly disagree and 7 = strongly agree), the mean score on this measure was 5.93, indicating that participants had a strong pretraining desire to learn the negotiation skills. Coefcient alpha for this scale was .94. 2.4.4. Pretraining willingness to expend effort to learn negotiation strategies A six-item measure of willingness to expend effort to learn negotiation strategies was also adapted from a scale used by Weissbein (2000). Sample items included I intend to try my best to become a good negotiator in this training and I intend to work hard to learn the material in this training course. On a 7-point Likert scale (1 = strongly disagree and 7 = strongly agree), the mean score on this measure was 5.93, indicating that participants had a strong pretraining intention to work hard to learn the negotiation skills. Coefcient alpha for this scale was .97.

2.4. Premeasures
In this research, we created four premeasures, which we expected to be at least moderately positively related to one or more dependent variables. The purpose of creating these premeasures was to use them as covariates in the research, thereby increasing the overall power of the research to detect differences in learning outcomes due to the choice of training method (Arvey & Cole, 1991). 2.4.1. Pretraining self-efcacy for learning negotiation skills Self-efcacy has been demonstrated to be a signicant predictor of training outcomes in many previous studies

2.5. Postmeasures
Postmeasures were developed to permit the assessment of whether conscientiousness, extraversion, and openness moderated the effect of training condition on negotiation performance.

International Journal of Selection and Assessment Volume 21 Number 1 March 2013

2013 Blackwell Publishing Ltd.

Individual Differences and Error Management Training 2.5.1. Personality Conscientiousness, extraversion, and openness were all measured using Goldbergs (1999) 50-item International Personality Item Pool. The scale is composed of 10 phrases related to standing on each of the Big Five traits conscientiousness, openness to experience, emotional stability, extraversion, and agreeableness. Goldberg (1999) reports the mean coefcient alpha for each of the ve scales (10 items each) to be .84, indicating an acceptable degree of internal consistency. The data for this study were consistent with this nding, with alphas of .81, .89, and .78 respectively, for the conscientiousness, extraversion, and openness scales.

11 tions are for the two measures of declarative knowledge (i.e., declarative knowledge acquisition and retention, r = .72) and the two measures of procedural knowledge (i.e., procedural knowledge acquisition and transfer performance, r = .71). However, the two measures of declarative knowledge are also strongly correlated with the measures of procedural knowledge since knowledge of the negotiation strategies was necessary to implement those strategies in the mock negotiation sessions. The correlations with individual difference variables were largely consistent with past ndings. As expected, openness was moderately correlated with three of the four negotiation performance measures (r = .19, .29, and .21, respectively, for declarative knowledge acquisition, procedural knowledge acquisition, and transfer performance). Somewhat unexpectedly, conscientiousness was only moderately positively correlated with one of the learning outcomes, declarative knowledge retention (r = .17). Extraversion was not correlated with any of the learning outcomes. 3.1.1. Main effects of training condition on negotiation performance Table 3 indicates that the coefcients for training condition were not signicant for any of the four outcome variables. Therefore, neither training method was superior to the other in promoting declarative knowledge acquisition, procedural knowledge acquisition, declarative knowledge retention, or transfer performance. 3.1.2. Interactions between training method and individual differences Hierarchical regression was used to test for potential interaction effects between training condition and conscientiousness, extraversion, and openness to experience. The covariates used in the analyses of main effects in Table 3 were entered individually rst. Subsequently, the training condition variable was entered, followed by the individual difference variable, followed by the interaction term. There was a statistically signicant interaction between training condition and conscientiousness for declarative knowledge acquisition (p < .05). In addition, the interaction between training condition and conscientiousness was marginally signicant in the hypothesized direction for declarative knowledge retention (p < .10). Finally, there was a marginally signicant interaction between training condition and extraversion for declarative knowledge acquisition (p < .10). Therefore, as described below, we found partial support for hypotheses 1 and 2. Table 4 displays the interactions for conscientiousness and extraversion. Results support the presence of an interaction between conscientiousness and training method for declarative knowledge acquisition (DR2 =

2.6. Analyses
Means, standard deviations, and correlations among all study variables were rst computed. To determine the effects of training condition on learning, the four negotiation performance indices were all separately regressed on training condition using a variety of covariates. We controlled for college afliation and class size, pretraining self-efcacy, and pretraining desire to learn in all analyses. While we intended to control for all four self-efcacy and motivation-related premeasures separately, subsequent analyses revealed that the two pretraining self-efcacy measures, and the two pretraining desire and willingness measures, were highly correlated (r = .67 and .86, respectively). Accordingly, to avoid analytical difculties arising from collinearity among these two sets of variables, we created two separate unit-weighted composites, one for the selfefcacy measures (i.e., a new pretraining self-efcacy composite) and one for the measures of desire and willingness (i.e., a new pretraining desire to learn composite). Power to detect a moderate (d = .5) effect size using a two-tailed test at alpha = .05 was .87 or higher for each of the four outcome variables. To examine whether the effectiveness of training methods was moderated by cognitive ability, openness to experience, or conscientiousness, the standard moderated multiple regression approach used in differential prediction research was used (Cohen & Cohen, 1983). As recommended by Aiken and West (1991), continuous predictor variables were centered prior to conducting the moderated regression analyses. Since we used a Big Five measure to assess the three personality variables for which we had specic hypotheses, we also conducted post hoc tests on the other Big Five traits.

3. Results 3.1. Study means and intercorrelations


Table 2 presents means, SDs, and correlations for key study variables. Not surprisingly, the strongest correla-

2013 Blackwell Publishing Ltd.

International Journal of Selection and Assessment Volume 21 Number 1 March 2013

12

Michael J. Cullen, John P. Muros, Rena Rasch and Paul R. Sackett

Table 3. Effect of training method on declarative knowledge acquisition, procedural knowledge acquisition, declarative knowledge retention, and transfer performance Step and variable Declarative knowledge acquisition B (SE B) 0.01 (3.32)** -4.49 (3.34)** -0.38 (0.26)** 0.31 (0.57)** -0.27 (0.55)** R2 .218** B (SE B) 2.39 (2.17)** R2 .226** DR2 .008** Procedural knowledge acquisition -0.38 (0.29)** -0.32 (0.29)** -0.05 (0.02)** -0.05 (0.05)** 0.05 (0.05)** .168** -0.22 (0.19)** .178** .010** Declarative knowledge retention 1.19 (3.37)** -1.53 (3.10)** -0.32 (0.26)** -0.12 (0.54)** -0.07 (0.51)** .129** -1.57 (2.02)** .134** .005** Transfer performance

Step 1: Control variables College 1 College 2 Class size Pretraining self-efcacy Pretraining desire to learn

-0.23 (0.32)** -0.42 (0.30)** -0.03 (0.03)** -0.05 (0.05)** 0.02 (0.05)** .124** -.01 (0.19)** .124** .000**

Step 2: Training method BMT versus EMT

Note: N = 126 for declarative knowledge acquisition; N = 122 for procedural knowledge acquisition; N = 119 for declarative knowledge retention and transfer performance. Betas are unstandardized coefcients. College 1 and College 2 are dummy variables; College 3 is the omitted reference variable. BMT = behavioral modeling training. EMT = error management training. BMT was dummy-coded 0 and EMT was dummy-coded 1. * = p < .05. ** = p < .01.

Table 4. Interactions between conscientiousness, extraversion, and training method for declarative knowledge acquisition and declarative knowledge retention Declarative knowledge acquisition Conscientiousness B (SE B) 2.26 (3.32)** -5.00 (3.30)** -0.23 (0.26)** 0.30 (0.56)** -0.47 (0.54)** R2 .208** B (SE B) 2.82 (2.13)** R2 .215** DR2 .007** B (SE B) -2.28 (2.12)** R2 .224** DR2 .009** B (SE B) 7.82 (2.97)** R2 .268** DR2 .044** Declarative knowledge retention Declarative knowledge acquisition Extraversion -0.56 (3.31)** -3.61 (3.34)** -0.45 (0.26)** 0.26 (0.60)** -0.31 (0.55)** .208** 1.95 (2.17)** .215** .007** -1.40 (1.79)** .219** .004** 4.45 (2.39)** .242** .023**

Step and variable Step 1: Control variables College 1 College 2 Class size Pretraining self-efcacy Pretraining desire to learn

2.64 (3.35)** -2.59 (3.04)** -0.22 (0.26)** -0.11 (0.52)** -.21 (0.50)** .129** -0.98 (1.97)** .134** .005** 0.68 (1.95)** .173** .039** 4.85 (2.74)** .196** .023**

Step 2: Training method BMT versus EMT

Step 3: Personality variable Conscientiousness or extraversion

Step 4: Interaction term Conscientiousness method or extraversion method

Note: N = 125 for declarative knowledge acquisition; N = 119 for declarative knowledge retention. Betas are unstandardized coefcients. College 1 and College 2 are dummy variables; College 3 is the omitted reference variable. BMT = behavioral modeling training. EMT = error management training. BMT was dummy-coded 0 and EMT was dummy-coded 1. * = p < .10. ** = p < .05.

International Journal of Selection and Assessment Volume 21 Number 1 March 2013

2013 Blackwell Publishing Ltd.

Individual Differences and Error Management Training

13

Declarative knowledge acquisition, net of control variables

Error management training (r = .27)

Behavioral modeling training (r = -.19)

Conscientiousness
Figure 1. Interaction between conscientiousness and training method for declarative knowledge acquisition.

.044, p < .05). The interaction is displayed graphically in Figure 1, which indicates that trainees who are higher in conscientiousness obtained higher scores on the measure of declarative knowledge acquisition in the EMT condition than in the BMT condition, and that trainees who are lower in conscientiousness obtained higher scores on this measure in the BMT condition. For the dependent variable declarative knowledge retention, the interaction between conscientiousness and training method was marginally signicant (DR2 = .023, p < .10). The interaction is displayed graphically in Figure 2, and indicates a similar pattern of results to that reected in Figure 1. As Table 4 reects, the hierarchical results support the presence of a marginally signicant interaction between extraversion and training method for declarative knowledge acquisition (DR2 = .023, p < .10). This interaction is displayed graphically in Figure 3, and indicates a similar pattern of interactions to those found for conscientiousness. Figure 3 illustrates that participants who are more extraverted scored higher on the measure of declarative knowledge acquisition in the EMT condition than in the BMT condition, and that participants who are more introverted scored better on this measure in the BMT condition. Table 5 indicates that the interactions between extraversion and procedural knowledge acquisition (b = .18, t = 1.47, p = .14), and extraversion and transfer perform-

ance (b = .20, t = 1.57, p = .11) were close to being statistically signicant. The form of these nonsignicant interactions was similar to the forms displayed in Figures 1, 2, and 3.

4. Discussion
The purpose of this research was to determine whether the effectiveness of an error management approach to training an important interpersonal skill such as negotiation is moderated by individual differences in conscientiousness, extraversion, or openness. To address this question, we created two negotiation training programs, which contained key features of BMT and EMT training methods, and assessed declarative knowledge acquisition, declarative knowledge retention, task performance, and transfer performance at different points in time. Interestingly, although we do not form any hypotheses concerning the main effects of training method on any of the learning outcomes, the results indicated that neither training method was superior to the other in developing declarative knowledge acquisition, procedural knowledge acquisition, declarative knowledge retention, or transfer performance. These ndings are interesting, but for at least two reasons, they are not necessarily surprising. First, both training methods have

2013 Blackwell Publishing Ltd.

International Journal of Selection and Assessment Volume 21 Number 1 March 2013

14

Michael J. Cullen, John P. Muros, Rena Rasch and Paul R. Sackett

Declarative knowledge retention, net of control variables

Error management training (r = .29)

Behavioral modeling training (r = -.04)

Conscientiousness
Figure 2. Interaction between conscientiousness and training method for declarative knowledge retention.

Declarative knowledge acquisition, net of control variables

Error management training (r = .12)

Behavioral modeling training (r = -.13)

Extraversion
Figure 3. Interaction between extraversion and training method for declarative knowledge acquisition.

International Journal of Selection and Assessment Volume 21 Number 1 March 2013

2013 Blackwell Publishing Ltd.

Individual Differences and Error Management Training


Table 5. Nonsignicant interactions for moderator variables investigated Moderator variable Declarative knowledge acquisition 1. Openness 2. Emotional stability 3. Agreeableness Procedural knowledge acquisition 1. Conscientiousness 2. Extraversion 3. Openness 4. Emotional stability 5. Agreeableness Declarative knowledge retention 1. Extraversion 2. Openness 3. Emotional stability 4. Agreeableness Transfer performance 1. Conscientiousness 2. Extraversion 3. Openness 4. Emotional stability 5. Agreeableness b .05 .07 .15 .02 .18 .01 .05 .15 .05 -.02 .15 .14 .09 .20 -.09 -.01 .01 t .39 .59 1.20 .16 1.47 .07 .37 1.18 .39 -.14 1.10 1.03 .69 1.57 -.65 -.09 .04 p .70 .55 .23 .87 .14 .94 .71 .24 .69 .89 .27 .30 .49 .11 .52 .93 .97

15

4.1. Interaction effects


Traditionally, the study of ATI has been largely ignored in the training domain, perhaps because the interactional portion of the variance accounted for by ATI is quite small (Warr & Allen, 1998). However, in recent years, training researchers have called for more study of ATI (Campbell, 1988, 1989; Campbell & Kuncel, 2002; Ford & Weissbein, 1997; Tannenbaum & Yukl, 1992). In training research, ATI are potentially important because, depending on the form of the interaction, they may suggest differential placement of trainees to training methods is warranted (Campbell, 1989; Campbell & Kuncel, 2002). For instance, in the case of a disordinal interaction between training method and aptitude (i.e., where the regression lines relating learning outcome and aptitude cross for two different treatments), trainees with high standing on a given characteristic will learn more when taught using one of the training methods, while trainees with lower standing on that characteristic will learn more when taught using the other training method. In this study, we obtained such disordinal interactions between training method and conscientiousness for (1) declarative knowledge acquisition and (2) declarative knowledge retention, and between training method and extraversion for declarative knowledge acquisition. Potentially, therefore, if these interactions could be reliably replicated, they may provide a rationale for differential placement of trainees into the two training conditions. The interactions displayed in Figures 1 and 2 between conscientiousness and the two learning outcomes suggest that highly conscientious individuals learned and recalled negotiation strategies and behaviors better in the EMT condition than in the BMT condition, and that individuals lower in conscientiousness learned and recalled the strategies and behaviors better in the BMT condition than in the EMT condition. In attempting to understand these ndings, it is important to appreciate the magnitude of the differences in the relationships between conscientiousness and declarative knowledge acquisition and retention for the two training conditions. For instance, for the outcome declarative knowledge acquisition, the correlation between conscientiousness and learning in the EMT condition is r = .27. In contrast, in the BMT condition, the correlation is r = -.19. Similarly, for the outcome declarative knowledge retention, the correlation between conscientiousness and learning is r = .29 in the EMT condition, and r = -.04 in the BMT condition. The strong, negative relationship between conscientiousness and declarative knowledge acquisition is particularly surprising since two meta-analyses have estimated the relationship between conscientiousness and

Note: b = the standardized beta weight for the interaction between the moderator variable and training method in the nal regression models. For all variables, N = 125 for declarative knowledge acquisition; N = 122 for procedural knowledge acquisition; N = 119 for declarative knowledge retention; N = 119 for transfer performance.

well-established empirical records attesting to their effectiveness in promoting learning and transfer, albeit for a more restricted task domain for EMT (e.g., Keith & Frese, 2008; Taylor et al., 2005). Second, from a theoretical standpoint, both training methods can marshal persuasive arguments as to why they should be effective methods for training interpersonal skills. We suspect that these results may not be surprising to BMT and EMT researchers either. In their meta-analyses of EMT, Keith and Frese (2008) suggested there may not be much difference between the amount of short-term learning and performance in EMT and more proceduralized training conditions. This is due to the fact that the difculties introduced by EMT during training, while important for promoting knowledge structures used during transfer, may not assist with short-term learning and performance. Regarding transfer, Keith and Frese (2008) argued that the cognitive processes elicited by EMT, such as metacognition, should be more helpful for developing knowledge structures used in structurally distinct adaptive transfer tasks than for structurally similar analogical transfer tasks such as the one used in this study. Accordingly, Keith and Frese (2008) hypothesized that there may be few differences between EMT and BMT for analogical transfer.

2013 Blackwell Publishing Ltd.

International Journal of Selection and Assessment Volume 21 Number 1 March 2013

16 learning to be between .31 and .39 (Barrick & Mount, 1991; Salgado, 1997). What could account for the rather surprising form of these disordinal interactions in Figures 1 and 2? One possible theoretical explanation is suggested by earlier ATI studies in the eld of strategy training, which found a positive relationship between cognitive ability and learning in one training condition, and a negative relationship between cognitive ability and learning in the other condition (e.g., Dansereau et al., 1979; Gray, 1983; Rigney, Munro, & Crook, 1979). At the time, such ndings were very puzzling because cognitive ability was presumed to be an effective predictor of learning outcomes regardless of the training method employed. In one representative study from this literature, Dansereau et al. (1979) randomly assigned students to a control condition in which no strategy training occurred, and a second condition in which a learning strategy called networking was taught. In the networking condition, students were shown how to transform text into conceptual networks. Students were later tested in their knowledge of conceptual networks, and an interesting ATI appeared. For students in the control condition, there was a strong, positive correlation between cognitive ability and learning. In the strategy condition, however, a negative relationship between cognitive ability and learning was observed. Thus, strategy training had helped low-ability participants, but had somehow hurt high-ability students. Dansereau speculated that the results stemmed from the lowered motivation of high-ability individuals to learn the strategies; in brief, the strategies may have interfered with the preferences of high g participants to take the test the way they wanted to take it. Very similar results were obtained in a study by Rigney et al. (1979), in which researchers attempted to teach procedural and monitoring strategies for self-directed learning, and a study by Gray (1983), which sought to teach the strategies of grouping, elaborating, and reorganizing materials to be learned. Due to the foresight of the researchers in this last study, it was possible to obtain some insight from the high-ability participants themselves as to possible reasons for their poorer performance in the strategy training condition. In this study, high-ability students in the strategy condition complained that the strategies forced on them by the investigators had interfered with their ability to use the strategies they normally used. Thus, Dansereaus intuition about cognitive interference as an explanation for the disordinal ATI seemed to be correct. Snow and Lohman (1984) joined Dansereau in this interpretation of results. They theorized that a negative slope between cognitive ability and learning occurred when an instructional method made cognitive demands on

Michael J. Cullen, John P. Muros, Rena Rasch and Paul R. Sackett the learner that were inconsistent with the style of cognitive processing preferred by the learner (Snow & Lohman, 1984). In our research, it is possible that a similar kind of interference is taking place for highly conscientious trainees in the BMT condition. For instance, it is possible that the highly structured, organized nature of the BMT program interfered with conscientious trainees ability to use, and preference for using, their own sophisticated mechanisms for organizing the material. In effect, in the BMT condition, the trainer was being conscientious for trainees during the strategy training. Prior to training, the trainer had already carefully organized all of the strategies and behaviors, and proceeded in a step-by-step fashion to explain them one by one. Subsequently, the trainer led trainees through a carefully crafted set of role plays illustrating the behaviors. Then, trainees watched actors who modeled effective behaviors for them. In brief, much as the strategy training in earlier studies interfered with the preference of high-ability trainees to devise their own learning strategies, it is possible that the highly organized and structured BMT condition interfered with highly conscientious trainees preference for organizing the training material the way they wanted. In contrast, the EMT condition required trainees to be extremely conscientious in paying attention to the strategies they generated as a group, and then organizing them in a way that worked for them. In sum, conscientious individuals may have preferred the ability to freely use their own organizing mechanisms during the key strategy training portion of the EMT condition. In contrast, less conscientious individuals may have preferred having the trainer organize and make sense of the materials for them, as was done in the strategy training portion of the BMT condition. If this hypothesis is correct, we would expect conscientious trainees to be more satised with EMT as an instructional method than BMT. The results for training program satisfaction tend to support this. In the EMT condition, the correlation between conscientiousness and self-reported satisfaction with the instructional method was r = .26; in the BMT condition, this correlation was r = -.22. More generally, the correlation between conscientiousness and overall satisfaction with the training program was r = .26 in the EMT condition and -.18 in the BMT condition. Interestingly, the hypothesized interactions for conscientiousness were not observed for procedural knowledge acquisition and transfer performance. One possibility is that any increased motivation to learn fostered by EMT for trainees high on conscientiousness heightened only those processes related to declarative knowledge acquisition and retention, such as attention, encoding, and retrieval of learned material, not processes related to skill performance, such as

International Journal of Selection and Assessment Volume 21 Number 1 March 2013

2013 Blackwell Publishing Ltd.

Individual Differences and Error Management Training automatization of learned behaviors (Anderson, 1982, 1989). We also found an interaction between training method and extraversion for declarative knowledge acquisition. The interactional pattern was similar to the ones found for conscientiousness. As Figure 3 indicates, highly extraverted trainees in the EMT condition achieved higher scores on the measure of declarative knowledge acquisition than highly extraverted trainees in the BMT condition. In contrast, more introverted trainees achieved higher scores on this measure in the BMT condition than in the EMT condition. Interestingly, as Table 5 makes clear, the interactions between extraversion and procedural knowledge acquisition and extraversion and transfer performance were very close to being signicant. Both of these nonsignicant interactions took the same form as the ones in Figure 3. Theoretically, there are two possible explanations for these results. One possibility is that extraverts, because of their active stance toward learning, preferred the EMT training program because of its more active, exploratory approach to learning the key negotiation skills than the more didactic, passive BMT approach to learning those skills during the key study manipulation where trainees learned the negotiation strategies. If this hypothesis is correct, we would expect extraverted trainees to be more satised with EMT as an instructional method than BMT. The results for training program satisfaction, while not conclusive, are suggestive. In the EMT condition, the correlation between extraversion and overall satisfaction with the training program was r = .14; in the BMT condition, it was -.14. A second possible reason for the observed interaction between extraversion and training method has to do with the fact that more active or stimulating training environments facilitate greater learning in extraverts than introverts, and less-active or stimulating environments stimulate greater learning in introverts. Past studies have demonstrated that learning and performance are higher for extraverts in more stimulating environments, and higher for introverts in less-stimulating environments. For instance, in an early study, Revelle, Amaral, and Turriff (1976) examined performance on graduate record examination (GRE) verbal items under a relaxed condition, a time pressure condition, and a time pressure condition in which participants were given 200 mg of caffeine. Results indicated that the performance of introverts decreased by 0.6 SDs over the three conditions, while the performance of extraverts increased by a like amount over those conditions. Similar results were found by other researchers (e.g., Anderson, 1985; Gilliland, 1976) for low to moderately high levels of arousal. However, at very high levels of arousal, the performance of extraverts decreased.

17 The theoretical explanation for these ndings rests on two postulates, each one of which has received a great deal of empirical support: (1) introverts have a higher resting state of arousal (i.e., resting internal stimulation) than extraverts (Eysenck & Eysenck, 1985) and (2) learning and performance are curvilinearly related (in an inverted U shape) to arousal (i.e., resting internal stimulation) level. The rst postulate was introduced by Eysenck, who argued that general stimulation level represents nonspecic activity in the ascending reticular activating system (Eysenck, 1967). The second postulate is a restatement of the YerkesDodson law (Yerkes & Dodson, 1908). The mechanisms responsible for the relationships underlying the YerkesDodson law have not been denitively identied, but could include distractibility, narrower cue utilization, and decreased efciency in storing, retrieving and rehearsing learned information in very high states of stimulation (Easterbrook, 1959; Eysenck, 1974). In combination, these postulates predict that introverts will learn and perform better in nonstimulating environments than stimulating ones. If the learning environment is very active and stimulating, the introverts relatively high resting state of internal arousal may be elevated to levels where learning is compromised. In contrast, if the learning environment is unstimulating, the extraverts relatively lower resting state of internal arousal may not be elevated to levels where learning is optimized. In our research, trainees responses to questions in a posttraining questionnaire indicate that they experienced the EMT condition to be quite a bit more active, and therefore, presumably, more stimulating and arousing, than the BMT condition. For instance, trainees mean rating for the statement The training program was a very active learning process was 5.48 in the EMT condition and 5.00 in the BMT condition (t = 2.15, p < .05; d = .39). In addition, trainees mean rating for the statement The training program really kept us busy with learning activities was 5.41 in the EMT condition and 4.89 in the BMT condition (t = 2.10, p < .05; d = .39). In this research, the heightened state of activity in the EMT condition may have led to more optimal levels of stimulation, and thus more optimal learning, for extraverts than introverts. In contrast, the less-active BMT condition may have provided the optimal level of arousal for introverts than extraverts, leading to greater learning for introverts, rather than extraverts, in the BMT condition. According to the Yerkes Dodson law, however, this explanation would only make sense if stimulatory levels in the EMT condition were moderate, not high. In accordance with this requirement, trainees moderately high mean ratings for the two questions inquiring about activity level suggest more moderate, rather than high, levels of stimulation in the EMT condition.

2013 Blackwell Publishing Ltd.

International Journal of Selection and Assessment Volume 21 Number 1 March 2013

18

Michael J. Cullen, John P. Muros, Rena Rasch and Paul R. Sackett bonuses. Third, we included negotiation tasks (e.g., the purchase and sale of manufactured items, and the negotiation for a salary compensation package) that would be relevant to workers in many organizations. However, it is possible that results may differ using an even more highly motivated group of participants, such as employees in organizations. More generally, our results are limited to the very narrowly dened interpersonal skill of negotiation in mixed-motive contexts. It is not necessarily the case that the results obtained in this research will generalize to other important interpersonal skills, such as supervisory or teamwork skills. A third limitation of this research relates to the duration of the training. For practical purposes, we limited this training to one 3-hour session. It is not clear whether the results obtained in this research would obtain if training was distributed over the course of several sessions, or if trainees were trained to mastery (e.g., May & Kahnweiler, 2000). A fourth limitation relates to the differences in structure in the two conditions. Hypothesized interactions for openness were premised on important differences in structure in the two conditions. Because those differences were largely conned to the key training component when negotiation strategies training took place, it may be that the differences in structure were not sufcient to obtain the expected interactions for openness. Finally, since we did not include pretraining measures of negotiation knowledge and skill, we are not able to claim with absolute certainty that either training method actually increased negotiation knowledge or skill at all. However, we believe it is reasonable to assume that scores on any such premeasures would have been close to zero. For instance, it is doubtful that anyone who had not undergone the specialized training in this research would be able to name the eight negotiation strategies or 18 key behaviors. In sum, BMT and EMT have both amassed signicant empirical and theoretical support for the proposition that they are effective methods for training different kinds of skills. This study suggests both methods can be used effectively for training interpersonal skills. However, it also suggests that neither EMT nor BMT may be the best method for everyone. Future research should continue to explore aptitudetreatment interactions using an expanded set of outcomes variables, such as motivation to learn.

4.2. Contributions, limitations, and directions for future research


This research makes several important contributions. From a practical perspective, this study reinforces Gully et al.s (2002) conclusion that the effectiveness of EMT may depend on individual differences. Thus, this study reinforces the point stressed by many other researchers (Campbell, 1988, 1989; Campbell & Kuncel, 2002; Ford & Weissbein, 1997; Tannenbaum & Yukl, 1992) that it is important to resist a one size ts all approach to training. Individual differences can have an important impact on relevant training outcomes, and thus individual differences ought to be incorporated into the needs assessment process (Gully et al., 2002). This research also highlights the important role motivation to learn may play in learning when considered in combination with individual difference characteristics. Although the focus of this research was on how individual differences differentially affected learning in EMT and BMT conditions, several of our hypotheses focused on how such differences may differentially affect motivation to learn. Future research should examine more directly whether individual differences differentially affect motivation to learn in EMT and more proceduralized conditions by using motivation to learn as an outcome measure. Finally, this research examined both short- and long-term declarative knowledge and procedural knowledge outcomes for a complex interpersonal skill. Because of the time investment required for interpersonal skills training, and the logistical hurdles involved in having participants return weeks later for transfer assessments, the examination of such a diverse range of learning outcomes over an extended time period is rare for interpersonal skills training research. However, it was important to include all of these learning measures because training researchers have suggested different hypotheses concerning the effectiveness of each method depending on whether the learning outcome was declarative or procedural knowledge, near- or far-term transfer, or whether the transfer task was analogical rather than adaptive in nature (Ivancic & Hesketh, 1995/ 1996; Keith & Frese, 2008; Smith, Ford, & Kozlowski, 1997). This research contains some limitations. One relates to the generalizability of the results to other learner populations and other interpersonal skills. In designing this study, we made an effort to enhance the generalizability of the study ndings to real-world populations in three ways. First, by recruiting students from classes for which negotiation training would be a relevant learning experience (e.g., communication studies, persuasion, sales, and marketing), we attempted to obtain a highly motivated study population. Second, we attempted to increase the motivation of our study population even more by offering performance-related

Acknowledgements
We thank the US Army Research Institute for the Behavioral and Social Sciences for providing the funding for this study. The authors would like to thank Hannah Jackson Foldes, Emily Duehr, and Stacy Davies for their assistance with data collection.

International Journal of Selection and Assessment Volume 21 Number 1 March 2013

2013 Blackwell Publishing Ltd.

Individual Differences and Error Management Training

19
Colquitt, J. A., LePine, J. A., & Noe, R. A. (2000). Toward an integrative theory of training motivation: A meta-analytic path analysis of 20 years of research. Journal of Applied Psychology, 85, 678707. Costa, P. T., & McCrae, R. R. (1992). Revised NEO personality inventory and NEO ve-factor inventory professional manual. Odessa, FL: Psychological Assessment Resources. Craik, F. I. M., & Lockhart, R. S. (1972). Levels of processing: A framework for memory research. Journal of Verbal Learning and Verbal Behavior, 11, 671684. Cronbach, L. J., & Gleser, G. C. (1965). Psychological tests and personnel decisions. Urbana: University of Illinois Press. Daniels, R. L., & Stevens, J. P. (1976). The interaction between the internal-external locus of control and two methods of college instruction. American Educational Research Journal, 13, 103113. Dansereau, D. F., McDonald, B. A., Collins, D. W., Garland, J., Holley, C. D., Diekhoff, G. M., & Evans, S. H. (1979). Evaluation of a teaching strategy system. In H. F. ONeill & C. D. Spielberger (Eds.), Cognitive and affective learning strategies (pp. 343). New York: Academic Press. Debowski, S., Wood, R. E., & Bandura, A. (2001). Impact of guided exploration and enactive exploration on selfregulatory mechanisms and information acquisition through electronic search. Journal of Applied Psychology, 86, 1129 1141. Dunn, R. (1990). Rita Dunn answers questions on learning styles. Educational Leadership, 48, 1519. Easterbrook, J. A. (1959). The effect of emotion on cue utilization and the organization of behavior. Psychological Review, 66, 183201. Eysenck, H. J. (1967). The biological basis of personality. Springeld, IL: Charles C. Thomas. Eysenck, H. J. (1974). Individual differences in speed of retrieval from semantic memory. Journal of Research in Personality, 8, 302323. Eysenck, H. J., & Eysenck, M. W. (1985). Personality and individual differences. New York: Plenum Press. Fisher, R., Ury, W., & Patton, B. (1999). Getting to yes: Negotiating agreement without giving in (3rd ed.) New York: Penguin Books. Ford, J. K., & Weissbein, D. A. (1997). Transfer of training: An updated review and analysis. Performance Improvement Quarterly, 10, 2241. Fosnot, C. T. (1996). Constructivism: A psychological theory of learning. In C. T. Fosnot (Ed.), Constructivism: Theory, perspectives and practice (pp. 833). New York: Teachers College Press. Frese, M. (1995). Error management in training: Conceptual and empirical results. In C. Zucchermaglio, S. Bagnara, & S. U. Stucky (Eds.), Organizational learning and technological change, series F: Computer and systems sciences (Vol. 141, pp. 112124). Berlin, Germany: Springer. Frese, M., & Zapf, D. (1994). Action as the core of work psychology: A German approach. In H. C. Triandis, M. D. Dunnette, & L. M. Hough (Eds.), Handbook of industrial and organizational psychology (Vol. 4, pp. 271340). Palo Alto, CA: Consulting Psychologists Press. Gilliland, K. (1976). The Interactive Effect of IntroversionExtraversion with Caffeine-Induced Arousal on Verbal Perform-

References
Aiken, L., & West, S. (1991). Multiple regression: Testing and interpreting interactions. Newbury Park, CA: Sage. Anderson, J. R. (1982). Acquisition of cognitive skill. Psychological Review, 89, 369403. Anderson, J. R. (1989). A theory of human knowledge. Articial Intelligence, 40, 313351. Anderson, K. J. (1985). Impulsivity, Caffeine and Task Difculty: A within-subjects test of the YerkesDodson law. Unpublished doctoral dissertation, Colgate University. Arthur, W., Bennett, W., Edens, P. S., & Bell, S. T. (2003). Effectiveness of training in organizations: A meta-analysis of design and evaluation features. Journal of Applied Psychology, 88, 234245. Arvey, R. D., & Cole, D. A. (1991). Evaluating change due to training. In I. L. Goldstein (Ed.), Training and development in organizations (pp. 89117). San Francisco: Jossey-Bass. Bandura, A. (1965). Inuences of models reinforcement contingencies on the acquisition of imitative responses. Journal of Personality and Social Psychology, 1, 589595. Bandura, A. (1969). Principles of behavior modication. New York: Holt, Rinehart & Winston. Barrick, M. R., & Mount, M. K. (1991). The Big Five personality dimensions and job performance: A meta-analysis. Personnel Psychology, 44, 126. Bell, B. S., & Ford, J. K. (2007). Reactions to skill assessment: The forgotten factor in explaining motivation to learn. Human Resource Development Quarterly, 18, 3362. Bell, B. S., & Kozlowski, S. W. J. (2008). Active learning: Effects of core training design elements on self-regulatory processes, learning, and adaptability. Journal of Applied Psychology, 93, 267306. Brooks, J. G. (1990). Teachers and students: Constructivists forging new directions. Educational Leadership, 47, 6871. Burke, M. J., & Day, R. R. (1986). A cumulative study of the effectiveness of management training. Journal of Applied Psychology, 71, 232245. Campbell, J. P. (1988). Training design for performance improvement. In J. Campbell & R. J. Campbell (Eds.), Productivity in organizations (pp. 177215). San Francisco: Jossey-Bass. Campbell, J. P. (1989). The agenda for theory and research. In I. L. Goldstein (Ed.), Training and development in organizations (pp. 469486). San Francisco: Jossey-Bass. Campbell, J. P., & Kuncel, N. R. (2002). Individual and team training. In N. Anderson, D. S. Ones, H. K. Sinangil, & C. Viswesvaran (Eds.), Handbook of industrial, work and organizational psychology (pp. 278312). Thousand Oaks, CA: Sage. Carlson, R. A., Lundy, D. H., & Schneider, W. (1992). Strategy guidance and memory aiding in learning problem-solving skill. Human Factors, 34, 129145. Carter, M., & Beier, M. E. (2010). The effectiveness of error management training with working-aged adults. Personnel Psychology, 63, 641675. Chandler, P., & Sweller, J. (1991). Cognitive load theory and the format of instruction. Cognition and Instruction, 8, 293 332. Cohen, J., & Cohen, P. (1983). Applied multiple regression/ correlation analysis for the behavioral sciences. Hillsdale, NJ: Erlbaum.

2013 Blackwell Publishing Ltd.

International Journal of Selection and Assessment Volume 21 Number 1 March 2013

20
ance. Unpublished doctoral dissertation, Northwestern University. Gist, M. E., Stevens, C. K., & Bavetta, A. G. (1991). Effects of self-efcacy and post-training intervention on the acquisition and maintenance of complex interpersonal skills. Personnel Psychology, 44, 837861. Goldberg, L. R. (1999). A broad-bandwidth, public domain, personality inventory measuring the lower-level facets of several ve-factor models. In I. Mervielde, I. Deary, F. De Fruyt, & F. Ostendorf (Eds.), Personality psychology in Europe (Vol. 7, pp. 728). Tilburg, the Netherlands: Tilburg University Press. Goldstein, A. P., & Sorcher, M. A. (1974). Changing supervisory behavior. New York: Pergamon Press. Gray, L. E. (1983). Aptitude Constructs, Learning Processes and Achievement. Unpublished doctoral dissertation, Stanford University. Gully, S. M., Payne, S. C., Kiechel Koles, K. L., & Whiteman, J. K. (2002). The impact of error training and individual differences on training outcomes: An attribute-treatment interaction perspective. Journal of Applied Psychology, 87, 143 155. Guzzo, R. A., Jette, R. D., & Katzell, R. A. (1985). The effects of psychologically based intervention programs on worker productivity: A meta-analysis. Personnel Psychology, 38, 275 291. Heimbeck, D., Frese, M., Sonnentag, S., & Keith, N. (2003). Integrating errors into the training process. The function of error management instructions and the role of goal orientation. Personnel Psychology, 56, 333361. Horak, V. M., & Horak, W. J. (1982). The inuence of student locus of control and teaching method on mathematics achievement. Journal of Experimental Education, 51, 18 21. Horak, W. J., & Slobodzian, K. A. (1980). Inuence on instructional structure and locus of control on achievement of preservice elementary science teachers. Journal of Research in Science Teaching, 17, 213222. Hough, L. M., Eaton, N. K., Dunnette, M. D., Kamp, J. D., & McCloy, R. (1990). Criterion-related validities of personality constructs and the effect of response distortion on those validities. Journal of Applied Psychology, 75, 581595. Ivancic, K., & Hesketh, B. (1995/1996). Making the best of errors during training. Training Research Journal, 1, 103 125. Keith, N., & Frese, M. (2005). Self-regulation in error management training: Emotion control and metacognition as mediators of performance effects. Journal of Applied Psychology, 90, 677691. Keith, N., & Frese, M. (2008). Effectiveness of error management training: A meta-analysis. Journal of Applied Psychology, 93, 5969. Keith, N., Richter, T., & Naumann, J. (2010). Active/ exploratory training promotes transfer even in learners with low motivation and cognitive ability. Applied Psychology, 59, 97123. Kolb, D. A. (1984). Experiential learning. Englewood Cliffs, NJ: Prentice-Hall. Martocchio, J. J., & Webster, J. (1992). Effects of feedback and cognitive playfulness on performance in microcomputer software training. Personnel Psychology, 45, 553578.

Michael J. Cullen, John P. Muros, Rena Rasch and Paul R. Sackett


Mathieu, J. E., Martineau, J. W., & Tannenbaum, S. I. (1993). Individual and situational inuences on the development of self-efcacy: Implications for training effectiveness. Personnel Psychology, 46, 125147. May, G. L., & Kahnweiler, W. M. (2000). The effect of a mastery practice design on learning and transfer in behavior modeling training. Personnel Psychology, 53, 353373. McLeod, D. B., & Adams, V. M. (1980/1981). Locus of control and mathematics instruction: Three exploratory studies. Journal of Experimental Education, 49, 9499. Nordstrom, C. R., Wendland, D., & Williams, K. B. (1998). To err is human: An examination of the effectiveness of error management training. Journal of Business and Psychology, 12, 269282. Pasher, H., McDaniel, M., Rohrer, D., & Bjork, R. (2008). Learning styles: Concepts and evidence. Psychological Science in the Public Interest, 9, 103119. Pruitt, D. G., & Rubin, J. Z. (1986). Social conict: Escalation, stalemate, & settlement. New York, NY: Random House. Quiones, M. A. (1995). Pretraining context effects: Training assignment as feedback. Journal of Applied Psychology, 80, 226238. Raiffa, H. (1982). The art and science of negotiation. Cambridge, MA: Harvard University Press. Revelle, W., Amaral, P., & Turriff, S. (1976). Introversionextraversion, time-stress, and caffeine: Effect on verbal performance. Science, 192, 149150. Rigney, J. W., Munro, A., & Crook, D. E. (1979). Teaching task-oriented selective reading. A learning strategy. In H. F. O. Neill & C. D. Spielberger (Eds.), Cognitive and affective learning strategies (pp. 177206). New York: Academic Press. Rotter, J. B. (1966). Generalized expectancies of internal versus external control of reinforcements. Psychological Monographs, 80 (Whole no. 609). Russell, J. S., Wexley, K. N., & Hunter, J. E. (1984). Questioning the effectiveness of behavioral modeling training in an industrial setting. Personnel Psychology, 40, 455488. Salgado, J. F. (1997). The ve factor model of personality and job performance in the European Community. Journal of Applied Psychology, 82, 3043. Schiefele, U., Krapp, A., & Winteler, A. (1992). Interest as a predictor of academic achievement: A meta-analysis of research. In K. A. Renninger, S. Hidi, & A. Krapp (Eds.), The role of interest in learning and development (pp. 183211). Hillsdale, NJ: Erlbaum. Smith, E. M., Ford, J. K., & Kozlowski, S. W. J. (1997). Building adaptive expertise: Implications for training design strategies. In M. A. Quiones & A. Ehrenstein (Eds.), Training for a rapidly changing workplace (pp. 89118). Washington, DC: American Psychological Association. Snow, R. E. (1989). Cognitive-conative aptitude interactions in learning. In R. Kanfer & P. L. Ackerman (Eds.), Abilities, motivation, and methodology: The Minnesota symposium on learning and individual differences (pp. 435474). Hillsdale, NJ: Erlbaum. Snow, R. E., & Lohman, D. F. (1984). Toward a theory of cognitive aptitude for learning from instruction. Journal of Educational Psychology, 76, 347376. Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12, 257285.

International Journal of Selection and Assessment Volume 21 Number 1 March 2013

2013 Blackwell Publishing Ltd.

Individual Differences and Error Management Training


Tannenbaum, S. I., & Yukl, G. (1992). Training and development in work organizations. Annual Review of Psychology, 43, 399441. Taylor, P. J., Russ-Eft, D. F., & Chan, D. W. L. (2005). A metaanalytic review of behavioral modeling training. Journal of Applied Psychology, 90, 692709. Walton, R. E., & McKersie, R. B. (1965). A behavioral theory of labor negotiations: An analysis of a social interaction system. New York, NY: McGraw-Hill. Warr, P., & Allen, C. (1998). Learning strategies and occupational training. In C. L. Cooper & I. T. Robertson (Eds.), International Review of Industrial and Organizational Psychology, 13, 83121. Weissbein, D. A. (2000). Improving Training Effectiveness through Motivation: Creating a psychological states intervention. Unpublished doctoral dissertation, Michigan State University, MI.

21
Werner, J. M., OLeary-Kelly, A., Baldwin, T. T., & Wexley, K. N. (1994). Augmenting behavior-modeling training: Testing the effects of pre- and post- training interventions. Human Resource Development Quarterly, 5, 169183. Wexley, K. N., & Latham, G. P. (2002). Developing and training human resources in organizations (3d ed.). Upper Saddle River, NJ: Prentice Hall. Wood, R. E., Kakebeeke, B. M., Debowski, S., & Frese, M. (2000). The impact of enactive exploration on intrinsic motivation, strategy, and performance in electronic search. Applied Psychology: An International Review, 49, 263283. Yerkes, R. M., & Dodson, J. D. (1908). The relation of strength of stimuli to rapidity of habit-formation. Journal of Comparative Neurology and Psychology, 18, 459482.

2013 Blackwell Publishing Ltd.

International Journal of Selection and Assessment Volume 21 Number 1 March 2013

You might also like