Professional Documents
Culture Documents
Individual Differences in the Effectiveness of Error Management Training for Developing Negotiation Skills
Michael J. Cullen*, John P. Muros**, Rena Rasch*** and Paul R. Sackett****
*Human Resources Research Organization, 100 Washington Avenue South, Suite 1660, Minneapolis, MN 55401, USA. mcullen@humrro.org **AT&T, Dallas, TX 75202, USA ***Kenexa High Performance Institute (KHPI), Minneapolis, MN 55402, USA ****Department of Psychology, University of Minnesota, Minneapolis, MN 55455, USA
This study investigated whether the effectiveness of an error management approach to training negotiation knowledge and skill depended on individual differences in conscientiousness, extraversion, and openness to experience. Participants were randomly assigned to two training programs that incorporated key elements of an error management and behavioral modeling approach to training, and were trained in the complex interpersonal skill of negotiation. At the end of training, declarative knowledge acquisition, procedural knowledge acquisition, declarative knowledge retention, and transfer performance were assessed at different points in time. Results suggested that the effectiveness of the error management training program was dependent on individual levels of conscientiousness and extraversion. For several learning outcomes, the performance of highly conscientious and extraverted individuals was superior in the error management condition, while the performance of less conscientious and introverted individuals was superior in the behavioral modeling condition. The implications of these ndings, and suggestions for future research, are discussed.
1. Introduction
esearchers have recently become interested in investigating the effectiveness of error management training (EMT) for a variety of learning and transfer outcomes. EMT is a relatively new approach to skill acquisition pioneered by cognitive psychologists, primarily in the educational domain (e.g., Brooks, 1990; Fosnot, 1996). The central premise of EMT is that the learning of complex, cognitively laden skills is best accomplished in an environment in which trainees actively engage in exploration, problem solving, hypothesis testing, making mistakes, and learning how to recover from mistakes (Ivancic & Hesketh, 1995/1996). The focus in an errorbased approach is on reducing training content, increasing participant involvement, and encouraging errors. Theoretically, EMT is believed to lead to increased depth of processing of information in training and im-
proved encoding, retention, and recall of information later on (Chandler & Sweller, 1991; Craik & Lockhart, 1972). By forcing trainees to interrupt training and reect, errors are hypothesized to lead to better mental models of the material to be learned, and to increased practice of behaviors that are the source of the error. Finally, EMT may lead trainees to practice errorrecovery strategies, which may increase performance (Frese & Zapf, 1994; Heimbeck, Frese, Sonnentag, & Keith, 2003; Ivancic & Hesketh, 1995/1996). Recent studies have supported the effectiveness of EMT in fostering learning. Several studies have reported medium to large effect sizes for EMT relative to other proceduralized training methods that advocate step-bystep instructions and the avoidance of errors (Frese, 1995; Nordstrom, Wendland, & Williams, 1998; Wood, Kakebeeke, Debowski, & Frese, 2000). In addition, a recent meta-analysis of EMT by Keith and Frese (2008)
2013 Blackwell Publishing Ltd., 9600 Garsington Road, Oxford, OX4 2DQ, UK and 350 Main St., Malden, MA, 02148, USA
bs_bs_banner
2 found that deliberately incorporating errors into training can be an effective means of promoting learning. Across 24 studies that investigated the effectiveness of EMT, they found that EMT led to superior overall learning outcomes compared to proceduralized methods that discouraged errors or exploratory training methods that did not involve explicit encouragement of errors (d = 0.44). Importantly, however, the training tasks included in these earlier studies, and in the meta-analysis, were conned to a very narrow range of tasks. In most cases, the training involved using a new computer software package. None of the studies in the Keith and Frese (2008) meta-analysis examined the effectiveness of EMT for training a complex interpersonal skill. The purpose of this study is to investigate whether the effectiveness of EMT in training the complex interpersonal skill of negotiation is moderated by individual differences in conscientiousness, extraversion, and openness to experience. To examine this issue, we develop two training programs that incorporate key elements of an EMT approach to training interpersonal skills, and a behavioral modeling approach (BMT). Because several different viable EMT and BMT negotiation training programs could be developed using fewer or more of the key elements of an EMT and BMT approach thus potentially leading to different learning outcomes depending on which features are included in the training programs our goal is not to compare the effectiveness of these training programs themselves. Rather, we focus on the role of conscientiousness, extraversion, and openness as moderators of the effectiveness of an EMT approach to training interpersonal skills. This research therefore responds to Keith and Freses (2005) call to continue the effort to understand the individual differences potentially affecting the effectiveness of EMT training. Although some efforts have been made to investigate this issue (e.g., Carter & Beier, 2010; Gully, Payne, Kiechel Koles, & Whiteman, 2002; Keith, Richter, & Naumann, 2010), no studies to date have examined the moderating role of important individual difference variables for a complex interpersonal skill such as negotiation.
Michael J. Cullen, John P. Muros, Rena Rasch and Paul R. Sackett oral modeling instruction are taught by a trainer in a group setting. In that group, trainees (1) receive an introduction to a topic, (2) watch a model perform the desired behaviors, (3) discuss what the model did right and wrong, (4) practice the desired behaviors via role playing, and (5) receive feedback about their performance. Learning is hypothesized to take place as a result of attentional, retentional, and motivational processes invoked by the procedure. Introducing an individual to a topic invokes attentional processes. Similarly, watching, discussing, and practicing the relevant behaviors invokes retentional processes. Finally, the provision of feedback invokes a motivational process (Bandura, 1969). Meta-analytic reviews have generally supported the efcacy of behavioral modeling as a method of training interpersonal skills. In their recent meta-analysis of BMT, Taylor, Russ-Eft, and Chan (2005) found large effect sizes for BMT on interpersonal knowledge and skill outcomes. Among studies employing control groups, the mean population effect size estimate was slightly more than 1.0 standard deviation (SD), which is comparable to those found in earlier meta-analyses for similar criteria (Arthur, Bennett, Edens, & Bell, 2003; Burke & Day, 1986). The mean population effect size estimate for attitudes was smaller, but still substantial, with an average change of one third of a standard deviation. Finally, the estimated population effect size for on-the-job performance behavior was approximately one quarter of a SD. This effect is smaller than the results reported in other meta-analyses for this criterion (Arthur et al., 2003; Burke & Day, 1986; Guzzo, Jette, & Katzell, 1985), which the authors attribute to the inclusion of more recent studies showing smaller BMT effects on behavioral outcomes (May & Kahnweiler, 2000; Russell, Wexley, & Hunter, 1984; Werner, OLeary-Kelly, Baldwin, & Wexley, 1994), and the inclusion of a greater number of unpublished studies than previous meta-analyses. Although the Taylor et al. (2005) effect size estimates were positive for all criteria, there was sufcient remaining variance in training effects, after removing sampling error variance, to warrant a search for moderators. On theoretical grounds and on the basis of prior empirical ndings, Taylor et al. decided to investigate four training design characteristics that may moderate the validity of BMT: (1) learning points (use of retention aids; display of learning points during modeling), (2) characteristics of the modeling display (presentation of positive-only models vs. mixed models), (3) characteristics of the behavioral rehearsal (instruction to symbolically rehearse newly learned skills; coaching prior to behavioral rehearsal; use of trainee-generated vs. trainer-provided practice scenarios), and (4) the number of hours of training. Results supported the benecial effects of many of the training design characteristics that had been advocated in the literature, including using learning points presented as rule codes,
1.1. Training interpersonal skills using behavioral modeling and error management approaches
Despite the availability of many different methods for training interpersonal skills (e.g., lecture, modeling, tutorial, simulation, discovery, error-based learning), BMT has long been the most popular method. Literally, millions of managers have been taught supervisory and teamwork communication skills using BMT (Wexley & Latham, 2002). Behavioral modeling has its roots in social learning theory, which posits that learning can take place by virtue of watching others perform a behavior (Bandura, 1965). Typically, individuals receiving behavi-
Individual Differences and Error Management Training using mixed rather than positive-only models, and having trainees generate some of the scenarios they practice. Not unexpectedly, more hours of training was associated with greater development of procedural knowledge. In our study, we take these moderator ndings into account by attempting to include in our behavioral modeling program the key features found in the Taylor et al. meta-analysis to enhance learning and transfer. Although BMT has predominated as a method for training interpersonal skills, other methods have been used. For instance, interpersonal skills have been taught using the lecture method, programmed instruction, discussion, and various combinations of these methods. In their meta-analysis of the effectiveness of different training methods for teaching different categories of skills, Arthur et al. (2003) found a wide range of effect sizes for these training methods and combinations of methods for teaching interpersonal skills. The effect sizes varied considerably (i.e., from d = .22 to d = 1.44) depending on whether reaction, learning, or behavioral criteria were employed. Another method for training interpersonal skills that holds promise is EMT. In contrast to a BMT, in which the focus is on guiding learners in an errorless, stepby-step fashion through a preestablished set of training material, the focus in an error-based approach is on reducing training content, increasing participant involvement, and encouraging errors. In contrast to BMT, the learning processes underlying EMT are only just beginning to be understood. As mentioned earlier, the chief benet of an error-based approach is that it forces individuals to generate and test hypotheses about a learning situation, thus increasing the cognitive load and depth of processing (Chandler & Sweller, 1991). Greater depth of processing of information, in turn, is anticipated to lead to better encoding of information, and to better recall (Craik & Lockhart, 1972). It also may promote more controlled, as opposed to automatic, processing of information (Ivancic & Hesketh, 1995/1996) because the errors that occur as a result of the training increase the attentional resources that must be devoted to understanding the error. More controlled processing may in turn play a role in how well principles are learned (Sweller, 1988). A third argument for EMT is that errors are a source of negative feedback, and such negative feedback may be necessary for learning (Frese & Zapf, 1994; Heimbeck et al., 2003). Advocates of EMT acknowledge that there are potential downsides to an error-based approach to learning. EMT may create negative emotions and reduced selfefcacy, which could potentially decrease learning. To address this, advocates of EMT promote the use of error management instructions (Keith & Frese, 2005; Keith & Frese, 2008), which point out that errors are an expected part of the learning process and are important,
3 if not essential, to learning. In this manner, the error management instructions aim to reduce the potentially negative emotions associated with making errors. Another way to reduce the anxiety of EMT is to correct errors that do occur. Once errors occur, there are potentially two routes the training designer might take. One route is to allow trainees to correct the error on their own, unaided by the trainer (Nordstrom et al., 1998). A second approach is to assist trainees in the resolution of errors (Carlson, Lundy, & Schneider, 1992; Debowski, Wood, & Bandura, 2001), thus potentially alleviating some of the negative feelings associated with errors. In this study, we aim to reduce the negative emotions associated with making errors by incorporating the kind of error management instructions suggested by other researchers. Once errors have occurred, we follow the second approach to error correction; we provide a set of hints to help trainees get back on track during the training.
4 engage in, or are able to engage in, active experimentation and exploration of material. Put another way, learners may vary in either their preference for dealing with, or ability to handle, different levels of structure in a training program. Within the educational domain, the idea that individuals may learn better in environments that suit their preferences is well reected in the learning styles literature (Pasher, McDaniel, Rohrer, & Bjork, 2008). A number of different learning styles models have been proposed and several assessments based on these models have been created (e.g., Dunn, 1990; Kolb, 1984). Although there is little evidence that any of the learning styles reected in these models reliably interacts with an educational treatment, there is evidence for an ATI between level of structure in a training program and two important individual difference variables: cognitive ability and locus of control. One of the most consistent ndings in the ATI literature is that individuals higher on cognitive ability learn more when the level of structure during training is low, and those lower on cognitive ability learn more when the level of structure is high (Snow, 1989; Snow & Lohman, 1984). Methods that are low in structure present material at a relatively rapid pace, under incomplete learning conditions where the students must infer much of the instructional message for themselves (Snow, 1989; Snow & Lohman, 1984). In contrast, highly structured methods allow the instructor to maintain a high level of control over learning activities. One explanation for this consistent ATI is that high-ability learners may prefer to develop their own strategies for learning material, and a low-structure condition best facilitates such self-directed learning (Snow & Lohman, 1984). In contrast, low-ability learners may prefer to have the trainer organize and present the material for them (Snow & Lohman, 1984). These differences in learner preferences may in turn affect the motivation of high- and low-ability learners in high- and low-structure training environments, thereby leading to differential learning for each group in these two conditions. Similar ATI ndings have been made for locus of control, which refers to an individuals belief about whether his or her successes or failures are a consequence of internal or external factors (Rotter, 1966). A number of studies have found that students with an external locus of control demonstrated better understanding of material taught in highly guided instruction conditions, whereas students with an internal locus of control demonstrated better understanding in less highly guided conditions (Daniels & Stevens, 1976; Horak & Horak, 1982; Horak & Slobodzian, 1980; McLeod & Adams, 1980/1981). Like the ATI studies centering on cognitive ability, these studies suggest that differences in preferences for structure may differentially affect learning in training conditions high and low in structure.
Michael J. Cullen, John P. Muros, Rena Rasch and Paul R. Sackett In our study, the BMT and EMT conditions were both highly structured throughout the majority of the training. In both conditions, instructional tasks were broken down into small units, the trainer claried material frequently, and maintained control over the content and pace of learning. However, during the key study manipulation, when the negotiation strategies were taught, the level of structure was quite different between the programs. In the BMT condition, the negotiation strategies were taught in a highly structured way. The trainer introduced the negotiation strategies and associated key behaviors, reviewed them one by one with trainees, and presented trainees with video models enacting the strategies and key behaviors. In contrast, in the EMT condition, the trainer did not inform participants what the effective negotiation strategies and behaviors were, and no modeling videos were employed. Instead, participants endeavored to learn the strategies and behaviors on their own by participating in a novel group brainstorming exercise. Accordingly, the level of structure was much lower in the EMT condition during this training component. We believe this difference in the amount of exploration and experimentation encouraged by the two programs during this key training component may have differentially affected learning for trainees with differential standing on three important individual difference variables: conscientiousness, extraversion, and openness to experience. 1.2.1. Conscientiousness Conscientious individuals are described as efcient, organized, dutiful, thorough, self-disciplined, and deliberate (Costa & McCrae, 1992). The Barrick and Mount (1991), Hough, Eaton, Dunnette, Kamp, and McCloy (1990), and Salgado (1997) meta-analyses indicate that conscientiousness is positively related to training outcomes. Individuals high on conscientiousness are likely to pay attention to the training material, organize training content, and use the meta-cognitive strategies of planning, monitoring and evaluating to enhance their learning. Conscientiousness may also affect learning by affecting the motivation to learn via its effect on selfefcacy (Colquitt, Lepine, & Noe, 2000). More specic facets of conscientiousness, such as achievement orientation, are also positively related to the motivation to learn (Mathieu, Martineau, & Tannenbaum, 1993). Consideration of the preferences of trainees with higher and lower levels of conscientiousness suggests a moderator hypothesis for this study. Just as less highly guided classroom conditions may be preferred by individuals with a greater sense of control over their ability to learn (e.g., Horak & Horak, 1982), a training environment with more student-oriented learning, such as the key training component in the EMT condition, may allow highly conscientious trainees to learn the training material the way they prefer, by organizing the material them-
Individual Differences and Error Management Training selves and being self-disciplined and effortful in their approach to learning. This should motivate highly conscientious trainees, leading to greater learning for them than in the more highly structured BMT component dealing with the negotiation strategy training. In contrast, the BMT condition may appeal to trainees low in conscientiousness since it allows them to be less industrious by letting the trainer organize the key negotiation strategy content for them. Additionally, relative to the more structured BMT condition that requires less organization of material and self-discipline to learn the negotiation strategies, conscientious trainees organized, self-disciplined and deliberate approach to tasks may make them much better equipped than those lower in conscientiousness to formulate and execute effective learning strategies during the key low-structured EMT training segment. Thus we hypothesize H1: Conscientiousness will interact with training condition to inuence learning and performance of negotiation skill (EMT > BMT for trainees high in conscientiousness; BMT > EMT for trainees low in conscientiousness). 1.2.2. Extraversion Extraversion is the tendency to be sociable, assertive, active, and to experience positive emotions such as energy and zeal (Costa & McCrae, 1992). In the Barrick and Mount (1991) meta-analysis, the estimated population validity of extraversion was higher than for any other Big Five personality variables (r = .26). One reason extraverted trainees may excel in training is the active stance they take toward learning. Taking an active stance toward learning may improve learning and transfer by increasing higher order metacognitive activities such as planning, monitoring, and evaluating, which are in turn related to learning (Bell & Kozlowski, 2008; Keith & Frese, 2008). Consideration of the preference of extraverts for an active training program suggests a moderator hypothesis for this study. As is typical of all EMT programs, our training study required active exploration of alternative hypotheses regarding the training material to be learned. In particular, in the EMT condition, trainees learned the negotiation strategies by brainstorming with other trainees, and testing hypotheses about what the effective strategies might be. In contrast, in the BMT condition, those strategies were taught directly by the trainer. Thus, during the key manipulation in the study, trainees were much more actively involved in determining the training content in the EMT condition than in the BMT condition. Extraverts may have preferred this more active approach to learning the strategies than introverts, thereby increasing motivation to learn, and thus learning itself, in extraverts in the EMT condition. Another reason one might expect an interaction between our training conditions and ex-
5 traversion has to do with (1) the differing levels of activity in the two training conditions, and (2) the fact that introverts have a higher resting state of arousal than introverts (Eysenck & Eysenck, 1985). In line with the YerkesDodson law, which predicts a curvilinear relationship between arousal level and performance (Yerkes & Dodson, 1908), the less-active, and presumably lessstimulating or arousing BMT condition may be optimal for introverts, who already have a high resting state of arousal. In like fashion, the more active EMT condition may raise extraverts lower resting state of arousal to more optimal levels for learning. These considerations suggest the following hypothesis: H2: Extraversion will interact with training condition to inuence learning and performance of negotiation skill (EMT > BMT for trainees high in extraversion; BMT > EMT for trainees low in extraversion). 1.2.3. Openness to experience Openness to experience is the tendency to be imaginative, nonconforming, unconventional, autonomous, inventive, curious, and innovative (Costa & McCrae, 1992). At least three meta-analysis indicate that openness to experience is positively related to learning (Barrick & Mount, 1991; Hough et al., 1990; Salgado, 1997). A high capacity for deep processing of information seems likely for learners high in openness in light of their active approach to learning and willingness to try new things (Barrick & Mount, 1991). Since individuals higher in openness are imaginative, curious, and enjoy learning things in different ways (Costa & McCrae, 1992), they may prefer to learn the negotiation strategies in the less-structured EMT component dealing with this content than in the BMT component dealing with this content. In contrast, individuals lower in openness may prefer the traditional teacher-centered BMT approach for learning the strategies. As with conscientiousness, these differences in learner preferences may lead to differential motivation to learn, and thus learning itself, for trainees higher and lower in openness in the two conditions. In addition, relative to the more structured BMT condition in which the negotiation strategy content is fairly easily processed by everyone, trainees higher in openness may be much better able than trainees lower in openness to deeply process the key content in the lower structured component of the EMT condition (Gully et al., 2002). Thus we hypothesize H3: Openness to experience will interact with training condition to inuence learning and performance of negotiation skill (EMT > BMT for trainees high in openness; BMT > EMT for trainees low in openness). To test these hypotheses, we created comprehensive BMT and EMT programs to train negotiation skills. Our
6 central goal in creating these training programs was to create fair representations of an EMT and BMT approach to training an interpersonal skill such as negotiation. Accordingly, for each training method, we endeavored to incorporate as many BMT or EMT features as we could that had been advocated by research or theory. In past studies, hypotheses concerning the effectiveness of EMT compared to other training methods have differed depending on the learning outcome. For instance, in their meta-analysis of the effectiveness of EMT, Keith and Frese (2008) hypothesized that shortterm learning and performance may not be superior for trainees in an EMT condition than those in a more proceduralized, errorless, training condition. With regard to transfer, Keith and Frese (2008) hypothesized that EMT may be as effective as other methods that avoid errors on measures of analogical transfer, but more effective than such methods on measures of adaptive transfer. Because the relative efcacy of EMT and BMT may vary by type of learning outcome, we included four different measures of learning in this study, spread out over time: (1) declarative knowledge acquisition, (2) procedural knowledge acquisition, (3) declarative knowledge retention, and (4) transfer performance. Measures of declarative and procedural knowledge acquisition were administered immediately following training; measures of declarative knowledge retention and transfer performance were administered 3 weeks after training. While declarative knowledge measures were traditional paper-and-pencil measures, procedural knowledge measures were actual negotiation simulations that required trainees to display their knowledge. In sum, this study contributes to training research in the following ways. First, this study responds to the call of Keith and Frese (2005) to continue to investigate the moderating role of individual difference variables for EMT programs. Second, just as importantly, it investigates the moderating role of important variables in relation to an interpersonal skill. Almost all prior studies investigating either the effectiveness of EMT for learning, or possible mediators or moderators of EMT have done so using a fairly narrow range of computer tasks. Thus, as advocated by Keith and Frese (2005), this study begins to look at the effectiveness of EMT programs for a broader class of skills. Finally, this study examines the moderating effect of individual differences for an especially wide range of learning and transfer outcomes spread out over time.
Michael J. Cullen, John P. Muros, Rena Rasch and Paul R. Sackett ing was a relevant instructional experience, such as courses in communication studies, persuasion, sales, and marketing. Participants were paid for their participation in the study, and the top performers received additional cash prizes. The sample was 53.7% female, and the mean age was 24.16 (SD = 7.68). Approximately 84% of participants were White, 5.9% were African American, 7.4% were Asian, and 2.2% were Hispanic. To participate in the study, participants were required to speak English uently. Only participants with no prior negotiation training were eligible for the study.
7 scenarios depicted interactions with which participants would be familiar, such as bargaining for the purchase of an automobile, or negotiating with a partner about where to spend a vacation. The scenarios contained examples of both effective and ineffective behaviors. Following each role play, the trainer led a group discussion about key behaviors used, providing corrective and reinforcing feedback as appropriate. Participants then watched two videos to help them model the behaviors they had learned. The rst was a negative display and showed the actor failing to use any of the correct strategies. The second showed the actor using all of the strategies correctly. In the second video, as each strategy was used, it was ashed on the video. According to Taylor et al.s (2005) BMT metaanalysis, skill development is greatest when learning points are presented as rule codes rather than descriptions or summaries of what the actor did. Accordingly, in the video displays, the strategies were presented as rule codes. As each video was played, participants answered questions in their workbook about which strategies were used effectively and ineffectively. At the conclusion of each video presentation, the trainer led a discussion about which strategies were used effectively and ineffectively, again providing corrective and reinforcing feedback as appropriate.
2.2.2. Strategy training in the BMT condition The typical sequence of learning activities in BMT is provision of the skills behaviors to be learned, prior to, or along with modeling, and then practice with feedback from either trainees and/or the trainer (Taylor et al., 2005). Feedback serves a correctional function when trainees do not use the behaviors correctly, and a motivational function, through a focus on reinforcement (i.e., praise) when a trainee performs behaviors correctly (Goldstein & Sorcher, 1974). We made a concerted effort to include all of these typical features in the BMT program. In the BMT condition, the trainer rst reviewed the declarative knowledge to be learned: the eight negotiation strategies and associated key behaviors. For each strategy, the trainer rst presented a denition of the strategy, presented a rationale for its use, and explained the key behaviors associated with the strategy. Next, to assist in knowledge compilation, the trainer asked two volunteers to role play eight scenarios in which each of the eight strategies and its associated key behaviors were implemented. Each scenario dealt with one strategy and its associated behaviors. The
2.2.3. Strategy training in the EMT condition EMT contains three characteristics that distinguish it from more proceduralized methods such as BMT. First, participants are given minimal guidance and are encouraged to explore material on their own (Keith & Frese, 2008). Second, EMT creates a learning situation in which errors are likely to occur (Keith & Frese, 2008). Finally, EMT participants are explicitly informed of the positive value of errors and are provided with error management instructions to reduce the frustration and anxiety that may be induced by errors (Heimbeck et al., 2003; Keith & Frese, 2005). In designing the strategy training for the EMT condition, we made a concerted effort to include these typical design elements. In the EMT condition, the trainer did not review the eight strategies and associated behaviors with participants. Instead, she informed participants that they would learn the strategies on their own by participating in a group exercise. Prior to commencing the exercise, the trainer issued a variety of error management instructions to participants that were similar to those used in past EMT studies. For instance, she told participants Dont worry about making wrong guesses, We expect you to make errors during this process, and that errors will help you in the learning process (Debowski et al., 2001; Heimbeck et al., 2003; Keith & Frese, 2005). The trainer then began the group exercise. First, participants were given transcripts of the eight scenarios that were
8 role played in the BMT condition. Unlike the BMT scenarios, however, these scenarios contained correct implementations of the key behaviors only. Second, participants were asked to read the scenarios on their own and take notes on which strategy and key behaviors they believed were being used. Third, two volunteers role played each scenario, after which the trainer led a group discussion about which key behaviors had been used in that scenario. Feedback similar to that provided in the BMT condition was then provided. When participants generated an incorrect key behavior, the trainer told participants it was not the behavior she was looking for. When participants generated a correct behavior, reinforcing feedback was provided and the group had a discussion about why that negotiation behavior was useful in that situation. If, after three successive tries, the group failed to come up with a correct key behavior for a scenario, the trainer took the class to the narrow passage in the scenario where the key behavior was demonstrated, and asked them a series of questions about the passage. Once all the key behaviors for a given scenario had been identied, the trainer asked for volunteers to identify the overall strategy being employed in the scenario. When participants identied the correct content for the strategy, the trainer revealed this strategy on a PowerPoint slide. The entire process was repeated for all eight scenarios. In both training conditions, we created a mock negotiation role play to facilitate the development of procedural knowledge. The purpose of the role play was to give trainees practice in using the strategies and effective behaviors, and to provide informational feedback about their use of the strategies. The role play contained two roles: a sales representative for a company seeking to purchase photocopiers, and an agent for the vendor. To facilitate the role play, each role was provided with a set of background materials. Sales representatives were presented with a tentative offer by the vendor. They were instructed that their negotiation objective was to satisfy as many of their nancial and nonnancial interests as possible by using as many of the key negotiation behaviors they could. To that end, they were provided with a list of their nonnancial interests. The agent for the vendor was provided with background information about the tentative offer that had already been made, and a list of her own nonprice-based interests. The agent for the vendor was also provided with a role play script to guide the mock negotiation. The script was designed to loosely structure the dialogue so that the sales representative would have the opportunity to use all of the key negotiation strategies and behaviors. At the conclusion of the role plays, the trainer led a group discussion about (1) how the negotiation went, (2) which aspects of the negotiation were challenging, and (3) which specic strategies participants had struggled to use.
Michael J. Cullen, John P. Muros, Rena Rasch and Paul R. Sackett Following the practice role plays, trainees were asked to consider how setting goals for using the strategies and associated behaviors could be a useful tool for remembering the strategies, and motivating them to use the strategies, in the weeks to come. The study utilized a between-subjects design. In both training conditions, participants began the training session by completing a set of premeasures. Subsequently, they completed the training program, a set of postmeasures, a knowledge acquisition measure, and participated in a mock negotiation role play to assess procedural knowledge acquisition. Three weeks later, participants returned to complete a measure of declarative knowledge retention and to participate in a transfer role play assessment. Before the transfer assessment commenced, participants also completed a short questionnaire about whether, and how often, they had attempted to use the negotiation strategies since the training program. The long duration of this training program made it efcient to administer the training to groups of students. Accordingly, small groups, rather than individuals, were randomly assigned to one of the two experimental conditions. Groups were roughly equally heterogeneous in terms of their ethnic, gender, and age composition. In addition, none of the personality variables investigated in this study were signicantly correlated with training condition (see Table 2), providing further evidence of the success of random assignment. In order to increase trainee motivation to learn the negotiation strategies, participants were told in Part I of the training that the top three performers on the assessments would receive cash bonuses of $100, $50, and $25, respectively, and that if they scored in the top one third of participants on the assessments, they would be entered into a lottery for three more cash bonuses in the same amount. We asked participants in a posttraining questionnaire whether they were motivated by those potential bonuses to learn the negotiation strategies. The mean response to these questions on a 7-point Likert scale (1 = strongly disagree and 7 = strongly agree) were 4.90 (SD = 1.01) for the effect of the bonuses for the three top performers, and 4.60 (SD = 1.01) for the effect of the lottery bonuses, indicating that both bonuses had a moderately positive effect on motivation to learn the negotiation strategies.
2.3. Measures
2.3.1. Performance measures 2.3.1.1. Declarative knowledge acquisition. Declarative knowledge acquisition was assessed with a 30-item test of the extent to which participants had learned the eight negotiation strategies and the key behaviors associated with each of the strategies. Participants were asked to
10
.70 -.07 .02 .21 -.10 .09 .95 .40 .17 .06 .36
.81 .06
.89
Note: N = 127. Training condition was dummy-coded 0 for the BMT condition and 1 for the EMT condition. Positive correlations with training condition indicate the EMT condition scored higher. Internal consistency reliability coefcients are on the diagonal where appropriate. Correlations greater than .18 are signicant at p < .05.
list the eight negotiation strategies and key behaviors associated with each (1 point for each strategy and behavior, for 26 possible points), to identify which strategies and key behaviors they would use in specic negotiation settings (12 total points), and to identify strategy use in a set of scenarios (13 total points). The total score for this test was the sum of scores for these questions, for a total possible score of 51. Scores ranged from 5 to 51, with a mean of 28.31 (SD = 11.61). Coefcient alpha for this test was .74. 2.3.1.2. Declarative knowledge retention. Declarative knowledge retention was assessed with a 36-item test assessing the same elements as those measured on the test of declarative knowledge acquisition, but using different items to avoid confounding retention of knowledge and memory for specic test items. The total possible test score on this test was 53. Scores ranged from 5 to 44, with a mean of 24.88 (SD = 9.91). Coefcient alpha for this test was .67. 2.3.1.3. Procedural knowledge acquisition. Procedural knowledge acquisition was assessed using a mock negotiation role play exercise that took place immediately following training. The role play was structurally identical to the one participants had practiced during training, but involved the purchase and sale of computers instead of photocopiers. Role plays were conducted with one of four confederates, who recorded whether each of the 18 key behaviors was used and whether the behaviors were used in the proper sequence. The role play commenced with one of the four confederates playing the role of the vendor. The confederates had either recently attained a PhD in psychology or were advanced industrial organizational psychology graduate students. As in the negotiation scenario practiced during training,
the vendor led participants through a scripted role play designed to give participants the opportunity to use all of the negotiation strategies and associated behaviors. To make the negotiation realistic, the vendor occasionally lowered the offer price in response to the correct use of behaviors during the negotiation. However, the vendor did not lower the price each time a correct behavior was used. Given the large number of key behaviors to be displayed in the negotiation, we reasoned that it would be too taxing on the vendor, and too unrealistic, for the vendor to lower the price whenever one of the 18 available key behaviors was displayed. Accordingly, rather than assigning a score based on the nal offer price negotiated, as has been done in some previous studies (e.g., Gist, Stevens, & Bavetta, 1991), participants were scored on their actual use of the key behaviors. The mock role plays were not videotaped. Rather, confederates kept track of the use of key behaviors on a scoring protocol in real time as the negotiation proceeded. The scoring protocol was divided into six separate sections, as follows: (1) opening strategy: focusing the discussion on interests (three points), (2) second strategy: appealing to interests (11 points), (3) attack 1: threat to end negotiation (three points), (4) third set of strategies: moving past stubbornness (three points), (5) attack 2: expression of frustration (three points), and (6) attack 3: criticism of position (two points). In general, one point was awarded for each key behavior used. However, for the second strategy, appealing to interests, we granted one point for each win-win solution proposed. Because there were nine possible nancial and nonnancial interests capable of being exchanged, nine total points were available for proposing exchanges of these interests. Finally, four points were available for using behaviors within the rst two strategies in the right order, as
10 taught in training. Additional points were not awarded for repeated use of key behaviors. Ideally, to eliminate the possibility of rater effects, procedural knowledge acquisition would have been assessed by the same person. From a practical and logistical standpoint, however, this was not possible, so we used four different raters. We employed two strategies to control for rater effects. First, to eliminate the possibility that raters judgments of transfer performance might be affected by prior knowledge of performance on the measure of task performance, we ensured that for any one participant, different raters rated task performance and transfer performance. Second, all four raters underwent rater training. The total possible score for the test was 29. Scores ranged from 1 to 25, with a mean of 14.79 (SD = 5.80). Four confederates were used, and each one received training from the lead confederate on how to score the behaviors using videotaped sessions of practice role plays. Ratings were standardized within rater to control for any rater differences in scale use. Coefcient alpha for this test was .75. 2.3.1.4. Transfer. For the transfer assessment, participants negotiated a compensation package for a new job. Participants attempted to use as many key negotiation behaviors as possible to help them achieve all of their nancial and nonnancial objectives, and behaviors were scored by the confederate using a scoring document identical in its structure to the one used for scoring procedural knowledge acquisition. Because, structurally, the transfer task was similar to the negotiation task practiced during training, the transfer task was an analogical rather than an adaptive transfer task (Ivancic & Hesketh, 1995/1996; Keith & Frese, 2008). The total possible score for this test was also 29. Scores ranged from 1 to 28, with a mean of 15.12 (SD = 6.54). Once again, four confederates trained in scoring the relevant behaviors were used. Ratings were standardized within rater to control for any rater differences in scale use. Coefcient alpha for this test was .70.
Michael J. Cullen, John P. Muros, Rena Rasch and Paul R. Sackett (Gist et al., 1991; Martocchio & Webster, 1992; Quiones, 1995). Accordingly, we created two specic negotiation-related measures of self-efcacy. The rst measure was a six-item measure of pretraining selfefcacy for learning negotiation skills. Sample items included I believe that I can meet the challenges of this negotiation training program and I am certain that I can manage the requirements of this negotiation training. Coefcient alpha for this scale was .93. 2.4.2. Pretraining negotiation self-efcacy We used a six-item measure adapted from a similar measure created by Bell and Ford (2007). Sample items included I can meet the challenge of negotiating in everyday conditions and I can negotiate effectively without help from others. Coefcient alpha for this scale was .94. 2.4.3. Pretraining desire to learn negotiation strategies Interest in a domain has also been demonstrated to be positively related to learning. For instance, Schiefele, Krapp, and Winteler (1992) investigated over 121 interestachievement correlations and found a mean r of .32. Accordingly, we created two interest-related premeasures. The rst measure was a six-item measure of desire to learn negotiation strategies adapted from a negotiation study conducted by Weissbein (2000). Sample items included I am motivated to learn the skills emphasized in the training program and I am interested in learning the training material. On a 7-point Likert scale (1 = strongly disagree and 7 = strongly agree), the mean score on this measure was 5.93, indicating that participants had a strong pretraining desire to learn the negotiation skills. Coefcient alpha for this scale was .94. 2.4.4. Pretraining willingness to expend effort to learn negotiation strategies A six-item measure of willingness to expend effort to learn negotiation strategies was also adapted from a scale used by Weissbein (2000). Sample items included I intend to try my best to become a good negotiator in this training and I intend to work hard to learn the material in this training course. On a 7-point Likert scale (1 = strongly disagree and 7 = strongly agree), the mean score on this measure was 5.93, indicating that participants had a strong pretraining intention to work hard to learn the negotiation skills. Coefcient alpha for this scale was .97.
2.4. Premeasures
In this research, we created four premeasures, which we expected to be at least moderately positively related to one or more dependent variables. The purpose of creating these premeasures was to use them as covariates in the research, thereby increasing the overall power of the research to detect differences in learning outcomes due to the choice of training method (Arvey & Cole, 1991). 2.4.1. Pretraining self-efcacy for learning negotiation skills Self-efcacy has been demonstrated to be a signicant predictor of training outcomes in many previous studies
2.5. Postmeasures
Postmeasures were developed to permit the assessment of whether conscientiousness, extraversion, and openness moderated the effect of training condition on negotiation performance.
Individual Differences and Error Management Training 2.5.1. Personality Conscientiousness, extraversion, and openness were all measured using Goldbergs (1999) 50-item International Personality Item Pool. The scale is composed of 10 phrases related to standing on each of the Big Five traits conscientiousness, openness to experience, emotional stability, extraversion, and agreeableness. Goldberg (1999) reports the mean coefcient alpha for each of the ve scales (10 items each) to be .84, indicating an acceptable degree of internal consistency. The data for this study were consistent with this nding, with alphas of .81, .89, and .78 respectively, for the conscientiousness, extraversion, and openness scales.
11 tions are for the two measures of declarative knowledge (i.e., declarative knowledge acquisition and retention, r = .72) and the two measures of procedural knowledge (i.e., procedural knowledge acquisition and transfer performance, r = .71). However, the two measures of declarative knowledge are also strongly correlated with the measures of procedural knowledge since knowledge of the negotiation strategies was necessary to implement those strategies in the mock negotiation sessions. The correlations with individual difference variables were largely consistent with past ndings. As expected, openness was moderately correlated with three of the four negotiation performance measures (r = .19, .29, and .21, respectively, for declarative knowledge acquisition, procedural knowledge acquisition, and transfer performance). Somewhat unexpectedly, conscientiousness was only moderately positively correlated with one of the learning outcomes, declarative knowledge retention (r = .17). Extraversion was not correlated with any of the learning outcomes. 3.1.1. Main effects of training condition on negotiation performance Table 3 indicates that the coefcients for training condition were not signicant for any of the four outcome variables. Therefore, neither training method was superior to the other in promoting declarative knowledge acquisition, procedural knowledge acquisition, declarative knowledge retention, or transfer performance. 3.1.2. Interactions between training method and individual differences Hierarchical regression was used to test for potential interaction effects between training condition and conscientiousness, extraversion, and openness to experience. The covariates used in the analyses of main effects in Table 3 were entered individually rst. Subsequently, the training condition variable was entered, followed by the individual difference variable, followed by the interaction term. There was a statistically signicant interaction between training condition and conscientiousness for declarative knowledge acquisition (p < .05). In addition, the interaction between training condition and conscientiousness was marginally signicant in the hypothesized direction for declarative knowledge retention (p < .10). Finally, there was a marginally signicant interaction between training condition and extraversion for declarative knowledge acquisition (p < .10). Therefore, as described below, we found partial support for hypotheses 1 and 2. Table 4 displays the interactions for conscientiousness and extraversion. Results support the presence of an interaction between conscientiousness and training method for declarative knowledge acquisition (DR2 =
2.6. Analyses
Means, standard deviations, and correlations among all study variables were rst computed. To determine the effects of training condition on learning, the four negotiation performance indices were all separately regressed on training condition using a variety of covariates. We controlled for college afliation and class size, pretraining self-efcacy, and pretraining desire to learn in all analyses. While we intended to control for all four self-efcacy and motivation-related premeasures separately, subsequent analyses revealed that the two pretraining self-efcacy measures, and the two pretraining desire and willingness measures, were highly correlated (r = .67 and .86, respectively). Accordingly, to avoid analytical difculties arising from collinearity among these two sets of variables, we created two separate unit-weighted composites, one for the selfefcacy measures (i.e., a new pretraining self-efcacy composite) and one for the measures of desire and willingness (i.e., a new pretraining desire to learn composite). Power to detect a moderate (d = .5) effect size using a two-tailed test at alpha = .05 was .87 or higher for each of the four outcome variables. To examine whether the effectiveness of training methods was moderated by cognitive ability, openness to experience, or conscientiousness, the standard moderated multiple regression approach used in differential prediction research was used (Cohen & Cohen, 1983). As recommended by Aiken and West (1991), continuous predictor variables were centered prior to conducting the moderated regression analyses. Since we used a Big Five measure to assess the three personality variables for which we had specic hypotheses, we also conducted post hoc tests on the other Big Five traits.
12
Table 3. Effect of training method on declarative knowledge acquisition, procedural knowledge acquisition, declarative knowledge retention, and transfer performance Step and variable Declarative knowledge acquisition B (SE B) 0.01 (3.32)** -4.49 (3.34)** -0.38 (0.26)** 0.31 (0.57)** -0.27 (0.55)** R2 .218** B (SE B) 2.39 (2.17)** R2 .226** DR2 .008** Procedural knowledge acquisition -0.38 (0.29)** -0.32 (0.29)** -0.05 (0.02)** -0.05 (0.05)** 0.05 (0.05)** .168** -0.22 (0.19)** .178** .010** Declarative knowledge retention 1.19 (3.37)** -1.53 (3.10)** -0.32 (0.26)** -0.12 (0.54)** -0.07 (0.51)** .129** -1.57 (2.02)** .134** .005** Transfer performance
Step 1: Control variables College 1 College 2 Class size Pretraining self-efcacy Pretraining desire to learn
-0.23 (0.32)** -0.42 (0.30)** -0.03 (0.03)** -0.05 (0.05)** 0.02 (0.05)** .124** -.01 (0.19)** .124** .000**
Note: N = 126 for declarative knowledge acquisition; N = 122 for procedural knowledge acquisition; N = 119 for declarative knowledge retention and transfer performance. Betas are unstandardized coefcients. College 1 and College 2 are dummy variables; College 3 is the omitted reference variable. BMT = behavioral modeling training. EMT = error management training. BMT was dummy-coded 0 and EMT was dummy-coded 1. * = p < .05. ** = p < .01.
Table 4. Interactions between conscientiousness, extraversion, and training method for declarative knowledge acquisition and declarative knowledge retention Declarative knowledge acquisition Conscientiousness B (SE B) 2.26 (3.32)** -5.00 (3.30)** -0.23 (0.26)** 0.30 (0.56)** -0.47 (0.54)** R2 .208** B (SE B) 2.82 (2.13)** R2 .215** DR2 .007** B (SE B) -2.28 (2.12)** R2 .224** DR2 .009** B (SE B) 7.82 (2.97)** R2 .268** DR2 .044** Declarative knowledge retention Declarative knowledge acquisition Extraversion -0.56 (3.31)** -3.61 (3.34)** -0.45 (0.26)** 0.26 (0.60)** -0.31 (0.55)** .208** 1.95 (2.17)** .215** .007** -1.40 (1.79)** .219** .004** 4.45 (2.39)** .242** .023**
Step and variable Step 1: Control variables College 1 College 2 Class size Pretraining self-efcacy Pretraining desire to learn
2.64 (3.35)** -2.59 (3.04)** -0.22 (0.26)** -0.11 (0.52)** -.21 (0.50)** .129** -0.98 (1.97)** .134** .005** 0.68 (1.95)** .173** .039** 4.85 (2.74)** .196** .023**
Note: N = 125 for declarative knowledge acquisition; N = 119 for declarative knowledge retention. Betas are unstandardized coefcients. College 1 and College 2 are dummy variables; College 3 is the omitted reference variable. BMT = behavioral modeling training. EMT = error management training. BMT was dummy-coded 0 and EMT was dummy-coded 1. * = p < .10. ** = p < .05.
13
Conscientiousness
Figure 1. Interaction between conscientiousness and training method for declarative knowledge acquisition.
.044, p < .05). The interaction is displayed graphically in Figure 1, which indicates that trainees who are higher in conscientiousness obtained higher scores on the measure of declarative knowledge acquisition in the EMT condition than in the BMT condition, and that trainees who are lower in conscientiousness obtained higher scores on this measure in the BMT condition. For the dependent variable declarative knowledge retention, the interaction between conscientiousness and training method was marginally signicant (DR2 = .023, p < .10). The interaction is displayed graphically in Figure 2, and indicates a similar pattern of results to that reected in Figure 1. As Table 4 reects, the hierarchical results support the presence of a marginally signicant interaction between extraversion and training method for declarative knowledge acquisition (DR2 = .023, p < .10). This interaction is displayed graphically in Figure 3, and indicates a similar pattern of interactions to those found for conscientiousness. Figure 3 illustrates that participants who are more extraverted scored higher on the measure of declarative knowledge acquisition in the EMT condition than in the BMT condition, and that participants who are more introverted scored better on this measure in the BMT condition. Table 5 indicates that the interactions between extraversion and procedural knowledge acquisition (b = .18, t = 1.47, p = .14), and extraversion and transfer perform-
ance (b = .20, t = 1.57, p = .11) were close to being statistically signicant. The form of these nonsignicant interactions was similar to the forms displayed in Figures 1, 2, and 3.
4. Discussion
The purpose of this research was to determine whether the effectiveness of an error management approach to training an important interpersonal skill such as negotiation is moderated by individual differences in conscientiousness, extraversion, or openness. To address this question, we created two negotiation training programs, which contained key features of BMT and EMT training methods, and assessed declarative knowledge acquisition, declarative knowledge retention, task performance, and transfer performance at different points in time. Interestingly, although we do not form any hypotheses concerning the main effects of training method on any of the learning outcomes, the results indicated that neither training method was superior to the other in developing declarative knowledge acquisition, procedural knowledge acquisition, declarative knowledge retention, or transfer performance. These ndings are interesting, but for at least two reasons, they are not necessarily surprising. First, both training methods have
14
Conscientiousness
Figure 2. Interaction between conscientiousness and training method for declarative knowledge retention.
Extraversion
Figure 3. Interaction between extraversion and training method for declarative knowledge acquisition.
15
Note: b = the standardized beta weight for the interaction between the moderator variable and training method in the nal regression models. For all variables, N = 125 for declarative knowledge acquisition; N = 122 for procedural knowledge acquisition; N = 119 for declarative knowledge retention; N = 119 for transfer performance.
well-established empirical records attesting to their effectiveness in promoting learning and transfer, albeit for a more restricted task domain for EMT (e.g., Keith & Frese, 2008; Taylor et al., 2005). Second, from a theoretical standpoint, both training methods can marshal persuasive arguments as to why they should be effective methods for training interpersonal skills. We suspect that these results may not be surprising to BMT and EMT researchers either. In their meta-analyses of EMT, Keith and Frese (2008) suggested there may not be much difference between the amount of short-term learning and performance in EMT and more proceduralized training conditions. This is due to the fact that the difculties introduced by EMT during training, while important for promoting knowledge structures used during transfer, may not assist with short-term learning and performance. Regarding transfer, Keith and Frese (2008) argued that the cognitive processes elicited by EMT, such as metacognition, should be more helpful for developing knowledge structures used in structurally distinct adaptive transfer tasks than for structurally similar analogical transfer tasks such as the one used in this study. Accordingly, Keith and Frese (2008) hypothesized that there may be few differences between EMT and BMT for analogical transfer.
16 learning to be between .31 and .39 (Barrick & Mount, 1991; Salgado, 1997). What could account for the rather surprising form of these disordinal interactions in Figures 1 and 2? One possible theoretical explanation is suggested by earlier ATI studies in the eld of strategy training, which found a positive relationship between cognitive ability and learning in one training condition, and a negative relationship between cognitive ability and learning in the other condition (e.g., Dansereau et al., 1979; Gray, 1983; Rigney, Munro, & Crook, 1979). At the time, such ndings were very puzzling because cognitive ability was presumed to be an effective predictor of learning outcomes regardless of the training method employed. In one representative study from this literature, Dansereau et al. (1979) randomly assigned students to a control condition in which no strategy training occurred, and a second condition in which a learning strategy called networking was taught. In the networking condition, students were shown how to transform text into conceptual networks. Students were later tested in their knowledge of conceptual networks, and an interesting ATI appeared. For students in the control condition, there was a strong, positive correlation between cognitive ability and learning. In the strategy condition, however, a negative relationship between cognitive ability and learning was observed. Thus, strategy training had helped low-ability participants, but had somehow hurt high-ability students. Dansereau speculated that the results stemmed from the lowered motivation of high-ability individuals to learn the strategies; in brief, the strategies may have interfered with the preferences of high g participants to take the test the way they wanted to take it. Very similar results were obtained in a study by Rigney et al. (1979), in which researchers attempted to teach procedural and monitoring strategies for self-directed learning, and a study by Gray (1983), which sought to teach the strategies of grouping, elaborating, and reorganizing materials to be learned. Due to the foresight of the researchers in this last study, it was possible to obtain some insight from the high-ability participants themselves as to possible reasons for their poorer performance in the strategy training condition. In this study, high-ability students in the strategy condition complained that the strategies forced on them by the investigators had interfered with their ability to use the strategies they normally used. Thus, Dansereaus intuition about cognitive interference as an explanation for the disordinal ATI seemed to be correct. Snow and Lohman (1984) joined Dansereau in this interpretation of results. They theorized that a negative slope between cognitive ability and learning occurred when an instructional method made cognitive demands on
Michael J. Cullen, John P. Muros, Rena Rasch and Paul R. Sackett the learner that were inconsistent with the style of cognitive processing preferred by the learner (Snow & Lohman, 1984). In our research, it is possible that a similar kind of interference is taking place for highly conscientious trainees in the BMT condition. For instance, it is possible that the highly structured, organized nature of the BMT program interfered with conscientious trainees ability to use, and preference for using, their own sophisticated mechanisms for organizing the material. In effect, in the BMT condition, the trainer was being conscientious for trainees during the strategy training. Prior to training, the trainer had already carefully organized all of the strategies and behaviors, and proceeded in a step-by-step fashion to explain them one by one. Subsequently, the trainer led trainees through a carefully crafted set of role plays illustrating the behaviors. Then, trainees watched actors who modeled effective behaviors for them. In brief, much as the strategy training in earlier studies interfered with the preference of high-ability trainees to devise their own learning strategies, it is possible that the highly organized and structured BMT condition interfered with highly conscientious trainees preference for organizing the training material the way they wanted. In contrast, the EMT condition required trainees to be extremely conscientious in paying attention to the strategies they generated as a group, and then organizing them in a way that worked for them. In sum, conscientious individuals may have preferred the ability to freely use their own organizing mechanisms during the key strategy training portion of the EMT condition. In contrast, less conscientious individuals may have preferred having the trainer organize and make sense of the materials for them, as was done in the strategy training portion of the BMT condition. If this hypothesis is correct, we would expect conscientious trainees to be more satised with EMT as an instructional method than BMT. The results for training program satisfaction tend to support this. In the EMT condition, the correlation between conscientiousness and self-reported satisfaction with the instructional method was r = .26; in the BMT condition, this correlation was r = -.22. More generally, the correlation between conscientiousness and overall satisfaction with the training program was r = .26 in the EMT condition and -.18 in the BMT condition. Interestingly, the hypothesized interactions for conscientiousness were not observed for procedural knowledge acquisition and transfer performance. One possibility is that any increased motivation to learn fostered by EMT for trainees high on conscientiousness heightened only those processes related to declarative knowledge acquisition and retention, such as attention, encoding, and retrieval of learned material, not processes related to skill performance, such as
Individual Differences and Error Management Training automatization of learned behaviors (Anderson, 1982, 1989). We also found an interaction between training method and extraversion for declarative knowledge acquisition. The interactional pattern was similar to the ones found for conscientiousness. As Figure 3 indicates, highly extraverted trainees in the EMT condition achieved higher scores on the measure of declarative knowledge acquisition than highly extraverted trainees in the BMT condition. In contrast, more introverted trainees achieved higher scores on this measure in the BMT condition than in the EMT condition. Interestingly, as Table 5 makes clear, the interactions between extraversion and procedural knowledge acquisition and extraversion and transfer performance were very close to being signicant. Both of these nonsignicant interactions took the same form as the ones in Figure 3. Theoretically, there are two possible explanations for these results. One possibility is that extraverts, because of their active stance toward learning, preferred the EMT training program because of its more active, exploratory approach to learning the key negotiation skills than the more didactic, passive BMT approach to learning those skills during the key study manipulation where trainees learned the negotiation strategies. If this hypothesis is correct, we would expect extraverted trainees to be more satised with EMT as an instructional method than BMT. The results for training program satisfaction, while not conclusive, are suggestive. In the EMT condition, the correlation between extraversion and overall satisfaction with the training program was r = .14; in the BMT condition, it was -.14. A second possible reason for the observed interaction between extraversion and training method has to do with the fact that more active or stimulating training environments facilitate greater learning in extraverts than introverts, and less-active or stimulating environments stimulate greater learning in introverts. Past studies have demonstrated that learning and performance are higher for extraverts in more stimulating environments, and higher for introverts in less-stimulating environments. For instance, in an early study, Revelle, Amaral, and Turriff (1976) examined performance on graduate record examination (GRE) verbal items under a relaxed condition, a time pressure condition, and a time pressure condition in which participants were given 200 mg of caffeine. Results indicated that the performance of introverts decreased by 0.6 SDs over the three conditions, while the performance of extraverts increased by a like amount over those conditions. Similar results were found by other researchers (e.g., Anderson, 1985; Gilliland, 1976) for low to moderately high levels of arousal. However, at very high levels of arousal, the performance of extraverts decreased.
17 The theoretical explanation for these ndings rests on two postulates, each one of which has received a great deal of empirical support: (1) introverts have a higher resting state of arousal (i.e., resting internal stimulation) than extraverts (Eysenck & Eysenck, 1985) and (2) learning and performance are curvilinearly related (in an inverted U shape) to arousal (i.e., resting internal stimulation) level. The rst postulate was introduced by Eysenck, who argued that general stimulation level represents nonspecic activity in the ascending reticular activating system (Eysenck, 1967). The second postulate is a restatement of the YerkesDodson law (Yerkes & Dodson, 1908). The mechanisms responsible for the relationships underlying the YerkesDodson law have not been denitively identied, but could include distractibility, narrower cue utilization, and decreased efciency in storing, retrieving and rehearsing learned information in very high states of stimulation (Easterbrook, 1959; Eysenck, 1974). In combination, these postulates predict that introverts will learn and perform better in nonstimulating environments than stimulating ones. If the learning environment is very active and stimulating, the introverts relatively high resting state of internal arousal may be elevated to levels where learning is compromised. In contrast, if the learning environment is unstimulating, the extraverts relatively lower resting state of internal arousal may not be elevated to levels where learning is optimized. In our research, trainees responses to questions in a posttraining questionnaire indicate that they experienced the EMT condition to be quite a bit more active, and therefore, presumably, more stimulating and arousing, than the BMT condition. For instance, trainees mean rating for the statement The training program was a very active learning process was 5.48 in the EMT condition and 5.00 in the BMT condition (t = 2.15, p < .05; d = .39). In addition, trainees mean rating for the statement The training program really kept us busy with learning activities was 5.41 in the EMT condition and 4.89 in the BMT condition (t = 2.10, p < .05; d = .39). In this research, the heightened state of activity in the EMT condition may have led to more optimal levels of stimulation, and thus more optimal learning, for extraverts than introverts. In contrast, the less-active BMT condition may have provided the optimal level of arousal for introverts than extraverts, leading to greater learning for introverts, rather than extraverts, in the BMT condition. According to the Yerkes Dodson law, however, this explanation would only make sense if stimulatory levels in the EMT condition were moderate, not high. In accordance with this requirement, trainees moderately high mean ratings for the two questions inquiring about activity level suggest more moderate, rather than high, levels of stimulation in the EMT condition.
18
Michael J. Cullen, John P. Muros, Rena Rasch and Paul R. Sackett bonuses. Third, we included negotiation tasks (e.g., the purchase and sale of manufactured items, and the negotiation for a salary compensation package) that would be relevant to workers in many organizations. However, it is possible that results may differ using an even more highly motivated group of participants, such as employees in organizations. More generally, our results are limited to the very narrowly dened interpersonal skill of negotiation in mixed-motive contexts. It is not necessarily the case that the results obtained in this research will generalize to other important interpersonal skills, such as supervisory or teamwork skills. A third limitation of this research relates to the duration of the training. For practical purposes, we limited this training to one 3-hour session. It is not clear whether the results obtained in this research would obtain if training was distributed over the course of several sessions, or if trainees were trained to mastery (e.g., May & Kahnweiler, 2000). A fourth limitation relates to the differences in structure in the two conditions. Hypothesized interactions for openness were premised on important differences in structure in the two conditions. Because those differences were largely conned to the key training component when negotiation strategies training took place, it may be that the differences in structure were not sufcient to obtain the expected interactions for openness. Finally, since we did not include pretraining measures of negotiation knowledge and skill, we are not able to claim with absolute certainty that either training method actually increased negotiation knowledge or skill at all. However, we believe it is reasonable to assume that scores on any such premeasures would have been close to zero. For instance, it is doubtful that anyone who had not undergone the specialized training in this research would be able to name the eight negotiation strategies or 18 key behaviors. In sum, BMT and EMT have both amassed signicant empirical and theoretical support for the proposition that they are effective methods for training different kinds of skills. This study suggests both methods can be used effectively for training interpersonal skills. However, it also suggests that neither EMT nor BMT may be the best method for everyone. Future research should continue to explore aptitudetreatment interactions using an expanded set of outcomes variables, such as motivation to learn.
Acknowledgements
We thank the US Army Research Institute for the Behavioral and Social Sciences for providing the funding for this study. The authors would like to thank Hannah Jackson Foldes, Emily Duehr, and Stacy Davies for their assistance with data collection.
19
Colquitt, J. A., LePine, J. A., & Noe, R. A. (2000). Toward an integrative theory of training motivation: A meta-analytic path analysis of 20 years of research. Journal of Applied Psychology, 85, 678707. Costa, P. T., & McCrae, R. R. (1992). Revised NEO personality inventory and NEO ve-factor inventory professional manual. Odessa, FL: Psychological Assessment Resources. Craik, F. I. M., & Lockhart, R. S. (1972). Levels of processing: A framework for memory research. Journal of Verbal Learning and Verbal Behavior, 11, 671684. Cronbach, L. J., & Gleser, G. C. (1965). Psychological tests and personnel decisions. Urbana: University of Illinois Press. Daniels, R. L., & Stevens, J. P. (1976). The interaction between the internal-external locus of control and two methods of college instruction. American Educational Research Journal, 13, 103113. Dansereau, D. F., McDonald, B. A., Collins, D. W., Garland, J., Holley, C. D., Diekhoff, G. M., & Evans, S. H. (1979). Evaluation of a teaching strategy system. In H. F. ONeill & C. D. Spielberger (Eds.), Cognitive and affective learning strategies (pp. 343). New York: Academic Press. Debowski, S., Wood, R. E., & Bandura, A. (2001). Impact of guided exploration and enactive exploration on selfregulatory mechanisms and information acquisition through electronic search. Journal of Applied Psychology, 86, 1129 1141. Dunn, R. (1990). Rita Dunn answers questions on learning styles. Educational Leadership, 48, 1519. Easterbrook, J. A. (1959). The effect of emotion on cue utilization and the organization of behavior. Psychological Review, 66, 183201. Eysenck, H. J. (1967). The biological basis of personality. Springeld, IL: Charles C. Thomas. Eysenck, H. J. (1974). Individual differences in speed of retrieval from semantic memory. Journal of Research in Personality, 8, 302323. Eysenck, H. J., & Eysenck, M. W. (1985). Personality and individual differences. New York: Plenum Press. Fisher, R., Ury, W., & Patton, B. (1999). Getting to yes: Negotiating agreement without giving in (3rd ed.) New York: Penguin Books. Ford, J. K., & Weissbein, D. A. (1997). Transfer of training: An updated review and analysis. Performance Improvement Quarterly, 10, 2241. Fosnot, C. T. (1996). Constructivism: A psychological theory of learning. In C. T. Fosnot (Ed.), Constructivism: Theory, perspectives and practice (pp. 833). New York: Teachers College Press. Frese, M. (1995). Error management in training: Conceptual and empirical results. In C. Zucchermaglio, S. Bagnara, & S. U. Stucky (Eds.), Organizational learning and technological change, series F: Computer and systems sciences (Vol. 141, pp. 112124). Berlin, Germany: Springer. Frese, M., & Zapf, D. (1994). Action as the core of work psychology: A German approach. In H. C. Triandis, M. D. Dunnette, & L. M. Hough (Eds.), Handbook of industrial and organizational psychology (Vol. 4, pp. 271340). Palo Alto, CA: Consulting Psychologists Press. Gilliland, K. (1976). The Interactive Effect of IntroversionExtraversion with Caffeine-Induced Arousal on Verbal Perform-
References
Aiken, L., & West, S. (1991). Multiple regression: Testing and interpreting interactions. Newbury Park, CA: Sage. Anderson, J. R. (1982). Acquisition of cognitive skill. Psychological Review, 89, 369403. Anderson, J. R. (1989). A theory of human knowledge. Articial Intelligence, 40, 313351. Anderson, K. J. (1985). Impulsivity, Caffeine and Task Difculty: A within-subjects test of the YerkesDodson law. Unpublished doctoral dissertation, Colgate University. Arthur, W., Bennett, W., Edens, P. S., & Bell, S. T. (2003). Effectiveness of training in organizations: A meta-analysis of design and evaluation features. Journal of Applied Psychology, 88, 234245. Arvey, R. D., & Cole, D. A. (1991). Evaluating change due to training. In I. L. Goldstein (Ed.), Training and development in organizations (pp. 89117). San Francisco: Jossey-Bass. Bandura, A. (1965). Inuences of models reinforcement contingencies on the acquisition of imitative responses. Journal of Personality and Social Psychology, 1, 589595. Bandura, A. (1969). Principles of behavior modication. New York: Holt, Rinehart & Winston. Barrick, M. R., & Mount, M. K. (1991). The Big Five personality dimensions and job performance: A meta-analysis. Personnel Psychology, 44, 126. Bell, B. S., & Ford, J. K. (2007). Reactions to skill assessment: The forgotten factor in explaining motivation to learn. Human Resource Development Quarterly, 18, 3362. Bell, B. S., & Kozlowski, S. W. J. (2008). Active learning: Effects of core training design elements on self-regulatory processes, learning, and adaptability. Journal of Applied Psychology, 93, 267306. Brooks, J. G. (1990). Teachers and students: Constructivists forging new directions. Educational Leadership, 47, 6871. Burke, M. J., & Day, R. R. (1986). A cumulative study of the effectiveness of management training. Journal of Applied Psychology, 71, 232245. Campbell, J. P. (1988). Training design for performance improvement. In J. Campbell & R. J. Campbell (Eds.), Productivity in organizations (pp. 177215). San Francisco: Jossey-Bass. Campbell, J. P. (1989). The agenda for theory and research. In I. L. Goldstein (Ed.), Training and development in organizations (pp. 469486). San Francisco: Jossey-Bass. Campbell, J. P., & Kuncel, N. R. (2002). Individual and team training. In N. Anderson, D. S. Ones, H. K. Sinangil, & C. Viswesvaran (Eds.), Handbook of industrial, work and organizational psychology (pp. 278312). Thousand Oaks, CA: Sage. Carlson, R. A., Lundy, D. H., & Schneider, W. (1992). Strategy guidance and memory aiding in learning problem-solving skill. Human Factors, 34, 129145. Carter, M., & Beier, M. E. (2010). The effectiveness of error management training with working-aged adults. Personnel Psychology, 63, 641675. Chandler, P., & Sweller, J. (1991). Cognitive load theory and the format of instruction. Cognition and Instruction, 8, 293 332. Cohen, J., & Cohen, P. (1983). Applied multiple regression/ correlation analysis for the behavioral sciences. Hillsdale, NJ: Erlbaum.
20
ance. Unpublished doctoral dissertation, Northwestern University. Gist, M. E., Stevens, C. K., & Bavetta, A. G. (1991). Effects of self-efcacy and post-training intervention on the acquisition and maintenance of complex interpersonal skills. Personnel Psychology, 44, 837861. Goldberg, L. R. (1999). A broad-bandwidth, public domain, personality inventory measuring the lower-level facets of several ve-factor models. In I. Mervielde, I. Deary, F. De Fruyt, & F. Ostendorf (Eds.), Personality psychology in Europe (Vol. 7, pp. 728). Tilburg, the Netherlands: Tilburg University Press. Goldstein, A. P., & Sorcher, M. A. (1974). Changing supervisory behavior. New York: Pergamon Press. Gray, L. E. (1983). Aptitude Constructs, Learning Processes and Achievement. Unpublished doctoral dissertation, Stanford University. Gully, S. M., Payne, S. C., Kiechel Koles, K. L., & Whiteman, J. K. (2002). The impact of error training and individual differences on training outcomes: An attribute-treatment interaction perspective. Journal of Applied Psychology, 87, 143 155. Guzzo, R. A., Jette, R. D., & Katzell, R. A. (1985). The effects of psychologically based intervention programs on worker productivity: A meta-analysis. Personnel Psychology, 38, 275 291. Heimbeck, D., Frese, M., Sonnentag, S., & Keith, N. (2003). Integrating errors into the training process. The function of error management instructions and the role of goal orientation. Personnel Psychology, 56, 333361. Horak, V. M., & Horak, W. J. (1982). The inuence of student locus of control and teaching method on mathematics achievement. Journal of Experimental Education, 51, 18 21. Horak, W. J., & Slobodzian, K. A. (1980). Inuence on instructional structure and locus of control on achievement of preservice elementary science teachers. Journal of Research in Science Teaching, 17, 213222. Hough, L. M., Eaton, N. K., Dunnette, M. D., Kamp, J. D., & McCloy, R. (1990). Criterion-related validities of personality constructs and the effect of response distortion on those validities. Journal of Applied Psychology, 75, 581595. Ivancic, K., & Hesketh, B. (1995/1996). Making the best of errors during training. Training Research Journal, 1, 103 125. Keith, N., & Frese, M. (2005). Self-regulation in error management training: Emotion control and metacognition as mediators of performance effects. Journal of Applied Psychology, 90, 677691. Keith, N., & Frese, M. (2008). Effectiveness of error management training: A meta-analysis. Journal of Applied Psychology, 93, 5969. Keith, N., Richter, T., & Naumann, J. (2010). Active/ exploratory training promotes transfer even in learners with low motivation and cognitive ability. Applied Psychology, 59, 97123. Kolb, D. A. (1984). Experiential learning. Englewood Cliffs, NJ: Prentice-Hall. Martocchio, J. J., & Webster, J. (1992). Effects of feedback and cognitive playfulness on performance in microcomputer software training. Personnel Psychology, 45, 553578.
21
Werner, J. M., OLeary-Kelly, A., Baldwin, T. T., & Wexley, K. N. (1994). Augmenting behavior-modeling training: Testing the effects of pre- and post- training interventions. Human Resource Development Quarterly, 5, 169183. Wexley, K. N., & Latham, G. P. (2002). Developing and training human resources in organizations (3d ed.). Upper Saddle River, NJ: Prentice Hall. Wood, R. E., Kakebeeke, B. M., Debowski, S., & Frese, M. (2000). The impact of enactive exploration on intrinsic motivation, strategy, and performance in electronic search. Applied Psychology: An International Review, 49, 263283. Yerkes, R. M., & Dodson, J. D. (1908). The relation of strength of stimuli to rapidity of habit-formation. Journal of Comparative Neurology and Psychology, 18, 459482.