You are on page 1of 9

Archives of Clinical Neuropsychology Advance Access published September 2, 2013

Effort Testing in Schizophrenia and Schizoaffective Disorder: Validity Indicator Prole and Test of Memory Malingering Performance Characteristics
Sandra Hunt1, James C. Root2,3,4,*, Brittany Lynn Bascetta 5
1 2

Department of Psychology, City University of New York, College of Staten Island, New York, NY, USA Assistant Attending Neuropsychologist, Memorial Sloan Kettering Cancer Center, New York, NY, USA 3 Department of Psychology in Psychiatry, Weill Cornell Medical College, New York, NY, USA 4 Department of Psychology in Anesthesiology, Weill Cornell Medical College, New York, NY, USA 5 The Graduate Center, City University of New York, New York, NY, USA

Downloaded from http://acn.oxfordjournals.org/ by guest on September 22, 2013

*Corresponding author at: Department of Psychiatry and Behavioral Sciences, Memorial Sloan Kettering Cancer Center, 641 Lexington Avenue, 7th Floor, New York, NY 10022, USA. Tel.: + 1-646-888-0035; fax: + 1-212-888-2584. E-mail address: rootj@mskcc.org (J.C. Root). Accepted 3 August 2013

Abstract While the Validity Indicator Prole (VIP) and the Test of Memory Malingering (TOMM) are designed to limit the inuence of actual cognitive impairment on successful performance, the extent to which cognitive dysfunction does play a role in the assessment of effort should be veried in distinct clinical groups. To date, little research has been conducted on VIP performance in individuals diagnosed with a psychotic disorder. Fifty-four patients with either schizophrenia or schizoaffective disorder were administered the VIP, TOMM, Short Test of Mental Status, and the Wide Range Achievement Test-4 Reading subtest. Specicity rates were compared between tests, with normative data, and with published specicity rates in psychiatric samples. Results indicate that the use of the VIP with psychotic-disordered individuals will generate increased invalid performance proles, whereas the TOMM is more resilient in this population. Signicantly, mental status and estimated intellectual ability were predictive of classications on the VIP Verbal subtest and the TOMM.
Keywords: Effort testing; Psychosis; Schizophrenia, Schizoaffective disorder

Introduction Tests of effort and motivation have been developed to detect feigned cognitive impairment within forensic settings; unfortunately, there is only limited research on the performance of psychiatric samples on these measures. It is imperative to understand how clinical populations perform on tests of effort and motivation to examine whether the assumptions of each measure are valid in the clinical group assessed, and so that performance validity can be effectively measured in these groups. The cognitive performance of patients with psychotic disorders on effort tests was identied early on as an area in need of future research; however, few empirical studies have been conducted on this population to date (Goldberg, Back-Madruga, & Boone, 2007). Schizophrenia, in particular, affects 1% of the population and is characterized by cognitive decits in memory (Lee & Park, 2005), attention (Sharma & Antonova, 2003), executive functions such as reasoning and problem solving (Kuperberg & Heckers, 2000), and motivation/mental effort (Gorissen, Sanz, & Schmand, 2005). Furthermore, these decits appear to be relatively stable in patients from the time of their rst hospitalization up to intervals of 10 years or more (Hoff, Svetina, Shields, Stewart, & DeLisi, 2005). In light of the fact that patients with schizophrenia and psychotic disorders more generally may be expected to exhibit cognitive decits in neuropsychological testing, the effects of these decits on formal effort testing remains an important issue to address.
# The Author 2013. Published by Oxford University Press. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com. doi:10.1093/arclin/act069

S. Hunt et al. / Archives of Clinical Neuropsychology

Previous studies have examined individuals with schizophrenia on neuropsychological measures of effort and motivation (Goldberg et al., 2007, for review). In general, results of studies on effort testing in individuals with psychotic disorders suggest higher failure rates in this group compared with normative data, although rates vary with the specic measure studied. Failure rates of 10% or higher have been found on a subset of effort tests in samples of individuals diagnosed with a psychotic disorder, including Digit Span measures (Lesser et al., 1991; Miller et al., 1991), The Digit Memory Test/Hiscock Forced Choice Test (Back et al., 1996), the Dot Counting Test (Back et al., 1996), the Rey 15-Item Test (Back et al., 1996), the Finger Tapping Test (Arnold et al., 2005), the B test (Back et al., 1996), and the Word Memory Test (Gorissen et al., 2005). Relatively preserved specicity rates ( 90%) were found for the Warrington Recognition Memory Test-Words (Egeland et al., 2003; Lesser et al., 1991; Miller et al., 1991), the Test of Memory Malingering (TOMM; Duncan, 2005), and the Victoria Symptom Validity Test (Egeland et al., 2003). Further research on the TOMM (Tombaugh, 1996) found results consistent with those reported by Duncan (2005). In a sample of 20 honest-responding, psychiatric inpatients, which included individuals diagnosed with schizophrenia (Gierok, Dickson, & Cole, 2005), 95% of participants exceeded the cutoff score. In a sample primarily of individuals diagnosed with schizophrenia (Pivovarova, Rosenfeld, Dole, & Green, 2010), 93% of participants exceeded the cutoff score on Trial 2 of the TOMM (Retention Trial not given). In a sample of 26 individuals, in which over half of subjects were diagnosed with a psychotic spectrum disorder (Weinborn, Orr, Woods, Conover, & Feix, 2003), 88% of participants exceeded the cutoff score. In a sample of 104 individuals with a psychotic-spectrum disorder (Hubbard, 2007), 84% of participants exceeded the cutoff score. To date, performance characteristics on concurrent Verbal and Non-verbal subtests of the Validity Indicator Prole (VIP; Frederick, 2003) in a non-litigating sample of individuals with a psychotic-spectrum disorder have not been reported. Frederick, Crosby, and Wynkoop (2000) reported the performance of a forensic sample, believed to be performing with sufcient effort and diagnosed with a psychotic disorder, on the VIP Nonverbal subtest. In that sample, psychotic disorder, affective disorder, and non-psychiatric groups were matched on adjusted score of the VIP Non-verbal subtest. Results indicated that of 60 subjects in the psychotic disorder group, 55% generated valid proles on the VIP Non-verbal subtest, with this rate not signicantly different from non-psychiatric pretrial defendants (48%). That the psychotic disorder group consisted solely of pretrial defendants suggests that either inadequate effort or a motivation to perform poorly cannot be entirely ruled out in this group, and the inuence of cognitive dysfunction associated with psychotic disorders cannot be specically identied. That poor effort or motivation to perform poorly may have been possible in this forensic sample is suggested by the fact that nearly one-half of the non-psychiatric sample generated invalid proles. In an effort to clarify the effects of actual cognitive dysfunction associated with psychosis on VIP performance in a non-forensic sample, Pivovarova and colleagues (2010) examined VIP Verbal subtest performance in a psychiatric sample composed primarily of individuals with a psychotic disorder and a community sample instructed to feign cognitive impairment. Of 81 psychiatric subjects, 70% generated valid proles on the Verbal subtest. Of 26 simulators, 27% generated valid proles. As the above review suggests, little research exists on performance proles of psychotic-disordered individuals on the VIP specically in the clinical setting, and no data exist on the performance of a clinical group on either concurrently administered Verbal and Non-verbal subtests or on the VIP Non-verbal subtest alone. Although few published reports are available that have examined the performance of individuals with psychosis on the VIP and TOMM, available evidence suggests that results of effort assessment in these individuals may be confounded with cognitive decits associated with psychosis depending on the measure used, with the TOMM being relatively resilient to cognitive dysfunction, and the VIP potentially less so. This may be partly due to the level of cognitive demand required by each measure and the specic cognitive decits typically associated with psychosis. Veried cognitive impairment has been found to affect effort test performance; previous research suggests that both intellectual ability (Dean, Victor, Boone, & Arnold, 2008), as well as mental status as determined by mental status screening measures (Dean, Victor, Boone, Philpott, & Hess, 2009), are associated with performance on effort measures. Given developmental, educational, and cognitive issues typically present in individuals diagnosed with schizophrenia, this cohort may be at higher risk for both lower general intellectual ability as well as lower mental status performance on formal measures. Additionally, specic cognitive difculties associated with psychosis may interact with task demands inherent in each measure. Cognitive decits associated with schizophrenia such as inattention, distractibility, and difculties in conceptualization and abstraction may be exploited by the different formats, difculty levels, and task demands of the TOMM and VIP. As an example, with the exception of task instructions that are given by the clinician, the VIP is largely self-administered, lengthy, and varies in item difculty. In contrast, the TOMM administration requires clinician-patient contact throughout, is shorter, and is less demanding in regard to item difculty. As a result, disagreement in classications between the VIP and the TOMM may be common given the difference in task demands and the ways in which these interact with cognitive difculties associated with schizophrenia. The research reported here had three main aims: (a) to establish performance proles on both the VIP Non-verbal and Verbal subtests in a group of non-forensic psychotic-disordered individuals; (b) to examine the concordance of VIP classications with

S. Hunt et al. / Archives of Clinical Neuropsychology

performance on the TOMM; and (c) to examine the effects of both premorbid estimated intellectual ability and mental status on VIP and TOMM classications. We administered the VIP and the TOMM to 54 consecutive residential-dwelling patients diagnosed with a psychotic disorder (schizophrenia; schizoaffective disorder) together with the Short Test of Mental Status (STMS) and the Reading subtest of the Wide Range Achievement Test-4th Edition (WRAT-4). Given previous research (Pivovarova et al., 2010), we hypothesized that the rate of Invalid classications on both VIP Verbal and Non-verbal subtests would be . 10%, with the majority of these classications falling in the Inconsistent category. Given previous research suggesting that the TOMM is relatively resilient to actual cognitive impairment associated with schizophrenia, we hypothesized that the rate of scores falling below the TOMM cutoff would be signicantly lower than that of the VIP. Finally, given research indicating that general intellectual ability and current mental status may inuence performance on measures of effort and motivation, we hypothesized that lower performance on the STMS and the WRAT-4 Reading subtest would be related to lower performance on the VIP and the TOMM. Methods Subjects The study sample consisted of 58 individuals diagnosed with schizophrenia or schizoaffective disorder residing in a community residential facility (Table 1). Of the original sample, four individuals were excluded from the nal analyses because they discontinued testing prior to the completion of the test battery and one individual did not complete the VIP Verbal subtest. The nal participant pool consisted of 54 individuals (34 schizophrenia; 20 schizoaffective disorder). Of the schizophrenia-diagnosed participants, 67% (23) were classied as Undifferentiated and 33% (11) were classied as Paranoid. All patients were previously diagnosed by board-certied psychiatrists using criteria set forth by the Diagnostic and Statistical Manual of Mental Disorders: Fourth Edition-Text Revision (American Psychiatric Association, 2000). Record of rst episode/rst psychiatric hospitalization for all participants was obtained from patient charts and is as follows: Ages 12 16 (26%), ages 17 21 (42%), over 21 (16%), unknown (16%). All participants were residents of a 200-bed assisted living facility that provides independent living and supportive care including housing, meals, psychiatric treatment, medical services, and other support staff to patients with chronic mental illness. Approximately 50% of residents in the facility are diagnosed with psychotic disorders and treated with antipsychotic medications, psychotherapy twice per month, and psychiatric services every 2 months or as needed. For this study, participants were excluded if they had dual diagnoses of substance abuse/dependence, neurological disorders, and/or mental retardation by medical chart review. Participants were recruited by means of sign-up sheets placed around the residential facility. Participants were also alerted to the study through staff members at the facility. Participants were compensated $5.00 for their time and were assured that they
Table 1. Demographic characteristics Demographics Age (year) Gender (F) Education (year) Race Caucasian Asian African American Hispanic Estimated Intellectual Ability (Standard Score) Diagnosis Schizophrenia Undifferentiated Paranoid Schizoaffective Age at First Episode 1216 1721 21 or over Unknown Mean (SD) or n (%) 51 (11.91), range 2672 28 (51%) 10 (2.47), range 4 16 34 (63%) 9 (16%) 7 (13%) 4 (7%) 89.6 (11.50) 34 (63%) 23 (67%) 11 (33%) 20 (37%) 14 (26%) 22 (42%) 9 (16%) 9 (16%)

S. Hunt et al. / Archives of Clinical Neuropsychology

would receive payment regardless of whether they completed the study. All tests were administered by the primary investigator, a clinical neuropsychologist, and testing was completed in two sessions. Measures Subjects were administered the VIP Verbal and Non-verbal subtests (Frederick, 2003), the TOMM (Tombaugh, 1996), the STMS (Kokmen, Naessens, & Offord, 1987), and the Reading subtest of the WRAT-4 (Wilkinson & Robertson, 2006). The VIP is a self-administered, paper-and-pencil test consisting of a 78-item verbal subtest and a 100-item non-verbal subtest designed to detect suboptimal effort or malingering in cognitive assessment. Performance on the VIP Verbal and Non-verbal subtests is classied as either Compliant, or as one of four Invalid classications: Inconsistentperformance is inconsistent; Irrelevant performance may reect random responding or little effort to answer correctly; and Suppressedperformance may have been affected by an intentional effort to respond incorrectly. The TOMM is a visual recognition task consisting of two learning trials (50 items each) and a retention trial (50 items) specically designed for the assessment of insufcient effort dened as likely malingering during neuropsychological evaluations. The STMS was developed and validated as a brief screening mental status test to be used for detecting mild to severe cognitive difculties, with a cutoff of 29 indicating a greater likelihood of impairment (Kokmen et al., 1987). Cognitive functions assessed include orientation, attention, learning/memory, calculation, abstraction, information, and construction. The WRAT-4 Reading subtest was used to estimate intellectual ability; scores on this subtest were transformed into standard scores with a mean of 100 and standard deviation of 15. The design and requirements of this study were reviewed and approved by the Institutional Review Board. Analysis For all comparisons, the z-test for equality of proportions was used to assess differential passing rates and classication agreement; because a subset of cell counts contained inadequate numbers, Fishers Exact Test was used to conrm signicance in affected comparisons. Passing rates on the VIP Verbal and Non-verbal subtests were calculated and compared with normative values for honest responders (Frederick & Crosby, 2000). Observed passing rates were then compared with published data for non-litigating psychiatric samples in which the majority of patients were diagnosed with a psychotic disorder (Non-verbal subtest [Frederick et al., 2000]; Verbal subtest [Pivovarova et al., 2010]). Passing rates for the Retention Trial of the TOMM were calculated and compared with normative data (Tombaugh, 1997) of cognitively intact and cognitively impaired compliant responders (no cognitive impairment; cognitive impairment; traumatic brain injury; aphasia). Observed passing rates on the Retention Trial of the TOMM were then compared with individual and pooled published data for non-litigating psychiatric samples in which the majority of patients were diagnosed with a psychotic disorder and a Retention classication was available (Duncan, 2005; Hubbard, 2007; Weinborn et al., 2003). For analysis of the inuence of general intellectual ability on effort test performance, WRAT-4 Reading standardized scores were dichotomized into two groups ( 79; 80). For analysis of the inuence of mental status on effort test performance, STMS raw scores were dichotomized into two groups ( 29; 30). Results VIP Classication Accuracy The VIP Verbal subtest exhibited a passing rate of 40%, with 21 of 53 subjects generating a Compliant classication. Invalid proles were predominantly classied as Inconsistent (n 21), with fewer Irrelevant classications (n 11), and no Suppressed classications. The VIP Non-verbal subtest exhibited a passing rate of 17%, with 9 of 54 subjects generating a Compliant classication. Invalid proles were again predominantly classied as Inconsistent (n 26), with fewer Irrelevant classications (n 14), and ve Suppressed classications (Table 2). Comparison with published results in a presumed compliant, non-psychiatric sample (Frederick, 2003) revealed signicantly lower passing rates in our sample for both Verbal (z 2 8.58, p , .001) and Non-verbal subtests (z 2 10.35, p , .001). Comparison with published results of presumed compliant, psychotic-disordered samples (Frederick et al., 2000; Pivovarova et al., 2010) revealed signicantly lower passing rates in our sample for both Verbal (z 2 3.68, p , .001) and Non-verbal subtests (z 2 4.24, p , .001) (Table 3). No signicant difference in the passing rate was observed between schizophrenia and schizoaffective subgroups for the VIP Verbal subtest (Fishers Exact Test, p .337). For the VIP Non-verbal subtest, a trend-signicant difference was observed, with the schizophrenia subgroup exhibiting more invalid proles than the schizoaffective subgroup (Fishers Exact Test, p .053).

S. Hunt et al. / Archives of Clinical Neuropsychology Table 2. Test performance VIP VIP Verbal VIP Non-verbal Test of Memory Malingering Trial 2 Retention WRAT-4 Reading 79 (n 12) VIP Verbal VIP Non-verbal TOMM Retention 80 (n 42) VIP Verbal VIP Non-verbal TOMM Retention Short Test of Mental Status 29 (n 19) VIP Verbal VIP Non-verbal TOMM Retention 30 (n 35) VIP Verbal VIP Non-verbal TOMM Retention Compliant (n [%]) 21 (40) 9 (17) Valid (n [%]) 39 (72) 45 (83) Valid (n [%]) 0 (0) 1 (8) 5 (42) 21 (50) 8 (19) 40 (95) Valid (n [%]) 2 (11) 1 (5%) 11 (58) 19 (56) 8 (23) 34 (97) Inconsistent (n [%]) 21 (40) 26 (48) Invalid (n [%]) 15 (28) 9 (17) Invalid (n [%]) 11 (100) 11 (92) 7 (58) 21 (50) 34 (81) 2 (5) Invalid (n [%]) 16 (89) 18 (95) 8 (42) 16 (34) 27 (77) 1 (3) Irrelevant (n [%]) 11 (20) 14 (26) Specicity (n [%]) 39/54 (72) 45/54 (83) Suppressed (n [%]) 0 (0) 5 (9)

Specicity (n [%]) 21/53 (40) 9/54 (17)

Notes: VIP Validity Indicator Prole; TOMM Test of Memory Malingering.

Table 3. Comparison of reported sample with previously published results VIP % Psychotic disorder Current sample VIP Verbal Compliant combined Frederick (2003) Psychosis Pivovarova and colleagues (2010) Frederick and colleagues (2000) TOMM Honest Responders Combined Tombaugh (1997) Psychosis Pivovarova and colleagues (2010) Hubbard (2007) Duncan (2005) Weinborn and colleagues (2003) TOMM Psychosis Pooled 0% 85% 100% % Psychotic Disorder 0% 85% 100% 72% 69% 84% z 2 8.58* z 2 3.68* Current sample Trial 2 z 2 4.29* z 2 3.40* z 2 1.45 z 2 2.93** z 2 1.22 z 2 3.35* VIP Non-verbal z 2 10.35* z 2 4.24* Retention z 2 3.78* z 2 0.09 z 2 1.70*** z 2 0.60 z 2 1.02

Notes: VIP Validity Indicator Prole; TOMM Test of Memory Malingering. *p .001. **p .01. ***p .05.

TOMM Classication Accuracy The TOMM Retention Trial exhibited a passing rate of 83%. Of protocols classied as invalid, no Retention score was signicantly below chance (40; 40; 39; 39; 39; 38; 34; 34; 34) (Table 2). Comparison with published results of a compliant, nonpsychiatric sample (Tombaugh, 1997) revealed signicantly lower passing rates in our sample (z 2 3.78, p , .001).

S. Hunt et al. / Archives of Clinical Neuropsychology

Table 4. VIP Non-verbal, Verbal, TOMM Trial 2, and TOMM Retention Agreement Compliant (n [%]) VIP Non-verbal VIP Verbal Compliant Inconsistent Irrelevant Suppressed TOMM Retention Valid Retention Invalid TOMM Retention Valid Retention Invalid 6 (11) 1 (2) 2 (4) 0 (0) VIP Non-verbal 9 (17) 0 (0) VIP Verbal 21 (31) 0 (0) 10 (19) 11 (20) 5 (9) 0 (0) 3 (6) 8 (15) 3 (6) 0 (0) 2 (4) 2 (4) 1 (2) 0 (0) Inconsistent (n [%]) Irrelevant (n [%]) Suppressed (n [%])

22 (41) 4 (7)

10 (19) 4 (7)

4 (7) 1 (2)

17 (32) 5 (9)

7 (13) 4 (7)

0 (0) 0 (0)

Comparison with published results of compliant, psychotic-disordered samples pooled across three previous studies (Duncan, 2005; Hubbard, 2007; Weinborn et al., 2003) was not signicant (z 2 1.02; Table 3). For the TOMM Retention Trial, a trend-signicant difference was observed between schizophrenia and schizoaffective subgroups, with the schizophrenia subgroup exhibiting more invalid proles than the schizoaffective subgroup (Fishers Exact Test, p .078). VIP and TOMM Agreement Disagreement between classications of the VIP and TOMM Retention Trial were signicant for both the VIP Verbal subtest (z-test 2 4.65, p , .001) and VIP Non-verbal subtest (z-test 2 6.93, p , .001). Of 45 valid TOMM performances, 21 subjects (47%) were classied as Compliant on the VIP Verbal subtest, and nine subjects (2%) were classied as Compliant on the VIP Non-verbal subtest (Table 4). Association with STMS Performance Performance on the Short Test of Mental Status was signicantly associated with classication on the TOMM (Fishers Exact Test, p , .001), and with classication on the VIP Verbal subtest (Fishers Exact Test, p , .01), with subjects in the low STMS group being more likely to be classied as invalid on either test (Table 2). No signicant relationship was found between STMS performance and VIP Non-verbal classication. Association with WRAT-4 Reading Score Performance on the WRAT-4 Reading subtest was signicantly associated with classication on the TOMM (Fishers Exact Test, p , .001), and with classication on the VIP Verbal subtest (Fishers Exact Test, p , .01), with subjects in the low WRAT-4 Reading group being more likely to be classied as invalid on either test (Table 2). No signicant relationship was found between WRAT-4 Reading performance and VIP Non-verbal classication. Discussion Standalone and embedded measures of effort and motivation rely to a greater or lesser extent on the assumption that true cognitive impairment does not affect performance on these tests. This assumption is based on prior work documenting the performance of cognitively-impaired clinical groups on such measures which nd adequate specicity ( 90) for honest or compliant responders. Although the performance of neurologically affected patient groups has been extensively studied on tests of effort and motivation, less is known about the effects of neurocognitive complications of psychiatric disorders, and specically of psychosis-spectrum disorders, on effort testing. Since individuals diagnosed with schizophrenia or schizoaffective disorder may be seen in a forensic context, more extensive normative data are needed to address the inuence of cognitive dysfunction on effort testing in affected individuals. We chose to focus on the VIP because of the comparatively few published studies that have evaluated this measure on patient groups and, more

S. Hunt et al. / Archives of Clinical Neuropsychology

specically, in individuals diagnosed with a schizophrenia or schizoaffective disorder. VIP performance was compared with normative data, with the limited, available data in similar psychiatric samples, and with TOMM classications to assess agreement between measures in this cohort. We further assessed the inuence of mental status and premorbid cognitive ability on effort test performance. First, our ndings indicate that performance on the VIP is signicantly decreased in this sample of individuals diagnosed with schizophrenia or schizoaffective disorder. Passing rates on the Verbal (40%) and Non-verbal (17%) subtests in our sample were signicantly lower than normative data for honest responders in the original standardization sample (94% and 90%, respectively; Frederick, 2003), and signicantly lower than those found in previous research in predominantly psychotic-disordered samples (Verbal 70%; Non-verbal 55%; Frederick et al., 2000; Pivovarova et al., 2010). The majority of invalid classications for both the Verbal and Non-verbal subtests were classied as Inconsistent, suggesting that performance was uneven for less demanding items in Sector 1. Although ve patients were classied as Suppressed on the Non-verbal subtest, the second most common invalid classication was that of Irrelevant for both Verbal and Non-verbal subtests. Of note, the passing rate on the TOMM (83%) was less affected by psychiatric and neurocognitive difculties in our sample, but still lower than normative data for a mixed sample of cognitively intact and cognitively impaired individuals (99%; Tombaugh, 1997). In contrast, when compared with a pooled sample of previously published passing rates in psychotic-spectrum individuals (Duncan, 2005; Hubbard, 2007; Weinborn et al., 2003), no signicant difference was observed (83% vs. 89%). Disagreement between the classications of the TOMM and the VIP was noted to be higher in the present study than in previous work (Farkas, Rosenfeld, Robbins, & van Gorp, 2006). In that study, using TOMM classications as the index variable, VIP classications exhibited higher sensitivity and specicity in a sample of forensic referrals for both Verbal (Sen: 75%; Spec: 93%) and Non-verbal (Sen: 75%; Spec: 50%) subtests than in the current study. The signicantly lower VIP passing rates in our sample than in Pivovarova and colleagues (2010) may be partly due to differences in participant characteristics between studies, specically in disease severity and intellectual ability. Although disease severity cannot be denitively compared between the two studies, participants in the Pivovarova study were recruited as outpatients through a psychiatric hospital, whereas all of our participants required supervised residential care in a facility for the chronically mentally ill. In regard to intellectual ability, which also cannot be directly compared between studies, we note that in our study, 12 of 54 participants had estimated intellectual ability below a standard score of 80 and that intellectual ability was signicantly associated with effort test performance. Although the VIP manual cautions against use of the VIP with individuals diagnosed with mental retardation (Frederick, 2003), lower intellectual ability without a documented history of mental retardation is not considered a contraindication to administration. Results from this study suggest that even in individuals not diagnosed with mental retardation, estimated borderline range of intellectual ability may signicantly inuence tested performance. These results serve to underscore the importance of testing the inuence of both psychiatric and neurocognitive symptoms on effort test performance in distinct clinical groups. Our results suggest that the VIP, specically, will be potentially inuenced by psychiatric symptomatology, associated neurocognitive decits, current mental status, as well as estimated intellectual ability. One important issue is the extent to which our sample is representative of schizophrenia or schizoaffective samples more broadly. We note that our sample consisted of chronically affected individuals whose psychiatric symptoms had necessitated primary placement in a residential facility. Certainly, the range of severity in psychotic disorders more generally is broad and the individuals included in our sample may represent the most severely affected. Of note, however, is the nding that the TOMM passing rate in our sample was not signicantly lower than the aggregate specicity rate of samples that included psychotic-disordered individuals in previous studies, while signicantly lower passing rates were observed only for the VIP Verbal and Non-verbal subtests. Although our results are suggestive, limitations are identied that should be addressed in future studies. One question that should be further claried is the issue of whether the invalid VIP classications are actually in error in our patient sample. As noted above, the majority invalid classication was for an Inconsistent prole, with the second most common classication being for an Irrelevant prole. Both of these classications would suggest either uneven attention and inconsistent focus or little effort to answer accurately, but at the same time no strong effort to answer incorrectly. Given the effects of schizophrenia and schizoaffective disorder on attention, focus, and disinhibition, it could be argued that the VIP classications are accurately representing the attitude and approach to testing taken by our patient sample. Furthermore, absent the direct effect of cognitive decits on performance, there is also the possibility that amotivation, disengagement, or lack of interest in the testing in a research setting may have adversely affected performance. Indeed, the VIP manual (Frederick, 2003) suggests that individuals with lower intellectual ability may stop trying after they encounter more difcult items mixed among easy ones. (p. 42). In such cases, the manual indicates that test administration may be altered by guiding evaluees through the administration using prompts and altered instructions for completion. In summary, results of this study indicate signicantly altered performance on standalone tests of effort and motivation in a sample of individuals diagnosed with schizophrenia and schizoaffective disorder. At this point, the independent inuence of

S. Hunt et al. / Archives of Clinical Neuropsychology

psychiatric, neurocognitive, intellectual, and motivational factors cannot be further assessed. Findings in our sample suggest that intellectual ability and mental status may both be implicated in higher failure rates, particularly on the VIP Verbal subtest. Future studies might clarify independent effects on effort test performance with the use of a broader battery that would assess psychiatric, neurocognitive, intellectual, and motivational factors. These observations notwithstanding, our results would recommend caution in using effort tests in individuals diagnosed with schizophrenia or schizoaffective disorder, particularly in those individuals in whom low intellectual ability is assessed or suspected. Funding This work was supported by the PSC-CUNY Research Award. Conict of Interest None declared. Acknowledgements The authors wish to thank Raf Leicht, who assisted in database entry and management of records. References
American Psychiatric Association. (2000). Diagnostic and Statistical Manual of Mental Disorders (4th ed.). Washington, DC: Author. Arnold, G., Boone, K. B., Lu, P., Dean, A., Wen, J., Nitch, S., et al. (2005). Sensitivity and specicity of nger tapping test scores for the detection of suspect effort. The Clinical Neuropsychologist, 19 (1), 105120. doi:10.1080/13854040490888567 Back, C. A., Boone, K. B., Edwards, C., Parks, C., Burgoyne, K., & Silver, B. (1996). The performance of schizophrenics on three Cognitive Tests of Malingering, Rey 15-Item Memory Test, Rey Dot Counting, and Hiscock Forced-Choice Method. Assessment, 3 (4), 449 457. Dean, A. C., Victor, T. L., Boone, K. B., & Arnold, G. (2008). The relationship of IQ to effort test performance. The Clinical Neuropsychologist, 22 (4), 705 722. doi:10.1080/13854040701440493 Dean, A. C., Victor, T. L., Boone, K. B., Philpott, L. M., & Hess, R. A. (2009). Dementia and effort test performance. The Clinical Neuropsychologist, 23 (1), 133152. doi:10.1080/13854040701819050 Duncan, A. (2005). The impact of cognitive and psychiatric impairment of psychotic disorders on the test of memory malingering (TOMM). Assessment, 12 (2), 123129. doi:10.1177/1073191105275512 Egeland, J., Sundet, K., Rund, B. R., Asbjornsen, A., Hugdahl, K., Landro, N. I., et al. (2003). Sensitivity and specicity of memory dysfunction in schizophrenia: A comparison with major depression. Journal of Clinical and Experimental Neuropsychology, 25 (1), 7993. doi:10.1076/jcen.25.1.79.13630 Farkas, M. R., Rosenfeld, B., Robbins, R., & van Gorp, W. (2006). Do Tests of Malingering concur? Concordance among malingering measures. Behavioral Sciences and the Law, 24 (5), 659671. Frederick, R. I. (2003). Validity Indicator Prole manual. Texas: Pearson. Frederick, R. I., & Crosby, R. D. (2000). Development and validation of the Validity Indicator Prole. Law and Human Behavior, 24 (1), 5982. Frederick, R. I., Crosby, R. D., & Wynkoop, T. F. (2000). Performance curve classication of invalid responding on the Validity Indicator Prole. Archives of Clinical Neuropsychology, 15 (4), 281 300. Gierok, S. D., Dickson, A. L., & Cole, J. A. (2005). Performance of forensic and non-forensic adult psychiatric inpatients on the Test of Memory Malingering. Archives of Clinical Neuropsychology, 20 (6), 755760. doi:10.1016/j.acn.2005.04.008 Goldberg, H. E., Back-Madruga, C., & Boone, B. B. (2007). The impact of psychiatric disorders on cognitive Symptom Validity Test scores. In B. B. Boone (Ed.), Assessment of feigned cognitive impairment: A neuropsychological perspective (pp. 281 309). New York: The Guilford Press. Gorissen, M., Sanz, J. C., & Schmand, B. (2005). Effort and cognition in schizophrenia patients. Schizophrenia Research, 78 (2 3), 199 208. doi:10.1016/ j.schres.2005.02.016 Hoff, A. L., Svetina, C., Shields, G., Stewart, J., & DeLisi, L. E. (2005). Ten year longitudinal study of neuropsychological functioning subsequent to a rst episode of schizophrenia. Schizophrenia Research, 78 (1), 27 34. doi:10.1016/j.schres.2005.05.010 Hubbard, K. L. (2007). Feigning cognitive decits among psychiatric inpatients: A validation of three measures of cognitive malingering. PhD, Graduate School of The University of Alabama, Proquest. Kokmen, E., Naessens, J. M., & Offord, K. P. (1987). A short test of mental status: Description and preliminary results. Mayo Clinic Proceedings, 62 (4), 281 288. Kuperberg, G., & Heckers, S. (2000). Schizophrenia and cognitive function. Current Opinion in Neurobiology, 10 (2), 205210. Lee, J., & Park, S. (2005). Working memory impairments in schizophrenia: A meta-analysis. Journal of Abnormal Psychology, 114 (4), 599 611. doi:10.1037/ 0021-843X.114.4.599 Lesser, I. M., Miller, B. L., Boone, K. B., Hill-Gutierrez, E., Mehringer, C. M., Wong, K., et al. (1991). Brain injury and cognitive function in late-onset psychotic depression. Journal of Neuropsychiatry and Clinical Neurosciences, 3 (1), 3340. Miller, B. L., Lesser, I. M., Boone, K. B., Hill, E., Mehringer, C. M., & Wong, K. (1991). Brain lesions and cognitive function in late-life psychosis. British Journal of Psychiatry, 158, 76 82. Pivovarova, E., Rosenfeld, B., Dole, T., Green, D., & Zapf, P. (2010). Are measures of cognitive effort and motivation useful in differentiating feigned from genuine psychiatric symptoms? International Journal of Forensic Mental Health, 9 (4), 271278.

S. Hunt et al. / Archives of Clinical Neuropsychology

Sharma, T., & Antonova, L. (2003). Cognitive function in schizophrenia. Decits, functional consequences, and future treatment. Psychiatric Clinics of North America, 26 (1), 2540. Tombaugh, T. N. (1996). Test of Memory Malingering manual. New York: MHS. Tombaugh, T. N. (1997). The Test of Memory Malingering (TOMM): Normative data from cognitively intact and cognitively impaired individuals. Psychological Assessment, 9 (3), 260 268. Weinborn, M., Orr, T., Woods, S. P., Conover, E., & Feix, J. (2003). A validation of the test of memory malingering in a forensic psychiatric setting. Journal of Clinical and Experimental Neuropsychology, 25 (7), 979 990. doi:10.1076/jcen.25.7.979.16481 Wilkinson, G., & Robertson, G. (2006). Wide Range Achievement Test-fourth edition. Florida: Psychological Assessment Resources.

You might also like