Educational Methodologies

Non-Graded Clinical Evaluation of Dental

Students in a Competency-Based Education
Program
Mohsen Taleghani, D.M.D.; Eric S. Solomon, D.D.S., M.A.; William F. Wathen, D.M.D.
Abstract: The objective of this article is to report the development, implementation, and early results of a non-graded normative
dental student clinical performance assessment system based on our competencies documents. The normative system (student
performance is compared to evidence-based clinical standards) was used and evaluated during the 2002-03 academic year and is
now gradually replacing the traditional summative (numerical) grading system previously used at Baylor College of Dentistry.
The methodology included: 1) consensus development of new clinical performance assessment forms over the summer of 2002;
2) concurrent fourth-year clinical faculty calibration to the new forms; 3) implementation at the beginning of the senior year for
the Class of 2003; and 4) faculty and student evaluation surveys in May 2003. Every step of each clinical procedure was
recorded, weekly performance summaries by both students and faculty were collected, and periodic workshops were held to
refine the forms and further calibrate faculty. The results showed strong positive responses to the new system by graduates and
faculty alike. We conclude that early results warrant broadened efforts toward a continuously improved schoolwide normative
student clinical performance assessment system.
Dr. Taleghani, who created this assessment program, is Professor and Chair, Department of General Dentistry, Baylor College of
Dentistry, The Texas A&M University System Health Science Center; Dr. Solomon is Executive Director for Institutional
Research, The Texas A&M University System Health Science Center; and Dr. Wathen is Associate Professor, Department of
General Dentistry, Baylor College of Dentistry. Direct correspondence and reprint requests to Dr. Mohsen Taleghani, Department
of General Dentistry, Baylor College of Dentistry, 3302 Gaston Ave., #313, Dallas, TX 75246; 214-828-8414 phone; 214-828-
8952 fax; mtaleghani@tambcd.edu.
Key words: clinical evaluation, non-graded assessment, competency-based dental education, clinical instruction, quality assess-
ment, remedial education
Submitted for publication 11/26/03; accepted 3/29/04

M
uch has been written in the past decade
about dentists and dental practices of the
future.1-3 Todays educational and practice
environments demand change at an ever-increasing
pace. The move to competency-based dental educa-
tion adds to that demand.
Dentistry rests on an educational foundation,
yet must thrive in the competitive milieu of a rapidly
changing world that demands continuous quality
improvement through both personal and professional
growth.4,5 Our dental schools must produce dentists
who are firmly rooted in the ethical/moral life of tra-
ditional professions, evidence-based science, and
sound clinical decision making: individuals who are
biologically oriented, technically capable, and so-
cially sensitive. At the same time, it is imperative
that our graduates maintain an abiding desire for life-
long collegial relationships with their dental school
and within the communities they will serve during
their practice lifetime. Our previous assessment sys-
tem was inadequate at measuring these factors.
Therefore, we recognized a need to develop and
implement a different student performance assess-
ment strategy that complemented a competency-
based curriculum and emphasized the nontechnical
as well as the procedural aspects of modern dental
practice.
Todays dental administrators and faculty
should be knowledgeable about our changing pro-
fession and supportive of appropriate innovation in
dental education. The scope and scale of such non-
technical innovation have been discussed by numer-
ous authors,6-8 who have suggested models for den-
tal schools to consider. A secondary objective of our
program innovation was to emphasize these aspects
of the dental education process and to include them
in our faculty calibration strategy.

644 Journal of Dental Education Volume 68, Number 6

 Furthermore, we appreciate the subtle, critical
change that occurs in maturing learners as they as-
sume increasing responsibility for control of their
learning environment (Figure 1). Understanding the
differences between levels of competencies and how
to successfully mentor our students through that
segue from passive to active learner9-15 must be mas-
tered and practiced by faculty (who have largely been
trained in a different model). Only after incorporat-
ing that understanding into the faculty mindset and
behavior can schools more successfully mentor stu-
dents into the professional worlds that await.
Collegial mentoring requires assessment and
feedback systems that enhance learning, the students
professional growth, and pride in being a dentist.
Calibration workshops help ensure that faculty can
and will model the defined competencies. Traditional
dental school clinical assessment has often consisted
HOW DO ADULT PROFESSIONALS LEARN?
Figure 1. As learners move beyond formal education and mature toward mastery in their professions, they assume
increasing responsibility for directing their own professional development strategies.
June 2004 Journal of Dental Education
of a faculty member simply looking at a students
work and making snap decisions about the product.
Such techniques are highly subjective and variable.16
Berrong et al.,17 as well as Mackenzie et al.,18 ana-
lyzed such discrepancies and identified common,
specific factors that reduce assessment agreement
among clinical faculty and create confusion among
students (Table 1). Our system attempts to eliminate
these factors by requiring faculty to enter written ex-
planation of any aspect of patient care that lies out-
side the clinically acceptable norms contained in our
faculty calibration manual.
Early attempts to diminish the inter- and intra-
rater discrepancies created by the traditional system
consisted of rating scales that translated into the
summative grading system with which many of us
are familiar.19 Baylor College of Dentistry previously
used a traditional block curriculum with the 0-4 grad-
645

Table 1. Contributing factors of disagreement in
clinical assessment
1. Checkpoint ambiguity
2. Faulty memory
3. Incomplete coverage of dimensions
4. Unspecified exceptions
5. Untrained estimations of size
6. Non-standardized aids to judgment
7. Unspecified methods of observing
8. Incomplete operational definition
9. Unsystematic inspection
10. Discrepancies in visual acuity
11. Degrees of leniency
12. Inadequacy of verbal definitions
13. Inadequate communications with non-verbal
examples
14. Unrecognized ambiguities in definitions
15. Differences in background
16. Differences in mental processing
Source: Hendricson WD, Kleffner JH. Curricular and
instructional implications of competency-based
dental education. J Dent Educ 1998;62(2):183-96.
ing system (F to A) to evaluate student clinical per-
formance. As the curriculum changed to a compre-
hensive, clinical competency-based approach, the
shortcomings of this evaluation method became in-
creasingly evident.
Our traditional system was highly subjective,
difficult to calibrate, and susceptible to individual
faculty personalities, and it offered insufficient teach-
25,000
20,000
15,000
10,000
5,000
0
0.0 0.5 1.0 1.5
Figure 2. Grade distribution fourth-year class, 2001-02
646
ing opportunities and discussion time with students.
Many faculty members tended to take the easier ap-
proach of giving out As because they didnt feel they
had the time to discuss or argue with students who
were graded down. Thus, the long-recognized assess-
ment errors of leniency (some faculty usually give
high grades), central tendency (some faculty usually
give a B or C), and severity (some faculty usually
give low grades) were predictably present, skewed
dramatically toward the path of least resistance: le-
niency (Figure 2). It was necessary to devise a si-
multaneous faculty evaluation and feedback system.20
To summarize, we wanted the new assessment
system to:
1. eliminate summative clinical grading,
2. establish clinical performance norms based on
the competency model,
3. transition to a normative assessment model that
compares student performance to those clinical
performance standards,
4. enhance faculty/student relationships and envi-
ronment to better express professional collegi-
ality,
5. diminish student feelings of threat in the clinics,
6. diminish faculty feelings of stress in a graded
situation,
7. bring collegial mentored education to the fore-
front of all clinical activities, as the primary fac-
ulty expectation, and
8. ensure student transition from passive to endur-
ing active learners with enhanced critical think-
ing and clinical decision-making skills.
2.0 2.5 3.0 3.5 4.0
Journal of Dental Education Volume 68, Number 6

Materials and Methods
The transition to competency-based education,
begun in 1992, is now a comprehensive clinical ap-
plication. As previously reported,21 we began to con-
struct a student evaluation system to pinpoint and
document performance deficiencies that could lead
to clinical failure. The Department of General Den-
tistry began the clinical implementation of our com-
petencies documents by establishing fundamental
assessment standards and criteria broadly based on
clinical faculty input and identified ideals.4,14-23 (See
Table 2.)
The process of developing new clinical perfor-
mance assessment forms began in January 2002 and
was concluded in the early summer of 2002. After
general consensus was reached among clinical fac-
ulty from all departments, the fourth-year general
dentistry faculty were calibrated to the clinical stan-
dards agreed to by consensus, and the forms were
introduced in June at the beginning of the senior year
for the Class of 2003.
A historical assessment problem involves the
behavioral (nontechnical) aspect of dentistry. Baylor
College of Dentistrys original competency document
identified issues of Professionalism, Procedure and
Patient Management, and Skills/Traits and detailed
them in objective elements that specify those
nonclinical aspects of being a dentist. These elements
of professionalism are included in the assessment of
all clinical activities that relate to specific non-
procedural aspects of practice (Figure 3). The lines
on the assessment forms under each of these head-
ings define what is meant by each term: for example,
under Skills and Traits the attending faculty mem-
ber assesses 1) the degree of confidence and inde-
pendence exhibited by a student during a patient en-
counter; 2) how the student reacts to stress that may
occur during that encounter; 3) the level of manual
skills demonstrated by the student during the encoun-
ter; and 4) the level of appropriate interpersonal skills
related to patients, faculty, and staff members.
Clinical procedures were similarly reviewed
and disaggregated into key steps, sequentially ar-
ranged. Using the assessment sheets as a fundamen-
tal checklist, faculty assess these critical steps (both
nontechnical and clinical) that predict success or fail-
ure of the procedure, privately discuss their obser-
vations with the student as the event occurs, and write
their requirements of performance improvement
(numbered to correspond to the line number of the
assessment form) in the comments section of the
clinical evaluation forms. Staff members then enter

Table 2. Selected competency assessment criteria

1. Assessments distinguish competent from incompetent practitioners.
2. Assessments are valid and reliable.
3. Assessments are applied uniformly.
4. Assessments measure objective standards.
5. Clear performance standards and criteria are formulated by the evaluators.
6. Standards/criteria are understood and applied uniformly by every evaluator.
7. Standards/criteria are reviewed and revised regularly under CQI feedback.
8. Standards/criteria are fully available to those being assessed.
9. Assessments focus on a defined scope of activities/procedures.
10. Student remediation programs are available.
11. Assessors are able to assess the standards/criteria with minimal variability.
12. Students experience enhanced formative learning opportunities.

Sources: Low DS, Kalkwarf KL. Assessing continued competency: an approach for dentistry. J Am Dent Assoc
1996;127(3):383-8; Mossey PA, Newton JP, Stirrups DR. Defining, conferring and assessing the skills of the dentist. Br Dent J
1997;182(4):123-5; Hendricson WD, Kleffner JH. Curricular and instructional implications of competency-based dental
education. J Dent Educ 1998;62(2):183-96; OConnor P, Lorey RE. Improving inter-rater agreement in evaluation in dentistry
by the use of comparison stimuli. J Dent Educ 1978;42(4):174-9; Berrong JM, Buchanan RN, Hendricson WD. Evaluation of
practical clinical examinations. J Dent Educ 1983;47(10):656-63; Mackenzie RS, Antonson DE, Weldy PL, Welsch BB,
Simpson WJ. Analysis of disagreement in the evaluation of clinical products. J Dent Educ 1982;46(5):284-9; Kerlingen FN.
Foundations of behavioral research. New York: Holt, Reinhart, and Winston, 1966:516-7; Chambers DW, Boyarsky H, Peltier
B, Fendler F. Development of a mission-focused faculty evaluation system. J Dent Educ 2003;67(1):10-22; McCann AL, Babler
WJ, Cohen PA. Lessons learned from the competency-based curriculum initiative at Baylor College of Dentistry. J Dent Educ
1998;62(2):197-207; Yip H-K, Smales RJ. Review of competency-based education in dentistry. Br Dent J 2000;189(6):324-6;
Yip H-K, Smales RJ, Newsome PRH, Chu FCS, Chow TW. Competency-based education in a clinical course in conservative
dentistry. Br Dent J 2001;191(9):517-22.

June 2004 Journal of Dental Education 647

 Figure 3. The assessment forms are two-part. At the conclusion of the appointment one copy of the form is given to
the student and one to data entry.

648 Journal of Dental Education Volume 68, Number 6

those comments into both the students and the
facultys individual databases. Thus, the traditional
Socratic teaching method is revived as targeted learn-
ing takes place in a mentored learning environment.
Daily input into the colleges clinical management
system allows extraction of such data and immedi-
ate identification of trends and areas needing im-
provement for both faculty and students.
The development of objective student evalua-
tion forms encompasses the totality of general den-
tal practice, tied to the Baylor College of Dentistry
competency document. Thirteen forms linked to spe-
cific categories (disciplines/departments) of dental
care are now in use as the college moves towards
consistency among departments (Table 3).
As clinical procedures are performed, appro-
priate assessment forms are begun as part of the pre-
operative record. As the procedure progresses, both
nontechnical and technical aspects are evaluated by
attending faculty. Each procedure step must be ad-
equately accomplished (meet the clinical standards
contained in our calibration manual) before the stu-
dent is allowed to proceed to the next step. When the
procedure is complete, the two-part assessment form
serves as instant feedback to the student and raw
material for data entry. One copy of the form is given
to the student immediately, and the other is submit-
ted to data entry personnel. The entries are added to
the students permanent electronic record, creating a
real-time individual competency profile that super-
vising faculty can access to evaluate the student in
all areas. Similar profiles of faculty performance are
recorded (Figures 4 and 5).
Fourth-year faculty and graduating students
completed evaluation surveys about the effectiveness
of the Non-Graded Clinical Evaluation System in
May 2003. In addition, a statistical test was conducted
that compared the grade distributions from the tradi-
tional grading system to the progress grades now used
with the Non-Graded Clinical Evaluation System.
A final note about our grading methodology is
important. A summative (4-0 or A-F) grading sys-
tem remains in place for specific interim evaluations
of competence. Under the Non-Graded Clinical
Evaluation System, grades are awarded on progress
examinations, conducted for specific procedures at
prearranged times under controlled conditions. These
grades are based on specific criteria, evaluated by
multiple calibrated faculty, and serve to corroborate
student performance in a traditional way that is con-
sistent with requirements for graduate school appli-
cations and class rankings.
June 2004 Journal of Dental Education
Results
Ninety-nine percent of the fourth-year students
completed the satisfaction survey of the Non-Graded
Clinical Evaluation System (NGCES). The students
were asked to compare this system, which was used
in their fourth year, to the traditional grading system
they worked under during their third year. The over-
all results were quite favorable for the new system
(Table 4). Almost four out of five students thought
the NGCES provided a better learning environment.
A similar percentage of respondents also believed
their interactions with clinical faculty were improved
using the system. Accordingly, it was not surprising
that two-thirds of the students indicated they received
more constructive comments on their techniques with
the new system. The level of stress in the clinic has
been a concern of the students for many years. Over
70 percent of the fourth-year students thought the
system helped foster a less stressful clinic environ-
ment. As far as performance evaluation is concerned,
over 85 percent of the respondents thought their per-
formance was adequately evaluated under the new
system. Since this was the first year for the NGCES,
we asked the students whether their concerns were
addressed. Eighty-two percent indicated their con-
cerns had been addressed. Finally, we asked the
fourth-year students whether they would recommend
that the Non-Graded Clinical Evaluation System be
instituted in the third year as well. Here the responses
were positive overall, but there was some dissent.
About 71 percent of the students were in favor of
using it in the third year; however, 16 percent were
neutral on the question, and 14 percent were not in
favor of changing the third-year grading system.
Table 3. Primary clinical assessment forms
1. Oral Diagnosis and Treatment Planning
2. Endodontics, Post and Core
3. Direct and Indirect Restorations
4. Preventive Dentistry
5. Oral Disease Risk Assessment
6. Oral and Maxillofacial Surgery
7. Oral and Maxillofacial Radiology
8. Removable Prosthodontics
9. Periodontal Diagnosis and Treatment Planning
10. Periodontal Scaling, Root Planing, and Prophylaxis
11. Periodontal Surgery
12. Periodontal Re-evaluation
13. Pediatric Direct and Indirect Restorations
649
 A satisfaction survey was also administered to
the faculty of the General Dentistry Department. One
hundred per cent of the department faculty (seven
full-time and twelve part-time) completed the satis-
faction survey of the Non-Graded Clinical Evalua-
tion System. The faculty were asked to compare the
NGCES used during the 2002-03 academic year to
the traditional grading system with which they
worked previously. As in the student survey, the over-
all results were quite favorable for the new system
(Table 5). About 95 percent of the faculty thought
the system provided a better learning environment.
Over two-thirds of the respondents thought the learn-
ing environment was much better. Eighty-four
percent of the faculty also believed their interactions
with students were improved and indicated they pro-
vided more constructive comments on their tech-
niques with the new system. About 74 percent of the
faculty thought the clinic environment was less stress-
ful using the NGCES although most of these respon-
dents thought the stress level was less, not much
less. As far as performance evaluation is concerned,
almost 90 percent of the faculty thought they were
better able to evaluate performance under the sys-
tem. We also asked the faculty whether they believed
the new system helped them to be a more effective
teacher, and almost 90 percent responded affirma-
tively. Finally, we asked the faculty whether they

D4ENDO 33. Preserv. of Adj. Tooth/Soft Tissue 1

6. Adhere to Rules and Procedures 1 33. Margins 1
43. Other 3 35. Finish/Polish/Texture/Integrity 1
32. Anatomy, Surface Texture 2
D4ODTX 35. Anatomy, Surface Texture 6
33. Margins 3
4. Wrk Habits/Time Utilization/Punctual 1 36. Caries/Decalcification Removal 1
25. Radiographic Interpretation 2 37. Occlusal Reduction 1
26. Radiographic Interpretation 1 38. Occlusal Reduction 3
26. Dental Caries/Defective Restoration 2 39. Circumferential (taper/undercut) 1
27. Dental Caries/Defective Restoration 4 40. Margins 1
34. Charting 1 41. Anatomical Form 1
51. Other 3 42. Caries/Decalcification Removal 1
51. Prophylaxis 3 42. Occlusion/Interproximal Contacts 1
52. Positive Feedback 1 43. Margins 3
53. Other 12 43. Prep for Fabrication 1
44. Contours/Surface Texture 1
D4REST 45. Excess Cement Removal 1
3. Patient Record Management 5 45. Margins 3
4. Wrk Habits/Time Utilization/Punctual 8 46. Excess Cement Removal 1
6. Adhere to Rules and Procedures 5 47. Contours/Surface Texture 1
8. Instrument & Material Set-Up 2 48. Custom Tray 1
9. Infection Control/Cleanliness 2 50. (Final Restoration) Margins 3
19. Extension 7
20. Extension 1 D0168
20. Extension 3 51. Margins 1
20. Unsupposted Enamel 1 50. (Final Restoration) Margins 2
22. Proximal and Ginival Extension 1 52. Interproximal/Occlusal Contact 1
23. Axial Wall/Axiopulpal Line Angle 1 51. Interproximal/Occlusal Contact 1
23. Proximal and Ginival Extension 1 53. Finish/Polish/Shade 1
24. Axial Wall/Axiopulpal Line Angle 1 54. Tissue Management/Cement Removal 1
26. Retention 6 55. Other 2
27. Caries/Decalcification Removal 4 56. Interproximal/Occlusal Contact 1
27. Caries/Decalcification Removal 4 55. Other 1
28. Caries/Decalcification Removal 11 58. Other 35
29. Pulp Expos, Carious/Mech. Rec/Unrec. 1
30. Cleanliness of the Prep 2 D4SCAL
6. Adhere to Rules and Procedures 1

Total QA Grades: 186 Total Cards Graded: 624 29.81%

Figure 4. Quality assessment by instructor and grade card criteria

650 Journal of Dental Education Volume 68, Number 6

 Summary by Student

Procedure Chart Date Instructor QA Marks Comments

D2391 169360 06/11/2003 Steglich 28. Caries/Decalcification Removal 28. Unrecognized decalcification.

D2140 166699 06/12/2003 Stromberg 19. Extension 19. Pulpal floor shallow.
28. Question of caries or stain - student
handled it well.
38. Small interference - removed by student.
58. Preparation excellent.
D2140 166699 06/12/2003 Stromberg 28. Caries/Decalcification Removal
D2140 166699 06/12/2003 Stromberg 38. Occlusal Reduction
D2140 166699 06/12/2003 Stromberg 58. Other
D1205 166692 06/20/2003 Nelson 51. Prophylaxis 51. Slight calculus anterior lingual.

D2790 165717 06/23/2003 Stromberg 51. Margins 51. Light contact. Occlusion high - use
occlusal registration strips.
58. Excellent restoration. Excellent margins.
D2790 165717 06/23/2003 Stromberg 58. Other
D2392 166699 06/26/2003 Stromberg 35. Anatomy, Surface Texture 35. Lack of marginal ridge on #5. Indistinct
anatomy on 3 & 5.
58. Excellent margins. Excellent interproximal
contours.
D2392 166699 06/26/2003 Stromberg 58. Other
D2140 166699 06/30/2003 Stromberg 19. Extension 19. Marginal ridge very thin. Removed
marginal ridge #15.
26. Inadequate retention for amalgams 15 & 20.
35. Indistinct anatomy.
58. All decay removed - excellent. Excellent
contours of restorations. Well condensed
amalgam. Excellent margins.
D2140 166699 06/30/2003 Stromberg 26. Retention
D2140 166699 06/30/2003 Stromberg 35. Anatomy, Surface Texture
D2140 166699 06/30/2003 Stromberg 58. Other
D0150 168801 07/02/2003 Stromberg 53. Other 53. Excellent diagnosis. Excellent presentation.

D2750 166692 07/10/2003 Stromberg 39. Circumferential (taper/undercut) 39. Overprepared on distal facial area.
56. Contact not broad - want contact area.
D2750 166692 07/10/2003 Stromberg 56. Interproximal/Occlusal Contact
D2790 166699 07/14/2003 Gonzales 58. Other 58. Excellent prep. Very nice crown. Bulky on
distal but student corrected.

D0120 166812 08/27/2003 Baker 27. Dental Caries/Defective Restoration 27. Stained grooves.
D2750 166827 08/29/2003 Stromberg 46. Excess Cement Removal 46. Temp cement remaining at provisional
check.

D1205 169541 09/11/2003 Truong-Mai 51. Prophylaxis 51. Still had tartar buildup on a few teeth.

D2790 167509 09/09/2003 Wathen 52. Interproximal/Occlusal Contact 52. Open contact, adjacent restoration to be
replaced.
D2998 166692 09/16/2003 Stromberg 40. Margins 40. Uneven margins, too heavy chamber on M.
43. No template made, student unfamiliar
with Integrity. Margins short and open, no
anatomy. Requested student re-fabricate.
D2998 166692 09/16/2003 Stromberg 43. Prep for Fabrication
D2998 166692 09/16/2003 Stromberg 45. Margins
D2998 166692 09/16/2003 Stromberg 47. Contours/Surface Texture
D7140.SE168569 09/30/2003 Frohberg Surgical Technique/Judgement Improper elevator use.

D6750 167509 10/10/2003 Stromberg 3. Patient Record Management Finished 4:40PM (10 min. late)
3. QA form not signed for prep,
provisional - student responsibility
to get appropriate signature.

Figure 5. Quality assessment comments

June 2004 Journal of Dental Education 651

 Table 4. Results of the fourth-year student satisfaction survey
1. Does the Non-Graded Clinical Evaluation System provide a better learning environment?
Much Better Somewhat Better Neutral Somewhat Worse Much Worse
43.8% 34.8% 18.0% 2.2% 1.1%

2. Were your interactions with clinical faculty improved using the Non-Graded Clinical Evaluation System?
Much Improved Somewhat Improved Neutral Somewhat Worse Much Worse
36.0% 41.6% 19.1% 2.2% 1.1%

3. Did you receive more comments that helped you improve your technique using the Non-Graded Clinical Evaluation
System?
A Lot More A Few More About the Same A Few Less A Lot Less
27.0% 40.4% 27.0% 3.4% 2.2%

4. Was the clinic environment less stressful using the Non-Graded Clinical Evaluation System?
Much Less Stress Less Stress No Difference More Stress Much More Stress
39.3% 32.6% 25.8% 1.1% 1.1%

5. Do you feel that your performance was adequately evaluated under the Non-Graded Clinical Evaluation System?
85.4% Yes 14.6% No

6. If you had concerns about the Non-Graded Clinical Evaluation System, were they addressed?
82.0% Yes 18.0% No

7. Would you recommend the Non-Graded Clinical Evaluation System be instituted for the 3rd year as well?
Strongly Recommend Recommend Neutral Not Recommend Strongly Not Recommend
39.3% 31.5% 15.7% 9.0% 4.5%

Table 5. Results of the general dentistry faculty satisfaction survey

1. In your opinion, does the Non-Graded Clinical Evaluation System provide a better learning environment?
Much Better Somewhat Better Neutral Somewhat Worse Much Worse
68.4% 26.3% 0.0% 5.3% 0.0%

2. Were your interactions with students improved using the Non-Graded Clinical Evaluation System?
Much Improved Somewhat Improved Neutral Somewhat Worse Much Worse
52.6% 31.6% 15.8% 0.0% 0.0%

3. Did you provide more feedback to the students on specific elements of their techniques using the Non-Graded Clinical
Evaluation System?
A Lot More A Little More About the Same A Few Less A Lot Less
47.4% 36.8% 15.8% 0.0% 0.0%

4. In your opinion, was the clinic environment less stressful using the Non-Graded Clinical Evaluation System?
Much Less Stress Less Stress No Difference More Stress Much More Stress
26.3% 47.4% 21.1% 5.3% 0.0%

5. Do you feel you were better able to evaluate student performance under the Non-Graded Clinical Evaluation System?
89.5% Yes 10.5% No

6. Do you believe the Non-Graded Clinical Evaluation System helped you be a more effective teacher?
89.5% Yes 10.5% No

7. Would you recommend the Non-Graded Clinical Evaluation System be instituted for all clinical and preclinical courses?
Strongly Recommend Recommend Neutral Not Recommend Strongly Not Recommend
47.4% 26.3% 10.5% 15.8% 0.0%

652 Journal of Dental Education Volume 68, Number 6

would recommend that the NGCES be instituted for
all clinical and preclinical courses. As with the stu-
dents, the responses were positive overall, but there
was some dissent. About 74 percent of the faculty
were in favor of using the system in all clinical and
preclinical courses; however, 11 percent were neu-
tral on the question, and 16 percent were not in favor
of changing the grading system. Differences between
the responses of full- and part-time faculty were ex-
plored, and no significant differences were found.
As noted, the traditional grading system had a
tendency to foster leniency in grading. Figure 2 shows
the distribution of grades in the fourth-year class
using the traditional grading system for academic
year 2001-02. The grades here are highly skewed,
with over 91 percent of all grades in the two highest
categories. The average grade was a 3.79 on a four-
point system.
Under the Non-Graded Clinical Evaluation
System, grades are only awarded on progress exami-
nations. These examinations are conducted for spe-
cific procedures, at pre-arranged times, under con-
trolled conditions, and are graded on specific criteria
by multiple evaluators. Figure 6 shows the distribu-
tion of grades for the 2002-03 academic year. By
using grades from progress examinations rather than
daily grading, there are far fewer student grades and
a greater distribution of grades. Grades still tend to
# Grades
500
74.4% of all grades 3.5
400
300
200
100
0
1.0 1.5 2.0
Grades
Figure 6. Grade distribution fourth-year class, 2002-03
June 2004 Journal of Dental Education
be quite high, with almost three out of four grades in
the top two categories and an average grade of 3.59.
Although the mean grade for each grading sys-
tem was relatively similar, the distribution of progress
grades appears to be greater. To verify this, a t-test
was conducted to determine whether the means of
each distribution of grades were statistically differ-
ent. The results indicate that the average grades un-
der each system were statistically significantly dif-
ferent (p.001).
Discussion
The results of the student and faculty evalua-
tion surveys demonstrated a high level of acceptance
of the NGCES. About 80 percent of students and 95
percent of faculty indicated the NGCES provided a
better learning environment. Other related evalua-
tion criteria were also positively rated by students
and faculty. While many refinements will no doubt
be incorporated in the future, students and faculty
agree that better collegial relationships have resulted
in a more relaxed atmosphere and a better teaching
environment. Elimination of daily grades allows fac-
ulty and students to work together more easily as a
team, which appears to have improved both the qual-
ity and quantity of procedures completed. Future
2.5 3.0 3.5 4.0
653
 satisfaction surveys of both faculty and students will
elucidate this opinion.
The new clinical performance assessment has
not only resulted in a high level of both student and
faculty acceptance, but has also produced a broader
grade distribution among the class. Under the old
grading system, grades were not just highly skewed;
there were a lot of them as well. During the 2001-02
academic year, fourth-year students received 33,473
grades. Only progress grades were awarded to fourth-
year students during the 2002-03 academic year, re-
sulting in a total of 817 grades. Thus, a largely mean-
ingless accumulation of data was eliminated.
The faculty calibration and student assessment
process recently implemented follows the mission-
focused model proposed by Chambers et al.,20 con-
centrating on objective identification and documen-
tation of areas that require improvement by students
and faculty alike. The daily entry of performance data
defined by the assessment forms allows identifica-
tion of students who are not making satisfactory clini-
cal progress, so remediation can begin immediately.
These elements comprise the balance of the assess-
ment forms.
From an educational standpoint, faculty mem-
bers now have familiar templates that provide pre-
cious opportunities to teach. Under the previous sys-
tem, typical verbal feedback at the chair went largely
unremembered, and there was no reliable documen-
tation trail for tracking either student or faculty per-
formance. Teaching opportunities abound now be-
cause any failure to attain a level of predictable
clinical success for each procedural step requires a
check mark and may have written documentation of
the step and the means used to correct the shortcom-
ing. Daily entry of such raw data from the assess-
ment forms allows both students and faculty to mea-
sure themselves against our normative clinical
models.
The challenge while moving to a competency-
based educational system was stated simply: elimi-
nate the maximum number of problems identified in
the introduction while creating a mentored learning
environment conducive to enduring adult education
and lifelong collegiality between our graduates and
the school. The creation and implementation of such
a program are difficult and have yet to be completed.
At this point, however, it seems to be worth the ef-
fort. Our new system centralizes all student perfor-
mance data for responsible faculty to review on a
654
regular basis. Problems can now be identified and
pinpointed. Faculty can rapidly engage appropriate
remedial strategies to overcome student deficiencies
in either technical or nontechnical areas.
Students meet regularly with their primary fac-
ulty mentor (Group Leader). These sessions cover
all issues of concern, and students now typically re-
port their realization that the new clinical assessment
strategy is a nonthreatening evaluation system
wherein the information gathered is used to help them
achieve increased levels of clinical competency. Stu-
dents and faculty alike are now more willing to en-
gage in clinical discussions and the remediation pro-
cess.
At the same time that daily and cumulative stu-
dent performance profiles are generated from the
individual assessment sheets, similar data are gener-
ated for faculty performance. Since our system is
focused on best practices teaching, the informa-
tion is now available for faculty to self-assess and
strengthen their own weaker areas. A calibration
manual has been written to detail each item of the
evaluation forms and to discuss current clinical best
practices.
Another way to use the data generated by this
system is to assess curriculum deficiencies. We dis-
covered early in the first year of using the evaluation
system, for example, that an unusually high number
of students were leaving caries and decalcification
in their preparations. It was determined that more
time in preclinical courses should be spent assisting
students to recognize and remove decay from prepa-
rations, more attention paid on the clinic floor, more
teaching opportunities sought, and immediate
remediation with extracted teeth when the deficiency
persists beyond rare isolated incidents.
After the 2002-03 year, the system was adopted
by the curriculum committee for application in the
second and third years. Therefore, the upcoming
evaluation surveys for 2003-04 and 2004-05 will
include students who have attended Baylor under
both the old and the new systems, and all clinical
faculty will be included henceforth.
Conclusions
The new clinical performance assessment re-
sulted in a high level of student and faculty accep-
tance and produced a broader grade distribution
among the class.
Journal of Dental Education Volume 68, Number 6
 We perceive better collegial relationships be-
tween faculty and students, a more relaxed atmo-
sphere, and a better teaching environment. Elimina-
tion of grades seems to have allowed faculty and
students to work together more easily as a team.
Faculty report that it is easier to give detailed
written feedback to students and that the system is
uncomplicated and user-friendly. Faculty calibration
is better organized and sequenced, and helps new
faculty in adjusting to a satisfying teaching career.
The system creates a centralized information file for
each student for ease of tracking their progress in all
disciplines and departments, making it a simple mater
to identify areas that require improvement. Faculty
are able to quickly create customized remedial pro-
grams to overcome specific areas of concern. De-
partment chairs and program directors can objectively
measure faculty activity in the clinics to ensure on-
going calibration and development of faculty mem-
bers.
The task is far from complete, but early out-
comes of this assessment strategy are positive. The
new evaluation process appears to have reduced or
eliminated most of the problems listed in the intro-
duction. Expanded surveys will now compare stu-
dent and faculty responses to both old and new as-
sessment systems. Performance on licensing boards
will be compared, as will career satisfaction surveys
among our graduates. Faculty surveys will search for
the strengths and weaknesses of the new system and
will continue to compare faculty responses to using
the system.
In summary, we believe the new system has
provided a better environment for students and fac-
ulty alike and will be a very positive factor in sup-
port of Baylors Lifelong Colleague initiative for
the practicing community.
REFERENCES
1. Field MJ, ed. Dental education at the crossroads: chal-
lenges and change. Washington, DC: National Academy
Press, 1995:94-100.
2. Future of dentistry. Chicago: American Dental Associa-
tion, Health Policies Resource Center, 2001:19,21,88-114.
3. Oral health in America: a report of the surgeon general.
Rockville, MD: National Institute of Dental and Cranio-
facial Research, National Institutes of Health, U.S. De-
partment of Health and Human Services, 2000.
June 2004 Journal of Dental Education
4. Low DS, Kalkwarf KL. Assessing continued competency:
an approach for dentistry. J Am Dent Assoc 1996;
127(3):383-8.
5. Drucker P. The new workforce in the next society: a sur-
vey of the near future. The Economist, November 3, 2001:
8-11.
6. Dugoni AA. University of the Pacific School of Dentistry:
a dental school that serves the practicing profession. Cal
Dent Assoc J 2002;30(10):763-8.
7. Christie CR, Coun M, Bowers DM, Paarmann CS. Cur-
riculum evaluation of ethical reasoning and professional
responsibility. J Dent Educ 2003;67(1):55-63.
8. Shuman SK. Doing the right thing: resolving ethical is-
sues in geriatric dental care. Cal Dent Assoc J 1999;
27(9):694-702.
9. Glassman P, Chambers DW. Developing competency sys-
tems: a never-ending story. J Dent Educ 1998;62(2):
173-82.
10. Merriam SB, Brockett RG. The profession and practice
of adult education. San Francisco: Jossey-Bass Publish-
ers, 1997.
11. Candy PC. Self-direction for life-long learning. San Fran-
cisco: Jossey-Bass Publishers, 1991.
12. Brookfield SD. Understanding and facilitating adult edu-
cation. San Francisco: Jossey-Bass Publishers, 1992.
13. Knowles MS. The modern practice of adult education:
andragogy versus pedagogy. Chicago: Follett Publishing
Company, 1970.
14. Mossey PA, Newton JP, Stirrups DR. Defining, confer-
ring and assessing the skills of the dentist. Br Dent J
1997;182(4):123-5.
15. Hendricson WD, Kleffner JH. Curricular and instructional
implications of competency-based dental education.
J Dent Educ 1998;62(2):183-96.
16. OConnor P, Lorey RE. Improving inter-rater agreement
in evaluation in dentistry by the use of comparison
stimuli. J Dent Educ 1978;42(4):174-9.
17. Berrong JM, Buchanan RN, Hendricson WD. Evaluation
of practical clinical examinations. J Dent Educ 1983;
47(10):656-63.
18. Mackenzie RS, Antonson DE, Weldy PL, Welsch BB,
Simpson WJ. Analysis of disagreement in the evaluation
of clinical products. J Dent Educ 1982;46(5):284-9.
19. Kerlingen FN. Foundations of behavioral research. New
York: Holt, Reinhart, and Winston, 1966:516-7.
20. Chambers DW, Boyarsky H, Peltier B, Fendler F. Devel-
opment of a mission-focused faculty evaluation system.
J Dent Educ 2003;67(1):10-22.
21. McCann AL, Babler WJ, Cohen PA. Lessons learned from
the competency-based curriculum initiative at Baylor
College of Dentistry. J Dent Educ 1998;62(2):197-207.
22. Yip H-K, Smales RJ. Review of competency-based edu-
cation in dentistry. Br Dent J 2000;189(6):324-6.
23. Yip H-K, Smales RJ, Newsome PRH, Chu FCS, Chow
TW. Competency-based education in a clinical course in
conservative dentistry. Br Dent J 2001;191(9):517-22.
655