Overview

In December 2009, The New Teacher Project (TNTP) launched a partnership with the
Houston Independent School District (HISD) to assist with the core initiative of the district’s

AUGUST 2010

Findings from Analysis of Human

Capital Policies and Practices in
Houston Independent School District
Strategic Direction to ensure that there is an effective teacher in every classroom. During
Phase I of its engagement with HISD, TNTP conducted an extensive analysis of current
human capital policies and practices in the district in order to guide the design and
implementation of the core initiative. This memorandum provides a final summary of TNTP’s
analysis, which yielded four primary findings:

1. In order to attract top teaching talent, HISD must hire earlier, strategically
recruit from candidate pipelines that produce the most effective teachers,
and utilize rigorous selection processes to identify candidates with the best
likelihood of being effective. Page 3

2. The district’s appraisal systems do not accurately measure teachers’ ability

to promote student achievement, preventing HISD from using instructional
effectiveness to inform key human capital decisions.
Page 6

3. Teachers do not receive professional development that helps them improve

their performance in the classroom.
Page 12

4. HISD does not maximize opportunities to retain and leverage effective

teachers or to address teachers who demonstrate poor instructional
performance. Page 14

TNTP shared the results of this analysis with the HISD Board of Trustees in two separate
reports on April 29 and June 3, 2010. TNTP is engaged in the process of presenting its
analysis to other stakeholders and has conducted briefings with teachers, HISD central
administration staff, and community members. As TNTP enters the planning and
implementation phases of its work with HISD, it will continue to pursue and receive feedback
from stakeholders through targeted focus groups, community forums, and educator working
committees.

The New Teacher Project 1 August 2010

Methodology1 Survey Response Rates
TNTP’s analysis utilized a review of HISD Te
personnel and academic records, as well as ac
Pr
surveys of teachers, principals, and teacher in Te
he
applicants. HISD provided TNTP with personnel r
ci ac
data including records regarding teacher Ap
p her
appraisals, district rosters, hiring, and pli
al s
ca
separations. This data includes school-level s
nt
information, such as annual accountability s
ratings and student demographics. Data
provided by HISD’s Department of Research Responde
and Accountability allowed TNTP to examine 144 6,279 3,167
nts
value-added measures of teacher impact on
student growth and records regarding teacher Response
compensation under the ASPIRE program. 56% 55% 12%
Rate
TNTP administered surveys of HISD teachers,
principals, and teacher applicants in March,
April, and May of 2010, covering topics such as teacher appraisal, professional development,
compensation, hiring, and working conditions.

Value-Added Data
TNTP’s analysis includes data from HISD’s measures of teacher impact on student learning
from the Education Value-Added Assessment System (EVAAS). Each teacher with EVAAS
data2 receives a cumulative growth index (CGI) in up to five subjects. These indexed scores
allowed for a consistent benchmark across multiple years of data. Additionally, in order to
improve confidence in the analysis of value-added data, wherever possible, TNTP applied
two or three year average EVAAS scores in place of single year value-added scores.

When examining teacher effectiveness, it was useful to devise categories of effectiveness

using value-added data against which teachers could be compared. Unless indicated
otherwise, high-performing teachers are defined as being in the top decile of performers in
at least one subject and not in the bottom quartile of performers in any other subject when
using two or three year average EVAAS percentile ranks. Low-performing teachers are
defined as being in the bottom decile in at least one subject and not in the top quartile in
any other subject.

1 The response rates that the teacher, principal, and applicant surveys received (see sidebar) are on par with, if not
higher than, the response rates in surveys of other large districts in which TNTP has worked. Surveys of applicants
to a district typically yield lower response rates because the respondents are not district employees and thus are
less likely to respond to a district email inquiry.

2 Teachers teaching the tested subjects of Language Arts, Mathematics, Reading, Science, and Social Studies in
grades 3 through 8 currently receive EVAAS scores. In 2008-09, approximately 30 percent of classroom teachers
received EVAAS scores.

The New Teacher Project 2 August 2010

Finding 1: In order to attract top teaching talent, HISD
must hire earlier, strategically recruit from candidate
pipelines that produce the most effective teachers, and
utilize rigorous selection processes to identify candidates
with the best likelihood of being effective.
TNTP’s surveys of applicants to HISD between 2007 and 2009 identified a number of
weaknesses in the district’s recruitment and hiring processes that have had the cumulative
effect of causing the district to lose desirable teacher candidates. With more efficient
processes and more strategic recruitment and selection, HISD will be able to optimize its
supply of new teachers to better meet the needs of schools and improve student
achievement.

It should be noted that HISD has significantly altered its recruitment practices for the 2010
hiring season, and data from that year was not captured in our survey. While it is expected
that many of the issues faced by candidates in prior years have been improved upon, HISD
must continue to use student outcome data to evaluate whether the changes in its
recruitment and hiring practices have had the result of attracting new teachers who are
ultimately more effective in the classroom.

Summary of Data:
i. In prior hiring seasons, HISD has lost desirable candidates due to flawed
communication processes and a late hiring timeline.
ii. HISD should recruit strategically from the candidate pipelines that produce the most
effective teachers and pair those efforts with selection processes that can
demonstrably identify candidates likely to be effective in the classroom.

i. In prior hiring seasons, HISD has lost desirable candidates due to

flawed communication processes and a late hiring timeline.

Over the last three hiring seasons, teacher applicants to HISD have faced a lack of prompt
and clear communication. Respondents to TNTP’s applicant survey felt confused and
devalued by the application process, often due to unresponsiveness from the district and a
lack of clarity about how the hiring process would proceed.

Figure 1.1: Unhired Teacher Applicant Survey Responses Regarding Hiring

Process

Teacher Response Percentage Responses

Unhired teacher applicants who indicate that they never received a
62% 1,519
response from HISD
Of those who did receive a response, applicants who say that
32% 571
communication from HISD about the hiring process was timely3
Applicants who agree or strongly agree that communication from HISD
28% 564
about the hiring process was clear

3 Percentages indicate applicants who responded “agree” or “strongly agree.”

The New Teacher Project 3 August 2010

These delays and flawed communication processes caused the district to lose teacher
candidates. The uncertainty created by delayed and unclear communications from HISD
forced applicants to withdraw their applications or decline job offers from HISD to pursue
timelier job opportunities. Of the reasons cited for withdrawing an application or declining a
job offer, the length of the hiring process and the candidate receiving another job offer were
the most common.

Figure 1.2: Teacher Applicant Responses Regarding Reason for Withdrawing

Application or Declining a Job Offer

Teacher Response Percentage

I received another job offer. 70%
The application and hiring process took too long. 44%
The salary and benefits in HISD were insufficient. 17%
I was dissatisfied with the customer service of the HISD Human Resources
17%
department.
I had a negative interaction with a school principal or other district representative. 14%
The hiring timeline was “important” or “very important” in my decision to withdraw my 56%
application or decline a job offer. “I wanted to work for HISD but it took HR
forever to get back with me about even the
It is important to note that simplest answers.”
candidates who withdrew their – Secondary English Language Arts
applications or declined a job offer Applicant
included desirable prospects with
characteristics indicating the “I never heard back from schools I applied
potential to promote student to…when I finally did hear back from one, I
achievement. The average had already been offered a position…[at]
undergraduate GPA of candidates YES Prep.”
withdrawing their application or – Middle School Math/Science Applicant
declining a job offer was 3.37, as
opposed to the 3.17 average GPA of survey respondents who did not receive or accept an
offer from HISD.4 Though the number of responses from withdrawing or declining candidates
was relatively low, that group also appears more likely to be licensed in high-need areas,
such as those listed in the table below.

Figure 1.3: Applicants Withdrawing an Application or Declining a Job Offer

Certified in High-Need Licensure Areas

% of Overall Applicant
% of Declining % of Withdrawing
Population in Each
Licensure Area Applicants in Each Applicants in Each
Licensure Area
Licensure Area (n=29) Licensure Area (n=24)
(n=461)

Bilingual – Spanish 14% 17% 10%

ESL – Generalist 35% 29% 23%

4 Survey of HISD teacher applicants. For research linking undergraduate GPA to applicant quality, see Andrew J.
Wayne and Peter Youngs, “Teacher Characteristics and Student Achievement Gains: A Review,” Review of
Educational Research, 73 (1) (2003), 89-122; and Cathy W. Hall, Kris M. Smith, and Rosina Chia, “Relationship
Between Metacognition and Affective Variables in College Achievement,” National Social Science Journal 19(1),
(2002), 43-50.

The New Teacher Project 4 August 2010

Special Education 10% 25% 20%
Science 14% 0% 7%
i. HISD should recruit strategically from the candidate pipelines that
produce the most effective teachers and pair those efforts with
selection processes that can demonstrably identify candidates likely to
be effective in the classroom.

TNTP’s analysis indicated that while some preparation programs are better than others at
producing high percentages of effective teachers, all pipelines produce both high and low
performers. Analysis linking teacher effectiveness to degree-granting institutions produces a
complex picture with regard to the number of both high and low performing teachers;
however, some conclusions can be drawn from the aggregate data. For example, HISD has
almost a one in five chance of receiving a low performer from Sam Houston State, but less
than a one in ten chance of receiving a low performer from Texas A&M - College Station. The
latter program also produced a higher percentage of high performers.

Figure 1.4: Most Common Degree-Granting Institutions by Number of HISD

Teachers and Percent of Teachers in Each EVAAS Performance Level, 2009-105

Going forward, HISD will need to refine this data by grade, license area, and effectiveness
with sub-groups of students (e.g., English Language Learners or students multiple years
below grade level) in order to get richer data with which to inform recruitment strategies. In
addition, these efforts must be paired with rigorous selection and evaluation processes to
ensure the district only hires candidates that have the potential to be effective in the
classroom. For any new selection models that are adopted, HISD should analyze the

5 HISD Human Resources and EVAAS data. Bolded numbers on the left side of the chart indicate number of total
teachers with a degree from the listed institution. Figures in parentheses are the total number of teachers with
degrees from the listed institution who have multi-year value-added data. High-performing teachers are defined as
being in the top decile of performers in at least one subject and not in the bottom quartile of performers in any
other subject when using two or three year average EVAAS percentile ranks. Low-performing teachers are defined
as being in the bottom decile in at least one subject and not in the top quartile in any other subject.

The New Teacher Project 5 August 2010

demonstrated student achievement growth of the teachers hired through those processes to
ensure the models can consistently identify effective candidates.

The New Teacher Project 6 August 2010

Finding 2: The district’s appraisal systems do not
accurately measure teachers’ ability to promote student
achievement, preventing HISD from using instructional
effectiveness to inform key human capital decisions.

A wide body of research has established that teacher effectiveness is the single most
important school-based factor impacting student achievement. In order to maximize student
achievement, HISD must base key human capital decisions on accurate data regarding
teacher performance levels. These decisions (including compensation, retention, dismissal,
the granting of term contracts, and professional development) are currently made in a
vacuum of information, as teacher appraisal and survey data demonstrate that the district
currently has no reliable data differentiating teachers’ instructional performance. HISD must
develop new, comprehensive systems to appraise teacher performance that include
measures of a teacher’s impact on student achievement as a predominant factor.

Summary of Data:
i. Neither teachers nor principals believe the district’s current teacher appraisal
practices accurately assess teachers’ instructional performance.
ii. By rating almost all teachers as “Proficient” or “High Performing” regardless of their
ability to promote student achievement, the teacher appraisal process fails to
provide a basis for continuous professional improvement.
iii. The Staff Review Process conducted in March 2010 yielded a greater degree of
differentiation among teacher performance levels, pointing to potential opportunities
for the improvement of the teacher appraisal system.
iv. Teachers want to be assessed based on multiple factors including student
achievement growth, but have concerns about the district’s current value-added
measures.

i. Neither teachers nor principals believe the district’s current teacher

appraisal practices accurately assess teachers’ instructional
performance.

The current teacher appraisal systems in HISD, the Professional Development and Appraisal
System (PDAS) and the Modified Professional Development and Appraisal System (MPDAS),
evaluate teachers’ performance in eight domains, each of which is assessed on a four point
scale that ranges from “High-Performing” to “Unsatisfactory.” Teachers are appraised once
each year. Term or continuing contract teachers who are rated “Proficient” or higher in each
domain can choose to be appraised using MPDAS during the following year. Under MPDAS, in
five of the eight appraisal domains, teachers accept the previous year’s ratings as the
highest possible ratings they can receive that year, and appraisers gather data for those five
domains using walk-through observations rather than formal observations.

The New Teacher Project 7 August 2010

According to teachers and principals, PDAS and MPDAS do not accurately differentiate
teacher effectiveness, hindering HISD’s ability to assess instructional performance,
meaningfully inform professional development, and address instances of poor performance.

Figure 2.1: Teacher and Principal Opinions Regarding the PDAS/MPDAS Process6

Percentage of Teachers Who “Agree” or “Strongly Teachers Teachers Principals

Agree” That the PDAS/MPDAS Process… (PDAS) (MPDAS) (PDAS/MPDAS)
… allows appraisers to accurately assess teachers'
instructional performance. (i.e., the appraisal ratings reflect 44% 55% 28%
teacher effectiveness in promoting student achievement).

…recognizes effective instructional performance. 48% 56% 29%

...identifies and offers concrete steps to remedy poor

39% 46% 19%
performance.

...helps teachers improve their instructional performance. 43% 50% 20%

Total respondents 5,108 3,183 114

ii. By rating almost all teachers as “Proficient” or “High Performing,”

regardless of their ability to promote student achievement, the teacher
appraisal process fails to provide a basis for continuous professional
improvement.

One likely reason for principals’ and teachers’ lack of confidence in the current appraisal
system is that its results do not match the reality of the instructional performance that they
observe in their schools daily. Survey responses indicate that principals and teachers
recognize a significant percentage of low-performing and developing teachers in their
buildings, yet almost no teachers are rated “Below Expectations” or “Unsatisfactory” on
PDAS or MPDAS.

Figure 2.2: Average Perceptions of Teacher Performance Held by Principals and

Teachers Compared with Actual Teacher Performance on PDAS and MPDAS
between 2006-07 and 2008-097

An analysis of teacher appraisal results as compared with school-level student outcomes

indicates some differentiation in the percentage of teachers who receive the top rating
versus the second highest, however even at low-performing schools, very few teachers
receive ratings of “Below Expectations” or “Unsatisfactory.” This communicates to teachers
that all performance is at or above the desired standard and contributes to a persistent

6 Surveys of HISD teachers. Percentages indicate respondents who selected “Strongly agree” or “Agree.”

7 Surveys of HISD teachers and principals and HISD Human Resources records. Survey respondents were asked to
identify the percentage of teachers at their school performing in each of the four groups. These percentages were
calculated by averaging all of the respondents’ distributions. Principal n=81. Teacher n=2,871. The PDAS rating
curve indicates the percentage of teacher appraisals that had all domains scored in the indicated rating category or
higher over the last three years. n=32,345.

The New Teacher Project 8 August 2010

culture of lowered expectations for teacher performance and the flawed assumption that
most teachers do not have any room for improvement in their instructional practices.

Figure 2.3: PDAS/MPDAS Results for Teacher by School TEA Rating for the 2008-
2009 School Year8

Texas Education All Domains All Domains

All Domains At Least One Domain
Rated Rated "Below
Agency Rating of Rated "Exceeds
"Proficient" Or Expectations" Or
Rated
Schools Expectations" “Unsatisfactory”
Higher Higher
Exemplary 41% 57% 2% 0.0%
Recognized 32% 66% 2% 0.1%
Acceptable 23% 73% 4% 0.0%
Unacceptable 8% 85% 7% 0.3%
All Teachers 30% 67% 3% 0.1%

Figure 2.4: 2008-09 PDAS/MPDAS Results by Teacher from Schools Rated

“Academically Unacceptable” by the Texas Education Agency in 2008-20099

All Domains Rated

Schools Rated All Domains Rated At Least One
"Below
"Academically "Proficient" Or Domain Rated Total Teachers
Expectations" Or
Unacceptable" Higher “Unsatisfactory”
Higher

Attucks Middle School 40 0 0 40

Dowling Middle School 85 4 1 90
Fondren Middle School 42 1 0 43
Jones High School 43 2 1 46
Kashmere High School 39 3 0 42
Lee High School 92 7 0 99
Ryan Middle School 17 1 0 18
Sharpstown High School 67 8 0 75
Sterling High School 59 3 0 62
Westbury High School 93 8 0 101
Worthing High School 38 7 0 45

8 HISD Human Resources Records.

9 HISD Human Resources Records.

The New Teacher Project 9 August 2010

iii. The Staff Review Process conducted in March 2010 yielded a greater
degree of differentiation among teacher performance levels, pointing
to potential opportunities for improvement of the teacher appraisal
system.

During the 2009-2010 school year, HISD implemented a Staff Review Process as a
performance management measure. In this process, principals categorized each of the
teachers at their schools according to their assessment of the teacher’s level of
effectiveness. Principals met with members of HISD’s administration to discuss the evidence
for these categorizations and the actions that principals were taking to retain high
performers, provide struggling teachers with professional development, and pursue
nonrenewal or the extension of three-year probationary contracts to a fourth year for
underperforming probationary teachers..

In the Staff Review Process, principals differentiated the performance of their teachers to a
greater degree than through the PDAS/MPDAS process, and were much more likely to rate
teachers in one of the two lowest categories. Differences between the implementation of the
Staff Review Process and PDAS/MPDAS shed light on needed reforms of teacher appraisal.
First, the review of teacher ratings by HISD administrators and the requirement that
principals provide evidence for performance categorizations discouraged principals from
inflating their decisions. Second, in cases where teacher-level EVAAS data was available, the
information was used by principals and HISD administrators to inform performance
categorization decisions. Finally, principals were held accountable to act on the
categorizations they assigned their teachers (e.g., developing professional growth plans for
struggling teachers and working with HISD Professional Development Services to plan
training resources) so that the Staff Review Process was meaningfully connected to the
principal’s primary role as the instructional leader of his/her school. These three elements –
norming and oversight of appraisal ratings; incorporation of student outcome data; and
meaningful connections to key decisions – must be part of any redesign of the appraisal
system.

Figure 2.5: Rating Category Distribution from PDAS/MPDAS for 2008-2009 School
Year and Staff Review Process for 2009-10 School Year10

Highly
Low- Total
Effective/
Proficient Developing Performing/ Teacher
Exceeds
Unsatisfactory s Rated
Expectations
2009-10 Staff Review Process 28% 45% 21% 6% 11,249
2008-09 PDAS/MPDAS 30% 67% 3% 0% 10,572

Figure 2.6: Staff Review Process Results by Texas Education Agency School Rating
from 2008-09

TEA School Total

Highly Effective Proficient Developing Low-Performing
Rating Schools
Exemplary 38% 42% 17% 3% 102
Recognized 28% 47% 20% 5% 68

10 HISD Human Resources Records.

The New Teacher Project 10 August 2010

Acceptable 21% 44% 26% 9% 41
Unacceptable 12% 54% 26% 8% 11
Total Teachers 2,909 4,787 2,190 587

However, even in the context of the Staff Review Process, rating inflation and a culture of
low expectations persist. While teachers with multiple years of strong value-added results
were much more likely to be rated as “Highly Effective” or “Proficient” during the Staff
Review Process a significant number of low performers continued to receive a satisfactory
categorization. Of those teachers identified as low performers by their multi-year EVAAS
data, 36 percent received a Staff Review Process categorization of “Proficient” or higher.

Figure 2.7: 2009-10 Staff Review Process Categorizations by Multi-Year EVAAS

Performance Level11

Percent of
Staff Review Percent of EVAAS Percent of EVAAS
EVAAS High
Categorization Middle Performers Low Performers
Performers
Highly Effective 78% 34% 8%
Proficient 20% 44% 28%
Developing 2% 16% 42%
Low-Performing 0% 5% 22%
Total Teachers 308 1396 275

iv. Teachers want to be assessed based on multiple factors including

student achievement growth, but have concerns about the district’s
current value-added measures.

Since the promotion of student learning is a teacher’s foremost responsibility, an effective

teacher appraisal system must include measures of student growth as the predominant
factor. HISD has collected EVAAS data using the results of the Texas Assessment of
Knowledge and Skills (TAKS) and the Stanford Achievement Test for the last four years, and
is technically well-positioned to utilize its current growth measures as a factor in teacher
appraisal. HISD should combine current student growth measures with additional
assessments and processes to assess student learning gains in a wider range of subjects
and grades, and use these measures as the foundation of teacher appraisals. Survey data
from teachers indicate that most agree multiple factors, including student growth, should be
included in their appraisals.

Figure 2.8: Teachers Who Believe the Following Criteria Are “Very Important” Or
“Important” In A Fair and Comprehensive Evaluation System12

Evaluation Criteria Percent

Teacher mastery of content knowledge 96%
Student engagement in the classroom 94%
Classroom management 95%
Professionalism 90%
Instructional technique 89%

11 HISD Staff Review Process data and EVAAS multi-year averages.

12 Surveys of HISD teachers Percentage of respondents who selected “Very Important” or “Important.”

The New Teacher Project 11 August 2010

 Measures of teacher impact on student learning 88%
Evidence of preparation and planning 86%
Measures of student academic progress 83%
Improvement since last evaluation 76%
Total Respondents 4,853

Teachers want appraisals to include their impact on student performance, and they support
the use of tests to measure growth, as 87 percent of teachers believe that some form of
test-based assessment should be used to measure their impact on student learning.
However, teachers lack confidence in EVAAS, as only 18 percent of teachers surveyed in
March and April 2010 believe that it provides a fair and accurate measure of a teacher’s
impact on student learning. This uneasiness with value-added measures can be partially
explained by teachers’ lack of understanding about the measures themselves. After HISD
conducted trainings of teachers on EVAAS methodology in June 2010, a significant
percentage of participants indicated that their confidence in value-added measures like
EVAAS had increased. After participating in trainings, 56 percent of teachers expressed
belief in the system’s fairness and accuracy.13

Figure 2.9: Teacher Responses to the Question: “I believe HISD’s measures of a

teacher’s effect of students’ standardized test score growth, such as EVAAS,
provide a fair and accurate measurement of a teacher’s impact on student
learning”

Furthermore, teachers indicate support for the use of school- and teacher-generated
assessments in their appraisals in addition to TAKS or other state standardized exams. This
suggests that in addition to using measures of standardized test score growth, teachers wish
to have their impact on students be assessed using methods that they consider to be
authentic and connected to their daily practice.

Figure 2.10: The Percentage of Teachers Who Believe That These Individual
Components Should Be Used As Part of the Measure of Their Impact on Student
Learning 14

Measure of Student Learning Percent

Student standardized test scores from assessments such as TAKS or
37%
standardized end of course exams
Student standardized test score growth that takes into account where students
48%
begin the year, such as EVAAS
Student academic growth based on school- or teacher-generated assessments 72%
Student work products assessed according to district-wide, standardized rubrics 48%
Student graduation rates 23%
Student engagement levels 52%
School-wide student achievement 43%
Surveys of students in secondary grades 21%
Surveys of parents 27%
Teacher self-assessments 51%
Total Respondents 4,663

13 It should be noted that teachers participating in trainings responded with higher levels of confidence in EVAAS
prior to trainings than the average teacher responding to the March and April teacher surveys.

14 Surveys of HISD teachers and HISD exit survey of teachers participating in Foundation Level EVAAS Trainings
held between June 8 and June 28. Teacher survey n=4,505. Exit survey n=636. Data accurate as of July 28, 2010.

The New Teacher Project 12 August 2010

Finding 3: Teachers do not receive professional
development that helps them improve their performance
in the classroom.
Just as teacher appraisal systems must reflect the differences in the performance levels of
individual teachers, professional development must be responsive to individual growth
needs and cannot be conducted using a “one size fits all” approach. TNTP’s analysis
indicates that current professional development efforts, both central and at the school-level,
have not been effective at improving teacher practice or student learning. As HISD
restructures its performance management systems, it must consider new ways of aligning
the identification of teacher development needs with individually targeted, job-embedded
professional development opportunities.

Summary of Data:
i. Teachers do not currently receive professional development that addresses their
individual growth needs or helps them improve their ability to promote student
achievement.
ii. Principals do not have the time and training necessary to appraise and develop
teachers’ instructional performance.

i. Teachers do not currently receive professional development that

addresses their individual growth needs or helps them improve their
ability to promote student achievement.

Among the many repercussions of an ineffective teacher appraisal system, one of the most
detrimental is the inability to provide teachers with targeted professional development to
improve their instructional performance. HISD teachers reported that current professional
development offerings do not address their individual needs or help them improve their
ability to promote student achievement.

Figure 3.1: Percentage of Teachers Who “Agree” or “Strongly Agree” to the

Following Statements: The professional development I receive from HISD... 15

Teacher Response Percentage Respondents

...adequately addresses my individual needs as an educator. 51% 5,414
...is tailored according to the needs of my students as determined by
45% 5,390
classroom-based or standardized assessments.
...is tailored according to feedback and/or development domains from my
39% 5,379
PDAS/MPDAS.
...helps me to improve my effectiveness in promoting student
52% 5,369
achievement.

15 Surveys of HISD teachers. Percentages indicate respondents who selected “Strongly agree” or “Agree.”

The New Teacher Project 13 August 2010

Teachers and principals believe that individualized supports, such as observing other
teachers and participating in one-on-one coaching, are more effective in improving teachers’
instructional performance than school-wide programs. In making decisions about
professional development offerings, HISD should prioritize the offerings that are most
responsive to individual needs as determined by a comprehensive appraisal system and that
can be shown to have demonstrable positive effects on teacher performance.

Figure 3.2: “Effective” or “Highly Effective” Professional Development Methods16

Professional Development Support Teachers Principals

School-wide PD generated by school staff 53% 60%
School-wide PD generated by central HISD staff 47% 31%
School-wide PD generated by external vendors 46% 38%
Observing other teachers 68% 79%
Grade/subject level learning communities 70% 80%
One-on-one coaching 66% 82%

ii. Principals do not have the time and training necessary to appraise and
develop teachers’ instructional performance.

TNTP’s surveys of principals indicate that one of the major barriers to effective teacher
appraisal and development at the school-level is the lack of time and training. As HISD
redesigns its teacher performance management processes, it is clear that significant
alterations to principal time allocation and training must be considered in order for teacher
professional development and appraisal to be conducted successfully.
• Only 18 percent of principals feel that they have sufficient time to focus on teacher
appraisals and development.
• Only 32 percent of principals feel that they are given adequate training and support
from HISD on how to appraise and develop teachers’ instructional performance.

Figure 3.3: Principals’ Reported Average Time Allocation17

Actual Time Desired

Principal Activity
Spent Time Spent
17% Visiting classrooms and observing teachers 32%
7% Providing teachers with feedback on instruction 14%
6% Planning and delivering professional development 7%
16% District required paperwork 5%
8% Reviewing student achievement data 9%
9% Attending staff or district meetings 3%
57 Total Respondents 55

16 Surveys of HISD teachers. Percentage of respondents who selected “Highly Effective” or “Effective.”

17 Surveys of HISD principals. Average “Actual’ and “Desired” time allocation taken from principals who indicated
that their time is not distributed in a way that best supports student growth.

The New Teacher Project 14 August 2010

Finding 4: HISD does not maximize opportunities to retain
and leverage effective teachers or to address teachers
who demonstrate poor instructional performance.

To make dramatic improvements in teacher effectiveness and student achievement, HISD

must implement specific strategies to retain its strongest teachers effectively while
leveraging their talents to improve the performance of other educators throughout the
district. Simultaneously, HISD must provide teachers who consistently demonstrate that
they are less effective at promoting student achievement growth with the opportunity to
improve their performance and take action to exit teachers who do not improve. To reward
strong performers, HISD has successfully built a school improvement model, ASPIRE, which
includes performance pay based on teachers’ and schools’ impact on student learning
growth. However, TNTP’s analysis indicates that those strategies to encourage the retention
of high-performers must be expanded upon and coupled with more efficient means of
removing ineffective teachers.

Summary of Data:
i. There is limited variability in the retention rates of HISD’s highest and lowest
performing teachers.
ii. HISD teachers do not feel confident that ASPIRE awards are reflective of their
classroom performance, likely due in part to the use of fluctuating single-year
EVAAS results.
iii. Many teachers support a restructuring of compensation that would make
advancement on the salary schedule reflective of student learning growth, which
could be accomplished by diverting resources from salary increases based on
factors unrelated to teacher effectiveness in HISD.
iv. HISD has failed to exit consistently ineffective teachers.

i. There is limited variability in the retention rates of HISD’s highest and

lowest performing teachers.

TNTP’s analysis indicates that the district’s highest and lowest performing teachers, as
identified by value-added measures of teacher impact on student growth, are retained at
similar rates. While in some subjects, such as Math, teachers in the EVAAS 90 th percentile or
above are retained at higher rates than those in the 10 th percentile or below, in other
subjects, such as Language, the retention rates are identical.

Figure 4.1: Retention Rates for Teachers Scores in the Top and Bottom Deciles of
EVAAS Scores in 2008-0918

Retention Rate between 2008-

EVAAS Subject EVAAS Percentile
09 and 2009-10

Reading (n=252) 90th and above 95%

10th and below 92%
Math (n=257) 90th and above 98%
10th and below 91%
Language (n=235) 90th and above 96%

18 HISD Human Resources and EVAAS Data. EVAAS percentile determined by two and three year EVAAS averages.

The New Teacher Project 15 August 2010

 10th and below 96%

The responses to TNTP’s survey of HISD teachers indicate that the district’s highest
performing teachers, those who have demonstrated high levels of student growth across
multiple years, do not consistently plan to remain in the district longer than low performers.

Figure 4.2: Teachers Planning to Leave HISD within Three Years in the Top and
Bottom Deciles of EVAAS Scores19

Percent of Teachers Who Plan

Subject EVAAS Percentile to Leave HISD Within Three
Years

Reading (n=99) 90th and above 19%

10th and below 24%
Math (n=97) 90th and above 23%
10th and below 13%
Language (n=91) 90th and above 11%
10th and below 23%

Comments from Teachers Planning to Leave HISD in the Next Three

Years

“There are very few opportunities for monetary/professional advancement for

effective teachers. Effective teachers are not being identified, and ineffective
teachers continue to make the load heavier for the whole school.”

– Elementary Bilingual Teacher

“Teacher leadership opportunities are limited. Although many of my colleagues

and I have come into teaching from other careers (lawyers, consultants, engineers)
we are seen as having a very small role and skill-set…Regarding professional
development, if I learn and master the limited high quality professional
development I occasionally receive, I am sentenced to be bored over the next
several years as I am forced to hear the same thing over and over. We differentiate
for student learners, but not for teachers…I wish the district wouldn't just focus on
how to get rid of the few bad teachers, but also on how to keep the good ones. ”

– Secondary Reading Teacher with Three-Year EVAAS Performance in the Top

Quartile

19 Surveys of HISD teachers.

The New Teacher Project 16 August 2010

i. HISD teachers do not feel confident that ASPIRE awards are reflective
of their classroom performance, likely due in part to the use of
fluctuating single-year EVAAS results.

Beginning in 2006, HISD launched the ASPIRE Award Program and has distributed rewards to
teachers, administrators, and other school-based staff using student achievement outcomes,
including value-added measures of teacher impact on student learning. The ASPIRE program
has had a measurable impact on the retention of effective teachers 20, but since the retention
rate of teachers receiving the highest amount of individual financial reward is only slightly
higher than that of teachers receiving no individual reward at all, it is clear that HISD has not
maximized the effect of its incentive program. HISD should take additional steps to refine its
teacher compensation structure in a way that allows it to reward effective teaching and
encourage high-performing teachers to remain in HISD.

Figure 4.3 2008-09 Teacher Retention Rate and Amount of 2008 Strand 2
Individual ASPIRE Award21

Amount of Strand 2 Individual Award Retention Rate

$0 92.3%
$1 - $625 92.7%
$626 - $1875 93.1%
$1876 or more 93.8%
Total Teachers Considered 10,370

HISD has conducted annual surveys of teachers on their opinions regarding the ASPIRE
Award Program. Those surveys indicate that while most teachers are not opposed to the
concept of performance pay, the limited impact of the ASPIRE program on the retention of
highly effective teachers can be partly explained by the fact that many teachers are
confused about the program’s implementation and unsure if it is executed accurately and
fairly.

Figure 4.4: HISD Surveys Regarding ASPIRE Awards 2006-07 through 2008-0922

2006- 2007- 2008-

Teacher Response
2007 2008 2009
Respondents “in favor” or “somewhat in favor” of the concept of teacher
57% 64% 55%
performance pay
Respondents “in favor” or “somewhat in favor” of the ASPIRE reward
45% 53% 47%
program

20 SAS Institute, “Analysis of a Teacher Pay-For-Performance Program: Determining the Treatment Effect and
Overall Impact.”

21 HISD data on ASPIRE individual awards, Human Resources separation data, and a survey of current HISD
teachers.

22 HISD Surveys on ASPIRE Program conducted annually beginning in 2008. Some questions were not asked in
every year of the survey. The 2008-2009 survey was distributed soon after the Board of Education approved the
use of value-added data in teacher appraisals. The uncertainty felt by many teachers as a result of this policy
change is likely reflected in their survey responses.

The New Teacher Project 17 August 2010

 Respondents who stated “absolutely” or “mostly” that there is a connection
40% 45% 38%
between classroom instruction and performance pay results
Respondents who state that they understood "most aspects" of the
performance pay model paid out that year or that they understood it 27% 39% 17%
"completely"
Respondents who rate their level of understanding of how the ASPIRE
22% 27% 22%
Awards were calculated as “high” or “very high”
Respondents who indicate that they have received training about ASPIRE 85% 79% 71%
Respondents who believe that the maximum ASPIRE award amount
N/A 32% 29%
adequately recognized their efforts to increase student progress

In part, these concerns may stem from the instability of the single year value-added scores
that are used to calculate ASPIRE Awards. Over three years, significant percentages of
teachers shift from the top and bottom quintile of EVAAS scores in each subject, and the
majority of teachers have had both positive and negative value-added scores. Fluctuation in
single-year scores may contribute to a lack of credibility among teachers regarding the
reward system’s ability to reward excellence accurately. Without teachers’ trust in the
district’s value-added measures, ASPIRE cannot effectively incentivize retention and
effective instructional practice. Therefore, HISD should consider using multi-year value-
added scores to inform financial incentives and other means of retaining effective teachers.

Figure 4.5: Teachers with Fluctuating EVAAS Scores across Three Years of Data23

Teacher Teacher Appeared Teacher Had Total Teachers

Appeared in Top in Either 1st and Positive and with Three
EVAAS Subject
and Bottom 4th or 2nd and 5th Negative CGI Years of
Quintile Quintile Scores EVAAS Data
Language 21% 39% 73% 694
Math 16% 34% 70% 549
Reading 18% 37% 71% 726
Science 19% 33% 75% 192
Social Studies 17% 36% 73% 240

ii. Many teachers support a restructuring of compensation that would

make advancement on the salary schedule reflective of student
learning growth, which could be accomplished by diverting resources
from salary increases based on factors unrelated to teacher
effectiveness in HISD.

In addition to financial bonuses, many teachers support a more significant restructuring of

the compensation system that would allow for differences in baseline pay according to
classroom performance. The majority of teachers stated they would agree to a differentiated
salary schedule based on student learning growth. A change such as this would be a
departure from the current compensation system, which includes a traditional salary
schedule along with performance-based bonuses.

Figure 4.6: Agreement with: “To what extent do you agree that the following
factors would be an appropriate part of a compensation system that fairly
rewards teachers for instructional excellence?”24

23 HISD Human Resources Records.

The New Teacher Project 18 August 2010

 Teacher Response Percent
Accelerated salary schedule advancement for teachers who consistently
61%
show high levels of student academic growth
Opportunities for school leadership positions for teachers who show high
65%
levels of student academic growth
Financial rewards for all teachers in a grade or subject based on the
62%
academic growth of students in that grade or subject
Financial rewards for all teachers in a school based on the academic
66%
growth of students in that school
Financial rewards based on annual appraisal results 47%
Increased baseline salaries and salary step increases for all teachers 78%
Total Respondents 4,615
The current teacher compensation structure in HISD and most districts around the country,
rewards teachers based on seniority and advanced degrees rather than on classroom
performance. TNTP’s analysis of teacher performance based on value-added measures in
HISD indicates that neither seniority nor advanced degrees are reliable predictors of a
teacher’s effectiveness. In some subjects, like Language and Reading, high performers tend
to be more experienced than low performers, but the opposite is true in subjects like Math
and Science. Similarly, across subjects, having an advanced degree does not appear to have
an impact on a teacher’s effectiveness in the classroom. This data indicates that while
teacher experience and advanced degrees can be valuable assets in the classroom, HISD
should implement financial incentives that directly encourage the promotion of student
achievement growth rather than indirect teacher inputs.

Figure 4.7: The Average Number of Years Taught by a Teacher in HISD by EVAAS
Percentile and Subject Area (n=2075)25

EVAAS 90th EVAAS 11th to 89th EVAAS 10th Percentile

EVAAS Subject
Percentile or Above Percentile or Below
Language (n=1760) 15.9 14.4 11.4
Math (n=1645) 10.5 13.1 13.5
Reading (n=1736) 14.2 14.0 13.5
Science (n=1229) 12.2 12.6 15.7
Social Studies
12.3 13.5 12.7
(n=1331)

Figure 4.8: Percent of Teachers with an Advanced Degree by EVAAS Percentile

and Subject Area 26

EVAAS 90th EVAAS 11th to 89th EVAAS 10th

EVAAS Subject
Percentile or Above Percentile Percentile or Below
Language (n=1760) 28% 30% 30%
Math (n=1645) 25% 29% 32%
Reading (n=1736) 31% 30% 34%

24 Surveys of HISD teachers. Percentages indicate teachers who responded “Strongly Agree” or “Agree.”

25 HISD Human Resources Records.

26 HISD Human Resources Records.

The New Teacher Project 19 August 2010

Science (n=1230) 25% 31% 33%
Social Studies
(n=1332) 27% 30% 32%

A restructuring of the teacher compensation system that would incentivize excellent

performance and retain highly effective teachers could reallocate financial resources away
from salary schedule increases based solely on seniority and advanced degrees and towards
salary increases based on demonstrated classroom performance.

The New Teacher Project 20 August 2010

iii. HISD has failed to exit consistently ineffective teachers.

HISD’s inability to retain high-performers at a significantly higher rate than low-performers

has been compounded by a failure on the part of the district to capitalize on opportunities to
exit consistently ineffective teachers. Very few teachers are exited from the district due to
their performance in the classroom, including those whom principals indicate were poor
performers. This includes probationary teachers whose contracts make nonrenewal for
performance reasons much easier than the nonrenewal of teachers who have received term
contracts after completing their third year of teaching.
• 0.6 percent of all probationary teachers between 2005-2006 and 2008-2009 were
nonrenewed for performance reasons.
• 56 percent of principals say that at least one poorly performing probationary teacher
at their school received a term contract in the last five years.
• In the last five years, 50 percent of principals have not even attempted to nonrenew
or terminate a poorly performing term or continuing contract teacher.

As in other school districts across the country, the number of continuing or term contract
teachers who have been nonrenewed or terminated for performance reasons is also
extraordinarily low. To address this problem, HISD must couple more effective means of
identifying poor performance through the appraisal system with more efficient methods of
exiting teachers whose performance does not improve after receiving targeted support.

Figure 4.9: Percentage of Continuing or Term Contract Teachers Terminated or

Nonrenewed for Performance by School Year27

Teachers Exited Total Continuing

Percentage Exited for
Year from HISD for or Term Contract
Performance
Performance Teachers
2005-2006 7 0.1% 8,482
2006-2007 12 0.1% 8,504
2007-2008 13 0.2% 8,459
2008-2009 18 0.2% 8,562

27 Termination and nonrenewal data provided by HISD Professional Standards Office.

The New Teacher Project 21 August 2010

Observers may question the utility of exiting underperforming teachers for fear that the
teachers replacing them may not be significantly more effective. However, when comparing
the performance of HISD’s lowest performing teachers to the performance of average first
year teachers, an analysis of value-added data indicates there are groups of teachers that
are demonstrably less effective than new teachers. Replacing HISD’s least effective teachers
would therefore yield immediate benefits to student performance.

Figure 4.10: Distribution of Teachers’ Mathematics 2- and 3-Year Average Value-

Added Scores28

28 HISD Human Resources Records. Average performance of lowest 10% of teachers is based on two and three
year value-added data. Average performance of first year teachers is based on single year data from SY 2008-09.

The New Teacher Project 22 August 2010

Conclusion
In response to data such as the findings presented above, HISD has made the
transformation of human capital management a top priority and included “An Effective
Teacher in Every Classroom” as one of the five core initiatives at the heart of its Strategic
Direction. This Core Initiative includes four key strategies to address the challenges
described above:
• Strengthen recruiting and staffing policies and practices to attract top talent.
• Establish a rigorous and fair teacher appraisal system to inform key decisions.
• Provide effective individualized support and professional development for teachers.
• Offer meaningful career pathways and differentiated compensation to retain and
leverage the most-effective teachers.

TNTP has been working with HISD leadership to develop a five-year master plan to transform
the district’s human capital systems and lead to HISD having an effective teacher in every
classroom. This plan includes the development of new tools and processes, coordination and
monitoring of school-level implementation activities, and building stakeholder buy-in for
reform. Throughout the five-year initiative, data on implementation outcomes will
continuously be monitored and real-time adjustments will be made to strategies and tactics,
as required. Ambitious, concrete goals have been set for the success of the Effective
Teachers Core Initiative by the 2014-15 school year, including:
• Fewer than 10 percent of new hires are ineffective in their first year in HISD as
identified through the Staff Review Process and annual teacher appraisal instrument,
and at least 50 percent of new hires are effective or better by their third year in the
district.
• The distribution of teacher appraisal ratings at the district and school-level is aligned
with student performance outcomes and with survey data indicating teachers’ and
principals’ assessments of teacher performance.
• 100 percent of probationary teachers who receive a term contract have performed at
the “Effective” level or above, and at least 90 percent of teachers rated “Ineffective”
either improve their performance to a level above “Ineffective” or are removed from
teaching in HISD within one year of receiving the “Ineffective” rating.
• In surveys, at least 85 percent of teachers and 90 percent of principals agree or
strongly agree that teacher appraisal in HISD is a fair and accurate measure of
teacher performance.
• Annual appraisal data shows a significant increase in the overall effectiveness of
HISD’s teaching force, and a significant percentage of teachers improve their
performance at least one rating level from year to year.
• At least 85 percent of teachers and at least 90 percent of principals agree or strongly
agree that HISD’s teacher support and development systems and processes address
teachers’ individual needs and help them improve their performance.
• HISD retains highly effective teachers at a rate at least twice the rate of teachers
rated “below effective” as measured through the Staff Review Process and teacher-
level value-added data, when available.
• At least 85 percent of teachers and at least 90 percent of principals agree or strongly
agree that HISD is doing enough to leverage and retain its most effective teachers.

The New Teacher Project 23 August 2010

Further information regarding HISD’s core initiative of “An Effective Teacher in Every
Classroom” can be obtained by emailing effectiveteachers@houstonisd.org.

The New Teacher Project 24 August 2010