PALUMBO: MAKING ART SMART

Making Art SMART: Using Goal Setting to Measure Student Achievement and Teacher Effectiveness Jill Elaine Palumbo May 7th, 2013 ARTE 600: Assessment Dr. Taylor & Dr. Burton

PALUMBO: MAKING ART SMART Introduction: SMART goals have been used by business and marketing agencies as a way to improve staff performances and close achievement gaps since they were developed by

market analyst George Doran (1981). Goal setting has been used in response to the call for accountability in Virginia schools since the Alexandria City Public Schools (ACPS) developed a Performance Evaluation Program (PEP) in 2000 (Stronge & Tucker, 2005). Currently, SMART goals are becoming increasingly popular within the education sector as a way to evaluate teachers based on measurable student learning (A. Stratton, personal communication, March 9, 2013). Leading researcher in educational assessment, Robert Marzano, writes, Arguably the most basic issue a teacher can consider is what he or she will do to establish & communicate learning goals & track student progress (2001). Based on new teacher evaluation standards in Virginia that place a 40% value based on student progress, art teachers are now required to work with administration to create SMART goals that use pre-tests and post-tests to measure student performance growth. What are SMART goals? The word SMART is an acronym broken down into five criteria: Specific, Measurable, Action-oriented, Realistic, Time-bound (ONeill, 2000). Art teachers may further understand the criteria by outlining them as follows: Specific Identify a specific group of students and their learning needs and/or skills. Establish baseline data to determine student readiness and prior knowledge while planning for differentiation of instruction. Align your goals to specific national, state, and local standards of learning. Measurable Develop your assessment tools before instruction by identifying desired

results, determining acceptable evidence before you plan the learning experience and

PALUMBO: MAKING ART SMART instructional delivery. Pre-asses your students by using a pre test, and re-assess your

students regularly. Make sure the identified skills and content your lesson focuses on can be measured over time and repeated. Attainable Your SMART goal should be doable, yet challenging. Baseline data is

required to establish an attainable goal. Consider your students willingness and your willingness to learn and teach the subjects at hand. Focus on progress for all students, rather than achievement. Results-oriented- Your SMART goals should support school and district goals, be standardsbased and strategic while focusing on learning outcomes, not activities. It is important to establish baseline data and differentiate instruction so that all students are able to make progress. Art teachers must also plan differentiation for students that already possess mastery. Time-boundYour SMART goal must have a clearly identified time frame and a prioritized

objective breakdown that outlines attainment. Art teachers should plan for holidays, student absences, and other time related variables accordingly.

Simply described, SMART goals are developed by the teacher to show their effectiveness by measuring the learning progress of specific student groups.

Literature Review: Student Academic Goal-Setting Model The call for accountability (Stronge & Tucker, 2005, p. 54) within the Commonwealth of Virginia spurred the Alexandria City Public School (ACPS) system to implement the Performance Evaluation Program (PEP), in 2000. The teacher evaluations are

PALUMBO: MAKING ART SMART comprised of five components: 1. Formal observation, 2. Informal observations, 3. Teacher portfolios, 4. Academic goal-setting, and 5. Student achievement, and focuses on painting a

more authentic picture of the complex nature of teaching (p. 54). In this evaluative model teachers must set annual measurable goals related to student achievement. Originally the ACPS evaluation program sought to create a merit pay system that rewarded teachers who could produce data that supported evidence of student achievement. The underlying controversy behind such a system lay in how the summative evaluation could be considered as a way to focus on faults, though it purports being based on promoting professional development. Despite this concern, teachers in 25 schools are currently participating in Governor Bob McDonnells Virginia Performance-Pay Incentives Initiative. In 2011 the General Assembly approved McDonnells request for $3 million to reward educators in hard-to-staff schools based on student growth and other performance measures during the 2011-2012 academic year. The legislation authorizes incentive payments up to $5,000 for teachers earning exemplary ratings. In addition, incentive payments up to $3,000 based on performance during 2012-2013 are available for exemplaryrated teachers in participating schools with federal School Improvement Grants (Guidelines for uniform performance standards and evaluation criteria for teachers, 2013). ACPSs goal-setting evaluation seeks to use granular data from multiple sources, including, but not limited to student outcomes, in a value-added approach to student learning in order to promote teacher improvement. The purpose of academic goal setting is not to replace classroom observations or other means of documenting performance (p. 58), or be the sole measure of teacher effectiveness. Of note the ACPS goal-setting process does not validate teachers personal or professional goals such as attaining a masters degree or creating a classroom management

PALUMBO: MAKING ART SMART

document. Rather, the goals focus directly on student academic progress. In order for this to happen, teachers must implement assessments within their classroom that are fair, consistent, and measurable. This can be challenging in the non-subject tested teachers classroom where student assessments may be varied, qualitative, and formative. As such, the ACPS teacher evaluation considers the students grade level, the content area, and ability level in order to select student assessment measures that are closely aligned with the curriculum (p. 60). Using Stronges Goals & Roles Evaluation Model, ACPSs PEP evaluators look at teacher performance via a 3-tiered approach: 1. Performance Domains, 2. Performance Standards, and 3. Performance Indicators. Under this umbrella there are 17 performance responsibilities within 5 categories: 1. Instruction, 2. Assessment, 3. Learning Environment, 4. Communications & Community Relations, and 5. Professionalism. The PEP specialists use two tools to complete teachers summative evaluations: the performance indicators & the performance rubric (p. 60). Continuing this thread, teachers must make SMART (specific, measureable, attainable, realistic, and time-bound) goals in order to satisfy Virginia state law, which requires the performance evaluation of instructional personnel include measures of student academic progress (p. 61; VDOE, 2013). This data collecting encourages a reflective praxis by which teachers are able to identify focal points for improvement for their students and themselves. However, the examples of data analysis and student assessment strategies provided in this evaluation synopsis did very little to address non-tested subject areas. In fact, the example of the complete Goal-Setting Form was drawn from Algebra and focused on crunching the numbers (p. 64). What, then, would an art teacher write for his or her goals? Even so, the ACPSs evaluation method claims to foster a collaborative effort

PALUMBO: MAKING ART SMART between teachers, evaluators, and PEP specialists that empowers teachers by allowing them to determine the selection of their own goals and student assessment measures (p. 65). The advantages of the ACPS evaluation system include the ability to foster teacher reflection and data-driven decision making by emphasizing formative as well as summative evaluation. Goal setting encourages teacher collegiality and collaboration, and the PEP

specialists can assist evaluators and serve as instructional leaders thus enabling teachers to be active participants in their evaluations. The disadvantages of the ACPS assessment system include factors such as how student data can be misused or misinterpreted and that effectiveness is contingent upon well-trained, accessible PEP specialists. Evaluating teachers based on student academic progress can be threatening and increase stress, as well as time consuming. The formative aspects of academic goalsetting encourage mastery-learning practices with increased feedback, opportunities for non-high-stakes failures, and flexibility in changing methods of instruction. By pre-testing students, teachers are able to determine the base-line level and set achievable benchmarks. However, according to Marzano, Pickering and Pollack, instructional goals can: 1. Narrow what students focus on. 2. Should not be too specific, and 3. Should be used in collaboration with the teacher and the student (2001). In other words, there is an inherent danger of bias to consider when using goal-setting, one must be careful to see the full picture within the context of curriculum, school culture, and student learning needs. The interesting component within ACPSs evaluation model is the pivotal role of the PEP specialist, whose responsibilities include staff development, teacher training, data analysis, and providing continuous support to teachers. The authors recommend that a PEP specialist be housed at each school to enhance the effectiveness of the goal-setting process

PALUMBO: MAKING ART SMART (p. 69). This position might be compared to the Dean of Faculty at a private school. Currently, Marzano promotes an iObservation web-based platform that includes the use of classroom video observation to be used in house in as a kind of individualized PEP (Education Week, 2013). Virginia Department of Education Guidelines for Uniform Performance Standards and Evaluation Criteria for Teachers

According to the Virginia DOE guidelines for uniform performance standards and evaluation criteria for teachers (n.d.), all teachers are to be evaluated based on seven performance standards: 1. Professional knowledge, 2. Instructional planning, 3. Instructional delivery, 4. Assessment of and for student learning, 5. Learning environment, 6. Professionalism, and 7. Student academic progress. The newly implemented performance standard #7 (Student Academic Progress) attaches 40% of the summative rating of the teacher performance evaluation to student academic progress and is outlined as follows: Performance Standard 7: Student Academic Progress The work of the teacher results in acceptable, measurable, and appropriate student academic progress. Sample Performance Indicators Examples of teacher work conducted in the performance of the standard may include, but are not limited to: 7.1 Sets acceptable, measurable and appropriate achievement goals for student academic progress based on baseline data. 7.2 Documents the progress of each student throughout the year.

PALUMBO: MAKING ART SMART 7.3 Provides evidence that achievement goals have been met, including the state-provided growth measure when available as well as other multiple measures of student growth. 7.4 Uses available performance outcome data to continually document and communicate student academic progress and develop interim learning targets (VDOE, 2013). All teachers, including visual art teachers, in Virginia must use SMART goals to measure

student learning growth and academic progress using pre-tests and post-tests. SMART goals are described as: Specific, Measurable, Achievable, Results-focused/Relevant, and Timebound (VDOE, 2013; Meyer, 2003). The benefits of creating SMART goals lay in the power that an art teacher has in personalizing and tailoring their student assessments. Actually creating and implementing SMART goals that incorporate the criteria required are rigorous and measure learning valued in the arts is a task worthy of deliberate consideration (Stratton, 2013).

Using Student Growth Measurements to Assess Visual Arts Teachers. In an article from the Education Week teacher blog, Teacher in a Strange Land, national board-certified arts educator Nancy Flanagan (2012) summarizes a collective opinion regarding the use of standardized testing in the arts to evaluate teachers. She claims, the tests tell us nothing about how students will apply artistic skill and expression to their real lives and careers. Further, they tell us nothing about the instructional quality of their teachers. She goes further to state in no uncertain terms, We measure what we value. We can shoot to expand teachers' own assessment literacy in the arts. We can enhance their

PALUMBO: MAKING ART SMART instructional and curricular repertoires. But we won't raise teaching quality in the arts by creating standardized tests. The varied opinions on how to assess students in the visual arts has, in fact, been quite well researched and documented (Boughton, 2004; Davis, 1993; Day & Eisner, 2004; Eisner, 1996; Hetland, Sheridan, Veenema & Winner, 2007; Strong & Tucker, 2005; Wehlage, Newmann & Secada, 1996). It is either a blessing or a curse (Boughton, p. 588) that there has been no commonly adopted state or national standardized measure implemented. Proponents of using legitimized assessments and standards of learning would argue that, the issue of including art in the assessed category is an interesting one. Assessment is what makes you legitimate. Being assessed is the price you pay for being important (G. Reich, personal communication, February 13, 2013). Flanagan (2012) opposes using standardized tests in the arts as a measure of job security and states, this is like saying thank goodness for all those infarctions, because now we can staff our high-tech cardiac unit. The simple reality is that students learn in multiple ways just as teachers teach in multiple ways. There is no way to standardize this, nor should there be. The way we as people interact with each other in society is reflected in this concept. Holding a teacher to standards that are not relevant within his or her curriculum or the subject they teach is demoralizing and counterproductive (Flanagan, 2012; Schmocker, 2012). It is disconcerting that there is such an obvious disconnect between the research regarding how art educators are evaluated when, now more than ever, their evaluations are directly correlated and weighted according to perceived student learning and academic

achievement. Educators may feel wary about the purposes and aims of their assessments and may believe that, teacher evaluation will continue to be nothing more than what teachers and administrators have aptly called a dog-and-pony show, with one difference: It will be

PALUMBO: MAKING ART SMART

10

even more confusing and time-consuming (Schmoker, 2012). Educators may even fear the process and perceive it as a way to weed out teachers: Most of the teachers at my school see the new evaluation method the way a victim would regard a sniper: As a way to pick them off one by one (Schmoker, 2012). These are strong concerns and this feeds directly into the question regarding who is actually responsible for performing the assessments of art teachers and how to provide them with the data that demonstrates measurable student learning in the visual arts. According to Stronge & Tucker (2005), there may be many obstacles that exist with the subjective use of evaluation data when it comes to the performance evaluations of educators. They stress the importance to maximize the benefits and minimize the liabilities in linking student learning and teacher effectiveness (p. 96). The ways in which a student learns in the art teachers classroom may not be apparent to the evaluator, may be unknowledgeable about the field of visual arts. Strong & Tucker address this question stating, measures of student learning are vitally important to judging the effectiveness of teachers and schools, but should never usurp professional judgment that integrates knowledge of other factors that affect instruction (p. 96). The dilemma arises when the evaluator does not have a background or appreciation of visual art. Steve Peah (2012) brings some clarity to the conversation of teacher assessment and accountability. He states, Teacher resistance to evaluation is a red herring. The skill of evaluators, not the nature of evaluations, is the real issue.

Amongst the resources I reviewed specifically related to art teaching, Campos outlines how to use SMART goals and defines what goals are versus objectives. According to Campos (2009), a goal is a specific accomplishment you seek. A goal statement can be

PALUMBO: MAKING ART SMART

11

something general that you desire to achieve. An example of a goal is Students will learn the methods for developing a graphic novel. A set of objectives informs this goal. Campos defines objectives as a series of specific steps, strategies, or tasks that help achieve the stated goal. Examples of objectives in this case could include, Deliver sixteen 30 minute sessions of art content to students in class, Give students ## minutes per week to research their project, and Give students ## minutes per week to create their artwork for their comic book.

A general goal would be, "Become a good artist. But a specific goal would say, I want to become proficient at sketching subjects in my sketchbook by drawing from direct observation and practicing different techniques for 30 minutes after school 3 days a week and evaluating my work with a rubric to help me chart my progress (Zimmerman, E. 2011)

http://courses.edtechleaders.org/documents/Data_Driven/smart_goals_smart_schools.pdf In his article SMART goals, SMART schools for the journal in Educational Leadership, ONeill writes about an interesting debate that arises when the art teachers relay their Art SMART Goal. ONeill uses this example of an art teachers SMART goal: "Increasing by 10 percent the number of 5th grade students who meet or exceed expectations for drawing a realistic hand. ONeill then goes further to voice the concerns that art teachers have regarding the formulation of SMART goals, A few classroom teachers decry the "mechanization of art," claiming, "It destroys art as an aesthetic process!" One art teacher replies, "We teach skills, too. It's important that students and their parents see this growth (2000, p. 47). Art teachers who are now required to create SMART goals have voiced similar opinions in educational forums, I did a google search on ART SMART

PALUMBO: MAKING ART SMART goals--got over a million sites--ZERO for art education and this is taking all the joy of teaching away for me (Holmgren, 2004).

12

Advantages and disadvantages of creating and using SMART goals Advantages: SMART goals help increase motivation, measure progress, allow the teacher and student to take control of the progress, and allow the teacher to set priorities and make decisions earlier (Livestrong, 2012). SMART goals can be used as a powerful way to engage teachers, students, and parents as they increase accountability for all parties because students are able to actively participate in the learning process (SMART Goals in Education, 2012). SMART goals are a good way to communicate with administration and enable the teacher to reflect upon how they teach. SMART goals encourage you to produce better assessment tools, are mutually developed between administration and teacher, and can be used as a way to advocate for visual art.

Disadvantages: Misunderstanding SMART Goals While the criteria for SMART goals are, in themselves, good; there is potential to misunderstand how to use SMART goals in alignment with the standards of learning, and with school or district goals. I have found examples of SMART goals provided within training videos to be problematic. For instance, would you want your teaching performance to be measured on this approved SMART goal students will attend class 80% of the time over the course of the next semester, or this, during the second semester, our school will reduce student referrals to the office by 50%? In the first goal, the teacher has little to no control over the actual outcome. In the latter goal, the teachers performance would be rated

PALUMBO: MAKING ART SMART based on the entire schools performance, not to mention that the methods for keeping students in classrooms could prove to be highly counterproductive to producing a good learning environment. These SMART goals were developed to be in alignment with the

13

district goal Provide a safe and orderly environment for teaching and learning. Ironically, they do not actively address the root issues that may be causing truancy or delinquent behavior. Ultimately, neither goal measure student learning or achievement, yet they are deemed to be good examples that should be modeled. SMART goals that address particular standards of learning objectives for art teachers may be hard to develop because of the complexity of the conceptual development targeted. Art teachers may develop SMART goals that are not rigorous, in that they focus on production skills or easily tested tasks, such as making vocabulary lists or measuring negative space. The very definition of measurable is problematic when attached to student achievement. Measuring progress under the SMART goal pre and post testing requirements assumes that the route from point a to point b is a straight and incremental path. It is no wonder that, when creating their SMART goals, art teachers may be tempted to select paths that are easier to traverse, but may not lead students anywhere significant. Developing an artistic vocabulary is important and easily tested, however many art teachers agree that testing in this area does not represent the core value of a good arts curriculum. SMART goals are not useful unless art teachers have a meaningful curriculum with good student assessments already in place.

Standards of learning comparison in grades 2, 7, and art III (advanced intermediate) As discussed previously, all art teachers in Virginia are now required to create SMART goals aligned with standards of learning that measure student achievement by using

PALUMBO: MAKING ART SMART pre and post tests. The visual art SOLs represent four areas of investigation: 1. Visual communication and production, 2. Cultural context and art history, 3. Judgement and criticism, and 4. Aesthetics, and have been developed through the efforts of classroom

14

teachers, curriculum specialists, administrators, college faculty, professional artists, fine arts organization representatives, and museum personnel (Emblidge et al., 2006). The SOLs represented as follows were selected in order to comparatively demonstrate the potential difficulty an art teacher may encounter while creating SMART goals that would effectively measure the outcomes of student learning. The SOLs in italics represent more complex arts education concepts, contrasted with the SOLs in non-italics, which are easier to break down into simple task-oriented objectives, thereby appear easier to assess.

Grade 2: Visual communication and production 2.2 The student will incorporate unanticipated results of art-making into works of art. 2.4 The student will identify and use: 1. Secondary colors orange, violet, and green; 2. Shapes geometric and organic; 3. Three-dimensional forms cube, cylinder, sphere, pyramid, and cone; and 4. Pattern complex, alternating, and repeating. Cultural context and art history 2.13 The student will compare the art, artifacts, and architecture of other cultures with that of their own culture. 2.14 The student will identify symbols from various cultures. Judgment and criticism 2.19 The student will interpret ideas and feelings expressed in personal and others works of art. 2.17 The student will categorize works of art by subject matter, including portrait, landscape, & still life. Aesthetics 2.22 The student will discuss the ways that the art of a culture reflects its peoples attitudes and beliefs. Grade 7: Visual communication and production 7.11 The student will create works of art by representing and interpreting ideas from other fields of knowledge. 7.9 The student will use two-point perspective to create the illusion of depth in a two-dimensional drawing.

PALUMBO: MAKING ART SMART

Cultural context and art history 7.18 The student will examine the uses and impact of persuasive techniques in print and electronic media. 7.16 The student will identify styles and themes in contemporary and historical works of art. Judgment and criticism

15

7.24 The student will compare and contrast personal experiences with the life experiences depicted in works of art from other cultures. 7.19 The student will explore and identify subjects, themes, and symbols as they relate to meaning in works of art. Aesthetics 7.30 The student will investigate the purposes of art. 7.27 The student will generate questions about the nature of art and possible answers to the questions. Art III (Advanced Intermediate): Visual communication and production AIII.8 The student will demonstrate initiative, originality, fluency, commitment to tasks, and openness to new ideas in the creation of works of art. AIII.4 The student will use technology to create works of art that integrate electronic and traditional media. Cultural context and art history AIII.12 The student will research and analyze personally influential artists, art styles, and cultures that have contributed to the students developing portfolio. AIII.17 The student will categorize works of art by styles and cultures. Judgment and criticism AIII.22 The student will analyze the attributes of a work of art in terms of its ability to evoke a viewer response and command sustained attention. AIII.18 The student will use an expanded art vocabulary related to design, composition, aesthetic concepts, and art criticism when discussing works of art. Aesthetics AIII.26 The student will debate the perceived intentions of those creating works of art. AIII.24 The student will research, compare, and contrast the aesthetic ideals of two or more artists.

Based on my attendance at assessment workshop seminars at the National Arts Education Association annual convention (2013) and personal discussion with art teachers in Virginia, I gathered that art teachers feel hedged in with SMART goal criteria. It is difficult

PALUMBO: MAKING ART SMART

16

to develop goals that accurately represent the necessarily complex nature of many SOLs for visual arts; many of which describe art learning in a more authentic way. Therefore, art teachers may feel inclined to create SMART goals that outline a simple, measurable path of student learning, but may not accurately reflect the valuable concept development that is one of art educations biggest strengths. Also, art teachers that do not have strong student assessments in place may have a more difficult time in coming up with SMART goals that their administration recognizes.

References Campos, J. A. Jr. (2009). Goals and objectives for teachers (S.M.A.R.T.). Retrieved from http://www.icoe.org/webfm_send/4715 Doran, G. T. (1981). There's a S.M.A.R.T. way to write management's goals and objectives. Management Review, Volume 70, Issue 11 (AMA FORUM), pp. 3536. Emblidge, M. E., Ward, E. P., Brewster, T. M., Castro, I. M., Johnson, D. L., Jones, G. L., . . . Wright, P. I. (2006). Visual arts standards of learning: For Virginia public schools. Retrieved from http://www.lcps.org/cms/lib4/VA01000195/Centricity/Domain/7127/Art_sols.p df Guidelines for uniform performance standards and evaluation criteria for teachers (2013). Retrieved from http://www.doe.virginia.gov/teaching/regulations/uniform_performance_stds.pdf

PALUMBO: MAKING ART SMART Holmgren, M. (2004). TeacherArtExchange: RE SMART Goals. Retrieved from http://www.getty.edu/education/teacherartexchange/archive/Oct04/0043.html

17

Marzano, R. J., Pickering, D.J., & Pollock, J. E. (2001). Classroom instruction that works: Researchbased strategies for increasing student achievement. Alexandria, VA: Association for Supervision and Curriculum Development. O'Neill, J. (2000). SMART Goals, SMART Schools. Educational Leadership, 57(5), 46. SMART Goals in Education. (March 5, 2012). Retrieved from http://www.youtube.com/watch?v=aOnN1iVGMO4 Sparks, D. (1999, Winter). The singular power of one goal. Journal of Staff Development, 20 (1), 54 58 Stiggins, R. J., Arter, J. A., Chappuis, J., & Chappuis, S. (2004). Classroom assessment for student learning: Doing it right using it well. Portland, OR: Assessment Training Institute, Inc. Stronge, J. H., &Tucker, P. D., (2005). Linking teacher evaluation and student learning. Alexandria, VA: Association for Supervision and Curriculum Development. Zimmerman, M. (2011). Assessment: Empower students to be accountable and resourceful via formative and summative assessments while using differentiated instruction. Retrieved from http://www.arteducators.org/news/national-c convention/Zimmerman_NAEA_Assessment_2011.pdf