Assessment Summit, Spring 2012

Authentic Assessment & Tools to Assess Suzanne Evans, sevans@nu.edu SHHS Cynthia Chandler, cchandler@nu.edu SOE

Cycle of Assessment Choosing the Right Assessment Tools Cognitive, Affective, and Psychomotor Domain Digital Blooms & Web 2.0

Identif y

Improv e

Assess

Evans & Sistek-Chandler, 2012, Assessment Summit, National University, La Jolla, CA

As

Interpr et

Evans & Sistek-Chandler, 2012, Assessment Summit, National University, La Jolla, CA

As

Choosing the Right Assessment Tools

Assessme nt Tool Abbreviatio n
Multiple Choice Exam Exams Standardized Tests

Dat a

Direc t or or D D D D

Bloom's Knowledge, Comprehension, Application or Analysis/Synthesis/Evaluation I K, C, A, K, ASE C if Pros

Easy to grade objective

carefully construct ed A, S, & E K, C, A K, C, A

Reduces assessment to multiple choice answers No authentic testing, outdated? Heavily dependent on exposure to topics on test.large Can minimize picture and interrelatedness Evaluation feedback is basically a Yes/no - present/absent Time consuming to grade, can be subjective Creating the case is time consuming, dependent on student knowledge form multiple to grade due to Difficult areas multiple Methods and potential multiple solutions Difficult for ESL students Stressful for students takes course time Must fairly grade course content beyond delivery Requires good rubric More than one evaluator is helpful difficult for ESL students stressful for students Takes course timedefined Must have clearly criteria and evaluative measures "The look" can not override the content grade, More difficult to requiring a Checklist or rubric for a variety of different answers Difficult for some students to do on the spot

Con s

Easy to score and compare Comparable between students Very useful for skills or performances

Checklists

Variable

Essay Case Study

D D

Problem Solving

Oral Speech

Debate

Product Creation & Special Reports Flowchart or Diagram

Students know exactly what is missing analytical and Displays K, C, A, synthetic thinking well ASE Displays analytical and synthetic thinking well K, C, A, Connects other knowledge to ASE topic Displays analytical and synthesis K, C, A, Authentic if real world ASE situations are used. Easily graded with rubric Allows other students to see Variabl and learn what each student e learned connects general education K, C, A, goals with ASE Provides immediate feedback to the Student reveals thinking and K, C, A, ability to respond based on ASE background knowledge and critical thinking ability Students can display skills. knowledge, Variabl e and abilities in a way that is K, C, A, suited to them ASE Displays original synthetic thinking on C, A, ASE the part of the student Perhaps the best way to display overall high level thinking and articulation

Evans & Sistek-Chandler, 2012, Assessment Summit, National University, La Jolla, CA

Data Bloom's Assessme Direct Knowledge, Comprehension, Application or nt or Analysis/Synthesis/Evaluation Indirect Tool Abbreviati D or I K, C, A, Pro Con Provides the students with a Time consuming to on ASE s s
Portfolios Variable clear record of their work and growth Best evidence of growth and change over time Students can display skills. knowledge, and abilities in a way that is suited to them promotesgood summative data Provides self-assessment Easy to manage data if Likert-scaled responses are used Provides best display of skills and abilities provides excellent opportunity for peer review students can display skills. knowledge, and measure in a Best method to abilities growth overtime with regards to a course or program cumulative

grade Different content in portfolio makes evaluating difficult and may require training Bulky to manage depending on size Likert scales limit feedback, openended responses are bulky to manage Stressful for students May take course time some students may take the Evaluation very hard evaluative statements must be carefully framed Focus on breadth of assessment is important Understanding all the variables to produce assessment results is also important May result in additional course requirements Requires coordination and Must fairly grade individuals as well as team Grading is slightly more complicated student interaction may be to support Must use evidence a conclusions, not just self- opinionated assessment may be Respondents influenced by factors other than those being considered Validity and reliability most be closely watched

Exit Surveys

D, I

ASE

D Performance Variabl e K, C, A, ASE

Capston e Project or Course

ASE

D Team Project

Reflective selfnt essay Satisfaction and Perception Surveys

D, I

Variabl e K, C, A, ASE ASE

Connects general education goals with discipline-specific courses

Provides invaluable ability to evaluate affective growth in students Provides good indirect data data can be compared longitudinally C, A, ASE can be used to determine outcomes over a long period of time

Evans & Sistek-Chandler, 2012, Assessment Summit, National University, La Jolla, CA

Model Microbiology SLOs from Janet Fulks Assessing Student Learning in Higher Education Draft Microbiology Student Learning Outcomes Domain Knowledge / Cognitive Cell Theory Specific Outcomes Following Completion of the Microbiology Course students will be able to: Use examples of infections, treatment, and epidemiologic control to compare and contrast the characteristics of prions, viruses, bacteria, protozoans, and multicellular Explain the dynamics of commensal and pathological relationships that occur between microbes and humans. Evaluate methods of microbial control and apply the proper methods necessary when given adescribe sample metabolic Briefly scenario. pathways found in microorganisms and their implicationsbasic bacterial genetic Summarize for food production and principles and analyze implications for mutation, genetic recombination, the role of the Articulate and diagram and bacterial immune system in maintaining homeostasis, challenging infections,Completion of the Microbiology Following and fighting cancer. Course students will be able to: Apply the scientific method by stating a question; researching the topic; determining appropriate tests; performing tests; collecting, analyzing, and presenting data; and finally Correctly perform microbiologic lab skills and display a habit of good lab practices which extends to relevant situations in the Following Completion of the Microbiology Course, students will be able to: Retrieve, evaluate, and use microbiologic information regarding contemporary issues in the world and relevant to their everyday lives. Summative Assessment Method Final exam essay question Take home case study question for Multiple final exam choice questions on final Diagram labeled on final exam Table completion on final exam Flow chart created by student on the final exam Two 50 point labs One team & one individual Senior Picnic Components of lab assignments above are used to assess skills Take home essay question on final exam and live patient interview

Microbial Interactions Microbial Control Microbial Metabolism Microbial Genetics Immune Response Skills/ Psychomotor Scientific Method Application Lab Safety Skills Attitudes and behavior/ Affective of Appraisal microbiolog ic information

Evans & Sistek-Chandler, 2012, Assessment Summit, National University, La Jolla, CA

Cognitive Domain
Learning Outcomes Related to Knowledge
Knowledge Comprehension Remembering Understanding Student remembers or recognizes information or specifics as communicat ed with little personal assimilation. Cite Labe l List Enumera te Identify Imitate Match Name Quote Recall Reproduc e State Write Student grasps the meaning behind the information and interprets, translates, or comprehends the information. Application Applying Student uses information to relate and apply it to a new situation with minimal instructor input. Apply Chart Compute Demonstra te Determine Dramatize Establish Make Manipulate Prepare Project Solve Use Analysis Analyzing Student discriminate s, organizes, and scrutinizes assumptions in an attempt to identify evidence for a conclusion. Analyze Compare Contrast Correlate Diagram Dissect Differenti ate Distinguis h Infer Investigat e Limit Outline Separate Synthesis Evaluating Student judges or evaluates information based upon standards and criteria, values and opinions. Access Apprais e Conclu de Critique Decide Defend Diagno se Evaluat e Hypoth esize Judge Justify Rank Evaluation Creating Student creativel y applies knowledg e and analysis to integrate concepts or construct an Assemble Create Construct Design Develop Formulate Generate Initiate Invent

Convert Define Describe Discuss Estimate Explain Generaliz e Identify Illustrate Locate Paraphras e Restate Summariz e

Cognitive Domain
Basic Knowledge Sophisticated More

Evans & Sistek-Chandler, 2012, Assessment Summit, National University, La Jolla, CA

Higher Level Thinking

Evans & Sistek-Chandler, 2012, Assessment Summit, National University, La Jolla, CA

Affectiv e Domain Learnin g Outcomes Related to Attitudes

Receiving Students become aware of an attitude, behavior, or value. Accept Attend Describe Explain Locate Observe Realize Receive Recogniz e Responding Students exhibit a reaction or change as a result of exposure to an attitude, behavior, or value. Behave Comply Coopera te Discuss Examine Follow Model Present Respond Show Studies Valuing Students recognize value and display this through involveme nt or commitme Accept Adapt Balance Choose Differenti ate Defend Influence Prefer Recognize Seek Value Organizing Students determine a new value or behavior as important or a priority. Characterizing Students integrate consistent behavior as a naturalized value in spite of discomfort or cost. The value is recognized as a part Authentica te Characteri ze Defend Display Embody Habituate Internalize Produce Represent Validate Verify

Adapt Adjust Alter Chang e Customiz e Develop Improve Manipula te Modify Practice Revise

Behaviors, and Values Elementary Values and Behaviors Highly More

Evans & Sistek-Chandler, 2012, Assessment Summit, National University, La Jolla, CA

Inherited Value System Attitudes

Developed Well Thought-out Value System

Evans & Sistek-Chandler, 2012, Assessment Summit, National University, La Jolla, CA

Psychomotor Domain Learning Outcomes Related To Skills

Observe Model Recognize Standards Students recognize standards or criteria important to perform a skill or task correctly. Check Detect Discrimina te Differenti ate Distinguis h Notice Perceive Recognize Select Correct Apply Coach

Students Transla te sensory input into physica l tasks or Hear Identif y Observ e See Smell Taste Touch Watch *Usually no outcomes

Students are able to replicate a fundamen tal skill or task.

Attempt Copy Follow Imitate Mimic Model Reenact Repeat Reprodu ce Show Try

Students use standards to evaluate their own performanc es and make corrections . Adapt Adjust Alter Chang e Correc t Customiz e Develop Improve Manipula te Modify

Students apply this skill to real life situation s. Build Compos e Constru ct Create Design Originat e Produce

Students are able to instruct or train others to perform this skill in other situations. Demonstra te Exhibit Illustrate Instruct Teach Train

Basic Knowledge Skills Basic Skills Abilities

More Sophisticated Higher Level

Evans & Sistek-Chandler, 2012, Assessment Summit, National University, La Jolla, CA

Evans & Sistek-Chandler, 2012, Assessment Summit, National University, La Jolla, CA

Digital Blooms
Web 2.0 Technology Tools Web-based applications arranged by their Blooms level.

Updated version by: Penney, S. (n. d.) Blooms Taxonomy Pyramid, http://www.usi.edu/distance/bdt.htm

Evans & Sistek-Chandler, 2012, Assessment Summit, National University, La Jolla, CA

Choosing the Right Web 2.0 Assessment Tools

Note: Many of the tools have free access and fee-based access. Most have administrative or instructor features if they are designed for education.

Bloom's Level Knowledge, Comprehension, Application or Analysis/Synthesis/Evaluation

Web 2.0 Tool Type of Assessm ent or Activity Content Review Blooms Level Pros Cons

TEDTalk iTunes University Wikipedia

K and C

Quia.com Quizlet.com eCollege Testing Engine (or other LMS) Surveymonkey.c om Zoomerang.com Questionpro.com My-ecoach.com

Online Exam or Quiz

K, C if carefully constructed A, S, & E

Multimedia, audio-based appealing to visual and auditory senses. Reusable Auto graded Data Analysis of Item Type
Provides good indirect data data can be compared longitudinally Can be used to determine outcomes over a long period of time Easily graded with rubric Allows other students to see and learn what each student learned connects general education goals with Discipline-

May not be vetted or authenticated

Reduces assessment to multiple choice answers

Respondents may be influenced by factors other than those being considered Validity and reliability most be closely watched Difficult for ESL students Stressful for students takes course time Must fairly grade course content beyond delivery

Survey Exit Survey MidCourse Survey

ASE

Screencast.com Voki.com Voicethread.com Adobe Connect

Oral presentati on using a synchrono us, VoIP tool or the individual student records for later

Variable K, C, A, ASE

Evans & Sistek-Chandler, 2012, Assessment Summit, National University, La Jolla, CA

assessme nt.

specific courses

Bloom's Level Knowledge, Comprehension, Application or Analysis/Synthesis/Evaluation

Web 2.0 Tool Type of Assessm ent or Activity
Debate using breakout rooms for preparation

Blooms Level

Pros

Cons

Skype elluminate Adobe Connect

K&C If carefully constructed rubric accompanies the debate and discussion A, S, & E

Wiki or Google Docs Wikispaces

Team Project for collaborative work

Variable K, C, A, A, S &E

Easily graded by rubric Allows other students to see and learn what each student learned Connects general education goals with Disciplinespecific courses Connects general education goals with disciplinespecific courses Students can display skills. knowledge, and abilities in a way that is suited to them Provides invaluable ability to evaluate affective growth in students. Provides the students with a record of their work and growth Best evidence of growth and change over time Students can display skills.

May be difficult for EL students. Must fairly grade course content beyond delivery

Twitter, Facebook Edmodo Social Media sites

Knowledge sharing

Variable K, C, A, ASE

Blog EduBlog Blogger

Selfreflectio n or essay. Web-based Constructed Portfolio

Variable K, C, A, A, S & E

Googlesites.com Weebly.com Moodle.org WordPress.com

Variable

Must fairly grade individuals as well as team Grading is slightly more complicated student interaction may be a challenge Must have clearly defined criteria and evaluative measures "The look" can not over-ride the content Must use evidence to support conclusions, not just opinions or I statements. Private vs. public? Time consuming to grade Different content in portfolio makes evaluating difficult and may require training Bulky to manage depending on size

Evans & Sistek-Chandler, 2012, Assessment Summit, National University, La Jolla, CA

Webspiration.com https://bubbl.us/ Mindomo.com

Flowchart or Diagram

C, A, A, S, E

Knowledge, and abilities in a way that is suited to them promotes selfassessment Collaborative Easy to share

Needs to be online

Evans & Sistek-Chandler, 2012, Assessment Summit, National University, La Jolla, CA

References
Fulks, J. (2004). Assessing Student Learning in Community Colleges. http://online.bakersfieldcollege.edu/courseassessment/default.htm Kirk, K. (n.d.). SERC. Image retrieved from: http://serc.carleton.edu/NAGTWorkshops/affective/ Mueller, J. Authentic Assessment Toolbox http://jfmueller.faculty.noctrl.edu/toolbox/ Overbaugh, R. C., & Schultz, L. Blooms Taxonomy, Old Dominion University, www.odu.edu/educ/roverbau/Bloom/blooms_taxonomy.htm Penney, S. (n. d.) Blooms Taxonomy Pyramid, http://www.usi.edu/distance/bdt.htm Sistek-Chandler, C. & Evans, S. (2012). Scribd, Assessment Summit, Spring Symposium, National University, La Jolla, CA. Uploaded to: http://www.scribd.com/ Sistek-Chandler, C. & Evans, S. (2012). Slideshare, Assessment Summit, Spring Symposium, National University, La Jolla, CA. Thinkfinity, Blog Reply, (Sep 26, 2011) How do web 2.0 tools fit into Blooms? http://community.thinkfinity.org/thread/3591

Evans & Sistek-Chandler, 2012, Assessment Summit, National University, La Jolla, CA