DESIGNING LEARNING WITH THE END IN MIND

Alignment of Assessment
In the Quality Matters Rubric that we will be using to evaluate DEE’s online courses,
five of the eight sections reference alignment to learning objectives. The QM rubric asks
course evaluators to consider if the learning objectives are measurable; if the
assessments are designed to actually measure the stated objectives; and if the course
materials, learning activities, and supporting technologies are designed to promote the
achievement of the learning objectives.

During today’s workshop, we will be walking through each of these QM sections to

double check that what we are saying we want the students to learn is actually what we
have designed the course to teach them. Below is a simple schematic for thinking about
how learning objectives are the central starting place for effective learning design.

Assessment  Strategies  

Learning  Objectives  

Instructional  Materials   Learning  Activities   Tools  and  Media  

University of Idaho Distance & Extended Education (208) 885-4024, dee@uidaho.edu  

Three Simple Steps to Keep Focused on Core Concepts:
1. Identify results that you want for your learners. These results will probably include
"enduring understanding" (similar to concepts), knowing the vocabulary and syntax
of a discipline domain and being familiar with the "exemplars of a discipline." The
exemplars of a discipline might include the most famous representatives and case
studies, etc.

2. Determine the "acceptance evidence" by which the learners can demonstrate their
knowledge, understanding and integration of ideas.

3. Design the learning experiences to ensure learner accomplishment of these

understandings and the processes for demonstrating their learning.

McTighe, J. & Wiggins, G. The Understanding by Design Handbook.

Alexandria, VA: Association for Supervision and Curriculum Development, 1999.

Five Quality Matters Standards for Aligning Objectives:

The course learning objectives describe outcomes that
are measurable. This is one of those statements that gets often
Learning   gets misinterpreted because rigorous measurement is usually
Objectives   equated with traditional testing. However, authentic assessment
(outlined below) can be just as effective at measuring what your
students can actually do. “Do” is the key word to keep in mind
here. For an outcome to be measureable, it needs to be
something students can do to demonstrate the knowledge, skills, and habits of mind they
have developed in your course.

Outline of Authentic Assessment

Performance-based:
Challenges learners to show what they can do along with what they know.

Integrative:
Requires learners to pull together skills and knowledge from multiple areas.

Useful: Motivates learners to improve in the areas where they are weakest.

Relevant: References the learner’s situation and life experiences.

Participatory: Solicits the learner’s involvement in the evaluation process.

University of Idaho Distance & Extended Education (208) 885-4024, dee@uidaho.edu  

 The types of assessments selected measure the stated
learning objectives. Again, this is an easy one to miss if you
don’t start with the end in mind. In other words, what do you want
the students to do? What will you accept as evidence that they can do those things? Most
of us start by pulling together material that we think is critical to cover, then we come up
with some assignments that seem appropriate and, at the last minute, we develop a
grading scheme to put in the syllabus. Unfortunately this often results in not measuring
what we are really interested in but rather in measuring what we think we can measure.

One of the simplest ways to get around this measurement problem is to begin by
describing what a successful demonstration of learning would look like for your course.
When you have done this you have developed criteria that you can use for evaluating
student work, communicating your expectations to students, and even guiding what
kinds of test questions to create.* Although there is a lot of fuss around the science of
assessment, the most effective strategy is to use series of assessments throughout the
semester, preferably in various forms. This allows you to measure from multiple
perspectives and provides richer information about your students’ learning.
* See “Articulating your expectations to students” below.

The instructional materials contribute to the

achievement of the stated learning objectives.
Instructors usually do a thorough job with this aspect of course
design, maybe to a fault. Since instructors typically love their
subject, they tend to be concerned about coverage, so much so that
they often overdo it on course materials. Since coverage isn’t the
same as understanding the concepts it is important to step back and ask yourself, “Is this
really necessary?” It may be that you can—gasp—eliminate some material, which will
allow you to structure what remains in ways that will be “stickier” in terms of learning.

The learning activities promote the achievement of the

stated learning objectives. As in the Three Simple Steps for
Focusing on Core Concepts above, the learning activities become
the natural outgrowth of your description of “acceptable evidence.”
If you can articulate what a credible demonstration of student
learning would look like, then you can design an activity that lets
you see what students can do.

The tools and media support the learning objectives, and

are appropriately chosen to deliver the content of the
course. We have been exploring a number of tools during in these
course design workshops and it is clear that everyone in the group is
open to exploring the most effective options for delivering content
as well as creating appropriate learning activities. The selection of
possible learning tools has grown exponentially since the Quality
Matters Rubric was crafted in 2008. Given all the choices, the best
approach is the one the group is already using, that is to remain flexible and look for the
tools that best fit your course’s learning objectives.

University of Idaho Distance & Extended Education (208) 885-4024, dee@uidaho.edu  

Articulating your expectations to the students
A critical—but often overlooked—aspect of assessment is to involve students in the
process, first by clearly articulating your expectations and second by requiring them to
evaluate their own work along with the work of their peers. As noted in the Outline of
Authentic Assessment above, assessment should be useful. Students need a clear map of
disciplinary expectations along with a description of what mastery of your course
material looks like. Students learn by systematically comparing your expectations to
their own work and the work of their peers.

One of the best ways to do this is to create a rubric that clearly spells out what you are
hoping to see in their work, what would suffice and what wouldn’t. Rubrics have gotten
some bad press but that is because there are too many awful rubrics being poorly
applied. Even though rubrics take some work to produce, they
make your job easier because students can self and peer-assess
coursework in the draft stages.

We will take a look at some good and bad examples of rubrics

during the workshop but remember that you don’t have to
develop a rubric entirely on your own. We will be happy to work
with you to create a rubric that aligns with your course objectives.
Also keep in mind that you don’t have to start with a blank page.
AAC&U’s VALUE rubrics provide a great starting point from
which to tailor something to your needs.

Aligning your course goals to program goals

This is something that we have not spent much time on during these workshops.
It is also not covered in the Quality Matters Rubric. However, the alignment of
institutional goals is currently getting attention at the national level with discussions
around accreditation. As online courses at the University of Idaho develop into full
online programs it will be important to describe how course-level objectives support
program goals and to communicate a coherent set of program-level learning
expectations to students.

Avoiding the expert blind spot

One of the reasons that it is difficult to articulate expectations and develop assessment
criteria is because it is hard to remember what being a novice in your discipline was like.
To avoid this expert blind spot we are going to pair you up so that you can try out your
assessment language on each other and see if it is clear.

University of Idaho Distance & Extended Education (208) 885-4024, dee@uidaho.edu