Portfolio-Remix Performance Intervention

Online Publishing with Movable Type 5.10

Jeanette Novakovich Professional Writing Minor Program English Department Concordia University Montreal, Quebec

PERFORMANCE INTERVENTION

2012 Novakovich

PERFORMANCE INTERVENTION

TABLE OF CONTENTS
EDITION NOTICE INTERVENTION #1: TWITTER MICROBLOGGING SESSION TWITTER MICROBLOGGING SESSION INSTRUCTORS GUIDE SETUP INSTRUCTIONS MICROBLOGGING INSTRUCTIONS MANAGING THE POST-ACTIVITY DEBRIEFING MICROBLOGGING INSTRUCTIONS (STUDENT HANDOUT) TWITTER MICROBLOGGING FEED INTERVENTION #2: STEP-BY-STEP QUICK GUIDE FOR PUBLISHING AN ENTRY THE ENTRY: STEP-BY-STEP QUICK GUIDE FOR PUBLISHING AN ENTRY INTERVENTION #3 FINDING CREATIVE COMMONS LICENSED IMAGES THE ENTRY: FINDING CREATIVE COMMONS LICENSED IMAGES INTERVENTION #4: GUIDELINES FOR PUBLISHING AN ENTRY THE ENTRY: GUIDELINES FOR PUBLISHING AN ENTRY INTERVENTION #5: ONLINE SURVEY CHECKLIST FOR PUBLISHING AN ENTRY THE ENTRY: ONLINE SURVEY CHECKLIST FOR PUBLISHING AN ENTRY APPENDIX A: EVALUATIONS SAMPLE SATISFACTION SURVEY (KIRKPATRICK LEVEL 1) SAMPLE CRITERION-REFERENCED ASSESSMENT (KIRKPATRICK LEVEL 2) SAMPLE OF A LEVEL THREE EVALUATION (KIRKPATRICK LEVEL 3) SAMPLE OF A LEVEL THREE EVALUATION (KIRKPATRICK LEVEL 3) APPENDIX B: FORMATIVE EVALUATIONS TECHNICAL REVIEW DRAFT EMAIL #1 DRAFT EMAIL #2 DRAFT EMAIL #3 DRAFT #4 BLANK FEEDBACK FORM HYPOTHETICAL PILOT TEST REVISION OF INTERVENTIONS BASED ON PILOT TEST EDITORIAL REVIEW 2012 Novakovich 5 6 7 10 11 14 15 16 18 19 23 24 26 28 30 31 32 33 34 35 37 40 41 42 43 45 46 47 48 49 50

PERFORMANCE INTERVENTION

DRAFT #1 DRAFT #2 DRAFT #3 PERFORMANCE INTERVENTION STYLESHEET REFERENCES 50 51 52 53 54

2012 Novakovich

PERFORMANCE INTERVENTION

EDITION NOTICE

Portfolio-Remix Performance Interventions: Online Publishing with Movable Type 5.10 Professional Writing Minor Program English Department Concordia University Montreal, QC

First Version Copyright 2012 Jeanette Novakovich. All rights reserved.

2012 Novakovich

PERFORMANCE INTERVENTION

INTERVENTION #1: TWITTER MICROBLOGGING SESSION

Description The objective of this intervention, a weekly microblogging session, is to provide learners with a support and recognition feedback system. It addresses the needs of the unit and individual performer primarily at the disinterested level of performance by providing a community of support and feedback. Learners will read published entries on the blog feed of Portfolio Remix and provide feedback in the form of Twitter tweets that include the following hashtag, #portfolio-remix, a keyword that will allow Twitter search engines to create a customized feed. Learners can then respond to the tweets as they are fed into a Twitter search feed on the Portfolio-Remix online help support website. This will allow students to reply and retweet memorable tweets. Tweets will consist of constructive advice, alerts for publishing mishaps and congratulatory notes. This weekly microblogging session provides learners with a feedback support and recognition system. Rationale The rationale for including this intervention is to provide a feedback support system. Performance needs addressed: Select [Convert Line Breaks] format 100% of the time Place images appropriately on the page, centered, at the top of entry in [Body] field set at 500 pixel width in order to fit the container space or size of entry field 100% of the time Attribute all outside sources 100% of the time Publish entry summaries or introductions in the [Body] field 100% of the time Publish the bulk of entry in the [Extended] field 100% of the time Materials provided: Instructors guide Student handout Entry: Twitter feed for microblogging session

2012 Novakovich

PERFORMANCE INTERVENTION

TWITTER MICROBLOGGING SESSION INSTRUCTORS GUIDE

Overview of the activity The purpose of this intervention, a weekly microblogging session, is to provide learners with a support and recognition feedback system.

Background of the activity More and more employers are asking applicants if they have a website or other web presence that they can access easily. For employers who want to see beyond the traditional one-page rsum, students can develop a rich alternative in the form of an eportfolio, which is an online digital form of the traditional paper-based portfolio. For this reason, an e-portfolio hub, which is a website that aggregates or pools student portfolios together in one common location, was established for students enrolled in the program. Essential to the success of the program is ensuring that students maintain high online publishing standards as defined by the Professional Writing Minor Program when using Movable Type 5.10 software, an online open source publishing Content Management System, to design and publish professional e-portfolios. The primary goal is to ensure that students comply with online publishing standards set by the program by providing a support and recognition system in the form of a community microblogging session.

Summary of the activity Learners will read published entries on the blog feed of Portfolio Remix and provide feedback and support in the form of Twitter tweets that include the following hashtag, #portfolio-remix, a keyword that allows Twitter search engines to create a customized search feed. Tweets will consist of constructive advice, alerts for publishing mishaps, and congratulatory notes. Learners can then respond to the tweets as they are fed into a Twitter search feed on the Portfolio-Remix online support website.

Main objectives Select [Convert Line Breaks] format 100% of the time 2012 Novakovich

PERFORMANCE INTERVENTION

Place images appropriately on the page, centered, at the top of entry in [Body] field set at 500 pixel width in order to fit the container space or size of entry field 100% of the time Attribute all outside sources 100% of the time Publish entry summaries or introductions in the [Body] field 100% of the time Publish the bulk of entry in the [Extended] field 100% of the time

Format of the activity During a thirty-minute microblogging group session, learners will tweet microblogs or messages that consist of 140 characters or less that include the #portfolio-remix hashtag.

Expectations that learners have Expect tweeters to be informal and polite Expect nonjudgmental alerts Expect a measure of recognition Expect a measure of fun Note: The instructor of the course will be both a Twitter expert and Movable Type expert. The goal is not to teach students how to tweet or how to publish online, but to lead them through a peer-based feedback and support activity. Prerequisites of the activity: When the activity takes place students should have advanced working knowledge of online publishing and social media, particularly, microblogging. As experienced micro bloggers, learners will know how to write an effective tweet, create a hashtag, and link tweets to blog entries. Time: The 30-minute formative evaluation activity is divided into two parts, including a brief PowerPoint presentation and a hands-on learning activity, followed up by a debriefing. Equipment and Services: PC lab Projector Access to Internet Material and Props: Power Point embedded on course website 2012 Novakovich

PERFORMANCE INTERVENTION

Printed instructors guide Printed student guide Online activity Online assessment Schedule: 5:45 Classroom setup 6:00 Introduction and explanation of activity 6:10 Formative evaluation activity 6:30 Bio break and snacks (informal Q&A) 6:45 Online satisfaction survey

2012 Novakovich

PERFORMANCE INTERVENTION

10

SETUP INSTRUCTIONS
Classroom: Training for the microblogging session will take place in Hall 923 during regularly scheduled class time. Materials: Activity guides for the students and instructor will be printed prior to the training session. Printed Instructors Guide Printed Student Handout Before the students arrive: Turn on the projector by pushing the red button under the projector. Login to the main computer located at the front of class: o Username: jnovako o Password: ******** Type Portfolio-Remix.com into the address bar. Open printed instructors guide. After the students arrive at 6:00 p.m.: Have students complete the following tasks: o Login to computer terminals. o Type Portfolio-Remix in the address bar. Pass out student handout for microblogging session. You are now ready to begin the training program. Administrivia: Your focus will be on explaining the purpose of the activity and the learning objectives while encouraging students to maintain a friendly and supportive tone. Read through the handout before the students begin the activity.

2012 Novakovich

PERFORMANCE INTERVENTION

11

MICROBLOGGING INSTRUCTIONS
1. Login to computer and go to Portfolio-Remix main website: http://www.portfolio-remix.com Administrivia: Common classroom activity. Problems and questions are not anticipated. 2. Select the [Blogs] tab on the navigation bar. Administrivia: Common classroom activity. Problems and questions are not anticipated. 3. Read through the entries presented in the blog feed. Administrivia: Common classroom activity. Problems and questions are not anticipated. 4. Respond to at least ten entries by tweeting information that evaluates the entry in terms of excellence and its adherence to publishing standards set by the program. See Tweet guidelines below. Administrivia: Common activity. Problems are not anticipated. Every entry page will have Share Us buttons, which include a Twitter icon. When students select the Twitter icon, a popup box will open with a link to the entry and its title. Students can then add a brief personal message. Remind them to include the #portfolioremix hashtag. This way the Tweet will enter the Twitter Feed embedded on the Portfolio-Remix online support help website.

Responding to a professional entry If an entry is excellent, encourage your Twitter followers to read the entry: Tweet#portfolio-remix BRAVO! + add positive word or two (Twitter will add a link to the entry and title to all tweets made from the entry page.)

2012 Novakovich

PERFORMANCE INTERVENTION

12

Administrivia: Common activity. Problems are not anticipated. If a student enjoys reading an entry, Tweet it out positively. The student can then Tweet out any format issues in subsequent Tweets. Students are not limited to one Tweet per entry.

Responding to text formatting If the author forgot to use [Convert Line Breaks] ): Tweet#portfolio-remix CLB!!! + add positive word or two Administrivia: Help students identify entries that have a Word document pasted into the [Rich Text Editor] format. This can be achieved by observing the blog feed on the home page. Entries with [Rich Text] will show script or coding, irregular font styles. If the author has a weak introduction: Tweet--#portfolio-remix INTRO? + add positive word or two Administrivia: Challenge students to be critical and supportive. Advise them in the introductory paragraph does not compel them to read further, click the Continue reading link, to help the author by letting them know in a friendly and supportive manner. If the entry does not have a Continue reading link: Tweet--#portfolio-remix Extended? + add positive word or two Administrivia: Show students that they can identify that an [Extended] entry field is empty if the blog feed is long and if the following link is not present: Continue reading Responding to images If the author forgot to include an image: Tweet#portfolio-remix Image? + add positive word or two Administrivia: No problems or questions anticipated. If the author failed to provide attribution for any images: o Tweet#portfolio-remix Attribute? + add positive word or two 2012 Novakovich

PERFORMANCE INTERVENTION

13

o Administrivia: No problems or questions anticipated.

If the author has off-size or off-center image: o Tweet#portfolio-remix Image center? + add positive word or two Administrivia: No problems or questions anticipated. 5. After responding to ten entries, read through the Twitter feed on the Portfolio-Remix twitter-feed page and respond to the Twitter feedback that refers to specifically to the blog entries that you authored and published. http://www.portfolio-remix.com/twitter-feed.html. Administrivia: Explain to the students that they will now read through the Twitter Feed and respond to tweets referring to their work, following the guidelines below. Responding to tweets If a tweet is positive in tone: [Reply] and tweet a warm thank you. If a tweet is constructive: [Reply] and tweet a warm thank you. If a tweet is positive and funny: Retweet. If a tweet is negative in tone: Ignore.

Demonstrating the activity Demonstrate the activity by asking the students to identify an entry that they think is polished and professional. Open the entry by clicking on the link in the blog feed to the entrys page. Demonstrate the positive Bravo! tweet by clicking on the Twitter sharing button, typing: #portfolio-remix Bravo! Enticing story by a dedicated foodie. url, and submitting the tweet. Next, ask the students if they have any questions. And finally, ask the students to continue the activity on their own and offer to assist any students who are experiencing technical difficulties.

2012 Novakovich

PERFORMANCE INTERVENTION

14

MANAGING THE POST-ACTIVITY DEBRIEFING

Review the results with the learners: What were the results? What did you enjoy about the activity? What did you dislike about the activity? What was helpful?

2012 Novakovich

PERFORMANCE INTERVENTION

15

MICROBLOGGING INSTRUCTIONS (STUDENT HANDOUT)

1. Login to your computer and go to the Portfolio-Remix main website: http://www.portfolio-remix.com 2. Select [Blogs] tab on the navigation bar. 3. Read through the entries presented in the blog feed. 4. Respond to at least ten entries by tweeting according to the guidelines below. Responding to a professional entry If an entry is excellent, encourage your Twitter followers to read the entry: Tweet#portfolio-remix BRAVO! + add positive word or two (Twitter will add a link to the entry.) Responding to text formatting If the author forgot to use [Convert Line Breaks] ): Tweet#portfolio-remix CLB!!! + add positive word If the author has a weak introduction: Tweet--#portfolio-remix Intro? + add positive word If the entry does not include a Continue reading link: Tweet--#portfolio-remix Extended? + add positive word Responding to images If the author forgot to include an image: Tweet#portfolio-remix Image? + add positive word If the author failed to provide attribution for any images: Tweet#portfolio-remix Attribute? + add positive word If the author has off-size or off-center image: Tweet#portfolio-remix Image Center? + add positive word 5. After responding to ten entries, read through the Twitter feed on the Portfolio-Remix home page and respond to the Twitter feedback that refers directly to the blog entries that you authored and published. Responding to tweets If a tweet is positive: [Reply] and tweet a warm thank you. If a tweet is constructive: [Reply] and tweet a warm thank you. If a tweet is positive and funny: Retweet. If a tweet is negative: Ignore. 2012 Novakovich

PERFORMANCE INTERVENTION

16

TWITTER MICROBLOGGING FEED

Goal: The following twitter feed will support the Microblogging Twitter Feedback Session. Performance objectives that will be addressed: Select [Convert Line Breaks] format 100% of the time Place images appropriately on the page, centered, at the top of entry in [Body] field set at 500 pixel width in order to fit the container space or size of entry field 100% of the time and provide consistency of expression Attribute all outside sources 100% of the time Publish entry summaries or introductions in the [Body] field 100% of the time Publish the bulk of entry in the [Extended] field 100% of the time Production Instructions: Choose the <A> button on the toolbar and paste the following HTML code onto a [New] page of the online help support website, tag @topnav, and publish the page. <script charset="utf-8" src="http://widgets.twimg.com/j/2/widget.js"></script> <script> new TWTR.Widget({version: 2, type: 'search', search: '#portfolio-remix OR portfolioremix', interval: 30000, title: 'Portfolio-Remix', subject: ' ', width: 500, height: 800, theme: {shell: { background: '#f9fae8', color: '#a1092f'},tweets: { background: '#ffffff', color: '#444444', links: '#7d0628' } }, features: { scrollbar: true, loop: true, live: true, behavior: 'default'} }).render().start(); </script>

2012 Novakovich

PERFORMANCE INTERVENTION

17

The published page will have the following appearance:

2012 Novakovich

PERFORMANCE INTERVENTION

18

INTERVENTION #2: STEP-BY-STEP QUICK GUIDE FOR PUBLISHING AN ENTRY

Description This intervention, a step-by-step procedure, provides learners with the skills and knowledge necessary to publish an entry according to the guidelines for publishing a portfolio entry provided by the Professional Writing Minor Program. Rationale It addresses the needs of the individual performer primarily at the newbie level and provides a refresher for more advanced learners. The rationale for including this intervention is to set up a newbie to perform successfully by providing fundamental directions that reflect the material covered during the experiential learning session. Performance needs addressed: Select [Convert Line Breaks] format 100% of the time Place images appropriately on the page, centered, at the top of entry in [Body] field set at 500 pixel width in order to fit the container space or size of entry field 100% of the time Attribute all outside sources 100% of the time Publish entry summaries or introductions in the [Body] field 100% of the time Publish the bulk of entry in the [Extended] field 100% of the time Material provided consists of a web entry. Production Instructions: The objectives will be presented in the form of a quick procedural step-by-step guide to online publishing. Screenshots of steps will be provided.

2012 Novakovich

PERFORMANCE INTERVENTION

19

THE ENTRY: STEP-BY-STEP QUICK GUIDE FOR PUBLISHING AN ENTRY

The following instruction set provides you with the steps you need to in order to publish an entry according to the guidelines established by the Professional Writing Minor Program at Concordia University. Before beginning this procedure, you should have an entry composed in a Word document, and you should have an image downloaded into your personal computer for uploading a Creative Commons licensed image with a pixel resolution of 500 pixel width or greater. The overall procedure should take no more than 10 to 15 minutes. 1. Creating a new entry 1.1. Log into dashboard. 1.2. In the sidebar, select [Entries] > [New] 1.3. In the top entry field, type your title. 1.4. Select a Format: [Convert Line Breaks].

2. Uploading an image 2.1. Select Image Icon from the toolbar. 2.2. On the popup screen, select [Upload a new image from computer]. 2.3. Select the image you prepared for your entry. 2.4. On the next popup screen, set pixel width to 500 pixels. 2012 Novakovich

PERFORMANCE INTERVENTION

20

2.5. Select [Center]. 2.6. Select [Finish]. 2.7. Provide attribution for your entry by linking caption to source. 3. Adding the text 3.1. Copy and paste the first paragraph of your entry. 3.2. Select [Extended] entry field. 3.3. Copy and paste the remainder of the entry. 3.4. Switch format back to [Rich Text Editor]. 4. Optimizing SEO 4.1. Select from the available links offered by Zemanta. 4.2. Select from the available tags offered by Zemanta.

5. Publishing an entry 5.1. Select [Preview]. 5.2. Select [Publish entry].

2012 Novakovich

PERFORMANCE INTERVENTION

21

6. Socially bookmarking your entry 6.1. Select [View Entry]. 6.2. Select a [Twitter icon] or [Facebook icon].

2012 Novakovich

PERFORMANCE INTERVENTION

22

6.3. Tweet (add #portfolio-remix and/or post to Facebook.

2012 Novakovich

PERFORMANCE INTERVENTION

23

INTERVENTION #3 FINDING CREATIVE COMMONS LICENSED IMAGES

Description This online resource will provide resources for locating Creative Commons licensed images. Rationale This intervention was chosen to provide learners with convenient access to information. Performance needs addressed: Include an appealing Creative Commons licensed image 100% of the time

Material will consist of a web entry.

Production Instructions The objective will be presented in a descriptive bullet list of resources. As well, the Creative Commons license will be defined and illustrated. Screenshots of the Creative Commons license will be provided and a Creative Commons licensed image will provide a visual argument.

2012 Novakovich

PERFORMANCE INTERVENTION

24

THE ENTRY: FINDING CREATIVE COMMONS LICENSED IMAGES

Creative Commons image by A. Diez Herrero A Creative Commons license allows the maker of an image to retain the copyright of their work and, at the same time, gives permission for others to reuse the work. The license provides the original artist a number of conditions to select from that the user must abide: "You let others copy, distribute, display, and perform your copyrighted work and derivative works based upon it but only if they give credit the way you request (CC Org). "You allow others to distribute derivative works only under a license identical to the license that governs your work (CC Org). 2012 Novakovich

PERFORMANCE INTERVENTION

25

"You let others copy, distribute, display, and perform your work and derivative works based upon it but for noncommercial purposes only (CC Org). "You let others copy, distribute, display, and perform only verbatim copies of your work, not derivative works based upon it (CC Org). The following websites provide a number of Creative Commons licensed images or search engines to help you locate images that are available for reuse: Google Image advanced search- This search engine allows you to search for images that are licensed for reuse. Image Links from Creative Commons This website provides numerous links to the Creative Commons licensed databases of images. Flickr Advanced Search This photo sharing website allows you to select a search for Creative Commons licensed images and videos. Wikimedia Commons - This website provides a number of public domain images. NASA Image Archive - NASA has a number of non-copyrighted government images related to space flight, astronomy, and aeronautics. US Government This link will lead you to public domain photos and images created by the United States government. Open Clip Art- This website provides an archive of free-to-use clipart.

When you locate an image that you would like to use on your portfolio, check the license to find out how to attribute the image and how you are allowed to use the image, whether it can be a derivative work or a verbatim copy of the work.

2012 Novakovich

PERFORMANCE INTERVENTION

26

INTERVENTION #4: GUIDELINES FOR PUBLISHING AN ENTRY

Description This online resource will provide an online resource in the form of a guideline for publishing. Rationale The rationale for including this intervention was to make the guidelines available on demand and easily accessible to all learners. This will allow learners to maintain knowledge and skills learned during the training by providing them with an online support system. Performance needs addressed: Select [Convert Line Breaks] format 100% of the time Include an appealing Creative Commons licensed image 100% of the time Place images appropriately on the page, centered, at the top of entry in [Body] field set at 500 pixel width in order to fit the container space or size of entry field 100% of the time and provide consistency of expression Attribute all outside sources 100% of the time Publish entry summaries or introductions in the [Body] field 100% of the time Publish the bulk of entry in the [Extended] field 100% of the time Provide search engine optimization 100% of the time

Material provided consists of a web entry.

Production Instructions: The objectives will be presented in the form of a quick procedural step-by-step guide to online publishing. Screenshots of steps will be provided.

How the objective(s) will be presented (either a sketch or a narrative) 2012 Novakovich

PERFORMANCE INTERVENTION

27

Production Instructions: Guidelines will be presented in a descriptive bullet list and will include a screenshot of the blog feed and words.

2012 Novakovich

PERFORMANCE INTERVENTION

28

THE ENTRY: GUIDELINES FOR PUBLISHING AN ENTRY

Maintaining a high level of professionalism is key to the success of the Professional Writing Minor Programs implementation of our portfolio hub. To ensure that you are maintaining consistent and professional publishing standards, the following guidelines are provided: Use [Convert Line Breaks] when pasting any text from a Word document. Include an image at the head of every entry to create an attractive entry feed on the main portfolio website. The image should have high pixel resolution of at least 500 pixel width; when you upload the image, it should be centered on the page and the width should be set to 500-pixels.

2012 Novakovich

PERFORMANCE INTERVENTION

29

Only one paragraph of text should be pasted into the [Body] of the entry. This introductory paragraph should be polished and appealing. The rest of your entry should be pasted into the [Extended] entry field where you can also add an assortment of images and videos. Avoid adding videos to the [Body] entry field. Videos will significantly slow down the blog feed. After entering the text, use the [Rich Text Editor] to activate the Zemanta plugin. This will give you an opportunity to create links to outside sources and add tags to your entry, thereby, establishing search engine optimization. Once you have published your entry, view it and socially bookmark the entry to increase web visibility and traffic. Thank you very much for following the guidelines!

2012 Novakovich

PERFORMANCE INTERVENTION

30

INTERVENTION #5: ONLINE SURVEY CHECKLIST FOR PUBLISHING AN ENTRY

Description This online resource will provide feedback and support in the form of an online survey checklist for publishing. It will address the needs of the individual performer and assist learners at the feeling humble stage, learners who know that there is a lot to know and who will find the resource invaluable. It can also help learners at the newbie and feeling arrogant stage as well.

Rationale The rationale for including this intervention was to provide immediate support to students by providing them with an online support system when publishing.

Performance needs addressed: Select [Convert Line Breaks] format 100% of the time Include an appealing Creative Commons licensed image 100% of the time Place images appropriately on the page, centered, at the top of entry in [Body] field set at 500 pixel width in order to fit the container space or size of entry field 100% of the time and provide consistency of expression Attribute all outside sources 100% of the time Publish entry summaries or introductions in the [Body] field 100% of the time Publish the bulk of entry in the [Extended] field 100% of the time Provide search engine optimization 100% of the time

Material provided consists of a web entry.

2012 Novakovich

PERFORMANCE INTERVENTION

31

Production Instructions: This intervention will take the form of an embedded online survey using Poll Daddy survey maker.

THE ENTRY: ONLINE SURVEY CHECKLIST FOR PUBLISHING AN ENTRY

Complete the following survey every time you publish a new blog entry on your portfolio.

Title of your entry __________________________________ Author_______________________________________________

Did you select [Convert Line Breaks] Yes or No Did you add an image? Was your image centered and set to 500-pixel width? Did you provide attribution? Did you paste the first paragraph of the entry into the [Body] field? Did you paste the remainder of the entry into the [Extended] entry field? Did you switch back to [Rich Text Editor]? Did you select [Links] and [Tags] to optimize SEO? Did you socially bookmark your published entry?

Submit

2012 Novakovich

PERFORMANCE INTERVENTION

32

APPENDIX A: EVALUATIONS

2012 Novakovich

PERFORMANCE INTERVENTION

33

SAMPLE SATISFACTION SURVEY (KIRKPATRICK LEVEL 1)

(Adapted from Carliner, 2003) Please provide feedback to the following questions: 1. How would you rate this activity on a scale of 1 - 5 with 5 being the highest rating? 1 2 3 4 5

2. How much did you know about online publishing software before doing this training course? Nothing Very little Little Some Somewhat

3. How much do you know after completing the program? Nothing Very little Little Some Somewhat

4. How likely are you to use the skills taught in this program on your own outside of class without supervision? Nothing Very little Little Some Somewhat

5. What was the most valuable part of this program in terms of skill sets taught? _____________________________________________________________________ 6. What was the most effective part of this program? _____________________________________________________________________ 7. What is one aspect of the training that could improve this program most? _____________________________________________________________________

2012 Novakovich

PERFORMANCE INTERVENTION

34

SAMPLE CRITERION-REFERENCED ASSESSMENT (KIRKPATRICK LEVEL 2)

Performers will be evaluated through observation. The course instructor of English 396 will ask a representative sample of students that include novice, average and expert performers to create a new entry. The instructor will record observations in a chart. Main Task: Within a span of 30 minutes, create a new entry for your portfolio. Observer will circle if the behavior is present. Composes on word (not necessary but common) Opens dashboard Creates new entry Composes a title for entry Selects [convert line breaks] format Finds a creative commons image Uploads image first, 500 px, centered [body] Provides attribution Locates 1st paragraph of text in [body] field Locates remainder of text in [extended] field Selects suggested links Selects suggested tags Saves entry Refreshes website Score /130 Present Present Present Present Present Present Present Present Present Present Present Present Present Present

2012 Novakovich

PERFORMANCE INTERVENTION

35

SAMPLE OF A LEVEL THREE EVALUATION (KIRKPATRICK LEVEL 3)

Twelve weeks after training, in order to evaluate whether performers are publishing entries according to the programs standards, 10 direct observations of the most recent web entries will be evaluated. Google analytics will also be used to gather web metric data on published entries. Entry #1 Selected [convert line breaks] format Used a creative commons image Image 500 px, centered, [body] field Provided attribution Included no more than 1 paragraph in [body] field Located bulk of entry in [extended] field Provided links Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Entry #2 Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Entry #3 Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Entry #4 Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Entry #5 Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Entry #6 Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Entry #7 Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Entr y #8 Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Entr y #9 Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Entr y #10 Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No

Provided tagssearch terms (found in tag cloud)

2012 Novakovich

PERFORMANCE INTERVENTION

36

Score Yes = 10 pts No = 0 pts /90 Google analytics will also be used to collect web metrics, which measure the success of individual web entries, in terms of readership, to identify entries that are tagged and linked effectively. Top ten entries over the last month Title of entry #1 #2 #3 #4 #5 #6 #7 #8 #9 #10 Number of visitors Engagement (average # of seconds spent on page or entry)

2012 Novakovich

PERFORMANCE INTERVENTION

37

SAMPLE OF A LEVEL THREE EVALUATION (KIRKPATRICK LEVEL 3)

After a twenty-four week period, a brief survey will be given to students to check for their understanding of the publishing procedure six months done the line. After this, a repeat of observations can be taken of the most recent published entries. Finally, Google Analytics will be used to gather web-publishing metrics. Student survey of students understanding of the publishing procedure: Provide an effective title Select [convert line breaks] format Circle Yes or Circle Yes or Circle Yes or Circle Yes or Circle Yes or Circle Yes or Circle Yes or Circle Yes or Circle Yes or No No No No No No No No No

Use creative commons licensed image Size and center images at 500 px Provide attribution or cite your sources Locate 1st paragraph of text in [body] field Locate remainder of text in [extended] field Select suggested links Select suggested tags Score /90

2012 Novakovich

PERFORMANCE INTERVENTION

38

In order to evaluate whether performers are publishing entries according to the programs standards, ten direct observations of the most recent web entries will be evaluated. Entry #1 Selected [convert line breaks] format Used a creative commons image Image 500 px, centered, [body] field Provided attribution Included no more than 1 paragraph in [body] field Located bulk of entry in [extended] field Provided links Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Entry #2 Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Entry #3 Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Entry #4 Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Entry #5 Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Entry #6 Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Entry #7 Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Entr y #8 Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Entr y #9 Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Entr y #10 Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No Yes or No

Provided tagssearch terms (found in tag cloud) Score Yes = 10 pts No = 0 pts /90

2012 Novakovich

PERFORMANCE INTERVENTION

39

Google analytics will also be used to collect web metrics, which measure the success of individual web entries, in terms of readership, to identify entries that are tagged and linked effectively. Top ten entries over the last month Title of entry Number of visitors #1 #2 #3 #4 #5 #6 #7 #8 #9 #10 References Carliner S. (2003) Training Design Basics. Baltimore, MD: ASTD press. Gilbert Model (1996) adapted by Stolovitch, Harold D. & Keeps, Erica J. (2004) Training Aint Performance. Baltimore, Md: ASTD press.

Engagement (average # of seconds spent on page or entry)

2012 Novakovich

PERFORMANCE INTERVENTION

40

APPENDIX B: FORMATIVE EVALUATIONS

A formative evaluation plan for Performance Intervention: Online Publishing with Movable Type 5.10 will take place in a series of stages to determine the effectiveness of the course in terms of its being accurate, understandable, and functional as described by Carliner (2003): Technical review Pilot test Editorial review

2012 Novakovich

PERFORMANCE INTERVENTION

41

TECHNICAL REVIEW
Technical Review Participants: EL Instructional design expert with a specialization implementing Movable Type MK Movable Type expert SK Educational technology expert at the university level Procedure: Date September 17, 2012 September 24, 2012 November 17, 2012 November 28, 2012 Action Email Email Email Email with attachment Draft 1: Instructional materials, learners handout and storyboards for web entries Materials

First Contact Affirm participation Reminder Delivery

Return date

November 29, 2012

Email with attachment

2012 Novakovich

PERFORMANCE INTERVENTION

42

DRAFT EMAIL #1 To: EL, MK, or BK From: Jeanette Novakovich Date: 17 September 2012 Subject: Portfolio-Remix Performance Intervention Hello, [Congratulations on your recent nuptials. I hope that you had an enjoyable honeymoon. I would like to thank you again for the work that you managed to complete for me during your exciting summer.] (Vary with recipient) This winter, I am completing a performance intervention for students using Movable Type software. I am writing to request a review of the performance intervention materials for the program. Coming from one of the few Movable Type experts, your feedback will provide an invaluable insight into the accuracy of the project. The review will take two forms: An evaluation of the effectiveness and accuracy of the technical content A walk-through of the training material The review materials will be sent out electronically in eight weeks, during the first week of November 2012. The review shouldnt take more than a few hours to complete. The scheduled review will take place between the following dates: November 28- Reviewer receives review materials electronically November 29- Reviewer returns reviews electronically Thank you for your consideration. Kindest regards, Jeanette Novakovich Assistant Professor, English Department Concordia University, Montreal QC
2012 Novakovich

PERFORMANCE INTERVENTION

43

DRAFT EMAIL #2 To: EL, MK, BK From: Jeanette Novakovich Date: 24 September 2012 Subject: Performance Intervention for Online Publishing with Movable Type 5.10 Hello, Thank you for agreeing to review the performance intervention materials that I am developing for the Professional Writing Minor Program at Concordia University. As one of the few Movable Type experts, your willingness to provide detailed feedback is truly appreciated. The review will take the form of an evaluation of the effectiveness of the content in terms of its accuracy, understandability, and the functionality of the materials. A feedback form will be provided. The review materials will be sent out electronically in six weeks, during the last week of November 2012. The review shouldnt take more than a few hours to complete. Please review the schedule: November 28- Reviewer receives review materials electronically November 29- Reviewer returns feedback electronically Please find attached an I-Tunes gift certificate as a gesture of goodwill and appreciation for your anticipated timely feedback. Kind regards, Jeanette Novakovich Assistant Professor, English Department Concordia University, Montreal QC
2012 Novakovich

PERFORMANCE INTERVENTION

44

2012 Novakovich

PERFORMANCE INTERVENTION

45

DRAFT EMAIL #3 To: EL, MK, BK From: Jeanette Novakovich Date: 17 November 2012 Subject: Portfolio-Remix Performance Intervention Hello, Thank you for agreeing to review the performance intervention materials for our program here at Concordia. The review materials will be sent out electronically in five days, during the first week of November 2012: November 4- Reviewer receives review materials electronically November 6- Reviewer returns feedback electronically I look forward to receiving your feedback. Kindest regards, Jeanette Novakovich Assistant Professor, English Department Concordia University, Montreal QC

2012 Novakovich

PERFORMANCE INTERVENTION

46

DRAFT #4 To: EL, MK, BK From: Jeanette Novakovich Date: 28 November 2012 Subject: Portfolio-Remix Performance Intervention Hello, Thank you for agreeing to review the performance intervention materials for our online publishing program. Please find the following items attached for the technical review: Instructors Manual Learners Handout Storyboards for web entries The overall objective of this review is to note any inaccuracies of content in terms of correctness, currency and completeness. In addition, the material can be reviews for accessibility issues for learners. The learners can be assumed to have no more than basic training using Movable Type software. And finally, please note any aspects of the materials that are not currently functional. Please feel free to make comments and corrections on the electronic drafts and be sure to complete the feedback form. I look forward to receiving your feedback on Tuesday, November 29th. Kindest regards, Jeanette Novakovich Assistant Professor, English Department Concordia University, Montreal QC

2012 Novakovich

PERFORMANCE INTERVENTION

47

BLANK FEEDBACK FORM

In addition to making comments directly on course material, please complete the following feedback form. Storyboards for Instructors manual Learners Material web entries, note When providing When providing slide #. feedback, note feedback, note section and page section and page #. #. Information complete? If not, note missing information. Is the information Accurate? Note inaccuracies. Is information current? Note missing updates. Is Information Understandable to a lay audience? Note any issues or confusion. Is information functional? Note any places where the material did not link or operate according to basic web standards. Any additional comments.

2012 Novakovich

PERFORMANCE INTERVENTION

48

HYPOTHETICAL PILOT TEST To: English 395 Technical Writing Students From: Jeanette Novakovich Date: 1 December 2012 Subject: Pilot test for Portfolio-Remix Performance Intervention Hello, As mentioned in class earlier today, you are invited to pilot a test of PortfolioRemix Performance Intervention Tuesday December 6, 2012. The test will take place at the start of class in Hall Room 923. Students who participate are eligible to take part in an I-Tunes gift card drawing at the close of the pilot test. I look forward to receiving your invaluable feedback. See you in class next week! Kindest regards, Jeanette Novakovich Assistant Professor, English Department Concordia University, Montreal QC

2012 Novakovich

PERFORMANCE INTERVENTION

49

REVISION OF INTERVENTIONS BASED ON PILOT TEST Category A: Showstopper Category B: Must change Category C: Nice to change

2012 Novakovich

PERFORMANCE INTERVENTION

50

EDITORIAL REVIEW DRAFT #1 To: <editorial reviewer> From: Jeanette Novakovich Date: 1 October 2012 Subject: Editorial review for Portfolio-Remix Performance Intervention Hello, Congratulations on your recent and well-deserved promotion. As we discussed last week, I am writing to formally request an editorial review of the training materials for Portfolio-Remix Performance Intervention. I appreciate your having expressed your willingness to participate in the final review process. The review materials and style sheet will be sent out electronically in ten weeks, during the second week of December 2012. The review shouldnt take more than a few hours to complete and will take place between the following dates: December 12- Reviewer receives review materials electronically December 13- Reviewer returns reviews electronically I look forward to receiving your keen feedback. Please identify any errors in grammar, style, spelling and layout. As we agreed, you will be sending me the materials to review your latest in-the-works project. Thank you for your consideration. Kindest regards, Jeanette Novakovich Assistant Professor, English Department Concordia University, Montreal QC

2012 Novakovich

PERFORMANCE INTERVENTION

51

DRAFT #2 To: <editorial reviewer> From: Jeanette Novakovich Date: 7 December 2012 Subject: Portfolio-Remix Performance Intervention

Hello, Thank you for agreeing to review the training materials for Portfolio-Remix Performance Intervention. The review materials will be sent out electronically in five days, during the second week of December: December 12- Reviewer receives review materials electronically December 13- Reviewer returns feedback electronically I look forward to receiving your feedback. Kindest regards, Jeanette Novakovich Assistant Professor, English Department Concordia University, Montreal QC

2012 Novakovich

PERFORMANCE INTERVENTION

52

DRAFT #3 To: <editorial reviewer> From: Jeanette Novakovich Date: 12 December 2012 Subject: Portfolio-Remix Performance Intervention

Hello, Thank you for agreeing to review the training materials for Portfolio-Remix Performance Intervention. Please find the following items attached for the editorial review: Instructors manual Learners handout Storyboard for web entries Stylesheet The overall objective of this review is to note any grammar, style, spelling or layout errors. Please feel free to make comments and corrections on the electronic drafts. I look forward to receiving your feedback on Tuesday, December 13th. Kindest regards, Jeanette Novakovich Assistant Professor, English Department Concordia University, Montreal QC

2012 Novakovich

PERFORMANCE INTERVENTION

53

PERFORMANCE INTERVENTION STYLESHEET

Canadian English Style Guides:

The Canadian Style: A Guide to Writing and Editing (1997). Dundurn Press, Public Works, Government Services Canada. Canadian Oxford Dictionary. Online reference: http://www.oupcanada.com/reference_trade/canadian_oxford_dictionaries/dicti onary_online.html

Preferred Web Entry Layout:

Introductory and close-out slides standard PPT heading and bullet list summary. Content or unit slides: Assertion-evidence style that includes an assertion in the header followed by visual evidence. Furthermore, each unit includes an introduction to the troubleshooting area, provides a tip to solve or prevent the problem, and concludes by explaining the best practice. Clean crisp slide layout. Warm beige background, crisp modern black font, and white background for text. Delicious font for heading: size 28pt Delicious font for text: size 16.5pt Red spot color in visuals

Course Materials:
Euphemia UCAS font for headings: size 14 pt + Euphemia UCAS font for text: size 12 pt Maroon red spot color Preferred active voice Technical jargon defined in text boxes
2012 Novakovich

PERFORMANCE INTERVENTION

54

REFERENCES
The Canadian Style: A Guide to Writing and Editing (1997). Works, Government Services Canada. Canadian Oxford Dictionary. Online reference: http://www.oupcanada.com/reference_trade/canadian_oxford_dictionaries/dictionary_ online.html Carliner S. (2003). Training Design Basics, Baltimore, MD: ASTD Press. Dundurn Press, Public

Note: Some text adapted from Carliner, 2003.

2012 Novakovich