Design and Evaluation of Training Programs - Week 8 - Assessment in Teacher Training

Introduction: Hello and welcome to Week 7 of the course Design and Evaluation of Teacher Training Programs and Workshops for the Master's in English Teaching at ULACIT. Today we will do several activities to explore the topics of reading groups and conferences for professional development, assessment strategies for training course development, and the Kirkpatrick Model for training evaluation. 

Today's Goals:

• Explore practical strategies for creating reading discussion groups as a staff professional development initiative.
• Discuss potential strategies for assessing trainee learning and the effectiveness of your training course.

Guiding Questions:

• How can I create practical opportunities for my teaching staff to learn from the professional literature of our field?
• How can I know if my training has been successfull?

Warm Up: Origin Stories

Last week we discussed how fictional depictions of teachers and teaching can be analyzed as a professional development strategy in order to gain insights about our practice. This week we will do a variation on that theme by creating a work of fiction based on a teaching reality. Click your group link and complete the information required. 

• Group 1: CLICK HERE
• Group 2: CLICK HERE

Now share your origin stories. CLICK HERE to see a digital comic book that I created to share with my team of teachers when we did this activity last December. How else might might teachers create fictional content to explore deeper issues regarding the teaching-learning process?

Task 1: Sharing your PD Journal

Let's take a moment to share one of the entries you made in your PD Journal in Week 7. As you share your highlight, let's think about how this tip connects to how we as individual teachers can develop in our practice and how we as trainers can use these techniques to support teacher growth with teachers we work with.

• Expand your Knowledge of the Field
• Form a Reading Group
• Learn from an Expert
• Join a Language Teachers' Association
• Attend a Conference

Click to view full size image.

Task 2: Teachers' Associations and Conferences

In your reading this week from Richards (2017), the author suggests joining a teachers' association and attending a professional conference as tips for professional development.

• What teachers' associations are you familiar with?
• What do they do?
• What are some benefits of joining them?

Association Membership and SIGs

• IATEFL Membership Benefits and Costs - CLICK HERE
• IATEFL Special Interest Groups (SIGs) - CLICK HERE

• TESOL Membership Benefits and Costs - CLICK HERE
• TESOL Interest Sections (ISs) - CLICK HERE

• ACTFL Membership Benefits and Costs - CLICK HERE
• ACTFL Special Interest Groups (SIGs) - CLICK HERE

Conferences

• Have you ever attendend a professional conference?
• If so, what was your experience?
• Have you attended a virtual conference or PD event? 
• If so, what was that like? 
• How did it compare with a face to face event?

CLICK HERE to join the CCCN Community of Practice for access to the recordings of National Conference for Teachers of English Costa Rica (NCTE) and the PD Talks: Professional Development Sessions.

Task 3: Ideas for Reading Groups

Richards (2017) discusses the idea of creating a reading group as a professional development initiative. Discuss the questions below.

• In your opinion, how important is it for language teachers to read from professional literature?
• How often do you read articles or books about teaching and language learning outside the context of a university assignment?
• Do you think you and your teacher colleagues would benefit from regularly reading and processing a text from the professional literature of our field? Why or why not?
• What barriers can you identify to establishing a reading group in your context?

 Practical Ideas

Tip 1: Preparation is Key

Give teachers a task to do! Simply asking teachers to read and discuss is unlikely to lead to an effective exploration of the assigned text. Help teachers process their thoughts about the assigned reading before coming to the discussion session with a preparation task. Faciliate the discussion session with a scaffolded series of tasks or questions.

• Sample Reading Group Preparation Task: CLICK HERE
• Sample Reading Group Faciliatation Sequence: CLICK HERE

Tip 2: Allow for Choice

An alternative to choosing a single article or chapter for all teachers to read, you might provide several articles about a specific topic (teaching reading, classroom management, assessment strategies, etc.) and let teachers choose the one they want to read and process. In the discussion session you can have them explore similar themes that came up in the different articles as well as differing points of view and potentially contradictory ideas.

Tip 3: Make it Last

It takes time and effort to carry out a successful reading group activity and you want that investment to make a difference in the teachers' thinking and practice. For one of the tasks in the discussion session, have teachers collaborate to create a tangible product that serves as a record of their thinking and analysis that can be archived for future reference and/or shared with other faculty members. This could be a blog post, a personal journal, a poster to be put up in the teacher's room, a short write up to be shared as part of a teacher newsletter, or a short video or audio recording are just a couple of ideas that might serve this purpose. 

Tip 4: Mix it Up

We read about reading groups, but what about "Watching Groups"? You could also select a webinar recording that you would like your team of teachers to watch and process. As with the reading groups, it is important to give teachers a task in order to facilitate a structured exploration of the content. You could create your own specific reflection question based on the content of the chosen webinar or use a "generic" processing guide that will work for any topic.

Mark's Generic Processing Guide for Webinars

• What were at least three key ideas of this session? For each one summarize the idea and what it means to you.
• What is something that you could take from this session and apply in your teaching context with no modifications needed?
• What is something you could apply in your teaching context with some modifications? How would you modify it and why?
• Is there anything from the session that would not work in your context or that you would not like to do because it does not fit your beliefs or personality?
• What additional insights, ideas, questions, or points of curiosity were sparked in by this presentation?

 

Task 5: Reading Response

Last week you read Chapter 10 "Designing an Assessment Plan" in Graves (2000) Designing Language Courses. Click your group link below to discuss some of the important ideas with your partners and explore how they can be applied to the training course proposal you are working on.

• Group 1: CLICK HERE
• Group 2: CLICK HERE
• Group 3: CLICK HERE 

Practical Ideas

Strategy 1: Use Pre-Post Course Reflections

One simple but powerful learner-centered formative assessment strategy is to design a pre-course and post-course reflection task. In the pre-course reflection have trainees answer questions similar to the following.

• What do you hope to get out of this course?
• What do you already know about the following topics...?
• What specific questions do you have regarding...?

Of course, you will need to personalize the prompts to fit your training context. You may also want to include a short diagnostic quiz in this task. The purpose of the pre-course reflection is to have trainees articulate their goals for the course, specific questions and doubts regarding the content areas, and (in the case of a diagnostic) find how how much they already know.

At the end of the course, have trainees revisit their pre-course reflections and answer prompts similar to the following:

• Look at the goals you had at the beginning of this course. To what degree do you think you have achieved them? Provide specific reasons to support your opinions.
• How did your knowledge of ... change or deepen as a result of this training? Provide specific examples. 
• What were your questions and doubts at the beginning of the course about...? Are you able to answer any of them now? Do you have any new questions or doubts?

You can also have students take the same diagnostic quiz again and reflect on their responses compared to the first one they took.

Strategy 2: Create Digital Checklists for "Complies / Does Not Comply Types" Assessments

One practical way of assessing trainees is asking yourself the question, "Did they do it or not?" This is especially useful when training teachers on procedural tasks where by completing them, they show that they are capable of carrying out the action. These checklists can also be used to establish a global perspective regarding the level of trainee participation. 

• Sample Checklist: CLICK HERE

You might consider adding a third category to your checkboxes: Fully Complies, Partially Complies, Does not Comply. 

Strategy 3: Establish Standard Criteria for Participant Feedback Surveys

By establishing a core set of evaluation criteria for training events, you can measure trainee satisfaction over time and set Key Performance Indicators (KPIs). 

Task 6: The Kirkpatrick Model 

Donald Kirkpatrick developed this popular four level model for evaluating the effectiveness of training on job performance. It starts with trainees' impressions of the training and evidence of learning and moves on to discover if the training led to any measurable changes in the trainees' on the job behavior and finally, it measures the results gained by the training from an institutional perspective. 

Click to see full size image.

For your training course proposal assignment, you are only required to develop an assessment plan for levels 1 and 2 in the Kirkpatrick model. However, it is important to consider strategies to address levels 3 and 4 in order to determine the true effectiveness of your training efforts. 

• What might be some follow up and support strategies to see to what degree teacher's apply what they learned in the training in their classes? 
• Once you see that teachers are actually applying the training on the job, how do you know if their new behaviors are actually making an impact at an instutional level? 
• What indicators might help you answer that question and how could you gather data regarding those indicators?

References:

Graves, K. (2000). Designing Language Courses. National Geographic Learning.

Kirkpatrick, D. (2020). The Kirkpatrick Model. Kirkpatrick Partners. https://www.kirkpatrickpartners.com/Our-Philosophy/The-Kirkpatrick-Model

Richards, J. (2017). Jack C Richard's 50 Tips for Teacher Development. Cambridge University Press.