What
evaluations don’t tell you
We all like to read nice things about our teaching in student evaluations. But how much do these comments really tell us about the effectiveness of our classes?
Many articles on CPR skills training and retention have been published in the last 20 years. They all show similar results. Most people who had taken CPR classes could not perform adequate CPR when tested as soon as two weeks after the class. Researchers who observed the classes noted that many instructors did not follow the course plan, did not allow enough time for skills practice, and did not correct errors, but passed everyone who completed the class. Yet both students and instructors almost always gave the classes positive evaluations.
Most of us can recall similar experiences from emergency care courses we have taken, observed, or helped to teach. For example, I and several instructors whom I trained ran the final practical exam for a First Responder class of another instructor, years ago. This instructor had many years of teaching and EMS experience, and the students made a point of telling us what a wonderful teacher he was. Yet almost none of them could demonstrate even the most basic skills at the stations we monitored, without coaching.
Their problem was not lack of skills practice but the instructor's style of teaching. He was so eager to share all of his knowledge and experiences that he kept his students in a passive learning mode even during skills practice. As a result, his students never had a chance to go into problem-solving mode and practice the skills on their own. They remained dependent on his coaching and explanations, while being entertained by his stories.
Another example reveals a different problem in skills teaching technique. The National Ski Patrol has its own EMT level course, Outdoor Emergency Care, for training patrollers. It features many role-playing scenarios that help patrollers put their skills together and learn NSP protocols. The final practical exam is a series of scenarios graded by many instructors. In the exam for one class, the instructors who monitored the scenarios agreed with me that very few students were showing adequate skills without coaching. Yet they had done many practice scenarios during the class. But the instructor had not prepared them for the scenarios by giving clear explanations or demonstrations of the skills; and in evaluating the practice scenarios he had focused on the protocols while paying little attention to the skills. In this class too, the students seemed unaware of the deficiencies in their training until the final exam.
So how can we tell whether students have really learned the skills, much less feel confident that they will retain the skills after the class? The new AHA CPR course plans and videos are a response to this question, which was a main theme at the conferences to decide on the 2005 guidelines and standards. Most of us have now taken or taught these courses. What can we learn about skills teaching and retention from them?
Realistic demonstrations : Introduce each skill with a realistic, role-playing demonstration that models care giver behavior as well as the mechanics of the skill. Let another instructor act as narrator, explaining the technique and answering questions. If you are teaching alone, you can still perform both functions by telling the patient what you are doing, which models good patient care. If on the other hand you keep interrupting your demonstration by turning to the class and explaining what you are doing, students will never see a realistic demonstration, and they may imitate you by verbalizing the skills (saying what they would do) instead of practicing them realistically.
Whole-part-whole : After the realistic demonstration of a whole skill, the AHA CPR video shows each component while students practice that component (e.g. chest compressions) along with the video. Since everyone is performing the same movements together, instructors can easily spot and correct errors. We can use the same technique when teaching other complex skills. For example, after giving a realistic demonstration of backboarding, demonstrate each component in detail, and have students practice it (e.g. manually aligning the neck, or applying a C-collar) before they put all the steps together into a complete spinal management scenario.
Drill & repetition : Instructor led drill (which the video does for us in the new AHA CPR courses) is another teaching technique that can reinforce skill performance. It works best for skills that are standardized, so that everyone should be doing them the same way. The trick for talking students through a skill is to give clear, concise directions that tell students exactly what to do, and to keep the drill moving. The mistake many instructors make is to ask students questions (instead of telling them what to do) which elicits a vocal rather than a motor response; or to interrupt the drill with explanations. You can have another instructor modeling the skill as a living video while you give directions.
Contests : You can reinforce student skills, and make practice more fun, by staging contests with individuals or teams competing to complete a skill correctly in the shortest time; or to complete the most techniques in a given time. This teaches teamwork as well as reinforcing the skills with repetition.
Realistic testing : We need to see students perform skills realistically, without coaching. Performance should include patient care and patient communication, which means students demonstrating skills in a practical exam should be telling the patient what they are doing (not verbalizing to the examiner). Practical exams should be rehearsals for real emergencies, where students are not just going through the motions, but getting into the role so that they integrate and reinforce all their patient care skills. This how the new AHA CPR courses work.
We can apply the same principles to all emergency care skills teaching. By teaching complex skills systematically (whole-part-whole) as coaches and athletic trainers do, and training students to imitate and practice realistic patient care behavior as well as the mechanics of skills, we can increase the chances that our students will perform effective patient care in the real world.