We all know firefighters who’d be excellent officers and may have served as an acting officer, but despite studying or mentoring, they just can’t pass the written exam. Or maybe when they do pass it, they’re bested by the academic candidate who outscores him or her every time. Some—not always the best candidate—are just good at taking written tests.
Even in applying for a firefighter’s job, the academic student may already have an advantage. According to a February 2015 post, Nancy Chapman found that the Norwalk (Connecticut) Fire Department had unintentionally given college graduates an advantage in its hiring process. The job requirement called for a high-school diploma.
The Struggle
People have legitimate phobias and high levels of anxiety when taking tests. In a document by Mark and Karen Gilbert, they asked if during exams people:
- Go blank
- Become frustrated
- Find themselves thinking they can't do this or are stupid
- Feel like the room is closing in on them
- Feel their heart racing or find it difficult to breathe
- Suddenly know the answers after turning in the test
These are some of the candidates we’re losing. Some have medical problems, like dyslexia or ADHD, and though they’re entitled to accommodations, they may not want to admit to a disability or ask for help. Actress and model Charlotte McKinney said,
Whenever people talk about dyslexia, it's important to know that some of the smartest people in the world, major owners of companies, are dyslexic. We just see things differently, so that's an advantage. I just learn a different way; there's nothing bad about it.
Others simply struggle to pass written tests; many will avoid, if possible, pursuing certifications or an EMT because of the test. It’s important for test takers to remember they don’t have to answer every question correctly to pass an exam; most require a 70% passing grade.
The Iowa Fire Service Training Bureau offers retests, for any level, on its certification tests. In May 2015, the Massachusetts Training Council changed the Career Recruit Firefighter Training Program’s Grading System for Written Examinations to allow one reexamination, but testers who don’t successfully pass the second exam are dismissed from the program without prejudice.
Standardized Testing
Promotional exams meet the criteria of standardized testing. In How to Assess Authentic Learning, Kathleen (Kay) Burke maintains that traditionally “standardized” meant that the test is standard or the same in three ways:
- Format/questions
- Instructions
- Time allotment
“Format/questions” means the test questions are the same for all students, asked in the same format, usually multiple choice. But in Tips for Teaching Adult Students, Brooks Doherty found that while younger test takers are accustomed to being compared to their peers, adults challenge themselves.
Written about students but applicable to anyone, The Dangerous Consequences of High-Stakes Standardized Testing from The National Center for Fair and Open Testing finds:
- High-stakes tests are unfair to many students.
- Some students simply don’t test well. Many students are affected by test anxiety or don’t show their learning well on a standardized test, resulting in inaccurately lower scores.
Assessment Equity
So, you ask, if they can’t pass the written promotional exam, could they document an incident scene or write a detailed report? The answer is yes. They’re good at their jobs, just not at tests.
Leon Nevfakh, in a Boston Globe article, in discussing the implications of the Harvard cheating scandal wrote
Being successful in today’s world, as we all now recognize, requires more than an ability to think quickly and recall facts on command. And our education system has, however fitfully, moved to address those values. The problem is that our tests still lag behind.
That could describe today’s fire service, but in promotions, there does need to be some sort of validity. How valid can test results be in those if participants simply don’t test well? If, for whatever reason, a standardized test may result in inaccurately lower scores?
Not all people can be tested fairly under the same conditions. Are there strategies to make testing better accommodate the vocational candidate without punishing the academic candidate?
Alternative Assessment Processes
Those departments that don’t have regularly scheduled promotional exams should reevaluate their testing processes. For someone who has testing phobias, nothing is worse than a last-minute exam for a sudden officer opening. Some departments give less than 30-days’ notice; with little lead time, firefighters who have difficulty in testing often simply won’t take the test.
Consider eliminating minimum written scores and have an overall test score. The academic candidate may score very high on the written exam while the vocational candidate may score low on the written exam but very high on the assessment center or practical portion. Requiring a minimum written exam score, usually 70%, to continue in the process is not fair to the vocational candidate.
Consider giving more credit for education other than college degrees. If a vocational candidate has received certificates (not necessarily certifications), perhaps it will help offset a low test score from someone who hasn’t pursued advanced education for whatever reason. Though the academic may have the greater formal education, in some instances, crediting certificates may give the vocational candidate a chance for a higher overall score.
The Illinois Fire Chiefs Association uses predominantly true/false tests, which may be easier than multiple choice, for the vocational candidate. Many departments schedule written exams with time limits; perhaps increasing the time or eliminating time limits altogether would benefit the vocational candidate.
In "Testing and Time Limits" (R&D Connections: November 2004), Brent Bridgeman, Amanda McBride and William Monaghan assert that limiting testing time too drastically can threaten a test’s validity or the ability of the test to accurately reflect what the test was designed to measure. This is particularly true if the test isn’t intended to measure how quickly the test taker can answer questions or if the testing time is so limited that a large number of examinees taking the test can’t complete it.
The passage of time is probably the most anxiety-provoking aspect of the testing situation for test-anxious examinees. As their symptoms make them unable to work, the approaching deadline increases their anxiety and so symptoms escalate with the passage of time. An untimed or extended exam may alleviate such symptoms by allowing the test-taker to relax about the time available.
Given the most common practices currently, vocational candidates must be prepared to study harder, learn relaxation techniques and take practice exams to help them through the process. There are a number of free practice test questions online and promotional exam books that can be purchased. Most importantly, they should ask someone who’s been through the process for help.
Alternate exams, elimination of time limits or the elimination of the written exam, so common in the fire service, becomes very controversial. Without changes in our current practices, we’re missing opportunities to promote some excellent firefighters. The worst part? We know who they are.