You might want to stand up to read this article. It's about wellness, fitness and exercise in the fire service, and it's hard to make any progress on those topics while driving a desk.

The Fire Chief report card on the ICHIEFS and IAFF Candidate Physical Ability Test addresses:

• Who's using it.
• How much it costs to implement and what it's worth.
• Sample pass/fail rates so far.
• How women fare on the test.
• How the ICHIEFS and IAFF are reshaping discussion about the controversial issue of incumbent testing.

By now, most chiefs know the roots of this unprecedented labor/management effort to address firefighter physical fitness. On Aug. 25, 1996, an “informational picket” of the Firefighter Combat Challenge by IAFF Local 42 and other Kansas City-area union members devolved into a brief but vivid brawl. At issue for the IAFF: Should local governments consider a modified version of the crowd-pleasing competition — regularly won by beings straight from Krypton — as a fitness test for current and prospective firefighters?

In the months that followed, the bones of the incident would be picked over by cops, newspaper and magazine editors, and mayors and city councils, as well as by fact-finding teams of chiefs and union members, would-be competitors and victims. The union bore the brunt of nearly everyone's displeasure for the fracas, which all but precluded any calm discourse about its concerns.

Initiating the fitness initiative

Still, the Phoenix rises from ashes, and the ICHIEFS and IAFF leadership seized the opportunity to turn a public-relations disaster into progress. In 1997, the Fire Service Joint Labor Management Wellness-Fitness Initiative brought together a task force of 10 fire chiefs and 10 union presidents from cities across the United States and Canada. In less than a year, the two organizations jointly released a wellness-fitness program described by ICHIEFS as “a model fitness program that has a holistic, positive, rehabilitative and educational focus.”

On Aug. 29, 1999, in Kansas City at Fire-Rescue International, the task force unveiled the program's second effort, the Candidate Physical Ability Test. Last year, incoming IAFF General President Harold Shaitberger, who had been instrumental in running the task force, mentioned CPAT's progress during his speech at Fire-Rescue International in Dallas.

“[The joint task force] worked closely with the Justice Department,” Shaitberger said, “to create a test that will help ensure new firefighter candidates are more physically capable of performing the challenging job of firefighting, while making it possible to improve the diversity of the fire service.”

Creating such a test would be a tall order, but the sustained cooperative efforts have surprised many fire service veterans, who are accustomed to change coming at a glacial pace. Now, inquiring minds want to know: How successful have the cooperative efforts been?

Before diving into the specifics of CPAT, it's instructive to remember the guiding principle behind the whole shooting match: CPAT's non-punitive nature. The wellness-fitness initiative came first. It doesn't make sense for a department to test its candidates or incumbents, the theory goes, before it has an infrastructure in place to prepare them for the test and help maintain their fitness afterward.

“The whole idea of the wellness-fitness program is to keep people healthy as opposed to allowing them to get fat and flabby and then trying to make them healthy again,” one observer notes. “The point is to maintain a fit and healthy fire service from the beginning through the course of a firefighter's career. The wellness-fitness initiative works.” According to the CPAT guidelines, departments must fully implement the wellness-fitness initiative before implementing CPAT.

Who's using it?

The task force validated CPAT extensively (more on that later). The test is designed to measure a candidate's physical ability to perform critical firefighting tasks as a part of a fire department's hiring process.

There are eight job tasks associated with the program: stair climb, hose drag, equipment carry, ladder raise and extension, forcible entry, search, rescue, and ceiling breach and pull. The exam's eight stations must be completed in one continuous sequence. Scoring is pass/fail, and the maximum time allowed for a passing score is 10 minutes, 20 seconds.

Why pass/fail and not rank order? Rich Duffy, the IAFF's assistant to the president for health and safety, says he gets that question all the time. “Rank scoring is difficult if not impossible to defend — technically and legally,” he says. “We know from experience that if you attempt to use rank scoring, you're going to end up with costly litigation.”

All 10 task force fire departments have implemented most of the seven CPAT components, according to Duffy, including recruiting and mentoring, test preparation, test validation, test administration and orientation, and offering the test itself. Phoenix, Los Angeles, Fairfax, Seattle and Dade County have run the test for candidates. Charlotte is still in the orientation phase, but it should be testing candidates with CPAT soon. Duffy says that FDNY would be using CPAT for its next recruiting class.

Dozens of fire departments beyond the task force have implemented CPAT, with concentrations in California and Arizona. Communities such as Boise, El Paso and San Antonio use the test for their entrance exams. Mississippi, Montana and New York have implemented CPAT at the state level, and at least one other state, Connecticut, was close behind. Manitoba adopted CPAT at the province level. Duffy says that 80 to 90 jurisdictions in California were using CPAT through a joint apprenticeship program. The ICHIEFS and IAFF have distributed between 4,000 and 5,000 copies of the program in the United States and Canada.

Cost to implement

The primary expense for the wellness-fitness phase tends to be the annual medical exam for all firefighters, with the cost ranging from $300 to $700 per person. According to the ICHIEFS's promotional material for CPAT, implementation costs, beyond those for the wellness-fitness phase, run between $10,000 and $15,000. Dayton, Ohio, budgeted $15,000 for its start-up CPAT program and $3,000 for annual maintenance thereafter. However, most sources contacted by Fire Chief placed the actual cost closer to the $22,000 to $25,000 range. For example, the Connecticut Fire Academy reported that it had spent about $25,000.

The material needed for the actual course set-up, including equipment, lumber, paint and nails, runs between $20,000 and $22,000, Duffy says. Phoenix Battalion Chief Scott Peltin suggests a figure closer to $25,000, though that includes items beyond course set-up.

“In addition to the materials to construct the course, a department needs to do a transportability study, and the cheapest transportability study I know is about $5,000,” Peltin says. Such studies, which confirm that an agency's job behaviors are substantially similar to the 10 departments that developed the exam, can carry a much higher price tag.

Duffy and Peltin agree that for the most part, CPAT doesn't require anything beyond what a department would already spend on recruiting, mentoring and proctor training for any physical ability test.

Neither Duffy nor Peltin's estimates cover outside help that departments might seek. A growing cadre of consultants offer help with course site selection, test equipment assembly, test coordinator training, candidate orientation and other preparatory programs.

ICHIEFS points out that implementation costs could be absorbed by several jurisdictions pooling their resources for a joint program. Phoenix has run CPAT tests for neighboring jurisdictions, and a number of departments are exploring regional agreements to implement CPAT.

What about the cost to offer the test itself? Los Angeles County has run CPAT twice for a total of 235 candidates, according to Chief P. Michael Freeman. He says that they have six people to set up the course and 16 proctors during each test. “Based on our hours and wages, it costs us approximately $9,000 to administer each test,” he says. “That will obviously vary from jurisdiction to jurisdiction.”

Sample pass/fail rates so far

Heart attacks and strokes kill more firefighters every year than any other cause. One would expect that a concerted, nationwide firefighter fitness regime would eventually reduce line-of-duty deaths. In the meantime, there are obvious advantages to having fit, healthy firefighters, not the least of which is being able to rescue civilians and each other from peril.

According to ICHIEFS, implementing a fitness program such as CPAT can also benefit fire departments by reducing lost work time due to injury and reducing the number of workers' compensation and disability claims. Although CPAT hasn't been challenged in court yet, one oft-touted advantage of a stringently-validated test is protection from such challenges.

It's still too early in the process to determine any effects on firefighter health and safety. Though it's not clear whether anyone's keeping a nationwide tally of overall pass/fail rates from CPAT tests, the relatively few tests administered so far offer a useful snapshot.

Phoenix and surrounding jurisdictions. “We've run the CPAT one time for the city of Phoenix,” Peltin says. “We invited almost 1,000 candidates, and we had 951 people show up. Of the 951 people who took the test, about 80% passed it. We've also run it for eight or nine neighboring jurisdictions. They all had about a 25% fail rate, a little bit higher [than ours].”

El Paso. “We offered CPAT twice in September last year,” says El Paso Fire Department Capt. Don Mehl. “On the first day, we had 27 candidate pass and 20 who failed, [a 57% pass rate.] On the second day, we had 28 candidates pass and 17 fail, [a 62% pass rate.]” There were no repeat candidates.

Los Angeles County. Freeman says the pass/fail rate for CPAT closely matches historical results from the department's former physical ability test. “On average, for the previous agility test we administered, 88% of the candidates who attempted, passed,” he says. “With CPAT, by comparison, 91% of the candidates completed it successfully.”

How women fare

To absolutely no one's surprise, CPAT's pass/fail rate for women candidates generates lively discussion. Although most generally agree that women candidates fare worse on the test than men overall, there are a number of tactics departments and individuals can use to improve women's chances.

As one might suspect from the strong overall pass rates mentioned above, Phoenix is widely hailed for its success rate with women candidates. As Peltin mentioned, the Phoenix Fire Department tested 951 candidates in its initial outing, with 80% passing. Forty-three of 120 women passed, nearly 36%. Peltin attributes the relatively high success rate to the department's aggressive pre-test training and mentoring programs.

Phoenix administered the test for other jurisdictions, however, and the differences showed. “[Other departments] had a lower pass rate for women, but they didn't do nearly the recruiting and mentoring that we've done,” Peltin says. The neighboring jurisdictions typically had fewer total candidates, though, which makes it more difficult to find the significance of the numbers. For instance, some only had three women take the test.

Still, Peltin says that some of the candidates those jurisdictions rounded up never stood a chance: “I can say, though, having been there and administered all the CPATs myself, some of those jurisdictions ran women who wouldn't pass San Francisco's test, which is by far the weakest physical ability test in the country.”

Of the 235 candidates Los Angeles County has tested using CPAT, 12 were women; seven, 58%, passed. That pass rate is slightly higher than the 33% rate for women seen under the department's previous physical agility test, Freeman says. All seven went through recruit school successfully and are now assigned to county fire stations, he says.

FDNY Lt. Brenda Berkman, an attorney and former president of the Women in the Fire Service board of directors, tracks women's pass/fail statistics closely. Based on initial observations and analysis, Berkman offers three primary conclusions about CPAT and women candidates:

• Women are passing CPAT in lower numbers than men, which may be evidence that CPAT has an adverse impact on women applicants.
• The ICHIEFS and the IAFF are not monitoring test administrators from non — task force departments who are ignoring the CPAT document's requirements regarding transportability studies, environmental conditions and restriction of the CPAT to entry-level applicants.
• CPAT's documentation does not sufficiently advocate pre-test programs, as there are no included model programs or best practices for pre-test training, specificity training and regular access to the actual test course before the test.

According to ICHIEFS President Mike Brown, the task force is already taking steps to address Berkman's concerns about how the test is being administered.

“The task force wants to look at the statistics of completed programs, and they want to make sure that every department is administering it the way it was designed,” Brown says. “They're working on how they modify the license to make sure that when somebody buys the CPAT process, that it's done correctly. If it's not, you've invalidated all the work that we did.”

Berkman says that even the few administrations of the CPAT test so far seem to indicate an adverse effect on women. “The discrepancies in terms of pass rates vary tremendously,” Berkman says. “In some instances, up to 44% of the women taking the exam have passed the test. In other case, it's much, much lower — one out of 10 was one department's experience.”

Jill Young, a 10-year veteran firefighter with the Fairfax County (Va.) Fire & Rescue Department, helped validate CPAT in Phoenix. “I know there are a lot of women who believe CPAT hurts women's chances to join departments, but it's a fair test of the job,” she said. “It requires a lot of upper-body strength, which genetically is harder for women to achieve. It's a hard test, but I do believe that it's attainable for women.

“How much does the average firefighter weigh with turnout gear? … Who are you trying to be prepared to save: the kid on the fifth floor or your fellow firefighter?”
— Dr. Paul C. DiVico

“First, you've got to have the physical ability, and second, you've got to have the mental ability to push yourself. The crew needs to rely on every single person, male or female,” Young says.

Phoenix Fire Chief Alan Brunacini, Peltin and Berkman agree that Phoenix has a higher success rate with women candidates than most departments because it recruits aggressively and has an extensive, well-organized pre-test training program available to everyone.

“To a major extent, that test is going to be effective based on who we can get to show up,” Brunacini says. “If we can't recruit appropriate candidates for it, you can give the best test in the world and it's just interesting, nothing more.”

Duffy cites two departments' implementation of CPAT where 70% of the women who participated in the mentoring program passed the test; fewer than 30% of women who didn't participate in such a program passed.

Watered down?

In this post — Lanning v. SEPTA world, there are bound to be those who accuse ICHIEFS and the IAFF of “dumbing down” CPAT to help women pass it.

Dr. Paul C. DiVico is an occupational physiologist and the coo of Health Metrics Inc., one of several private companies offering CPAT some competition in the arena of job-based physical performance testing.

“I think CPAT is an outstanding representation of the fundamental physical challenges a structural firefighter has to face,” DiVico says. “But I think it's disappointing that its format isn't significantly modifiable to reflect each agency's unique demands. It's way better than what came before, but it doesn't go far enough.” He believes that CPAT's expense and logistics may make it burdensome for smaller agencies.

The other primary argument DiVico has about CPAT is that it takes a one-size-fits-all approach and doesn't require a department to gather data and create a test that addresses its particular missions. For example, CPAT doesn't research average patient weight or differentiate between a city full of high-rises and a community where the tallest building is only three stories.

DiVico offers an example of CPAT's “arbitrary requirements”: the size of the mannequin used for the victim rescue portion of the test. “When was the last time you picked up an adult patient having a heart attack who was not overweight, who weighed 165 pounds, 145 pounds or less?” he asks.

“How much does the average firefighter weigh with turnout gear? There's not a 150-pound female firefighter who's not going to weigh close to 200 pounds with her turnouts on. Who are you trying to be prepared to save: the kid on the fifth floor or your fellow firefighter?”

Intimations that CPAT isn't tough enough or has been watered down in the hopes women will pass rankle the test's developers. “I run our training academy, and what some of CPAT's competitors say should be required is a little out there,” Peltin says. “I don't think there's a tougher fire academy than Phoenix; it's fair but physically demanding.”

According to Duffy, CPAT itself helps prevent watered-down tests. “Our task force agreed unanimously that recruiting goals should never come from lowering any validated standards,” he says. “All of the task force departments are 90% or more white male firefighters.

“And yet, many of these jurisdictions did lower hiring standards [before CPAT], and the number of women and minority recruits were not represented despite those lowered standards,” Duffy says. “Unfortunately, what we found in many cases was that by lowering standards, more white, male candidates — of lesser physical ability — came to get on the job.”

Suggestions from competitors that CPAT has been inadequately or erroneously validated draw similar ire. Duffy said CPAT development costs were between $600,000 and $750,000, excluding salaries. He outlined the development and validation process during a recent ICHIEFS teleconference.

“We spent a lot of time and resources developing CPAT,” he said. “It was a year's worth of direct work with almost 200 people. Beyond the two organizations and the 10 jurisdictions' firefighters and exercise physiologists, we hired outside lawyers who were familiar with the Department of Justice and EEOC regulations.” Seven physicians were involved in the process as well, Duffy said, as were an industrial psychologist and a psychometrician, a specialist in the theory and practice of administering, scoring and interpreting the results of mental tests.

“CPAT is the most accurately, authentically validated test that we've ever used in the fire service,” Brunacini says. “There isn't another test, including the Phoenix Fire Department [candidate] test that we gave before, with anywhere close to the validation methodology that was used on CPAT.

“Phoenix has never had anybody enjoin us on a test, so we've done pretty well with tests,” Brunacini says. Even so, he says, no one has ever validated an entrance test like CPAT. “What came before was the best you could do at the time. But what CPAT did was take us to the future.”

The incumbent testing debate

Task force members took up the thorny issue of incumbent testing last September in Calgary, where it was announced that the task force wouldn't support incumbent performance testing until certain criteria were met. Here's an excerpt from the task force's statement:

“The members of the Wellness/Fitness Task Force have concluded that before an incumbent physical ability test is developed and implemented, the fire department must:

• Establish a policy that all incumbent evaluation be non-punitive.
• Fully implement all components of the Wellness/Fitness Initiative. The full initiative shall be in place for a minimum of 24 months. The requisite components include medical fitness, physical fitness, rehabilitation, behavioral health and data management.
• Fully implement all components of the CPAT program. The requisite components include: recruitment, mentoring, pre-test orientation, training and education, transportability study, administration (proctor training, evaluation and data collection), and CPAT test.
• Establish an internal quality assurance program to review fire department programs (operations, training, fitness and/or wellness) that may be deficient.

“Furthermore, the Task Force reiterates its position that the failure of an incumbent firefighter demonstrates the fire department's (including labor, management and the individual) inability to prepare and/or maintain uniformed personnel's training skill and conditioning to perform the job-specific functions required for fire department operations.”

DiVico expressed his disappointment about the decision, saying that ICHIEFS and the IAFF are not adequately addressing the needs of incumbents. “The people at risk for illness and injury due to the physical exertion and stress of firefighting are the incumbent firefighters, particularly those who are less physically fit or less healthful, regardless of age,” he says. The people who are suffering these coronary events are not nearly as likely to be young new hires, candidates or volunteers new to the service, he says.

“The people who are responsible for meeting the public safety mission today are today's incumbent firefighters,” Di Vico says. “That's why I believe the physical demand analysis process should be devoted to those currently ‘on the line.’”

ICHIEFS President Mike Brown attended the Calgary meeting, and he supported the decision to delay incumbent testing. “The biggest fear I would have is that if they dealt with an incumbent standard, people would use that all by itself,” Brown says. “They would say, ‘You want to keep your job, you've got to pass the incumbent test,’ and they would stop there. They wouldn't provide all the support that is necessary in the wellness initiative.”

Brunacini concurs: “I don't think the wellness initiatives in some cities are as well-refined as they are in others. And one of the things we've said from the very beginning is that the key to the whole process is the wellness part. The focus is wellness. It doesn't make very much sense to give candidates this space-age test of their physical capability if, after you hire them, you don't have any way to maintain it.”

With their various wellness/fitness programs, ICHIEFS and the IAFF are changing the face of fire service fitness, but it's going to take years before its ripples reach very deep. Statistically, only a handful of career and combination fire departments have implemented CPAT and similar physical performance tests. It will be at least two years, and likely longer, before the organizations release any sort of mandatory incumbent test.

And what of the volunteer fire service? CPAT certainly wasn't designed with volunteers in mind, but according to 1999 U.S. Fire Administration data, volunteer fatalities occurred at almost twice the rate of career firefighters. In 1999, nearly 58% of the alarm-response fatalities were attributed to cardiovascular disease — related events, and 88% of these firefighters were volunteers.

Finally, what sort of fitness example are you, as a chief, setting for your personnel? Take stock of your own fitness/wellness regime; you can be sure that the troops will notice your condition as the topic of physical testing comes increasingly to the fore.

---

Timothy Elliott is a freelance writer based in beautiful downtown Burbank, Calif.