What gets measured, gets done. If someone is keeping track of performance, then someone else is probably doing everything he or she can to meet that expectation. There are thousands of people looking at the statistics produced during the Olympic Games held in China this summer. On the other hand, if nobody measures it, frankly, most of the time nobody cares.

The phenomenon of performance measurement in the fire service is nothing new. More than 20 years ago, public officials recognized that performance measurement would be either embraced by or forced upon government. Notwithstanding wholesale failures of various levels of government to actually do much with performance measurement, the more successful fire organizations today use performance measures as if they were playing in Major League Baseball.

If you follow that sport very closely, you know that statisticians keep a myriad of numbers. The numbers track more than games won or lost; they also track the various reasons why they were won or lost. The numbers focus on more than team statistics; win or lose, every player's performance is tallied. I was always amazed listening to baseball as a kid when the announcer very casually mentioned that this was the 27th time that a third baseman had struck out against the left-handed pitcher.

Just like home runs are not the only indication of a baseball team's performance, response time is not the only indication of a fire department's performance. To place too much emphasis on one statistic places the focus on one failure instead of the performance.

The statistical array of a contemporary fire department is less broad than many people think and simultaneously more broad than what many fire departments develop. The opposite of keeping track of only one statistic is trying to keep track of everything. And that track is almost equally dangerous.

Some of baseball's individual statistics track the number of at bats, on-base percentage, and doubles and triples hit. There is a win and loss score measuring the number of times a player actually took the bat and could score.

When it comes to teams, the most basic statistic is the one that determines the game's winner: how many runs a team scored versus how many it gave up. We also look first at the number of wins versus the number of losses over the course of a season and how it compares with each teams' competitors. Some team statistics shed light on why these more basic statistics exist. For instance, the number of times a team leaves runners at third base with less than two outs is an indicator of how many one- or two-run-margin games it will win or lose.

This creates a cascade of data on the activity of each team member that emerges as team performance. Notably everybody in professional baseball has some form of batting average. And when runners are on base, the coach wants the player with the best batting average to be at the plate. The game being played in the fire service is called initial attack, and it is one that has a strong need for a high batting average.

Over the last two decades I have looked at hundreds of communication centers in an attempt to understand how good a job we are doing in collecting some of our basic statistics, the data that defines our performance in the field. A very large number of fire agencies never question the accuracy or validity of their time statistics.

That is a mistake. In way too many dispatch centers, the programmers and program managers who are not in charge of the outcome establish the element of time. Spend some time in the dispatch center to really examine what time elements are collected and how accurate they are. For example, time elements recorded whole minutes, instead of in seconds, are like umpires calling balls and strikes with their eyes closed. Only keeping track of minutes without the seconds allows some time components to be inaccurate by as much as two full minutes.

Next, observe how accurately information is compiled on the different types of responses. Many fire agencies include non-emergency calls in their response-time calculations and subsequently inflate their average response time. On the other hand, many departments that only document code-3 responses fail to take that analysis to the next step, which is to discriminate between structure fires, vehicle fires and medical aids. These are almost the equivalent to a single, double and triple when counting the number of bases a player gets per hit. Non-emergencies are like taking a walk — nobody cares how long you take to get there. However, we should be very focused on the time elements of what it takes to get an effective response force on the scene of a structure fire. Many fire departments fail to realize that keeping track of an effective response force assembly has more to do with the outcome than just a distribution of engine companies.

The failure to equate fire prevention efforts to emergency operations is another serious weakness in our statistics. The essence of fire prevention is to prevent fires from happening. Every time a fire does happen, fire prevention has failed. There are very few links in our system that look back at whether or not a building had been inspected within a reasonable time frame before it had a fire. And, frankly, that really only applies to commercial buildings because most fire departments do not inspect single-family dwellings.

Unfortunately, keeping statistics has gotten a bad name in the fire service. Ask any fire officer what his or her least favorite duty is in the fire station; you will likely get an answer of paperwork. Yet, the paperwork that we accumulate at the engine company, department, county, state and national level literally is fuel for an ongoing debate about what we are doing about fire in this country.

There is the problem of not analyzing whether or not hose lines were laid and whether or not fire crews invoked the two-in/two-out rule. I have yet to come across a department that does a really good job of analyzing outcomes on working fires. There is a tendency at the end of the year to lump them in with a category called major events. However, there were causal factors in all major events.

One exception to that rule of course is when we kill someone on our team. There are nationwide investigations over the death of a single firefighter, and rightly so. Yet there can be multiple resident deaths in which the investigation does not extend beyond a press release issued to the local newspaper the following morning. That is not the right thing to do.

Pardon the cynicism, but who cares? Who really cares whether or not we are winning or losing the fire battle? At one level, everybody should care, especially those assigned the responsibility of protecting our communities. But the reality is that we have grown so concerned about the process that we do not talk much about the outcome. Those who are beginning to care are the watchdogs in our society. It is likely that they are elected officials, but in some cases, they belong to groups that are beginning to focus on organizational performance in government.

This last observation may be even more cynical. In the baseball world there are sandlot teams, independent leagues, minor leaguers and major leaguers. By the same token, there are fire departments that have practically no data to offer for analysis and there are major metropolitan departments that are in a data-rich environment. We don't need to collect everything from everybody, but we better be collecting the most important data from those who have the greatest experience.

Each chief must decide if his or her department is going to be a contributor or an observer. Contributors go out of their way to make sure that their information is accurate and comprehensive. Each chief will determine whether both of those conditions exist. To do so takes a certain amount of professional curiosity. It raises some levels of internal stress and pain. But in the long run, it really makes the chief a more effective contributor to the overall profession.

For decades, New York Yankee slugger Babe Ruth held the home-run record; he also held the strikeout record for many decades. The thing we remember him for was the moment that brought him glory and success — the called shot. The other home runs were just a byproduct and we have chosen to ignore them.

While response time remains the Holy Grail of the fire service, the true glory in providing fire protection is to have a community where lives and careers are not disrupted by the intrusion of fire.

---

With more than 40 years in the fire service, Ronny J. Coleman has served as fire chief in Fullerton and San Clemente, Calif., and was the fire marshal of the State of California from 1992 to 1999. He is a certified fire chief and a master instructor in the California Fire Service Training and Education System. A Fellow of the Institution of Fire Engineers, he has an associate's degree in fire science, a bachelor's degree in political science and a master's degree in vocational education.