Insufficient evidence! It's a phrase you often hear on television shows involving cops and robbers and attorneys and perps. If there's not enough evidence, there isn't much of a chance of convicting anybody in court. Circumstantial evidence is almost always looked at as being inadequate, ineffective and insufficient.

Well, let me put that in a totally different context. What if you were trying to collect enough evidence to prove you have a fire problem? Would you be kicked out of court for the quality of evidence that you would show the judge about your department's fire problem? Or would he accept those exhibits A through Z as accurate, reproducible and relevant to the case of running a good fire department.

In the fire service we don't talk about rules of evidence very often. Granted, our arson investigators run across them when attempting to obtain a conviction of some individual with a propensity for burning buildings. But we don't look at the data we collect as evidence, even though it's needed to justify many of the things that we want the public to invest in, such as staffing, apparatus, fire stations and other infrastructure components that build a community's fire defense system.

What would you say if your mayor, in response to one of your budget justifications, simply declared “insufficient evidence”? I've been in enough budget hearings to feel that this is an issue for the fire service. I have seen politicians dismiss fire service information because they believed it to be biased and inaccurate, and I've seen city administrators dismiss fire service evidence because they believed it to be biased and incomplete. The ability to seek an improvement is diminished because of a lack of evidence to support it.

Let me replace the word “evidence” with “records.” If you want to put firefighters to sleep quickly, just tell them you're going to conduct a class on records and reports. It's not only a “ho, hum” thing, it's also a “bah, humbug” thing. But when you're in front of the city council proposing a new fire station, increased staffing or any other major infrastructure change, you'll almost always be hammered with questions about the facts.

This is the dilemma we face. For the most part we aren't doing a very good job of collecting the facts. Some of you are, but if you have any serious doubts about whether you're collecting sufficient evidence to justify your decisions, perhaps you should read on.

The Commission on Fire Accreditation International's performance document asks, “Does the fire department perform periodic appraisal of the program to make policy shifts?” People often will check “yes” without any serious consideration. The question doesn't refer to glancing at records and then ignoring them. Appraisal of performance means looking into the information to the degree that you can determine if it's measuring what you want to have measured and if those measurements are making a difference in the outcomes of the program to which they're applied.

Some chiefs have told me that they no longer send monthly and annual reports to the authority having jurisdiction because nobody reads them. Excuse me! Even if monthly and annual reports aren't used for external evaluation, they're still an essential management tool to figure out whether you're doing your job. Do you think that your local McDonald's owner is totally unaware of his franchise's statistics?

Here's a pathetic example of how bad this can get. A friend and I were talking about auditing fire records, and he admitted that he had worked with an officer who used the same report for every fire. All of his fires were “suspicious.” All of the points of origin were “unknown.” The narrative was limited to “responded to fire, put it out” — end of story. Are fire captains deliberately ignoring a basic responsibility by misrepresenting facts that may be used to justify their fire departments' next incremental improvement in infrastructure?

Before you say it's not happening in my fire department, let me ask you some questions. Are you reporting on an annual basis through the National Fire Incident Reporting System at the state and national level? If the answer is no, then your department is contributing to the dearth of information to support modern fire protection. I guess you could rationalize that your department is small potatoes and that nobody cares, but if enough small potatoes are put together they can fill up a bushel basket.

If you are participating in NFIRS, who's responsible for the quality control of reports as they're produced by the fire suppression division? If nobody's in charge, then you have no idea how much garbage is in your record-keeping system. You have little or no idea of your fire officers' competency in addressing their responsibilities of report-writing.

If your department collects data and oversees the quality of the information, how often do you conduct a review of the data with respect to potential policy qualifications? If the answer is once a year, that might be a bare minimum. But if you haven't done anything with the data in the last 10 years, don't expect to come up with some compelling information when you're challenged across the budget table over spending more money on your fire department.

If you're on track so far, are you compiling your data into a meaningful document? We need to do more than just create a bunch of tables, charts and graphs; we need to develop our staff's skill at writing reports that inform policy-makers, both immediately and over time, of the requirements to build a strong and adequate fire protection defense system.

Finally, do you have any evidence that you're using data to manage any component of your fire department? In other words, is there a track record of taking information and translating it into organizational improvements, or is the record-keeping system a repository of dead data?

Everybody knows that firefighters hate paperwork. However, the very same firefighters expect their chief to fall on the sword to justify decisions that cost taxpayers millions of dollars. Everybody from the fire chief down to the individual firefighter must understand that we can't fabricate facts. Our problems are big enough without falsifying data. We need to regard a firefighter's record-keeping as the equivalent of a detective's homicide investigation. This isn't a game: It's competition for public trust and public investment.

The opposite of insufficient evidence is when the attorney is able to hold up an item and say, “I would like to submit Exhibit A for consideration in this proceeding.” That's what we need to be doing with our data. Creating supportive evidence that the data generated by our emergency calls are a reflection of the data that will be generated by subsequent emergency calls is a key factor in convincing public policy-makers that fire protection is not just a necessary evil, but an essential infrastructure component of a modern community. It's a quality-of-life issue.

Wouldn't it be wonderful if we could collect enough evidence to prosecute a case for improved fire protection?

---

With more than 40 years in the fire service, Ronny J. Coleman has served as fire chief in Fullerton and San Clemente, Calif., and was the fire marshal of the State of California from 1992 to 1999. He is a certified fire chief and a master instructor in the California Fire Service Training and Education System. A Fellow of the Institution of Fire Engineers, he has an associate's degree in fire science, a bachelor's degree in political science and a master's degree in vocational education.