As the incidence of working fires in most urban departments continues to decline, and calls for medical, hazmat and other emergency-related services increase dramatically, almost every urban or suburban department in the United States has wrestled with how to define itself. And there's no more fundamental way for a fire department to define itself than through the personnel it hires.

I've been involved in entry-level firefighter selection for more than 20 years. During this time, I've seen departments move from the selection of almost any (male) person who walked through the door to very sophisticated and challenging selection criteria.

When the profession consisted mainly of pulling hose, crawling through burning buildings and other labor-intensive work, the key selection criteria were physical strength and stamina. This, coupled with the traditional, close-knit male camaraderie, created the basis from which other firefighters would be selected.

Very often, the selection test consisted of nothing more than an interview to see if you were the "kind of guy" who would fit in well with the existing team. As departments moved under civil service rules, however, and the job began to appeal to a new generation of individuals, competitive entry-level tests were instituted.

The first hiring program I developed for a fire department over 20 years ago included tests of cognitive ability, common-sense reasoning, personality and mechanical aptitude. Many of these are still in use today, although as women have entered the fire service, the mechanical-ability tests have fallen into disfavor, even as increased attention has been paid to physical-ability testing.

Today, the career of firefighting remains extremely appealing, resulting in situations where hundreds or even thousands of applicants compete for a handful of available jobs. They will submit to written tests, physical-ability tests, psychological examinations, medical screens, background investigations and much more before they're hired.

I began to wonder how, as firefighting evolves, the means of selecting the personnel to fill the profession have changed, and whether there are trends that point in a particular direction. To that end, I conducted a national survey on the selection requirements and testing methods for entry-level hiring.

Summary of the survey sample One hundred fifteen fire departments, representing well over half the states, responded to the survey. The South and Midwest had the greatest number of respondents, but all geographical areas of the United States were represented.

I recognize that these categories are somewhat arbitrary, and that most fire departments in the United States have fewer than 100 firefighters. However, examining the results according to these categories allows the reader to compare his or her department against similarly sized departments.

Minimum educational requirements There's no doubt that the job of firefighter has moved from one that relies predominantly on brute strength to one with considerable intellectual demands. Over the years I've obtained anecdotal evidence to suggest that the entry-level educational requirements of fire personnel have increased.

As can be seen, while most fire departments require an educational level of only a high school diploma, the actual practice is to hire above that level. For large and medium-sized departments, almost 80% of the people hired over the past two years have had either some college or an associate's degree. For small departments, over one-third of those recently hired had at least some college or an associate's degree. Nonetheless, the majority of those hired by small departments (52%) had only a high school degree or ged.

These data are consistent with my experience from recruiting for fire chief and other high-ranking positions: While most departments may require a minimum level of education, the competition generally possesses a much higher level. This suggests that departments (at least medium-sized to large ones) could raise their minimum educational requirements and not greatly change the kind of people they're currently selecting.

However, while raising minimum educational standards might not greatly affect the kinds of people who apply, it would most likely decrease the number of people applying. In other words, if a department raised its entry-level educational requirement, it could reduce the number of applicants and thus save some of the time and costs associated with testing.

Raising minimum educational requirements immediately causes concern regarding adverse impact, however. Would raising the minimum educational requirements have a negative impact on the selection of minorities and/or women? Our survey didn't directly examine this question, but we've explored this issue with some of the fire departments we've worked with over the years.

It may come as a surprise, but raising minimum educational requirements hasn't had the expected adverse impact. Basically, the results have been smaller numbers of applicants, but in relatively the same ethnic and gender percentages as when the educational standards were lower.

As a matter of fact, those departments (the majority, according to our survey) that maintain low educational requirements, yet select predominantly those applicants from a higher level, often set themselves up for adverse impact in their selection processes. One reason is that, because of the larger numbers of applicants, lower educational levels force many departments to abandon those selection devices that show no adverse impact (such as assessment centers) and instead use selection tests that have traditionally demonstrated adverse impact (such as written tests). As a result, the lower educational requirements are actually working against organizations that wish to reduce the adverse impact of their test processes.

Furthermore, departments that have low minimum educational levels convey a deceptive message to the public that anyone with a high school diploma or ged can apply for the job. While it's true that anyone meeting this minimum qualification can apply, the reality is that those who are hired typically have attained a higher level of education. Over the long term, this can hurt a department's recruiting efforts, particularly the recruitment of minorities.

Over half the survey respondents thought that at least some college and/or an associate's degree should be the minimum entry-level educational standard. Of the responding departments, 14% indicated that they'd recently raised their entry-level educational standards, and 43% had given serious consideration to raising them. This percentage did not differ greatly by department size.

Minimum experience requirements The amount of experience the entry-level applicants had was also examined, and this differed somewhat by department size. The smaller departments relied more heavily on candidates with volunteer experience than either the medium-sized or large departments did.

It's interesting to note that experience wasn't confined to fire suppression. Indeed, for small to medium-sized departments surveyed, anywhere from one-third to one-half of the firefighters hired during the last two years had experience with the delivery of medical services. This reflects the trend in the fire service as a whole toward providing ems.

Fewer than 10% of the responding departments reported raising their minimum experience levels, but over 25% indicated that they'd at least considered raising them. It's also interesting to note that half of the smaller departments believe that entry-level applicants should have one to three years' experience, while less than a third of the large and medium-sized departments felt that this amount of experience was necessary.

As a matter of fact, over three-quarters of the large departments and over half of the medium-sized departments reported that no experience was appropriate for entry-level candidates. The relatively large percentage of departments that prefer candidates to have no experience is consistent with comments from training personnel, who report that they often have to "re-train" experienced personnel on various aspects of firefighting or ems delivery.

There may be a legitimate concern regarding the potential for adverse impact against racial minorities should experience requirements be raised. If one assumes that racial minorities are more concentrated in urban settings, while volunteer departments are more commonly found in rural areas, there may indeed be a potential for bias.

On the other hand, since women are often in larger proportions in volunteer fire departments than in paid departments, raising minimum experience requirements might favor them. Perhaps what's needed is a closer examination of the type of experience departments want, more than the amount.

Entry-level testing processes The survey also explored the types of tests used at the entry level and the order in which they were used, as well as the percentage of applicants passing each test. Among all departments, 96% use some type of written test as a screening device. Similarly, 93% use some type of oral board or interview at the entry level.

The written exam was almost always the first step in the selection process. Physical-ability tests were also frequently used (94%), typically early in the selection process. Physical-ability testing, like written testing, has significant potential to be associated with adverse impact. As a result, written and physical-ability tests tend to eliminate both minorities and women early in the selection process. In addition, these are the tests that are most likely to lead to legal challenges.

Perhaps the likelihood of adverse impact, coupled with the real possibility of expensive lawsuits (and the very real pressures to get more women and minorities into the firefighting profession), is the reason that the average pass rate for written tests was over 70% and the average pass rate for physical-ability tests was an even higher 83%. Quite frankly, these figures suggest that the value, or, to use testing terminology, the utility, of written and physical-ability tests needs to be questioned.

If an organization has 500 applicants, and 70% or 350 pass the test, is there much value in giving the test? Furthermore, when that test discriminates against protected classes, there appears to be even less rationale for using it.

Sometimes there are sound rationales for using tests with remarkably high pass rates. The medical examination and drug screening tests reported by the respondents had pass rates near 95%. Nevertheless, the accuracy of these tests, combined with the seriousness associated with hiring a failure, suggests their continued use. However, for tests such as the written exam, which have modest to low validities and high levels of adverse impact, the case is much less in favor of continuing to use them.

It was interesting to note that written and oral psychological examinations were used by less than a third of the respondents. Generally speaking, these methods have demonstrated only a modest ability to predict future behavior, and their costs are often quite high.

Biographical inventories or questionnaires that explore an individual's personal history are used by less than 25% of the respondents to this survey. Again, the validity and reliability of these types of measures probably contribute to their lack of use by a wider section of the surveyed organizations.

Common problems with testing processes An open-ended question at the end of the survey solicited common problems that fire departments were experiencing with their testing processes. The two most common problems cited were the large numbers of applicants for the small number of jobs (and the costs associated with testing and processing so many applicants), and the problems associated with getting enough minorities and women successfully through the process.

I believe that these two problems are actually related. When organizations are inundated with large numbers of candidates, they usually decide to use the least expensive and most easily administered method, such as a written test. But 50 years of research on written tests has shown, without qualification, that written tests result in adverse impact against protected classes.

Organizations are left with three options. The first is to continue along the path that has been in use for most of the past 20 years: a written test and probably a physical-ability test. This method is appealing because of its relatively low initial costs.

However, the legal costs associated with court challenges, the problems with consent decrees and similar outcomes often outweigh the initial advantages of this approach. Perhaps the costliest result of using this approach, however, is an organization that lacks diversity and doesn't represent the community it serves.

A second option is to accept the fact that large numbers of people will apply for positions in the fire service, and then use techniques that don't demonstrate adverse impact, such as assessment centers. The logistics of conducting an assessment center for 100, 500 or more candidates may seem overwhelming, but I've had great success at increasing the validity of the overall selection process while at the same time eliminating adverse impact by using assessment centers at the entry level.

Furthermore, I've conducted a modified assessment center for over 700 candidates, and the process was completed in less than two weeks. This process involved citizens, firefighters and a significant commitment of resources, but the results were extremely positive. The people at the top of the list were among the best and most diverse candidates the department had seen in years.

Obviously, an assessment center process for many hundreds or possibly several thousands of candidates requires a major commitment of resources and money. But the increased validity of the process (the fact that the assessment center can better select candidates who will succeed on the job), coupled with its contribution to a diverse and representative workplace, might justify its use in your community.

The third option is to limit the number of applicants so you can conduct a more rigorous and less problematic process. For example, by raising the entry-level standards, an organization might be faced with one-tenth the number of people who usually apply. With the proper criteria, it's easier to select well-qualified candidates from this smaller number than it is to select good candidates from the much larger group of people who apply without these more stringent standards.

The case for more education But what criteria should be raised? From our survey, it would appear reasonable for an organization to look at its educational requirements. First of all, most departments are already hiring people with some college. Secondly, the perception from over half the surveyed departments was that firefighters perform better with some college. Raising the educational standard to two years of college might therefore be worth considering.

Finally, a study by the Police Executive Research Foundation found 17 reasons that a college education benefits candidates coming into the protective services:

1) Develops a broader base of information for decision making.

2) Provides additional years and experiences for increasing maturity.

3) Inculcates responsibility in the individual through course requirements and achievements.

4) Permits the individual to learn more about the societal and historical forces that shape our country.

5) Engenders the ability to handle difficult or ambiguous situations with greater creativity and innovation.

6) Develops a greater empathy for minorities and their discriminatory experiences both through coursework and interactions in the academic environment.

7) Engenders understanding and tolerance for persons with different lifestyles and ideologies.

8) Leads to less rigid decision making.

9) Helps individuals communicate to the service needs of the public in a competent manner.

10) Makes individuals more innovative and flexible when dealing with complex problems.

11) Allows individuals to better perform tasks with little or no supervision.

12) Helps individuals develop better overall community relation skills.

13) Engenders more "professional" demeanor and performance.

14) Enables individuals to better cope with stress and to be more likely to seek assistance with personal or stress-related problems.

15) Enables individuals to adapt their styles of communication and behavior to a wider range of social conditions.

16) Tends to make individuals less authoritarian and less cynical.

17) Enables individuals to more readily accept and adapt to organizational change.

Some readers might wonder whether higher levels of education can be supported by their current job descriptions, or whether such higher standards would be defensible in a court challenge. To the first concern I say, if your organization's job description for firefighter doesn't support at least some college, it has probably been several years since a hard look has been taken at the job's real demands.

As for the courts supporting higher educational standards for firefighters, it's worth examining those jobs in which the courts did not support such higher standards. Typically, three factors were present in such cases. First, the jobs were considered vocational or labor positions, and there were no factors supporting the jobs as a profession. Second, the jobs did not involve any unique public responsibility. And third, there was no unusual degree of risk associated with the jobs.

Clearly, the job of today's firefighter can be recognized for its professionalism, its public responsibility and its high risk. Raising the educational standard for firefighters could be easily defended in most departments.

Raising the required experience level, on the other hand, is not as clear. While it seems intuitive that a department would benefit from hiring a new firefighter who had experience delivering medical services rather than delivering pizzas, our survey suggests that many departments prefer to train their personnel "fresh" and without the interference of other experiences related to the fire service.

This does, however, vary with the size of the department, with smaller departments preferring more experienced personnel and larger departments preferring to rely on their own training resources. Finally, if the minimum experience level is a concern, the survey results indicate that raising it above three years isn't worthwhile.

As the fire service enters a new era, it's important to re-examine the ways in which people are selected for this profession. The written test is the most widely used technique, and has been for many years. Unfortunately, its numerous weaknesses call into question its value for today's fire departments.

First, the written test doesn't measure those qualities that distinguish between good and poor firefighters. While written tests can adequately measure knowledge, I seldom hear about firefighters who were fired because they lacked knowledge. Instead, qualities such as work ethic, motivation to learn, ability to work in groups and ability to get along with other people are the factors that distinguish the good firefighters from the poor ones. These are qualities that are more accurately measured with tests such as assessment centers.

Second, written tests, particularly the less expensive, generic ones favored by many departments, aren't amenable to incorporating an individual department's values, goals and missions. This is a distinct disadvantage and a significant lost opportunity. I firmly believe that all tests, from entry-level through promotional tests, should reinforce a department's mission and values. Furthermore, by incorporating these factors into the test, departments can give candidates a more realistic perspective of what the job actually entails.

Finally, written tests have, for over 50 years, been proved to produce adverse impact against protected classes. This results in increased liability for fire departments, and perhaps most importantly, results in organizations that are less prepared for the next century.

The results of this survey indicate that fire departments are taking steps to better prepare themselves for the new millennium. However, this change needs to move at a faster pace, since the time of change is already here.