Applicant Questionnaires and Gaming the System

Only 36 percent of employees who responded to the 2017 Federal Employee Viewpoint Survey said promotions in their work unit are based on merit. There are many reasons for the lack of employee and hiring manager confidence in promotion programs. Concerns about “preselection” and favoritism are high on the employee list. Those reasons assume that managers and HR folks are deliberately gaming the system to get to favored applicants.

Maybe those reasons drive some of the problems with merit promotion and competitive hiring programs, but they are not the only problems. In fact, I believe they are not the biggest problem. That honor goes to the assessment process most agencies use for their job announcements. Anyone who has applied for a job in recent years has come across an applicant questionnaire. Let’s just call it an AQ for short.

The AQ is exactly what it sounds like – a lot of questions that purport to distinguish the best candidates from the rest of the pack. In theory, they are a great idea. Simple questions that tell a manager who is among the best qualified (and who is not), without putting a large burden on applicants, should make the hiring process work better. Except when they don’t.

Before we get too far into the AQ, let’s get in the wayback machine and go back in time, to the era before we had the AQ. Then we had KSAs. Remember those? KSAs are knowledge, skills and abilities, and job applicants had to write lengthy narratives describing how their education, training and experience matched the KSAs for the job. Virtually no one liked KSA narratives. They were time-consuming to write, miserable to read, and many folks believed they encouraged applicants to lie embellish their qualifications. The AQ was intended to eliminate those problems and improve the hiring process for HR, applicants and hiring managers. It is safe to say that it did not work out quite like it was expected.

What happened? Why did what seemed like a great idea not make the kind of difference everyone expected? I believe there are three reasons.

  • Hiring managers are not involved in the process as much as they should be. A good AQ requires input from someone who has in-depth understanding of the job being filled. That means an experienced supervisor or subject-matter expert (SME). Too often they are done by HR alone. Even the best HR specialists are not SMEs. The Office of Personnel Management (OPM) recognized the problem when it created its hiring excellence program, which is intended to bring HR and hiring managers together to fill jobs. Unless agencies expand their use of SMEs, the problem with poor quality AQs will not go away.
  • People lie. This one should not be a surprise. The people who lied when they wrote KSA narratives are still lying. The difference is that it is easier and faster now. No need to write 500 words to inflate your experience – just check the box and go on to the next question. Sadly, the number of people who lie on job applications seems to be going up. When we started using the AQ (about 25 years ago), lying in response to the questions did not appear to be a big problem. Now it is.
  • Many of the AQs are just terrible. Even if we solved the problems with lying and lack of SME involvement in the process, the AQs we see for many jobs have to be better. There are two big problems with them. The first is that they take a basic format – five answers ranging from “I don’t know anything” to “I am an expert” and use them for every question. The second is that there are far too many questions. Here is an example of the first problem (from a current job announcement):
Research, collect data, and provide advice and recommendations on research techniques to determine the most efficient and cost effective means of accomplishment.
  1. I have not had education, training, or experience in performing this task.
  2. I have had education or training in how to perform this task, but have not yet performed it on the job.
  3. I have performed this task on the job. My work on this task was monitored closely by a supervisor or senior employee to ensure compliance with proper procedures.
  4. I have performed this task as a regular part of a job. I have performed it independently and normally without review by a supervisor or senior employee.
  5. I am considered an expert in performing this task. I have supervised performance of this task or am normally the person who is consulted by other workers to assist or train them in doing this task because of my expertise.

Finding an example of this type of AQ took 30 seconds. They are everywhere. From the perspective of simplicity, they seem great. Come up with a job requirement, then drop in these responses. The first three answers mean you have never done the work, or you have only done it with close supervision. You have to get to answer #4 to find someone who actually has decent experience. Even #4 does not mean you did it well. It did not take long for a people to figure out that selecting answer 5 would get the most points. It is also hard to prove an applicant who selects answer #5 lied about it. “Considered an expert” is vague, as is “supervised performance.” Almost anyone can say they are “consulted by other workers.” The fact that the hiring and promotion processes are not highly regarded adds to the problem, making it easier for people who would never consider themselves to be liars to pick answer #5, even if they are not objectively what anyone would call an expert. People have told me that they pick the top answer because “It is a way to get to the interview process.” With this type of pattern, we should not be surprised when hiring managers complain that they get lists of people who cannot do the job.

The second part of the problem with AQs is the number of questions. It is common to see 20 or more questions per job. Many of those are junk questions that do not differentiate the great applicants from the average ones. If there are five questions that address the core requirements of the job, and 15 questions that are more general, such as “ability to communicate orally,” it is likely that people who have the necessary skills will not make the cut.

If we want to make the hiring process better, and we want to build employee trust in promotion programs, we have to stop misusing the AQ. They must be tailored to the job and they must have something other than the same five answers. They need to differentiate between good candidates and bad candidates in a way that the five standard responses cannot. And finally, we have to recognize that the AQ is only one part of the evaluation process. When OPM got rid of KSAs, they told agencies they could not use them in the initial screening process. They also said agencies could use multiple assessments. What happened was only the first part. We got rid of the KSAs, but we did not use additional assessments. We may not see comprehensive hiring reform any time soon, but we can and must improve the assessment process now.


Are You a Good Bureaucrat, or a Bad Bureaucrat?

I read an article recently that was mainly an exercise in badmouthing government employees as “bureaucrats” (I’m not going to link to it and give the writer any more traffic). Bureaucrat is one of those words that can have multiple meanings, depending on who uses the word, the context where it is used, and even the tone of voice used when saying it. Bureaucrats make government run, but they can also gum up the works. With that in mind, if you work in government, are you a good bureaucrat, or a bad bureaucrat? If you do not work in government, do you use the term bureaucrat as a term of derision? Or do you accept that there are both good and bad bureaucrats?

Let’s start with the definition of “bureaucrat.” A bureaucrat is an official of a bureaucracy. So – what is a bureaucracy? defines it as:

  1. Government by many bureaus, administrators, and petty officials.
  2. The body of officials and administrators, especially of a government or government department.
  3. Excessive multiplication of, and concentration of power in, administrative bureaus or administrators.
  4. Administration characterized by excessive red tape and routine.

Only one of those definitions (#2) could be characterized as good or neutral. The others are negative. Petty officials? Excessive multiplication and concentration of power? Excessive red tape and routine? The sad fact is that many people view federal, state and local employees through a similar lens. Rather than looking at the good characteristics of government workers, they look at the red tape, or the unresponsiveness of some government workers, and think that bureaucrat is the longest four-letter word in their vocabulary.

Like the witches in the Wizard of Oz, there are good bureaucrats and bad bureaucrats. The people who beat other folks over the head with rules are bad bureaucrats. The ones who go out of their way to help people understand government services are good bureaucrats. Those who lack any sense of urgency about their work are bad bureaucrats. Those who find ways to save money for the taxpayers are good bureaucrats, while those who can waste millions of dollars as though it is Monopoly money are bad bureaucrats.

In the 33 years I was in government, most of the people I encountered were good bureaucrats. They cared about the work they did. They respected the taxpayers and spent their money wisely. They put in (at least) 8 hours of work for 8 hours of pay, and they viewed their work as public service. They were proud of the work they did, and they were good at it.

The number of people who deserve to be called bad bureaucrats is not that large. Yes, there are folks in government who love red tape, and who think you should not put off until tomorrow what you can put off until next week. Or next month. There are people who are delighted to use whatever power they may have to make themselves feel special. Those are the folks who give government work a bad reputation. That small number of people do a great deal of harm, and they are just as unpopular with their coworkers as they are with the public.

Even though bureaucrat can have a neutral meaning, I think it is time to stop using the term to describe government employees. For the vast majority of those folks, “public servant” is a better term to use. It is more descriptive, and certainly more accurate. We can keep “bureaucrat” for that small number of government employees who do not serve the public. Or maybe we can get people like that out of government and make everyone (except the bureaucrats) a little bit happier.