The Washington Post did an interesting (and excellent) article on the increased Federal Employee Viewpoint Survey (FEVS) response rates at the Department of Housing and Urban Devlelopment (HUD). Various parts of the Department encouraged their employees to respond to the survey by offering excused absence for 59 minutes, gift cards, and lunches with the boss. (Full disclosure – I was interviewed for the Post article)
Giving someone an incentive to complete FEVS is not unique to HUD. Other agencies have offered incentives such as time off, ice cream socials, pizza and other non-monetary incentives to drive up their response rate for the survey. Why would they do that? And is it fair, or stacking the deck?
Why Response Rates Matter
Some leaders want higher response rates because they believe employees who are unhappy with the agency are more likely to complete the survey than those who are happy, resulting in a high risk of nonresponse bias. The thinking is that you can improve your results by getting more people to participate and drawing in the quiet happy people. As the Washington Post article pointed out, that is not a sure bet by any means. Some agencies getter better response rates only to learn that their workers truly are unhappy, and are less satisfied than they were in the previous year. That fear of nonresponse bias is a bit of a bias itself. Why should we assume that the people who are satisfied or happy are less likely to respond? It is also possible that exactly the opposite is true – that disengaged employees are also less likely to believe their employer will do anything with the results will not respond to the survey.
There are others who want higher response rates because they believe It will produce more accurate, defensible data that they can use to get good measures and hold people accountable. They want to have high confidence in the survey results because they plan to act on them and want to ensure they are not being misled by flawed survey data.
I believe the most important reason for encouraging a high response rate is that it takes away the excuses for management inaction. When I was HR Director for the Defense Logistics Agency, almost every new field activity Commanding Officer would challenge their survey results. They would argue that so many people did not complete the survey that the validity of the survey was questionable. For some reason, that argument came most often from the people whose organizations had poor results. It even surfaced in organizations with extremely high, but not universal, response rates. When an agency increases its response rate, they have one more reason to quash this type of lame “blame the survey” excuse. For that reason, I believe driving response rates up is a very good thing.
Incentives and Outcomes
The question that most people have when they hear about incentives (such as 59 minutes off) is “Do they skew the results?” That is a fair question. I think the answer is no, it does not. Like any other good survey, the FEVS is completely anonymous. There is no way for an agency to be certain that an employee responded to the survey, and there is certainly no way to know how they answered the questions. There is no reason to believe an employee who might have had negative opinions will magically change them to positive based on getting 59 minutes off (or any other token incentive). There is some research that shows that the number of “I don’t know” responses may go down when an incentive is offered. But – it is just as likely that the “I don’t know” will go negative as positive. I doubt anyone in government is going to do a split sample and provide incentives to half the employees and no incentives to the other half, just to measure the effects of incentives.
The argument that incentivizing people to respond might cause more satisfied, higher-graded employees to respond is not really based on any solid research. It is, like responses to the survey itself, an opinion. The weakness of that argument is that it also supports the case for nonresponse bias. If it is true that getting more people to respond makes the positive numbers go up, then it may be more likely that there really is a nonresponse bias and it is driving numbers down rather than up.
I have seen one real risk of trying to drive up response rates. Sometimes managers will choose their words badly (or deliberately) and tell employees it is important to get good results on the survey. There is a BIG difference between pushing for good participation and good results.
Perceptions Matter
My concern when I first saw this issue at DLA was one of perception. I believe employee surveys are important tools that can help an organization become a better and more effective workplace. They work only when employees (a) believe they are anonymous and unbiased, and (b) believe the agency will do something positive in response to the survey data. Anything that brings the bias argument into play risks the positive outcomes that can result from a good survey. For that reason, my answer when someone asks about incentives has always been that I do not generally like them. That is my personal bias. I talked with one of my ICF colleagues who does survey research and he said he does not view properly managed non-monetary incentives as a problem. He neither encourages nor discourages them.
The HUD story shows that perceptions are important. In this case, they were important enough for one of the most prominent newspapers in the country to write a story about it. That does not mean they cooked the results of their survey, just that it is an interesting and legitimate issue for the press to cover.
The Front Office Matters
Regardless of the perception issue, HUD already has the makings of a big win. It is clear that Secretary Julian Castro is focused on engagement and workforce issues. Personal involvement and time from the head of an agency is absolutely critical on employee engagement issues. The fact that the Secretary is personally engaged makes it much more likely that the improvement they saw this year (and last year to a lesser degree) will continue. The fact that other senior leaders in HUD are paying attention to the survey is also a win.
The most recent results on the Partnership for Public Service “Best Places to Work” rankings show clearly that agencies can make big moves in employee perceptions. They can do it even in the political climate we have today, and without wholesale Civil Service reform. Those big moves (like at HUD) require a full-court press from the senior leadership of an organization. If they delegate it to others, assume good things will happen, and do not stay fully engaged, they will see numbers that stagnate or drop.
Evolving Views
I have to admit that the research I did for this blog post is making me rethink my earlier distaste for participation incentives. They are more common than I believed and there is no strong research that shows they have a significant effect on survey results. In fact, it is common practice to offer incentives to people to participate in focus groups and some marketing surveys. Given the benefit good survey data offers to agencies and their workers, I believe OPM should issue a set of guidelines for use of incentives agencies can use to encourage FEVS participation.
Share this ChiefHRO post:
Despite all of the slings and arrows directed at the federal workforce, America’s public servants go to work every day to serve the American people, and most of them do it very well. Thanks to each and every one of you, and a Happy Hanukkah (belated), Merry Christmas, Joyous Kwanzaa and Happy Holidays to Federal employees, retirees and their families everywhere!
ChiefHRO.com 