Data Collection for Program Evaluation 10
Keep your questions simple3.
Compound sentences force respondents to keep a lot of information in their
heads, and are likely to produce unpredictable results. Example: “Imagine
a situation where the production supervisor is away from the line, a series
of defective parts is being manufactured, and you just heard that a new
client requires ten thousand of these parts in order to make their production
schedule. How empowered do you feel by your organization to stop the line
and make the repairs to the manufacturing equipment?” This question is too
complex for a clear, usable answer. Try breaking it down into component
parts.
Stay focused—avoid vague issues 4.
If you ask “When did you last see a movie?” you might get answers that refer
to the last time your respondent rented a video, when you are really inter-
ested in the last time the respondent went out to a movie theater. Consider
too, “Please rate your satisfaction with the service you have received from
this company.” This is a fine general question, but will not likely lead to
any specific action steps. Particular elements of service must be probed if
responses are to result in specific recommendations.
If a question can be misinterpreted, it will be 5.
“What time do you normally eat dinner?” will be answered differently by
people living in different regions; “dinner” can refer to either the midday or
the evening meal. Be clear, concise, always beware of imprecise language
and avoid double negatives.
Include only one topic per question (avoid “double-barreled” questions) 6.
How would you interpret the responses to “Please rate your satisfaction with
the amount and kind of care you received while in the hospital.” or, a ques-
tion asking about speed and accuracy? If you want to be able to come up
with specific recommended actions, you need specific questions.
Avoid leading questions 7.
It is easy, and incorrect, to write a question that the respondent believes has
a “right” answer. “Most doctors believe that exercise is good for you. Do you
agree?” is an example of a leading question. Even the most well-meaning
researcher can slant results by including extraneous information in a question.
Leading questions can be used to prejudice results.
Consider alternate ways to ask sensitive questions8.
Some questions are obviously sensitive. Income, drug or alcohol consump-
tion and sexual habits are clear examples of topics that must be asked about
carefully. The question: “Did you vote in the last election?” has an element
of sensitivity in it as well. Respondents might be unwilling to admit that they
did not vote, because of civic pride or embarrassment. To avoid respondent
alienation, it can be useful to mitigate the cost of answering “No” by includ-
ing a way out. For example: “There are many reasons why people don’t get a
chance to vote. Sometimes they have an emergency, or are ill, or simply can’t
get to the polls. Thinking about the last election, do you happen to remember