Start with an analysis plan
State what decision the survey should inform and what you will compare. If you need to understand why prospects abandon evaluation, a general satisfaction question will not resolve the issue. Define the relevant population, eligibility rules, recruitment method and the period respondents should consider.
Pew Research Center explains that question wording, response options and order can affect answers. That is a design constraint, not a reason to treat all survey data as unusable. Make the choices explicit and pretest whether respondents understand them as intended.
Repair common question problems
| Weak question | Problem | More useful direction |
|---|---|---|
| How fast and reliable is the service? | Two judgments in one answer | Ask speed and reliability separately |
| Do you agree our process is simple? | Leading wording | Ask about effort on a recent task |
| How often do you buy regularly? | Undefined frequency | Specify a period and meaningful ranges |
| Which feature will make you buy? | Assumes a future purchase | Ask current criteria and test later behavior |
The revised direction still needs testing with the intended audience. A precise-looking question can contain terminology that means different things to buyers, users and internal staff.
Make response options answerable
Check whether categories overlap and whether important answers are missing. Provide an appropriate way to indicate that a question does not apply or that the respondent does not know. Do not force a procurement decision from someone who has never participated in procurement.
Use routing to keep questions relevant. If a respondent did not evaluate an alternative, skip the detailed comparison. Test every route, including disqualification and incomplete-response cases, before launch.
Pretest comprehension and operation
Ask a small set of suitable participants to explain what they think each question means and how they chose an answer. This is a pretest, not the final market estimate. Then test the survey on different screen sizes, confirm required fields and inspect the exported data. A questionnaire can read well and still produce an unusable file.
Report the boundaries
Publish the question wording, field dates, recruitment method, eligible base and treatment of missing answers when interpreting results. Report the denominator for each percentage. A voluntary convenience sample should not be presented with the confidence claims of a probability sample. If different segments have small bases, show the counts and uncertainty instead of creating a league table from unstable differences.
A practical next step
Write the decision brief before the questionnaire. Check whether respondents can answer each question from experience, pretest the wording and report the sample limitations. A precise response percentage does not repair an ambiguous question.
Discuss the question with Nancy, or continue with the related guides below.