Every form you build starts with the same tradeoff. Closed questions — radio buttons, dropdowns, scales — give you clean data you can count. Open-ended questions — textareas — give you answers you didn't expect. You can have structure, or you can have discovery. Most form creators pick one and live with the consequences.
This is the wrong framing. The tradeoff is real, but the choice is false. There is a third option that survey researchers have known about for decades: the structured interview. A question that starts open, then narrows based on what the respondent says. Discovery first, structure second. The problem was always that interviews don't scale to forms. Until they do.
The question spectrum
What the research actually shows
Howard Schuman and Stanley Presser published Questions and Answers in Attitude Surveys in 1981. It remains one of the most cited books in survey methodology. Their central finding: question format changes the distribution of answers. The same topic, asked as a closed question or an open-ended question, produces meaningfully different results. People don't just express the same opinion in different forms — they express different opinions.
Closed questions anchor the respondent. When you offer five options, people pick from those five. They rarely write in an “other” response even when the option exists. Schuman and Presser showed that response categories act as frames — they define the space of acceptable answers. If your five options miss the real issue, the real issue doesn't surface. The respondent picks the closest match and moves on.
Open-ended questions avoid this anchoring. But they introduce a different problem. Reja, Manfreda, Hlebec, and Vehovar studied the difference in web surveys in 2003, published in Developments in Applied Statistics. Their results were stark: open-ended questions in web surveys produce drastically shorter answers than the same questions in face-to-face interviews. The median open-ended response was a few words. But — and this is the finding that matters — those short responses discovered categories that the closed-question version missed entirely.
That is the tradeoff in a sentence. Closed questions are complete but narrow. Open-ended questions are incomplete but wide. You need both properties, and no single question format gives you both.
Why “just add a comment box” fails
The most common compromise is a closed question with an optional comment field. Rate us 1-5, and if you want to say more, here is a box. Don Dillman, Jolene Smyth, and Leah Melani Christian addressed this pattern in Internet, Phone, Mail, and Mixed-Mode Surveys (4th edition, 2014). Their guidance is clear: the optional comment field underperforms both pure formats.
The closed question already anchored the respondent. Their score is locked in. The comment field is an afterthought — the respondent has already given their answer and feels done. The few who do write a comment typically rephrase what they already selected. “I gave a 4 because it's mostly good.” You learn nothing new. The discovery benefit of an open-ended question evaporates when you put it downstream of a closed one.
Reverse the order — open-ended first, then closed — and you get better discovery but worse completion rates. The open-ended field demands effort upfront, and a significant percentage of respondents drop off before reaching the structured questions. You got the discovery, but you lost the data.
Neither order works. The format itself is the constraint.
Interviews work. They just don't scale.
User researchers have always known the answer: interviews. You ask an open question, listen to the response, and then ask a focused follow-up based on what you heard. “You mentioned onboarding. What part of onboarding?” The interviewer narrows the scope in real time. The respondent gets the discovery benefit of an open question and the specificity of a closed one. Both parties do less cognitive work than either format alone would require.
The problem is cost. An interview takes 30 minutes, a trained researcher, and manual synthesis afterward. At 20 respondents, that is a week of work. At 200, it is impossible. Forms exist because interviews don't scale.
So form creators accept the tradeoff: closed questions for data, open-ended questions for color, and a comment box as a halfhearted bridge. Every form you have ever filled out reflects this compromise.
Open-ended field · before and after probes
Follow-up probes close the gap
A probe is a conditional follow-up that fires based on what the respondent writes. It is the form doing what an interviewer does — reading the answer and asking the next question that matters.
“Make it better” is a legitimate open-ended response. It tells you the respondent has a complaint but not what the complaint is. A probe asks: “Which part feels off — the setup, daily use, or something else?” That single follow-up converts a three-word throwaway into a categorized, specific answer. The respondent wrote an open-ended response and got the narrowing benefit of a closed question, without the anchoring that a closed question would have imposed upfront.
This is not a new idea. Cognitive interviewing — the practice of probing respondents about their answers — has been a standard method in survey development since the 1980s. What is new is that a form can do it. The form reads the response, identifies when it is thin or ambiguous, and asks a targeted follow-up. The respondent never sees a generic “tell us more.” They see a question that references what they just wrote.
AI pre-fill adds another dimension. When the respondent hands the form to their AI assistant, the draft starts with specifics drawn from real context — tickets, messages, documents. The open-ended field arrives pre-populated with a structured answer that the respondent reviews. The discovery already happened, because the AI synthesized information the respondent had access to but would not have recalled unprompted. The respondent edits and approves, and the form sender gets an answer that is both open (it found things a closed question would have missed) and structured (it contains specific, categorized detail).
The real question format
Schuman and Presser were right that format shapes answers. Reja and colleagues were right that open questions discover what closed questions miss. Dillman was right that the comment-box compromise underperforms both. The research is consistent: no static format gives you both discovery and structure.
The missing variable was time. A static form asks its questions all at once. An interview unfolds over time — each question responds to the previous answer. Forms never had that capability, so the tradeoff persisted.
Probes give forms a time dimension. The first question is open. The follow-up is narrow. The respondent's experience feels like a short conversation, not a questionnaire. You get discovery on the first answer and structure on the second. No comment box. No forced choice between formats.
It is not open-ended versus closed. It is both, in sequence, responding to what the person actually said.
Try it yourself — or see how it works across workflows.