Survey Design: Building, Distributing, and Analysing Results

Likert scales (1–5 or 1–7 agree/disagree) measure attitudes — use odd-numbered scales to allow neutrality. Semantic differentials ('complicated 1–7 simple') measure perception of specific attributes. Rating scales (0–10 NPS style) measure satisfaction and loyalty. Single-select and multi-select closed questions measure behaviour and demographics efficiently. Open-ended questions capture unexpected themes but require manual analysis — limit them to two per survey to preserve completion rates. The golden rule: each question must map to a decision. If you cannot describe how the answer would change what you build, cut the question.

Double-barrelled questions ask two things at once: 'Is the interface fast and easy to use?' Fix: split into two questions. Loaded questions embed an assumption: 'How frustrated were you by the checkout process?' Fix: 'How would you describe your experience with the checkout process?' Leading questions suggest a desired answer: 'Would you agree that the new design is better?' Fix: 'How does the new design compare to the previous version?' Acquiescence bias causes participants to agree with statements regardless of content — balance positive and negative framing. Order effects occur when early questions prime responses to later ones — randomise question order where possible.

For in-product surveys, a lightweight HTML form with a fetch POST to your analytics endpoint is sufficient for many cases. Typeform and SurveyMonkey embed scripts add 50–200 KB to your page weight. A minimal accessible survey form looks like this:

1<!-- Accessible in-product NPS survey -->
2<form id="nps-survey" aria-label="Quick feedback survey">
3  <fieldset>
4    <legend>How likely are you to recommend this tool to a colleague?</legend>
5    <div role="group" aria-label="Rating from 0 to 10">
6      <!-- Generate 0-10 radio buttons -->
7      <label><input type="radio" name="nps" value="0" required>0</label>
8      <label><input type="radio" name="nps" value="1">1</label>
9      <label><input type="radio" name="nps" value="2">2</label>
10      <label><input type="radio" name="nps" value="3">3</label>
11      <label><input type="radio" name="nps" value="4">4</label>
12      <label><input type="radio" name="nps" value="5">5</label>
13      <label><input type="radio" name="nps" value="6">6</label>
14      <label><input type="radio" name="nps" value="7">7</label>
15      <label><input type="radio" name="nps" value="8">8</label>
16      <label><input type="radio" name="nps" value="9">9</label>
17      <label><input type="radio" name="nps" value="10">10</label>
18    </div>
19    <span id="nps-low">Not at all likely</span>
20    <span id="nps-high">Extremely likely</span>
21  </fieldset>
22  <label for="nps-reason">What's the main reason for your score? (optional)</label>
23  <textarea id="nps-reason" name="reason" rows="3" maxlength="500"></textarea>
24  <button type="submit">Submit feedback</button>
25</form>
26
27<script>
28document.getElementById('nps-survey').addEventListener('submit', async (e) => {
29  e.preventDefault();
30  const data = Object.fromEntries(new FormData(e.target));
31  await fetch('/api/feedback/nps', {
32    method: 'POST',
33    headers: { 'Content-Type': 'application/json' },
34    body: JSON.stringify({ ...data, page: location.pathname, ts: Date.now() })
35  });
36  e.target.innerHTML = '<p>Thank you for your feedback!</p>';
37});
38</script>

Store responses in a time-series table keyed by user ID and page path so you can segment by cohort.

Sample size determines whether findings are generalisable. For a 5% margin of error at 95% confidence with a 50% response proportion (worst case), you need 385 respondents from a large population. Use an online calculator (SurveyMonkey's sample size calculator or Qualtrics) rather than guessing. Distribution channels in order of response quality: triggered in-product surveys (highest quality — users are in context), email to existing customers, social media (lowest quality — self-selection bias is severe). Incentives increase response rates but can bias toward incentive-seekers — prefer small fixed incentives over lottery-style prizes. A/B test your survey invitation copy to improve open rates before investing in incentives.

Start with frequency distributions: what percentage of respondents chose each option? For Likert scales, report mean and standard deviation, not just averages — a mean of 3.5 with SD 2.1 (bimodal) tells a different story than mean 3.5 with SD 0.4 (neutral). Cross-tabulate by segment: do power users rate the feature differently than new users? Use statistical significance tests (chi-square for categorical data, t-test for continuous) before claiming a difference is real. For open-ended responses, code them into themes using affinity mapping — two coders independently then reconcile divergence. Always report response rate (responders / invitations sent) and note any known selection biases in your write-up.