
Three decisions that used to be settled by whoever talked longest now get settled by a form. That's the honest summary.
The first was a homepage direction for a utilities client. Two layouts, twelve stakeholders, each rating both on clarity and trust plus a sentence about what they'd click first. The loudest voice in the room had already declared for the bolder option; the written answers went the other way, largely because the operations manager — who never speaks in meetings — noted that she couldn't find the outage map. We shipped the calmer layout, and that comment was the reason.
The second was error copy. We put five messages in front of forty people and asked, for each, what they thought had happened and what they'd do next. Three of the five were read as user error when the fault was ours, including my favourite, "We couldn't verify your details," which everyone interpreted as "you typed something wrong." Rewriting took an afternoon; testing the rewrite took twenty minutes.
The third was navigation labels for a portal. Same method, simpler question: given a task, which of these six words would you click? "Resources" lost to "Guides" by a margin wide enough to end the argument permanently. Support tickets asking where to find things thinned out within a month.
What I value most is the second round. A study that took a day to build gets duplicated and re-run with fresh stimulus the same week, which means we test the fix rather than only the problem. Research that can't afford a follow-up tends to stop at the diagnosis.
It's a questionnaire engine, not a research practice: no weighting, no panel, no repository. The thinking still comes from us — and so does the honesty. A leading question returns a confident answer that means nothing. Review collected by and hosted on G2.com.
The recurring theme is what the tool won't show me. Four blind spots, each costing a round of manual work.
Who rushed. A study recruited through a social post brought 260 completes in two days, and a dozen were plainly worthless — one-word answers, straight-lining, the same phrase pasted down a column. No attention check, no duplicate guard, no way to flag a suspicious row. I read every one by hand before trusting a single percentage.
Where they quit. A nine-question form lost a third of its participants, and there's no per-question abandonment view. I could see the total, not the cliff. We split it into two shorter forms and the rate recovered, but I inferred the cause rather than measured it.
What the comments actually mean. Four hundred open responses export as a spreadsheet, and the sorting, tagging, and counting are mine. When thirty people describe the same problem in thirty different phrasings, that becomes an evening of reading, not analysis.
When the logic broke. Editing a question inside a branched screener quietly invalidated a rule, and a group of participants skipped the stimulus image entirely. We discovered it because one respondent asked what they were supposed to be looking at. There's no validation pass before publishing, so the first test is live.
Media is the fifth. We wanted to show an animated mark and the answer was a link to an external host, which adds a click and loses people at exactly the wrong moment.
None of this makes it unusable — for a clean, short questionnaire it's quick and the price matches the scope. But the moment a study needs quality control or an audit trail, I am the control and I am the trail. That's a strange division of labour for a research tool. Review collected by and hosted on G2.com.