Blog

The biggest risk in any research engagement isn't a poorly written discussion guide or a last-minute participant dropout. It's starting a project when different stakeholders have fundamentally different ideas about what the research is supposed to answer — and why.

Over the years, I've watched teams spend weeks on research plans only to deliver findings that landed with a thud. "This is interesting, but it doesn't help us decide X." Sound familiar? Almost always, that disconnect existed before the first interview was scheduled. My fix: a one-hour stakeholder alignment session before anything else.

What the session covers

I run five core questions with all stakeholders present — in the room or on the call. First: what decisions will this research inform? Second: what does success look like, and how will we know if the research moved the needle? Third: what do we already believe, and how strongly? Surfacing existing hypotheses early prevents scope creep later. Fourth: what constraints exist — timeline, budget, access to participants, platform limitations? Fifth: how will the findings be shared, and who has the authority to act on them?

What you actually learn

The answers to those questions matter, but the disagreements matter more. When the product manager says the research should validate the roadmap feature and the designer says it should explore unmet needs, that's a scoping crisis you just surfaced in hour one — not after six weeks of fieldwork.

The session also reveals who the real decision-maker is. Often, it's not the person who scheduled the kickoff. And it tells you which stakeholders will push back on findings that challenge their priors — so you can plan how to present difficult results before you have difficult results to present.

Running the session well

Provide a pre-read document outlining the proposed research scope. This gives people something concrete to react to rather than an open-ended brainstorm. Use a shared digital whiteboard so everyone can annotate simultaneously. Close by summarizing the agreed-upon research questions in writing — that document becomes your north star when scope creep tries to sneak in at week three.

Approximately one in four adults in the United States has a disability. If your research consistently excludes them, your findings aren't representative — they're incomplete. And yet, disabled participants are routinely left out of UX research, often with explanations that don't hold up to scrutiny: "Our product isn't designed for disabled users." "Recruiting is too hard." "We'll do a separate accessibility study." Each of these is a way of deferring inclusion until it's too late to change the product.

The quality argument

Accessibility research isn't a niche specialty sitting beside general UX research — it's part of it. When we design tasks assuming participants will use a mouse, see all interface elements, and process content at a neurotypical reading pace, we're designing away an entire category of data. Users with motor impairments, low vision, cognitive disabilities, and ADHD interact with our products every day. Their struggle points are often our sharpest signal about where the design is genuinely broken.

Recruiting inclusive participants

The most common barrier is recruitment, and the fix is more systematic than heroic. Partner with disability-focused organizations and advocacy groups early in your research planning — not as an afterthought. Adjust your screener to ask directly about assistive technology use and access needs. Compensate participants fairly and transparently; disabled users are frequently undercompensated for their time in research studies, which both limits your recruiting pool and sends a signal about how you value their perspective.

Adapting your methods

Write technology-agnostic tasks that describe the goal rather than the gesture: "complete your purchase" instead of "click the blue button." Allow extended task time — typically 1.5 to 2 times your standard estimate. Schedule a brief tech check 15 minutes before the session to confirm screen reader compatibility, zoom levels, and audio settings. These aren't special accommodations; they're good research practice.

I've moderated hundreds of usability sessions. The most valuable lessons haven't come from watching users struggle — they've come from catching myself assuming they wouldn't. Here are five assumptions that usability testing has repeatedly corrected.

1. "Users will read the instructions."

They don't. And this isn't a failure of users — it's a failure of design. When I write a task scenario and a participant immediately asks "Wait, what am I supposed to do?", I've learned that my instructions were the problem, not their attention span. I now test my task wording in a pilot session before the real research begins.

2. "If it worked in the lab, it works in real life."

Lab conditions — whether in-person or remote — strip out the environmental variables that define real use. A checkout flow that takes 90 seconds in a quiet usability session might take 9 minutes on a mobile device in a noisy kitchen. I've started asking participants to describe their typical environment at the start of every session. It changes how I interpret everything I observe.

3. "Silence means understanding."

In early career moderation, I interpreted silence during a task as the user successfully making progress. It's often the opposite. Experienced moderators learn to read silence as a cue to probe: "What are you thinking right now?" The most important moments in a session often begin with a long, uninterrupted pause.

4. "My task wording is neutral."

No task wording is neutral. When I write "find the most affordable plan," I'm implying that price should be the decision factor. "Find the plan that fits your needs" opens the decision space entirely. I now have a co-moderator read every discussion guide specifically looking for leading or loaded language before any session runs.

5. "One round of testing is enough."

Usability testing is iterative by definition. A single round tells you what broke. A second round — after the team has addressed the findings — tells you whether you fixed it and what new problems emerged. Five participants twice is almost always more useful than ten participants once.

Research impact is measured in decisions changed, not documents delivered. A 40-page findings report sitting in a shared Google Drive is not research impact — it's a very expensive archive. I learned this lesson the hard way. Early in my career, I spent three weeks on a meticulous report with color-coded themes, verbatim quotes, and severity ratings. It was received politely and never discussed again.

The readout problem

The traditional research readout — a synchronous meeting where you present slides to stakeholders — is one of the least effective formats for driving action. By the time you're presenting, people have already formed opinions. By the time the meeting ends, the next sprint has started. Research findings need to be embedded in the moments when decisions are being made, not delivered ceremonially afterward.

Shorter, more frequent formats

I've replaced the 40-page report with a "3-minute research brief" — a one-pager with three findings, three supporting quotes, and three recommended next steps. It takes longer to write than a long report, because clarity requires effort. But it gets read. I also post a Slack summary the same day as the readout, for the people who couldn't attend and for the searchable institutional record.

Embed in design rituals

The most influential thing I've done for research impact was showing up to design critiques uninvited. Bringing one relevant participant quote to a design review — "We watched five participants miss this button because they expected it to be in the top navigation" — is worth more than a well-formatted report slide. Research becomes visible when it shows up where design decisions happen, not in a separate meeting three weeks later.

Track what changed

I keep a simple log of design decisions that were influenced by research findings. This serves two purposes: it helps me measure actual impact, and it makes the value of research visible to stakeholders during planning conversations. "Three of the features in this sprint were directly shaped by the Q1 research" is a more compelling argument for continued investment than any methodology slide.

"Navigate to the settings page." It seems like a simple task prompt. But for a participant using a screen reader, this instruction raises immediate questions: Do I use the keyboard shortcut? Do I search by heading? Do I look in the navigation landmark — and if so, which one? The task assumes a visual, mouse-driven interaction model that the participant isn't using. Writing a strong discussion guide for assistive technology users requires a different starting point than standard task design.

Write technology-agnostic tasks

The fix is straightforward but requires discipline: describe the goal, not the gesture. Instead of "click the blue button to submit your order," use "complete your order." Instead of "scroll down to the reviews section," use "find the customer reviews." The participant's job is to accomplish the goal using whatever method works for them. Your job is to observe how they do it — and that navigation path is often your most actionable finding.

Extend your time estimates

Screen reader navigation is methodical and precise. A task that takes a sighted mouse user 45 seconds may take 3–4 minutes with a screen reader — not because the product is unusable, but because the interaction pattern is different. Build realistic time buffers into your session guide. Resist the instinct to move on before a task is complete. The full navigation path is the data.

Schedule a tech-check call

A 15-minute call the day before the session to verify screen reader compatibility, confirm audio settings, and test screen sharing is not optional for AT research. Prototype tools and screen sharing software frequently conflict with assistive technology in ways that won't surface until the session has already started. Finding out at minute two is too late and unfair to the participant.

What to listen for

The most valuable signal in AT research is often not failure — it's workaround behavior. When a participant uses a non-obvious navigation path to accomplish a task, they're showing you a mental model your design doesn't accommodate. Those compensatory behaviors are where your most specific, actionable findings live.

Every research team I've worked with has the same problem: too many questions and not enough time. The backlog is full. Stakeholders are competitive. And "we'll study everything eventually" is not a prioritization strategy. The challenge with research prioritization isn't usually a lack of frameworks — it's the political dynamics that make saying "not now" feel like a professional risk.

The political problem

When stakeholders submit research questions, they're often advocating for their domain. The product manager wants to validate the roadmap feature. Sales wants to understand why deals are lost. Customer success wants to map the renewal journey. All legitimate, all urgent — and they can't all happen in Q2. A prioritization process that doesn't acknowledge the political reality will produce a ranked list that no one owns and everyone ignores.

The framework: confidence × impact

I use a simple 2×2 matrix: decision impact on one axis, existing confidence on the other. High-impact decisions with low confidence are the clearest research priorities. High-impact decisions with high confidence don't need research — they need execution. Low-impact questions, regardless of confidence level, get explicitly labeled "not now" during the session. Naming it helps teams accept it.

Running the session

The session starts with silent individual voting — everyone ranks submitted questions privately before any discussion. This prevents the loudest voice from anchoring the group. After voting, we review results together and discuss only the questions where there's significant disagreement. Items with agreement are accepted quickly; disagreement items get five minutes of structured discussion with a time-keeper holding the line.

After the session

The most important output isn't the ranked list — it's the "not now" list with documented reasons. Communicating to stakeholders why their question was deprioritized — too low impact to move the needle, high existing confidence, blocked by upstream decisions — is an act of respect. It closes the loop and prevents the same conversation from recurring next quarter with the same emotional charge.

In 2011, GrubHub was growing fast and facing a problem that looked, on the surface, like a technology problem: restaurants were missing orders. The fax machine — still the primary order delivery mechanism for most restaurants — was failing. Paper ran out during peak hours with no digital backup. Transmissions smudged. Handwriting was illegible. The proposed solution was a tablet-based order management system. My job was to understand whether that solution would actually work in the environments where it would live.

What 500 hours in restaurant kitchens during peak dinner service taught me is that fieldwork changes the research question.

Going where the work actually happens

Before the fieldwork, the team's mental model of the problem was essentially technological: fax bad, tablet good. After the first week in kitchens, the problem looked completely different. The issue wasn't just the medium — it was the workflow that had grown up around it. Cooks had memorized fax machine sounds. Managers had rituals around paper management. The desktop workflows also prevented staff from managing orders where the actual work happened: at the preparation stations. The fax machine, for all its failures, had been integrated into the muscle memory of kitchen operations over years. No prototype test in a lab would have surfaced that. You had to be there.

What ethnography reveals that interviews can't

I interviewed kitchen staff about their order management experience — and those interviews were useful. But what people tell you about their work and what they actually do are reliably, systematically different. In interviews, staff described an orderly process of checking faxes, reading orders, and delegating tasks. In the kitchen at 7pm on a Friday, that process looked nothing like the description. Ethnography surfaces the gap between the espoused process and the enacted process. That gap is almost always where the most important design opportunities live.

The outcome

The research established what became the industry pattern for tablet-first order management — a design that accounted not just for the technology, but for the physical environment, the social dynamics of kitchen hierarchy, and the workflow rhythms of peak service. Placing the tablet where orders were acted on, not just received, was a direct result of spending time in the space. None of that would have been visible from a conference room.

In 2010, Grainger had a problem that needed an answer fast. Business customers — trade professionals buying industrial supplies in bulk — were abandoning the online registration flow before completing it. The team had hypotheses. They needed data. And they needed it in 48 hours. I ran intercept testing with 20+ participants in Grainger retail locations using an Axure RP prototype, and delivered findings that produced 100% stakeholder alignment and immediate design modifications.

The right method for the constraint

Rapid research is not sloppy research. It's research scoped specifically to the decision at hand. The team needed to know why business customers weren't completing registration — not a full exploration of the customer journey, not a competitive analysis. A focused scope enables a fast timeline without sacrificing validity. Intercepting participants in retail locations was the right call for two reasons: Grainger's customers were physically present in the stores, and the retail environment was the actual context in which many users would first encounter the digital product. Watching a trade professional attempt registration on an in-store terminal told us more than a remote session ever could have.

8 barriers in 48 hours

The testing surfaced eight critical barriers to completion. Low computer literacy among trade professionals. Customers who were entirely unaware Grainger operated online. Registration perceived as an unnecessary obstacle to something they were already doing by phone without friction. B2B and B2C workflows conflated in ways that didn't match how business buyers think. Trust and security concerns. A deep dependency on the physical catalog. And — most unexpectedly — genuine anxiety about job elimination. If ordering moves online, what happens to the people who currently manage phone orders? Several participants raised this unprompted. These weren't usability problems. They were trust problems, awareness problems, and transition anxiety problems requiring design responses well beyond form field optimization.

Why the findings landed immediately

The brief was delivered within 48 hours of the final session. Because the scope was narrow and the findings were grounded in direct observation, the team aligned on priorities without debate. No methodology discussion. No waiting for a longer synthesis process. The constraints of rapid research, paradoxically, made the output more actionable — there was no room for findings that didn't directly inform the decision in front of the team.

When 7-Eleven introduced self-checkout machines in their stores, the initial hypothesis was the predictable one: if adoption is low, the machines must be hard to use. Run a usability study. Find the friction. Fix the interface. I've seen this pattern in nearly every low-adoption situation I've encountered — and it's almost always the wrong starting point. At one location, I observed approximately 80 customers come through the store during the study window. Nine used the machines.

What 9 of 80 actually tells you

An 11% adoption rate on a new feature could mean many things: a confusing interface, an unclear value proposition, a trust problem, a visibility problem. The right research response is to find out which one it is before designing a solution. In this case, in-store observation made the answer clear quickly. The machines were positioned outside the natural flow of the customer journey. Customers arrived, moved to the products, and walked to the attended checkout counter — all before the self-checkout machines entered their visual field. The problem wasn't usability. It was placement.

Zero adoption at the second location

At a second location, adoption was zero. Not low — zero. The machines were placed in a way that made customers genuinely uncertain whether they were permitted to use them. No prompts, no staff direction, no signage that invited engagement. Customers assumed the machines were non-operational or reserved for staff. The interface was never even reached. Competitor benchmarking conducted alongside the observation revealed that better-performing self-checkout deployments shared two things: prominent placement in the natural path to exit, and clear contextual signals that the equipment was available and ready.

Reframing the problem

The deliverable from this research wasn't a list of UI issues — it was a recommendation to reframe the problem entirely. The team shifted from "how do we make the machines easier to use" to "how do we make customers aware the option exists and feel invited to use it." The mobile app also lacked a store mode that connected browsing, deals, checkout, and loyalty earning into a coherent in-store journey. Placement strategy and in-app awareness features were both added to the roadmap as a result.

Most research engagements have a defined end. A project scope, a delivery date, a final report. My partnership with 7-Eleven and Speedway started that way — and then kept going. For four years, across six product areas (Fuel Hub, self-checkout, mobile barcode scanning, promotional games, loyalty programs, and the Speedway brand integration), I was embedded as a research partner while the organization navigated product launches, brand consolidation, and significant internal change. That longevity changed everything about how I practice research.

What changes when you stay

In a short-term engagement, the most valuable thing you can deliver is an accurate answer to a specific question. In a long-term partnership, the most valuable thing you can deliver is a research practice that outlasts your involvement. Over four years I built ResearchOps infrastructure, established research rhythms across teams, and helped internal stakeholders engage with research as a continuous activity rather than a periodic event. Organizations that treat research as a discrete deliverable make decisions without it between engagements. Organizations that build it into their operating rhythm make better decisions consistently.

Trust as a research condition

Short-term research relationships require you to establish credibility quickly. Long-term partnerships accumulate it. By year two, I had enough institutional trust to bring difficult findings to leadership without softening the message. When research on the Price Lock feature revealed that customers were confused in ways that contradicted the product team's assumptions — expecting the locked price to drop when fuel costs decreased — I could deliver that finding directly. It was received as useful information rather than criticism. That kind of relationship takes time to build and is irreplaceable in terms of research impact.

The patterns that only become visible over time

Across all six product areas, the recurring theme was the same: customers wanted a simpler, more unified experience than the organization's internal structure naturally produced. Win/loss iconography in promotional games was indistinguishable. Loyalty program benefits lacked customization. Customers wanted unified loyalty recognition across both the 7-Eleven and Speedway brands. Each of these was a discrete finding in isolation. Together, over four years, they told a consistent story about the gap between how the organization was structured and how customers experienced it. Research was one of the few functions with visibility across all six product areas — and surfacing those cross-product patterns became one of the most valuable contributions of the engagement.

For most of my career, the boundary between researcher and engineer was clear. I generated insights; someone else built the thing. The AI Research Playground changed that. Over the past year, I've used AI-assisted development to build a collection of functional web applications — a time tracker, a job search dashboard, a client research portal, an event site, a CMS, a consultancy site — and the experience has fundamentally changed how I think about the relationship between research and product.

What I set out to do

The Research Playground started as an experiment in capability, not a product initiative. I wanted to understand what AI-assisted development actually meant in practice — not from reading about it, but from doing it. As a researcher, I'm professionally skeptical of secondhand accounts of user experience. So I became the user. The collection includes Kaos Time Tracker (freelance invoicing and time tracking with authentication), Job Tracker (a job search dashboard with application pipeline management), a Client Portal (a research intake form with database routing), the McDowell Family Reunion event site with genealogy integration, Stop Sitting and Stand (an author CMS for a faith-based publisher), and Kindred Impact LLC (a strategic philanthropy consultancy site).

What building changes about research

When you've struggled to implement a feature you once recommended in a deliverable, you develop a different relationship with the gap between insight and execution. I now write recommendations that account for implementation complexity in ways I didn't before. I also write better specifications, because I understand what specification clarity actually costs a developer when it's absent. The empathy cuts in both directions: I have more patience with engineering constraints, and engineers have responded with more engagement with the research findings I bring.

What AI makes possible — and what it doesn't replace

AI-assisted development lowered the barrier to building enough that I could ship functional products on a research timeline without a development partner. That's a meaningful capability shift. But the decisions that determined whether those products were useful — what problem they solved, who they were for, what the flows should feel like — were entirely human. AI accelerated the execution. It had nothing to say about the strategy.

A portfolio case study can tell someone what you did. A live, interactive system can show them how you think. The UX Intelligence Platform™ started from a simple frustration: the best way to demonstrate research operations expertise is to operate research — and a static PDF can't do that. So I built something that could.

What the platform does

The UX Intelligence Platform is an interactive design system that demonstrates research operations in practice: research intake workflows, pattern libraries for common research artifacts, accessibility leadership documentation, and a living record of research methods and their appropriate use cases. It's not a concept — it's functional, and it's built to the same standards I'd apply to a client-facing research tool. AI-assisted development made it possible to build at this scope without a dedicated engineering partner.

Why ResearchOps needed its own format

ResearchOps is one of the hardest things to communicate in a portfolio. The outcomes are organizational — faster research cycles, better stakeholder alignment, reduced duplicated effort — but the work that produces those outcomes is largely invisible: templates built, processes documented, tools configured, onboarding designed. A static case study can describe this work. An interactive platform lets a visitor experience the structure firsthand, which is a more honest representation of what good research operations actually feels like to work within.

What building it taught me about accessibility

Building the platform required making every decision I usually recommend to clients: color contrast ratios, keyboard navigation, screen reader compatibility, semantic HTML structure, focus management. Making those decisions at the implementation level — not just specifying them in a deliverable — revealed edge cases I had never surfaced in research. The gap between "this needs to be accessible" and "here is how to make this accessible" is where a significant amount of real design work lives. I came out of this project a more specific accessibility advocate than I went in.

AI-assisted synthesis is one of the most discussed topics in UX research right now, and the conversation tends toward extremes: either AI will replace research synthesis entirely, or it's a parlor trick that produces plausible-sounding nonsense. After a year of experimenting with AI tools across real research contexts — including building my own AI-assisted applications in the Research Playground — I have a more nuanced view.

Where AI genuinely helps

AI is useful for the parts of synthesis that are mechanical rather than interpretive. Transcription cleaning, deduplication of similar observations, initial grouping of raw notes into candidate themes — these are tasks where AI can save significant time without meaningfully compromising quality. Asking an AI to identify recurring themes across 40 interview transcripts produces a useful starting point that a researcher can interrogate, challenge, and refine. The key phrase is starting point. Treated as a first draft rather than a finding, AI-generated groupings are a legitimate time-saver.

Where AI falls short

The interpretive core of synthesis — deciding what's significant, what's surprising, what contradicts the team's existing assumptions, what should change the roadmap — requires human judgment that AI cannot reliably replicate. AI produces patterns. Researchers produce meaning. The difference matters enormously when findings need to be actionable rather than just accurate. An AI can tell you that participants mentioned price in 60% of sessions. It cannot tell you whether that finding should change the product strategy, or whether the team already knows it and what matters is something buried in the other 40%.

The question of accountability

Research findings carry accountability. When a design decision is made based on research, someone is responsible for the quality of the interpretation behind it. AI-assisted synthesis distributes that accountability in ways the field is still working through. My current practice: use AI for mechanical tasks, make all interpretive decisions myself, and document the division of labor explicitly in my research briefs. That transparency isn't just an ethical position — it's good science, and it's what allows the findings to be trusted and acted on.

A research plan used to take me the better part of a day to write. Methodology rationale, recruitment criteria, discussion guide outline, timeline, stakeholder sign-off — it's a document that requires thought, but also a lot of boilerplate that looks similar from project to project. AI has changed that calculus significantly, not by replacing the thinking, but by eliminating the blank-page problem.

What AI does well in research planning

AI is excellent at producing structured first drafts when given clear inputs. I provide the research question, the product context, the team constraints, and the intended outputs, then ask for a draft plan. What comes back usually covers 70–80% of what I'd write myself. The methodology selection, participant criteria, and timeline scaffolding are almost always usable as a starting point. The rationale sections and stakeholder framing still need my voice and judgment — but starting from a rough structure is significantly faster than starting from a blank document.

Where you still have to do the work

The parts of a research plan that require genuine expertise are the parts AI handles worst: choosing the right method for the specific uncertainty on the table, scoping the study to the actual decision that needs to be made, and writing recruitment criteria that will surface the right participants rather than convenient ones. These require knowledge of the specific product context, the team dynamics, and the research backlog that no AI has access to. AI can scaffold the document. It can't make the calls that make the research good.

My current workflow

I write the research question and key constraints myself — this is the part that requires the most clarity of thought, and AI rushing past it produces shallow plans. I then use AI to generate a first-draft structure and populate the boilerplate sections. I review and revise with research judgment, then share for stakeholder feedback. The plan takes roughly a third of the time it used to, and the quality is the same or better because I'm spending my time on the parts that actually require expertise rather than on formatting and boilerplate.

Personas are one of the most debated artifacts in UX research, and AI-generated personas have added a new layer to that debate. I've seen both extremes: teams using AI personas as a substitute for actual user research, and teams using them as a structured starting point for a research-informed iteration process. The difference isn't in the tool. It's in how you treat what comes out of it.

What AI personas are good for

AI can generate plausible, structured persona documents quickly when given relevant inputs: product category, user segment descriptions, behavioral patterns from existing sources. These are useful in three specific scenarios. First, when a team needs a shared vocabulary for user types before primary research has started — a placeholder that creates alignment without claiming empirical authority. Second, as a prompt for assumption-surfacing: "here's what we think the user looks like; what are we assuming that we don't actually know?" Third, as a comparison point for validating research-derived personas once fieldwork is complete — where do the AI assumptions diverge from what participants actually said?

Why AI personas can't replace research

A persona is only as good as the behavioral data behind it. AI generates personas from training data — aggregated, generalized, and not specific to your users, your product, or your market moment. The nuance that makes a persona actionable — the specific workaround this user type relies on, the emotional context of the task, the vocabulary they actually use to describe their problem — comes from direct observation and interviews. AI personas are educated guesses with good formatting. Research-derived personas are evidence with appropriate uncertainty acknowledged.

The right framing

I treat AI personas as hypothesis documents, not research artifacts. They live in the "assumptions to test" column, not in the "validated understanding" column. That framing matters for how teams use them. An assumption is expected to be updated; a validated finding is something to design from. Keeping that distinction visible — in how the personas are labeled, presented, and referenced — is what prevents a useful hypothesis from becoming a dangerous substitute.

I've written about whether researchers should use AI for synthesis — the short answer is yes for mechanical tasks, no for interpretation. This post is about what that looks like in practice, stage by stage, from raw transcript to actionable theme.

Stage 1 — Transcript preparation

Before AI touches your data, transcripts need to be clean and de-identified. I use AI for this step: remove personally identifiable information, correct obvious speech-to-text errors, and flag unclear or incomplete passages. This is pure mechanical work with no interpretive risk, and it saves 30–60 minutes per transcript. It also creates a consistent transcript format across sessions, which makes the subsequent stages faster.

Stage 2 — Observation extraction

I give AI a single transcript and ask it to extract discrete observations: specific things the participant said, did, or expressed — not themes, just observations. This produces a structured list that mirrors the manual sticky-note extraction step that synthesis traditionally begins with. I review every extracted observation against the original transcript before using it. AI occasionally conflates two separate moments or loses the emotional register of a quote; the review step catches those errors.

Stage 3 — Candidate theme generation

Only after observations are extracted and verified do I ask AI to suggest candidate themes across the full observation set. I treat these strictly as a starting point for affinity mapping, not as conclusions. AI reliably surfaces obvious clusters, which frees my attention for the non-obvious patterns — the contradictions, the edge cases, the moments that don't fit neatly anywhere. Those outliers are often where the most important findings live.

Stage 4 — Human interpretation

This stage has no AI in it. Deciding which candidate themes are significant, which are artifacts of the sample, which contradict team assumptions, and which should drive recommendations — that is the research work. Every finding in my final deliverable is something I can trace back to specific participants and specific moments in the transcripts.

A discussion guide is one of the most personal documents in a researcher's toolkit. The opening that establishes rapport, the probes that surface what's beneath the first answer, the transitions that feel like conversation rather than interrogation — these reflect years of moderation practice. So what happens when you ask AI to write one? In my experience: good structure, reasonable coverage, and almost no voice. Here's how to use it productively anyway.

What AI produces by default

Ask AI to write a discussion guide for a usability study and it will produce something technically competent: logical topic flow, sensible probe questions, reasonable time estimates. It will also produce questions that are slightly too formal, transitions that sound scripted, and opening sequences that feel like onboarding rather than conversation. The document is usable as raw material. It won't sound like you, and it won't reflect the specific dynamics of the research context it's designed for.

Where the edits are concentrated

AI-generated guides require the heaviest editing in three places. First, the opening and closing — AI writes these as procedural scripts rather than human exchanges; rewrite them entirely in your voice. Second, the follow-up probes — AI generates generic probes ("Can you say more about that?") rather than contextually specific ones; replace them with probes tailored to what you're actually trying to learn. Third, task wording — AI often tells participants what to do rather than presenting a goal to accomplish; rewrite tasks to be technology-agnostic and goal-oriented.

What you can keep

The topic structure and sequencing are usually solid and can be kept largely as-is. The time estimates per section are generally reasonable. The consent and recording language is boilerplate that AI handles well. These aren't the parts that make a guide effective — but they're real time-savers when starting from zero. I've found AI most useful for generating a first-draft structure for an unfamiliar research context, where I don't yet have a template to work from.

One of the most persistent problems in UX research is institutional memory. Findings from last quarter's study are buried in a shared drive. The interview that surfaced that insight about the onboarding flow — which study was that? Which participant? Research teams generate enormous amounts of knowledge and lose most of it between engagements. Marvin is one of the tools I've used to address this problem, and it's changed how I think about the research repository as an active research function rather than a passive archive.

What Marvin does

Marvin is an AI-powered research repository that stores interview recordings, transcripts, and notes, then surfaces relevant insights across studies through semantic search and AI-assisted tagging. The core value proposition is findability: when a stakeholder asks "what do we know about how users experience the checkout flow?", the answer isn't buried in a six-month-old report — it's surfaced through a search that reaches across every study that touched the topic, with direct links back to the source moments in the original recordings.

The repository as a living research function

Most research repositories fail because they require manual tagging and maintenance that researchers don't have bandwidth for after a study closes. Marvin's AI-assisted tagging reduces that burden significantly — insights are tagged automatically based on content, and researchers review and refine rather than starting from scratch. This changes the adoption calculus: a repository that requires two hours of maintenance per study gets abandoned; one that requires twenty minutes gets used. And a repository that gets used consistently compounds its value over time as the knowledge base grows.

What to watch for

AI-assisted tagging is not perfect tagging. Marvin occasionally surfaces insights that are topically adjacent but contextually irrelevant — and users who haven't been trained to evaluate results critically may treat everything that surfaces as applicable. I make a point of training any stakeholder who uses the repository to trace every insight back to its source study and participant context before using it to inform a decision. The repository is a research tool, not a research substitute.

The broader implication

A well-maintained research repository changes the economics of a research function. When teams can quickly surface what they already know, they stop commissioning research to re-answer questions that have already been answered. They also arrive at new research engagements with better-informed hypotheses, which makes subsequent studies more efficient. At its best, Marvin isn't just a filing system — it's organizational research memory made searchable and accessible to everyone who needs it.

A research plan used to take me the better part of a day to write. I'd start with a blank document, work through the research objectives, draft the methodology section, map out the screener criteria, estimate timelines, and write the stakeholder summary. Each section required thinking, but much of the writing was structural scaffolding — boilerplate that looked different each time but followed the same underlying logic. AI has changed that significantly. Not by replacing the thinking, but by eliminating the blank-page problem so I can focus on the decisions that actually require judgment.

What I ask AI to do

I start with a brief: the product area, the decision the research needs to inform, the timeline, and any known constraints. From that, I ask the AI to draft a research plan outline. The output is never complete — it's always a starting point — but it's a starting point with all the sections present, which is most of the battle. The structural scaffolding is done. What's left is the judgment work that actually earns my presence on the project.

Where I still do the work

The AI draft needs significant editing in three areas. First, research questions are almost always too broad and too similar to each other — AI tends toward comprehensive rather than focused. I cut and sharpen. Second, the methodology section is often default-obvious: "conduct user interviews." The reasoning behind why this method rather than another, given this question and this timeline, requires the thinking I bring. Third, screening criteria are generic until I make them specific to this product's actual user population and the hypotheses we're testing.

What this changed about my practice

The speed gain is real but secondary. The more interesting change is that working from a draft rather than a blank page has made my planning process more iterative. I push back on the AI's structure, rearrange sections, add constraints it missed. That active editing process has clarified my thinking about what a research plan actually needs to accomplish — and some of the structural improvements I've made to AI drafts have made their way into my templates permanently.

In my post on whether UX researchers should use AI for synthesis, I argued that AI handles the mechanical parts well and the interpretive parts poorly. This is the practical follow-up: what that distinction actually looks like as a workflow, stage by stage, from raw transcript to actionable theme.

Stage 1 — Transcript cleaning

I run raw transcripts through AI with a specific prompt: identify and correct obvious transcription errors, remove filler words, and flag places where the transcript is unclear. The output is a cleaner document, not an analyzed one. AI is reliable at this stage because the task is purely mechanical — no interpretation required, no judgment calls about significance.

Stage 2 — Observation extraction

I ask the AI to extract discrete observations from the cleaned transcript: specific statements, behaviors, or moments where the participant expressed something notable. I ask it to preserve the participant's own language rather than paraphrase. This gives me a structured set of raw data points to work with — significantly faster than reading through and pulling quotes manually, and without the selection bias that comes from reading linearly.

Stage 3 — Candidate theme grouping

I give the AI the full observation set from all sessions and ask it to propose candidate theme groupings. This is where I start editing heavily. AI groupings are often too broad, too literal, or organized around topic rather than insight. I use them as a starting point — splitting some clusters, merging others, renaming all of them in participant language rather than the abstract labels AI tends to produce.

Stage 4 — Interpretation (human only)

The final step — deciding what the themes mean, what's surprising about them, what they suggest for the design — is entirely human. AI interpretation is competent-sounding but shallow. It doesn't know what the team already believed, what the organizational constraints are, or what would actually change the product roadmap. That context is mine to bring, and no prompt engineering changes that.

One of the most persistent problems in UX research is institutional memory. Studies get completed, reports get filed, and six months later a team makes a decision that directly contradicts a finding nobody remembered existed. Marvin is an AI-powered research repository that addresses this problem at the source — not by producing better reports, but by making existing insights findable when a decision is being made.

What Marvin does

Marvin ingests research artifacts — session recordings, transcripts, reports, highlight reels — and makes them searchable by insight rather than by document. Instead of searching for "Q3 registration research," you search for "what do users think about account setup" and surface relevant observations from across your full research history. The AI layer handles tagging, semantic search, and connections between related findings across studies.

What changed about how I commission research

The most significant impact has been on research scoping. When I can quickly check whether a question has already been studied — and retrieve relevant findings in under a minute — I commission research differently. I stop re-studying things that have been studied well enough. I focus new research specifically on the gaps. That's a meaningful change to how research resources get allocated, and it's one of the harder arguments to make without a tool that makes the existing knowledge visible.

What it can't do

Marvin surfaces what's in your repository. If your past research was narrow, shallow, or focused on the wrong questions, it will surface those limitations faithfully. The tool is only as good as the research you've put into it — and it can't tell you whether an insight is still valid given how much the product has changed since the study was run. That judgment remains human, and it's one of the more consequential calls a researcher makes when recommending whether to re-run a study or rely on existing data.

I spent several years writing code before I wrote a single discussion guide. For most of that time, I thought I was doing the work that mattered — shipping features, closing tickets, watching new functionality move from backlog to production. Then I started watching people actually use what I'd built, and the work I thought mattered started to look different.

What changed

The moment that reoriented my career wasn't dramatic. It was a series of moments, each one a small variation on the same observation: someone was struggling with something I had built, and I didn't understand why. I had made what seemed like the obvious design choice. They weren't finding it obvious. That gap — between my mental model and theirs — became the most interesting problem in the room. More interesting than the implementation. More interesting than the next feature. I wanted to understand it systematically, and research was the discipline that gave me a framework for doing that.

Why watching beats building

Development teaches you to solve for the requirements in front of you. Research teaches you to question whether those are the right requirements. After several years of building, I had become good at solving specified problems. What I hadn't developed was the discipline for investigating whether we were specifying the right problems in the first place. That's a different kind of work — and it turned out to be the kind I wanted to do.

What the developer background gave me

Leaving development didn't mean leaving what I'd learned there. The most durable thing I carried into research was credibility with the engineers receiving my recommendations. I know the difference between a finding that requires a two-day fix and one that requires an architectural rethink. I write recommendations that account for implementation complexity because I've lived that complexity. I can attend a sprint review, read a technical specification, and spot when a design recommendation will create downstream problems nobody has considered yet. These aren't skills most researchers arrive with — they're ones I didn't have to learn from scratch.

What I had to unlearn

Development rewards certainty. You're building toward a specification; the question is whether your implementation satisfies it. Research rewards uncertainty. The best research questions are the ones you don't already know the answer to, and the best findings are often the ones that contradict your priors. Learning to hold ambiguity as a productive state — rather than a problem to be closed as quickly as possible — was the hardest part of the transition. It took longer than I expected and was more valuable than I could have anticipated.

Why I Always Start with a Stakeholder Alignment Session

Accessibility Research Is UX Research: The Case for Inclusive Methods

How to Synthesize 200 Sticky Notes Without Losing Your Mind

Customer Journey Workshops: When Everyone Has a Different Map

5 Things Usability Testing Taught Me About My Own Assumptions

From Findings to Action: Closing the Gap Between Research and Design

Running Remote Research: Lessons from Three Years of Distributed Teams

Writing a Discussion Guide That Works for Screen Reader Users

Research Prioritization Workshops: Getting Teams to Say Not Now

What 500 Hours in Restaurant Kitchens Taught Me About Field Research

48-Hour Turnaround: Rapid Research in High-Stakes B2B Contexts

The 9 of 80 Problem: Adoption Failures Are Often Placement Failures

Four Years Embedded: What Long-Term Research Partnerships Teach You

What I Learned Building Apps with AI in the Research Playground

The UX Intelligence Platform: A Living Research Operations Demo

Should UX Researchers Use AI for Synthesis? An Honest Assessment

Using AI to Build a Research Plan in Half the Time

AI-Generated Personas: Useful Starting Point or Dangerous Shortcut?

AI-Assisted Synthesis: A Practical Workflow for Faster Theming

Writing Interview Scripts with AI: What to Keep, What to Cut

Marvin as a Research Repository: AI-Powered Insight Management

From Code to Curiosity: My Path from Developer to Researcher