Delphi Method Overview

What is Delphi? A guide to panels and the process

The Delphi method is a structured process for building consensus among experts or stakeholders through anonymous input, controlled feedback, and repeated rating or revision.

Plan a Delphi Study View References

What is a Delphi panel?

A Delphi panel is the selected group of people who contribute judgments to the study. Depending on the research question, expertise may come from professional training, research, practical experience, or lived experience. A patient, caregiver, practitioner, or service user can therefore contribute a relevant perspective alongside a technical specialist.

Panel selection should follow clear eligibility criteria and the purpose of the study. A Delphi panel is usually purposively recruited; it should not automatically be described as statistically representative of an entire population. See our guides to panel size and stakeholder balance.

What is a Delphi process?

The core sequence is independent judgment, controlled feedback, and reconsideration. Traditional designs organize that sequence into rounds. Real-Time Delphi can make feedback available while the study remains open. In either case, define how ratings will be interpreted and when the process will close.

The Delphi method guide explains how to apply this process in research and decision-making.

A structured way to turn expert judgment into consensus

Many important decisions cannot be answered by a single dataset, one expert, or an informal meeting. Clinical definitions, research priorities, quality indicators, care standards, and core outcome sets often require judgment from a diverse group of experts and stakeholders.

The Delphi method helps organize that judgment. Participants respond to structured questions, often rating the importance, feasibility, agreement, clarity, validity, priority, or appropriateness of each item. When more than one judgment matters, the dimensions should be rated and interpreted separately. After the first round, participants receive controlled feedback, such as group statistics and anonymized comments. They then have an opportunity to reconsider their responses in later rounds or during a real-time Delphi process.

The goal is not to force agreement. The goal is to make agreement, disagreement, uncertainty, and priorities visible in a transparent and repeatable way.

What makes Delphi different from a regular survey?

Anonymity

Participants can provide honest judgments without being influenced by hierarchy, reputation, seniority, or dominant voices.

Iteration

Participants can reconsider their responses after seeing structured feedback from the group.

Controlled Feedback

Feedback is organized through statistics, consensus indicators, and anonymized comments.

Defined Consensus Rules

The study team defines how consensus will be measured for each required rating dimension before interpreting the results.

Transparent Reporting

A well-run Delphi study documents the panel, rounds, response rates, rating scales, feedback, thresholds, item decisions, and stopping rules.

Stakeholder Inclusion

Modern Delphi studies often include clinicians, researchers, patients, caregivers, advocates, and other stakeholders.

Consensus may require more than agreement. An item can be important but unclear, valid but difficult to implement, or appropriate only in certain settings. When several dimensions matter, predefine whether an item must pass every criterion. Read our guide to multidimensional consensus.

Multi-round Delphi and real-time Delphi

Multi-Round Delphi

In a traditional multi-round Delphi study, participants complete a survey round, the study team analyzes the results, feedback is prepared, and a new round is launched.

Core outcome set development
Clinical guidelines and recommendations
Formal research-priority setting
Studies requiring steering-committee review between rounds

Real-Time Delphi

In a real-time Delphi study, participants can often see updated group feedback during the active survey period and revise their responses without waiting for a separate round.

Time-sensitive consensus work
Distributed expert panels
Studies where rapid feedback is valuable
Projects seeking to reduce multiple separate survey rounds

Hybrid approaches are also possible. Some projects combine real-time feedback within a round with steering-committee review, item revision, or consensus meetings between phases.

Common uses of Delphi in medical and health research

Core Outcome Sets

Determine which outcomes should be measured and reported in all studies of a condition, intervention, or population.

Clinical Definitions

Refine definitions and criteria where clinical practice varies or evidence is incomplete.

Practice Recommendations

Develop consensus recommendations, care pathways, or implementation guidance.

Research Priorities

Help funders, networks, and investigators prioritize unanswered research questions.

Quality Indicators

Identify measurable indicators of care quality, performance, or program effectiveness.

Patient and Stakeholder Consensus

Include patients, caregivers, advocates, administrators, and other stakeholders in structured decision-making.

How a Delphi study usually works

Define the questionThe study team clarifies the purpose of the Delphi process and what the final output should be.

Identify participantsThe team defines expert and stakeholder groups, eligibility criteria, recruitment targets, and panel structure.

Develop itemsCandidate outcomes, statements, criteria, recommendations, or questions are developed from literature reviews, interviews, steering-committee input, or prior work.

Round 1Participants rate or comment on the initial items. Depending on the design, they may also suggest new items.

Feedback and revisionThe study team summarizes results and provides feedback. Items may be retained, removed, revised, or carried forward.

Additional rounds or real-time iterationParticipants review group feedback and reconsider ratings until the study’s predefined consensus, stability, or stopping rules are reached.

Consensus meeting or final reviewSome studies include a final meeting to review items, resolve uncertainty, and confirm the final set.

ReportingThe final report documents the methodology, panel characteristics, response rates, criterion-specific results, stakeholder patterns, stopping decisions, and final recommendations.

Why use Surveylet for Delphi?

Generic survey systems are usually designed for one-time data collection. Delphi studies require a more specialized workflow: participant tracking, structured feedback, revision across rounds, consensus indicators, and round-by-round analysis.

Surveylet was built specifically for Delphi and real-time Delphi studies. It helps teams manage the practical and methodological complexity of consensus research while keeping the participant experience clear and focused.

Plan and report the study transparently

A credible Delphi study explains not only what reached consensus, but how the panel was selected, who participated in each round, what feedback was returned, how items changed, and whether important stakeholder groups viewed the issues differently.

Design before launch

Predefine panel criteria, stakeholder groups, rating dimensions, consensus and disagreement rules, feedback, item decisions, and stopping logic.

Monitor representation

Track participation and attrition by round and assess whether the panel that completed the study still reflects the perspectives required by the research question.

Report more than a percentage

Document distributions, disagreement, qualitative reasoning, subgroup patterns, protocol changes, limitations, and unresolved items. Persistent polarization may represent two defensible positions—not a failed Delphi.

There is no universal Delphi panel size. Panel size should be justified in relation to the research question, the expertise and stakeholder perspectives required, expected attrition, and the analyses planned—not selected from a single arbitrary minimum. Read our practical panel-size guide.

Use the Delphi Design & Reporting Checklist

Not sure what kind of Delphi design you need?

Calibrum can help you choose between a multi-round Delphi, real-time Delphi, modified Delphi, hybrid process, or core outcome set workflow.

Discuss Your Delphi Study