Skip to content
CalibrumDelphi knowledge hub
Calibrum Blog

Delphi consensus, expert panels, and research guides

Delphi consensus is agreement assessed using predefined rules after independent ratings, structured feedback, and an opportunity to reconsider. Explore the consensus process and criteria, expert panels, the modified Delphi method, and Real-Time Delphi in our research guides.

Find an article

Search the Delphi knowledge hub

Search article titles, topics, methods, applications, and summaries. The topic chapters below remain available for browsing.

44 articles across 6 topics.

New on the blog

Latest articles

Explore our latest practical guidance on Delphi studies, questionnaires, and structured group decisions.

NewExternal review

When External Review Changes a Delphi Definition

External feedback, statement versions, and renewed assessment

Plan external review, preserve statement versions, define renewed rating, and report evidence, agreement, and reviewer roles with an auditable decision record.

NewRAM interpretation

Why a RAND/UCLA Item Can Remain Uncertain Without Disagreement

Interpret the median and disagreement result together

Use original rating patterns and a recent research example to distinguish uncertain judgments, polarization, and separate validity, importance, and resource ratings.

4 articles

Delphi foundations

What the Delphi method is, why it works for expert consensus, and how it differs from the surveys and adaptations it is often confused with.

Next: Study design & panels
Guide

What Is the Delphi Method?

Benefits, uses, and why Delphi works so well for expert consensus

Learn how the Delphi method helps researchers and organizations gather expert input, refine opinions over multiple rounds, reduce groupthink, and measure consensus in a structured and transparent way.

Benefits

Why Use the Delphi Method?

Key benefits for expert consensus studies

Understand why Delphi is useful for reducing dominant voices, encouraging independent judgment, supporting structured feedback, measuring consensus, and creating a transparent record of expert input.

Comparison

Delphi vs. Traditional Surveys

When Delphi is a better fit than a standard questionnaire

Learn how Delphi studies differ from traditional surveys and why expert consensus often requires structured feedback, multiple rounds, participant reflection, and clear consensus criteria.

Method design

Modified Delphi Studies

How teams adapt Delphi while preserving credibility

Learn what modified Delphi studies are, how they differ from traditional Delphi, and how teams can adapt the method while preserving transparency, credibility, and clear reporting.

10 articles

Study design & panels

Decisions to make before the first round opens: who sits on the panel, how many experts you need, and which pitfalls weaken a study.

Next: Consensus & analysis
Guide

How to Plan a Delphi Study

Panel selection, rounds, rating scales, feedback, and reporting

A practical guide to planning expert panels, Delphi rounds, rating scales, structured feedback, consensus thresholds, administration, analysis, and reporting before launching a study.

Panel design

How Many Experts Does a Delphi Study Need?

Why panel-size justification matters more than a universal minimum

Plan panel size around the research question, required expertise and stakeholder perspectives, expected attrition, and intended analyses—and report the rationale transparently.

Stakeholder analysis

Can One Large Stakeholder Group Dominate a Delphi Result?

Overall consensus and stakeholder consensus answer different questions

A pooled percentage gives every participant equal weight. See when the largest group can decide the result even if smaller, essential groups disagree—and how to analyze and report by stakeholder group.

Pitfalls

Common Delphi Study Mistakes

Practical issues that can weaken Delphi studies

Learn common Delphi study mistakes and how to avoid problems with unclear objectives, weak panel design, poor item wording, undefined consensus criteria, and weak feedback workflows.

Study design

Two Delphi Design Errors That Can Undermine Consensus

Restricted early input and compound statements can distort the result

Learn why every relevant perspective needs a defined role in content development—and why each survey item should ask panelists to make one interpretable judgment.

Study design

Where Bias Enters a Delphi Study—and How to Reduce It

Practical safeguards for panel selection, framing, feedback, attrition, and analysis

Identify where investigator, panel, statement, feedback, attrition, and reporting bias can influence a Delphi study—and build proportionate safeguards into the protocol and final report.

Checklist

Delphi Study Design and Reporting Checklist

A practical resource for planning and transparent reporting

Review panel composition, stakeholder representation, scales, consensus rules, feedback, attrition, analysis, and transparent reporting decisions before and after your study.

Reporting guide

What Should You Report in a Delphi Study?

A practical record of design, participation, feedback, and decisions

Document the panel, item development, planned sample size, rounds, feedback, consensus rules, attrition, revisions, and results so readers can understand and reproduce the process.

NewExternal review

When External Review Changes a Delphi Definition

External feedback, statement versions, and renewed assessment

Plan external review, preserve statement versions, define renewed rating, and report evidence, agreement, and reviewer roles with an auditable decision record.

11 articles

Consensus & analysis

How to define agreement, judge items on more than one dimension, and report disagreement honestly when it does not resolve.

Next: Real-time, hybrid & live formats
Thresholds

Consensus Thresholds

How to define, interpret, and report agreement clearly

Learn how Delphi study teams define consensus thresholds, interpret agreement and disagreement, compare stakeholder groups, and report consensus criteria clearly.

Consensus criteria

Why a Simple Majority Is Not Necessarily Delphi Consensus

A 51% decision rule can settle a vote without demonstrating strong agreement

Learn why Delphi consensus usually needs more than a bare majority, how thresholds should be chosen before results are reviewed, and what else to examine besides one percentage.

Consensus thresholds

How Consensus Thresholds Work in Small Delphi Panels

Translate percentages into exact votes before the study opens

Compare 75% and 80% rules, check whole-vote boundaries, and report valid item counts, non-ratings, and changes in participation.

Analysis and reporting

Reporting Preferences, Abstentions, and Consensus in Delphi Studies

A leading option and a consensus result answer different questions

Distinguish preferred options from predefined consensus, explain valid-choice denominators, and retain explicit non-ratings and missing answers in the reporting record.

Participation and agreement

Delphi Quorum and Agreement: Two Checks Before Accepting an Item

Item participation and support have separate denominators

Use original worked examples to define quorum, distinguish abstentions from missing answers, apply exact thresholds, and report participation alongside agreement.

Consecutive-round consensus

When Delphi Consensus Must Hold Across Two Rounds

Repeated agreement, item versions, and individual stability

Plan consecutive-round consensus rules, preserve comparable item versions, and use an original matched-panel example to interpret repeated agreement and stability.

NewRAM interpretation

Why a RAND/UCLA Item Can Remain Uncertain Without Disagreement

Interpret the median and disagreement result together

Use original rating patterns and a recent research example to distinguish uncertain judgments, polarization, and separate validity, importance, and resource ratings.

Study design

Delphi Consensus Is Not Enough

Clarity, importance, feasibility, validity, and appropriateness may require separate decisions

Learn when an item should be evaluated on more than one dimension, how to predefine multidimensional acceptance rules, and how to report which criterion caused an item to pass, fail, or require revision.

Disagreement

When Delphi Should Produce Two Answers, Not One

Recognizing stable polarization and context-dependent consensus

Learn when persistent disagreement represents meaningful competing positions—and why a defensible Delphi should preserve them rather than force one artificial conclusion.

Analysis guide

What Changed Between Delphi Rounds?

Consensus, stability, matched ratings, and attrition

Plan comparable round data and distinguish aggregate movement from individual rating changes, accounting for participation and attrition.

Consensus analysis

Can a Delphi Panel Look Like It Agrees While Remaining Divided?

What medians, percentages, and small panels can conceal

Inspect response distributions and minority views, distinguish a protocol classification from the strength of its evidence, and report information limits without inventing a universal sample-size rule.

4 articles

Real-time, hybrid & live formats

Choosing between asynchronous rounds, Real-Time Delphi, and facilitated live sessions—or combining them so each is used where it is strongest.

Next: Applications & fields
Fast feedback

Real-Time Delphi

A faster way to build expert consensus

Learn how Real-Time Delphi allows participants to review feedback and refine responses while a study is still active, helping teams build consensus more flexibly and efficiently.

Conference decisions

How Live Consensus Sessions Help Groups Make Better Decisions

A structured alternative to long, uneven, and difficult meeting discussions

Learn how a Surveylet Live Consensus Session can help conferences, workshops, advisory boards, committees, and leadership groups collect independent input, share anonymized feedback, and document a decision during the meeting.

Live voting guide

How to Run and Report a Live Delphi Voting Session

Keep voters, wording versions, and repeat votes interpretable

Plan eligible voters, item denominators, feedback, reflection, repeat votes, and a session record that distinguishes changed opinions from changed wording or participation.

12 articles

Applications & fields

Delphi at work: clinical consensus and core outcome sets, appropriateness ratings, and expert judgment in fields well beyond healthcare.

Next: Questions, risks and group decisions
Clinical consensus

Delphi in Medical Research

Clinical consensus, core outcome sets, care guidance, and research priorities

See how Delphi studies support medical research, including clinical consensus, core outcome sets, quality indicators, care pathways, public health, and publication-ready reporting.

RAM guide

RAND/UCLA Appropriateness Method: A Practical Overview

Medical applications, ratings, disagreement, and online study setup

Understand the RAM process, distinguish appropriateness from general consensus, and prepare a Surveylet study with a practical setup guide and worked examples.

Medical research

When Clinical Evidence Is Incomplete

Expert judgment with a transparent research record

Plan a medical consensus study around an evidence gap, preserve disagreement, and define useful reporting outputs.

Multidimensional RAM

Effective Does Not Mean Feasible: Multidimensional RAND/UCLA Consensus

Separate expected benefit from realistic implementation

Plan separate effectiveness and feasibility ratings, classify each dimension under the approved protocol, and report paired results and implementation barriers without blending the judgments.

Implementation planning

Turning Delphi Findings into an Implementation Plan

Connect feasibility ratings and comments to a reviewable delivery plan

Link item-level findings to reviewed barriers, proposed actions, owners, dependencies, and evaluation measures while preserving the panel's original decisions.

Audit standards

From Delphi Consensus to Measurable Audit Standards

Preserve agreement as recommendations become measurable definitions

Specify populations, numerators, denominators, wording versions, approval decisions, and field testing when developing Delphi-derived audit standards.

Workflow validation

Does Consensus on a Checklist Validate the Workflow?

Agreement on steps and review of their sequence need separate records

Define what the panel assessed, distinguish approved statements from later sequencing and role decisions, and plan application testing before implementation.

Appropriateness

Using RAND/UCLA for Real-World Cases and What-If Scenarios

Case-based appropriateness ratings under controlled comparison

Appropriateness decisions often depend on context. A case-based RAND/UCLA design lets experts rate a realistic base scenario, then test how controlled changes affect the median, disagreement, and final classification.

Vignette testing

Can Experts Apply Your Delphi Recommendations Consistently?

Use standardized cases to examine the interpretation of agreed guidance

Plan case testing after consensus, collect independent application decisions, and report stakeholder agreement and uncertainty separately from later validation.

Use cases

Delphi Studies Beyond Healthcare

Examples from policy, education, space, technology, and planning

Explore how Delphi and Real-Time Delphi studies can support structured expert consensus in fields beyond medicine, including policy, education, space, technology, forecasting, and strategic planning.

Policy research

Using Delphi to Reveal Policy Trade-Offs—not Manufacture Agreement

Who benefits, who bears costs, and under which assumptions?

Rate policy impacts separately for affected constituencies, keep panelist roles distinct from impact dimensions, and preserve benefits, burdens, uncertainty, and implementation assumptions.

3 articles

Questions, risks and group decisions

Practical examples for writing Delphi questions, identifying risks, and choosing software for structured group decisions.

Planning a Delphi study or live consensus session?

Surveylet helps researchers and organizations manage expert panels, multiple rounds, structured feedback, Real-Time Delphi workflows, and live consensus-oriented decision-making during conferences and meetings.