Modified Delphi Studies
What makes a Delphi study “modified,” when this approach is useful, and how teams can adapt the method while preserving transparency and credibility.
What is a modified Delphi study?
A modified Delphi study is an adapted version of the Delphi method. It keeps the core purpose of Delphi — structured expert input, feedback, reconsideration, and consensus-building — but adjusts one or more parts of the process to fit the study objective, timeline, field, or stakeholder needs.
For example, a traditional Delphi study may begin with an open-ended first round where participants generate ideas. A modified Delphi study may instead begin with a predefined list of items developed from a literature review, steering-committee input, interviews, clinical guidelines, prior studies, or other evidence sources.
Modified Delphi approaches are common because real-world research often needs a practical balance between methodological rigor and project constraints.
What usually stays the same?
Even when a Delphi study is modified, the core principles should remain clear. These principles are what separate Delphi from a simple survey or informal expert consultation.
Structured expert input
Participants are selected because of relevant expertise, experience, stakeholder role, or lived experience.
Feedback and reflection
Participants can review summarized group feedback and reconsider their views.
Consensus criteria
The study uses predefined rules to interpret agreement, disagreement, priority, or stability.
Iteration
The process allows refinement through multiple rounds, Real-Time Delphi feedback, or structured review steps.
Transparency
The final report explains how items were developed, rated, revised, retained, or excluded.
Independent judgment
The design should reduce undue influence from hierarchy, dominant voices, or group pressure.
What can be modified?
Different studies use the term “modified Delphi” in different ways. In general, it means that the team has adjusted part of the standard Delphi process to better fit the study.
| Possible modification | What it means | Why teams use it |
|---|---|---|
| Starting with predefined items | The first round uses items from literature, interviews, guidelines, or expert preparation. | Reduces participant burden and grounds the study in existing evidence. |
| Changing the number of rounds | The study may use two rounds, three rounds, or a Real-Time Delphi format. | Fits the timeline, topic complexity, and degree of consensus needed. |
| Adding a steering committee | A small group helps prepare items, review comments, or interpret round results. | Supports quality control and methodological oversight. |
| Including meetings or workshops | A structured discussion may occur before, between, or after survey rounds. | Helps resolve complex items or finalize recommendations. |
| Using stakeholder-specific feedback | Participants may see results by group, such as clinicians, patients, policymakers, or educators. | Highlights differences across stakeholder perspectives. |
| Using Real-Time Delphi | Participants can review updated feedback and revise responses during the active study period. | Supports faster and more flexible consensus-building. |
Why use a modified Delphi approach?
Modified Delphi designs are often used when a fully open-ended, traditional Delphi process would be inefficient, too burdensome, or not well aligned with the study objective.
Existing evidence is available
When a literature review or prior work has already identified candidate items, starting from scratch may not be necessary.
Participants are busy
Clinicians, researchers, executives, policy experts, and other panelists may need a more efficient workflow.
The timeline is limited
Modified designs can reduce delays while still supporting structured expert feedback.
The study needs stakeholder input
Different stakeholder groups may require tailored instructions, feedback, or analysis.
The topic is complex
A steering committee or structured review step can help refine items before they are shown to the full panel.
The output must be practical
Modified Delphi can help translate expert input into usable recommendations, priorities, standards, or criteria.
Traditional Delphi vs. modified Delphi
Traditional and modified Delphi studies share the same general goal: structured expert consensus. The difference is usually how the study begins, how rounds are managed, and how much the process is adapted to the project.
| Traditional Delphi | Modified Delphi |
|---|---|
| Often begins with open-ended idea generation. | May begin with predefined items from literature, interviews, prior work, or steering-committee input. |
| Usually follows a more standardized multi-round structure. | May adapt the number of rounds, feedback format, item-generation process, or consensus workflow. |
| May rely mainly on anonymous survey rounds. | May include structured meetings, workshops, panels, or steering-committee review. |
| Useful when the topic is less defined and the panel needs to generate items. | Useful when the topic is already partly defined and the panel needs to rate, refine, or prioritize items. |
Common modified Delphi workflows
Modified Delphi studies can be designed in many ways. The right workflow depends on the study question, stakeholder groups, timeline, and level of consensus needed.
Modified Delphi in medical research
Modified Delphi studies are common in medical and clinical research because many projects begin with existing evidence, guidelines, prior studies, or expert-developed item lists. The panel is then asked to rate, refine, or prioritize those items.
Core outcome sets
Candidate outcomes may come from literature reviews, interviews, registries, or prior studies before panel rating.
Clinical recommendations
Statements may be drafted by a steering committee and refined through expert-panel feedback.
Appropriateness criteria
Clinical scenarios may be pre-specified and then rated for appropriateness, importance, or feasibility.
Modified Delphi beyond healthcare
Modified Delphi studies are also useful in policy, education, technology, forecasting, space research, workforce planning, and organizational decision-making. These fields often begin with existing frameworks, candidate criteria, scenario lists, or planning options that need expert review.
Policy and planning
Rate and prioritize policy options, implementation barriers, risk factors, or decision criteria.
Education
Refine competency frameworks, curriculum priorities, standards, and assessment criteria.
Technology and forecasting
Evaluate trends, adoption barriers, future scenarios, risks, and strategic priorities.
Design decisions for a modified Delphi study
Modified Delphi studies need clear documentation because the modifications affect how readers interpret the results. The team should explain what was changed, why it was changed, and how the modified process still supports credible consensus-building.
Item source
Where did the initial items come from: literature, interviews, guidelines, prior studies, or expert input?
Panel selection
Who was invited, why were they appropriate, and what stakeholder groups were represented?
Round structure
How many rounds were planned, and what determined whether another round was needed?
Feedback format
What summary statistics, comments, prior responses, or stakeholder comparisons were shown?
Consensus criteria
What thresholds determined whether an item reached consensus, disagreement, or uncertainty?
Finalization process
How were final items accepted, excluded, revised, ranked, or prepared for publication or implementation?
Modified Delphi and consensus thresholds
Consensus thresholds are especially important in modified Delphi studies because the study may include adapted workflows, predefined items, or stakeholder-specific criteria. Clear thresholds help make the modified design transparent and defensible.
Inclusion rules
What level of agreement, rating, ranking, or support is needed for an item to be accepted?
Exclusion rules
What criteria will remove items from later rounds or final recommendations?
Uncertain items
What happens to items that do not clearly meet inclusion or exclusion thresholds?
Common mistakes in modified Delphi studies
Modified Delphi designs can be very useful, but they can also be misunderstood. The main risk is that the study becomes “modified” in a way that weakens transparency or makes the process hard to interpret.
Not explaining the modification
The report says “modified Delphi” but does not describe what was modified or why.
Weak item-development process
Initial items are presented without explaining how they were generated or reviewed.
Undefined consensus rules
The study does not define how items will be accepted, rejected, or classified.
Too much steering-committee control
The panel’s role becomes unclear if decisions are made outside the Delphi process without explanation.
Weak feedback design
Participants do not receive enough information to reconsider their responses meaningfully.
Poor reporting
The final report does not show how ratings changed, how items were revised, or why decisions were made.
How to keep a modified Delphi study credible
The goal is not to follow a rigid template. The goal is to make thoughtful design choices and document them clearly. A modified Delphi study can be credible when the modifications are intentional, justified, and consistently applied.
Before launch
- Define the study objective and final output.
- Document how initial items were generated.
- Define the expert or stakeholder panel.
- Choose rating scales that match the study purpose.
- Set consensus thresholds before analysis.
- Plan what feedback participants will receive.
During and after the study
- Apply item-retention rules consistently.
- Document item revisions and removals.
- Track response rates across rounds.
- Report consensus and disagreement clearly.
- Explain any steering-committee decisions.
- Prepare analysis-ready exports and reporting tables.
Modified Delphi vs. Real-Time Delphi
Modified Delphi and Real-Time Delphi are not the same thing, but they can overlap. A modified Delphi study describes an adapted Delphi design. Real-Time Delphi describes a workflow where participants may review updated feedback and revise responses while the study is active.
| Modified Delphi | Real-Time Delphi |
|---|---|
| Refers to adaptations in the study design. | Refers to dynamic feedback and response revision during the active study period. |
| May start with predefined items, steering-committee input, or adjusted rounds. | May reduce delays by allowing feedback and revision without waiting for separate rounds. |
| Can be traditional, online, hybrid, or real-time. | Can be used as part of a modified Delphi design. |
How Surveylet supports modified Delphi studies
Surveylet supports Delphi and Real-Time Delphi studies, including adapted workflows for modified Delphi designs. Study teams can use Surveylet for expert panels, structured questionnaires, feedback, response revision, multi-round workflows, and consensus-oriented exports.
Calibrum can also help teams think through the design choices that matter most: item development, rating scales, consensus thresholds, feedback, round structure, stakeholder groups, and reporting.
Flexible workflows
Support traditional Delphi, modified Delphi, and Real-Time Delphi study designs.
Structured feedback
Provide participants with group results, comments, consensus status, or other feedback needed for reconsideration.
Reporting support
Export results for consensus analysis, steering committees, manuscripts, and supplemental materials.
Planning a modified Delphi study?
Surveylet helps researchers and organizations manage Delphi, modified Delphi, and Real-Time Delphi studies online, including expert panels, structured feedback, multiple rounds, response tracking, and consensus-oriented reporting.