What Changed Between Delphi Rounds?
Plan inter-round analysis that separates consensus, stability, participant change, and attrition.
Consensus and stability answer different questions
Consensus describes agreement within a round under a predefined rule. Stability describes how judgments change between rounds. A panel can remain stably divided, or cross a consensus threshold while individual ratings still move. Neither alone demonstrates clinical validity.
Compare the same item under comparable conditions
Match stable item identifiers, wording versions, scales, dimensions, and anonymous panelist identifiers. Substantially revised statements should be identified as revised items rather than treated as directly comparable. Preserve matrix row and column identifiers and distinguish skipped items from explicit non-ratings.
Use two complementary views
All responders in each round
Report valid counts, median, IQR, distribution, and agreement percentage for each round. Show percentage-point change and consensus-status transitions. These results reflect both rating changes and panel composition.
The matched panel
Compare the same panelists on the same item across two rounds. Report the paired count, unchanged ratings, upward and downward movement, and absolute rating changes. Aggregate stability can conceal offsetting individual shifts.
Participation and item history
Show retained participants, attrition, non-ratings, revised items, new items, and items removed under the protocol. Compare retained participants’ earlier ratings with those of dropouts descriptively where appropriate.
Define stopping rules before reviewing results
Specify the metrics, acceptable changes, minimum valid and paired counts, and maximum rounds. There is no universal stability threshold suitable for every Delphi design. If conditions are not met, report insufficient evidence of stability rather than equating missing data with no change.
An illustrative reporting record
For each item and dimension, retain round pair, wording status, valid counts, paired count, median and IQR by round, agreement percentage by round, percentage-point difference, classification transition, paired movement, and the protocol stopping decision. Use consistent denominators and label descriptive findings clearly.
Plan round comparisons with Surveylet
Surveylet supports snapshots and exports for preserving round results. A snapshot captures a study state; it is separate from the minimum-answer threshold controlling when analytics are shown. Define the round pairs, comparable items, participant matching, and stability measures required by your study protocol.
Define the study and reporting scope
Share your research question, planned rounds, available data, and intended outputs.