When Delphi Consensus Must Hold Across Two Rounds

Define the rating scale, supporting categories, denominator, agreement threshold, participation requirement, number of consecutive qualifying rounds, and stopping rule before reviewing results. Say whether the same panelists must respond in both rounds and how incomplete participation will be handled.

Also define the effect of wording changes. A correction to punctuation and a change to the population or recommended action have different implications for comparability. Preserve every version and record the methodological decision about whether the earlier result can contribute to the consecutive-round criterion.

Keep three questions separate: did the item meet the agreement rule in each round; did the aggregate distribution change; and did individual panelists change their ratings? Each requires its own data and definition. Repeated high support alone cannot answer all three.

Where an item falls below the required threshold in a later round, apply the planned continuation or unresolved-item rule. Do not combine nonconsecutive successful rounds and describe the result as consecutive agreement.

Give every item a stable identifier and every material wording version its own version label. Link a revised or merged statement to the items that preceded it. Record which rounds contained the version, which results were comparable, and why the item moved forward or left the questionnaire.

New items need a clear route through the remaining rounds. If the planned last round cannot provide enough assessments to satisfy the rule, report that limitation and the decision taken. Keep resource-driven closure distinct from evidence that an unresolved item would never achieve consensus.

An item-flow record should distinguish accepted under the rule, retained for another assessment, revised, merged, removed for a stated reason, and unresolved at closure. Use labels that explain the event instead of assigning the same failure label to every item that leaves the questionnaire.

Preserve the feedback shown between rounds, including the statistics, source comments, and the panelist’s previous rating where available. Document how revisions followed the feedback and who approved them.

Report each qualifying round with its wording version, eligible panel, valid ratings, supporting count, non-ratings, missing answers, and classification. If the study requires agreement within stakeholder groups, show the rule and result for every required group.

Then report aggregate change and matched individual change under their own definitions. Explain the consequences of attrition, replacement, item removal, or revised wording for those comparisons. Describe exploratory checks as exploratory.

For Real-Time Delphi, specify the observation times or snapshots and what opportunity participants had to consider feedback and revise their ratings. Two stored views of the same responses cannot by themselves establish two distinct reconsideration opportunities. If repeated snapshots are used for a temporal criterion, define their purpose and interpretation in advance.

Surveylet supports round management and snapshots, with response data available for analysis. Agree the version history and any consecutive-round classification or matched-rating analysis as part of the study’s reporting specification. Do not assume that the general consensus indicator establishes every additional protocol condition.

See comparing Delphi rounds for broader guidance on stability and attrition, and quorum and agreement for separate participation checks.