Consensus & statistics · Administrator
Configure the study’s consensus rules
Match question-level measures and thresholds to your protocol, then verify them with known test data.
Screenshots from the original Surveylet guide illustrate the controls below. Layouts and labels may vary with your survey configuration. Select an image or “View full size” to open it in a new tab.
Specify the rule before data collection
Decide what constitutes consensus in, consensus out, and an undetermined result. State the scale, the criterion or combination of criteria, and the denominator used for percentages. Decide how non-ratings, missing answers, and stakeholder groups are treated.
A majority choice, median, or displayed consensus score does not by itself prove the study’s acceptance criterion has been reached. Interpret the status using the rule configured for that item.
Configure and verify the item
- Open the question or matrix-column settings.
- Select the appropriate Consensus Measure.
- Set the Stability Measure where the selected method uses it. Range consensus uses its own percentage-based rules.
- Enter the Stability Threshold required by the protocol. Do not assume a default threshold is appropriate.
- Check scale order, the Likert Scale flag where required, and the virtual “I don’t know” option.
- Enter known test answers, including cases at and around each boundary. Compare the displayed status and exports with the intended criterion.
Keep labels separate from criteria
The survey’s Consensus tab lets you change labels such as Consent, Dissent, and Undetermined. Renaming a status does not change the underlying calculation. In that tab, “Consent” is a consensus label; it is separate from informed consent to participate.
Use a supported built-in method that matches the protocol. If you need a combined or custom rule not represented by the available options, ask Calibrum about configuration and any separately scoped work before launch.
Document what you report
Keep the configured rule and a test example in the study records. State whether a median or IQR is part of the acceptance rule or merely a descriptive statistic. Review the consensus threshold guidance for planning context.
Common questions
Does an IQR above 1 automatically mean no consensus?
Only if your pre-specified rule requires IQR ≤ 1. If the acceptance rule is solely percentage agreement, an IQR is descriptive unless the protocol says otherwise. Configure and report the rule consistently.
Is 75% the right threshold for every study?
No. The study protocol determines the criterion. Surveylet supports configuration; do not substitute a default or a common threshold for your pre-specified rule.