Consensus & statistics · Administrator
Calculate consensus and stability for numeric answers
Follow a worked example using the mean, population standard deviation, and the permitted numeric range.
Worked calculation
Surveylet offers different statistical measures for Consensus and Group Stability calculation, which settings are configurable for survey questions individually. Depending on the type of the question, different options are available for us to choose from. See Consensus Measurement for more information.
This example is for the Numeric Entry question type when using Mean for Consensus measurement, and Coefficient of Variation over Range (CVR) for Group Stability measurement.
When a Numeric Entry question's consensus measure is set to use Mean, and its group stability measure is set to use CV, Surveylet will automatically use this CV calculation in the background.
In this example below, we will calculate the consensus score, group stability and consensus status.
For the sample question, we ask the expected temperature for a projection requesting a numerical value from -20F to 120F.
We have 9 panelists who submitted a response to this question. Here are their answers:
|
user name
|
user score
(Farenheit)
|
|---|---|
|
User A
|
-1
|
|
User B
|
[I don't know]
|
|
User C
|
41
|
|
User D
|
-8
|
|
User E
|
11
|
|
User F
|
41
|
|
User G
|
99
|
|
User H
|
[I don't know]
|
|
User I
|
110
|
Let's calculate stability and consensus from the above data set.
There are 9 participants who answered the question.
2 of the 9 participants selected the [I don't know] option, therefore their answers are removed from calculation.
7 of 9 participants provided answers to the question.
Number of users: 7
Total sum of all user scores: 293
Average (Arithmetic Mean) = 293 / 7 = 41.85714286
Standard Deviation Population (StDev.P) = 43.39989655
Minimum = -20
Maximum = 120
Range = Maximum - Minimum = 120 - (-20) = 140
CVR = StDev.P / Range = 43.39989655 / 140 = 0.3100 = 31.00%
Group Stability = 100% - (2 * CVR) = 100% - (2 * 31.00%) = 100% - 62.00% = 38.00%
Group Stability is rounded to the nearest whole number, when displayed in the stability gauge.
The consensus score is calculated from the Average (Arithmetic Mean), which is 41.85714286 for this question (as calculated above).
For this example, the Consensus Threshold value which is set to 50%, meaning that the group stability value must be greater than 50 for the group consensus to be established.
Now we need to calculate the status of consensus, which is based on the Consensus Threshold value which is 50%. This means that we need over 50% group stability for the group-decision for this question to be flagged as Agreement, otherwise it is flagged as Disagreement. Since the group stability is 38.00% which is lower than the required 50%, the group did not reach agreement.
Use the permitted input range
The denominator is the configured maximum minus the configured minimum, not the difference between the highest and lowest observed answers. Here it is 120 − (−20) = 140. The two “I don’t know” responses are excluded; the mean and standard deviation use seven valid numeric answers.