Effective does not mean feasible: multidimensional RAND/UCLA consensus
A panel can agree that an action would help while remaining uncertain that it can be delivered. Collect those judgments separately so a promising recommendation does not become an unsupported implementation claim.
A recent study makes the distinction concrete
In an October 2026 PLOS Biology study on antibody validation, 32 participants evaluated 33 proposed actions using separate nine-point effectiveness and feasibility ratings. The modified Delphi applied RAND/UCLA analysis across two rounds: 15 actions were judged both effective and feasible, while another 15 were effective but of uncertain feasibility. Read the study.
The practical lesson is broader than antibody research: agreement that an intervention has value does not establish that the resources, authority, infrastructure, or time needed to implement it are available. The planning suggestions below are Calibrum’s interpretation of that distinction, not a new RAM standard.
Ask two questions, not one compound question
“Is this effective and feasible?” gives a panelist no clear way to endorse the expected benefit while questioning delivery. Keep one stable action identifier, then collect a separate response for each dimension.
- Effectiveness: How much would this action contribute to the specified objective?
- Feasibility: How realistically could the specified actors implement it in the stated setting and timeframe?
Illustrative wording—not a validated questionnaire
Action: introduce a standardized review before a recommendation is implemented.
Rating A: expected effectiveness in improving the stated outcome, from 1 (very low) to 9 (very high).
Rating B: feasibility of implementation within two years under the stated resource assumptions, from 1 (very low) to 9 (very high).
Define the population, setting, responsible actor, resources, and time horizon. Otherwise, two panelists can give different feasibility ratings while imagining different conditions. Pilot the scale anchors as well as the item wording.
Classify each dimension before combining the results
A high median does not establish agreement if the chosen disagreement rule is met. Analyze each dimension under its own predefined rule, then apply the protocol’s combination rule. Do not average effectiveness and feasibility into one score unless a justified scoring model explicitly requires it.
| Effectiveness result | Feasibility result | Possible action |
|---|---|---|
| High, without disagreement | High, without disagreement | Consider for implementation, subject to evidence and governance review. |
| High, without disagreement | Uncertain or contested | Retain as promising; investigate barriers or revise the implementation conditions. |
| Uncertain or low | High, without disagreement | Do not treat ease of delivery as evidence of effectiveness. |
| Uncertain, low, or disputed | Uncertain, low, or disputed | Review the evidence, clarify the action, defer it, or report it as unresolved. |
State the rating bands, valid-response denominator, missing-response policy, median boundary handling, disagreement calculation, and stopping rule. Adaptations involving effectiveness or feasibility should be identified as modified applications of RAM; they are not automatically equivalent to a clinical appropriateness judgment.
Use comments to explain the implementation gap
For items with strong expected benefit but uncertain feasibility, ask what would make implementation possible. Responses might point to staffing, funding, training, regulatory approval, coordination, or a longer timeline.
Distinguish participant explanations from the research team’s interpretation. If comments are formally coded, report the coding process and retain an appropriately anonymized audit trail. If the team instead summarizes the material informally, describe it as an interpretive summary rather than a formal qualitative analysis.
Show the paired profile, not only a final pass/fail label
A publication-ready item table should retain both medians, response counts, distributions or spread measures, disagreement results, dimension-specific classifications, and the final item action. Report the round from which each result comes, especially when accepted items are not re-rated.
An effectiveness-versus-feasibility plot can complement that table. Show the predefined boundaries and mark disagreement explicitly; two identical medians can conceal different distributions. Use accessible labels and a companion table so readers do not have to infer the result from color alone.
Separate “not judged feasible under these conditions” from “ineffective.” They imply different next steps and should not be collapsed into one rejection category. Stakeholder comparisons may help explain differences, but small subgroup counts need to remain visible.
Configure the study around the approved protocol
Surveylet supports separate structured rating questions, nine-point scales, RAM-related statistical outputs, controlled feedback, stakeholder grouping, and exports. Keep effectiveness and feasibility items paired through consistent identifiers and instructions, then verify the configured calculations against worked examples before launch.
See the RAM setup guide for the distinction between RAM-related outputs and general consensus indicators. The research team must decide how dimension-specific results translate into final recommendations.
Calibrum’s Complete Survey Care can help configure your approved questionnaire and workflow. Discuss custom interpretation and figures separately through the Delphi Analysis & Final Report service.
Sources and methodological boundaries
- Blades K, et al. Actionable solutions to address antibody validation failures. PLOS Biology; October 6, 2026.
- Fitch K, et al. The RAND/UCLA Appropriateness Method User’s Manual. RAND; 2001.
The recent study is an example of separate dimensions in practice, not evidence that every Delphi requires this design or that consensus proves real-world effectiveness. Predefine and disclose adaptations rather than copying a published study’s rules without checking their fit.
Keep expected benefit and implementation readiness distinct
Bring your approved dimensions, decision rules, and reporting needs. We can help translate them into a clear Surveylet workflow.