RAND/UCLA Appropriateness Method for medical consensus.
Explore the RAND/UCLA Appropriateness Method (RAM), understand its ratings and disagreement rules, and plan how to implement your study in Surveylet.
What is the RAND/UCLA Appropriateness Method?
RAM combines an evidence review with structured expert judgment about a proposed clinical action in a specified patient scenario. It is useful when research evidence leaves practical questions unresolved.
The focus is appropriateness: whether the expected health benefit outweighs the expected harm for the scenario being rated. A research team develops the scenarios and supporting materials; the panel provides judgments through a planned rating and review process. The RAND manual explains the foundational approach.
| Approach | Main question | Analysis planning |
|---|---|---|
| RAND/UCLA appropriateness | Is the proposed action appropriate in this defined clinical scenario? | Prespecify median classification and a disagreement rule. |
| Other Delphi consensus designs | Does the panel agree on importance, priority, feasibility, or another dimension? | Prespecify the criteria suited to that dimension and study purpose. |
| Modified RAM / online RAM | How will the appropriateness process be adapted for this panel and delivery format? | Describe the retained RAM elements and each modification explicitly. |
For a broader introduction, see Delphi in medical research. For questions evaluated on several dimensions, see multidimensional criteria.
Where RAM can fit in medical research
Define the clinical context before asking the panel to judge the action. Appropriateness may change when that context changes.
Diagnostic decisions
Evaluate a proposed test or investigation for a defined presentation, risk profile, and available clinical information.
Treatment indications
Rate a procedure or treatment across scenarios with different symptoms, prior care, contraindications, or disease characteristics.
Care processes and indicators
Assess a precisely defined care action or indicator. Keep its clinical appropriateness separate from feasibility or implementation priority.
A published burn inhalation injury study illustrates a modified RAM process using an international clinical panel, virtual discussion, repeat ratings, and median/disagreement analysis.
For your questionnaire, separate the scenario, proposed action, and rating dimension. For example, an item could ask panelists to judge a diagnostic action for a clearly specified presentation, then use a separate item to assess feasibility. This is a questionnaire-design example, not a clinical recommendation.
For an applied example of separate dimensions, read Effective Does Not Mean Feasible. It explains how to retain each dimension’s median, disagreement result, and classification before applying a documented combined decision rule. Collecting two ratings does not imply an automatic paired classification or quadrant export; confirm custom analysis and reporting requirements before launch.
How ratings become an appropriateness classification
The usual nine-point scale spans inappropriate (1–3), uncertain or equivocal (4–6), and appropriate (7–9). The final item classification combines the median with the selected disagreement rule.
| Panel median | Disagreement | Conventional classification |
|---|---|---|
| 1–3 | Absent | Inappropriate |
| 4–6 | Either | Uncertain |
| 7–9 | Absent | Appropriate |
| Any median | Present | Uncertain |
This table shows the conventional median/disagreement approach used in published RAM research. Specify how fractional medians and insufficient valid ratings will be handled in your protocol.
A high median can conceal disagreement
An item with a median of 8 can still be uncertain if its ratings meet the study’s disagreement rule. Report the disagreement result alongside the median.
“Not sure” is a separate response
A panelist unable to rate an item should use a non-rating option. That is different from giving a numeric uncertain/equivocal score and should be counted separately.
IPR and IPRAS: checking the spread of ratings
One established approach compares the 30th–70th interpercentile range (IPR) with a symmetry-adjusted threshold (IPRAS). A disagreement index can be expressed as IPR divided by IPRAS. The denominator adjusts for asymmetry; it is not a fixed 2.35 for every item. The selected comparison and its boundary convention belong in the analysis plan.
The RAM setup guide explains the components with worked examples. An absence of disagreement does not necessarily meet a separate strict definition of agreement, and neither result proves that ratings are stable between rounds.
Plan the process, as well as the calculation
The review of web-based RAM panels found differences in how online studies implemented and reported the process. Report whether discussion and feedback occurred, how participation changed, and how the delivery differed from the intended design.
Implement your RAM workflow in Surveylet
Surveylet supports the rating and feedback workflow and provides RAM-related statistical outputs. Confirm the required definitions and outputs against your study protocol before launch.
Nine-point rating tasks
Organize clinical scenarios as questions or matrix items, with clearly labeled appropriateness ratings.
Non-ratings and comments
Use a separate “Not sure” option and an “Offer feedback” comment field for each relevant item.
Panel workflows
Manage participant access, invitations, reminders, and planned rating stages.
RAM statistics
Configure available agreement/disagreement definitions and relevant outputs such as IPRAS, asymmetry, and appropriateness.
Controlled feedback
Choose which group summaries and comments panelists see and when that feedback is available.
Data for analysis
Export the relevant ratings and comments and confirm round, item, subgroup, and valid-rating counts.
For alternative or study-specific agreement/disagreement definitions, contact Calibrum to confirm the available implementation and parameters. Read the RAM setup guide for screenshots, definitions, worked examples, and pre-launch checks.
Recommended: Complete Survey Care for RAM setup
Get hands-on help translating your questionnaire and agreed protocol into a tested Surveylet configuration.
Within the purchased hours and agreed scope, Complete Survey Care can cover survey programming, configuration of confirmed RAM settings, non-ratings and comments, pre-launch checks, feedback and round setup, and data assistance.
Your team provides the study content and protocol and retains scientific and clinical decisions. Broader methodology leadership, comprehensive statistical analysis, and a final report can be scoped separately when required.
Common questions about RAM studies
Is RAM the same as any nine-point Delphi survey?
No. RAM focuses on appropriateness and a defined evidence-informed panel process. Other Delphi designs can use nine-point scales for different judgments and different decision rules.
Can we include “Not sure” and written feedback?
Yes. Surveylet can provide a virtual non-rating labeled “Not sure” and an item-level comment field. Test both controls and specify how non-ratings and comments enter the study analysis.
Does an appropriate median guarantee an appropriate item?
No. Under the conventional RAM approach, an item meeting the disagreement rule is uncertain, even when the median lies in the appropriate region.
Can we add feasibility or validity ratings?
Yes, when the protocol defines them as separate dimensions. Explain their analysis and how they affect the final decision; they do not automatically replace appropriateness.
Is Complete Survey Care required?
It is optional and recommended for teams wanting hands-on implementation support. The service hours and scope are agreed for the project.
References and further reading
- Fitch K, et al. The RAND/UCLA Appropriateness Method User’s Manual. RAND; 2001. See Chapter 8 for classification and disagreement definitions.
- Sparks JB, et al. Planning and Reporting Effective Web-Based RAND/UCLA Appropriateness Method Panels: Literature Review and Preliminary Recommendations. Journal of Medical Internet Research. 2022;24(8):e33898. PubMed record.
- Milton-Jones H, et al. An international RAND/UCLA expert panel to determine the optimal diagnosis and management of burn inhalation injury. Critical Care. 2023;27:459. A published example of median ratings and disagreement-index analysis.
Related guidance: RAM case scenarios, Delphi reporting, and RAM setup in Surveylet.
Interpret disagreement and change between rounds
Report the median, response distribution, disagreement rule, and classification together. An uncertain classification can reflect mid-range ratings or disagreement; retain the distinction when explaining the findings.
When comparing ratings across rounds, keep the clinical scenario and scale consistent, account for panel attrition, and document changes to the evidence or wording. Stable ratings do not by themselves establish clinical validity. Subgroup findings require suitable counts and confidentiality safeguards.
Plan comparisons between rounds · Interpret persistent disagreement
A modified RAND/UCLA study may use an explicitly defined percentage-in-band rule. Report that adaptation and its denominator; the label alone does not identify the calculation. A 75% rule with seven valid ratings needs six supporting votes, but that support count does not replace a protocol's median and disagreement requirements. See the small-panel examples and method distinction.
Planning a medical appropriateness study?
Share your questionnaire, intended RAM rules, panel size, planned rounds, and launch date. We can confirm the Surveylet configuration and recommend the implementation support your team needs.