Skip to main content

Table 3 Criteria for good measurement properties

From: Patient-reported outcome measures for acute rhinosinusitis in adults and children: a systematic review of the quality of existing instruments

Measurement property

Rating

Criteria

Structural validity

+

CTT

CFA: CFI or comparable measure > 0.95 OR RMSEA < 0.06 OR SRMR

< 0.08a

IRT/Rasch

No violation of unidimensionalityb: CFI or TLI or comparable measure

> 0.95 OR RMSEA < 0.06 OR SRMR < 0.08

AND

no violation of local independence: residual correlations among the items after controlling for the dominant factor < 0.20 OR Q3’s < 0.37

AND

no violation of monotonicity: adequate looking graphs OR item scalability > 0.30

AND

adequate model fit

IRT: χ2 > 0.001

Rasch: infit and outfit mean squares ≥ 0.5 and ≤ 1.5 OR Z-standardized values > -2 and < 2

 

?

CTT: not all information for ‘+’ reported

IRT/Rasch: model fit not reported

 

-

Criteria for ‘+’ not met

Internal consistency

+

At least low evidencec for sufficient structural validityd” AND Cronbach’s alpha(s) ≥ 0.70 for each unidimensional scale or subscalee

 

?

Criteria for “At least low evidencec for sufficient structural validityd” not met

 

-

At least low evidencec for sufficient structural validityd and Cronbach’s alpha(s) < 0.70 for each unidimensional scale or subscalee

Reliability

+

ICC or weighted Kappa ≥ 0.70

 

?

ICC or weighted Kappa not reported

 

-

ICC or weighted Kappa < 0.70

Measurement error

+

SDC or LoA < MICd

 

?

MIC not defined

 

-

SDC or LoA > MIC

Hypotheses testing for construct validity

+

The result is in accordance with the hypothesisf

 

?

No hypothesis defined (by the review team)

 

-

The result is not in accordance with the hypothesisf

Cross-cultural validity/measurement invariance

+

No important differences found between group factors (such as age, gender, language) in multiple group factor analysis OR no important DIF for group factors (McFadden’s R2 < 0.02)

 

?

No multiple group factor analysis OR DIF analysis performed

 

-

Important differences between group factors OR DIF was found

Criterion validity

+

Correlation with gold standard ≥ 0.70 OR AUC ≥ 0.70

 

?

Not all information for ‘+’ reported

 

-

Correlation with gold standard < 0.70 OR AUC < 0.70

Responsiveness

+

The result is in accordance with the hypothesisf OR AUC ≥ 0.70

 

?

No hypothesis defined (by the review team)

 

-

The result is not in accordance with the hypothesisf OR AUC < 0.70

  1. The criteria are based on Terwee et al. and Prinsen et al.
  2. AUC area under the curve, CFA confirmatory factor analysis, CFI comparative fit index, CTT classical test theory, DIF differential item functioning, ICC intraclass correlation coefficient, IRT Item response theory, LoA limits of agreement, MIC minimal important change, RMSEA root mean square error of approximation, SEM standard error of measurement, SDC smallest detectable change, SRMR standardized root mean residuals, TLI Tucker-Lewis index
  3. “+” = sufficient, “-“ = insufficient, “?” = indeterminate
  4. aTo rate the quality of the summary score, the factor structure should be equal across studies
  5. bUnidimensionality refers to a factor analysis per subscale, while structural validity refers to a factor analysis of a (multidimensional) patient-reported outcome measure
  6. cAs defined by grading the evidence according to the GRADE approach
  7. dThis evidence may come from different studies
  8. eThe criteria ‘Cronbach’s alpha < 0.95’ was deleted, as this is relevant in the development phase of a PROM and not when evaluating an existing PROM
  9. fThe results of all studies should be taken together and it should then be decided if 75% of the results are in accordance with the hypotheses
Back to article page

Health and Quality of Life Outcomes

ISSN: 1477-7525

Contact us