About the repository
For more information on the systematic review that this repository is based upon, please refer to the following publications:
Khadjesari Z, Boufkhed S, Vitoratou S, Schatte L, Ziemann A, Daskalopoulou C, Uglik-Marucha E, Sevdalis N, Hull L. Methodological and psychometric quality of quantitative implementation outcome instruments for use in physical healthcare settings: a systematic review.
Khadjesari Z, Vitoratou S, Sevdalis N, Hull L. Implementation outcome assessment instruments used in physical healthcare settings and their measurement properties: a systematic review protocol.
Where rights and permissions allow, you can download instruments according to the implementation outcome(s) you wish to measure. You will also be able to download resources relevant to each instrument (e.g. peer-reviewed publication(s), user manuals and scoring guides).
Psychometric quality
The psychometric quality of each instrument included in this repository has been assessed using the Contemporary Psychometrics Checklist (ConPsyCL).
The Contemporary Psychometrics checklist (ConPsyCL) was developed for the purposes of the systematic review that this repository is based upon, at the Psychometrics and Measurement Lab, Institute of Psychiatry, Psychology and Neuroscience, King’s College London. Based on a literature review of seminal papers in the field of contemporary psychometrics and popularity of methods, the ConPsyCL represents a consolidation of the most up-to-date statistical tools that complement the recommendations included within the COSMIN checklist.
Reliability
(consistency, reproducibility of the measurement)
- Internal consistency
- Test retest (stability)
- Inter-rater (agreement)
Validity
(accuracy of the measurement)
- Content related validity (content and face)
- Criterion related validity (convergent, discriminant and/or discriminative, either concurrent or predictive)
Factor analysis
(dimensionality, structural validity, latent trait measurement)
- Diagnostics (adequacy measures)
- Number of factors justification (Kaiser criterion, Variance explained / Parallel analysis, item loadings)
- Goodness of fit (EFA – CFA)
ConPsyCL scoring
Instruments are scored on three psychometric qualities: 1) Reliability (maximum score of 5); 2) Validity (maximum score of 5); 3) Factor analysis (maximum score of 12). These scored are combined to provide a global score (minimum score of 0 and maximum score of 22, with higher score indicating great psychometric quality).
For more information on ConPsyCL, please contact:
Dr Silia Vitoratou
Email: silia.vitoratou@kcl.ac.uk
| ConPsyCL scoring domains and scoring range | |
|---|---|
| Reliability | 0 – 5 |
| Validity | 0 – 5 |
| Factor analysis | 0 – 12 |
| Penalty | -2 – 0 |
| Maximum global score | 22 |
Methodological quality
The methodological quality of each instrument has been assessed using the COnsensus-based Standards for the selection of health Measurement Instruments (COSMIN) checklist.
The COSMIN checklist is a global measure of methodological quality that distinguishes three measurement instrument quality domains; reliability, validity, and responsiveness. Each domain contain one or more measurement properties, presented below.
Reliability
(the degree to which the measurement is free from measurement error)
- Internal consistency
- Reliability (test-retest, inter-rater, intra-rater)
- Measurement error (test-retest, inter-rater, intra-rater)
Validity
(the degree to which an outcome measure measures the construct it purports to measure)
- Content validity (face validity, structural validity, cross cultural validity)
- Hypotheses testing
- Criterion validity (concurrent validity, predictive validity)
Responsiveness
(the ability of an outcome measure to detect change over time in the construct to be measured)
- Responsiveness
COSMIN scoring
Each measurement property is rated using a 4-point scale: ‘excellent’, ‘good’, ‘fair’ or ‘poor’. The lowest rating of any item for a particular measurement property (i.e. reliability, validity, responsiveness) is selected as the global score for each measurement property. The way in which the global score is calculated has led to some criticism and therefore we suggest that you review all measurement property ratings and not the global domain score in isolation.
For more information on COSMIN please visit the COSMIN website.
| COSMIN scale range |
|---|
| Excellent |
| Good |
| Fair |
| Poor |
| Global domain score = lowest rating |
Usability
The usability of instruments included in this repository has been assessed as the number of items in the instrument and categorised in-line with a previous systematic review assessing the usability of implementation outcome instruments in mental and behavioural health.
Lewis CC, Fischer S, Weiner BJ, Stanick C, Kim M, Martinez RG. Outcomes for implementation science: an enhanced systematic review of instruments using evidence-based rating criteria. Implement Sci. 2015;10:155
Usability (instrument length) |
|
|---|---|
Excellent |
<10 items |
Good |
10-49 items |
Adequate |
50-99 items |
Minimal |
>99 items |