---
title: "Inter-rater agreement/reliability"
description: One way to assess how good, how usable, a rating system is is to get more than one person to use the system to do ratings of the same things where what is rated has a broad coverage of the…
url: "https://www.psyctc.org/psyctc/glossary2/inter-rater-agreement-reliability/"
updated: "2026-10-09"
category: All OM book glossary entries
---

# Inter-rater agreement/reliability

One way to assess how good, how usable, a rating system is is to get more than one person to use the system to do ratings of the same things where what is rated has a broad coverage of the possible ratings in the system. Ratings may be undimensional and continuous, e.g. "how much warmth in this clip do you think this carer is expressing about ...", or categorical "into which of the categories would you put this AAI interview? The index of inter-rater agreement you are most likely to see for categorical ratings is [Cohen's kappa](https://www.psyctc.org/psyctc/glossary2/cohens-kappa/), a [correlation](https://www.psyctc.org/psyctc/glossary2/correlation/) coefficient is better for continuous ratings.

#### Details

Inter-rater reliability/agreement is an extremely general way of assessing a rating system. Cohen's kappa, the first really widely recognised and used description of the issues and how to use joint ratings in psychological situations, applied to just two raters rating a set of things but there are extensions and alternatives that can be used for more than two raters.

#### Try also

AAI
[Bias](https://www.psyctc.org/psyctc/glossary2/bias/)Classification
[Cohen's kappa](https://www.psyctc.org/psyctc/glossary2/cohens-kappa/)
[Weighted kappa](https://www.psyctc.org/psyctc/glossary2/weighted-kappa/)
[Interview measures](https://www.psyctc.org/psyctc/glossary2/interview-measures/)
[Nominal/category scaling](https://www.psyctc.org/psyctc/glossary2/categorical-nominal-data-scaling/)
[Ordinal scaling](https://www.psyctc.org/psyctc/glossary2/ordinal-scaling/)
[Rating scales](https://www.psyctc.org/psyctc/glossary2/rating-scales/)
[Reliability](https://www.psyctc.org/psyctc/glossary2/reliability/)
[Stevens' levels of measurement](https://www.psyctc.org/psyctc/glossary2/ratio-scaling/)

#### Chapters

Chapter 3.

#### History

Created 26/1/22, tweaks to map links 9.x.26.