In quantitative methods this means pretty much what you’d expect it to. There is just one cautionary aspect when it comes to re-weighting survey data to make it more representative of the population from which it was taken. More next!
Details #
Weighted kappa #
So one simple example of weighting is already in the glossary: “weighted kappa”. Like simple kappa (or “Cohen’s kappa”) this is an index of agreement/disagreement typically used in our field for comparing data from different raters to see how well they agree (or not). We might think that the disagreement where one rater codes a fantasy as “lethal violence” and the other codes it as “rude” is more of a disagreement than between “lethal violence” and “non-lethal violence”. Weighted kappa allows such weighting. (Yes, I know it’s a silly example. Do contact me to suggest a better one or even an open source published example of using weighted kappa!)
Item response scoring/weighting #
Another sort of weighting is differential weighting of responses to items on a questionnaire. We commonly score responses as sequential integers so for item 1 of the CORE-OM we have these scores.
CORE-OM item 1 #
| Response | Score |
|---|---|
| Not at all | 0 |
| Only occasionally | 1 |
| Sometimes | 2 |
| Often | 3 |
| Most or all the time | 4 |
However, some questionnaires use different weighting/scoring of responses. For example, for the GHQ-28 (28 item General Health Questionnaire) the scoring is this.
GHQ-28 item scoring #
| Response | Score |
|---|---|
| Better than usual | 0 |
| Same as usual | 0 |
| Worse than usual | 1 |
| Much worse than usual | 1 |
You may also see responses weighted or scored using weights determined by IRT (Item Response Theory) where the scores are generally non-integer values.
Weighting of grouped data #
A much more general and simple example is of computing weighted statistics where you have grouped data from 258 people which looks like this:
Grouped scores #
| Score | n |
|---|---|
| 1 | 15 |
| 2 | 33 |
| 3 | 45 |
| 4 | 90 |
| 5 | 75 |
Well we can see that the mean score across the 258 people isn’t 3 (the mean of 1, 2, 3, 4 and 5)! To get the correct mean we use the weighted scores for each row: the score multiplied by the number of people who used that score, then we get the mean by dividing that total by 258, the number of people involved. So this:
Grouped scores2 #
| Score | n | Weighted score |
|---|---|---|
| 1 | 15 | 15 |
| 2 | 33 | 66 |
| 3 | 45 | 135 |
| 4 | 90 | 360 |
| 5 | 75 | 375 |
| Total | 258 | 951 |
| Mean | 3.69 |
Epidemiological/survey weighting #
The final example of weighting that I can think of at this moment is epidemiological weighting. This is commonly used for survey data by re-weighting the data from a survey to make the membership more like that of a known population distribution. Say you collected data about satisfaction with the service (rated 1 to 10 say) from voluntary participants. However, when looked at the participation rates compared with the numbers of attendees broken down by the categories the clients ticked in the clinic’s routine data form, you might find that there were clear and statistically significant differences in participation rates say by gender with women participating more than men and men more than those who ticked the “other” category. (Assuming the routine clinic form and the satisfaction survey used the same three-level categories.) Then you could re-weight the data you have from the participants to make the balance of the numbers across those gender categories match the proportions across all the clinic attenders. Sometimes this may change the generalised mean from the one you have from the participants quite considerably and, assuming that those who did participate in each gender group are representative of all attendees in that group then the new mean should be more representative of overall satisfaction. The catch here is that you can’t just weight by the numbers as in simple weighting because you will have variance in satisfaction ratings within the gender groups so reweighting has to look at that as well as the raw numbers in each group. There are some assumptions you are making when re-weighting like this and there is software that will handle the data sensibly but it’s not just a simple spreadsheet process unlike simple weighting or the weighting in weighted kappa.
Try also #
- CORE system
- General Health Questionnaires
- Variance: introduction
- Variance: computation and bias
- Weighted kappa
Chapters #
Not covered in the OMbook.
Online resources #
Not currently. I’m toying with creating a trivial shiny app to give you weighted means and SDs from grouped data tables!
Dates #
First created 29.iii.26.
