This is mainly about typical multi-item nomothetic self-rating questionnaires or rating scales. Content validity is the second most simple component of “validity” and what I said there was:
The more rigorous extension of face validity: does the measure appear to address all facets of what you want to measure and does it not appear to include items, or parts of items, that seem to be slipping away from the target thing you want to measure, or which might introduce bias. For example some items might introduce “social desirability response bias”: they may ask something, or ask in a way, that makes it likely that some respondents will feel ashamed to answer with their first instinct and will modify their answer to one they feel is more “socially desirable”. This is introducing systematic error or bias and hence compromising validity.
Details #
So the ideal for a high validity measurement tool is that it should me measuring all aspects of what you want it to and it should be as little contaminated by other systemic sources of variance, i.e. of things that might cause people to scores themselves, or be scored by others, higher or lower on the measure but because of something other than the thing you want to measure (that’s bias). So a measure that is high on content validity should have items covering all or the majority of aspects of this latent variable that you want to measure. At the same time it should be as free of items that would introduce bias because responses to them might be strongly affected by something other than the latent variable.
Content validity is largely a qualitative assessment: you want to see that the items in the measure satisfy, as far as possible, those two requirements.
This is much harder than it sounds as you want all the items to be answerable/rateable for everyone you want to be able to use the questionnaire or who might be rated using the rating scale. Particularly for questionnaires this can exclude a lot items that might only make sense for a subset of people even if they do reflect the latent variable. A fairly simple might be “I have found it hard to concentrate at work” in a measure of functioning: this is only going to work for people who have employment. Another challenge is that you might want to cover all possible indicators of a complex latent variable but you also want to keep the measure short enough to be usable.a
Completely excluding bias in the shape of different responding by say gender or by age is hard but if designing a new measure or looking at existing ones do ask yourself if some items have overt or cover gender or age aspects.
Try also #
Chapters #
These issues run through chapters 1 to 5 but are particularly central to chapter 3 in the OMbook.
Online resources #
None likely.
Dates #
First created 8.viii.25.
