Assessing the validity of idiographic data by the method of derangements

Page started by Chris Evans 28.v.01, last updated 26.x.08
This page describes a way of testing the validity of information transfer through idiographic assessments. The key feature of the method is that if a judge can match four or more idiographic descriptors to the people or objects they describe, then the chances of this having arisen by chance alone are p<.05.

A paper showing the maths of this is:
Evans, C., Hughes, J. & Houston, J. (2002) Significance testing the validity of ideographic methods: a little derangement goes a long way. British Journal of Mathematical and Statistical Psychology, 55:385-390.

I've mounted a very quick introduction to the method in PDF. That's a variant of a powerpoint poster given at the Chicago International meeting of the Society for Psychotherapy Research in 2000.

In the unlikely event that you can't read that there's a horrid HTML version as spat out of M$ Powerpoint in very nasty html. Assuming you're a M$ user, then a more recent version of Powerpoint spits it out more usably as an mht file this works fine in M$ Internet Explorer but which doesn't work for me in Firefox and I don't know about other browers.

Table 1 from that paper tabulates the probabilities of various scores for various n (Hm, horrid HTML, I must find a better way to get from Word to HTML than Word.) If you observe a score of, say 6 correct matches across 8 sets of data, you can read that the probability of hitting exactly six by chance alone was .0007, and that the crucial probability of hitting six or more, is also .0007 to any sensible accuracy (since the probability of getting eight of eight by chance alone is a .00002 which adds little to .0007). Incidentally, there is no possibility of scoring 7 here since it's always impossible to match n-1 from n: if you've got the first n-2 right, you either get the last two the right way round or the wrong way around.

In addition I've mounted a few tools that can give you these probabilities for any arbitrary n:

Work on the statistical power of the method and on some extensions of the method is in progress.

Chris Evans,
Rampton Hospital, Retford, Notts. DN22 0PD Britain
by telephone at [+44|0] 1777 247242
and 'fax at: [+44|0] 1777 247213