… for psychotherapy and counselling research
- The personal background: context and history (skip if you just want the tools but may help understand things).
- Pieces thinking aloud about issues and methods
- Utilities of various sorts
Depending on the size of the text, things may be in this page or jump from it.
In my adolescence I was fascinated by “science” and maths, partly I suspect to mark a difference from my historian father and linguist mother. I’ve started rebalancing that with some Open University courses in my early medical training but a love of, and respect for, the rigour of the natural sciences and maths has stayed with me. Most of my research career from about 1983 has been quantitative though I have done some rather amateur qualitative work and believe that many questions of interest in therapy research can only be usefully explored qualitatively.
These pages started way back as way to share some tools I thought weren’t easily available elsewhere. I’m slowly moving things from the old site to individual pages here and reorganising and expanding this area. Looking at what I created over the last twenty years I see that most of it falls into categories around things I learned that felt absolute game changers for me: methods and ways of thinking and handling data that were clearly dramatically better than what they were replacing. These were things I felt a kind of moral duty to support and I was fascinated and appalled at how slowly they seemed to be made available in statistics packages, in papers in psychology and mental health, and in journals’ stipulations. Even when journals said they would require them they often went on publishing papers breaking their own guides for years (I think the British Journal of Psychiatry holds a record probably now in decades for saying it would be expecting papers to use confidence intervals in preference to p values: it’s still not doing that!)
These discoveries for me were probably as follows. The first three were real abrupt changes, not exactly the bolt of lightning on the road to Damascus but things I grasped over months say and which seemed to leave thinking that didn’t use these ideas simply inadequate. The next
- Paradigm shifts for me:
- The advantages of estimation and use of confidence intervals (CIs) in preference to p values and inferential hypothesis testing.
- The “Reliable and clinically significant change” way of complementing group summary data about change with categories that had some rationale to them.
- The discovery of permutation tests as a way to get significance tests based on sample data and not imputing populations and basing the inferential testing on usually completely violated requirements about the sampling.
- Then bootstrapping, related in many ways to permutation tests as a way to get confidence intervals again without sampling/population assumptions.
- More gradual changes:
- A growing grasp of various psychometric theories, with an increasing disquiet about psychometric theory wars and absolutism.
- Some small grasp of, but a huge shift in my respect form, some epistemological position as a foundation for any empirical work.
- A concern about abstraction versus embodiment, relational location and personal responsibility in research (and life). For me this is associated with utter disgust with growing societal and political reductionism, commoditisation and managerialism as part of a “neo-liberal” late 20th and early 21st malignant capitalism.
- Embracing open source software, open standards and, where I can, open access dissemination. (Perhaps this has really been an operationalisation attitudes and aspirations that took me into research.)
- Pretty much renouncing SPSS and, with a bit more regret, SAS and other commercial statistical packages and committing myself to use R (or occasionally, other open source software) for any statistical and psychometric work wherever I can.
Between adding pieces branching off here, and blogging about these issues, I’m hoping this will develop into a useful resource for people who share my view that too much that is published in psychology, mental health and therapy research neglects these issues disastrously.
- What I call “rigorous idiography“, so big as an issue that it’s got its own heading on the site. Biggest area I hope to develop usefully before my brain stops working!
- The problem of subgroup mean differences in correlation, regression and factor analyses.
- Statistical inference and estimation of internal reliability (Cronbach’s alpha). About the need to treat internal reliability as a sample statistic, like any other sample statistic and not reify reliability. Also in the utilities as it links to online calculators.
- Parametric confidence interval calculators: online forms allowing you to input observed summary statistics and perhaps raw data and get the correct confidence intervals (CIs), 95% or other, around the observed sample values. Can be used for your own data or for values without CIs you see in others’ work.
- Statistical inference and estimation of internal reliability (Cronbach’s alpha). Overlaps with the above as it gives CIs for alpha but also p value for the difference between two values. Also in the thinking pieces as it’s a more general page about the need to treat internal reliability as a sample statistic, like any other sample statistic.
- Then there’s information about determining reliable and clinically significant change.
- My work on probabilities for matching scores “derangements”. This is a very simple method of demonstrating the statistical significance of information transfer through idiographic techniques. Mounted 25.v.01, updated 21.xii.03