--- title: Inferential testing, “tests” description: "This starts with us wanting to know if the relationship between two (or more variables) seem to be systematic. This approach turns on testing whether the data from your sample might easily have been seen if there were *no* systematic…" url: "https://www.psyctc.org/psyctc/glossary2/inferential-testing-tests/" updated: "2026-11-04" category: All OM book glossary entries --- # Inferential testing, “tests” This starts with us wanting to know if the relationship between two (or more variables) seem to be systematic. This approach turns on testing whether the data from your sample might easily have been seen if there were *no* systematic relationship in the population from which you took the sample, given the size(s) of your dataset(s). #### Details To take a very simple example you might want to know about gender differences in baseline CORE-OM scores of clients entering a service. Your inferential test is "How likely is it that I would have seen a difference in mean scores between the gender groups as large as I did, or larger, if in the population of people coming for therapy there is no mean difference in scores by gender?" The method is based on a null model or null hypothesis, here: "There is no difference in mean scores in the population" juxtaposed against an alternative ("alternate" in the American) hypothesis: "there is some non-zero difference in the population". Given some other assumptions about the population and about the data being a random sample from the population then you have a mathematical model, really a simulation of the research process, that allows a statistician to say how unlikely it is that you would have seen differences by gender (I'm being careful not to presume binary gender) as big or bigger than you did on that model. By convention (usually) a probability lower than .05 ("p < .05"), i.e. lower than one in twenty is dubbed "statistically significant" and one above that dubbed "non-significant", "NS", "p > .05". Despite varying waves of challenges, this has been the dominant statistical approach to psychological and MH data since at least the 1940s. However, despite most of the concerns having been around at least that long, the dominance of the approach is waning though sadly not yet replaced by thoughtful data analysis. (OK, that's not the name of a new statistical paradigm but perhaps it should be!) Sadly, some of the current critique of inferential testing is often rather unthoughtful. Used appropriately and understood for what it is, the simple inferential test still has a lot going for it for particular questions, gender differences are not such a bad example. However, good graphical descriptions of the observed data (histograms, boxplots, violinplots, ecdfs) and confidence intervals around observed means will complement and enrich the simple inferential test even for as simple a question as this. Inferential tests are at the heart of the [Null hypothesis significance testing (NHST) paradigm](https://www.psyctc.org/psyctc/glossary2/null-hypothesis-significance-testing-nhst-paradigm/). #### Try also Hm, huge list but all of these can be involved in inferential tests, the problems with that paradigm, and the alternatives. [Analysis of variance (ANOVA)](https://www.psyctc.org/psyctc/glossary2/analysis-of-variance-anova/) [Alternative ("alternate") hypothesis](https://www.psyctc.org/psyctc/glossary2/alternative-alternate-hypothesis/) [Boxplot (or box plot)](https://www.psyctc.org/psyctc/glossary2/boxplot-or-box-plot/) [Confidence intervals](https://www.psyctc.org/psyctc/glossary2/confidence-intervals-cis/) [Distribution](https://www.psyctc.org/psyctc/glossary2/distribution/) [Distribution shape](https://www.psyctc.org/psyctc/glossary2/distribution-shape/) [Empirical Cumulative Distribution Function (ECDF)](https://www.psyctc.org/psyctc/glossary2/empirical-cumulative-distribution-function-ecdf/) [Estimate & estimation](https://www.psyctc.org/psyctc/glossary2/estimate-estimation/) [Heteroscedasticity](https://www.psyctc.org/psyctc/glossary2/heteroscedasticity-2/) [Histograms and bar plots](https://www.psyctc.org/psyctc/glossary2/histograms-and-barplots/) Homoscedasticity [Independence of observations](https://www.psyctc.org/psyctc/glossary2/independence-of-observations-2/) [Mean (arithmetic mean, average)](https://www.psyctc.org/psyctc/glossary2/mean-arithmetic-mean-average/) Modelling [Multiple tests problem](https://www.psyctc.org/psyctc/glossary2/multiple-tests-problem/) [Non-parametric tests](https://www.psyctc.org/psyctc/glossary2/non-parametric-tests/) [Null hypothesis](https://www.psyctc.org/psyctc/glossary2/null-hypothesis/) [Null hypothesis significance testing (NHST) paradigm](https://www.psyctc.org/psyctc/glossary2/null-hypothesis-significance-testing-nhst-paradigm/) [Parametric tests](https://www.psyctc.org/psyctc/glossary2/parametric-tests/) [Robustness & robust tests](https://www.psyctc.org/psyctc/glossary2/robustness-robust-tests/) [Sensitivity (analysis)](https://www.psyctc.org/psyctc/glossary2/sensitivity-analyses/) [Statistical assumptions](https://www.psyctc.org/psyctc/glossary2/statistical-assumptions/) [Statistical power](https://www.psyctc.org/psyctc/glossary2/statistical-power/) [t-test = “Student’s t-test”](https://www.psyctc.org/psyctc/glossary2/t-test-students-t-test/) [Type I error](https://www.psyctc.org/psyctc/glossary2/type-i-error/) [Type II error](https://www.psyctc.org/psyctc/glossary2/type-ii-error/) [Violin plot (or violinplot)](https://www.psyctc.org/psyctc/glossary2/violin-plot-or-violinplot/) #### Chapters Chapter 5. #### Online resources None of mine yet. #### Dates First created 18.viii.23, wording improved and most links connected 4.xi.26.