{"id":5412,"date":"2026-07-19T21:16:53","date_gmt":"2026-07-19T19:16:53","guid":{"rendered":"https:\/\/www.psyctc.org\/psyctc\/?post_type=docs&#038;p=5412"},"modified":"2026-07-20T22:50:23","modified_gmt":"2026-07-20T20:50:23","password":"","slug":"mapping-psychometric-statistics-to-words-rules-of-thumb","status":"publish","type":"docs","link":"https:\/\/www.psyctc.org\/psyctc\/glossary2\/mapping-psychometric-statistics-to-words-rules-of-thumb\/","title":{"rendered":"Mapping psychometric statistics to words: rules of thumb"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">This seems to me to be a particular feature of psychometrics: it loves rules of thumb that map from the values of a psychometric statistic observed in a dataset to words like &#8220;adequate&#8221;, &#8220;good&#8221; or &#8220;excellent&#8221;. I have increasing clarity that these mappings are deeply problematical and feed into treating questionnaire measures as if they were physical science measures.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Details<\/h4>\n\n\n\n<p class=\"wp-block-paragraph\">One commonly seen mapping is for Cronbach&#8217;s alpha index of internal reliability.  As one source has it:<br>&#8220;<em><strong>The most commonly accepted threshold is 0.70.<\/strong> Most dissertation committees will want to see \u03b1 \u2265 0.70 for each scale or subscale you use. Some fields accept 0.60 for exploratory research.<\/em>&#8221; (<a href=\"https:\/\/statisticsforresearch.com\/blog\/cronbachs-alpha-explained\/\">https:\/\/statisticsforresearch.com\/blog\/cronbachs-alpha-explained\/<\/a>).  That source also has a very commonly used mapping:<\/p>\n\n\n\n    <h3 class=\"wpdt-c\"\n        id=\"wdt-table-title-47\">Cronbach&#039;s alpha<\/h3>\n<div class=\"wpdt-c row wpDataTableContainerSimpleTable wpDataTables wpDataTablesWrapper\n\"\n    >\n        <table id=\"wpdtSimpleTable-47\"\n           style=\"border-collapse:collapse;\n                   border-spacing:0px;\"\n           class=\"wpdtSimpleTable wpDataTable\"\n           data-column=\"2\"\n           data-rows=\"6\"\n           data-wpID=\"47\"\n           data-responsive=\"0\"\n           data-has-header=\"0\">\n\n                    <tbody>        <tr class=\"wpdt-cell-row \" >\n                                <td class=\"wpdt-cell \"\n                                            data-cell-id=\"A1\"\n                    data-col-index=\"0\"\n                    data-row-index=\"0\"\n                    style=\" width:59.839357429719%;                    padding:10px;\n                    \"\n                    >\n                                        Alpha values                    <\/td>\n                                                <td class=\"wpdt-cell wpdt-bold wpdt-align-center\"\n                                            data-cell-id=\"B1\"\n                    data-col-index=\"1\"\n                    data-row-index=\"0\"\n                    style=\" width:40.160642570281%;                    padding:10px;\n                    \"\n                    >\n                                        Translation                    <\/td>\n                                        <\/tr>\n                            <tr class=\"wpdt-cell-row \" >\n                                <td class=\"wpdt-cell wpdt-align-left\"\n                                            data-cell-id=\"A2\"\n                    data-col-index=\"0\"\n                    data-row-index=\"1\"\n                    style=\"                    padding:10px;\n                    \"\n                    >\n                                        >= .9                    <\/td>\n                                                <td class=\"wpdt-cell wpdt-align-left\"\n                                            data-cell-id=\"B2\"\n                    data-col-index=\"1\"\n                    data-row-index=\"1\"\n                    style=\"                    padding:10px;\n                    \"\n                    >\n                                        Excellent                    <\/td>\n                                        <\/tr>\n                            <tr class=\"wpdt-cell-row \" >\n                                <td class=\"wpdt-cell wpdt-align-left\"\n                                            data-cell-id=\"A3\"\n                    data-col-index=\"0\"\n                    data-row-index=\"2\"\n                    style=\"                    padding:10px;\n                    \"\n                    >\n                                        .8 <= alpha < .9                    <\/td>\n                                                <td class=\"wpdt-cell wpdt-align-left\"\n                                            data-cell-id=\"B3\"\n                    data-col-index=\"1\"\n                    data-row-index=\"2\"\n                    style=\"                    padding:10px;\n                    \"\n                    >\n                                        Good                    <\/td>\n                                        <\/tr>\n                            <tr class=\"wpdt-cell-row \" >\n                                <td class=\"wpdt-cell wpdt-align-left\"\n                                            data-cell-id=\"A4\"\n                    data-col-index=\"0\"\n                    data-row-index=\"3\"\n                    style=\"                    padding:10px;\n                    \"\n                    >\n                                        .7 <= alpha < .8                    <\/td>\n                                                <td class=\"wpdt-cell wpdt-align-left\"\n                                            data-cell-id=\"B4\"\n                    data-col-index=\"1\"\n                    data-row-index=\"3\"\n                    style=\"                    padding:10px;\n                    \"\n                    >\n                                        Acceptable                    <\/td>\n                                        <\/tr>\n                            <tr class=\"wpdt-cell-row \" >\n                                <td class=\"wpdt-cell wpdt-align-left\"\n                                            data-cell-id=\"A5\"\n                    data-col-index=\"0\"\n                    data-row-index=\"4\"\n                    style=\"                    padding:10px;\n                    \"\n                    >\n                                        .6 <= alpha < .7                    <\/td>\n                                                <td class=\"wpdt-cell wpdt-align-left\"\n                                            data-cell-id=\"B5\"\n                    data-col-index=\"1\"\n                    data-row-index=\"4\"\n                    style=\"                    padding:10px;\n                    \"\n                    >\n                                        Questionable                    <\/td>\n                                        <\/tr>\n                            <tr class=\"wpdt-cell-row \" >\n                                <td class=\"wpdt-cell wpdt-align-left\"\n                                            data-cell-id=\"A6\"\n                    data-col-index=\"0\"\n                    data-row-index=\"5\"\n                    style=\"                    padding:10px;\n                    \"\n                    >\n                                        < .6                    <\/td>\n                                                <td class=\"wpdt-cell wpdt-align-left\"\n                                            data-cell-id=\"B6\"\n                    data-col-index=\"1\"\n                    data-row-index=\"5\"\n                    style=\"                    padding:10px;\n                    \"\n                    >\n                                        Poor                    <\/td>\n                                        <\/tr>\n                    <\/table>\n<\/div><style id='wpdt-custom-style-47'>\n<\/style>\n<style>\n                    \n                                                                                        \/* table font color *\/\n    .wpdt-c.wpDataTablesWrapper table.wpdtSimpleTable,\n    .wpdt-c .wpDataTablesWrapper table.wpDataTable {\n        font-family: Lucida Sans Unicode, Lucida Grande, sans-serif !important;\n    }\n\n            \/* table font size *\/\n    .wpdt-c.wpDataTablesWrapper table.wpdtSimpleTable,\n    .wpdt-c .wpDataTablesWrapper table.wpDataTable {\n        font-size: 20px !important;\n    }\n\n            \n                <\/style>\n\n\n\n\n<p class=\"wp-block-paragraph\">There is so much that is problematical here:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>it doesn&#8217;t distinguish between an alpha value from the data that is to be used and one that is being assumed to generalise from some other dataset to these data (though the &#8220;dissertation committee&#8221; suggests alpha calculated in the data being analysed for purposes beyond reliability estimation)<\/li>\n\n\n\n<li>as the comment about &#8220;dissertation committees&#8221; shows, these values are given huge value, they can pass or fail a dissertation<\/li>\n\n\n\n<li>the mapping completely ignores what will be done with values  that are collected with the measure.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">The first of those problems is fairly trivial: the mapping is really intended to be used to describe a measure whose alpha has been determined in that dataset.  However, we see it in generalisations like: &#8220;Tolkein&#8217;s measure of orcishness has excellent reliability (alpha .95, Gollum &amp; Serkis, 2000)&#8221;. This begs huge questions about whether a measure will have the same alpha value in a different dataset from that of Gollum &amp; Serkis, 2000. Questionnaire measures don&#8217;t have the same (relative) independence of their reliability from the application area that many physical measures have across a reasonable range of settings and temperatures a thermometer has steady reliability (usually very high by our standards).  This is not true when questionnaires are moved across cultures or even from help-seeking to non-help-seeking participants.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">However, it&#8217;s the next two problems that are really dangerous. If a PhD student&#8217;s new measure of orcishness (or something more useful one hopes) only has an alpha of .45 in the development dataset but the scores on the measure are also being used to look at the correlation between scores and say teenage age to the nearest day then we can treat age as measured with very good reliability. So we can now look at the impact of a reliability of .45 for the measure for the candidate&#8217;s NHST (Null Hypothesis Significance Testing) for a population Pearson correlation of .7 with older adolescents more orchish. If we ignore reliabilities of either measure as is typical in power calculations the candidate only needed a dataset <em>n<\/em> of 14 to have a power of .8 for an alpha of .05.  We know that an &#8220;unacceptable&#8221; alpha of .45 will attenuate a population correlation measured without unreliability from that .7 to .46 (<a href=\"https:\/\/shiny.psyctc.org\/apps\/Attenuation2\/\">https:\/\/shiny.psyctc.org\/apps\/Attenuation2\/<\/a>). Now if we go back to the power calculation (e.g. <a href=\"https:\/\/www.pwranalysis.com\/power-analysis-pearson-correlation\/\">https:\/\/www.pwranalysis.com\/power-analysis-pearson-correlation\/<\/a>) and we need an <em>n<\/em> of 35 rather than 14 to have that power of .8. Is this &#8220;unacceptable&#8221;? The alpha is certainly an indication that the items in the measure may be contaminated by a lot of random variation between participants that is not about orcishness but there is still quite sufficient statistical power if the candidate had more than 34 participants for her\/him to argue that the data suggest rejection of the null of no relationship between orcishness and age.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">At the other extreme if the planned use of the questionnaire is in screening then an alpha of over .95 may still give poor positive and negative predictive validity values unless the issue screened for is very common.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">We should stop using these mappings and work through the implications of the observed alpha values for the intended use of the scores when the alpha is for the dataset to be used.  The above is an example of doing this for a low alpha and a correlation design.  We should simply not use these mappings at all when generalising from an alpha from one dataset to use of the measure in another dataset.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The issue is not particular to internal reliability and Cronbach&#8217;s alpha, these mappings are seen for effect sizes, for confirmatory analysis fit indices and other psychometric statistics.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">How to suspect a mapping should be ignored.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Equal steps as for the alpha values are suspicious: why such a neat mapping?<\/li>\n\n\n\n<li>Can you translate the adjectives in your head into something meaningful like statistical power or the probable precision of estimation?  (The answer to that is &#8220;never&#8221;!)<\/li>\n<\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">Try also<\/h4>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Attenuation by unreliability of measurement<\/li>\n\n\n\n<li>Confidence intervals<\/li>\n\n\n\n<li>Correlation<\/li>\n\n\n\n<li>Cronbach&#8217;s alpha<\/li>\n\n\n\n<li>Estimation<\/li>\n\n\n\n<li>Internal reliability<\/li>\n\n\n\n<li>Null Hypothesis Significance Testing (NHST)<\/li>\n\n\n\n<li>Statistical power<\/li>\n<\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">Chapters<\/h4>\n\n\n\n<p class=\"wp-block-paragraph\">Not really covered in the OMbook.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Online resources<\/h4>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/shiny.psyctc.org\/apps\/Attenuation2\/\" target=\"_blank\" rel=\"noreferrer noopener\">Attentuation of correlation by unreliability<\/a> in my <a href=\"https:\/\/shiny.psyctc.org\" target=\"_blank\" rel=\"noreferrer noopener\">shiny apps<\/a>.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Dates<\/h4>\n\n\n\n<p class=\"wp-block-paragraph\">First created 19.vii.26, tweaked 20.vii.26.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>This seems to me to be a particular feature of psychometrics: it loves rules of thumb that map from the values of a psychometric statistic observed in a dataset to words like &#8220;adequate&#8221;, &#8220;good&#8221; or &#8220;excellent&#8221;. I have increasing clarity that these mappings are deeply problematical and feed into treating questionnaire measures as if they &hellip; <a href=\"https:\/\/www.psyctc.org\/psyctc\/glossary2\/mapping-psychometric-statistics-to-words-rules-of-thumb\/\" class=\"more-link\">Continue reading <span class=\"screen-reader-text\">Mapping psychometric statistics to words: rules of thumb<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","template":"","meta":{"footnotes":""},"doc_category":[],"glossaries":[],"doc_tag":[],"knowledge_base":[],"class_list":["post-5412","docs","type-docs","status-publish","hentry"],"year_month":"2026-08","word_count":820,"total_views":"39","reactions":{"happy":"0","normal":"0","sad":"0"},"author_info":{"name":"chris","author_nicename":"chris","author_url":"https:\/\/www.psyctc.org\/psyctc\/author\/chris\/"},"doc_category_info":[],"doc_tag_info":[],"knowledge_base_info":[],"knowledge_base_slug":[],"_links":{"self":[{"href":"https:\/\/www.psyctc.org\/psyctc\/wp-json\/wp\/v2\/docs\/5412","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.psyctc.org\/psyctc\/wp-json\/wp\/v2\/docs"}],"about":[{"href":"https:\/\/www.psyctc.org\/psyctc\/wp-json\/wp\/v2\/types\/docs"}],"author":[{"embeddable":true,"href":"https:\/\/www.psyctc.org\/psyctc\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.psyctc.org\/psyctc\/wp-json\/wp\/v2\/comments?post=5412"}],"version-history":[{"count":4,"href":"https:\/\/www.psyctc.org\/psyctc\/wp-json\/wp\/v2\/docs\/5412\/revisions"}],"predecessor-version":[{"id":5417,"href":"https:\/\/www.psyctc.org\/psyctc\/wp-json\/wp\/v2\/docs\/5412\/revisions\/5417"}],"wp:attachment":[{"href":"https:\/\/www.psyctc.org\/psyctc\/wp-json\/wp\/v2\/media?parent=5412"}],"wp:term":[{"taxonomy":"doc_category","embeddable":true,"href":"https:\/\/www.psyctc.org\/psyctc\/wp-json\/wp\/v2\/doc_category?post=5412"},{"taxonomy":"glossaries","embeddable":true,"href":"https:\/\/www.psyctc.org\/psyctc\/wp-json\/wp\/v2\/glossaries?post=5412"},{"taxonomy":"doc_tag","embeddable":true,"href":"https:\/\/www.psyctc.org\/psyctc\/wp-json\/wp\/v2\/doc_tag?post=5412"},{"taxonomy":"knowledge_base","embeddable":true,"href":"https:\/\/www.psyctc.org\/psyctc\/wp-json\/wp\/v2\/knowledge_base?post=5412"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}