Difference between revisions of "Theme:B6c091e2-cbb4-47d4-a14c-560d3da11db8"
(Created page with "{{Theme |Theme Type=Other |Title=Inferring from P-values |Is About=Inferring from P-values is considered to be a conventional scientific procedure <ref>Kennedy-Shaffer L. Befo...") |
|||
| (3 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
{{Theme | {{Theme | ||
| − | |Theme Type= | + | |Theme Type=Misconduct & Misbehaviors |
| + | |Has Parent Theme=Theme:48185295-9e1e-41fb-ab70-948596e588d5 | ||
|Title=Inferring from P-values | |Title=Inferring from P-values | ||
|Is About=Inferring from P-values is considered to be a conventional scientific procedure <ref>Kennedy-Shaffer L. Before p < 0.05 to Beyond p < 0.05: Using History to Contextualize p-Values and Significance Testing. Am Stat. 2019;73(sup1):82-90.</ref>. However, this statistical method is frequently misused, resulting in the publication of false positive results <ref>Ioannidis JPA. What Have We (Not) Learnt from Millions of Scientific Papers with P Values. Am Stat. 2019;73(sup1):20-25.</ref>, which is one of the reasons why the American Statistical Association (ASA) released a policy statement on P-values. There is an ongoing discussion in the scientific community regarding the statement. | |Is About=Inferring from P-values is considered to be a conventional scientific procedure <ref>Kennedy-Shaffer L. Before p < 0.05 to Beyond p < 0.05: Using History to Contextualize p-Values and Significance Testing. Am Stat. 2019;73(sup1):82-90.</ref>. However, this statistical method is frequently misused, resulting in the publication of false positive results <ref>Ioannidis JPA. What Have We (Not) Learnt from Millions of Scientific Papers with P Values. Am Stat. 2019;73(sup1):20-25.</ref>, which is one of the reasons why the American Statistical Association (ASA) released a policy statement on P-values. There is an ongoing discussion in the scientific community regarding the statement. | ||
| + | <references /> | ||
|Important Because=In early 20<sup>th</sup> century, the concept of a P-value was introduced, along with a decision rule that stated that if p<0.05, then the null hypothesis should be rejected <ref name=":0">Kennedy-Shaffer L. Before p < 0.05 to Beyond p < 0.05: Using History to Contextualize p-Values and Significance Testing. Am Stat. 2019;73(sup1):82-90.</ref>. In other words, when a P-value is less than 0.05, the results are regarded as “statistically significant” <ref name=":1">Singh Chawala D. Statistics experts urge scientists to rethink p-value. Spectrum. 2019 March 25. [cited 2020 Aug 18]. Available from: https://www.spectrumnews.org/news/statistics-experts-urge-scientists-rethink-p-value/.</ref>. | |Important Because=In early 20<sup>th</sup> century, the concept of a P-value was introduced, along with a decision rule that stated that if p<0.05, then the null hypothesis should be rejected <ref name=":0">Kennedy-Shaffer L. Before p < 0.05 to Beyond p < 0.05: Using History to Contextualize p-Values and Significance Testing. Am Stat. 2019;73(sup1):82-90.</ref>. In other words, when a P-value is less than 0.05, the results are regarded as “statistically significant” <ref name=":1">Singh Chawala D. Statistics experts urge scientists to rethink p-value. Spectrum. 2019 March 25. [cited 2020 Aug 18]. Available from: https://www.spectrumnews.org/news/statistics-experts-urge-scientists-rethink-p-value/.</ref>. | ||
| Line 8: | Line 10: | ||
Developments in decision theory, information theory, mathematical modelling and computing in the second half of the 20<sup>th</sup> century shed a completely different light on the use of P-values and statistical inference in general <ref name=":0" />. By 2016, mounting criticisms of the use and interpretation of P-values prompted the ASA to publish a policy statement <ref>Wasserstein RL, Lazar NA. The ASA Statement on p-Values: Context, Process, and Purpose. Am Stat. 2016;70(2):129-33.</ref>. | Developments in decision theory, information theory, mathematical modelling and computing in the second half of the 20<sup>th</sup> century shed a completely different light on the use of P-values and statistical inference in general <ref name=":0" />. By 2016, mounting criticisms of the use and interpretation of P-values prompted the ASA to publish a policy statement <ref>Wasserstein RL, Lazar NA. The ASA Statement on p-Values: Context, Process, and Purpose. Am Stat. 2016;70(2):129-33.</ref>. | ||
| + | <references /> | ||
|Important For=Researchers; PhD Students; Universities; Journals; Policy makers | |Important For=Researchers; PhD Students; Universities; Journals; Policy makers | ||
|Has Best Practice=The ASA statement on P-values gives instructions on the correct use of P-values, with the goal of improving interpretation in quantitative science. The overall conclusion of the ASA is that scientific inferences should not be based exclusively on P-value threshold, because that, in itself, does not provide substantial evidence regarding a model or hypothesis, nor does it measure the size of a certain effect or determine the importance of the results. Researchers should use P-values within a proper context, because otherwise it can lead to selective reporting <ref name=":0">Wasserstein RL, Lazar NA. The ASA Statement on p-Values: Context, Process, and Purpose. Am Stat. 2016;70(2):129-33.</ref>. Good scientific inference requires the full and transparent reporting of data and methods <ref name=":0" />. There are other methods that researchers can use with or instead of P-values, which mostly focus on estimations as opposed to testing. These include confidence, credibility or prediction intervals, Bayesian methods, decision-theoretic modeling and false discovery rates <ref name=":0" />. | |Has Best Practice=The ASA statement on P-values gives instructions on the correct use of P-values, with the goal of improving interpretation in quantitative science. The overall conclusion of the ASA is that scientific inferences should not be based exclusively on P-value threshold, because that, in itself, does not provide substantial evidence regarding a model or hypothesis, nor does it measure the size of a certain effect or determine the importance of the results. Researchers should use P-values within a proper context, because otherwise it can lead to selective reporting <ref name=":0">Wasserstein RL, Lazar NA. The ASA Statement on p-Values: Context, Process, and Purpose. Am Stat. 2016;70(2):129-33.</ref>. Good scientific inference requires the full and transparent reporting of data and methods <ref name=":0" />. There are other methods that researchers can use with or instead of P-values, which mostly focus on estimations as opposed to testing. These include confidence, credibility or prediction intervals, Bayesian methods, decision-theoretic modeling and false discovery rates <ref name=":0" />. | ||
| Line 14: | Line 17: | ||
Critiques, initiatives and recommendations require not only further academic discussion, but also significant educational reforms in statistics <ref>Hubbard R, Haig BD, Parsa RA. The Limited Role of Formal Statistical Inference in Scientific Inference. Am Stat. 2019;73(sup1):91-98.</ref>. | Critiques, initiatives and recommendations require not only further academic discussion, but also significant educational reforms in statistics <ref>Hubbard R, Haig BD, Parsa RA. The Limited Role of Formal Statistical Inference in Scientific Inference. Am Stat. 2019;73(sup1):91-98.</ref>. | ||
| + | <references /> | ||
}} | }} | ||
{{Related To | {{Related To | ||
| + | |Related To Resource=Resource:A2dfa1c6-22c6-43e1-9bc9-483e87ee701e;Resource:D1fda917-43c6-4e43-9ebe-0aed1f4fa404 | ||
|Related To Theme=Theme:26631aa0-18f0-4635-b71b-80a6f4e58d33;Theme:6b584d4e-2c9d-4e27-b370-5fbdb983ab46;Theme:349f9eb9-b796-46cb-9a98-214c06db9046 | |Related To Theme=Theme:26631aa0-18f0-4635-b71b-80a6f4e58d33;Theme:6b584d4e-2c9d-4e27-b370-5fbdb983ab46;Theme:349f9eb9-b796-46cb-9a98-214c06db9046 | ||
}} | }} | ||
Revision as of 10:36, 28 October 2020
ThemesInferring from P-values
What is this about?
Inferring from P-values is considered to be a conventional scientific procedure [1]. However, this statistical method is frequently misused, resulting in the publication of false positive results [2], which is one of the reasons why the American Statistical Association (ASA) released a policy statement on P-values. There is an ongoing discussion in the scientific community regarding the statement.
Why is this important?
In early 20th century, the concept of a P-value was introduced, along with a decision rule that stated that if p<0.05, then the null hypothesis should be rejected [1]. In other words, when a P-value is less than 0.05, the results are regarded as “statistically significant” [2].
Journals widely encourage the use of the method of inferring from P-values for publication, which puts researchers under a lot of pressure to publish “statistically significant” results [2]. According to recent findings, 96% of abstracts and full-text articles in the biomedical literature from 1990 to 2015 presented p<0.05, which is considered “too good to be true”, and indicates that there is a practice of selective reporting [3].
Developments in decision theory, information theory, mathematical modelling and computing in the second half of the 20th century shed a completely different light on the use of P-values and statistical inference in general [1]. By 2016, mounting criticisms of the use and interpretation of P-values prompted the ASA to publish a policy statement [4].
- ↑ 1.0 1.1 Kennedy-Shaffer L. Before p < 0.05 to Beyond p < 0.05: Using History to Contextualize p-Values and Significance Testing. Am Stat. 2019;73(sup1):82-90.
- ↑ 2.0 2.1 Singh Chawala D. Statistics experts urge scientists to rethink p-value. Spectrum. 2019 March 25. [cited 2020 Aug 18]. Available from: https://www.spectrumnews.org/news/statistics-experts-urge-scientists-rethink-p-value/.
- ↑ Ioannidis JPA. What Have We (Not) Learnt from Millions of Scientific Papers with P Values. Am Stat. 2019;73(sup1):20-25.
- ↑ Wasserstein RL, Lazar NA. The ASA Statement on p-Values: Context, Process, and Purpose. Am Stat. 2016;70(2):129-33.
For whom is this important?
What are the best practices?
The ASA statement on P-values gives instructions on the correct use of P-values, with the goal of improving interpretation in quantitative science. The overall conclusion of the ASA is that scientific inferences should not be based exclusively on P-value threshold, because that, in itself, does not provide substantial evidence regarding a model or hypothesis, nor does it measure the size of a certain effect or determine the importance of the results. Researchers should use P-values within a proper context, because otherwise it can lead to selective reporting [1]. Good scientific inference requires the full and transparent reporting of data and methods [1]. There are other methods that researchers can use with or instead of P-values, which mostly focus on estimations as opposed to testing. These include confidence, credibility or prediction intervals, Bayesian methods, decision-theoretic modeling and false discovery rates [1].
Since its release in 2016, the ASA statement has been cited about 1,700 times and downloaded nearly 300,000 times. In 2017, the ASA organized a symposium on statistical methods, which resulted in 43 articles on the topic of the responsible use of P-values[2] . Statisticians and scientists are currently considering “a world beyond p<0.05” ([3]), suggesting a wide spectrum of solutions and possibilities. One solution involves changing the P-value threshold for statistical significance from 0.05 to 0.005 ([3][4]). By contrast, others argue that reproducibility of results and pre-registration are the best means for preventing selection bias [5]. Others still recommend including more information when reporting P-values, such as the researcher’s confidence in the P-value or their assessment of the likelihood that a statistically significant finding is, in fact, a false positive result [6].
Critiques, initiatives and recommendations require not only further academic discussion, but also significant educational reforms in statistics [7].
- ↑ 1.0 1.1 1.2 Wasserstein RL, Lazar NA. The ASA Statement on p-Values: Context, Process, and Purpose. Am Stat. 2016;70(2):129-33.
- ↑ Singh Chawala D. Statistics experts urge scientists to rethink p-value. Spectrum. 2019 March 25. [cited 2020 Aug 18]. Available from: https://www.spectrumnews.org/news/statistics-experts-urge-scientists-rethink-p-value/
- ↑ 3.0 3.1 Kennedy-Shaffer L. Before p < 0.05 to Beyond p < 0.05: Using History to Contextualize p-Values and Significance Testing. Am Stat. 2019;73(sup1):82-90.
- ↑ Betensky RA. The p-Value Requires Context, Not a Threshold. Am Stat. 2019;73(sup1):115-117.
- ↑ Ioannidis JPA. What Have We (Not) Learnt from Millions of Scientific Papers with P Values. Am Stat. 2019;73(sup1):20-25.
- ↑ Halsey LG. The reign of the p-value is over: what alternative analyses could we employ to fill the power vacuum. Biol Lett. 2019;15:1-8.
- ↑ Hubbard R, Haig BD, Parsa RA. The Limited Role of Formal Statistical Inference in Scientific Inference. Am Stat. 2019;73(sup1):91-98.
Natalie Evans, Andrijana Perković Paloš, Fran Šaler contributed to this theme. Latest contribution was Feb 09, 2023
