Difference between revisions of "Theme:B6c091e2-cbb4-47d4-a14c-560d3da11db8"

From The Embassy of Good Science
(Created page with "{{Theme |Theme Type=Other |Title=Inferring from P-values |Is About=Inferring from P-values is considered to be a conventional scientific procedure <ref>Kennedy-Shaffer L. Befo...")
 
Line 3: Line 3:
 
|Title=Inferring from P-values
 
|Title=Inferring from P-values
 
|Is About=Inferring from P-values is considered to be a conventional scientific procedure <ref>Kennedy-Shaffer L. Before p < 0.05 to Beyond p < 0.05: Using History to Contextualize p-Values and Significance Testing. Am Stat. 2019;73(sup1):82-90.</ref>. However, this statistical method is frequently misused, resulting in the publication of false positive results <ref>Ioannidis JPA. What Have We (Not) Learnt from Millions of Scientific Papers with P Values. Am Stat. 2019;73(sup1):20-25.</ref>, which is one of the reasons why the American Statistical Association (ASA) released a policy statement on P-values. There is an ongoing discussion in the scientific community regarding the statement.
 
|Is About=Inferring from P-values is considered to be a conventional scientific procedure <ref>Kennedy-Shaffer L. Before p < 0.05 to Beyond p < 0.05: Using History to Contextualize p-Values and Significance Testing. Am Stat. 2019;73(sup1):82-90.</ref>. However, this statistical method is frequently misused, resulting in the publication of false positive results <ref>Ioannidis JPA. What Have We (Not) Learnt from Millions of Scientific Papers with P Values. Am Stat. 2019;73(sup1):20-25.</ref>, which is one of the reasons why the American Statistical Association (ASA) released a policy statement on P-values. There is an ongoing discussion in the scientific community regarding the statement.
 +
<references />
 
|Important Because=In early 20<sup>th</sup> century, the concept of a P-value was introduced, along with a decision rule that stated that if p<0.05, then the null hypothesis should be rejected <ref name=":0">Kennedy-Shaffer L. Before p < 0.05 to Beyond p < 0.05: Using History to Contextualize p-Values and Significance Testing. Am Stat. 2019;73(sup1):82-90.</ref>. In other words, when a P-value is less than 0.05, the results are regarded as “statistically significant” <ref name=":1">Singh Chawala D. Statistics experts urge scientists to rethink p-value. Spectrum. 2019 March 25. [cited 2020 Aug 18]. Available from:  https://www.spectrumnews.org/news/statistics-experts-urge-scientists-rethink-p-value/.</ref>.  
 
|Important Because=In early 20<sup>th</sup> century, the concept of a P-value was introduced, along with a decision rule that stated that if p<0.05, then the null hypothesis should be rejected <ref name=":0">Kennedy-Shaffer L. Before p < 0.05 to Beyond p < 0.05: Using History to Contextualize p-Values and Significance Testing. Am Stat. 2019;73(sup1):82-90.</ref>. In other words, when a P-value is less than 0.05, the results are regarded as “statistically significant” <ref name=":1">Singh Chawala D. Statistics experts urge scientists to rethink p-value. Spectrum. 2019 March 25. [cited 2020 Aug 18]. Available from:  https://www.spectrumnews.org/news/statistics-experts-urge-scientists-rethink-p-value/.</ref>.  
  
Line 8: Line 9:
  
 
Developments in decision theory, information theory, mathematical modelling and computing in the second half of the 20<sup>th</sup> century shed a completely different light on the use of P-values and statistical inference in general <ref name=":0" />. By 2016, mounting criticisms of the use and interpretation of P-values prompted the ASA to publish a policy statement <ref>Wasserstein RL, Lazar NA. The ASA Statement on p-Values: Context, Process, and Purpose. Am Stat. 2016;70(2):129-33.</ref>.
 
Developments in decision theory, information theory, mathematical modelling and computing in the second half of the 20<sup>th</sup> century shed a completely different light on the use of P-values and statistical inference in general <ref name=":0" />. By 2016, mounting criticisms of the use and interpretation of P-values prompted the ASA to publish a policy statement <ref>Wasserstein RL, Lazar NA. The ASA Statement on p-Values: Context, Process, and Purpose. Am Stat. 2016;70(2):129-33.</ref>.
 +
<references />
 
|Important For=Researchers; PhD Students; Universities; Journals; Policy makers
 
|Important For=Researchers; PhD Students; Universities; Journals; Policy makers
 
|Has Best Practice=The ASA statement on P-values gives instructions on the correct use of P-values, with the goal of improving interpretation in quantitative science. The overall conclusion of the ASA is that scientific inferences should not be based exclusively on P-value threshold, because that, in itself, does not provide substantial evidence regarding a model or hypothesis, nor does it measure the size of a certain effect or determine the importance of the results. Researchers should use P-values within a proper context, because otherwise it can lead to selective reporting <ref name=":0">Wasserstein RL, Lazar NA. The ASA Statement on p-Values: Context, Process, and Purpose. Am Stat. 2016;70(2):129-33.</ref>.  Good scientific inference requires the full and transparent reporting of data and methods <ref name=":0" />. There are other methods that researchers can use with or instead of P-values, which mostly focus on estimations as opposed to testing. These include confidence, credibility or prediction intervals, Bayesian methods, decision-theoretic modeling and false discovery rates <ref name=":0" />.
 
|Has Best Practice=The ASA statement on P-values gives instructions on the correct use of P-values, with the goal of improving interpretation in quantitative science. The overall conclusion of the ASA is that scientific inferences should not be based exclusively on P-value threshold, because that, in itself, does not provide substantial evidence regarding a model or hypothesis, nor does it measure the size of a certain effect or determine the importance of the results. Researchers should use P-values within a proper context, because otherwise it can lead to selective reporting <ref name=":0">Wasserstein RL, Lazar NA. The ASA Statement on p-Values: Context, Process, and Purpose. Am Stat. 2016;70(2):129-33.</ref>.  Good scientific inference requires the full and transparent reporting of data and methods <ref name=":0" />. There are other methods that researchers can use with or instead of P-values, which mostly focus on estimations as opposed to testing. These include confidence, credibility or prediction intervals, Bayesian methods, decision-theoretic modeling and false discovery rates <ref name=":0" />.
Line 14: Line 16:
  
 
Critiques, initiatives and recommendations require not only further academic discussion, but also significant educational reforms in statistics <ref>Hubbard R, Haig BD, Parsa RA. The Limited Role of Formal Statistical Inference in Scientific Inference. Am Stat. 2019;73(sup1):91-98.</ref>.
 
Critiques, initiatives and recommendations require not only further academic discussion, but also significant educational reforms in statistics <ref>Hubbard R, Haig BD, Parsa RA. The Limited Role of Formal Statistical Inference in Scientific Inference. Am Stat. 2019;73(sup1):91-98.</ref>.
 +
<references />
 
}}
 
}}
 
{{Related To
 
{{Related To
 +
|Related To Resource=Resource:A2dfa1c6-22c6-43e1-9bc9-483e87ee701e
 
|Related To Theme=Theme:26631aa0-18f0-4635-b71b-80a6f4e58d33;Theme:6b584d4e-2c9d-4e27-b370-5fbdb983ab46;Theme:349f9eb9-b796-46cb-9a98-214c06db9046
 
|Related To Theme=Theme:26631aa0-18f0-4635-b71b-80a6f4e58d33;Theme:6b584d4e-2c9d-4e27-b370-5fbdb983ab46;Theme:349f9eb9-b796-46cb-9a98-214c06db9046
 
}}
 
}}

Revision as of 08:08, 22 October 2020

Inferring from P-values

What is this about?

Inferring from P-values is considered to be a conventional scientific procedure [1]. However, this statistical method is frequently misused, resulting in the publication of false positive results [2], which is one of the reasons why the American Statistical Association (ASA) released a policy statement on P-values. There is an ongoing discussion in the scientific community regarding the statement.

  1. Kennedy-Shaffer L. Before p < 0.05 to Beyond p < 0.05: Using History to Contextualize p-Values and Significance Testing. Am Stat. 2019;73(sup1):82-90.
  2. Ioannidis JPA. What Have We (Not) Learnt from Millions of Scientific Papers with P Values. Am Stat. 2019;73(sup1):20-25.

Why is this important?

In early 20th century, the concept of a P-value was introduced, along with a decision rule that stated that if p<0.05, then the null hypothesis should be rejected [1]. In other words, when a P-value is less than 0.05, the results are regarded as “statistically significant” [2].  

Journals widely encourage the use of the method of inferring from P-values for publication, which puts researchers under a lot of pressure to publish “statistically significant” results [2]. According to recent findings, 96% of abstracts and full-text articles in the biomedical literature from 1990 to 2015 presented p<0.05, which is considered “too good to be true”, and indicates that there is a practice of selective reporting [3].

Developments in decision theory, information theory, mathematical modelling and computing in the second half of the 20th century shed a completely different light on the use of P-values and statistical inference in general [1]. By 2016, mounting criticisms of the use and interpretation of P-values prompted the ASA to publish a policy statement [4].

  1. 1.0 1.1 Kennedy-Shaffer L. Before p < 0.05 to Beyond p < 0.05: Using History to Contextualize p-Values and Significance Testing. Am Stat. 2019;73(sup1):82-90.
  2. 2.0 2.1 Singh Chawala D. Statistics experts urge scientists to rethink p-value. Spectrum. 2019 March 25. [cited 2020 Aug 18]. Available from:  https://www.spectrumnews.org/news/statistics-experts-urge-scientists-rethink-p-value/.
  3. Ioannidis JPA. What Have We (Not) Learnt from Millions of Scientific Papers with P Values. Am Stat. 2019;73(sup1):20-25.
  4. Wasserstein RL, Lazar NA. The ASA Statement on p-Values: Context, Process, and Purpose. Am Stat. 2016;70(2):129-33.

For whom is this important?

What are the best practices?

The ASA statement on P-values gives instructions on the correct use of P-values, with the goal of improving interpretation in quantitative science. The overall conclusion of the ASA is that scientific inferences should not be based exclusively on P-value threshold, because that, in itself, does not provide substantial evidence regarding a model or hypothesis, nor does it measure the size of a certain effect or determine the importance of the results. Researchers should use P-values within a proper context, because otherwise it can lead to selective reporting [1].  Good scientific inference requires the full and transparent reporting of data and methods [1]. There are other methods that researchers can use with or instead of P-values, which mostly focus on estimations as opposed to testing. These include confidence, credibility or prediction intervals, Bayesian methods, decision-theoretic modeling and false discovery rates [1].

Since its release in 2016, the ASA statement has been cited about 1,700 times and downloaded nearly 300,000 times. In 2017, the ASA organized a symposium on statistical methods, which resulted in 43 articles on the topic of the responsible use of P-values[2] . Statisticians and scientists are currently considering “a world beyond p<0.05” ([3]), suggesting a wide spectrum of solutions and possibilities. One solution involves changing the P-value threshold for statistical significance from 0.05 to 0.005 ([3][4]). By contrast, others argue that reproducibility of results and pre-registration are the best means for preventing selection bias [5]. Others still recommend including more information when reporting P-values, such as the researcher’s confidence in the P-value or their assessment of the likelihood that a statistically significant finding is, in fact, a false positive result [6].

Critiques, initiatives and recommendations require not only further academic discussion, but also significant educational reforms in statistics [7].

  1. 1.0 1.1 1.2 Wasserstein RL, Lazar NA. The ASA Statement on p-Values: Context, Process, and Purpose. Am Stat. 2016;70(2):129-33.
  2. Singh Chawala D. Statistics experts urge scientists to rethink p-value. Spectrum. 2019 March 25. [cited 2020 Aug 18]. Available from:  https://www.spectrumnews.org/news/statistics-experts-urge-scientists-rethink-p-value/
  3. 3.0 3.1 Kennedy-Shaffer L. Before p < 0.05 to Beyond p < 0.05: Using History to Contextualize p-Values and Significance Testing. Am Stat. 2019;73(sup1):82-90.
  4. Betensky RA. The p-Value Requires Context, Not a Threshold. Am Stat. 2019;73(sup1):115-117.
  5. Ioannidis JPA. What Have We (Not) Learnt from Millions of Scientific Papers with P Values. Am Stat. 2019;73(sup1):20-25.
  6. Halsey LG. The reign of the p-value is over: what alternative analyses could we employ to fill the power vacuum. Biol Lett. 2019;15:1-8.
  7. Hubbard R, Haig BD, Parsa RA. The Limited Role of Formal Statistical Inference in Scientific Inference. Am Stat. 2019;73(sup1):91-98.
Cookies help us deliver our services. By using our services, you agree to our use of cookies.
5.1.6