Testing for significance

Decide on your significance level α
Calculate your statistical value p
If p < α, then the result is significant, else it is not significant

An alternative view is:
$LaTeX: \text{confidence} = (\text{signal}/\text{noise}) \times \sqrt{\text{sample size}}$ $\text{confidence} = (\text{signal}/\text{noise}) \times \sqrt{\text{sample size}}$

For details of the above equation see: David L. Sackett, Why randomized controlled trials fail but needn't:2. Failure to employ physiological statistics, or the only formula a clinician-trialist is ever likely to need (or understand!). [Sackett2001] http://www.cmaj.ca/cgi/content/full/165/9/1226 Links to an external site.

See also: Understanding Hypothesis Testing: Example #1, Department of Statistics, West Virginia University,
last modified 4 April 2000 formerly available from http://www.stat.wvu.edu/SRS/Modules/HypTest/exam1.html

[Sackett2001] David L. Sackett, ‘Why randomized controlled trials fail but needn’t: 2. Failure to employ physiological statistics, or the only formula a clinician-trialist is ever likely to need (or understand!)’, Canadian Medical Association Journal (CMAJ), vol. 165, no. 9, pp. 1226–1237, Oct. 2001. PubMedID (PMID): 11706914

[WVU2000] Department of Statistics, West Virginia University, ‘Understanding Hypothesis Testing: Example #1’, 04-Apr-2000. [Online]. Formerly available from http://www.stat.wvu.edu/SRS/Modules/HypTest/exam1.html. [Accessed: 03-Aug-2015]

Transcript

So how can we test for significance? Well, the first thing we need to decide on is the significance level, in this case, alpha, then we need to calculate the statistical variable p. and then we say if p is less than alpha, then the result is significant; otherwise, it's not significant. Now for those of you who like to think in terms of signal to noise ratio, it is very simple to understand the confidence is equal to the signal to noise ratio times the square root of the sample size. And immediately now it should become apparent to you that if I have a very small signal and a lot of noise, then I need a big sample size. And to get twice the confidence, I need the square of the sample size for a given signal to noise ratio. So, either think of it in the statistical sense or think of it in the signal to noise [ratio] sense. Voilà, you can start to get a feel for what it means for something to be statistically significant.