Leonard Bickman and Michele Athay
- Published in print:
- 2009
- Published Online:
- April 2010
- ISBN:
- 9780195325522
- eISBN:
- 9780199893850
- Item type:
- chapter
- Publisher:
- Oxford University Press
- DOI:
- 10.1093/acprof:oso/9780195325522.003.0009
- Subject:
- Social Work, Research and Evaluation
This chapter addresses every researcher's nightmare: what to do when you have null results. What does it mean? How can an investigator extract value from the data and move on in one's career from ...
More
This chapter addresses every researcher's nightmare: what to do when you have null results. What does it mean? How can an investigator extract value from the data and move on in one's career from that point? The chapter starts with steps toward launching scientifically valid field research. The community intervention development model includes manualizing the protocol in the context of the practice setting, initial controlled efficacy trials, single-case applications, initial effectiveness tests, full effectiveness tests, and, finally, evaluating and disseminating results. The relationship with the host organization or practice setting is emphasized throughout.Less
This chapter addresses every researcher's nightmare: what to do when you have null results. What does it mean? How can an investigator extract value from the data and move on in one's career from that point? The chapter starts with steps toward launching scientifically valid field research. The community intervention development model includes manualizing the protocol in the context of the practice setting, initial controlled efficacy trials, single-case applications, initial effectiveness tests, full effectiveness tests, and, finally, evaluating and disseminating results. The relationship with the host organization or practice setting is emphasized throughout.
ames Kirby and Morgan Sonderegger
- Published in print:
- 2018
- Published Online:
- January 2019
- ISBN:
- 9780226562452
- eISBN:
- 9780226562599
- Item type:
- chapter
- Publisher:
- University of Chicago Press
- DOI:
- 10.7208/chicago/9780226562599.003.0011
- Subject:
- Linguistics, Phonetics / Phonology
Statistical and empirical methods are in widespread use in present-day phonological research. Researchers are often interested in the problem of model selection, or determining whether or not a ...
More
Statistical and empirical methods are in widespread use in present-day phonological research. Researchers are often interested in the problem of model selection, or determining whether or not a particular term in a model is statistically significant, in order to make a judgement about whether or not that term is theoretically significant. If a term is not significant, it is often tempting to conclude that it is not relevant. However, such inferences require an assessment of statistical power, a dimension independent from significance. Assessing power is more difficult than assessing significance because it depends on factors including the true (or expected) effect size, sample size, and degree of noise. In this paper, we provide a non-technical introduction to the issue of power, illustrated with simulations based on experimental investigations of incomplete neutralization, to illustrate how not all null results are equally informative. In particular, depending on the statistical power, a non-significant result can either be uninformative, or reasonably interpreted as providing evidence consistent with a small or zero effect.Less
Statistical and empirical methods are in widespread use in present-day phonological research. Researchers are often interested in the problem of model selection, or determining whether or not a particular term in a model is statistically significant, in order to make a judgement about whether or not that term is theoretically significant. If a term is not significant, it is often tempting to conclude that it is not relevant. However, such inferences require an assessment of statistical power, a dimension independent from significance. Assessing power is more difficult than assessing significance because it depends on factors including the true (or expected) effect size, sample size, and degree of noise. In this paper, we provide a non-technical introduction to the issue of power, illustrated with simulations based on experimental investigations of incomplete neutralization, to illustrate how not all null results are equally informative. In particular, depending on the statistical power, a non-significant result can either be uninformative, or reasonably interpreted as providing evidence consistent with a small or zero effect.
Zoltan Dienes
- Published in print:
- 2015
- Published Online:
- June 2015
- ISBN:
- 9780199688890
- eISBN:
- 9780191801785
- Item type:
- chapter
- Publisher:
- Oxford University Press
- DOI:
- 10.1093/acprof:oso/9780199688890.003.0012
- Subject:
- Psychology, Cognitive Psychology, Developmental Psychology
Inferring that a mental state is unconscious often rests on affirming a null hypothesis. For example, for perception to be below an objective threshold, discrimination about stimulus properties must ...
More
Inferring that a mental state is unconscious often rests on affirming a null hypothesis. For example, for perception to be below an objective threshold, discrimination about stimulus properties must be at chance. Similarly, for perception to be below a subjective threshold by the zero correlation criterion, the ability to discriminate one’s own accuracy must be at chance. However, a non-significant result in itself does not mean there is evidence for the null hypothesis; a non-significant result may just mean that the data are insensitive. Orthodox statistics does not provide a practical solution to this problem. Bayes factors provide a simple solution. The solution is vital for progress in the field, as so many claims that mental states are unconscious have relied on non-significant results with no indication of whether that outcome is due purely to data insensitivity.Less
Inferring that a mental state is unconscious often rests on affirming a null hypothesis. For example, for perception to be below an objective threshold, discrimination about stimulus properties must be at chance. Similarly, for perception to be below a subjective threshold by the zero correlation criterion, the ability to discriminate one’s own accuracy must be at chance. However, a non-significant result in itself does not mean there is evidence for the null hypothesis; a non-significant result may just mean that the data are insensitive. Orthodox statistics does not provide a practical solution to this problem. Bayes factors provide a simple solution. The solution is vital for progress in the field, as so many claims that mental states are unconscious have relied on non-significant results with no indication of whether that outcome is due purely to data insensitivity.
