Jie W Weiss and David J Weiss
- Published in print:
- 2008
- Published Online:
- January 2009
- ISBN:
- 9780195322989
- eISBN:
- 9780199869206
- Item type:
- chapter
- Publisher:
- Oxford University Press
- DOI:
- 10.1093/acprof:oso/9780195322989.003.0019
- Subject:
- Psychology, Cognitive Psychology
An important element in using evidence to select therapy is the determination of whether a treatment is clinically superior to its competitors. This chapter argues that the determination is a ...
More
An important element in using evidence to select therapy is the determination of whether a treatment is clinically superior to its competitors. This chapter argues that the determination is a decision requiring both kinds of information that are necessary in decision analysis: the probabilities and values associated with the possible outcomes. It is debatable whether significance tests answer questions about probabilities in a form suitable for decision making. But significance tests cannot answer questions about the comparative values of different treatments. The preferable option is the one with the highest expected utility, where expected utility is the product of probability times utility.Less
An important element in using evidence to select therapy is the determination of whether a treatment is clinically superior to its competitors. This chapter argues that the determination is a decision requiring both kinds of information that are necessary in decision analysis: the probabilities and values associated with the possible outcomes. It is debatable whether significance tests answer questions about probabilities in a form suitable for decision making. But significance tests cannot answer questions about the comparative values of different treatments. The preferable option is the one with the highest expected utility, where expected utility is the product of probability times utility.
Ezra Susser, Sharon Schwartz, Alfredo Morabia, and Evelyn J. Bromet
- Published in print:
- 2006
- Published Online:
- September 2009
- ISBN:
- 9780195101812
- eISBN:
- 9780199864096
- Item type:
- chapter
- Publisher:
- Oxford University Press
- DOI:
- 10.1093/acprof:oso/9780195101812.003.23
- Subject:
- Public Health and Epidemiology, Public Health, Epidemiology
Statistical significance testing can be viewed as a formal method for assessing the evidence with regard to a scientific conjecture. It reflects the degree to which we can be confident that an ...
More
Statistical significance testing can be viewed as a formal method for assessing the evidence with regard to a scientific conjecture. It reflects the degree to which we can be confident that an association observed in the study sample is also present in the target population. Significance tests play a central role in psychiatric epidemiology, and in fact across the full gamut of human research. Given the omnipresence of statistical significance tests, it is essential for all investigators to understand the principles behind these tests and guidelines for their interpretation. This chapter provides this background and makes recommendations for sensible evaluation of test results.Less
Statistical significance testing can be viewed as a formal method for assessing the evidence with regard to a scientific conjecture. It reflects the degree to which we can be confident that an association observed in the study sample is also present in the target population. Significance tests play a central role in psychiatric epidemiology, and in fact across the full gamut of human research. Given the omnipresence of statistical significance tests, it is essential for all investigators to understand the principles behind these tests and guidelines for their interpretation. This chapter provides this background and makes recommendations for sensible evaluation of test results.
Theodore M. Porter
- Published in print:
- 2009
- Published Online:
- February 2010
- ISBN:
- 9780199546350
- eISBN:
- 9780191720048
- Item type:
- chapter
- Publisher:
- Oxford University Press
- DOI:
- 10.1093/acprof:oso/9780199546350.003.0014
- Subject:
- Business and Management, Organization Studies, Finance, Accounting, and Banking
The standing of accounting among the academic disciplines has never been very high, in part because the work of accounting is not regarded as suitably creative. Yet when we think of knowledge as a ...
More
The standing of accounting among the academic disciplines has never been very high, in part because the work of accounting is not regarded as suitably creative. Yet when we think of knowledge as a set of institutionalized practices, the subservience of accounting to the sciences and to the ancient professions may be reversed. Fields like economics, engineering, biology, and medicine cannot escape the twin imperatives of commensuration and accountability, especially when these are brought to bear on matters of recognized public importance. In this regard, the career of cost-benefit quantification is exemplary. Knowledge and rationality, whenever they touch on politics and policy, have become closely bound up with a logic of accountancy.Less
The standing of accounting among the academic disciplines has never been very high, in part because the work of accounting is not regarded as suitably creative. Yet when we think of knowledge as a set of institutionalized practices, the subservience of accounting to the sciences and to the ancient professions may be reversed. Fields like economics, engineering, biology, and medicine cannot escape the twin imperatives of commensuration and accountability, especially when these are brought to bear on matters of recognized public importance. In this regard, the career of cost-benefit quantification is exemplary. Knowledge and rationality, whenever they touch on politics and policy, have become closely bound up with a logic of accountancy.
J. H. Abramson and Z. H. Abramson
- Published in print:
- 2001
- Published Online:
- September 2009
- ISBN:
- 9780195145250
- eISBN:
- 9780199864775
- Item type:
- chapter
- Publisher:
- Oxford University Press
- DOI:
- 10.1093/acprof:oso/9780195145250.003.0004
- Subject:
- Public Health and Epidemiology, Public Health, Epidemiology
The exercises in this section deal with the assessment of associations between variables, with special reference to possible effects of shortcomings in the study methods, the strength and other ...
More
The exercises in this section deal with the assessment of associations between variables, with special reference to possible effects of shortcomings in the study methods, the strength and other qualities of the association, its consistency, possible confounding effects, and the appraisal of causality. Specific topics include effects of misclassification, statistical significance and significance tests, methods of appraising the possibility and likely direction of confounding effects, measures of the strength of associations, synergism, the appraisal of associations in stratified data, the use of multivariate analysis (including multiple linear regression, multiple logistic regression analysis, and proportional hazards regression), the use of matched samples, synergism, risk factors and risk markers, and the appraisal of risk markers. A self-test concludes the section.Less
The exercises in this section deal with the assessment of associations between variables, with special reference to possible effects of shortcomings in the study methods, the strength and other qualities of the association, its consistency, possible confounding effects, and the appraisal of causality. Specific topics include effects of misclassification, statistical significance and significance tests, methods of appraising the possibility and likely direction of confounding effects, measures of the strength of associations, synergism, the appraisal of associations in stratified data, the use of multivariate analysis (including multiple linear regression, multiple logistic regression analysis, and proportional hazards regression), the use of matched samples, synergism, risk factors and risk markers, and the appraisal of risk markers. A self-test concludes the section.
David A. Savitz
- Published in print:
- 2003
- Published Online:
- September 2009
- ISBN:
- 9780195108408
- eISBN:
- 9780199865765
- Item type:
- chapter
- Publisher:
- Oxford University Press
- DOI:
- 10.1093/acprof:oso/9780195108408.003.0010
- Subject:
- Public Health and Epidemiology, Public Health, Epidemiology
This chapter discusses random error. Topics covered include taking a sequential approach to considering random and systematic error, special considerations in evaluating random error in observational ...
More
This chapter discusses random error. Topics covered include taking a sequential approach to considering random and systematic error, special considerations in evaluating random error in observational studies, statistical significance testing, multiple comparisons and related issues, interpretation of confidence intervals, and integrated assessment of random error.Less
This chapter discusses random error. Topics covered include taking a sequential approach to considering random and systematic error, special considerations in evaluating random error in observational studies, statistical significance testing, multiple comparisons and related issues, interpretation of confidence intervals, and integrated assessment of random error.
Bradley E. Alger
- Published in print:
- 2019
- Published Online:
- February 2021
- ISBN:
- 9780190881481
- eISBN:
- 9780190093761
- Item type:
- chapter
- Publisher:
- Oxford University Press
- DOI:
- 10.1093/oso/9780190881481.003.0005
- Subject:
- Neuroscience, Techniques
This chapter covers statistics from a conceptual rather than a quantitative perspective. The objective is to distinguish statistical from scientific hypotheses and provide background for later ...
More
This chapter covers statistics from a conceptual rather than a quantitative perspective. The objective is to distinguish statistical from scientific hypotheses and provide background for later discussions. The fact that scientific and statistical hypotheses are dissimilar is often overlooked, and the chapter shows how they differ and why the differences matter. The chapter uses simple, nontechnical examples to present the main ideas in frequentist statistics; the next chapter covers the basics of Bayesian statistics and explores how the divergent philosophical viewpoints of frequentists and Bayesians affect scientific reasoning. The present chapter covers statistical error and its role in null hypothesis significance testing, which is still the dominant mode of scientific significance testing, as well as the origins, weaknesses of, and alternatives to, null-hypothesis testing. A philosophical objective of the chapter is to show how the concept of probability fits with Karl Popper’s program of falsification. The background material in this chapter will be especially important for evaluating the Reproducibility Crisis in Chapter 7.Less
This chapter covers statistics from a conceptual rather than a quantitative perspective. The objective is to distinguish statistical from scientific hypotheses and provide background for later discussions. The fact that scientific and statistical hypotheses are dissimilar is often overlooked, and the chapter shows how they differ and why the differences matter. The chapter uses simple, nontechnical examples to present the main ideas in frequentist statistics; the next chapter covers the basics of Bayesian statistics and explores how the divergent philosophical viewpoints of frequentists and Bayesians affect scientific reasoning. The present chapter covers statistical error and its role in null hypothesis significance testing, which is still the dominant mode of scientific significance testing, as well as the origins, weaknesses of, and alternatives to, null-hypothesis testing. A philosophical objective of the chapter is to show how the concept of probability fits with Karl Popper’s program of falsification. The background material in this chapter will be especially important for evaluating the Reproducibility Crisis in Chapter 7.
Julia H. Littell, Jacqueline Corcoran, and Vijayan Pillai
- Published in print:
- 2008
- Published Online:
- January 2009
- ISBN:
- 9780195326543
- eISBN:
- 9780199864959
- Item type:
- chapter
- Publisher:
- Oxford University Press
- DOI:
- 10.1093/acprof:oso/9780195326543.003.0001
- Subject:
- Social Work, Research and Evaluation
This chapter defines and compares systematic reviews and meta-analyses. Systematic reviews carefully locate relevant evidence and appraise study qualities, while meta-analyses provide quantitative ...
More
This chapter defines and compares systematic reviews and meta-analyses. Systematic reviews carefully locate relevant evidence and appraise study qualities, while meta-analyses provide quantitative summaries of evidence, showing central trends, variations, and possible explanations for differences in results across studies. The advantages and disadvantages of these review methodologies are discussed, along with appropriate applications and quality standards.Less
This chapter defines and compares systematic reviews and meta-analyses. Systematic reviews carefully locate relevant evidence and appraise study qualities, while meta-analyses provide quantitative summaries of evidence, showing central trends, variations, and possible explanations for differences in results across studies. The advantages and disadvantages of these review methodologies are discussed, along with appropriate applications and quality standards.
Brian D. Haig
- Published in print:
- 2018
- Published Online:
- January 2018
- ISBN:
- 9780190222055
- eISBN:
- 9780190871734
- Item type:
- chapter
- Publisher:
- Oxford University Press
- DOI:
- 10.1093/oso/9780190222055.003.0003
- Subject:
- Psychology, Social Psychology
Chapter 3 provides a brief overview of null hypothesis significance testing and points out its primary defects. It then outlines the neo-Fisherian account of tests of statistical significance, along ...
More
Chapter 3 provides a brief overview of null hypothesis significance testing and points out its primary defects. It then outlines the neo-Fisherian account of tests of statistical significance, along with a second option contained in the philosophy of statistics known as the error-statistical philosophy, both of which are defensible. Tests of statistical significance are the most widely used means for evaluating hypotheses and theories in psychology. A massive critical literature has developed in psychology, and the behavioral sciences more generally, regarding the worth of these tests. The chapter provides a list of important lessons learned from the ongoing debates about tests of significance.Less
Chapter 3 provides a brief overview of null hypothesis significance testing and points out its primary defects. It then outlines the neo-Fisherian account of tests of statistical significance, along with a second option contained in the philosophy of statistics known as the error-statistical philosophy, both of which are defensible. Tests of statistical significance are the most widely used means for evaluating hypotheses and theories in psychology. A massive critical literature has developed in psychology, and the behavioral sciences more generally, regarding the worth of these tests. The chapter provides a list of important lessons learned from the ongoing debates about tests of significance.
Kristin Shrader-Frechette
- Published in print:
- 2014
- Published Online:
- October 2014
- ISBN:
- 9780199396412
- eISBN:
- 9780199396436
- Item type:
- chapter
- Publisher:
- Oxford University Press
- DOI:
- 10.1093/acprof:oso/9780199396412.003.0008
- Subject:
- Philosophy, Philosophy of Science
The chapter evaluates the well-known statistical-significance rule for discovering hypotheses and shows that, because scientists routinely misuse this rule, especially with observational and not ...
More
The chapter evaluates the well-known statistical-significance rule for discovering hypotheses and shows that, because scientists routinely misuse this rule, especially with observational and not experimental data, they can miss discovering important causal hypotheses in statistics. In particular, scientists forget how subjective the rule is, and that the rule tells nothing about scientific significance, only mathematical significanceLess
The chapter evaluates the well-known statistical-significance rule for discovering hypotheses and shows that, because scientists routinely misuse this rule, especially with observational and not experimental data, they can miss discovering important causal hypotheses in statistics. In particular, scientists forget how subjective the rule is, and that the rule tells nothing about scientific significance, only mathematical significance
Marcel Boumans
- Published in print:
- 2015
- Published Online:
- May 2015
- ISBN:
- 9780199388288
- eISBN:
- 9780199388318
- Item type:
- chapter
- Publisher:
- Oxford University Press
- DOI:
- 10.1093/acprof:oso/9780199388288.003.0004
- Subject:
- Philosophy, Philosophy of Science
This chapter discusses the epistemological scope of field science by exploring the econometric methodology of Trygve Haavelmo. The theories of social field phenomena are inexact; they do not provide ...
More
This chapter discusses the epistemological scope of field science by exploring the econometric methodology of Trygve Haavelmo. The theories of social field phenomena are inexact; they do not provide the complete set of causes that affects the phenomenon. Haavelmo’s epistemology shows that a social field science explanation can never be made complete by statistics and statistical analysis only. In other words, a social field science cannot be a purely inductive science. Data observed “passively” might not display enough variations to reveal the relevant causal influences. Variation shows statistical significance, not potential influence. We also need theory. But because theory if fundamentally incomplete—inexact—we need additional sources of knowledge.Less
This chapter discusses the epistemological scope of field science by exploring the econometric methodology of Trygve Haavelmo. The theories of social field phenomena are inexact; they do not provide the complete set of causes that affects the phenomenon. Haavelmo’s epistemology shows that a social field science explanation can never be made complete by statistics and statistical analysis only. In other words, a social field science cannot be a purely inductive science. Data observed “passively” might not display enough variations to reveal the relevant causal influences. Variation shows statistical significance, not potential influence. We also need theory. But because theory if fundamentally incomplete—inexact—we need additional sources of knowledge.
Gerald P. Dwyer
- Published in print:
- 2016
- Published Online:
- August 2016
- ISBN:
- 9780198704324
- eISBN:
- 9780191773761
- Item type:
- chapter
- Publisher:
- Oxford University Press
- DOI:
- 10.1093/acprof:oso/9780198704324.003.0031
- Subject:
- Economics and Finance, Economic History, Macro- and Monetary Economics
Milton Friedman’s empirical research is very different in tone and substance to the research in a typical journal article. This chapter points out that the difference in tone and substance is ...
More
Milton Friedman’s empirical research is very different in tone and substance to the research in a typical journal article. This chapter points out that the difference in tone and substance is directly related to Friedman’s views on the foundations of statistics. Instead of viewing statistics as a classical statistician might, Friedman viewed statistics through the lens of personal probability. This had substantial implications for his overall research agenda and the particulars of his empirical work. Friedman emphasized agreement across various sets of data because such evidence was more likely to produce consensus among economists than tests on a single set of data. He put less emphasis on statistical significance of results, at least partly because of the problems involved in interpreting multiple tests on the same data.Less
Milton Friedman’s empirical research is very different in tone and substance to the research in a typical journal article. This chapter points out that the difference in tone and substance is directly related to Friedman’s views on the foundations of statistics. Instead of viewing statistics as a classical statistician might, Friedman viewed statistics through the lens of personal probability. This had substantial implications for his overall research agenda and the particulars of his empirical work. Friedman emphasized agreement across various sets of data because such evidence was more likely to produce consensus among economists than tests on a single set of data. He put less emphasis on statistical significance of results, at least partly because of the problems involved in interpreting multiple tests on the same data.
Dale W. Jorgenson, Richard J. Goettle, Mun S. Ho, and Peter J. Wilcoxen
- Published in print:
- 2014
- Published Online:
- September 2014
- ISBN:
- 9780262027090
- eISBN:
- 9780262318563
- Item type:
- chapter
- Publisher:
- The MIT Press
- DOI:
- 10.7551/mitpress/9780262027090.003.0009
- Subject:
- Economics and Finance, Development, Growth, and Environmental
This chapter describes a method for constructing confidence intervals for general equilibrium model results using the covariance matrices routinely computed in the course of estimating the model's ...
More
This chapter describes a method for constructing confidence intervals for general equilibrium model results using the covariance matrices routinely computed in the course of estimating the model's parameters. The method is then applied to IGEM to produce confidence intervals for several types of results: (i) the levels of variables in IGEM's base case, (ii) deviations in variables resulting from a policy experiment consisting of a carbon tax used to reduce the rate of tax on capital income, and (iii) the effects of the carbon tax swap on the components of social welfare. For each case, a decomposition analysis is used to identify the key uncertainties underlying the confidence interval. The carbon tax swap is shown to produce statistically significant gains in key variables, including social welfare, along with significant reductions in pollution. The method is also shown to have important advantages over alternative approaches. The confidence intervals it produces are very similar to those generated by Monte Carlo simulation but are much faster to compute for large models. In addition, the decomposition analysis shows that off-diagonal terms in the parameter covariance matrices often contribute significantly to the confidence intervals for key results. By capturing these effects, the method provides better characterization of uncertainty than alternative methodologies that rely on independent perturbations of parameters.Less
This chapter describes a method for constructing confidence intervals for general equilibrium model results using the covariance matrices routinely computed in the course of estimating the model's parameters. The method is then applied to IGEM to produce confidence intervals for several types of results: (i) the levels of variables in IGEM's base case, (ii) deviations in variables resulting from a policy experiment consisting of a carbon tax used to reduce the rate of tax on capital income, and (iii) the effects of the carbon tax swap on the components of social welfare. For each case, a decomposition analysis is used to identify the key uncertainties underlying the confidence interval. The carbon tax swap is shown to produce statistically significant gains in key variables, including social welfare, along with significant reductions in pollution. The method is also shown to have important advantages over alternative approaches. The confidence intervals it produces are very similar to those generated by Monte Carlo simulation but are much faster to compute for large models. In addition, the decomposition analysis shows that off-diagonal terms in the parameter covariance matrices often contribute significantly to the confidence intervals for key results. By capturing these effects, the method provides better characterization of uncertainty than alternative methodologies that rely on independent perturbations of parameters.