Search Results

This chapter takes on the problem of model adequacy and makes an argument for reformulating the way model-based statistical inference is carried out. In the new formulation, it does not treat the model as “truth.” It is instead an approximation to truth. Rather than testing for model fit, an integral part of the proposed statistical analysis is to assess the degree to which the model provides adequate answers to the statistical questions being posed. One method for doing so is to create a single overall measure of inadequacy that evaluates the degree of departure between the model and truth. The chapter argues that there are two components of errors in any statistical analysis. One component is due to model misspecification; that is, the working model is different from the true data-generating process. The chapter compares confidence intervals on model misspecification error with external knowledge of the scientific relevance of prediction variability to address the issue of scientific significance. The chapter also analyzes several familiar measures of statistical distances in terms of their possible use as inadequacy measures.

Robust prediction under model misspecification focuses on the important topic of how to ensure unbiased prediction even when the assumed population model is not precisely specified. The general role of sample balance in ensuring this unbiasedness is explored in the context of the homogeneous and the ratio population models, and the problem of maintaining a suitable trade-off between prediction efficiency under a working model and unbiasedness under alternative population models is discussed. A general result that provides the necessary conditions for both unbiasedness and efficiency is provided and the extension of balanced sampling to the clustered population model is discussed. A misspecification-robust alternative to balanced sampling is flexible estimation, and the chapter concludes with a development of finite population prediction based on a non-parametric regression fit to the sample data.

Using transformations in sample survey inference is the final chapter of this book and describes the extension of the empirical best prediction approach to the situation where the population values of interest do not follow a linear model in their original scale of measurement, but can be transformed so that this is the case. In particular, it focuses on the situation where the logarithm of the survey variable can be modelled linearly, and develops methodology for correcting the transformation biases of empirical best predictors of the population mean. The logarithmic transformation is particularly useful when there are outliers in the data, and outlier robust versions of these predictors are developed. Empirical results based on actual business survey data are used to demonstrate the efficacy of the transformation-based predictors. Both estimation and sample design issues caused by model-misspecification in the transformed scale are also discussed.

This chapter studies adaptive learning with multiple models. An agent is aware of potential model misspecification, and tries to detect it, in realtime, using an econometric specification test. If the current model passes the test, it is used to construct an optimal policy. If it fails the test, a new model is randomly selected from a fixed set of models. As the rate of coefficient updating decreases, one model becomes dominant, and is used ‘almost always’. Dominant models can be characterized using the tools of large deviations theory. The analysis is applied to a standard cobweb model.

In this chapter, we develop theory and methodology to support mode adjustment and hindcasting/forecasting in the presence of different possible mode effect types, including additive effects and odds-multiplicative effects. Mode adjustment is particularly important if the ultimate goal is to report one aggregate estimate of response parameters, and also to allow for comparison to historical surveys performed with different modes. Effect type has important consequences for inferential validity when the baseline response changes over time (i.e. when there is a time trend or time effect). We present a methodology to provide inference for additive and odds-multiplicative effect types, and demonstrate its performance in a simulation study. We also show that if the wrong effect type is assumed, the resulting inference can be invalid as confidence interval coverage is greatly reduced and estimates can be biased.

For almost 30 years, Lande and Arnold's approximation of individual fitness surfaces through multiple regression has provided a common framework for comparing the strength and form of phenotypic selection across traits, fitness components and sexes. This chapter provides an overview of the statistical and geometric approaches available for the multivariate analysis of phenotypic selection that build upon the Lande and Arnold approach. First, it details least squares based approaches for the estimation of multivariate selection in a single population. Second, it shows how these approaches can be extended for the statistical comparison of individual fitness surfaces among groups such as populations or experimental treatments, addressing the inferential differences between analyses of randomly chosen groups versus situations in which groups are experimentally fixed. In each case, it points out known issues and caveats associated with the approaches. Finally, using case studies, the chapter shows how these estimates of multivariate selection can be integrated with quantitative genetic analyses to better understand issues such as the maintenance of genetic variance under selection and how genetic constraints can bias evolutionary responses to selection.

This chapter provides an introduction to covariate shift adaptation toward machine learning in a non-stationary environment. It begins by discussing cover machine learning under covariate shift. It then describes the core idea of covariate shift adaptation, using an illustrative example. Next, it formulates the supervised learning problem, which includes regression and classification. It pays particular attention to covariate shift and model misspecification. An overview of the subsequent chapters is also presented.