Search Results

This chapter explains the differences between primary and secondary data sets. It explains how secondary data sets are typically collected, the ability of large institutions to collect both sizable and representative data sets and their ability to use sophisticated sampling designs in their data collection, how these large data sets can utilize statistical techniques that may not be available to smaller data sets, and differences between cross-sectional and longitudinal data sets.

Software for cladistic analysis has been widely available for more than twenty years, and a series of advances made during this time have facilitated the analysis of matrices of ever-increasing size. This chapter provides an overview of the development of parsimony methods for cladistic analysis, describes strategies that have allowed large data matrices to be analysed by conventional methods, and in doing so, demonstrates that data sets historically considered intracable could in fact have been readily approached using then-available hardware and software. Preliminary analyses, even when unsuccessful at discovering most-parsimonious trees, can be used to identify appropriate software settings for use during thorough analyses. A useful indicator of the settings that yield the most efficient searches is the excess branch swapping ratio, which is the ratio between the number of tree rearrangements conducted during a particular phase of branch swapping in which shorter trees are being discovered, and the minimum possible number of rearrangements during this phase. It is concluded that two-stage search strategies, with intensive branch swapping conducted on a small percentage of the most optimal sets of trees obtained by a large number of relatively short searches, are more efficient than one-stage searches.

This chapter examines methodologies for dimensional analysis and linear decomposition of multivariate data sets, and discusses their implicit hypotheses and interpretations for muscle coordination of movement. It presents tutorials to compare how two common methods—principal components analysis (PCA) and non-negative matrix factorization (NMF)—decompose electromyographic signals into underlying components. To facilitate the integration of such mathematical techniques with physiological hypothesis testing, the chapter focuses on developing an intuitive understanding to the two techniques. It provides a simple two-dimensional tutorial, focusing on how orthogonality constraints in PCA and non-negativity constraints in NMF impact the resulting data decomposition and physiological relevance. Examples are presented using real data sets from human balance control and locomotion. The chapter examines the structure of the resulting components, their robustness across tasks, and their implications for various muscle synergy hypotheses. The chapter addresses practical issues and caveats in organizing datasets, the selection of the appropriate number of components, and considerations and pitfalls of experimental design and analysis, as well as offering suggestions and cautions for interpreting results. Based on these comparisons and on the work in the visual system over the last decade, evidence is presented for the increased neurophysiological relevance of the factors derived from NMF compared to PCA.

This chapter focuses on empirical orthogonal functions (EOFs). One of the most useful and common eigen-techniques in data analysis is the construction of EOFs. EOFs are a transform of the data; the original set of numbers is transformed into a different set with some desirable properties. In this sense the EOF transform is similar to other transforms, such as the Fourier or Laplace transforms. In all these cases, we project the original data onto a set of functions, thus replacing the original data with the set of projection coefficients on the chosen new set of basis vectors. However, the choice of the specific basis set varies from case to case. The discussions cover data matrix structure convention, reshaping multidimensional data sets for EOF analysis, forming anomalies and removing time mean, missing values, choosing and interpreting the covariability matrix, calculating the EOFs, projection time series, and extended EOF analysis.

This chapter discusses initial transformations that may be useful before embarking on a Procrustes analysis proper. Data-scaling and configuration-scaling are the terms adopted for the many kinds of transformations that may be deemed desirable before embarking on the actual Procrustes matching. The aim of these is to eliminate possible incommensurabilities of variables within the individual data sets (data-scaling) and size differences between data sets (configuration-scaling). Although some choices of data-scaling have clear justification, other choices are largely subjective. The form and source of the data should help decide what, if any, data-scaling is needed. Three main types of data are considered: (i) sets of coordinates derived from some form of multidimensional scaling (MDS) or, in the case of shape studies, directly measured landmark coordinates; (ii) data matrices whose columns refer to different variables; and (iii) sets of loadings derived from factor analysis.

This chapter examines factors to consider when thinking about using large secondary data sets. These include advantages of using such data, e.g., cost and time advantages; the ability to follow people over a long period of time as can be done with longitudinal data sets that already span a great number of decades; the packaging of the data into SAS, SPSS, STATA, and other formats, facilitating data programming and analysis; and the ability to do a number of research projects from a single data set. Some disadvantages include lack of control over the framing and wording of survey items, potentially small sample sizes for particular populations, and an inability to recontact the survey respondent. The chapter also presents a list of questions to determine whether using secondary data is both feasible and appropriate given the research study.

This chapter discusses how there is little of what economists call data on markets in Roman times, despite lots of information about prices and transactions. Data, as economists consider it, consist of a set of uniform prices that can be compared with each other. According to scholars, extensive markets existed in the late Roman Republic and early Roman Empire. Even though there is a lack of data, there are enough observations for the price of wheat, the most extensively traded commodity, to perform a test. The problem is that there is only a little bit of data by modern standards. Consequently, the chapter explains why statistics are useful in interpreting small data sets and how one deals with various problems that arise when there are only a few data points.

This chapter examines how the qualitative and quantitative research traditions treat within-case analysis versus cross-case analysis for causal inference. In qualitative research, the primary focus is on specific events and processes taking place within each individual case. Leading qualitative methodologies of hypothesis testing, such as process tracing and counterfactual analysis, are fundamentally methods of within-case analysis. By contrast, quantitative research traditionally involves exclusively cross-case comparison. The chapter begins with a comparison of the typical roles (or nonroles) of within-case and cross-case analysis in case studies versus experiments. It then considers how causal inference in quantitative and qualitative research is linked to the use of “data-set observations” and “causal-process observations,” respectively. It also explains the differences between process-tracing tests and statistical tests and concludes by suggesting that cross-case analysis and within-case analysis can and often should be combined.

This chapter analyses the time series properties of panel data sets, focusing on short panels. It discusses time effects and moving average covariances. It presents estimates of covariance structures and tests the permanent income hypothesis.

Science has become a data-intensive enterprise. Data sets are commonly being stored in public data repositories and are thus available for others to use in new, often unexpected ways. Such re-use of data sets can take the form of reproducing the original analysis, analyzing the data in new ways, or combining multiple data sets into new data sets that are analyzed still further. A scientist who re-uses a data set collected by another must be able to assess its trustworthiness. This chapter reviews the types of errors that are found in metadata referring to data collected manually, data collected by instruments (sensors), and data recovered from specimens in museum collections. It also summarizes methods used to screen these types of data for errors. It stresses the importance of ensuring that metadata associated with a data set thoroughly document the error prevention, detection, and correction methods applied to the data set prior to publication.

This chapter studies the history of the first generation of indicators of governmental institutional quality. These (international) indicators include labels such as ‘bureaucratic efficiency’ and ‘rule of law.’ This discussion also addresses the argument that it is the reversal, and not the creation, of indicators designed to justify large-scale development policies by leading multilateral agencies that is problematic. This chapter emphasizes the importance of using alternative data sets and making raw data easily available, in order to challenge the present assumptions instead of merely aiming to validate them and the policy choices with which they are associated with.

This chapter assesses the strengths and weaknesses of different semi-supervised learning (SSL) algorithms through inviting the authors of each chapter in this book to apply their algorithms to eight benchmark data sets. These data sets encompass both artificial and real-world problems. Details are provided on how the algorithms were applied, especially how hyperparameters were chosen given the few labeled points. Finally, the chapter concludes by presenting and discussing the empirical performance.

As citizen-electors and voters are different bodies of people, particularly in terms of the choice situation within which they express their preferences, we cannot simply assume that they are the same. They have to be compared explicitly. Democracy has to guarantee a necessary connection between public policy and citizen preferences, so we have in the first place to see how far voter and electoral preferences correspond. It cannot simply be assumed that they do. Comparing Eurobaromoter Left-Right self placements by electors with the positions stated out by voters in elections we find only limited correspondence in terms of election-by-election congruence and responsiveness but very substantial long term correspondence, especially between their mean positions. This gives an assurance that elections and voters do represent electors fairly over the long term.

This chapter focuses on the occurrence of Benford's law within the natural sciences, emphasizing that Benford's law is to be expected within many scientific data sets. This is a consequence of the reasonable assumption that a particular scientific process is scale invariant, or nearly scale invariant. The chapter reviews previous work from many fields showing a number of data sets that conform to Benford's law. In each case the underlying scale invariance, or mechanism that leads to scale invariance, is identified. Having established that Benford's law is to be expected for many data sets in the natural sciences, the second half of the chapter highlights generic potential applications of Benford's law. Finally, direct applications of Benford's law are highlighted, whereby the Benford distribution is used in a constructive way rather than simply assessing an already existing data set.

This chapter reviews Benford's law as it relates to detecting fraud and errors. It starts with an introduction and a review of selected parts of Benford's original 1938 paper, which shows the results of his analysis of 20,229 records from a total of 20 sets of data. Thereafter, four complaint data sets are reviewed, pertaining to interest received amounts from tax return data, census results from the 2000 United States census, results of an analysis of streamflow data, and results of analysis of accounts payable data. This is then followed by a discussion of several Benford analyses of fraudulent data related to accounts payable amounts, payroll data, and reported corporate numbers. The concluding discussion talks about these findings and sheds some light on when Benford's law might, or might not, detect fraud or errors.

This chapter provides a brief overview of Benford's law. It states Benford's law of digit bias and describes its history. The chapter then discusses the origins of Benford's law and gives numerous examples of data sets that follow this law, as well as some that do not. From these examples this chapter extracts several explanations as to the prevalence of Benford's law. Finally, the chapter closes with a quick summary of many of the diverse situations in which Benford's law holds, and why an observation that began in looking at the wear and tear in tables of logarithms has become a major tool in subjects as diverse as detecting tax fraud and building efficient computers.

The metadata that describe how scientific data are created and analyzed are typically limited to a general description of data sources, software used, and statistical tests applied and are presented in narrative form in the methods section of a scientific paper or a data set description. Recognizing that such narratives are usually inadequate to support reproduction of the analysis of the original work, a growing number of journals now require that authors also publish their data. However, finer-scale metadata that describe exactly how individual items of data were created and transformed and the processes by which this was done are rarely provided, even though such metadata have great potential to improve data set reliability. This chapter focuses on the detailed process metadata, called “data provenance,” required to ensure reproducibility of analyses and reliable re-use of the data.

Hurricane data originate from careful analysis of past storms by operational meteorologists. The data include estimates of the hurricane position and intensity at 6-hourly intervals. Information related to landfall time, local wind speeds, damages, and deaths, as well as cyclone size, are included. The data are archived by season. Some effort is needed to make the data useful for hurricane climate studies. In this chapter, we describe the data sets used throughout this book. We show you a work flow that includes importing, interpolating, smoothing, and adding attributes. We also show you how to create subsets of the data. Code in this chapter is more complicated and it can take longer to run. You can skip this material on first reading and continue with model building in Chapter 7. You can return here when you have an updated version of the data that includes the most recent years. Most statistical models in this book use the best-track data. Here we describe these data and provide original source material. We also explain how to smooth and interpolate them. Interpolations are needed for regional hurricane analyses. The best-track data set contains the 6-hourly center locations and intensities of all known tropical cyclones across the North Atlantic basin, including the Gulf of Mexico and Caribbean Sea. The data set is called HURDAT for HURricane DATa. It is maintained by the U.S. National Oceanic and Atmospheric Administration (NOAA) at the National Hurricane Center (NHC). Center locations are given in geographic coordinates (in tenths of degrees) and the intensities, representing the one-minute near-surface (∼10 m) wind speeds, are given in knots (1 kt = .5144 m s−1) and the minimum central pressures are given in millibars (1 mb = 1 hPa). The data are provided in 6-hourly intervals starting at 00 UTC (Universal Time Coordinate). The version of HURDAT file used here contains cyclones over the period 1851 through 2010 inclusive. Information on the history and origin of these data is found in Jarvinen et al (1984). The file has a logical structure that makes it easy to read with a FORTRAN program. Each cyclone contains a header record, a series of data records, and a trailer record.