Search Results

This chapter begins with a discussion of psychometric aspects of neuropsychological tests. It then discusses the classical test theory model, criterion-referenced testing, probabilistic test models, individual case classification, psychometric single-case analysis, and comparison of a single case to a small control or normative sample.

Unobservable characters called constructs or latent traits are frequently measured by social scientists though certain educational and psychological tests. It is observed in this chapter that measuring the observable characteristics is not as deterministic as measuring physical characteristics and that measurement error exists. In this first chapter, general concepts are presented. Reliability is defined as “the extent to which test scores are consistent with another set of test scores produced from a similar process.” The measurement procedure includes all aspects of testing the situation. Sampling can be done by the sampling of one or more aspects of the measurement procedure or the sampling of examinees. The classical test theory and the generalizability theory are also presented. Concepts under reliability are discussed in more detail in the succeeding chapters.

Absolutely no judgment or evaluation should be included when reporting a reliability analysis. Estimates should speak for themselves and it is the task of the readers to judge their values. The discussion should focus on limitations of analysis. In this chapter, example discussions for the Benchmark Assessment of English Language Arts (ELA), the Palmetto Achievement Challenge Test (PACT) of mathematics, and the MSCAN are provided. Several books for future reading are recommended. These include books on classical test theory. Some of the recommended book include Theory of Mental Tests by Gulliksen, Statistical Theories of Mental Test Scores by Lord and Novick, and Reliability for the Social Sciences: Theory and Applications by Traub. The chapter also advises that reading be done on original references so that one can read the original words of the author. Other references are given after this chapter.

This chapter provides a theoretical overview of reliability, as well as pragmatic considerations in establishing different types of reliability. To illustrate key points, it draws from two scales: the Family Responsibility Scale and the Parental Self-Care Scale. Various forms of reliability are addressed, including interrater, test-retest, and internal consistency. Guidelines for interpreting reliability coefficients for clinical and research purposes are provided, including computation of stratified alpha for multidimensional measures. Computation of the standard error of measurement (SEM) is illustrated. The chapter concludes by asserting that a solid reliability coefficient is indispensable as a primary principle in assessing the quality of scores from a scale or test.

Item response theory (IRT) models show the relationship between the ability or trait (θ) measured by the instrument and an item response. Both the IRT and classical test theory (CTT) indices and how they are related in terms of item parameters, conceptualization of reliability and standard error of measurement, and population invariance are described. The IRT indices are more readily understood in the context of the formal mathematical models for describing the item response probabilities, in which these models are discussed. The IRT indices are introduced only in general terms. The three-parameter logistic (3PL), two-parameter logistic (2PL), and one-parameter logistic (1PL) dichotomous models are presented, along with the Graded Response (GR) and Generalized Partial Credit (GPC) polytomous models.

Reliability is the degree to which measurements are free from error, making reliability inversely related to error. Reliability goes hand-in-hand with validity, as any reputable introductory research methods book will emphasize that reliability is a condition for validity. The amount of literature on the topic of reliability is further suggestive of its importance as the size of this literature dwarfs that of validity. Reliability and measurement error mirror each other. Understanding reliability requires understanding measurement error. In this chapter, the concept of measurement error is introduced from the perspective of classical test theory (CTT). The focus is on CTT because it serves as the foundation of reliability in social work research. An overview of the domain sampling model and the various types and sources of measurement error are then discussed, followed by different methods for assessing reliability. The chapter concludes with an overview of generalizability theory and item response theory, which are also used to conceptualize and evaluate measurement error.

Limited choices exist for assessing the signed language development of deaf and hard of hearing children. Over the past 30 years, the American Sign Language Assessment Instrument (ASLAI) has been one of the top choices for norm-referenced assessment of deaf and hard of hearing children who use American Sign Language. Signed language assessments can also be used to evaluate the effects of a phenomenon known as language deprivation, which tends to affect deaf children. They can also measure the effects of impoverished and idiosyncratic nonstandard signs and grammar used by educators of the deaf and professionals who serve the Deaf community. This chapter discusses what was learned while developing the ASLAI and provides guidelines for educators and researchers of the deaf who seek to develop their own signed language assessments.