David L. Streiner and Geoffrey R. Norman
- Published in print:
- 2008
- Published Online:
- September 2009
- ISBN:
- 9780199231881
- eISBN:
- 9780191724015
- Item type:
- chapter
- Publisher:
- Oxford University Press
- DOI:
- 10.1093/acprof:oso/9780199231881.003.0009
- Subject:
- Public Health and Epidemiology, Public Health, Epidemiology
Generalizability theory is a powerful method for simultaneously assessing many factors that affect reliability. Its use has been limited by its conceptual complexity and the lack of user-friendly ...
More
Generalizability theory is a powerful method for simultaneously assessing many factors that affect reliability. Its use has been limited by its conceptual complexity and the lack of user-friendly software. This chapter first discusses G-theory, and then describes how to use a free program that does the actual calculations, allowing for the analysis of complicated reliability studies.Less
Generalizability theory is a powerful method for simultaneously assessing many factors that affect reliability. Its use has been limited by its conceptual complexity and the lack of user-friendly software. This chapter first discusses G-theory, and then describes how to use a free program that does the actual calculations, allowing for the analysis of complicated reliability studies.
J. Patrick Meyer
- Published in print:
- 2010
- Published Online:
- March 2012
- ISBN:
- 9780195380361
- eISBN:
- 9780199847914
- Item type:
- chapter
- Publisher:
- Oxford University Press
- DOI:
- 10.1093/acprof:oso/9780195380361.003.0001
- Subject:
- Psychology, Social Psychology
Unobservable characters called constructs or latent traits are frequently measured by social scientists though certain educational and psychological tests. It is observed in this chapter that ...
More
Unobservable characters called constructs or latent traits are frequently measured by social scientists though certain educational and psychological tests. It is observed in this chapter that measuring the observable characteristics is not as deterministic as measuring physical characteristics and that measurement error exists. In this first chapter, general concepts are presented. Reliability is defined as “the extent to which test scores are consistent with another set of test scores produced from a similar process.” The measurement procedure includes all aspects of testing the situation. Sampling can be done by the sampling of one or more aspects of the measurement procedure or the sampling of examinees. The classical test theory and the generalizability theory are also presented. Concepts under reliability are discussed in more detail in the succeeding chapters.Less
Unobservable characters called constructs or latent traits are frequently measured by social scientists though certain educational and psychological tests. It is observed in this chapter that measuring the observable characteristics is not as deterministic as measuring physical characteristics and that measurement error exists. In this first chapter, general concepts are presented. Reliability is defined as “the extent to which test scores are consistent with another set of test scores produced from a similar process.” The measurement procedure includes all aspects of testing the situation. Sampling can be done by the sampling of one or more aspects of the measurement procedure or the sampling of examinees. The classical test theory and the generalizability theory are also presented. Concepts under reliability are discussed in more detail in the succeeding chapters.
Patrick Meyer
- Published in print:
- 2010
- Published Online:
- March 2012
- ISBN:
- 9780195380361
- eISBN:
- 9780199847914
- Item type:
- book
- Publisher:
- Oxford University Press
- DOI:
- 10.1093/acprof:oso/9780195380361.001.0001
- Subject:
- Psychology, Social Psychology
This is a book in the Understanding Statistics series, which is designed to provide guides to understanding, presenting, and critiquing analyses and associated inferences. Each book in the series ...
More
This is a book in the Understanding Statistics series, which is designed to provide guides to understanding, presenting, and critiquing analyses and associated inferences. Each book in the series demonstrates how the relevant topic should be reported—including detail surrounding what can be said, and how it should be said, as well as drawing boundaries around what cannot appropriately be claimed or inferred. This volume addresses reliability, which is a fundamental aspect of any social science study that involves educational or psychological measurement. It not only has implications for the quality of test scores themselves, but also any statistical analysis conducted using those scores. Topics addressed in this book cover three different types of reliability methods and appropriate standard errors of measurement: classical test theory methods, decision consistency indices, and generalizability theory coefficients. After a brief introduction to the topic, the book outlines how to report reliability in professional journal articles. It includes examples of both good and bad write-ups for methods sections of journal articles.Less
This is a book in the Understanding Statistics series, which is designed to provide guides to understanding, presenting, and critiquing analyses and associated inferences. Each book in the series demonstrates how the relevant topic should be reported—including detail surrounding what can be said, and how it should be said, as well as drawing boundaries around what cannot appropriately be claimed or inferred. This volume addresses reliability, which is a fundamental aspect of any social science study that involves educational or psychological measurement. It not only has implications for the quality of test scores themselves, but also any statistical analysis conducted using those scores. Topics addressed in this book cover three different types of reliability methods and appropriate standard errors of measurement: classical test theory methods, decision consistency indices, and generalizability theory coefficients. After a brief introduction to the topic, the book outlines how to report reliability in professional journal articles. It includes examples of both good and bad write-ups for methods sections of journal articles.
Brian E. Perron and David F. Gillespie
- Published in print:
- 2015
- Published Online:
- April 2015
- ISBN:
- 9780199855483
- eISBN:
- 9780190235963
- Item type:
- chapter
- Publisher:
- Oxford University Press
- DOI:
- 10.1093/acprof:oso/9780199855483.003.0004
- Subject:
- Social Work, Research and Evaluation
Reliability is the degree to which measurements are free from error, making reliability inversely related to error. Reliability goes hand-in-hand with validity, as any reputable introductory research ...
More
Reliability is the degree to which measurements are free from error, making reliability inversely related to error. Reliability goes hand-in-hand with validity, as any reputable introductory research methods book will emphasize that reliability is a condition for validity. The amount of literature on the topic of reliability is further suggestive of its importance as the size of this literature dwarfs that of validity. Reliability and measurement error mirror each other. Understanding reliability requires understanding measurement error. In this chapter, the concept of measurement error is introduced from the perspective of classical test theory (CTT). The focus is on CTT because it serves as the foundation of reliability in social work research. An overview of the domain sampling model and the various types and sources of measurement error are then discussed, followed by different methods for assessing reliability. The chapter concludes with an overview of generalizability theory and item response theory, which are also used to conceptualize and evaluate measurement error.Less
Reliability is the degree to which measurements are free from error, making reliability inversely related to error. Reliability goes hand-in-hand with validity, as any reputable introductory research methods book will emphasize that reliability is a condition for validity. The amount of literature on the topic of reliability is further suggestive of its importance as the size of this literature dwarfs that of validity. Reliability and measurement error mirror each other. Understanding reliability requires understanding measurement error. In this chapter, the concept of measurement error is introduced from the perspective of classical test theory (CTT). The focus is on CTT because it serves as the foundation of reliability in social work research. An overview of the domain sampling model and the various types and sources of measurement error are then discussed, followed by different methods for assessing reliability. The chapter concludes with an overview of generalizability theory and item response theory, which are also used to conceptualize and evaluate measurement error.
