Joaquin Quiñonero-Candela, Masashi Sugiyama, Anton Schwaighofer, and Neil D. Lawrence (eds)
- Published in print:
- 2008
- Published Online:
- August 2013
- ISBN:
- 9780262170055
- eISBN:
- 9780262255103
- Item type:
- book
- Publisher:
- The MIT Press
- DOI:
- 10.7551/mitpress/9780262170055.001.0001
- Subject:
- Computer Science, Machine Learning
Dataset shift is a common problem in predictive modeling that occurs when the joint distribution of inputs and outputs differs between training and test stages. Covariate shift, a particular case of ...
More
Dataset shift is a common problem in predictive modeling that occurs when the joint distribution of inputs and outputs differs between training and test stages. Covariate shift, a particular case of dataset shift, occurs when only the input distribution changes. Dataset shift is present in most practical applications, for reasons ranging from the bias introduced by experimental design to the irreproducibility of the testing conditions at training time. (An example is email spam filtering, which may fail to recognize spam that differs in form from the spam the automatic filter has been built on.) Despite this, and despite the attention given to the apparently similar problems of semi-supervised learning and active learning, dataset shift has received relatively little attention in the machine learning community until recently. This book offers an overview of current efforts to deal with dataset and covariate shift. The chapters offer a mathematical and philosophical introduction to the problem; place dataset shift in relationship to transfer learning, transduction, local learning, active learning, and semi-supervised learning; provide theoretical views of dataset and covariate shift (including decision theoretic and Bayesian perspectives); and present algorithms for covariate shift.Less
Dataset shift is a common problem in predictive modeling that occurs when the joint distribution of inputs and outputs differs between training and test stages. Covariate shift, a particular case of dataset shift, occurs when only the input distribution changes. Dataset shift is present in most practical applications, for reasons ranging from the bias introduced by experimental design to the irreproducibility of the testing conditions at training time. (An example is email spam filtering, which may fail to recognize spam that differs in form from the spam the automatic filter has been built on.) Despite this, and despite the attention given to the apparently similar problems of semi-supervised learning and active learning, dataset shift has received relatively little attention in the machine learning community until recently. This book offers an overview of current efforts to deal with dataset and covariate shift. The chapters offer a mathematical and philosophical introduction to the problem; place dataset shift in relationship to transfer learning, transduction, local learning, active learning, and semi-supervised learning; provide theoretical views of dataset and covariate shift (including decision theoretic and Bayesian perspectives); and present algorithms for covariate shift.
Cyril Goutte, Nicola Cancedda, Marc Dymetman, and George Foster (eds)
- Published in print:
- 2008
- Published Online:
- August 2013
- ISBN:
- 9780262072977
- eISBN:
- 9780262255097
- Item type:
- book
- Publisher:
- The MIT Press
- DOI:
- 10.7551/mitpress/9780262072977.001.0001
- Subject:
- Computer Science, Machine Learning
The Internet gives us access to a wealth of information in languages we don’t understand. The investigation of automated or semi-automated approaches to translation has become a thriving research ...
More
The Internet gives us access to a wealth of information in languages we don’t understand. The investigation of automated or semi-automated approaches to translation has become a thriving research field with enormous commercial potential. This book investigates how Machine Learning techniques can improve Statistical Machine Translation, currently at the forefront of research in the field. It looks first at enabling technologies—technologies that solve problems which are not Machine Translation proper but are linked closely to the development of a Machine Translation system. These include the acquisition of bilingual sentence-aligned data from comparable corpora, automatic construction of multilingual name dictionaries, and word alignment. The book then presents new or improved statistical Machine Translation techniques, including a discriminative training framework for leveraging syntactic information, the use of semi-supervised and kernel-based learning methods, and the combination of multiple Machine Translation outputs in order to improve overall translation quality.Less
The Internet gives us access to a wealth of information in languages we don’t understand. The investigation of automated or semi-automated approaches to translation has become a thriving research field with enormous commercial potential. This book investigates how Machine Learning techniques can improve Statistical Machine Translation, currently at the forefront of research in the field. It looks first at enabling technologies—technologies that solve problems which are not Machine Translation proper but are linked closely to the development of a Machine Translation system. These include the acquisition of bilingual sentence-aligned data from comparable corpora, automatic construction of multilingual name dictionaries, and word alignment. The book then presents new or improved statistical Machine Translation techniques, including a discriminative training framework for leveraging syntactic information, the use of semi-supervised and kernel-based learning methods, and the combination of multiple Machine Translation outputs in order to improve overall translation quality.
Masashi Sugiyama and Motoaki Kawanabe
- Published in print:
- 2012
- Published Online:
- September 2013
- ISBN:
- 9780262017091
- eISBN:
- 9780262301220
- Item type:
- book
- Publisher:
- The MIT Press
- DOI:
- 10.7551/mitpress/9780262017091.001.0001
- Subject:
- Computer Science, Machine Learning
As the power of computing has grown over the past few decades, the field of machine learning has advanced rapidly in both theory and practice. Machine learning methods are usually based on the ...
More
As the power of computing has grown over the past few decades, the field of machine learning has advanced rapidly in both theory and practice. Machine learning methods are usually based on the assumption that the data generation mechanism does not change over time. Yet real-world applications of machine learning, including image recognition, natural language processing, speech recognition, robot control, and bioinformatics, often violate this common assumption. Dealing with non-stationarity is one of modern machine learning’s greatest challenges. This book focuses on a specific non-stationary environment known as covariate shift, in which the distributions of inputs (queries) change but the conditional distribution of outputs (answers) is unchanged, and presents machine learning theory, algorithms, and applications to overcome this variety of non-stationarity. After reviewing the state-of-the-art research in the field, the book discusses topics that include learning under covariate shift, model selection, importance estimation, and active learning. It describes such real-world applications of covariate shift adaption as brain-computer interface, speaker identification, and age prediction from facial images.Less
As the power of computing has grown over the past few decades, the field of machine learning has advanced rapidly in both theory and practice. Machine learning methods are usually based on the assumption that the data generation mechanism does not change over time. Yet real-world applications of machine learning, including image recognition, natural language processing, speech recognition, robot control, and bioinformatics, often violate this common assumption. Dealing with non-stationarity is one of modern machine learning’s greatest challenges. This book focuses on a specific non-stationary environment known as covariate shift, in which the distributions of inputs (queries) change but the conditional distribution of outputs (answers) is unchanged, and presents machine learning theory, algorithms, and applications to overcome this variety of non-stationarity. After reviewing the state-of-the-art research in the field, the book discusses topics that include learning under covariate shift, model selection, importance estimation, and active learning. It describes such real-world applications of covariate shift adaption as brain-computer interface, speaker identification, and age prediction from facial images.
Olivier Chapelle, Bernhard Scholkopf, and Alexander Zien (eds)
- Published in print:
- 2006
- Published Online:
- August 2013
- ISBN:
- 9780262033589
- eISBN:
- 9780262255899
- Item type:
- book
- Publisher:
- The MIT Press
- DOI:
- 10.7551/mitpress/9780262033589.001.0001
- Subject:
- Computer Science, Machine Learning
In the field of machine learning, semi-supervised learning (SSL) occupies the middle ground, between supervised learning (in which all training examples are labeled) and unsupervised learning (in ...
More
In the field of machine learning, semi-supervised learning (SSL) occupies the middle ground, between supervised learning (in which all training examples are labeled) and unsupervised learning (in which no label data are given). Interest in SSL has increased in recent years, particularly because of application domains in which unlabeled data are plentiful, such as images, text, and bioinformatics. This overview of SSL presents state-of-the-art algorithms, a taxonomy of the field, selected applications, benchmark experiments, and perspectives on ongoing and future research. It first presents the key assumptions and ideas underlying the field: smoothness, cluster or low-density separation, manifold structure, and transduction. The core of the book is the presentation of SSL methods, organized according to algorithmic strategies. After an examination of generative models, the book describes algorithms that implement the low-density separation assumption, graph-based methods, and algorithms which perform two-step learning. It then discusses SSL applications and offers guidelines for SSL practitioners by analyzing the results of benchmark experiments. Finally, the book looks at interesting directions for SSL research. It closes with a discussion of the relationship between semi-supervised learning and transduction.Less
In the field of machine learning, semi-supervised learning (SSL) occupies the middle ground, between supervised learning (in which all training examples are labeled) and unsupervised learning (in which no label data are given). Interest in SSL has increased in recent years, particularly because of application domains in which unlabeled data are plentiful, such as images, text, and bioinformatics. This overview of SSL presents state-of-the-art algorithms, a taxonomy of the field, selected applications, benchmark experiments, and perspectives on ongoing and future research. It first presents the key assumptions and ideas underlying the field: smoothness, cluster or low-density separation, manifold structure, and transduction. The core of the book is the presentation of SSL methods, organized according to algorithmic strategies. After an examination of generative models, the book describes algorithms that implement the low-density separation assumption, graph-based methods, and algorithms which perform two-step learning. It then discusses SSL applications and offers guidelines for SSL practitioners by analyzing the results of benchmark experiments. Finally, the book looks at interesting directions for SSL research. It closes with a discussion of the relationship between semi-supervised learning and transduction.
