Andrew J. Connolly, Jacob T. VanderPlas, Alexander Gray, Andrew J. Connolly, Jacob T. VanderPlas, and Alexander Gray
- Published in print:
- 2014
- Published Online:
- October 2017
- ISBN:
- 9780691151687
- eISBN:
- 9781400848911
- Item type:
- chapter
- Publisher:
- Princeton University Press
- DOI:
- 10.23943/princeton/9780691151687.003.0009
- Subject:
- Physics, Particle Physics / Astrophysics / Cosmology
Chapter 6 described techniques for estimating joint probability distributions from multivariate data sets and for identifying the inherent clustering within the properties of sources. This approach ...
More
Chapter 6 described techniques for estimating joint probability distributions from multivariate data sets and for identifying the inherent clustering within the properties of sources. This approach can be viewed as the unsupervised classification of data. If, however, we have labels for some of these data points (e.g., an object is tall, short, red, or blue) we can utilize this information to develop a relationship between the label and the properties of a source. We refer to this as supervised classification, which is the focus of this chapter. The motivation for supervised classification comes from the long history of classification in astronomy. Possibly the most well known of these classification schemes is that defined by Edwin Hubble for the morphological classification of galaxies based on their visual appearance. This chapter discusses generative classification, k-nearest-neighbor classifier, discriminative classification, support vector machines, decision trees, and evaluating classifiers.Less
Chapter 6 described techniques for estimating joint probability distributions from multivariate data sets and for identifying the inherent clustering within the properties of sources. This approach can be viewed as the unsupervised classification of data. If, however, we have labels for some of these data points (e.g., an object is tall, short, red, or blue) we can utilize this information to develop a relationship between the label and the properties of a source. We refer to this as supervised classification, which is the focus of this chapter. The motivation for supervised classification comes from the long history of classification in astronomy. Possibly the most well known of these classification schemes is that defined by Edwin Hubble for the morphological classification of galaxies based on their visual appearance. This chapter discusses generative classification, k-nearest-neighbor classifier, discriminative classification, support vector machines, decision trees, and evaluating classifiers.
Thomas P. Trappenberg
- Published in print:
- 2019
- Published Online:
- January 2020
- ISBN:
- 9780198828044
- eISBN:
- 9780191883873
- Item type:
- chapter
- Publisher:
- Oxford University Press
- DOI:
- 10.1093/oso/9780198828044.003.0003
- Subject:
- Neuroscience, Behavioral Neuroscience
This chapter’s goal is to show how to apply machine learning algorithms in a general setting using some classic methods. In particular, it demonstrates how to apply three important machine learning ...
More
This chapter’s goal is to show how to apply machine learning algorithms in a general setting using some classic methods. In particular, it demonstrates how to apply three important machine learning algorithms, a support vector classifier (SVC), a random forest classifier (RFC), and a multilayer perceptron (MLP). While many of the methods studied later go beyond these now classic methods, this does not mean that these methods are obsolete. Also, the algorithms discussed here provide some form of baseline to discuss advanced methods like probabilistic reasoning and deep learning. The aim here is to demonstrate that applying machine learning methods based on machine learning libraries is not very difficult. It offers an opportunity to discuss evaluation techniques that are very important in practice.Less
This chapter’s goal is to show how to apply machine learning algorithms in a general setting using some classic methods. In particular, it demonstrates how to apply three important machine learning algorithms, a support vector classifier (SVC), a random forest classifier (RFC), and a multilayer perceptron (MLP). While many of the methods studied later go beyond these now classic methods, this does not mean that these methods are obsolete. Also, the algorithms discussed here provide some form of baseline to discuss advanced methods like probabilistic reasoning and deep learning. The aim here is to demonstrate that applying machine learning methods based on machine learning libraries is not very difficult. It offers an opportunity to discuss evaluation techniques that are very important in practice.
Max A. Little
- Published in print:
- 2019
- Published Online:
- October 2019
- ISBN:
- 9780198714934
- eISBN:
- 9780191879180
- Item type:
- chapter
- Publisher:
- Oxford University Press
- DOI:
- 10.1093/oso/9780198714934.003.0006
- Subject:
- Mathematics, Logic / Computer Science / Mathematical Philosophy, Mathematical Physics
This chapter describes in detail how the main techniques of statistical machine learning can be constructed from the components described in earlier chapters. It presents these concepts in a way ...
More
This chapter describes in detail how the main techniques of statistical machine learning can be constructed from the components described in earlier chapters. It presents these concepts in a way which demonstrates how these techniques can be viewed as special cases of a more general probabilistic model which we fit to some data.Less
This chapter describes in detail how the main techniques of statistical machine learning can be constructed from the components described in earlier chapters. It presents these concepts in a way which demonstrates how these techniques can be viewed as special cases of a more general probabilistic model which we fit to some data.
Joachims Thorsten
- Published in print:
- 2006
- Published Online:
- August 2013
- ISBN:
- 9780262033589
- eISBN:
- 9780262255899
- Item type:
- chapter
- Publisher:
- The MIT Press
- DOI:
- 10.7551/mitpress/9780262033589.003.0006
- Subject:
- Computer Science, Machine Learning
This chapter discusses the transductive learning setting proposed by Vapnik where predictions are made only at a fixed number of known test points. Transductive support vector machines (TSVMs) ...
More
This chapter discusses the transductive learning setting proposed by Vapnik where predictions are made only at a fixed number of known test points. Transductive support vector machines (TSVMs) implement the idea of transductive learning by including test points in the computation of the margin. This chapter provides some examples for why the margin on the test examples can provide useful prior information for learning, in particular for the problem of text classification. The resulting optimization problems, however, are difficult to solve. The chapter reviews exact and approximate optimization methods and discusses their properties. Finally, the chapter discusses connections to other related semi-supervised learning approaches such as co-training and methods based on graph cuts, which can be seen as solving variants of the TSVM optimization problem.Less
This chapter discusses the transductive learning setting proposed by Vapnik where predictions are made only at a fixed number of known test points. Transductive support vector machines (TSVMs) implement the idea of transductive learning by including test points in the computation of the margin. This chapter provides some examples for why the margin on the test examples can provide useful prior information for learning, in particular for the problem of text classification. The resulting optimization problems, however, are difficult to solve. The chapter reviews exact and approximate optimization methods and discusses their properties. Finally, the chapter discusses connections to other related semi-supervised learning approaches such as co-training and methods based on graph cuts, which can be seen as solving variants of the TSVM optimization problem.
Masashi Sugiyama and Motoaki Kawanabe
- Published in print:
- 2012
- Published Online:
- September 2013
- ISBN:
- 9780262017091
- eISBN:
- 9780262301220
- Item type:
- chapter
- Publisher:
- The MIT Press
- DOI:
- 10.7551/mitpress/9780262017091.003.0002
- Subject:
- Computer Science, Machine Learning
This chapter discusses function learning methods under covariate shift. Ordinary empirical risk minimization learning is not consistent under covariate shift for misspecified models, and this ...
More
This chapter discusses function learning methods under covariate shift. Ordinary empirical risk minimization learning is not consistent under covariate shift for misspecified models, and this inconsistency issue can be resolved by considering importance-weighted loss functions. Here, various importance-weighted empirical risk minimization methods are introduced, including least squares and Huber’s method for regression, and Fisher discriminant analysis, logistic regression, support vector machines, and boosting for classification. Their adaptive and regularized variants are also described. The numerical behavior of these importance-weighted learning methods is illustrated through experiments.Less
This chapter discusses function learning methods under covariate shift. Ordinary empirical risk minimization learning is not consistent under covariate shift for misspecified models, and this inconsistency issue can be resolved by considering importance-weighted loss functions. Here, various importance-weighted empirical risk minimization methods are introduced, including least squares and Huber’s method for regression, and Fisher discriminant analysis, logistic regression, support vector machines, and boosting for classification. Their adaptive and regularized variants are also described. The numerical behavior of these importance-weighted learning methods is illustrated through experiments.
De Bie Tijl and Cristianini Nello
- Published in print:
- 2006
- Published Online:
- August 2013
- ISBN:
- 9780262033589
- eISBN:
- 9780262255899
- Item type:
- chapter
- Publisher:
- The MIT Press
- DOI:
- 10.7551/mitpress/9780262033589.003.0007
- Subject:
- Computer Science, Machine Learning
This chapter discusses an alternative approach that is based on a convex relaxation of the optimization problem associated with support vector machine transduction. The result is a semi-definite ...
More
This chapter discusses an alternative approach that is based on a convex relaxation of the optimization problem associated with support vector machine transduction. The result is a semi-definite programming (SDP) problem which can be optimized in polynomial time, the solution of which is an approximation of the optimal labeling as well as a bound on the true optimum of the original transduction objective function. To further decrease the computational complexity, this chapter proposes an approximation that allows solving transduction problems of up to 1,000 unlabeled samples. Finally, the formulation is extended to more general settings of semi-supervised learning, where equivalence and inequivalence constraints are given on labels of some of the samples.Less
This chapter discusses an alternative approach that is based on a convex relaxation of the optimization problem associated with support vector machine transduction. The result is a semi-definite programming (SDP) problem which can be optimized in polynomial time, the solution of which is an approximation of the optimal labeling as well as a bound on the true optimum of the original transduction objective function. To further decrease the computational complexity, this chapter proposes an approximation that allows solving transduction problems of up to 1,000 unlabeled samples. Finally, the formulation is extended to more general settings of semi-supervised learning, where equivalence and inequivalence constraints are given on labels of some of the samples.
Weston Jason, Leslie Christina, Ie Eugene, and Noble William Stafford
- Published in print:
- 2006
- Published Online:
- August 2013
- ISBN:
- 9780262033589
- eISBN:
- 9780262255899
- Item type:
- chapter
- Publisher:
- The MIT Press
- DOI:
- 10.7551/mitpress/9780262033589.003.0019
- Subject:
- Computer Science, Machine Learning
This chapter describes an experimental study of large-scale semi-supervised learning for the problem of protein classification. The protein classification problem, a central problem in computational ...
More
This chapter describes an experimental study of large-scale semi-supervised learning for the problem of protein classification. The protein classification problem, a central problem in computational biology, is to predict the structural class of a protein given its amino acid sequence. Such a classification helps biologists to understand the function of a protein. Building an accurate protein classification system, as with many tasks, depends critically upon choosing a good representation of the input sequences of amino acids. Early work using string kernels with support vector machines (SVMs) for protein classification achieved state-of-the-art classification performance. However, such representations are based only on labeled data—examples with known three-dimensional (3D) structures, organized into structural classes-while in practice, unlabeled data are far more plentiful.Less
This chapter describes an experimental study of large-scale semi-supervised learning for the problem of protein classification. The protein classification problem, a central problem in computational biology, is to predict the structural class of a protein given its amino acid sequence. Such a classification helps biologists to understand the function of a protein. Building an accurate protein classification system, as with many tasks, depends critically upon choosing a good representation of the input sequences of amino acids. Early work using string kernels with support vector machines (SVMs) for protein classification achieved state-of-the-art classification performance. However, such representations are based only on labeled data—examples with known three-dimensional (3D) structures, organized into structural classes-while in practice, unlabeled data are far more plentiful.
Jeffrey S. Racine
- Published in print:
- 2019
- Published Online:
- January 2019
- ISBN:
- 9780190900663
- eISBN:
- 9780190933647
- Item type:
- chapter
- Publisher:
- Oxford University Press
- DOI:
- 10.1093/oso/9780190900663.003.0007
- Subject:
- Economics and Finance, Econometrics
This chapter covers two advanced topics: a machine learning method (support vector machines useful for classification) and nonparametric kernel regression.
This chapter covers two advanced topics: a machine learning method (support vector machines useful for classification) and nonparametric kernel regression.