Search Results

Classification

Andrew J. Connolly, Jacob T. VanderPlas, Alexander Gray, Andrew J. Connolly, Jacob T. VanderPlas, and Alexander Gray

in Statistics, Data Mining, and Machine Learning in Astronomy: A Practical Python Guide for the Analysis of Survey Data

Chapter 6 described techniques for estimating joint probability distributions from multivariate data sets and for identifying the inherent clustering within the properties of sources. This approach ... More

Chapter 6 described techniques for estimating joint probability distributions from multivariate data sets and for identifying the inherent clustering within the properties of sources. This approach can be viewed as the unsupervised classification of data. If, however, we have labels for some of these data points (e.g., an object is tall, short, red, or blue) we can utilize this information to develop a relationship between the label and the properties of a source. We refer to this as supervised classification, which is the focus of this chapter. The motivation for supervised classification comes from the long history of classification in astronomy. Possibly the most well known of these classification schemes is that defined by Edwin Hubble for the morphological classification of galaxies based on their visual appearance. This chapter discusses generative classification, k-nearest-neighbor classifier, discriminative classification, support vector machines, decision trees, and evaluating classifiers.Less

Machine learning with sklearn

Thomas P. Trappenberg

in Fundamentals of Machine Learning

This chapter’s goal is to show how to apply machine learning algorithms in a general setting using some classic methods. In particular, it demonstrates how to apply three important machine learning ... More

This chapter’s goal is to show how to apply machine learning algorithms in a general setting using some classic methods. In particular, it demonstrates how to apply three important machine learning algorithms, a support vector classifier (SVC), a random forest classifier (RFC), and a multilayer perceptron (MLP). While many of the methods studied later go beyond these now classic methods, this does not mean that these methods are obsolete. Also, the algorithms discussed here provide some form of baseline to discuss advanced methods like probabilistic reasoning and deep learning. The aim here is to demonstrate that applying machine learning methods based on machine learning libraries is not very difficult. It offers an opportunity to discuss evaluation techniques that are very important in practice.Less

Statistical machine learning

Max A. Little

in Machine Learning for Signal Processing: Data Science, Algorithms, and Computational Statistics

This chapter describes in detail how the main techniques of statistical machine learning can be constructed from the components described in earlier chapters. It presents these concepts in a way ... More

This chapter describes in detail how the main techniques of statistical machine learning can be constructed from the components described in earlier chapters. It presents these concepts in a way which demonstrates how these techniques can be viewed as special cases of a more general probabilistic model which we fit to some data.Less

Transductive Support Vector Machines

Joachims Thorsten

in Semi-Supervised Learning

This chapter discusses the transductive learning setting proposed by Vapnik where predictions are made only at a fixed number of known test points. Transductive support vector machines (TSVMs) ... More

This chapter discusses the transductive learning setting proposed by Vapnik where predictions are made only at a fixed number of known test points. Transductive support vector machines (TSVMs) implement the idea of transductive learning by including test points in the computation of the margin. This chapter provides some examples for why the margin on the test examples can provide useful prior information for learning, in particular for the problem of text classification. The resulting optimization problems, however, are difficult to solve. The chapter reviews exact and approximate optimization methods and discusses their properties. Finally, the chapter discusses connections to other related semi-supervised learning approaches such as co-training and methods based on graph cuts, which can be seen as solving variants of the TSVM optimization problem.Less

Function Approximation

Masashi Sugiyama and Motoaki Kawanabe

in Machine Learning in Non-Stationary Environments: Introduction to Covariate Shift Adaptation

This chapter discusses function learning methods under covariate shift. Ordinary empirical risk minimization learning is not consistent under covariate shift for misspecified models, and this ... More

This chapter discusses function learning methods under covariate shift. Ordinary empirical risk minimization learning is not consistent under covariate shift for misspecified models, and this inconsistency issue can be resolved by considering importance-weighted loss functions. Here, various importance-weighted empirical risk minimization methods are introduced, including least squares and Huber’s method for regression, and Fisher discriminant analysis, logistic regression, support vector machines, and boosting for classification. Their adaptive and regularized variants are also described. The numerical behavior of these importance-weighted learning methods is illustrated through experiments.Less

Semi-Supervised Learning Using Semi-Definite Programming

De Bie Tijl and Cristianini Nello

in Semi-Supervised Learning

This chapter discusses an alternative approach that is based on a convex relaxation of the optimization problem associated with support vector machine transduction. The result is a semi-definite ... More

This chapter discusses an alternative approach that is based on a convex relaxation of the optimization problem associated with support vector machine transduction. The result is a semi-definite programming (SDP) problem which can be optimized in polynomial time, the solution of which is an approximation of the optimal labeling as well as a bound on the true optimum of the original transduction objective function. To further decrease the computational complexity, this chapter proposes an approximation that allows solving transduction problems of up to 1,000 unlabeled samples. Finally, the formulation is extended to more general settings of semi-supervised learning, where equivalence and inequivalence constraints are given on labels of some of the samples.Less

Semi-Supervised Protein Classification Using Cluster Kernels

Weston Jason, Leslie Christina, Ie Eugene, and Noble William Stafford

in Semi-Supervised Learning

This chapter describes an experimental study of large-scale semi-supervised learning for the problem of protein classification. The protein classification problem, a central problem in computational ... More

This chapter describes an experimental study of large-scale semi-supervised learning for the problem of protein classification. The protein classification problem, a central problem in computational biology, is to predict the structural class of a protein given its amino acid sequence. Such a classification helps biologists to understand the function of a protein. Building an accurate protein classification system, as with many tasks, depends critically upon choosing a good representation of the input sequences of amino acids. Early work using string kernels with support vector machines (SVMs) for protein classification achieved state-of-the-art classification performance. However, such representations are based only on labeled data—examples with known three-dimensional (3D) structures, organized into structural classes-while in practice, unlabeled data are far more plentiful.Less

Advanced Topics

Jeffrey S. Racine

in Reproducible Econometrics Using R

This chapter covers two advanced topics: a machine learning method (support vector machines useful for classification) and nonparametric kernel regression.