Search Results

This contribution describes an algorithm for the unsupervised learning of root-and-pattern morphology. The algorithm relies on a phonological heuristic to bootstrap the morphological analysis and identify a preliminary set of reliable roots and patterns. The analysis is then incrementally extended based on the minimum description length principle, in line with the approach to morphological learning embodied in John Goldsmith’s Linguistica algorithm. The algorithm is implemented as a computer program named Arabica and evaluated with regard to its ability to learn the system of Arabic noun plurals. The tension between the universality of the consonant-vowel distinction and the specificity of root-and-pattern morphology turns out to be crucial for understanding the strengths and weaknesses of this approach.

This chapter discusses basic concepts from control theory and machine learning to facilitate a formal understanding of animal learning and motor control. It first distinguishes between feedback and feed-forward control strategies, and later introduces the classification of machine learning applications into supervised, unsupervised, and reinforcement learning problems. Next, it links these concepts with their counterparts in the domain of the psychology of animal learning, highlighting the analogies between supervised learning and classical conditioning, reinforcement learning and operant conditioning, and between unsupervised and perceptual learning. Additionally, it interprets innate and acquired actions from the standpoint of feedback vs anticipatory and adaptive control. Finally, it argues how this framework of translating knowledge between formal and biological disciplines can serve us to not only structure and advance our understanding of brain function but also enrich engineering solutions at the level of robot learning and control with insights coming from biology.

Marr’s theory of neocortex is the ghost in his trinity of seminal contributions to theoretical neuroscience, being the most spirited and least appreciated. In this chapter, we review its computational and algorithmic content; others in this volume assess anatomical and physiological aspects of this theory and the related accounts of the cerebellum and the hippocampus. Marr offered an early treatment of what we would now understand as unsupervised learning in the context of how the neocortex acquires representations of sensory input. We analyze this problem, elucidating Marr’s central claims and contrasting them with intervening and current work.

On what linguistic theory is about, Chomsky (1957) discussed three views: the discovery procedure, the decision procedure, and the evaluation procedure. Chomsky rejected the discovery view for being “unreasonable” and argued for the evaluation view for being “practical”. Among the researchers who take the discussion seriously is John Goldsmith, with long-standing work on unsupervised learning of linguistic structure. While Goldsmith himself strongly sees his work as a response and realization of the evaluation procedure, this chapter makes the stronger claim that his research program is the higher-order goal of the discovery procedure. This chapter ends with methodological remarks on the importance of shaping linguistic research as an enterprise towards the discovery procedure.

Recent advances in artificial intelligence (AI) technologies have transformed our lives in profound ways. Indeed, AI has not only enabled machines to see (eg, face recognition), hear (eg, music retrieval), speak (eg, speech synthesis), and read (eg, text processing), but also, so it seems, given machines the ability to think (eg, board game-playing) and create (eg, artwork generation). This chapter introduces the key technical elements of machine learning (ML), which is a rapidly growing sub-field in AI and drives many of the aforementioned applications. The goal is to elucidate the ways human efforts are involved in the development of ML solutions, so as to facilitate legal discussions on intellectual property issues.

Machine learning is involved in search, translation, detecting depression, likelihood of college dropout, finding lost children, and to sell all kinds of products. While barely beyond its inception, the current machine learning revolution will affect people and organizations no less than the Industrial Revolution’s effect on weavers and many other skilled laborers. Machine learning will automate hundreds of millions of jobs that were considered too complex for machines ever to take over even a decade ago, including driving, flying, painting, programming, and customer service, as well as many of the jobs previously reserved for humans in the fields of finance, marketing, operations, accounting, and human resources. This section explains how automated machine learning addresses exploratory data analysis, feature engineering, algorithm selection, hyperparameter tuning, and model diagnostics. The section covers the eight criteria considered essential for AutoML to have significant impact: accuracy, productivity, ease of use, understanding and learning, resource availability, process transparency, generalization, and recommended actions.

This chapter provides an overview of unsupervised learning algorithms that can be viewed as spectral methods for linear and nonlinear dimensionality reduction. Spectral methods have recently emerged as a powerful tool for nonlinear dimensionality reduction and manifold learning. These methods are able to reveal low-dimensional structure in high-dimensional data from the top or bottom eigenvectors of specially constructed matrices. To analyze data that lie on a low-dimensional submanifold, the matrices are constructed from sparse weighted graphs whose vertices represent input patterns and whose edges indicate neighborhood relations. The main computations for manifold learning are based on tractable, polynomial-time optimizations, such as shortest-path problems, least-squares fits, semi-definite programming, and matrix diagonalization.

This chapter moves from studying developed countries to a sample of countries in Latin America over time. The chapter presents evidence that an increase in the information environment, in terms of its level of precision, exerts an attenuating and significant effect on the mean and standard deviation of forecasters’ inflation expectations, ultimately lowering inflation outcomes. The finding is robust to the inclusion of policy credibility, persistence in inflation, economic output, and month and country effects. When conducting instrumental variable analysis, similarly signed results hold. The main results imply that an increase in information precision helps to lower aggregate levels of inflation and that the channel that this works through is by lowering the weight of prior expectations, as predicted by the theoretical argument. Importantly, the results persist even when considering a sample of countries with relatively variable inflation outcomes and less established (and therefore less credible) economic institutions.

Using archival, textual information from Federal Open Market Committee (FOMC) transcript data as well as FOMC policy statements, chapter 4 demonstrates that when the chair and median member have opposing inflation preferences, the FOMC communicates with greater precision than when the chair and median member have aligned inflation preferences. The author finds this is true, however, when computing the median members able to cast a public vote. The chapter also provides supportive evidence that when committee members are more dissimilar, the number of textual changes to the policy announcement is higher than otherwise. Theoretically, the chapter shows that a combination of members’ preferences and voting rights matter for the level of uncertainty words used in the official policy statement. Methodologically, the chapter demonstrates the innovative use of supervised and unsupervised learning techniques to construct quantitative measures from text as data, applied to central bank committees.

This paper describes a neuro-symbolic system for distilling interpretable logical theories out of streams of raw, unprocessed sensory experience. We combine a binary neural network, that maps raw sensory input to concepts, with an inductive logic programming system, that combines concepts into declarative rules. Both the inductive logic programming system and the binary neural network are encoded as logic programs, so the weights of the neural network and the declarative rules of the theory can be solved jointly as a single SAT problem. This way, we are able to jointly learn how to perceive (mapping raw sensory information to concepts) and apperceive (combining concepts into declarative rules). We apply our system, the Apperception Engine, to the Sokoban domain. Given a sequence of noisy pixel images, the system has to construct objects that persist over time, extract attributes that change over time, and induce rules explaining how the attributes change over time. We compare our system with a neural network baseline, and show that the baseline is significantly outperformed by the Apperception Engine.