Motivation. Examples. Theoretical Problems. Results. Perspectives. Learning. Machine Learning: develop algorithms to aut
Outline
Motivation
Examples
Theoretical Problems
Manifold Learning Olivier Bousquet
Curves and Surfaces, Avignon, 2006
Olivier Bousquet
Manifold Learning
Results
Perspectives
Outline
Motivation
Examples
1
Motivation
2
Examples
3
Theoretical Problems
4
Results
5
Perspectives
Olivier Bousquet
Theoretical Problems
Manifold Learning
Results
Perspectives
Outline
Motivation
Examples
Theoretical Problems
Results
Acknowledgements
Most of the work presented here has been done by or in collaboration with Ulrike von Luxburg Matthias Hein (special thanks for the pictures) Mikhail Belkin Jean-Yves Audibert Olivier Chapelle
Olivier Bousquet
Manifold Learning
Perspectives
Outline
Motivation
Examples
Theoretical Problems
Results
Perspectives
Learning
Machine Learning: develop algorithms to automatically extract ”patterns” or ”regularities” from data (generalization) Typical tasks Clustering: Find groups of similar points Dimensionality reduction: Project points in a lower dimensional space while preserving structure Semi-supervised: Given labelled and unlabelled points, build a labelling function Supervised: Given labelled points, build a labelling function All these tasks are not well-defined
Olivier Bousquet
Manifold Learning
Outline
Motivation
Examples
Theoretical Problems
Results
Perspectives
Learning
Machine Learning: develop algorithms to automatically extract ”patterns” or ”regularities” from data (generalization) Typical tasks Clustering: Find groups of similar points Dimensionality reduction: Project points in a lower dimensional space while preserving structure Semi-supervised: Given labelled and unlabelled points, build a labelling function Supervised: Given labelled points, build a labelling function All these tasks are not well-defined
Olivier Bousquet
Manifold Learning
Outline
Motivation
Examples
Theoretical Problems
Results
Perspectives
Learning
Machine Learning: develop algorithms to automatically extract ”patterns” or ”regularities” from data (generalization) Typical tasks Clustering: Find groups of similar points Dimensionality reduction: Project points in a lower dimensional space while preserving structure Semi-supervised: Given labelled and unlabelled points, build a labelling function Supervised: Given labelled points, build a labelling function All these tasks are not well-defined
Olivier Bousquet
Manifold Learning
Outline
Motivation
Examples
Theoretical Problems
Results
Perspectives
Learning
Machine Learning: develop algorithms to automatically extract ”patterns” or ”regularities” from data (generalization) Typical tasks Clustering: Find groups of similar points Dimensionality reduction: Project points in a lower dimensional space while preserving structure Semi-supervised: Given labelled and unlabelled points, build a labelling function Supervised: Given labelled points, build a labelling function All these tasks are not well-defined
Olivier Bousquet
Manifold Learning
Outline
Motivation
Examples
Theoretical Problems
Results
Perspectives
Learning
Machine Learning: develop algorithms to automatically extract ”patterns” or ”regularities” from data (generalization) Typical tasks Clustering: Find groups of similar points Dimensionality reduction: Project points in a lower dimensional space while preserving structure Semi-supervised: Given labelled and unlabelled points, build a labelling function Supervised: Given labelled points, build a labelling function All these tasks are not well-defined
Olivier Bousquet
Manifold Learning
Outline
Motivation
Examples
Theoretical Problems
Results
Perspectives
Learning
Machine Learning: develop algorithms to automatically extract ”patterns” or ”regularities” from data (generalization) Typical tasks Clustering: Find groups of similar points Dimensionality reduction: Project points in a lower dimensional space while preserving structure Semi-supervised: Given labelled and unlabelled points, build a labelling function Supervised: Given labelled points, build a labelling function All these tasks are not well-defined
Olivier Bousquet
Manifold Learning
Outline
Motivation
Examples
Theoretical Problems
Clustering 1.5
1
0.5
0
−0.5
−1 −1.5
−1
−0.5
0
0.5
1
1.5
2
2.5
Identify the two ”groups” Olivier Bousquet
Manifold Learning
Results
Perspectives
Outline
Motivation
Examples
Theoretical Problems
Results
Dimensionality Reduction
Objects in R4096 ? But only 3 parameters: 2 angles and 1 for illumation. They span a 3-dimensional submanifold of R4096 ! Olivier Bousquet
Manifold Learning
Perspectives
Outline
Motivation
Examples
Theoretical Problems
Results
Dimensionality Reduction
Objects in R4096 ? But only 3 parameters: 2 angles and 1 for illumation. They span a 3-dimensional submanifold of R4096 ! Olivier Bousquet
Manifold Learning
Perspectives
Outline
Motivation
Examples
Theoretical Problems
Semi-supervised Learning 1.5
1
0.5
0
−0.5
−1 −1.5
−1
−0.5
0
0.5
1
1.5
2
2.5
Assign a label to the black points Olivier Bousquet
Manifold Learning
Results
Perspectives
Outline
Motivation
Examples
Theoretical Problems
Results
Supervised Learning 1.5
1
0.5
0
−0.5
−1 −1.5
−1
−0.5
0
0.5
1
1.5
2
2.5
Build a function which predicts the label of all points in the space Olivier Bousquet
Manifold Learning
Perspectives
Outline
Motivation
Examples
Theoretical Problems
Results
Formal Definitions
Clustering: Given {X1 , . . . , Xn }, build a function f : X → {1, . . . , k}. Dimensionality reduction: Given {X1 , . . . , Xn } ∈ RD , build a function f : RD → Rd Semi-supervised: Given {X1 , . . . , Xn } and {Y1 , . . . , Ym } with m
