Introduction to Neural Network

Introduction to Neural Network Instructors:

Joseph Santarcangelo, Ph.D. .

1

What Is a Neural Network • A neural network is a function that can use to approximate ‘something’ using a set of parameters 𝑦

x 𝑦" = 𝑓(𝑥|𝑊)

Neural Networks and Classification • There a many applications of Neural Networks 1. Regression 2. Clustering 3. Recommendation Systems • We will only Cover Classification

The Big Picture

Train Dataset

𝑦" = 𝑓(𝑥|𝑊)

𝑊

neural network

Prediction 𝑦"

Unknown Sample

𝑥

Targets :Iris Flower Data Set • Targets are denoted as 𝑦 • For example 𝑦 ∈ {0,1,2} means 𝑦 can equal: 0, 1, 2 𝑦=0

𝑦=1

𝑦=2

Features : Flower Data Set Features Sepal Sepal Petal Petal Species length width length width 𝒙3 𝒙 4 𝒙 5 𝒙 6 y

www.pinterest.com/aussiels/irises/

Iris Flower Data Set Fisher's Iris Data Sepal Sepal Petal Petal Species length width length width 𝒙3 𝒙 4 𝒙 5 𝒙 6 y

www.pinterest.com/aussiels/irises/

Training Set

y5 = 0

y4 = 1 y3 = 0

𝐱 4 = 4.2,2,1.4,0.3

𝐱 5 = 4.5,2.9,1.3,0.4

y> = 1 𝐱 > = 4.5,2.1.3,0.4

𝐱3 = 4.9,3,1.4,0.2

y6 = 1 𝐱 6 = 3.9,2,3.4,0.6

y@ = 1 𝐱 @ = 3.9,2,3.4,0.6

Features and targets: Example • Colors are used to indicates the class 1. 2. 3. 4. 5. 6. 7.

𝑦3 = 0, 𝑥3 = −3 𝑦4 = 0, 𝑥4 =-2 𝑦5 = 1, 𝑥5 =-1 𝑦6 = 1, 𝑥6 =-1 𝑦> = 1, 𝑥> = 1 𝑦> = 0, 𝑥@ =2 𝑦C = 0, 𝑥C =3

−3

𝑥3

−2

−1

0

1

2

3

x

𝑥4 𝑥5 𝑥6 𝑥> 𝑥@ 𝑥C © 2015 BigDataUniversity.com

9

Features and targets: Example • It is helpful to view the sample 𝑦 as a decision function of x 𝑦(x)

1

−3

𝑥3

−2

−1

0

1

2

3

x

𝑥4 𝑥5 𝑥6 𝑥> 𝑥@ 𝑥C © 2015 BigDataUniversity.com

10

Features and targets: Example • We can do the same in 2D 1. 2. 3. 4. 5. 6. 7. 8.

𝐱

4

y3 = 0, 𝐱3 = [−2, 0] y4 = 0, 𝐱 4 = [ 2, 0] y5 = 0, 𝐱 5 = [ 0, −2]

𝐱6

y6 = 0, 𝐱 5 = [ 0, 2]

𝐱H

𝐱3 𝐱 >

y> = 1, 𝐱 4 = [−1, 0] y@ = 1, 𝐱 5 = [0, −1]

𝐱4

𝐱C 𝐱@

yC = 1, 𝐱3 = [1, 0] yH = 1, 𝐱 4 = [0, 1]

𝐱

𝐱5

© 2015 BigDataUniversity.com

4

11

3

Features and targets: Example • We can also interpret these samples as as a function

𝐱

y=1 y=0 © 2015 BigDataUniversity.com

4

12

3

Features

2

0

1

Features • Usually more features can give better performance

1

0

2

Features and Targets §Data matrix and class vector Samples

Data Matrix 𝑋

1 2 3 4 5

Class vector 𝑦 Features © 2015 BigDataUniversity.com

15

Features • Color Images are comprised of different channels • Each channel has an element represented with a number

Features

…… 𝐱3 =

Linear Classifiers

Linear Classifier • The equation of a line in 1 dimension is given by:

𝑤x + 𝑏 • This generalizes in D dimensions to : V 𝒘 𝐱 • Lets see what happens for different values of 𝐱 ? © 2015 BigDataUniversity.com

19

Linear Classifier • Consider the following data set • If we can separate the data with a line, we can use that line to classify the sample 𝑧

𝑧 = 𝑤x + 𝑏

𝑦" = 0

= 1x − 1

x 𝑦" = 1


20

Linear Classifier • If x is on the left side of the line we get a positive number

𝑧 = 1x − 1 𝑧 = 1(3) − 1 =2

𝑧 2

x 3


21

Linear Classifier • If x is on the right side of the line we get a positive number

𝑧 = 1x − 1 𝑧 = 1(−2) − 1 = −3

𝑧

-2

x

-3


22

Linear Classifiers • If we use the line to calculate the class of a point, it always returns a positive or negative numbers, such as 3, -2, and so on. • But, we need class 0 and class 1. How we can convert the numbers into 0 and 1? 𝑦" = 1 • 𝑦" = 1, if 𝑧>= 0 • 𝑦" = 0, if 𝑧< 0

𝑧 𝑦" = 0

Linear Classifier: Threshold Function 𝑦" = 1

𝑦" = 0 𝑧 2 -2 3

x

-3 © 2015 BigDataUniversity.com

24

Logistic regression • Logistic Function 𝜎 𝑧

𝜎 −4 ≈ 0

𝜎 −1 = 0.25

𝑧

𝜎 1 = 0.75

𝜎 4 ≈1

1 𝜎 𝑧 = 1 + 𝑒 ]^ © 2015 BigDataUniversity.com

25

Logistic regression 𝑦" = 1

𝜎 𝑧 =0.5

𝜎 𝑧 >0.5 𝑦" = 0 𝑧

𝜎 𝑧