model

Getting Started with TensorFlow Strata Singapore Dec. 5 2017

Yufeng Guo

Welcome and Logistics - About the workshop, intros - Break at 10:30am - TA - Steve These slides: bit.ly/tf-strata-sing

Google Cloud Platform

2

Your guides

These slides: bit.ly/tf-strata-sing

Yufeng [email protected] @YufengG http://bit.ly/tf-strata-sing

bit.ly/tensorflow-workshop

Overview of the Workshop ● Quick look at TensorFlow’s high-level APIs: using ‘canned’ Estimators for Linear models and Deep Neural Networks ● Intro to TensorFlow, its APIs, and its ecosystem ● “Wide & Deep” models with TensorFlow, & , vocabulary_list=["female", "male"]) education = categorical_column_with_hash_bucket( key="education", hash_bucket_size=1000) ... # Continuous base columns. age = numeric_column("age") ...

http://bit.ly/tf-strata-sing

Feature columns continued # Transformations. age_buckets = bucketized_column( age, boundaries=[18, 25, 30, 35, 40, 45, 50, 55, 60, 65]) education_occupation =crossed_column( ["education", "occupation"], hash_bucket_size=int(1e4)) … # embeddings for deep learning embedding_column(workclass, dimension=8)


Feature columns continued # Transformations. age_buckets = layers.bucketized_column( age, boundaries=[18, 25, 30, 35, 40, 45, 50, 55, 60, 65]) education_occupation = layers.crossed_column( [education, occupation], hash_bucket_size=int(1e4)) … # embeddings for deep learning layers.embedding_column(workclass, dimension=8)


Typical structure ● Load ) } prediction_output = tf.estimator.export.PredictOutput({"classes": tf.argmax(input=logits, axis=1), "probabilities": tf.nn.softmax(logits, name="softmax_tensor")}) if mode == tf.estimator.ModeKeys.PREDICT: return tf.estimator.EstimatorSpec(mode=mode, predictions=predictions, export_outputs={tf.saved_model.signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY: prediction_output}) ... http://bit.ly/tf-strata-sing

... # Calculate Loss (for both TRAIN and EVAL modes) onehot_labels = tf.one_hot(indices=tf.cast(labels, tf.int32), depth=10) loss = tf.losses.softmax_cross_entropy( onehot_labels=onehot_labels, logits=logits) # Generate some summary info tf.summary.scalar('loss', loss) # Configure the Training Op (for TRAIN mode) if mode == tf.estimator.ModeKeys.TRAIN: optimizer = tf.train.AdamOptimizer(learning_rate=1e-4) train_op = optimizer.minimize( loss=loss, global_step=tf.train.get_global_step()) return tf.estimator.EstimatorSpec(mode=mode, loss=loss, train_op=train_op) # Add evaluation metrics (for EVAL mode) eval_metric_ops = { "accuracy": tf.metrics.accuracy( labels=labels, predictions=predictions["classes"])} return tf.estimator.EstimatorSpec( mode=mode, loss=loss, eval_metric_ops=eval_metric_ops) http://bit.ly/tf-strata-sing

# Convolutional Layer #1 Using conv1 = tf.layers.conv2d( tf.layers inputs=input_layer, filters=32, kernel_size=[5, 5], padding="same", activation=tf.nn.relu) to define our # Pooling Layer #1 CNN pool1 = tf.layers.max_pooling2d(inputs=conv1, pool_size=[2, 2], strides=2) # Convolutional Layer #2 conv2 = tf.layers.conv2d( inputs=pool1, filters=64, kernel_size=[5, 5], padding="same", activation=tf.nn.relu) # Pooling Layer #2 pool2 = tf.layers.max_pooling2d(inputs=conv2, pool_size=[2, 2], strides=2) # Flatten tensor into a batch of vectors pool2_flat = tf.reshape(pool2, [-1, 7 * 7 * 64]) # Dense Layer dense = tf.layers.dense(inputs=pool2_flat, units=1024, activation=tf.nn.relu, name="dense1") # Add dropout operation; 0.6 probability that element will be kept dropout = tf.layers.dropout( inputs=dense, rate=0.4, training=mode == tf.estimator.ModeKeys.TRAIN) # Logits layer logits = tf.layers.dense(inputs=dropout, units=10) http://bit.ly/tf-strata-sing

if mode == tf.estimator.ModeKeys.TRAIN: K.set_learning_phase(1) else: K.set_learning_phase(0)

Using keras.layers to define our CNN

conv1 = Convolution2D(32, (5, 5), activation='relu', input_shape=(28,28,1))(input_layer) pool1 = MaxPooling2D(pool_size=(2,2))(conv1) conv2 = Convolution2D(64, (5, 5), activation='relu')(pool1) pool2 = MaxPooling2D(pool_size=(2,2))(conv2) pool2_flat = Flatten()(pool2) dense = Dense(1024, activation='relu')(pool2_flat) dropout = Dropout(0.4)(dense) logits = Dense(10, activation='linear')(dropout)



Comparing ‘regular’ with Fashion MNIST


Maybe there are even better models than CNNs? (There's always a better model…)



Thank you!

@YufengG [email protected] yufengg.com http://bit.ly/tf-strata-sing

bit.ly/tensorflow-workshop

What next?

Enjoyed this? Rate our session: bit.ly/tf-workshop-stratasing17

Tutorials and code tensorflow.org

These Slides bit.ly/tf-strata-sing

TensorFlow Dev Summit https://goo.gl/bv9mm7

AI Adventures (on YouTube) bit.ly/ai-adventures

Intro to Deep Learning with TensorFlow Udacity class goo.gl/iHssII

Deep Learning (Goodfellow, Bengio, Courville) http://www.deeplearningbook.org/

Stanford’s CS231n cs231n.github.io

Chris Olah’s blog colah.github.io

TensorFlow Playground goo.gl/mXhncM

Michael Nielsen’s book neuralnetworksanddeeplearning.com

end