In this blog, I will create a deep learning model that uses the negative log-likelihood of Gaussian distribution as a loss. For this purpose, I will use Tensorflow.

Why not Keras?¶

Keras has been my first-choice deep learning framework in the last 1 year. However, if you want to create personal loss functions or layers, Keras requires to use backend functions written in either TensorFlow or Theano. As the negative log-likelihood of Gaussian distribution is not one of the available loss in Keras, I need to implement it in Tensorflow which is often my backend. So this motivated me to learn Tensorflow and write everything in Tensorflow rather than mixing up two frameworks.

My Kaggle's final score using the model explained in this ipython notebook.

In this blog post, I will learn a semantic segmentation problem and review fully convolutional networks. In an image for the semantic segmentation, each pixcel is usually labeled with the class of its enclosing object or region. For example, a pixcel might belongs to a road, car, building or a person. The semantic segmentation problem requires to make a classification at every pixel.

When you make your own website using third-party clouds, e.g. Heroku, there is a restriction for the uploaded data size. For example, Heroku only allows 500MB memory space. This might become a too tight constraint if you want to add some pictures to your websites as high resolution pictures nowadays could easily be about 10MB.

I have been using Suunto Ambit 2 as the device for recording any sports activities e.g. running, trekking, or swimming. Recorded activities can be updated in the cloud Suunto Movescount, and see the summary of activities, e.g., speed, pace, elevation gain or elevation loss. Suunto Movescount is a growing sports community where you can create your own sports diary to collect and share your activities as well as customize your compatible Suunto watch.

I will revisit Driver's facial keypoint detection. In this blog, I will improve the landmark detection model performance with data augmentation. ImageDataGenerator for the purpose of landmark detection is implemented at my github account and discussed in my previous blog - Data augmentation for facial keypoint detection-.

The python class ImageDataGenerator_landmarks is available at my github account. This blog explains about his class.

Why data augmentation?¶

Deep learning model is data greedy and the performance of the model may be surprisingly bad when testing images vary from training images a lot. Data augmentation is an essential technique to utilize limited amount of training images. In my previous blog post, I have seen poor performance of a deep learning model when testing images contain the translation of the training images. However, the model performance improves when training data also contains translated images. See Assess the robustness of CapsNet

In this blog post, I would like to review the traditional Gaussian process modeling. This blog was motivated by the blog post Fitting Gaussian Process Models in Python by Christ at Domino which explains the basic of Gaussian process modeling.

When I was reading his blog post, I felt that some mathemtatical details are missing. Therefore, I am writing this blog to digest his blog post.

Seminal blog post of Yarin Gal from Cambridge machine learning group What my deep model doesn't know... motivated me to learn how Dropout can be used to describe the uncertainty in my deep learning model.

This blog post is dedicated to learn how to use Dropout to measure the uncertainty using Keras framework.

The above gif shows the object detection results from the Haar cascades implemented in OpenCV.

In ths previous blog post Driver's facial keypoint detection, I used public dataset CVC11 to train a facial keypoint detection model. The crucial step in the modeling procedure was to trim image using the face's bounding box. In practice, you might not have access to nice bounding box. In such cases, pre-trained facial detection classifier such as Haar cascade can be useful.