Deep Learning on Boston’s K80 System

Deep Learning and Neural Networks

Deep Learning is a new area of Machine Learning research, which has been introduced with the objective of moving Machine Learning closer to one of its original goals: Artificial Intelligence. Deep Learning is about learning multiple levels of representation and abstraction that help to make sense of data such as images, sound, and text.

Deep Learning is a higher level set of techniques, which is using Neural Networks to achieve its goals. Neural Networks is a biologically-inspired programming paradigm which enables a computer to learn from observational data. The way Neural Networks work, is by taking a large number of training examples, such as images, and then developing a system which can learn from those examples. For instance, a self-driving car with no training images will certainly crash, whereas if we train it with a large number of road images, which will serve as a map, it will start driving around obstacles and manage to go further. A simple neural network example, including its layers, is shown in the following picture.

CTDB2

Another example, widely used in deep learning, is giving a picture as input to a computer and then waiting for it to describe that picture in a natural language. The following picture shows the output of such a use case, after the neural network was trained.

deeplearning2

Impressive Results

Alison B. Lowndes, a researcher in Artificial Intelligence at the University of Leeds, wanted to test a Deep Learning code on the newest NVIDIA Tesla Card, the Tesla K80. For this reason, Boston Ltd provided Alison with a K80 test system, which included:

  • 2x Intel Xeon E5-2660v3 (Haswell)
  • 128GB DDR4 2133MHz RAM
  • 2x NVIDIA Tesla K80s (4 GPUs in total)

Alison’s Deep Learning build required CUDA 6.5, CuDNN, Torch7 and LuaJIT4, which we helped to install. Alison then ran the opensource, refactored for multi-GPU code, from Facebook AI Research (FAIR), fbcunn, for which she chose the AlexNetOWT (AlexNet “One Weird Trick”) Neural Network. The input dataset was the ImageNet ILSVRC2012 dataset, which consisted of more than 1.2 million images.

The results, as Alison said were “very impressive”:

I’ve asked Soumith Chintala (Facebook AI Research) to take a look at my work as I find it hard to believe … Results were very impressive with 6.9% error rates being achieved at speeds of around 1.7s per epoch … the system achieved 1000 epochs in just over half an hour (32 minutes) with a 6.9% error rate.

Alison also received the following response from Professor LeCun:

lecun