The purpose of this project was to build a toy implementation of a neural network that supports convolutional and max pooling layers, and compare this approach to classifying the MNIST handwritten number set against my previous toy implementations that leveraged less sophisticated neural networks and decision forests.

Although my implementation is relatively primitive, it achieved a classification accuracy of almost 98% and outperformed both my randomized decision forest and neural network ensemble in terms of training speed, memory footprint, and accuracy. The following table illustrates this comparison.


 Approach Training speed Memory footprint Accuracy
 Convolutional Network <10 minutes 0.7 MB 97.68%
 Neural Network Ensemble <30 minutes 12 MB 96.75%
 Randomized Decision Forest >24 hours 40 MB 96.5%


Features

  • Configuration: set epoch size, learning rate, mini-batch size, and layer dimensions (size and types).

  • Tuning: quadratic and cross-entropy cost, L2 regularization, multiple activation functions (tanh, relu, sigmoid).

  • Serialization: save and load network ensembles for quick reuse.



Downloads

  Convolutional Neural Network Demo for Windows (32 KB)
  MNIST Training and Test Samples (10.5 MB)
  Convolutional Network Ensemble (97.65% accuracy) (0.7 MB)
  Convolutional Neural Network Source Code (C++)



Demo Instructions

If you'd like to test out the pre-built app, or use the source as-is, then you'll need to download the MNIST training and test data and unzip it into the same folder as the demo program. Next, you could train your own ensemble, but I'd recommend that you save yourself some time and use my existing 3 network ensemble by unzipping it to the same folder as the program.

Running the program (on Windows) without any parameters will display the prompt below. You train one network at a time and then combine them into an ensemble that you can use for verification. If you'd like to modify any of the hyper-parameters then you'll need to download, modify, and build the source code. 


Usage: simple-cnn-x64.exe [options]
  --train  [output network filename]                            Trains a single neural network based on the MNIST dataset.
  --combine [output ensemble filename] [network filenames]      Combines a set of network files into a single ensemble file.
  --verify [input ensemble filename]                            Tests the accuracy of a neural network ensemble against the MNIST test set.