Web Design Blog

A Beginner’s Guide to Keras: Digit Recognition in 30 Minutes

A Beginner’s Guide to Keras: Digit Recognition in 30 Minutes – SitePointSkip to main contentFree JavaScript Book!Write powerful, clean and maintainable JavaScript.RRP $11.95 Over the last decade, the use of artificial neural networks (ANNs) has increased considerably. People have used ANNs in medical diagnoses, to predict Bitcoin prices, and to create fake Obama videos! With all the buzz about deep learning and artificial neural networks, haven’t you always wanted to create one for yourself? In this tutorial, we’ll create a model to recognize handwritten digits.
We’ll use the Keras library for training the model in this tutorial. Keras is a high-level library in Python that is a wrapper over TensorFlow, CNTK and Theano. By default, Keras uses a TensorFlow backend by default, and we’ll use the same to train our model.
Artificial Neural Networks

An artificial neural network is a mathematical model that converts a set of inputs to a set of outputs through a number of hidden layers. An ANN works with hidden layers, each of which is a transient form associated with a probability. In a typical neural network, each node of a layer takes all nodes of the previous layer as input. A model may have one or more hidden layers.
ANNs receive an input layer to transform it through hidden layers. An ANN is initialized by assigning random weights and biases to each node of the hidden layers. As the training data is fed into the model, it modifies these weights and biases using the errors generated at each step. Hence, our model “learns” the pattern when going through the training data.
Convoluted Neural Networks
In this tutorial, we’re going to identify digits — which is a simple version of image classification. An image is essentially a collection of dots or pixels. A pixel can be identified through its component colors (RGB). Therefore, the input data of an image is essentially a 2D array of pixels, each representing a color.
If we were to train a regular neural network based on image data, we’d have to provide a long list of inputs, each of which would be connected to the next hidden layer. This makes the process difficult to scale up.

In a convoluted neural network (CNN), the layers are arranged in a 3D array (X-axis coordinate, Y-axis coordinate and color). Consequently, a node of the hidden layer would only be connected to a small region in the vicinity of the corresponding input layer, making the process far more efficient than a traditional neural network. CNNs, therefore, are popular when it comes to working with images and videos.

The various types of layers in a CNN are as follows:
convolutional layers: these run input through certain filters, which identify features in the image
pooling layers: these combine convolutional features, helping in feature reduction
flatten layers: these convert an N-dimentional layer to a 1D layer
classification layer: the final layer, which tells us the final result
Let’s now explore the data.
Explore MNIST Dataset
As you may have realized by now that we need labelled data to train any model. In this tutorial, we’ll use the MNIST dataset of handwritten digits. This dataset is a part of the Keras package. It contains a training set of 60,000 examples, and a test set of 10,000 examples. We’ll train the data on the training set and validate the results based on the test data. Further, we’ll create an image of our own to test whether the model can correctly predict it.
First, let’s import the MNIST dataset from Keras. The .load_data() method returns both the training and testing datasets:
from keras.datasets import mnist

(x_train, y_train), (x_test, y_test) = mnist.load_data()

Let’s try to visualize the digits in the dataset. If you’re using Jupyter notebooks, use the following magic function to show inline Matplotlib plots:
%matplotlib inline

Next, import the pyplot module from matplotlib and use the .imshow() method to display the image:
import matplotlib.pyplot as plt

image_index = 35
print(y_train[image_index])
plt.imshow(x_train[image_index], cmap=’Greys’)
plt.show()

The label of the image is printed and then the image is displayed.

Let’s verify the sizes of the training and testing datasets:
print(x_train.shape)
print(x_test.shape)

Notice that each image has the dimensions 28 x 28:
(60000, 28, 28)
(10000, 28, 28)

Next, we may also wish to explore the dependent variable, stored in y_train. Let’s print all labels till the digit that we visualized above:
print(y_train[:image_index + 1])

[5 0 4 1 9 2 1 3 1 4 3 5 3 6 1 7 2 8 6 9 4 0 9 1 1 2 4 3 2 7 3 8 6 9 0 5]

Cleaning Data
Now that we’ve seen the structure of the data, let’s work on it further before creating the model.
To work with the Keras API, we need to reshape each image to the format of (M x N x 1). we’ll use the .reshape() method to perform this action. Finally, normalize the image data by dividing each pixel value by 255 (since RGB values can range from 0 to 255):

img_rows, img_cols = 28, 28

x_train = x_train.reshape(x_train.shape[0], img_rows, img_cols, 1)
x_test = x_test.reshape(x_test.shape[0], img_rows, img_cols, 1)

x_train /= 255
x_test /= 255

Next, we need to convert the dependent variable in the form of integers to a binary class matrix. This can be achieved by the to_categorical() function:
from keras.utils import to_categorical
num_classes = 10

y_train = to_categorical(y_train, num_classes)
y_test = to_categorical(y_test, num_classes)

We’re now ready to create the model and train it!
Design a Model
The model design process is the most complex factor, having a direct impact on the performance of the model. For this tutorial, we’ll use this design from the Keras documentation.
To create the model, we first initialize a sequential model. It creates an empty model object. The first step is to add a convolutional layer, which takes the input image:
from keras.models import Sequential
from keras.layers import Dense, Dropout, Flatten, Conv2D, MaxPooling2D

model = Sequential()
model.add(Conv2D(32, kernel_size=(3, 3),
activation=’relu’,
input_shape=(img_rows, img_cols, 1)))

A relu activation stands for “Rectified Linear Units”, which takes the max of a value or zero. Next, we add another convolutional layer, followed by a pooling layer:
model.add(Conv2D(64, (3, 3), activation=’relu’))
model.add(MaxPooling2D(pool_size=(2, 2)))

Next, we add a “dropout” layer. While neural networks are trained on huge datasets, a problem of overfitting may occur. To avoid this issue, we randomly drop units and their connections during the training process. In this case, we’ll drop 25% of the units:
model.add(Dropout(0.25))

Next, we add a flattening layer to convert the previous hidden layer into a 1D array:
model.add(Flatten())

Once we’ve flattened the data into a 1D array, we can add a dense hidden layer, which is normal for a traditional neural network. Next, add another dropout layer before adding a final dense layer which classifies the data:
model.add(Dense(128, activation=’relu’))
model.add(Dropout(0.5))
model.add(Dense(num_classes, activation=’softmax’))

The softmax activation is used when we’d like to classify the data into a number of pre-decided classes.
Compile and Train Model
In the model design process, we’ve created an empty model without an objective function. We need to compile the model and specify a loss function, an optimizer function, and a metric to assess model performance.
We need to use a sparse_categorical_crossentropy() loss function in case we have an integer-dependent variable. For a vector-based dependent variable like a ten-size array as the output of each test case, use categorical_crossentropy. In this example, we’ll use the adam optimizer. The metric is the basis for the assessment of our model performance, though it’s only for us to judge and isn’t used in the training step:
model.compile(loss=’sparse_categorical_crossentropy’,
optimizer=’adam’,
metrics=[‘accuracy’])

We’re now ready to train the model using the .fit() method. We need to specify an epoch and batch size when training the model. An epoch is one forward pass and one backward pass of all training examples. A batch size is the number of training examples in one forward or backward pass.
Finally, save the model once the training is complete to use its results at a later stage:
batch_size = 128
epochs = 10

model.fit(x_train, y_train,
batch_size=batch_size,
epochs=epochs,
verbose=1,
validation_data=(x_test, y_test))
score = model.evaluate(x_test, y_test, verbose=0)
print(‘Test loss:’, score[0])
print(‘Test accuracy:’, score[1])
model.save(“test_model.h5”)

When we run the code above, the following output is shown as the model runs. It takes about ten minutes in a 2018 Macbook Air running Jupyter notebooks:
Train on 60000 samples, validate on 10000 samples
Epoch 1/10
60000/60000 [==============================] – 144s 2ms/step – loss: 0.2827 – acc: 0.9131 – val_loss: 0.0612 – val_acc: 0.9809
Epoch 2/10
60000/60000 [==============================] – 206s 3ms/step – loss: 0.0922 – acc: 0.9720 – val_loss: 0.0427 – val_acc: 0.9857
…
Epoch 9/10
60000/60000 [==============================] – 142s 2ms/step – loss: 0.0329 – acc: 0.9895 – val_loss: 0.0276 – val_acc: 0.9919
Epoch 10/10
60000/60000 [==============================] – 141s 2ms/step – loss: 0.0301 – acc: 0.9901 – val_loss: 0.0261 – val_acc: 0.9919
Test loss: 0.026140549496188395
Test accuracy: 0.9919

At the end of the final epoch, the accuracy of the test dataset is 99.19%. It’s difficult to comment on how high the accuracy needs to be. For a test run, accuracy over 99% is very good. However, there’s a lot of scope for improvement by tweaking the model parameters. There’s a submission from a digit recognizer contest on Kaggle that reached 99.7% accuracy.
Test with Handwritten Digits
Now that the model is ready, let’s use a custom image to assess the performance of the model. I’ve hosted a custom 28×28 digit on Imgur. First, let’s read the image using the imageio library and explore how the input data looks:
import imageio
import numpy as np
from matplotlib import pyplot as plt

im = imageio.imread(“https://i.imgur.com/a3Rql9C.png”)

Next, convert the RGB values to grayscale. We can then use the .imshow() method as explored above to display the image:
gray = np.dot(im[…,:3], [0.299, 0.587, 0.114])
plt.imshow(gray, cmap = plt.get_cmap(‘gray’))
plt.show()

Next, reshape the image and normalize the values to make it ready to be used in the model that we’ve just created:

gray = gray.reshape(1, img_rows, img_cols, 1)

gray /= 255

Load the model from the saved file using the load_model() function and predict the digit using the .predict() method:

from keras.models import load_model
model = load_model(“test_model.h5”)

prediction = model.predict(gray)
print(prediction.argmax())

The model correctly predicts the digit shown in the image:
5

Final Thoughts
In this tutorial, we created a neural network with Keras using the TensorFlow back end to classify handwritten digits. Although we reached an accuracy of 99%, there are still opportunities for improvement. We also learned how to classify custom handwritten digits, which were not a part of the test dataset. This tutorial, however, has just scratched the surface of the artificial neural networks field. There are endless uses for neural networks that are only limited by our imagination.
Are you able to improve the accuracy of the model? What other techniques can you think of using? Let me know on Twitter.
Related ArticlesUsing Python to Parse Spreadsheet DataProgrammingBy

A Comparison of Ruby Version Managers for macOS

A Comparison of Ruby Version Managers for macOS – SitePointSkip to main contentFree JavaScript Book!Write powerful, clean and maintainable JavaScript.RRP $11.95 In this article, I’ll compare the most popular version managers for Ruby on a Mac: Chruby, Rbenv, and RVM, as well as Asdf, a version manager for multiple languages, and Frum, a brand new version manager written in Rust. I’ll make recommendations based on what you’re most likely to need.
Pre-installed Ruby is Not for Developers
Folks who are new to Ruby are delighted to discover that Ruby comes pre-installed on macOS. They’re often disappointed when a more experienced developer tells them, “Don’t use the Mac system Ruby.” I’ve written elsewhere about the reasons. In short, the system Ruby is there for macOS, not for you. It’s fine for running utility scripts, but for development, the system Ruby is out of date and, if you’re not careful, installation of gems can lead to headaches (including system security violations).
Homebrew’s One Ruby Problem
Homebrew is the package manager that has become a near-standard for adding developer software to macOS. Homebrew makes it easy to install a new Ruby version to avoid relying on the system Ruby. But there are two problems with Homebrew. First, Homebrew’s Ruby version can be automatically and unexpectedly replaced by a newer version when other Homebrew packages are installed (a problem solved by brew pin ruby). More importantly, Homebrew only lets you install one version of Ruby. If you’re developing or maintaining more than one project in Ruby, you’ll likely need to switch among Ruby versions. That’s where you’ll need a version manager.
Asdf for Multiple Languages
Asdf is a version manager that accommodates multiple languages. If you’re using Ruby to develop web applications with Rails 6, you’ll need to install Node and Yarn to build a Rails starter application. (Rails 7 may eliminate the need for Node, but that’s not yet certain at this time.)
So, for a Rails developer, Asdf is an ideal version manager. It will also serve you well if you develop applications in Python, Elixir, or many other languages (see a list of supported languages). Is there a downside to using Asdf? Just two. It uses shims to intercept common commands, which can make troubleshooting difficult. Also, it can be a little slow for some operations. If you’re developing in multiple languages, I recommend you try Asdf to see if you like it. Switch to Frum, the Ruby-only version manager written in Rust, if you find Asdf slows you down.
Ruby-only Version Managers
If you only intend to develop applications using Ruby, choose among the four Ruby-only version managers: Chruby, Frum, Rbenv, and RVM. Differences among the four lie in speed, complexity, and how each switches Ruby versions. Let’s look at the oldest first: RVM.
RVM
RVM was the first popular version manager for Ruby. It’s still maintained and widely used. RVM modifies the system cd command to set environment variables when you change directories. To override the cd command, the RVM installation script changes the shell configuration file (the ~/.zshrc file on newer Macs).
RVM properly displays environment variables to reveal the Ruby and gem version and location, which can help diagnose problems. However, RVM includes additional features such as gemsets. Gemsets are no longer needed, since Bundler was added to the core Ruby to manage gem dependencies. The additional complexity isn’t needed, so many developers have switched to a newer version manager such as Rbenv, Chruby, or Frum.
Rbenv
Along with RVM, Rbenv has long been the most popular version manager for Ruby. Rbenv uses shims to intercept common Ruby commands. (Asdf also uses shims.)
After installing Rbenv with Homebrew, you must modify your ~/.zshrc file so the Rbenv shims take precedence over ordinary Ruby commands. The shim calls the rbenv exec command, which determines which version of Ruby to use before running any Ruby command.
Shims have two drawbacks. First, shims make troubleshooting more difficult by hiding the actual command. For example, the which ruby command shows the Rbenv shim, not the actual Ruby version (you must set the RBENV_DEBUG environment variable if you want to see the actual Ruby command). Second, a shim can add microseconds of delay to execution of a Ruby command. (Rbenv intervenes every time you run a Ruby command unlike RVM, which intervenes only when you change directories.) For developers who notice the delay, or don’t like the idea of shims, there is Chruby or Frum.
Chruby
Chruby is not as well-known as RVM or Rbenv, but it’s championed by several prominent Ruby developers. Unlike RVM, it doesn’t override the cd command. Unlike Rbenv or Asdf, it doesn’t intercept commands using shims. You’ll need to modify your ~/.zshrc file so Chruby runs in your local shell environment. There’s no need to modify your $PATH, as Chruby does that for you, based on the Ruby version you’ve selected.
Chruby also sets certain environment variables for Ruby. That’s all that’s needed for Chruby to switch among Ruby versions. It works fine for troubleshooting with which. I used Chruby for several years and had no problems with it. Chruby is a great choice, but you may want to try Frum, the newest Ruby version manager.
Frum
Frum is written in Rust, a good language for fast command-line execution of system software. Frum is new (released in early 2021). Unlike Asdf, Chruby, or Rbenv, it includes a built-in Ruby installer command so there’s no extra program needed to install Ruby. The installer is fast, and Frum requires no dependencies (it’s an all-in-one Rust executable you can install with Homebrew).
After you install Frum, you must modify your ~/.zshrc file so Frum runs in your local shell environment. Like the other version managers, it checks for a .ruby-version file in a project directory and, if there’s no version specified for a project, it will default to a global Ruby version. There are no shims (unlike Asdf or Rbenv) and it doesn’t override the cd command (unlike RVM). Though it’s new, I recommend it because it’s all-in-one and fast.
Installing a Version Manager for Ruby
I’ve written a complete guide to installing Ruby on Mac. In the guide, I provide instructions for:

If you want to explore RVM or Rbenv, see:

A Note about Docker
If you mention Ruby version managers, someone will likely say, “I use Docker.” Docker is a containerization tool that some developers use for version management. The primary use case for Docker is to create a reproducible virtual server that contains a configured version of any software dependency needed to run an application (language, databases, message queues). As such, it’s ideal for creating a “frozen” version of a development environment for deployment to a server.
You can also develop locally within a container, but it will run slower, require more memory, and add configuration complexity compared to a version manager. To keep things simple, I avoid Docker for local development unless an application is disturbingly complex.
Conclusion
You’re not just a casual Ruby user if you’re working on multiple projects in Ruby. Make sure you install a great version manager in your local development environment.
I like Asdf because I develop in Ruby and JavaScript and Asdf gives me one version manager for both languages. However, Asdf uses shims (like Rbenv) so occasionally it’s slower than I like. I also find it annoying that Asdf hides the actual command from troubleshooting with the which command. If these drawbacks outweigh the convenience of managing multiple languages, I recommend using Frum to switch among your Ruby versions. Version managers should be simple and stay out of your way when you’re working.
Related ArticlesUsing Python to Parse Spreadsheet DataProgrammingBy