Understanding Deep Belief Networks in Python

In this tutorial, we will be Understanding Deep Belief Networks in Python. Before reading this tutorial it is expected that you have a basic understanding of Artificial neural networks and Python programming.

So, let’s start with the definition of Deep Belief Network. It is nothing but simply a stack of Restricted Boltzmann Machines connected together and a feed-forward neural network. Now the question arises here is what is Restricted Boltzmann Machines.

Restricted Boltzmann Machines

Before stating what is Restricted Boltzmann Machines let me clear you that we are not going into its deep mathematical details. We are just learning how it functions and how it differs from other neural networks.

RBM has three parts in it i.e. One Hidden layer, One Input layer, and bias units. In the input layer, we will give input and it will get processed in the model and we will get our output. That output is then passed to the sigmoid function and probability is calculated. Now again that probability is retransmitted in a reverse way to the input layer and difference is obtained called Reconstruction error that we need to reduce in the next steps. This process will reduce the number of iteration to achieve the same accuracy as other models.

DBN is just a stack of these networks and a feed-forward neural network. Now we will go to the implementation of this.

Code in Python Programming Language

from sklearn.model_selection import train_test_split
from dbn.tensorflow import SupervisedDBNClassification
import numpy as np
import pandas as pd
from sklearn.metrics.classification import accuracy_score

We will start with importing libraries in python. There are many datasets available for learning purposes. For this tutorial, we are using https://www.kaggle.com/c/digit-recognizer.

digits = pd.read_csv("train.csv")
from sklearn.preprocessing import standardscaler
X = np.array(digits.drop(["label"], axis=1))
Y = np.array(digits["label"])
ss=standardscaler()
X = ss.fit_transform(X)
x_train, x_test, y_train, y_test = train_test_split(X, Y, test_size=0.25, random_state=0)
classifier = SupervisedDBNClassification(hidden_layers_structure =[256, 256], learning_rate_rbm=0.05, learning_rate=0.1, n_epochs_rbm=10, n_iter_backprop=100, batch_size=32, activation_function='relu', dropout_p=0.2)
classifier.fit(x_train, y_train)
y_pred = classifier.predict(x_test)
print('\nAccuracy of Prediction: %f' % accuracy_score(x_test, y_pred))

Output:

Accuracy of Prediction: 93.2%

Then we will upload the CSV file fit that into the DBN model made with the sklearn library. And split the test set and training set into 25% and 75% respectively. Then we predicted the output and stored it into y_pred. And in the last, we calculated Accuracy score and printed that on screen. That’s it!

Leave a Reply