All IT Courses 50% Off
Artificial Intelligence Tutorials

Creating Deep Learning Model with Keras Functional API

In the last tutorial, it was said that there are two ways of creating deep learning models in Keras. The first was by using the sequential class where layers of the network are stacked on each other. Although these models are common amongst learners, they, however, have limitations. For instance, with the sequential model, the deep learning architecture cannot be made to share two different layers or have more than one input or output. In real-life applications, this is a bottleneck.

The good news is that the second method; using the functional API, solves this problem. The functional API gives room for more flexibility in the model architecture and is typically ideal for a more complex model. In this tutorial, we shall talk about the Keras functional API and go further to implement it using the same example in the last tutorial where the sequential model was used. By the end of this tutorial you will learn the following:

  • Sequential Models 
  • Using the Keras Functional Models 
  • Using a Functional Model to Fit a Linear Regression Problem. 
  • Building a Model with Shared Input Layer

Let’s begin with an overview of the Sequential model. 

Sequential Models 

Keras sequential model API is used to build deep learning models that allow the different layers to be stacked upon each other sequentially.  First, the sequential class must be instantiated then the layers are created and added to it. See an example below 

model = Sequential()
model.add(Dense(64, input_dim=1, activation='relu'))
model.add(Dense(32, input_dim=1, activation='relu'))
model.add(Dense(1, activation='linear'))

As seen in the code example above,  the sequential class was instantiated with a variable name model.  A fully connected layer with 64 nodes was added after which another fully connected layer with 32 nodes was added on top of the first hidden layer. Finally, the output layer with just one node was added. As explained earlier these kinds of models are not perfect for multiple input sources or output. 

All IT Courses 50% Off

Using the Keras Functional Models 

The Functional Model is another way of creating a deep learning model in Keras. It allows you to create layers that can be reused and have shared inputs and output data. The functional model is typically used for creating a more sophisticated model. Using the Functional Model method can be done in three steps. 

Step 1: Define the input 

When using the Functional model, you must first define the input layer as a standalone object and pass its shape. This is unlike the case in Sequential models where the shape of the input data is passed alongside the first hidden layer. 

The shape of the input data is a tuple of the number of features contained in the data. Note that the tuple must make room for the data splitting (in mini-batch size) that would take place during training. So for data with one feature, the shape is defined as (1, ). 

To define the input layer, the Input class must be imported from Keras layers.

from keras.layers import Input
#define the input layer
input_layer = Input(shape=(1, ))

Step 2: Create and Connect the Layers. 

The next step is to create the layers. When creating the layers, you will need to define the number of nodes the layer would have and importantly where the input data would come from. The input data is placed inside a bracket after defining the type of layer you wish to create. See the example below.

 from keras.layers import Input
 
#define the hidden layers and the data to be received. 
hidden_1 = Dense(64)(input_layer)

Now, this is the interesting part. Because you define where the input data comes from for every created layer, you are at liberty to use shared data inputs/outputs for different layers. Thus, making the possibilities with the Functional Models largely sophisticated. 

Step 3: Create the model

The final step is to create the model. The Model class has to be first imported from the models module of Keras. The class must then be instantiated, passing the inputs and outputs parameters as variable. The input parameters are the input layer already defined while the output data is the last hidden layer of the model.

from keras.models import Model
model = Model(inputs=input_layer, outputs=hidden_1)

The model can then be created and trained on the data. Let’s take an example. 

Using a Functional Model to Fit a Linear Regression Problem. 

In the last tutorial, we built a Sequential model to be trained on data points and produced the line of best fit. Here, we will do the same with a Function model with Keras. Let’s begin by importing the necessary libraries. 

#import the necessary libraries
import numpy as np
from matplotlib import pyplot as plt
from keras.models import Model
from keras.layers import Input, Dense, Dropout

Next is to create the dataset. 

#define a random seed
np.random.seed(42)
 
#create random points for the x and y axis from 0 to 20
xs = np.linspace(0, 20, 50)
ys = np.linspace(20, 0, 50)
 
# add some positive and negative noise on the y axis
ys += np.random.uniform(-2, 2, 50)
 
#plot the graph
plt.plot(xs, ys, '.');

Output:

Creating Deep Learning Model with Keras Functional API

Next is to build the model. Remember that the Input layer has to be instantiated with the shape of the expected data, Then the other hidden layers will be defined and called on the data it is expected to receive. The model architecture is built in the code below.

#create the input layer
input_layer = Input(shape=(1, ))
 
#create the other hidden layers and the output layer
hidden_1 = Dense(64, activation='relu')(input_layer)
hidden_2 = Dense(32, activation='relu')(hidden_1)
output = Dense(1, activation='linear')(hidden_2)

Finally, the Model class is instantiated receiving the already defined input layer and output layer as arguments.

model = Model(inputs=input_layer, outputs=output)

Before going ahead to train the model, we can visualize the model architecture using the plot_model module of keras. 

from keras.utils import plot_model
 
plot_model(model)

Output:

Creating Deep Learning Model with Keras Functional API

As seen, the model has an input layer, two fully connected layers and an output layer. You can get more information about the model architecture by using the summary() method.

model.summary()
Output:
Model: "model"
Creating Deep Learning Model with Keras Functional API

Total params: 2,241
Trainable params: 2,241
Non-trainable params: 0
The model can finally be compiled and trained.

model.compile(optimizer='adam', loss='mse')
history = model.fit(xs, ys, epochs=200)

Output:

Epoch 1/200
2/2 [==============================] - 1s 6ms/step - loss: 122.2916
Epoch 2/200
2/2 [==============================] - 0s 15ms/step - loss: 122.3255
Epoch 3/200
2/2 [==============================] - 0s 5ms/step - loss: 109.8412
Epoch 4/200
2/2 [==============================] - 0s 10ms/step - loss: 113.5361
Epoch 5/200
...
Epoch 196/200
2/2 [==============================] - 0s 10ms/step - loss: 3.6998
Epoch 197/200
2/2 [==============================] - 0s 10ms/step - loss: 4.8764
Epoch 198/200
2/2 [==============================] - 0s 5ms/step - loss: 4.6869
Epoch 199/200
2/2 [==============================] - 0s 5ms/step - loss: 4.5548
Epoch 200/200
2/2 [==============================] - ETA: 0s - loss: 2.816 - 0s 0s/step - loss: 3.6678

The prediction can then be made and the line plotted. 

#model makes prediction
y_pred = model.predict(xs)
 
#plot the graph
plt.plot(xs, y_pred, 'k', label='Predicted')
plt.scatter(xs, ys, label='True')
plt.legend()
plt.xlabel('X Axis')
plt.ylabel('Y Axis')
plt.title('A plot to show the prediction of the model')
;

Output:

Creating Deep Learning Model with Keras Functional API

This is a pretty decent result. 

Building a Model with Shared Input Layer

The Functional API model particularly shines when you wish to share the data on different layers or have multiple inputs and outputs. If we wish to build a model that shares its input layer, we just need to call the input layer function to different hidden layers. Then you can concatenate it with the concatenate sub-module in Keras. See an example below.

from keras.layers import merge
#create the input layer
input_layer = Input(shape=(1, ))
 
#create two hidden layers with shared input layer
hidden_1 = Dense(64, activation='relu')(input_layer)
hidden_2 = Dense(64, activation='relu')(input_layer)
 
#merge the two layers 
merged_layers = merge.concatenate([hidden_1, hidden_2])
 
#built the rest of the layers
hidden_3 = Dense(32, activation='relu')(merged_layers)
output = Dense(1, activation='linear')(hidden_3)
 
model = Model(inputs=input_layer, outputs=output)

Let’s visualize what the model looks like

from keras.utils import plot_model
plot_model(model)

Output:

Creating Deep Learning Model with Keras Functional API

We can also check its summary. 

model.summary()
Model: "model"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_1 (InputLayer)            [(None, 1)]          0                                            
__________________________________________________________________________________________________
dense (Dense)                   (None, 64)           128         input_1[0][0]                    
__________________________________________________________________________________________________
dense_1 (Dense)                 (None, 64)           128         input_1[0][0]                    
__________________________________________________________________________________________________
concatenate (Concatenate)       (None, 128)          0           dense[0][0]                      
                                                                 dense_1[0][0]                    
__________________________________________________________________________________________________
dense_2 (Dense)                 (None, 32)           4128        concatenate[0][0]                
__________________________________________________________________________________________________
dense_3 (Dense)                 (None, 1)            33          dense_2[0][0]                    
==================================================================================================
Total params: 4,417
Trainable params: 4,417
Non-trainable params: 0

Running this model also produces good results. 

model.compile(optimizer='adam', loss='mse')
history = model.fit(xs, ys, epochs=200)
Output:
Epoch 1/200
2/2 [==============================] - 1s 5ms/step - loss: 108.9526
Epoch 2/200
2/2 [==============================] - 0s 5ms/step - loss: 105.3066
Epoch 3/200
2/2 [==============================] - 0s 5ms/step - loss: 99.1887
Epoch 4/200
2/2 [==============================] - 0s 10ms/step - loss: 108.9373
Epoch 5/200
2/2 [==============================] - 0s 5ms/step - loss: 103.2374
...
Epoch 196/200
2/2 [==============================] - 0s 5ms/step - loss: 4.4243
Epoch 197/200
2/2 [==============================] - 0s 5ms/step - loss: 3.9060
Epoch 198/200
2/2 [==============================] - 0s 5ms/step - loss: 4.6090
Epoch 199/200
2/2 [==============================] - 0s 5ms/step - loss: 3.5898
Epoch 200/200
2/2 [==============================] - 0s 0s/step - loss: 3.4363

Now, we can plot the predicted values of the model

y_pred = model.predict(xs)
k
plt.plot(xs, y_pred, 'k', label='Predicted')
plt.scatter(xs, ys, label='True')
plt.legend()
plt.xlabel('X Axis')
plt.ylabel('Y Axis')
plt.title('A plot to show the prediction of the model')
;

Output:

Creating Deep Learning Model with Keras Functional API

In summary, 

We have explained how to use this Functional API in Keras. You learned that using this model creates more flexibility in building the architecture of the model and overall its performance. This is why it is mostly used in more complex situations than the sequential model. If you have any questions as to how the model was created, please leave them in the comment section and I’d do my best to answer them.

Facebook Comments

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Related Articles

Back to top button