1
\$\begingroup\$

I am working on a multiple classification problem and after dabbling with multiple neural network architectures, I settled for a stacked LSTM structure as it yields the best accuracy for my use-case. Unfortunately the network takes a long time (almost 48 hours) to reach a good accuracy (~1000 epochs) even when I use GPU acceleration.

At this point, I would like some feedback about my code and my results, did I miss something? and hopefully some suggestions on how to further speed-up and improve the training process/code.

My code is the following:

# -*- coding: utf-8 -*-
import keras
import numpy as np
from time import time
from utils import dmanip, vis
from keras.models import Sequential
from keras.layers import LSTM, Dense
from keras.utils import to_categorical
from keras.callbacks import TensorBoard
from sklearn.preprocessing import LabelEncoder
from tensorflow.python.client import device_lib
from sklearn.model_selection import train_test_split

###############################################################################
####################### Extract the data from .csv file #######################
###############################################################################
# get data
data, column_names = dmanip.get_data(file_path='../data_one_outcome.csv')

# split data
X = data.iloc[:, :-1]
y = data.iloc[:, -1:].astype('category')

###############################################################################
########################## init global config vars ############################
###############################################################################
# check if GPU is used
print(device_lib.list_local_devices())

# init
n_epochs = 1500
n_comps = X.shape[1]

###############################################################################
################################## Keras RNN ##################################
###############################################################################
# encode the classification labels
le = LabelEncoder()
yy = to_categorical(le.fit_transform(y))

# split the dataset
x_train, x_test, y_train, y_test = train_test_split(X, yy, test_size=0.35,
                                                    random_state=True,
                                                    shuffle=True)

# exapand dimensions
x_train = np.expand_dims(x_train, axis=2)
x_test = np.expand_dims(x_test, axis=2)

# define model
model = Sequential()
model.add(LSTM(units=n_comps, return_sequences=True,
               input_shape=(x_train.shape[1], 1),
               dropout=0.2, recurrent_dropout=0.2))
model.add(LSTM(64, return_sequences=True, dropout=0.2, recurrent_dropout=0.2))
model.add(LSTM(32, dropout=0.2, recurrent_dropout=0.2))
model.add(Dense(4 ,activation='softmax'))

# print model architecture summary
print(model.summary())

# compile model
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

# Create a TensorBoard instance with the path to the logs directory
tensorboard = TensorBoard(log_dir='./logs/rnn/{}'.format(time()))

# fit the model
history = model.fit(x_train, y_train, epochs=n_epochs, batch_size=100,
                    validation_data=(x_test, y_test), callbacks=[tensorboard])

# plot results
vis.plot_nn_stats(history=history, stat_type="accuracy", fname="RNN-accuracy")
vis.plot_nn_stats(history=history, stat_type="loss", fname="RNN-loss")

The resulting accuracy and loss functions are:

My data is essentially a big matrix (38607, 150), where 150 is the number of features and 38607 is the number of samples, with a target vector including 4 classes.

\$\endgroup\$
  • \$\begingroup\$ Hi, if you have an NVIDIA GPU card then you could use the cudnnLSTM to speed up your network training time. However, cudnnLSTM has fewer options compared to the LSTM. \$\endgroup\$ – Nestoras Chalkidis Dec 19 '19 at 10:31
  • \$\begingroup\$ You'd be better off posting this exact same question in the Keras subreddit, for example. I'm not sure you'd get the advice you need in this general code review site. \$\endgroup\$ – Zchpyvr Dec 31 '19 at 21:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.