# Keras stacked LSTM model for multiclass classification

I am working on a multiple classification problem and after dabbling with multiple neural network architectures, I settled for a stacked LSTM structure as it yields the best accuracy for my use-case. Unfortunately the network takes a long time (almost 48 hours) to reach a good accuracy (~1000 epochs) even when I use GPU acceleration.

At this point, I would like some feedback about my code and my results, did I miss something? and hopefully some suggestions on how to further speed-up and improve the training process/code.

My code is the following:

# -*- coding: utf-8 -*-
import keras
import numpy as np
from time import time
from utils import dmanip, vis
from keras.models import Sequential
from keras.layers import LSTM, Dense
from keras.utils import to_categorical
from keras.callbacks import TensorBoard
from sklearn.preprocessing import LabelEncoder
from tensorflow.python.client import device_lib
from sklearn.model_selection import train_test_split

###############################################################################
####################### Extract the data from .csv file #######################
###############################################################################
# get data
data, column_names = dmanip.get_data(file_path='../data_one_outcome.csv')

# split data
X = data.iloc[:, :-1]
y = data.iloc[:, -1:].astype('category')

###############################################################################
########################## init global config vars ############################
###############################################################################
# check if GPU is used
print(device_lib.list_local_devices())

# init
n_epochs = 1500
n_comps = X.shape[1]

###############################################################################
################################## Keras RNN ##################################
###############################################################################
# encode the classification labels
le = LabelEncoder()
yy = to_categorical(le.fit_transform(y))

# split the dataset
x_train, x_test, y_train, y_test = train_test_split(X, yy, test_size=0.35,
random_state=True,
shuffle=True)

# exapand dimensions
x_train = np.expand_dims(x_train, axis=2)
x_test = np.expand_dims(x_test, axis=2)

# define model
model = Sequential()
input_shape=(x_train.shape[1], 1),
dropout=0.2, recurrent_dropout=0.2))

# print model architecture summary
print(model.summary())

# compile model

# Create a TensorBoard instance with the path to the logs directory
tensorboard = TensorBoard(log_dir='./logs/rnn/{}'.format(time()))

# fit the model
history = model.fit(x_train, y_train, epochs=n_epochs, batch_size=100,
validation_data=(x_test, y_test), callbacks=[tensorboard])

# plot results
vis.plot_nn_stats(history=history, stat_type="accuracy", fname="RNN-accuracy")
vis.plot_nn_stats(history=history, stat_type="loss", fname="RNN-loss")


The resulting accuracy and loss functions are:

My data is essentially a big matrix (38607, 150), where 150 is the number of features and 38607 is the number of samples, with a target vector including 4 classes.

• Hi, if you have an NVIDIA GPU card then you could use the cudnnLSTM to speed up your network training time. However, cudnnLSTM has fewer options compared to the LSTM. – Nestoras Chalkidis Dec 19 '19 at 10:31
• You'd be better off posting this exact same question in the Keras subreddit, for example. I'm not sure you'd get the advice you need in this general code review site. – Zchpyvr Dec 31 '19 at 21:22

If I understand your problem, your samples are 150-dimensional vectors which is to be classified into one of four categories. If so, an LSTM architecture is a very poor fit because there are no relations between your samples. I.e what is in the n:th sample doesn't impact what is in the n + 1:th sample.

from keras.layers import Dense
from keras.models import Sequential
from keras.utils import to_categorical
import numpy as np

n = 38607
d = 150
k = 4

# Generate data
X = np.random.randn(n, d)
Y = to_categorical(np.random.randint(k, size = n), k)

model = Sequential()
model.add(Dense(128, input_dim = d, activation = 'relu'))