Can stock prices be predicted with LSTM neural networks?

One of the first questions I asked myself when I started working with neural networks was, can I write a program to predict the price of a stock with a neural network? There are tons of examples floating around the Internet on forecasting or predicting stock market prices but most of them give only example code without actually addressing the question.

I’ve worked fairly extensively with machine learning technology, mainly using neural networks for object detection and facial recognition. One thing that I have learned is that neural network models need to be trained on data within a set of known parameters.

Can the human brain learn without first experiencing or seeing something? Can you use an orange to teach a young child that apples are red?

Most neural networks take a set of inputs to be trained for a given output or set of outputs. In the case of predicting stock prices the output will be the stock price. Recurrent neural networks (RNN) are very good at working with sequential data as are long short-term memory models (LSTM). The key difference between a RNN model and LSTM model is the LSTM model has a special memory cell that can retain historical information for longer periods of time.

When it comes to me experimenting with predicting stock prices with neural networks, I prefer to use the LSTM model due to it’s ability to remember historical information. Lets start off with some common LSTM Python code for predicting (forecasting) stock prices.

Python code for predicting stock prices.

(download the stock market dataset for the example code from my downloads page)

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, LSTM
from tensorflow.keras.callbacks import Callback
from numpy import array
import pandas as pd
import matplotlib.pyplot as plt


#Note: This code is part of a blog post exlaining why predicting stock prices from squential data isn't the greatest idea.  
#      (see jamesburnett.io for more information)



#This is a Callback class that is used to 
#provide reporting and training maintenance.
class ModelTrainingCallback(Callback):
	
	target_loss = 0.05

	def __init__(self,target_loss=0.05):
		self.target_loss = target_loss

	def on_epoch_end(self, epoch, logs={}):
		if(logs.get('loss') <= self.target_loss):
			print(F"Target Loss Reached: {self.target_loss}. Stopping.")
			self.model.stop_training = True
		else:
			current_loss = logs.get('loss')
			print(F"Epoch: {epoch} Loss: {current_loss}")

			

#This will split a Python list into
#sequencial data (multi dimentional array)
def split_sequence(sequence, n_steps):
	X, y = list(), list()
	for i in range(len(sequence)):
		end_ix = i + n_steps
		if end_ix > len(sequence)-1:
			break
		seq_x, seq_y = sequence[i:end_ix], sequence[end_ix]
		X.append(seq_x)
		y.append(seq_y)
	return array(X), array(y)


#Load the stock data. (get this at jamesburnett.io/downloads)
#and convert into a pandas data frame
data = pd.read_csv("DIA.csv")

stock_data = pd.DataFrame(index=range(0,len(data)),columns=['idx','close','volume'])

idx = 0

for i in range(len(data)-1,-1,-1):    
    stock_data['idx'][i] = idx  
    stock_data['close'][i] = float(data['close'][i])
    stock_data['volume'][i] = int(data['volume'][i])
    idx = idx + 1

stock_data.index = stock_data.idx

stock_data =stock_data.iloc[::-1]

stock_data.drop('idx', axis=1, inplace=True)

close_data = []

#Resort the data to be in a range
#from oldest to newest.
for idx in stock_data.index:
    c = stock_data['close'][idx]
    close_data.append(c)



#Normalized data between 0 and 1

normalized = [float(i)/max(close_data) for i in close_data]

n_features = 1

n_steps = 4

#Shape our normalied data into N steps of
#squential data using the functon above.
X, y = split_sequence(normalized,n_steps)

X_t = X.reshape((X.shape[0], X.shape[1], n_features))


#Get the next 4 squences and predict the output.
P = X[-4:]
P_t = P.reshape((P.shape[0], P.shape[1], n_features))


#Created our LSTM neural network model.
model = Sequential()

model.add(LSTM(units=32, return_sequences=True, activation='sigmoid',input_shape=(4,1)))

model.add(LSTM(units = 64, return_sequences = False))

model.add(Dense(1))

model.compile(optimizer='adam', loss='mean_squared_error',metrics=['accuracy'])

# Use a high loss (0.01) for testing the training, testing and prediction 
# and then switch to a loss of 0.0006 to train. (
# saves time finding code bugs)
modelCallback = ModelTrainingCallback(0.0006)


# Fit our training data (X_t) into the model and train the model.
history = model.fit(X_t, y, epochs=2000, verbose=0, callbacks=[modelCallback])


# Test our model by predicting data it has seen before.
test = model.predict(X_t)

test_c = [float(i)*max(close_data) for i in test]


# Predict the next sequence of data to determine the stock price (output)
pred = model.predict(P_t)

pred_c = [float(i)*max(close_data) for i in pred]


# Chart the stock price, tested price and predicted price.
fig, axs = plt.subplots(3, 1,figsize=(8,5))

axs[0].plot(close_data,color="#04b0c8", alpha=0.6)

axs[1].plot(test_c,color="#04b0c8", alpha=0.6)

axs[2].plot(pred_c,color="#04b0c8", alpha=0.6)

fig.tight_layout()

plt.show()

We can run this code and see how it produces predicted stock prices for the Dow Jones Industrial ETF DIA. As you can see it predicts that the price is going to go down.

Is this accurate? Is this code working correctly? Yes…..and ….no……maybe.

Why this code (idea) is wrong.

Let me tell you why this is not a well thought out idea. First, we must understand that historically the stock market has never crashed and not recovered. You heard me right, the stock market always recovers and always goes up. Don’t believe me? Leave a comment below and show me where the stock market has crashed and not recovered and I’ll have a six pack of your favorite beer delivered to you.

In addiction to the fact the stock market has never cashed and not recovered, the neural network has also not seen a stock price ceiling. The stock market is gaining new high prices every month, prices the neural network hasn’t seen before. If the neural network hasn’t seen these prices before, how can it predict or forecast the price?

When predicting the weather we can send neural network models information that has happened before. The weather models have seen their outputs before, rain, snow, ice etc. In the case of rising stock prices, those rising prices have never been seen by a stock predicting neural network model.

We attempting to solve the wrong problem.

I live outside the box. This means I think differently, I break rules and I sometimes think unconventionality. Many people are trained to think a certain way. Not me, I question everything, including these half-assed blog articles that post code with the sole purpose of generating traffic. My code compiles, trains and predicts data but that doesn’t mean you should blindly copy and paste it because it’s on the Internet.

When it comes to predicting the stock price I feel we are trying to solve the wrong problem, we are asking the wrong question. Instead of asking, can we predict the stock price, maybe we should be asking questions like:

If the stock market closes below it’s open on Wednesday, what does it do on Friday? Output: Win / Loss (win = daily close > daily open etc)

Can we use a convolutional neural network to detect common reversal patterns? Output: Image Classification for a certain pattern like a hammer.

Can we take advantage of LSTM’s to determine if a stock price will go up or down based on corporate earnings? Output: Up / Down on very specific dates based on historical data etc.

Can we look at the price movements of all underlying assets of an ETF to predict the direction the ETF may go on any given day? Output: Up / Down

Do any stocks move up or down in price on any particular holiday?

You can build legitimate outputs based on the questions above. These outputs that have been seen before which means we might be able to build a model to forecast or predict those outputs. Remember, neural networks are designed around how the brain operates, not how a programmer wants them to operate.

I encourage everyone to step outside their mind cages for a little while, stop coding and think about the scenarios and how. Ask yourself if you could teach a child to learn what you want your neural network to learn. Can you teach your child what the letters of the alphabet are? Yes, because they are defined. Can you teach your child what the stock price will be this time next year? No, because that event has not happened yet and like space, there is no limit. We can’t calculate the slope to a ceiling if there is no roof.

Now I’m not here to be a party pooper, I’m here to explain that there are better ways to predict and forecast data when it comes to the stock market. The key is to make sure that you are asking a question that can be answered.

Thanks for reading and remember, you can’t teach a child what an apple looks like if an apple has never been seen before.

Leave a Reply

Your email address will not be published. Required fields are marked *