Styjun

I am experiencing with Tensorflow via the Keras library and before diving into predicting uncertainty, I thought it might be a good idea to predict something certain. Therefore, I tried to predict weekly returns using daily price level data. My input shape looks like this: (1000, 5, 2), i.e. 1000 matrices of the form:

Stock A Stock B
 110 100
 95 101
 90 100 
 89 99
 100 110

For Stock A the price at day t=0is 100, 95 at t-1 and 100 at t-5. Thus, the weekly return for Stock A would be 110/100=10%and -10% for Stock B. Because I focus on only predicting Stock As return for now, my y for this input matrix would just be the scalar 0.01. Furthermore, I want to make it a classification problem and thus make a one-hot encoded vector via to_categorical with 1 if the y is above 5%, 2 if it is below -5% and 0 if it is in between. Hence my classification output for the aforementioned matrix would be:

0 1 0

To simplify: I want my model to learn to calculate returns, i.e. divide the first value in the input matrix by the last value of the input matrix for stock A and ignore the input for stock B. This would give the y. It is just a practice task for me before I get to more difficult tasks and the model should achieve a loss of zero because there is no uncertainty. What model do you propose to do that? I tried the following and it does not converge at all. Training and validation weights are calculated via compute_sample_weight('balanced', ).

Earlystop = EarlyStopping(monitor='val_loss', patience=150, mode='min', verbose=1, min_delta=0.0002, restore_best_weights=True) 
checkpoint = ModelCheckpoint('nn', monitor='val_loss', verbose=1, save_best_only=True, mode='min', save_weights_only=False)
Plateau = ReduceLROnPlateau(monitor='val_loss', factor=0.5, patience=30, verbose=1)
optimizer = optimizers.Adam(lr=0.0001, beta_1=0.9, beta_2=0.999, amsgrad=True)

input_ = Input(batch_shape=(batch_size, 1, 5, 2)) 

model = LocallyConnected2D(16, kernel_size=(5, 1), padding='valid', data_format="channels_first")(input_)
model = LeakyReLU(alpha=0.01)(model)
model = Dense(128)(model)
model = LeakyReLU(alpha=0.01)(model)
model = Flatten()(model)

x1 = Dense(3, activation='softmax', name='0')(model)
final_model = Model(inputs=input_, outputs=[x1])
final_model.compile(loss='categorical_crossentropy' , optimizer=optimizer, metrics=['accuracy'])
history = final_model.fit(X_train, y_train, epochs=1000, batch_size=batch_size, verbose=2, shuffle=False, validation_data=[X_valid, y_valid, valid_weight], sample_weight=train_weight, callbacks=[Earlystop, checkpoint, Plateau])

I thought convolution might be good for this and because every return is calcualted individually I decided to go for a LocallyConnected layer. Do I need to add more layers for such a simple task?

EDIT: transformed my input matrix to returns and the model converges successfully. So the input must be correct but the model fails to find the division function. Are there any layers that would be suited to do that?