Styjun

I'm using a neural network to learn an equation(Black-Scholes formula for option pricing). I've managed to let the NN to approximate the equation output well but I also want the derivative of the equation to be learned as well. In order to examine the result, I need to calculate the derivative of the NN output w.r.t to the first variable of my input.

My model is a 5 layers fully connected NN. I've already got the gradients for each layer but I'm not sure whether I'm calculating the derivative in the right way.

In the code below I'm calculating the derivative for the first variable by doing dot product for all layers and ignoring the bias. I'm just wondering whether this is the right way to do? Because after calculating the derivatives for all the test cases, I found them to be quite far away from their theoretical values. So either my code is wrong or I'm just failing to learn the equation correctly.

P.S. This is what the gradient structure looks like:

[<tf.Tensor 'gradients/dense_1/MatMul_grad/MatMul_1:0' shape=(2, 128) dtype=float32>,
 <tf.Tensor 'gradients/dense_1/BiasAdd_grad/BiasAddGrad:0' shape=(128,) dtype=float32>,
 <tf.Tensor 'gradients/dense_2/MatMul_grad/MatMul_1:0' shape=(128, 64) dtype=float32>,
 <tf.Tensor 'gradients/dense_2/BiasAdd_grad/BiasAddGrad:0' shape=(64,) dtype=float32>,
 <tf.Tensor 'gradients/dense_3/MatMul_grad/MatMul_1:0' shape=(64, 32) dtype=float32>,
 <tf.Tensor 'gradients/dense_3/BiasAdd_grad/BiasAddGrad:0' shape=(32,) dtype=float32>,
 <tf.Tensor 'gradients/dense_4/MatMul_grad/MatMul_1:0' shape=(32, 16) dtype=float32>,
 <tf.Tensor 'gradients/dense_4/BiasAdd_grad/BiasAddGrad:0' shape=(16,) dtype=float32>,
 <tf.Tensor 'gradients/dense_5/MatMul_grad/MatMul_1:0' shape=(16, 1) dtype=float32>,
 <tf.Tensor 'gradients/dense_5/BiasAdd_grad/BiasAddGrad:0' shape=(1,) dtype=float32>]

[enter image description here][1]

# My NN structure: 

def build_nn_model(feats):
 model = models.Sequential()

 model.add(layers.Dense(128, activation='relu',
 input_shape=(len(feats),))) 

 model.add(layers.Dense(64, activation='relu')) 

 model.add(layers.Dense(32, activation='relu')) 

 model.add(layers.Dense(16, activation='relu'))

 model.add(layers.Dense(1))
 model.compile(optimizer='rmsprop', loss='mse', metrics=['mse'])

 return model

# to get the gradient:

outputs = nn_model.output
trainable_variables = nn_model.trainable_weights
gradients = K.gradients(outputs, trainable_variables)

sess = tf.InteractiveSession()
sess.run(tf.initialize_all_variables())

# to calculate the derivative

trainingExample = X_val[['var_1', 'var_2']].iloc[0].values.reshape(1, 2)
evaluated_gradients = sess.run(gradients,feed_dict=nn_model.input:trainingExample)

layer_1 = evaluated_gradients[0][0].reshape(1, 128)
layer_2 = evaluated_gradients[2]
layer_3 = evaluated_gradients[4]
layer_4 = evaluated_gradients[6]
layer_5 = evaluated_gradients[8]

derivative = layer_1.dot(layer_2).dot(layer_3).dot(layer_4).dot(layer_5)


 [1]: https://i.stack.imgur.com/Kuidb.png