Styjun

I have tried to implement a true positive metric in Keras :

def TP(y_true, y_pred):
 estimated = K.argmax(y_pred, axis=1)
 truth = K.argmax(y_true, axis=1)
 TP = K.sum(truth * estimated)
 return TP

based on my last layer output shape : (batch, 2).
The function has been tested with numpy argmax equivalent and works well.

I use a cross_entropy loss function and each epochs it gives me the metric value. But how this value could be a decimal number ? What am I doing wrong ? Thanks !

Edited : here is a sample code for the Keras model :

def TP(y_true, y_pred):
 estimated = K.argmax(y_pred, axis=1)
 truth = K.argmax(y_true, axis=1)
 TP = K.sum(truth * estimated)
 return TP

epochs = 10
batch_size = 2

model = Sequential([
 Dense(32, input_shape=(4,)),
 Activation('relu'),
 Dense(2),
 Activation('softmax'),
])
model.compile(optimizer='adam',
 loss='categorical_crossentropy',
 metrics=['accuracy', TP])

model.summary()

train = np.array([[17,0,1,0],[17,0,1,0],[17,0,1,0],[17,0,1,0],[17,0,1,0], [2,1,0,1],[0,1,0,1],[0,1,0,1],[0,1,0,1],[0,1,0,1]])
labels = np.array([ [1,0],[1,0],[1,0],[1,0],[1,0], [0,1],[0,1],[0,1],[0,1],[0,1] ])

model.fit(train, labels, epochs=epochs, batch_size=batch_size, verbose=2)

And here a test showing the TP function seems to work

def npTP(y_true, y_pred):
 estimated = np.argmax(y_pred, axis=1)
 truth = np.argmax(y_true, axis=1)
 TP = np.sum(truth * estimated)
 return TP

y_true = np.array([ [1,0],[1,0],[1,0],[1,0],[1,0], [0,1],[0,1],[0,1],[0,1],[0,1] ])
y_pred = np.array([ [0,1],[0,1],[0,1],[0,1],[0,1], [0,1],[0,1],[0,1],[0,1],[0,1]])
print("np check : ")
print(npTP(y_true, y_pred))

Running this code gives the following output :

Using TensorFlow backend.

_________________________________________________________________
Layer (type) Output Shape Param # 
=================================================================
dense_1 (Dense) (None, 32) 160 
_________________________________________________________________
activation_1 (Activation) (None, 32) 0 
_________________________________________________________________
dense_2 (Dense) (None, 2) 66 
_________________________________________________________________
activation_2 (Activation) (None, 2) 0 
=================================================================
Total params: 226
Trainable params: 226
Non-trainable params: 0
_________________________________________________________________
Epoch 1/10
 - 0s - loss: 0.3934 - acc: 0.6000 - TP: 0.2000
Epoch 2/10 ^^^^^^^^^^ here are the decimal values
 - 0s - loss: 0.3736 - acc: 0.6000 - TP: 0.2000
Epoch 3/10 ^^^^^^^^^^
 - 0s - loss: 0.3562 - acc: 0.6000 - TP: 0.2000
Epoch 4/10 ^^^^^^^^^^
 - 0s - loss: 0.3416 - acc: 0.7000 - TP: 0.4000
Epoch 5/10 ^^^^^^^^^^
 - 0s - loss: 0.3240 - acc: 1.0000 - TP: 1.0000
Epoch 6/10
 - 0s - loss: 0.3118 - acc: 1.0000 - TP: 1.0000
Epoch 7/10
 - 0s - loss: 0.2960 - acc: 1.0000 - TP: 1.0000
Epoch 8/10
 - 0s - loss: 0.2806 - acc: 1.0000 - TP: 1.0000
Epoch 9/10
 - 0s - loss: 0.2656 - acc: 1.0000 - TP: 1.0000
Epoch 10/10
 - 0s - loss: 0.2535 - acc: 1.0000 - TP: 1.0000

np check : 
5

Thanks !

As desertnaut pointed out, answer is explained in this thread.

Keras is doing a running average between batches & epochs.

Here with batch_size=2 and 10 samples, each epoch runs 5 trainings (10/2=5).

To understand output metric of epoch 1, the total number of TP after the 5 trainings had to be 1, so the metric gives 1/5 = 0.2. Epoch 4 had 2 TP's in the 5 trainings giving 2/5 = 0.4 in the metric.