Styjun

Is it possible to have a career in SciComp without contributing to arms research?

Why are there few or no black super GMs?

How long were the Apollo astronauts allowed to breathe 100% oxygen at 1 atmosphere continuously?

How to tell readers that I know my story is factually incorrect?

Pure functions instead of For loop

Can error correction and detection be done without adding extra bits?

How to interpret a promising preprint that was never published in peer-review?

How did J. J. Thomson establish the particle nature of the electron?

Why teach C using scanf without talking about command line arguments?

How important are the Author's mood and feelings for writing a story?

Improving an O(N^2) function (all entities iterating over all other entities)

Who determines when road center lines are solid or dashed?

Do higher dimensions have axes?

literal `0` beeing a valid candidate for int and const string& overloads causes ambiguous call

Who or what determines if a curse is valid or not?

Pauli exclusion principle - black holes

"Je suis petite, moi?", purpose of the "moi"?

Left crank keeps coming loose

May I use a railway velocipede on actively-used British railways?

What's a German word for »Sandbagger«?

Apex Legends stuck at 60 FPS (G-Sync 144hz monitor)

How can I help our ranger feel special about her beast companion?

What would be the safest way to drop thousands of small, hard objects from a typical, high wing, GA airplane?

When designing an adventure, how can I ensure a continuous player experience in a setting that's likely to favor TPKs?

Keras deviance custom loss

customised loss function in keras using theano functionImplementing a custom loss function in KerasException in Tensorflow function used as Keras custom lossMake a custom loss function in kerasHow does Keras read input data?CTC Loss not decreasing in KerasPrecision@n and Recall@n in Keras Neural NetworkModel not converge (loss not decrease)Is it possible to train a CNN starting at an intermediate layer (in general and in Keras)?'Sequential' object has no attribute 'loss' - When I used GridSearchCV to tuning my Keras model

.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;

0

I am trying to use deviance as a custom loss function for optimizing a neural network wuth keras. I tried this :

#building model
model = keras.Sequential()
model.add(Dense(10, input_dim = 6, activation = "relu"))
model.add(Dense(5, activation = "relu"))
model.add(Dense(1, activation = "sigmoid"))

#DEF CUSTOM LOSS
def custom_loss():
 def loss(y_true, y_pred):
 return (2. *(KB.log(y_true) - KB.log(y_pred)))
 return loss


model.compile(loss = custom_loss(), optimizer = 'sgd')
model.fit(factorsTrain, yTrain, epochs = 2)

But it gives -inf as the loss so i guess it doesn't work properly at all, is there something i did wrong in there ?

Edit : I changed the activation to exponential in last layer to ensure values are between 0 and 1. I also noticed that since some of my y_true (most of them in fact) are equal to 0 i changed the loss function to this (also added epsilon which is 1e-07 to make sure i don't calculate the ln(0) :

#DEF CUSTOM LOSS
def custom_loss():
 def loss(y_true, y_pred):

 return (( KB.sqrt( KB.square(2 * (KB.log(y_true + KB.epsilon()) - KB.log(y_pred + KB.epsilon())) ))))
 return loss

Now i don't get -inf anymore but i still get NaN instead

python machine-learning keras

share|improve this question

edited Mar 26 at 11:51

Lucien Ledune

asked Mar 26 at 10:36

Lucien LeduneLucien Ledune

1166 bronze badges

add a comment | 

0

I am trying to use deviance as a custom loss function for optimizing a neural network wuth keras. I tried this :

#building model
model = keras.Sequential()
model.add(Dense(10, input_dim = 6, activation = "relu"))
model.add(Dense(5, activation = "relu"))
model.add(Dense(1, activation = "sigmoid"))

#DEF CUSTOM LOSS
def custom_loss():
 def loss(y_true, y_pred):
 return (2. *(KB.log(y_true) - KB.log(y_pred)))
 return loss


model.compile(loss = custom_loss(), optimizer = 'sgd')
model.fit(factorsTrain, yTrain, epochs = 2)

But it gives -inf as the loss so i guess it doesn't work properly at all, is there something i did wrong in there ?

Edit : I changed the activation to exponential in last layer to ensure values are between 0 and 1. I also noticed that since some of my y_true (most of them in fact) are equal to 0 i changed the loss function to this (also added epsilon which is 1e-07 to make sure i don't calculate the ln(0) :

#DEF CUSTOM LOSS
def custom_loss():
 def loss(y_true, y_pred):

 return (( KB.sqrt( KB.square(2 * (KB.log(y_true + KB.epsilon()) - KB.log(y_pred + KB.epsilon())) ))))
 return loss

Now i don't get -inf anymore but i still get NaN instead

python machine-learning keras

share|improve this question

edited Mar 26 at 11:51

Lucien Ledune

asked Mar 26 at 10:36

Lucien LeduneLucien Ledune

1166 bronze badges

add a comment | 

I am trying to use deviance as a custom loss function for optimizing a neural network wuth keras. I tried this :

#building model
model = keras.Sequential()
model.add(Dense(10, input_dim = 6, activation = "relu"))
model.add(Dense(5, activation = "relu"))
model.add(Dense(1, activation = "sigmoid"))

#DEF CUSTOM LOSS
def custom_loss():
 def loss(y_true, y_pred):
 return (2. *(KB.log(y_true) - KB.log(y_pred)))
 return loss


model.compile(loss = custom_loss(), optimizer = 'sgd')
model.fit(factorsTrain, yTrain, epochs = 2)

But it gives -inf as the loss so i guess it doesn't work properly at all, is there something i did wrong in there ?

Edit : I changed the activation to exponential in last layer to ensure values are between 0 and 1. I also noticed that since some of my y_true (most of them in fact) are equal to 0 i changed the loss function to this (also added epsilon which is 1e-07 to make sure i don't calculate the ln(0) :

#DEF CUSTOM LOSS
def custom_loss():
 def loss(y_true, y_pred):

 return (( KB.sqrt( KB.square(2 * (KB.log(y_true + KB.epsilon()) - KB.log(y_pred + KB.epsilon())) ))))
 return loss

Now i don't get -inf anymore but i still get NaN instead

python machine-learning keras

share|improve this question

edited Mar 26 at 11:51

Lucien Ledune

asked Mar 26 at 10:36

Lucien LeduneLucien Ledune

1166 bronze badges

#building model
model = keras.Sequential()
model.add(Dense(10, input_dim = 6, activation = "relu"))
model.add(Dense(5, activation = "relu"))
model.add(Dense(1, activation = "sigmoid"))

#DEF CUSTOM LOSS
def custom_loss():
 def loss(y_true, y_pred):
 return (2. *(KB.log(y_true) - KB.log(y_pred)))
 return loss


model.compile(loss = custom_loss(), optimizer = 'sgd')
model.fit(factorsTrain, yTrain, epochs = 2)

#DEF CUSTOM LOSS
def custom_loss():
 def loss(y_true, y_pred):

 return (( KB.sqrt( KB.square(2 * (KB.log(y_true + KB.epsilon()) - KB.log(y_pred + KB.epsilon())) ))))
 return loss

share|improve this question

edited Mar 26 at 11:51

Lucien Ledune

asked Mar 26 at 10:36

Lucien LeduneLucien Ledune

1166 bronze badges

share|improve this question

edited Mar 26 at 11:51

Lucien Ledune

asked Mar 26 at 10:36

Lucien LeduneLucien Ledune

1166 bronze badges

asked Mar 26 at 10:36

Lucien LeduneLucien Ledune

1166 bronze badges

asked Mar 26 at 10:36

Lucien LeduneLucien Ledune

1166 bronze badges

1 Answer
1

active

oldest

votes

0

Managed to fix it by changing the formula a little bit in order to force the values inside logs to be >= 0

def Deviance_loss():
 def loss(y_true, y_pred):
 y_true = KB.max(y_true, 0)
 return (KB.sqrt(KB.square( 2 * KB.log(y_true + KB.epsilon()) - KB.log(y_pred))))
 return loss

share|improve this answer

answered Mar 26 at 17:38

Lucien LeduneLucien Ledune

1166 bronze badges

add a comment | 

Your Answer

StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55355048%2fkeras-deviance-custom-loss%23new-answer', 'question_page');

);

Post as a guest

Name

Email

Required, but never shown

0

Managed to fix it by changing the formula a little bit in order to force the values inside logs to be >= 0

def Deviance_loss():
 def loss(y_true, y_pred):
 y_true = KB.max(y_true, 0)
 return (KB.sqrt(KB.square( 2 * KB.log(y_true + KB.epsilon()) - KB.log(y_pred))))
 return loss

share|improve this answer

answered Mar 26 at 17:38

Lucien LeduneLucien Ledune

1166 bronze badges

add a comment | 

0

Managed to fix it by changing the formula a little bit in order to force the values inside logs to be >= 0

def Deviance_loss():
 def loss(y_true, y_pred):
 y_true = KB.max(y_true, 0)
 return (KB.sqrt(KB.square( 2 * KB.log(y_true + KB.epsilon()) - KB.log(y_pred))))
 return loss

share|improve this answer

answered Mar 26 at 17:38

Lucien LeduneLucien Ledune

1166 bronze badges

add a comment | 

Managed to fix it by changing the formula a little bit in order to force the values inside logs to be >= 0

def Deviance_loss():
 def loss(y_true, y_pred):
 y_true = KB.max(y_true, 0)
 return (KB.sqrt(KB.square( 2 * KB.log(y_true + KB.epsilon()) - KB.log(y_pred))))
 return loss

share|improve this answer

answered Mar 26 at 17:38

Lucien LeduneLucien Ledune

1166 bronze badges

def Deviance_loss():
 def loss(y_true, y_pred):
 y_true = KB.max(y_true, 0)
 return (KB.sqrt(KB.square( 2 * KB.log(y_true + KB.epsilon()) - KB.log(y_pred))))
 return loss

share|improve this answer

answered Mar 26 at 17:38

Lucien LeduneLucien Ledune

1166 bronze badges

answered Mar 26 at 17:38

Lucien LeduneLucien Ledune

1166 bronze badges

answered Mar 26 at 17:38

Lucien LeduneLucien Ledune

1166 bronze badges