Styjun

Old French song lyrics with the word "baiser."

How many oliphaunts died in all of the Lord of the Rings battles?

How to judge a Ph.D. applicant that arrives "out of thin air"

Copying an existing HTML page and use it, is that against any copyright law?

Did the IBM PC use the 8088's NMI line?

If my pay period is split between 2 calendar years, which tax year do I file them in?

Why force the nose of 737 Max down in the first place?

Why is 'n' preferred over "n" for output streams?

Use cases for M-0 & C-0?

Why is drive/partition number still used?

Help with one interview question --expected length of numbers drawn

What is the most convenient way to prepare ferrous oxide (FeO) in the laboratory?

What is the most common end of life issue for a car?

Sci-fi change: Too much or Not enough

Trapped in an ocean Temple in Minecraft?

Why did House of Representatives need to condemn Trumps Tweets?

How should we understand λαμβάνω in John 5:34?

When does Haskell complain about incorrect typing in functions?

Sci fi story: Clever pigs that help a galaxy lawman

Why does Canada require mandatory bilingualism in all government posts?

What do you call a flexible diving platform?

What is the most efficient way to write 'for' loops in Matlab?

What is the difference between position, displacement, and distance traveled?

How to apply the changes to my `.zshrc` file after edit?

How to deal with the non differentiability of argmax

Getting around tf.argmax which is not differentiableHow to merge two dictionaries in a single expression?How do I check if a list is empty?How do I check whether a file exists without exceptions?How can I safely create a nested directory?How do I sort a dictionary by value?How to make a chain of function decorators?How to make a flat list out of list of listsHow do I list all files of a directory?Is it possible to train an SVM or Random Forest on the final layer feature of a Convolutional Neural Network using Keras?Getting around tf.argmax which is not differentiable

.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;

3

I have a neural network that has the final layer as a flattened layer that provides an output as a curve. The curve might have one or more local maxima, but I am only interested in finding the global maxima. The ground truth I have is the integer index (or argument) of the global maxima. I tried to write a custom loss in Keras like this:

def custom_loss(y_output,y_idx_pred):
 return K.mean(K.sum((K.argmax(y_output)-y_idx_pred)**2))

I also cast the integers to float32. But I am getting error that there is 'None' gradient. I searched for an answer and found that argmax doesn't have any defined gradient. The suggestions I found was to either create a custom Argmax layer or to use softmax instead.

How do I even use softmax here? Softmax only provides me with the approximation of [0 0 1... ], not the integer index itself. How am I supposed to work with this? I even tried to treat the problem as a classification problem by turning the ground truth to [0 1 0... ] and used crossentropy, but the network could not learn anything. It did better when I just added a dense(1) layer and then trained the model. It seems like classification treats all the arguments equally, but that is not the case here. I need the euclidean l2 distance.

Where can I get a proper instruction of creating the custom Argmax layer? Will it even help in my situation? Is it possible to implement a custom loss function that is differentiable? What should I do?

python tensorflow keras

share|improve this question

edited Mar 26 at 17:44

Arkistarvh Kltzuonstev

3,50833 gold badges1313 silver badges3535 bronze badges

asked Mar 26 at 17:38

happypandahappypanda

1622 bronze badges

• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  
  Take a look stackoverflow.com/questions/46926809/…
  
  – Sharky
  Mar 26 at 17:46
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  That was really helpful. Thank you very much.
  
  – happypanda
  Mar 26 at 19:54
  

  
  
  
  

add a comment | 

3

I have a neural network that has the final layer as a flattened layer that provides an output as a curve. The curve might have one or more local maxima, but I am only interested in finding the global maxima. The ground truth I have is the integer index (or argument) of the global maxima. I tried to write a custom loss in Keras like this:

def custom_loss(y_output,y_idx_pred):
 return K.mean(K.sum((K.argmax(y_output)-y_idx_pred)**2))

I also cast the integers to float32. But I am getting error that there is 'None' gradient. I searched for an answer and found that argmax doesn't have any defined gradient. The suggestions I found was to either create a custom Argmax layer or to use softmax instead.

How do I even use softmax here? Softmax only provides me with the approximation of [0 0 1... ], not the integer index itself. How am I supposed to work with this? I even tried to treat the problem as a classification problem by turning the ground truth to [0 1 0... ] and used crossentropy, but the network could not learn anything. It did better when I just added a dense(1) layer and then trained the model. It seems like classification treats all the arguments equally, but that is not the case here. I need the euclidean l2 distance.

Where can I get a proper instruction of creating the custom Argmax layer? Will it even help in my situation? Is it possible to implement a custom loss function that is differentiable? What should I do?

python tensorflow keras

share|improve this question

edited Mar 26 at 17:44

Arkistarvh Kltzuonstev

3,50833 gold badges1313 silver badges3535 bronze badges

asked Mar 26 at 17:38

happypandahappypanda

1622 bronze badges

• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  
  Take a look stackoverflow.com/questions/46926809/…
  
  – Sharky
  Mar 26 at 17:46
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  That was really helpful. Thank you very much.
  
  – happypanda
  Mar 26 at 19:54
  

  
  
  
  

add a comment | 

I have a neural network that has the final layer as a flattened layer that provides an output as a curve. The curve might have one or more local maxima, but I am only interested in finding the global maxima. The ground truth I have is the integer index (or argument) of the global maxima. I tried to write a custom loss in Keras like this:

def custom_loss(y_output,y_idx_pred):
 return K.mean(K.sum((K.argmax(y_output)-y_idx_pred)**2))

I also cast the integers to float32. But I am getting error that there is 'None' gradient. I searched for an answer and found that argmax doesn't have any defined gradient. The suggestions I found was to either create a custom Argmax layer or to use softmax instead.

How do I even use softmax here? Softmax only provides me with the approximation of [0 0 1... ], not the integer index itself. How am I supposed to work with this? I even tried to treat the problem as a classification problem by turning the ground truth to [0 1 0... ] and used crossentropy, but the network could not learn anything. It did better when I just added a dense(1) layer and then trained the model. It seems like classification treats all the arguments equally, but that is not the case here. I need the euclidean l2 distance.

Where can I get a proper instruction of creating the custom Argmax layer? Will it even help in my situation? Is it possible to implement a custom loss function that is differentiable? What should I do?

python tensorflow keras

share|improve this question

edited Mar 26 at 17:44

Arkistarvh Kltzuonstev

3,50833 gold badges1313 silver badges3535 bronze badges

asked Mar 26 at 17:38

happypandahappypanda

1622 bronze badges

def custom_loss(y_output,y_idx_pred):
 return K.mean(K.sum((K.argmax(y_output)-y_idx_pred)**2))

share|improve this question

edited Mar 26 at 17:44

Arkistarvh Kltzuonstev

3,50833 gold badges1313 silver badges3535 bronze badges

asked Mar 26 at 17:38

happypandahappypanda

1622 bronze badges

share|improve this question

edited Mar 26 at 17:44

Arkistarvh Kltzuonstev

3,50833 gold badges1313 silver badges3535 bronze badges

asked Mar 26 at 17:38

happypandahappypanda

1622 bronze badges

edited Mar 26 at 17:44

Arkistarvh Kltzuonstev

3,50833 gold badges1313 silver badges3535 bronze badges

edited Mar 26 at 17:44

Arkistarvh Kltzuonstev

3,50833 gold badges1313 silver badges3535 bronze badges

asked Mar 26 at 17:38

happypandahappypanda

1622 bronze badges

asked Mar 26 at 17:38

happypandahappypanda

1622 bronze badges