Styjun

Why would the US President need briefings on UFOs?

Is "stainless" a bulk or a surface property of stainless steel?

Why doesn't the Falcon-9 first stage use three legs to land?

Why does The Ancient One think differently about Doctor Strange in Endgame than the film Doctor Strange?

Do ability scores have any effect on casting Wish spell

Why can't an Airbus A330 dump fuel in an emergency?

Brexit and backstop: would changes require unanimous approval by all Euro countries? Does Ireland hold a veto?

Was Switzerland really impossible to invade during WW2?

What can I do to keep a threaded bolt from falling out of its slot?

What is the hex versus octal timeline?

(Why) May a Beit Din refuse to bury a body in order to coerce a man into giving a divorce?

How to dismiss intrusive questions from a colleague with whom I don't work?

Why didn’t Doctor Strange stay in the original winning timeline?

The economy of trapping

confused about grep and the * wildcard

Can you feel passing through the sound barrier in an F-16?

Descent a representation over finite field

Most practical knots for hitching a line to an object while keeping the bitter end as tight as possible, without sag?

Get info from plist file

Is it best to use a tie when using 8th notes off the beat?

IndexOptimize - Configuration

Potential new partner angry about first collaboration - how to answer email to close up this encounter in a graceful manner

What is the evidence on the danger of feeding whole blueberries and grapes to infants and toddlers?

Avoiding racist tropes in fantasy

Will tf.gradients pass through tf.cond?

Custom operation implementation for RBM/DBN with tensorflow?Confused by the behavior of `tf.cond`Understanding why tensorflow RNN is not learning toy dataSeparate gradients in tf.gradientsHow to combine FCNN and RNN in Tensorflow?How can i input boolean tensors to tf.cond() not just one boolean?tensor forest estimator value error at fitting the training part3darray training/testing TensorFlow RNN LSTMWhat's the meaning of grad_ys of tf.gradients?Efficiency of tf.gradients()

.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;

2

I would like to create a pair of recurrent neural networks, say NN₁ and NN₂, where NN₂ reproduces its output from the previous time step and does not update its weights at the current time step whenever NN₁ outputs a different value from the previous time step.

To do this, I was planning to use tf.cond() together with tf.stop_gradients(). However, in all toy examples I have run, I cannot get tf.gradients() to pass through tf.cond(): tf.gradients() simply returns [None].

Here is a simple toy example:

import tensorflow as tf

x = tf.constant(5)
y = tf.constant(3)

mult = tf.multiply(x, y)
cond = tf.cond(pred = tf.constant(True),
 true_fn = lambda: mult,
 false_fn = lambda: mult)

grad = tf.gradients(cond, x) # Returns [None]

Here is another simple toy example where I define true_fn and false_fn in tf.cond() (still no dice):

import tensorflow as tf

x = tf.constant(5)
y = tf.constant(3)
z = tf.constant(8)

cond = tf.cond(pred = x < y,
 true_fn = lambda: tf.add(x, z),
 false_fn = lambda: tf.square(y))

tf.gradients(cond, z) # Returns [None]

I originally thought that the gradient should flow through both true_fn and and false_fn, but clearly no gradient is flowing at all. Is this the expected behavior of gradients computed through tf.cond()? Might there be a way around this issue?

tensorflow

share|improve this question

asked Mar 27 at 15:48

cjurbancjurban

1722 bronze badges

add a comment | 

2

I would like to create a pair of recurrent neural networks, say NN₁ and NN₂, where NN₂ reproduces its output from the previous time step and does not update its weights at the current time step whenever NN₁ outputs a different value from the previous time step.

To do this, I was planning to use tf.cond() together with tf.stop_gradients(). However, in all toy examples I have run, I cannot get tf.gradients() to pass through tf.cond(): tf.gradients() simply returns [None].

Here is a simple toy example:

import tensorflow as tf

x = tf.constant(5)
y = tf.constant(3)

mult = tf.multiply(x, y)
cond = tf.cond(pred = tf.constant(True),
 true_fn = lambda: mult,
 false_fn = lambda: mult)

grad = tf.gradients(cond, x) # Returns [None]

Here is another simple toy example where I define true_fn and false_fn in tf.cond() (still no dice):

import tensorflow as tf

x = tf.constant(5)
y = tf.constant(3)
z = tf.constant(8)

cond = tf.cond(pred = x < y,
 true_fn = lambda: tf.add(x, z),
 false_fn = lambda: tf.square(y))

tf.gradients(cond, z) # Returns [None]

I originally thought that the gradient should flow through both true_fn and and false_fn, but clearly no gradient is flowing at all. Is this the expected behavior of gradients computed through tf.cond()? Might there be a way around this issue?

tensorflow

share|improve this question

asked Mar 27 at 15:48

cjurbancjurban

1722 bronze badges

add a comment | 

I would like to create a pair of recurrent neural networks, say NN₁ and NN₂, where NN₂ reproduces its output from the previous time step and does not update its weights at the current time step whenever NN₁ outputs a different value from the previous time step.

To do this, I was planning to use tf.cond() together with tf.stop_gradients(). However, in all toy examples I have run, I cannot get tf.gradients() to pass through tf.cond(): tf.gradients() simply returns [None].

Here is a simple toy example:

import tensorflow as tf

x = tf.constant(5)
y = tf.constant(3)

mult = tf.multiply(x, y)
cond = tf.cond(pred = tf.constant(True),
 true_fn = lambda: mult,
 false_fn = lambda: mult)

grad = tf.gradients(cond, x) # Returns [None]

Here is another simple toy example where I define true_fn and false_fn in tf.cond() (still no dice):

import tensorflow as tf

x = tf.constant(5)
y = tf.constant(3)
z = tf.constant(8)

cond = tf.cond(pred = x < y,
 true_fn = lambda: tf.add(x, z),
 false_fn = lambda: tf.square(y))

tf.gradients(cond, z) # Returns [None]

I originally thought that the gradient should flow through both true_fn and and false_fn, but clearly no gradient is flowing at all. Is this the expected behavior of gradients computed through tf.cond()? Might there be a way around this issue?

tensorflow

share|improve this question

asked Mar 27 at 15:48

cjurbancjurban

1722 bronze badges

import tensorflow as tf

x = tf.constant(5)
y = tf.constant(3)

mult = tf.multiply(x, y)
cond = tf.cond(pred = tf.constant(True),
 true_fn = lambda: mult,
 false_fn = lambda: mult)

grad = tf.gradients(cond, x) # Returns [None]

import tensorflow as tf

x = tf.constant(5)
y = tf.constant(3)
z = tf.constant(8)

cond = tf.cond(pred = x < y,
 true_fn = lambda: tf.add(x, z),
 false_fn = lambda: tf.square(y))

tf.gradients(cond, z) # Returns [None]

share|improve this question

asked Mar 27 at 15:48

cjurbancjurban

1722 bronze badges

share|improve this question

asked Mar 27 at 15:48

cjurbancjurban

1722 bronze badges

asked Mar 27 at 15:48

cjurbancjurban

1722 bronze badges

asked Mar 27 at 15:48

cjurbancjurban

1722 bronze badges

1 Answer
1

active

oldest

votes

0

Yes, the gradients will pass through tf.cond(). You just need to use floats instead of integers and (preferably) use variables instead of constants:

import tensorflow as tf

x = tf.Variable(5.0, dtype=tf.float32)
y = tf.Variable(6.0, dtype=tf.float32)
z = tf.Variable(8.0, dtype=tf.float32)

cond = tf.cond(pred = x < y,
 true_fn = lambda: tf.add(x, z),
 false_fn = lambda: tf.square(y))

op = tf.gradients(cond, z) 
# Returns [<tf.Tensor 'gradients_1/cond_1/Add/Switch_1_grad/cond_grad:0' shape=() dtype=float32>]

with tf.Session() as sess:
 sess.run(tf.global_variables_initializer())
 print(sess.run(op)) # [1.0] 

share|improve this answer

edited Mar 27 at 16:48

answered Mar 27 at 16:37

VladVlad

3,87655 gold badges1414 silver badges3131 bronze badges

add a comment | 

Your Answer

StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55381342%2fwill-tf-gradients-pass-through-tf-cond%23new-answer', 'question_page');

);

Post as a guest

Name

Email

Required, but never shown

0

Yes, the gradients will pass through tf.cond(). You just need to use floats instead of integers and (preferably) use variables instead of constants:

import tensorflow as tf

x = tf.Variable(5.0, dtype=tf.float32)
y = tf.Variable(6.0, dtype=tf.float32)
z = tf.Variable(8.0, dtype=tf.float32)

cond = tf.cond(pred = x < y,
 true_fn = lambda: tf.add(x, z),
 false_fn = lambda: tf.square(y))

op = tf.gradients(cond, z) 
# Returns [<tf.Tensor 'gradients_1/cond_1/Add/Switch_1_grad/cond_grad:0' shape=() dtype=float32>]

with tf.Session() as sess:
 sess.run(tf.global_variables_initializer())
 print(sess.run(op)) # [1.0] 

share|improve this answer

edited Mar 27 at 16:48

answered Mar 27 at 16:37

VladVlad

3,87655 gold badges1414 silver badges3131 bronze badges

add a comment | 

0

Yes, the gradients will pass through tf.cond(). You just need to use floats instead of integers and (preferably) use variables instead of constants:

import tensorflow as tf

x = tf.Variable(5.0, dtype=tf.float32)
y = tf.Variable(6.0, dtype=tf.float32)
z = tf.Variable(8.0, dtype=tf.float32)

cond = tf.cond(pred = x < y,
 true_fn = lambda: tf.add(x, z),
 false_fn = lambda: tf.square(y))

op = tf.gradients(cond, z) 
# Returns [<tf.Tensor 'gradients_1/cond_1/Add/Switch_1_grad/cond_grad:0' shape=() dtype=float32>]

with tf.Session() as sess:
 sess.run(tf.global_variables_initializer())
 print(sess.run(op)) # [1.0] 

share|improve this answer

edited Mar 27 at 16:48

answered Mar 27 at 16:37

VladVlad

3,87655 gold badges1414 silver badges3131 bronze badges

add a comment | 

Yes, the gradients will pass through tf.cond(). You just need to use floats instead of integers and (preferably) use variables instead of constants:

import tensorflow as tf

x = tf.Variable(5.0, dtype=tf.float32)
y = tf.Variable(6.0, dtype=tf.float32)
z = tf.Variable(8.0, dtype=tf.float32)

cond = tf.cond(pred = x < y,
 true_fn = lambda: tf.add(x, z),
 false_fn = lambda: tf.square(y))

op = tf.gradients(cond, z) 
# Returns [<tf.Tensor 'gradients_1/cond_1/Add/Switch_1_grad/cond_grad:0' shape=() dtype=float32>]

with tf.Session() as sess:
 sess.run(tf.global_variables_initializer())
 print(sess.run(op)) # [1.0] 

share|improve this answer

edited Mar 27 at 16:48

answered Mar 27 at 16:37

VladVlad

3,87655 gold badges1414 silver badges3131 bronze badges

import tensorflow as tf

x = tf.Variable(5.0, dtype=tf.float32)
y = tf.Variable(6.0, dtype=tf.float32)
z = tf.Variable(8.0, dtype=tf.float32)

cond = tf.cond(pred = x < y,
 true_fn = lambda: tf.add(x, z),
 false_fn = lambda: tf.square(y))

op = tf.gradients(cond, z) 
# Returns [<tf.Tensor 'gradients_1/cond_1/Add/Switch_1_grad/cond_grad:0' shape=() dtype=float32>]

with tf.Session() as sess:
 sess.run(tf.global_variables_initializer())
 print(sess.run(op)) # [1.0] 

share|improve this answer

edited Mar 27 at 16:48

answered Mar 27 at 16:37

VladVlad

3,87655 gold badges1414 silver badges3131 bronze badges

answered Mar 27 at 16:37

VladVlad

3,87655 gold badges1414 silver badges3131 bronze badges

answered Mar 27 at 16:37

VladVlad

3,87655 gold badges1414 silver badges3131 bronze badges