Styjun

Why would a fighter use the afterburner and air brakes at the same time?

Why does Canada require a minimum rate of climb for ultralights of 300 ft/min?

What exactly is a web font, and what does converting to one involve?

My passport was stamped with an exit stamp while transiting to another Schengen country via Turkey. Was this a mistake?

EU compensation - fire alarm at the Flight Crew's hotel

Manager manipulates my leaves, what's in it for him?

How can I check that parent has more than 1 child?

What is the word for a person who destroys monuments?

Which block cipher parameters should be kept secret?

What the did the controller say during my approach to land (audio clip)?

Plausibility and performance of a composite longbow

Is Zack Morris's 'time stop' ability in "Saved By the Bell" a supernatural ability?

What was the deeper meaning of Hermione wanting the cloak?

Amiga 500 OCS/ECS vs Mega Drive VDP

Why can't we use uninitialized local variable to access static content of its type?

Minimum number of lines to draw 111 squares

Applications of mathematics in clinical setting

Should the pagination be reset when changing the order?

Inquiry answerer

Account creation and log-in system

With a 500GB SSD and a 250GB SSD is it possible to mirror a 250GB partition on the 500GB with the 250GB SSD using ZFS?

Exam design: give maximum score per question or not?

Can one guy with a duplicator trigger a nuclear apocalypse?

Does rpcpassword need to be non-obvious in bitcoind?

How to compute weighted average of tensor A along an axis with weights specified by tensor B in tensorflow?

Numpy averaging with multi-dimensional weights along an axisTensorFlow: how to batch mut-mul a batch tensor by a weight variable?In tensorflow, how to iterate over a sequence of inputs stored in a tensor?Sample from a tensor in Tensorflow along an axisWhat is the difference between tensors and sparse tensors?how to average a tensor axis with specified mask in tensorflowN-D tensor matrix multiplication with tensorflowslicing a tensor along a dimension with given indexTensor Operation in Tensorflow

.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;

1

I am trying to apply a weighted average scheme on RNN output.

RNN output is represented by tensor A having dimension (a,b,c).

I can simply take tf.reduce_mean(A,axis=1) to get the tensor C having dimension (a,c).

However, I want to do the "weighted average" of tensor A along axis = 1.

Weights are specified in the matrix B having dimension (d,b).

For d = 1, I can do tf.tensordot(A,B,[1,1]) to get the result of dimension (a,c).

Now for d=a, I am unable to compute the weighted average.

Can someone suggest a solution?

python tensorflow recurrent-neural-network weighted-average

share|improve this question

edited Mar 28 at 13:53

abhi

asked Mar 28 at 13:46

abhiabhi

9877 bronze badges

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Try tf.reduce_sum(A * B[:, :, None], axis=1) / tf.reduce_sum(B, axis=1, keepdims=True). If it works, I'll turn it into an answer.
  
  – Siyuan Ren
  Mar 28 at 15:30
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  @SiyuanRen can you please explain what it does?
  
  – abhi
  Mar 29 at 8:29
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  @SiyuanRen weights are already normalized. So no need for division operation. I think division operation is normalizing weights so that it sums to 1. Can you explain the evaluation of the first part?
  
  – abhi
  Mar 29 at 9:21
  

  
  
  
  

add a comment
 | 

1

I am trying to apply a weighted average scheme on RNN output.

RNN output is represented by tensor A having dimension (a,b,c).

I can simply take tf.reduce_mean(A,axis=1) to get the tensor C having dimension (a,c).

However, I want to do the "weighted average" of tensor A along axis = 1.

Weights are specified in the matrix B having dimension (d,b).

For d = 1, I can do tf.tensordot(A,B,[1,1]) to get the result of dimension (a,c).

Now for d=a, I am unable to compute the weighted average.

Can someone suggest a solution?

python tensorflow recurrent-neural-network weighted-average

share|improve this question

edited Mar 28 at 13:53

abhi

asked Mar 28 at 13:46

abhiabhi

9877 bronze badges

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Try tf.reduce_sum(A * B[:, :, None], axis=1) / tf.reduce_sum(B, axis=1, keepdims=True). If it works, I'll turn it into an answer.
  
  – Siyuan Ren
  Mar 28 at 15:30
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  @SiyuanRen can you please explain what it does?
  
  – abhi
  Mar 29 at 8:29
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  @SiyuanRen weights are already normalized. So no need for division operation. I think division operation is normalizing weights so that it sums to 1. Can you explain the evaluation of the first part?
  
  – abhi
  Mar 29 at 9:21
  

  
  
  
  

add a comment
 | 

I am trying to apply a weighted average scheme on RNN output.

RNN output is represented by tensor A having dimension (a,b,c).

I can simply take tf.reduce_mean(A,axis=1) to get the tensor C having dimension (a,c).

However, I want to do the "weighted average" of tensor A along axis = 1.

Weights are specified in the matrix B having dimension (d,b).

For d = 1, I can do tf.tensordot(A,B,[1,1]) to get the result of dimension (a,c).

Now for d=a, I am unable to compute the weighted average.

Can someone suggest a solution?

python tensorflow recurrent-neural-network weighted-average

share|improve this question

edited Mar 28 at 13:53

abhi

asked Mar 28 at 13:46

abhiabhi

9877 bronze badges

share|improve this question

edited Mar 28 at 13:53

abhi

asked Mar 28 at 13:46

abhiabhi

9877 bronze badges

share|improve this question

edited Mar 28 at 13:53

abhi

asked Mar 28 at 13:46

abhiabhi

9877 bronze badges

asked Mar 28 at 13:46

abhiabhi

9877 bronze badges

asked Mar 28 at 13:46

abhiabhi

9877 bronze badges

2 Answers
2

active

oldest

votes

1

I don't quite get why B should have dimensions (d,b). If B contains the weights to do a weighted average of A across only one dimension, B only has to be a vector (b,), not a matrix.

If B is a vector, you can do:

C = tf.tensordot(A,B,[1,0]) to get a vector C of shape (a,c) which contains the weighted average of A across axis=1 using the weights specified in B.

Update:

You can do something like:

A = A*B[:,:,None] 

which is doing element wise multiplication of A and B, where B stores the weights given to each element in A.
Then:

C = tf.reduce_mean(A,axis=1)

will do the weighted average since each element in A has been multiplied by its weight.

share|improve this answer

edited Mar 30 at 8:37

answered Mar 28 at 15:24

Guillem CucurullGuillem Cucurull

1,36111 gold badge1717 silver badges2727 bronze badges

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  I have obtained the results for the d=1 case. I need d=a case because I have weights for each input and batch size is specified by a and d.
  
  – abhi
  Mar 29 at 8:12
  
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  
  Ok, sorry, then you can do something like: A = A*B[:,:,None] (this is doing element wise multiplication of A and B, where B stores the weights given to each element in A) and then C = tf.reduce_mean(A,axis=1), which will do the weighted mean since each element in A has been multiplied by its weight.
  
  – Guillem Cucurull
  Mar 30 at 8:35
  

  
  
  
  

add a comment
 | 

1

Since B is already normalized, the answer is

tf.reduce_sum(A * B[:, :, None], axis=1)

Indexing with None adds a new dimension, a behavior inherited from numpy.B[:,:, None] adds a last dimension so the result has shape (a, b, 1). You can achieve the same thing with tf.expand_dims, whose name may make more sense to you.

A has shape (a, b, c) while B[:, :, None] has shape (a, b, 1). When they are multiplied, expanded B will be treated as having shape (a, b, c) too, with the last dimension being c copies of the same value. This is called broadcasting.

Because of how broadcasting works, the same answer also works if B has shape (1, b).

share|improve this answer

answered Mar 30 at 2:05

Siyuan RenSiyuan Ren

4,73022 gold badges3434 silver badges4646 bronze badges

add a comment
 | 

Your Answer

StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/4.0/"u003ecc by-sa 4.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55399197%2fhow-to-compute-weighted-average-of-tensor-a-along-an-axis-with-weights-specified%23new-answer', 'question_page');

);

Post as a guest

Name

Email

Required, but never shown

1

I don't quite get why B should have dimensions (d,b). If B contains the weights to do a weighted average of A across only one dimension, B only has to be a vector (b,), not a matrix.

If B is a vector, you can do:

C = tf.tensordot(A,B,[1,0]) to get a vector C of shape (a,c) which contains the weighted average of A across axis=1 using the weights specified in B.

Update:

You can do something like:

A = A*B[:,:,None] 

which is doing element wise multiplication of A and B, where B stores the weights given to each element in A.
Then:

C = tf.reduce_mean(A,axis=1)

will do the weighted average since each element in A has been multiplied by its weight.

share|improve this answer

edited Mar 30 at 8:37

answered Mar 28 at 15:24

Guillem CucurullGuillem Cucurull

1,36111 gold badge1717 silver badges2727 bronze badges

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  I have obtained the results for the d=1 case. I need d=a case because I have weights for each input and batch size is specified by a and d.
  
  – abhi
  Mar 29 at 8:12
  
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  
  Ok, sorry, then you can do something like: A = A*B[:,:,None] (this is doing element wise multiplication of A and B, where B stores the weights given to each element in A) and then C = tf.reduce_mean(A,axis=1), which will do the weighted mean since each element in A has been multiplied by its weight.
  
  – Guillem Cucurull
  Mar 30 at 8:35
  

  
  
  
  

add a comment
 | 

1

I don't quite get why B should have dimensions (d,b). If B contains the weights to do a weighted average of A across only one dimension, B only has to be a vector (b,), not a matrix.

If B is a vector, you can do:

C = tf.tensordot(A,B,[1,0]) to get a vector C of shape (a,c) which contains the weighted average of A across axis=1 using the weights specified in B.

Update:

You can do something like:

A = A*B[:,:,None] 

which is doing element wise multiplication of A and B, where B stores the weights given to each element in A.
Then:

C = tf.reduce_mean(A,axis=1)

will do the weighted average since each element in A has been multiplied by its weight.

share|improve this answer

edited Mar 30 at 8:37

answered Mar 28 at 15:24

Guillem CucurullGuillem Cucurull

1,36111 gold badge1717 silver badges2727 bronze badges

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  I have obtained the results for the d=1 case. I need d=a case because I have weights for each input and batch size is specified by a and d.
  
  – abhi
  Mar 29 at 8:12
  
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  
  Ok, sorry, then you can do something like: A = A*B[:,:,None] (this is doing element wise multiplication of A and B, where B stores the weights given to each element in A) and then C = tf.reduce_mean(A,axis=1), which will do the weighted mean since each element in A has been multiplied by its weight.
  
  – Guillem Cucurull
  Mar 30 at 8:35
  

  
  
  
  

add a comment
 | 

I don't quite get why B should have dimensions (d,b). If B contains the weights to do a weighted average of A across only one dimension, B only has to be a vector (b,), not a matrix.

If B is a vector, you can do:

C = tf.tensordot(A,B,[1,0]) to get a vector C of shape (a,c) which contains the weighted average of A across axis=1 using the weights specified in B.

Update:

You can do something like:

A = A*B[:,:,None] 

which is doing element wise multiplication of A and B, where B stores the weights given to each element in A.
Then:

C = tf.reduce_mean(A,axis=1)

will do the weighted average since each element in A has been multiplied by its weight.

share|improve this answer

edited Mar 30 at 8:37

answered Mar 28 at 15:24

Guillem CucurullGuillem Cucurull

1,36111 gold badge1717 silver badges2727 bronze badges

A = A*B[:,:,None] 

C = tf.reduce_mean(A,axis=1)

share|improve this answer

edited Mar 30 at 8:37

answered Mar 28 at 15:24

Guillem CucurullGuillem Cucurull

1,36111 gold badge1717 silver badges2727 bronze badges

answered Mar 28 at 15:24

Guillem CucurullGuillem Cucurull

1,36111 gold badge1717 silver badges2727 bronze badges

answered Mar 28 at 15:24

Guillem CucurullGuillem Cucurull

1,36111 gold badge1717 silver badges2727 bronze badges

1

Since B is already normalized, the answer is

tf.reduce_sum(A * B[:, :, None], axis=1)

Indexing with None adds a new dimension, a behavior inherited from numpy.B[:,:, None] adds a last dimension so the result has shape (a, b, 1). You can achieve the same thing with tf.expand_dims, whose name may make more sense to you.

A has shape (a, b, c) while B[:, :, None] has shape (a, b, 1). When they are multiplied, expanded B will be treated as having shape (a, b, c) too, with the last dimension being c copies of the same value. This is called broadcasting.

Because of how broadcasting works, the same answer also works if B has shape (1, b).

share|improve this answer

answered Mar 30 at 2:05

Siyuan RenSiyuan Ren

4,73022 gold badges3434 silver badges4646 bronze badges

add a comment
 | 

1

Since B is already normalized, the answer is

tf.reduce_sum(A * B[:, :, None], axis=1)

Indexing with None adds a new dimension, a behavior inherited from numpy.B[:,:, None] adds a last dimension so the result has shape (a, b, 1). You can achieve the same thing with tf.expand_dims, whose name may make more sense to you.

A has shape (a, b, c) while B[:, :, None] has shape (a, b, 1). When they are multiplied, expanded B will be treated as having shape (a, b, c) too, with the last dimension being c copies of the same value. This is called broadcasting.

Because of how broadcasting works, the same answer also works if B has shape (1, b).

share|improve this answer

answered Mar 30 at 2:05

Siyuan RenSiyuan Ren

4,73022 gold badges3434 silver badges4646 bronze badges

add a comment
 | 

Since B is already normalized, the answer is

tf.reduce_sum(A * B[:, :, None], axis=1)

Indexing with None adds a new dimension, a behavior inherited from numpy.B[:,:, None] adds a last dimension so the result has shape (a, b, 1). You can achieve the same thing with tf.expand_dims, whose name may make more sense to you.

A has shape (a, b, c) while B[:, :, None] has shape (a, b, 1). When they are multiplied, expanded B will be treated as having shape (a, b, c) too, with the last dimension being c copies of the same value. This is called broadcasting.

Because of how broadcasting works, the same answer also works if B has shape (1, b).

share|improve this answer

answered Mar 30 at 2:05

Siyuan RenSiyuan Ren

4,73022 gold badges3434 silver badges4646 bronze badges

tf.reduce_sum(A * B[:, :, None], axis=1)

share|improve this answer

answered Mar 30 at 2:05

Siyuan RenSiyuan Ren

4,73022 gold badges3434 silver badges4646 bronze badges

answered Mar 30 at 2:05

Siyuan RenSiyuan Ren

4,73022 gold badges3434 silver badges4646 bronze badges

answered Mar 30 at 2:05

Siyuan RenSiyuan Ren

4,73022 gold badges3434 silver badges4646 bronze badges