Styjun

Why would non-kinetic weapons be used for orbital bombardment?

LED glows slightly during soldering

Does Lufthansa weigh your carry on luggage?

When I press the space bar it deletes the letters after it

For a hashing function like MD5, how similar can two plaintext strings be and still generate the same hash?

Extracting points from 3D plot that lie along an arbitrarily oriented line

Is there any reason why MCU changed the Snap to Blip

What is a "shilicashe?"

Fast validation of time windows in a routing problem

Great Unsolved Problems in O.R

Which star / galaxy is moving away from us the fastest?

How do native German speakers usually express skepticism (using even) about a premise?

Salt, pepper, herbs and spices

Are there any balance issues in allowing two half-feats to be taken without the Ability Score Increase instead of a feat?

How can I truly shut down ssh server?

Is "I do not want you to go nowhere" a case of "DOUBLE-NEGATIVES" as claimed by Grammarly?

OR-backed serious games

How are mathematicians paid to do research?

Would a non-attacking Barbarian's rage end the same turn he started it?

Shortest hex dumping program

Are there any medieval light sources without fire?

Addressing unnecessary daily meetings with manager?

Why does this potentiometer in an op-amp feedback path cause noise when adjusted?

Why are they 'nude photos'?

weight matrix dimension intuition in a neural network

Role of Bias in Neural Networksuse matlab neural function weight and bias matrix valueHow to determine the weights in a neural network?Multiplying weight matrix of a trained neural network with a test vectorregard Neural Network input formatMATLAB neural network weight and bias initializaitonHow to apply weights in neural network?How to train a neural network with inputs of different dimensions in Matlab?Neural network: weights and biases convergenceStandard parameter representation in neural networks

.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;

0

I have been following a course about neural networks in Coursera and came across this model:

I understand that the values of z1, z2 and so on are the values from the linear regression that will be put into an activation function. The problem that I have is when the author says that there should be one matrix of weights and a vector of the inputs, like this:

I know that the vector of Xs has a dimension of 3 x 1 because there are three inputs, but why the array of Ws is of dimensions 4 x 3?. I can deduct that it has four rows because those are the weights w1, w2, w3 and w4 that they correspond to each ones of the values of a1...a4, but what is inside that array? Its elements are something like:

w1T w1T w1T
w2T w2T w3T
... ?

so when I multiply by x1, for example, I will get:

w1Tx1+w1Tx2+w1Tx3=w1T(x1+x2+x3)=w1TX

I have think about it, but I cannot really get a grasp about what this array contains, even though I know that at the end I will have a vector of 4 x 1 that corresponds to the values of z. Any help?

Thanks

neural-network

share|improve this question

edited Mar 26 at 2:15

Little

asked Mar 26 at 1:33

LittleLittle

1,02866 gold badges2525 silver badges4545 bronze badges

add a comment | 

0

I have been following a course about neural networks in Coursera and came across this model:

I understand that the values of z1, z2 and so on are the values from the linear regression that will be put into an activation function. The problem that I have is when the author says that there should be one matrix of weights and a vector of the inputs, like this:

I know that the vector of Xs has a dimension of 3 x 1 because there are three inputs, but why the array of Ws is of dimensions 4 x 3?. I can deduct that it has four rows because those are the weights w1, w2, w3 and w4 that they correspond to each ones of the values of a1...a4, but what is inside that array? Its elements are something like:

w1T w1T w1T
w2T w2T w3T
... ?

so when I multiply by x1, for example, I will get:

w1Tx1+w1Tx2+w1Tx3=w1T(x1+x2+x3)=w1TX

I have think about it, but I cannot really get a grasp about what this array contains, even though I know that at the end I will have a vector of 4 x 1 that corresponds to the values of z. Any help?

Thanks

neural-network

share|improve this question

edited Mar 26 at 2:15

Little

asked Mar 26 at 1:33

LittleLittle

1,02866 gold badges2525 silver badges4545 bronze badges

add a comment | 

I have been following a course about neural networks in Coursera and came across this model:

I understand that the values of z1, z2 and so on are the values from the linear regression that will be put into an activation function. The problem that I have is when the author says that there should be one matrix of weights and a vector of the inputs, like this:

I know that the vector of Xs has a dimension of 3 x 1 because there are three inputs, but why the array of Ws is of dimensions 4 x 3?. I can deduct that it has four rows because those are the weights w1, w2, w3 and w4 that they correspond to each ones of the values of a1...a4, but what is inside that array? Its elements are something like:

w1T w1T w1T
w2T w2T w3T
... ?

so when I multiply by x1, for example, I will get:

w1Tx1+w1Tx2+w1Tx3=w1T(x1+x2+x3)=w1TX

I have think about it, but I cannot really get a grasp about what this array contains, even though I know that at the end I will have a vector of 4 x 1 that corresponds to the values of z. Any help?

Thanks

neural-network

share|improve this question

edited Mar 26 at 2:15

Little

asked Mar 26 at 1:33

LittleLittle

1,02866 gold badges2525 silver badges4545 bronze badges

w1T w1T w1T
w2T w2T w3T
... ?

w1Tx1+w1Tx2+w1Tx3=w1T(x1+x2+x3)=w1TX

share|improve this question

edited Mar 26 at 2:15

Little

asked Mar 26 at 1:33

LittleLittle

1,02866 gold badges2525 silver badges4545 bronze badges

share|improve this question

edited Mar 26 at 2:15

Little

asked Mar 26 at 1:33

LittleLittle

1,02866 gold badges2525 silver badges4545 bronze badges

asked Mar 26 at 1:33

LittleLittle

1,02866 gold badges2525 silver badges4545 bronze badges

asked Mar 26 at 1:33

LittleLittle

1,02866 gold badges2525 silver badges4545 bronze badges

1 Answer
1

active

oldest

votes

0

If x is 3x1, then a weight matrix of size Nx3 will give you a hidden layer with N units. In your case N = 4 (see the network schematic). This follows from the fact that multiplying a Nx3 matrix with a 3x1 vector gives a Nx1 vector as output, hence, N hidden units.

Each row of the weight matrix defines the weights for a single hidden unit, so the scalar product of w_1 and x (plus bias) gives z_1:

In the end, writing all quantities as vectors and matrices simply allows you to use succinct linear algebra notation:

where we assume that the activation is applied element-wise.

share|improve this answer

edited Mar 26 at 12:22

answered Mar 26 at 8:06

cheersmatecheersmate

1,14244 gold badges88 silver badges2020 bronze badges

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  thank you @cheersmate, but I know from where the las formula came from. Actually, I am curious about what information does the array W contains. For example, in the case that I wrote, is it contains w1 repeated three times in the first row, w2 repeated three times in the second raw and so on?
  
  – Little
  Mar 26 at 12:06
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Actually, you write individual weights with ^T in your question, but that's a transpose which only makes sense for vectors: this already gives you a hint about what's going on. The notation in the sketch means just that: w_1 is a vector with 3 (different) elements.
  
  – cheersmate
  Mar 26 at 12:20
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  I've also updated the formula to explicitly show the scalar product and the individual elements of w_i.
  
  – cheersmate
  Mar 26 at 12:23
  

  
  
  
  

add a comment | 

Your Answer

StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55348647%2fweight-matrix-dimension-intuition-in-a-neural-network%23new-answer', 'question_page');

);

Post as a guest

Name

Email

Required, but never shown

0

If x is 3x1, then a weight matrix of size Nx3 will give you a hidden layer with N units. In your case N = 4 (see the network schematic). This follows from the fact that multiplying a Nx3 matrix with a 3x1 vector gives a Nx1 vector as output, hence, N hidden units.

Each row of the weight matrix defines the weights for a single hidden unit, so the scalar product of w_1 and x (plus bias) gives z_1:

In the end, writing all quantities as vectors and matrices simply allows you to use succinct linear algebra notation:

where we assume that the activation is applied element-wise.

share|improve this answer

edited Mar 26 at 12:22

answered Mar 26 at 8:06

cheersmatecheersmate

1,14244 gold badges88 silver badges2020 bronze badges

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  thank you @cheersmate, but I know from where the las formula came from. Actually, I am curious about what information does the array W contains. For example, in the case that I wrote, is it contains w1 repeated three times in the first row, w2 repeated three times in the second raw and so on?
  
  – Little
  Mar 26 at 12:06
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Actually, you write individual weights with ^T in your question, but that's a transpose which only makes sense for vectors: this already gives you a hint about what's going on. The notation in the sketch means just that: w_1 is a vector with 3 (different) elements.
  
  – cheersmate
  Mar 26 at 12:20
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  I've also updated the formula to explicitly show the scalar product and the individual elements of w_i.
  
  – cheersmate
  Mar 26 at 12:23
  

  
  
  
  

add a comment | 

0

If x is 3x1, then a weight matrix of size Nx3 will give you a hidden layer with N units. In your case N = 4 (see the network schematic). This follows from the fact that multiplying a Nx3 matrix with a 3x1 vector gives a Nx1 vector as output, hence, N hidden units.

Each row of the weight matrix defines the weights for a single hidden unit, so the scalar product of w_1 and x (plus bias) gives z_1:

In the end, writing all quantities as vectors and matrices simply allows you to use succinct linear algebra notation:

where we assume that the activation is applied element-wise.

share|improve this answer

edited Mar 26 at 12:22

answered Mar 26 at 8:06

cheersmatecheersmate

1,14244 gold badges88 silver badges2020 bronze badges

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  thank you @cheersmate, but I know from where the las formula came from. Actually, I am curious about what information does the array W contains. For example, in the case that I wrote, is it contains w1 repeated three times in the first row, w2 repeated three times in the second raw and so on?
  
  – Little
  Mar 26 at 12:06
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Actually, you write individual weights with ^T in your question, but that's a transpose which only makes sense for vectors: this already gives you a hint about what's going on. The notation in the sketch means just that: w_1 is a vector with 3 (different) elements.
  
  – cheersmate
  Mar 26 at 12:20
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  I've also updated the formula to explicitly show the scalar product and the individual elements of w_i.
  
  – cheersmate
  Mar 26 at 12:23
  

  
  
  
  

add a comment | 

If x is 3x1, then a weight matrix of size Nx3 will give you a hidden layer with N units. In your case N = 4 (see the network schematic). This follows from the fact that multiplying a Nx3 matrix with a 3x1 vector gives a Nx1 vector as output, hence, N hidden units.

Each row of the weight matrix defines the weights for a single hidden unit, so the scalar product of w_1 and x (plus bias) gives z_1:

In the end, writing all quantities as vectors and matrices simply allows you to use succinct linear algebra notation:

where we assume that the activation is applied element-wise.

share|improve this answer

edited Mar 26 at 12:22

answered Mar 26 at 8:06

cheersmatecheersmate

1,14244 gold badges88 silver badges2020 bronze badges

share|improve this answer

edited Mar 26 at 12:22

answered Mar 26 at 8:06

cheersmatecheersmate

1,14244 gold badges88 silver badges2020 bronze badges

answered Mar 26 at 8:06

cheersmatecheersmate

1,14244 gold badges88 silver badges2020 bronze badges

answered Mar 26 at 8:06

cheersmatecheersmate

1,14244 gold badges88 silver badges2020 bronze badges