Keras share weights between custom layers The Next CEO of Stack OverflowWhat is the difference between @staticmethod and @classmethod?Difference between append vs. extend list methods in PythonDifference between __str__ and __repr__?Keras Maxpooling2d layer gives ValueErrorHow to access weight variables in Keras layers in tensor form for clip_by_weight?What is the advantage of using an InputLayer (or an Input) in a Keras model with Tensorflow tensors?Tensorflow compute_output_shape() Not Working For Custom LayerInput tensors to a Model must come from `tf.layers.Input` when I concatenate two models with Keras API on TensorflowTFE_Py_RecordGradient error using Keras with Tensorflow back endIs there a method to find the min, max of a Tensorflow/Keras layer input during training?
How should I support this large drywall patch?
Return the Closest Prime Number
How to write papers efficiently when English isn't my first language?
When airplanes disconnect from a tanker during air to air refueling, why do they bank so sharply to the right?
How to use tikz in fbox?
Opposite of a diet
What size rim is OK?
Solution of this Diophantine Equation
Would this house-rule that treats advantage as a +1 to the roll instead (and disadvantage as -1) and allows them to stack be balanced?
Explicit solution of a Hamiltonian system
Is it a good idea to use COLUMN AS (left([Another_Column],(4)) instead of LEFT in the select?
Whats the best way to handle refactoring a big file?
Trouble understanding the speech of overseas colleagues
Was a professor correct to chastise me for writing "Prof. X" rather than "Professor X"?
What is the point of a new vote on May's deal when the indicative votes suggest she will not win?
What does this shorthand mean?
Customer Requests (Sometimes) Drive Me Bonkers!
What does "Its cash flow is deeply negative" mean?
MAZDA 3 2006 (UK) - poor acceleration then takes off at 3250 revs
Does it take more energy to get to Venus or to Mars?
How to write the block matrix in LaTex?
How to be diplomatic in refusing to write code that breaches the privacy of our users
Anatomically Correct Strange Women In Ponds Distributing Swords
Inappropriate reference requests from Journal reviewers
Keras share weights between custom layers
The Next CEO of Stack OverflowWhat is the difference between @staticmethod and @classmethod?Difference between append vs. extend list methods in PythonDifference between __str__ and __repr__?Keras Maxpooling2d layer gives ValueErrorHow to access weight variables in Keras layers in tensor form for clip_by_weight?What is the advantage of using an InputLayer (or an Input) in a Keras model with Tensorflow tensors?Tensorflow compute_output_shape() Not Working For Custom LayerInput tensors to a Model must come from `tf.layers.Input` when I concatenate two models with Keras API on TensorflowTFE_Py_RecordGradient error using Keras with Tensorflow back endIs there a method to find the min, max of a Tensorflow/Keras layer input during training?
I am working with the keras-capsnet implementation of Capsule Networks, and am trying to apply the same layer to 30 images per sample.
The weights are initialized within the init and build arguments for the class, shown below. I have successfully shared the weights between the primary routing layers which just use tf.layers.conv2d, where I can assign them the same name and set reuse = True.
Does anyone know how to initialize weights in a Keras custom layer so that they may be reused? I am much more familiar with the tensorflow API than with the Keras one!
def __init__(self, num_capsule, dim_capsule, routings=3,
kernel_initializer='glorot_uniform',
**kwargs):
super(CapsuleLayer, self).__init__(**kwargs)
self.num_capsule = num_capsule
self.dim_capsule = dim_capsule
self.routings = routings
self.kernel_initializer = initializers.get(kernel_initializer)
def build(self, input_shape):
assert len(input_shape) >= 3, "The input Tensor should have shape=[None, input_num_capsule, input_dim_capsule]"
self.input_num_capsule = input_shape[1]
self.input_dim_capsule = input_shape[2]
# Weights are initialized here each time the layer is called
self.W = self.add_weight(shape=[self.num_capsule, self.input_num_capsule,
self.dim_capsule, self.input_dim_capsule],
initializer=self.kernel_initializer,
name='W')
self.built = True
python tensorflow keras deep-learning
asked Mar 21 at 16:31
John BrandtJohn Brandt
12
add a comment |
I am working with the keras-capsnet implementation of Capsule Networks, and am trying to apply the same layer to 30 images per sample.
The weights are initialized within the init and build arguments for the class, shown below. I have successfully shared the weights between the primary routing layers which just use tf.layers.conv2d, where I can assign them the same name and set reuse = True.
Does anyone know how to initialize weights in a Keras custom layer so that they may be reused? I am much more familiar with the tensorflow API than with the Keras one!
def __init__(self, num_capsule, dim_capsule, routings=3,
kernel_initializer='glorot_uniform',
**kwargs):
super(CapsuleLayer, self).__init__(**kwargs)
self.num_capsule = num_capsule
self.dim_capsule = dim_capsule
self.routings = routings
self.kernel_initializer = initializers.get(kernel_initializer)
def build(self, input_shape):
assert len(input_shape) >= 3, "The input Tensor should have shape=[None, input_num_capsule, input_dim_capsule]"
self.input_num_capsule = input_shape[1]
self.input_dim_capsule = input_shape[2]
# Weights are initialized here each time the layer is called
self.W = self.add_weight(shape=[self.num_capsule, self.input_num_capsule,
self.dim_capsule, self.input_dim_capsule],
initializer=self.kernel_initializer,
name='W')
self.built = True
python tensorflow keras deep-learning
asked Mar 21 at 16:31
John BrandtJohn Brandt
12
How would you do it in tensorflow?
– Sharky
Mar 21 at 16:45
Well the layer is a Keras custom layer, so I don't know how to do it in tensorflow. I am used to either manually creating a weight matrix and just using it within a layer (not having to use self.add_weight argument), or using the same name scope and passing "reuse = tf.AUTO_REUSE" - the Keras docs say nothing about layer sharing in custom layers, unfortunately (tensorflow.org/api_docs/python/tf/keras/layers/Layer)
– John Brandt
Mar 21 at 16:55
The documentation says that Keras should share weights by calling the same layer multiple times on different inputs. Like layer = Dense(2), layer1 = layer(input), layer2 = layer(input2). I tried that in this case and it says that the tensor is not callable, since the layer returns the tensor.
– John Brandt
Mar 21 at 17:07
Can you show us the code you used to do what you described? This is exactly the way to go in Keras, perhaps there is an issue in the way you set up your layers.
– IonicSolutions
Mar 21 at 17:55
add a comment |
I am working with the keras-capsnet implementation of Capsule Networks, and am trying to apply the same layer to 30 images per sample.
The weights are initialized within the init and build arguments for the class, shown below. I have successfully shared the weights between the primary routing layers which just use tf.layers.conv2d, where I can assign them the same name and set reuse = True.
Does anyone know how to initialize weights in a Keras custom layer so that they may be reused? I am much more familiar with the tensorflow API than with the Keras one!
def __init__(self, num_capsule, dim_capsule, routings=3,
kernel_initializer='glorot_uniform',
**kwargs):
super(CapsuleLayer, self).__init__(**kwargs)
self.num_capsule = num_capsule
self.dim_capsule = dim_capsule
self.routings = routings
self.kernel_initializer = initializers.get(kernel_initializer)
def build(self, input_shape):
assert len(input_shape) >= 3, "The input Tensor should have shape=[None, input_num_capsule, input_dim_capsule]"
self.input_num_capsule = input_shape[1]
self.input_dim_capsule = input_shape[2]
# Weights are initialized here each time the layer is called
self.W = self.add_weight(shape=[self.num_capsule, self.input_num_capsule,
self.dim_capsule, self.input_dim_capsule],
initializer=self.kernel_initializer,
name='W')
self.built = True
python tensorflow keras deep-learning
asked Mar 21 at 16:31
John BrandtJohn Brandt
12
I am working with the keras-capsnet implementation of Capsule Networks, and am trying to apply the same layer to 30 images per sample.
The weights are initialized within the init and build arguments for the class, shown below. I have successfully shared the weights between the primary routing layers which just use tf.layers.conv2d, where I can assign them the same name and set reuse = True.
Does anyone know how to initialize weights in a Keras custom layer so that they may be reused? I am much more familiar with the tensorflow API than with the Keras one!
def __init__(self, num_capsule, dim_capsule, routings=3,
kernel_initializer='glorot_uniform',
**kwargs):
super(CapsuleLayer, self).__init__(**kwargs)
self.num_capsule = num_capsule
self.dim_capsule = dim_capsule
self.routings = routings
self.kernel_initializer = initializers.get(kernel_initializer)
def build(self, input_shape):
assert len(input_shape) >= 3, "The input Tensor should have shape=[None, input_num_capsule, input_dim_capsule]"
self.input_num_capsule = input_shape[1]
self.input_dim_capsule = input_shape[2]
# Weights are initialized here each time the layer is called
self.W = self.add_weight(shape=[self.num_capsule, self.input_num_capsule,
self.dim_capsule, self.input_dim_capsule],
initializer=self.kernel_initializer,
name='W')
self.built = True
python tensorflow keras deep-learning
python tensorflow keras deep-learning
asked Mar 21 at 16:31
John BrandtJohn Brandt
12
asked Mar 21 at 16:31
John BrandtJohn Brandt
12
asked Mar 21 at 16:31
John BrandtJohn Brandt
12
asked Mar 21 at 16:31
John BrandtJohn Brandt
12
asked Mar 21 at 16:31
John BrandtJohn Brandt
12
12
How would you do it in tensorflow?
– Sharky
Mar 21 at 16:45
Well the layer is a Keras custom layer, so I don't know how to do it in tensorflow. I am used to either manually creating a weight matrix and just using it within a layer (not having to use self.add_weight argument), or using the same name scope and passing "reuse = tf.AUTO_REUSE" - the Keras docs say nothing about layer sharing in custom layers, unfortunately (tensorflow.org/api_docs/python/tf/keras/layers/Layer)
– John Brandt
Mar 21 at 16:55
The documentation says that Keras should share weights by calling the same layer multiple times on different inputs. Like layer = Dense(2), layer1 = layer(input), layer2 = layer(input2). I tried that in this case and it says that the tensor is not callable, since the layer returns the tensor.
– John Brandt
Mar 21 at 17:07
Can you show us the code you used to do what you described? This is exactly the way to go in Keras, perhaps there is an issue in the way you set up your layers.
– IonicSolutions
Mar 21 at 17:55
add a comment |
How would you do it in tensorflow?
– Sharky
Mar 21 at 16:45
Well the layer is a Keras custom layer, so I don't know how to do it in tensorflow. I am used to either manually creating a weight matrix and just using it within a layer (not having to use self.add_weight argument), or using the same name scope and passing "reuse = tf.AUTO_REUSE" - the Keras docs say nothing about layer sharing in custom layers, unfortunately (tensorflow.org/api_docs/python/tf/keras/layers/Layer)
– John Brandt
Mar 21 at 16:55
The documentation says that Keras should share weights by calling the same layer multiple times on different inputs. Like layer = Dense(2), layer1 = layer(input), layer2 = layer(input2). I tried that in this case and it says that the tensor is not callable, since the layer returns the tensor.
– John Brandt
Mar 21 at 17:07
Can you show us the code you used to do what you described? This is exactly the way to go in Keras, perhaps there is an issue in the way you set up your layers.
– IonicSolutions
Mar 21 at 17:55
How would you do it in tensorflow?
– Sharky
Mar 21 at 16:45
How would you do it in tensorflow?
– Sharky
Mar 21 at 16:45
Well the layer is a Keras custom layer, so I don't know how to do it in tensorflow. I am used to either manually creating a weight matrix and just using it within a layer (not having to use self.add_weight argument), or using the same name scope and passing "reuse = tf.AUTO_REUSE" - the Keras docs say nothing about layer sharing in custom layers, unfortunately (tensorflow.org/api_docs/python/tf/keras/layers/Layer)
– John Brandt
Mar 21 at 16:55
Well the layer is a Keras custom layer, so I don't know how to do it in tensorflow. I am used to either manually creating a weight matrix and just using it within a layer (not having to use self.add_weight argument), or using the same name scope and passing "reuse = tf.AUTO_REUSE" - the Keras docs say nothing about layer sharing in custom layers, unfortunately (tensorflow.org/api_docs/python/tf/keras/layers/Layer)
– John Brandt
Mar 21 at 16:55
The documentation says that Keras should share weights by calling the same layer multiple times on different inputs. Like layer = Dense(2), layer1 = layer(input), layer2 = layer(input2). I tried that in this case and it says that the tensor is not callable, since the layer returns the tensor.
– John Brandt
Mar 21 at 17:07
The documentation says that Keras should share weights by calling the same layer multiple times on different inputs. Like layer = Dense(2), layer1 = layer(input), layer2 = layer(input2). I tried that in this case and it says that the tensor is not callable, since the layer returns the tensor.
– John Brandt
Mar 21 at 17:07
Can you show us the code you used to do what you described? This is exactly the way to go in Keras, perhaps there is an issue in the way you set up your layers.
– IonicSolutions
Mar 21 at 17:55
Can you show us the code you used to do what you described? This is exactly the way to go in Keras, perhaps there is an issue in the way you set up your layers.
– IonicSolutions
Mar 21 at 17:55
add a comment |
1 Answer
1
active
oldest
votes
The answer was simple. Set up a layer without calling it on input, and then use that built layer to call the data individually.
answered Mar 21 at 19:47
John BrandtJohn Brandt
12
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55285139%2fkeras-share-weights-between-custom-layers%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
The answer was simple. Set up a layer without calling it on input, and then use that built layer to call the data individually.
answered Mar 21 at 19:47
John BrandtJohn Brandt
12
add a comment |
The answer was simple. Set up a layer without calling it on input, and then use that built layer to call the data individually.
answered Mar 21 at 19:47
John BrandtJohn Brandt
12
add a comment |
The answer was simple. Set up a layer without calling it on input, and then use that built layer to call the data individually.
answered Mar 21 at 19:47
John BrandtJohn Brandt
12
The answer was simple. Set up a layer without calling it on input, and then use that built layer to call the data individually.
answered Mar 21 at 19:47
John BrandtJohn Brandt
12
answered Mar 21 at 19:47
John BrandtJohn Brandt
12
answered Mar 21 at 19:47
John BrandtJohn Brandt
12
answered Mar 21 at 19:47
John BrandtJohn Brandt
12
12
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55285139%2fkeras-share-weights-between-custom-layers%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
How would you do it in tensorflow?
– Sharky
Mar 21 at 16:45
Well the layer is a Keras custom layer, so I don't know how to do it in tensorflow. I am used to either manually creating a weight matrix and just using it within a layer (not having to use self.add_weight argument), or using the same name scope and passing "reuse = tf.AUTO_REUSE" - the Keras docs say nothing about layer sharing in custom layers, unfortunately (tensorflow.org/api_docs/python/tf/keras/layers/Layer)
– John Brandt
Mar 21 at 16:55
The documentation says that Keras should share weights by calling the same layer multiple times on different inputs. Like layer = Dense(2), layer1 = layer(input), layer2 = layer(input2). I tried that in this case and it says that the tensor is not callable, since the layer returns the tensor.
– John Brandt
Mar 21 at 17:07
Can you show us the code you used to do what you described? This is exactly the way to go in Keras, perhaps there is an issue in the way you set up your layers.
– IonicSolutions
Mar 21 at 17:55