Styjun

I'm trying to create a baseline model, for an NER task, using a Bi-directional LSTM with the functional API provided by Keras

The embedding layer I've used is a 100-dimensional feature vector

Input to the layer is a padded sequence of length

MAX_LEN = 575

(Note : The input and output are of the same dimensions)

I want an output at each time-step therefore I've set

return_sequences = True

The output is just the activations passed through a soft-max layer

But while compiling the Model I keep getting this warning

UserWarning: Model inputs must come from `keras.layers.Input`
(thus holding past layer metadata), they cannot be the output of a 
previous non-Input layer. Here, a tensor specified as input to your model was
not an Input tensor, it was generated by layer embedding_3.
Note that input tensors are instantiated via `tensor = keras.layers.Input(shape)`.
The tensor that caused the issue was: embedding_3_40/embedding_lookup/Identity:0 str(x.name))

Accompanied by an

AssertionError:

Traceback:

---> 37 model = Model(inputs = nn_input, outputs = nn_output)
---> 91 return func(*args, **kwargs)
---> 93 self._init_graph_network(*args, **kwargs)
 222 # It's supposed to be an input layer, so only one node
 223 # and one tensor output.
--> 224 assert node_index == 0

I tried debugging the code to check the dimensions but they seem to match as highlighted by the comments in the code

nn_input = Input(shape = (MAX_LEN,) , dtype = 'int32')

print(nn_input.shape) #(?, 575)

nn_input = embedding_layer(nn_input)

print(nn_input.shape) #(?, 575, 100)

nn_out, forward_h, forward_c, backward_h, backward_c = Bidirectional(LSTM(MAX_LEN, return_sequences = True, return_state = True))(nn_input)

print(forward_h.shape) #(?, 575)
print(forward_c.shape) #(?, 575)
print(backward_h.shape) #(?, 575)
print(backward_c.shape) #(?, 575)

print(nn_out.shape) #(?, ?, 1150)

state_h = Concatenate()([forward_h, backward_h])
state_c = Concatenate()([forward_c, backward_c])

print(state_h.shape) #(?, 1150)
print(state_c.shape) #(?, 1150)

densor = Dense(100, activation='softmax')
nn_output = densor(nn_out)

print(nn_output.shape) #(?, 575, 100)

model = Model(inputs = nn_input, outputs = nn_output)

This might be seem trivial to some but I fear there's a flaw in my understanding of LSTMs or atleast Keras for that matter

I'll provide additional details in the edits if necessary

Any help would be highly appreciated!

As the error indicates, you have to pass tensor that is the output of layer keras.layers.Input to Model API. In this case, the tensor nn_input is the output of embedding_layer. Change the variable name used to assign the output of embedding_layer from nn_input to something else.

nn_input = Input(shape = (MAX_LEN,) , dtype = 'int32')
# the line below is the cause of the error. Change the output variable name to like nn_embed. 
nn_input = embedding_layer(nn_input)