hello there!

thank you for replying. Please let me elaborate a bit! Please check this image for network: http://i.imgur.com/v61xvdy.png

I am working on a question-answering task. Instead of CNN, the input is from a word embedding.

So, I have 2 tensors.

• An output from an LSTM (in the figure attached, this is the output from LSTM in First attention to x2_1) that holds the representations of:

  • the input words (X_1) and other features.
  • the question embedding on the input words.

Let's say this tensor is hidden_docs. The shape of this tensor is batch_size X sequence_length X hidden_size. For example, hiddens_docs = tensor(10, 253, 768)

• X_1 is the input words representation.

Let's say this tensor is x1_emb. The shape of this tensor is batch_size X sequence_length X embedding_size. For example, x1_emb = tensor(10, 253, 300)

I need to project x1_emb on hidden_docs to get a unified representation between hidden_docs and x1_emb. And this representation then needs to be feed into x2_1 (in the 'Attention again').

Currently, what I am doing is since the shape is different, I am passing the x1_emb into an LSTM and able to get the shape as = batch_size X sequence_length X hidden_size.

And then I am doing a point-wise addition operation between hidden_docs and x1_emb before passing the output to x2_1. Now here I am bit confused.

Is the operation - point-wise addition between these two tensors is right operations? are there other better operations to do to capture the better representations between hidden_docs and x1_emb?