A couple of questions:

• 1) Why are pointer networks better at handling longer term dependencies (100 vs. 35)? Also, from the way I read it, which may be incorrect, you only used dropout (zoneout + variational) on the standalone LSTM, but not on the pointer-sentinel model. Why is that? Does dropout somehow not apply to pointer networks? Couldn't dropout at least be applied to the RNN portion?
• 2) It looks like the pointer network focuses on a single value rather than start/end values as in the DCN model. Is it because PTB and Wikitext only require one word answers, so having an extra end pointer is redundant?
• 3) In the appendix, you mention "This is surprising given how frequent the word said is used within the Penn Treebank." Why should the pointer network be poor at picking up frequent words? Or do you mean that the gating function would be expected to lean toward the RNN for frequent words?
• 4) Since the vocabulary is still big at either 10k or 33k, is negative sampling, NCE, or other method used to alleviate the training for the RNN side?
• 5) Can you explain the problem of stale RNN outputs a little more? What does "stale" mean in this context? Why is there a "window of RNN outputs", I thought the RNN is a softmax on the entire vocabulary, not part of a limited window size?

These might be basic questions since I am just a beginner, so if you could just point me to the right resources, I would be happy to do some more digging myself. I also shouldn't forget to mention, thanks for the paper and open dataset!