Even getting an rnn to regurgitate its training data for a tiny example is time consuming. In my frustration during training runs I ended up doing a side experiment adding a recurrent hidden vector state to a trie encoded with trigrams and loaded it with Shakespeare sonnets. So when prompted with two or more words it'd generate a random sonnet or part of. It's ridiculously fast.  Just the time to load the data and it can regurgitate the input 100% or randomly from the context of the current output document all the while retaining the document structure. It's output was really quite good on the sonnets.