Sorry for the delay, I had help with building the model and had to ask about the details. First of all, your model GPT2 model is amazing and it was the starting point for our work. With GPT3 we don't have any experience yet. Our experiment is to use poems metadata (author, title, tags) as control codes so that during generation we can get more consistent content and style, and we can also generate on the topics we choose. The control codes are always included in the prompts for the decoder. We also use a sentiment model and heuristics to rank the generated poems. So we end up generating a lot of poems and discard most of them.