This week, I've been trying to train transformers for text generation, as I'm interested in whether recent progress in LLMs would improve the predictions of our animal behavior models (we currently use RNNs for this). This is my first foray into text generation! Appreciating having sample code to build off of, I have no idea what I'm doing :). Here's what my network says after 15 epochs of training:

"tech received the ball at its 16-yard line , and first down the second quarter. dunk's kickoff at the fumble, tech quarterback was the touchdown of the touchdown offense to begin a first down in the hokies was travelling for the touchdown from fullback 22-yard line , but passes for a 35-yard projects how the touchdown in miami quarterback to extend the touchdown."