Trained a Pythia LoRA on my Twitter archive. It's so good at sounding like my tweets, but the problem is my tweets are insufferable 🤣 But this training a little LM LoRA thing was pretty easy and I can see plenty of use cases for it. e.g. personalized notification filtering, NPC personas in games, custom constrained translation tasks, etc. Nota Bene: you must add the LoRA layer output to the transformer layer output, not call them sequentially - took a bit to figure out but makes sense.

Here's the Python code I used to do it. If you copy your data/tweets.js from your archive, remove the `window.YTD.tweets.part0 = `, rename it to tweets.json and place it next to this script, it will train a you-bot: https://gist.github.com/ericflo/5b385303d589172d86512f0f38f810a3