Simon @spoltier@qoto.org

Here's my end-of-year review of things we learned about LLMs in 2024 - we learned a LOT of things https://simonwillison.net/2024/Dec/31/llms-in-2024/

Table of contents:

I am SO happy when I find just one other person who agrees with me that the skills of following long form narrative fiction (with all its causality and many layers of information) can be the skills of understanding software

"The analogy between the “complexity of a large and thoughtful literary novel” and that of “a large computer program” holds because both literature and computer programs deal in “sophisticated information representations” (Blackwell, 2023)."
- Jain, 2024 (link next post)

@RuthMalan 1 fav video of 2024 shared on LinkedIn, of all places, by Caroline A Wanga, CEO Essence. She has a heuristics of expecting & giving yourself a “Daily Recommended Failure Allowance(DRFA)” of 5, like vitamins. You set your number & mitigating heuristic
This approach is life changing! #CarefulCactusHugging
e.g.
🎶When I🎶
-repeat myself, giving the same instruction 3 times, write it down in shareable form.
-try something & fail 2 times, ask for help & be specific.
https://mastodon.social/@dahukanna/111703530399450454

Yet I also feel very strongly that if I am focused on walking on eggshells, trying to "sound like the ingroup," trying to be trendy, trying to be Online Like a Tech Person, I won't do good science and I will write defensively and my thinking will suffer for it.

How I use LLMs – neat tricks with Simon’s LLM command

Earlier this year I co-authored a report about the direct environmental impact of AI, which might give the impression I’m massively anti-AI, because it talks about the signficant social and environmental of using it. I’m not. I’m (slowly) working through the content of the Climate Change AI summer school, and I use it a fair amount in my job. This post shows some examples I use

I’ve got into the habit of running an LLM locally on my machine in the background, having it sit there so I can pipe text or quick local queries into it.

I’m using Ollama, and the Simon Willison’s wonderful llm tool. I use it like this:

llm "My query goes here"

I’m able to continue discussions using the -c flag like so:

llm -c "continue discussion in a existing conversation"

It’s very handy, and because it’s on the command line, I can pipe text into and out of it.

Doing this with multi line queries

Of course, you don’t want to write every query on the command line.

If I have a more complicated query, I now do this:

cat my-longer-query.txt | llm

Or do this, if I want the llm to respond a specific way I can send a system prompt to like so:

cat my-longer-query.txt | llm -s "Reply angrily in ALL CAPS"

Because llm can use multiple models, if I find that the default local (currently llama 3.2) is giving me poor results, I can sub in a different model.

So, let’s say I have my query, and I’m not happy with the response from the local llama 3.2 model. I could then pipe the same output into Claude instead (I’d need an API key and the rest set up, but that’s an exercise left to the reader, as the LLM docs are fantastic)

cat my-longer-query.txt | llm -m claude-3.5-sonnet

Getting the last conversation

Sometimes you want to fetch the last thing you asked an llm, and the response.

llm logs -r

Or maybe the entire conversation:

llm logs -c

In both cases I usually either pipe it into my editor, which has handy markdown preview:

llm logs -c | code -

Or if I want to make the conversation visible to others, the github gh command has a handy way to create a gist in a single CLI invocation.

llm logs -c | gh gist create --filename chat-log.md -

This will return an URL for a publicly accessible secret gist, that I can share with others.

Addendum – putting a handy wrapper around these commands

I have a very simple shell function,ve that opens a temporary file, for me to jot stuff into, and upon save, echoes the content to STDOUT, using cat.

I use the fish shell

This then lets me write queries in editor, which I usually have open, without needing to worry about cleaning up the file I was writing in. Because LLM stores every request and response in a local sqlite database, I’m not worried about needing to keep these files around.

function ve --description "Open temp file in VSCode and output contents when closed" # Create a temporary file set tempfile (mktemp) # Open VSCode and wait for it to close code --wait $tempfile # If the file has content, output it and then remove the file if test -s $tempfile cat $tempfile rm $tempfile else rm $tempfile return 1 endend

This lets me do this now for queries:

ve | llm

One liner queries

I’ve also since set up another shortcut like this for quick questions I’d like to see the output from, like so:

function ask-llm --description "Pipe a question into llm and display the output in VS Code" set -l question $argv llm $question | code -end

This lets me do this now:

ask-llm "My question that I'd like to ask"

Do you use this all the time?

Not really. I started using Perplexity last year, as my way in to experimenting with Gen AI after hearing friends explain it was a significant improvement on using Search as it enshittifies. I also sometimes use Claude because Artefacts are such a neat feature.

I also experimented with Hugging Face’s Hugging Chat thing, but over time, I’ve got more comfortable using llm.

If I wanted a richer interface than what I use now, I’d probably spend some time using Open Web UI. If was to strategically invest in building a more diverse ecosystem for Gen AI, it’s where I would spend some time. Mozilla, this is where you should be investing time and money if you insist on jamming AI into things.

In my dream world, almost every Gen AI query I make is piped through llm, because that means all the conversations are stored in a local sqlite database that I can do what I like with.

In fact, I’d probably pay an annual fee (preferably to Simon!) to have my llm sqlite database backed up somewhere safe, or accessible from multiple computers, because as I use llm more, it becomes more valuable to me, and the consequences of losing it, or corrupting it in some way become greater.

If you have had success using llm that way, I’d love to hear from you.

#AI #datasette #generativeAi #llm

Paris and Berlin are now linked with an 8 hour daily rail service, that starts at €59. It produces 100th of the emissions of flying between the cities.
https://www.euronews.com/travel/2024/12/13/high-speed-train-from-paris-to-berlin-to-launch-this-winter-with-daily-connections-from-59

The man who was, arguably, the last American president with a chance to save the country just died.

Welp.

Rest in Peace Jimmy Carter, one of the few American leaders who called out apartheid and who wasn't just calling for "peace" in Israel/Palestine, but also for justice for Palestinians.

#Israel #Palestine #Gaza

The Oxide and Friends 2024 retrospective is here! @bcantrill and I look back on a year of the podcast and pick our favorite episode s, moments, and—at length—images!

https://youtu.be/mZd1KZuQ0gE?si=4LeCJMELxSZNfmnW

Will the "no" win this poll?

Boost if you like formal logic.

Soft launching this here before I talk about it anywhere else.

My book, "Permissionless: A Manifesto for the Future of Everything" is live. It explores how progress happens when we stop waiting for approval and start building.

You can read the full book free on the website (it was designed for it!) but if you want more, the eBook, PDF, and book notes are also available.

https://www.permissionlessbook.com/

ahh, we have infected the dataset

The DeepSeek v3 paper came out this morning, added a few notes about that here https://simonwillison.net/2024/Dec/26/deepseek-v3/

Do you have a big monitor and often lose your mouse pointer on the screen?

Are you in denial about being old enough to turn on mouse trails?

Well, I have the solution for you!

A desk kitten!

Desk kittens can locate your lost mouse pointer in approximately 3 nanoseconds and feature a clear and reliable indicator that ensures you know EXACTLY where your mouse pointer is.

Get your desk kitten today!

Act now and we’ll throw in the relaxing “purr” mode upgrade, guaranteed to relieve stress while responding to people who are wrong on the internet.