Martin Ruskov @mapto@qoto.org

I find these timelines very useful. Only that for me the interesting starts with Parti and Midjourney v4 (there's also a thing called Structured Diffusion Guidance), because these models also work on scene composition. Everything before that is pure randomness #text2image #AIart

Image from https://arxiv.org/abs/2307.07521v1

RT @BenDoBrown@twitter.com

Last week I spoke at @journalismfest@twitter.com on how open-source intelligence is used to combat propaganda and uncover new information during Russia's war on Ukraine. Thank you for having me.

I promised to share a list of links to tools and sources. Here it is (I hope it helps) 👇

🐦🔗: https://twitter.com/BenDoBrown/status/1650533044145586186

RT @rao2z@twitter.com

Interestingly, I was just telling someone today how several of the papers on "LLMs for Task Planning by Prompting" are rife with the Clever Hans effect (c.f. https://en.wikipedia.org/wiki/Clever_Hans ). I guess I will have to do a thread.. 2/

🐦🔗: https://twitter.com/rao2z/status/1648876535070457856

RT @mcxfrank@twitter.com

What does it mean for a large language model (LLM) to "have" a particular ability? Developmental psychologists argue about these questions all the time and have for decades. There are some ground rules. 🧵

🐦🔗: https://twitter.com/mcxfrank/status/1645459383554568193

RT @RSprugnoli@twitter.com

1st workshop on Ancient Language Processing (at RANLP-2023, Varna Bulgaria)

Paper submission due: July 3, 2023
Date: September 7, 2023
Website: https://www.ancientnlp.com/alp2023/
#NLProc #digiclass @ranlp2023@twitter.com

🐦🔗: https://twitter.com/RSprugnoli/status/1640995540920856576

RT @Mylovanov@twitter.com

I asked people why they think Russians did it. “To use us as a protection against the Ukrainian army” is the only answer I heard. Russians paraded kids in front of the building when Ukrainian drones were nearby. 9/

🐦🔗: https://twitter.com/Mylovanov/status/1643614587353280514

RT @babiejenks@twitter.com

Photos of Britney Spears on magazine covers are literally how I realized Americans couldn't tell the difference between a rictus and a smile.

🐦🔗: https://twitter.com/babiejenks/status/1442996955257479168

RT @ICFJ@twitter.com

Journalist @nameesarnous@twitter.com founded a magazine about, for, & from women in the Arab world. But her outlet was struggling financially. So she joined Elevate, ICFJ’s news business hub. Through her mentorship, she decided to develop 2 new revenue streams: https://buff.ly/3neLpVG

🐦🔗: https://twitter.com/ICFJ/status/1643101087623176194

RT @PopovaProf@twitter.com

Stumbled upon this 2003 quote from Ukr president Kuchma. Ru had just tried to take over the Ukr island of Tuzla in the Kerch Straight and build a dam to connect it to Ru mainland. Perhaps Pu was already planning his Kerch Bridge, which indeed passes over and is anchored in Tuzla.

🐦🔗: https://twitter.com/PopovaProf/status/1642737144866172928

Изборите в България бяха проведени успешно. Мнозинството избра още от същото негласувайки. Гласуващите също ги подкрепиха

RT @timnitGebru@twitter.com

Since we've been looking for more things to do, @emilymbender@twitter.com @mmitchell_ai@twitter.com @mcmillan_majora@twitter.com and I wrote a statement about the horrible "letter" on the AI apocalypse, the very first citation of which, was our #StochasticParrots paper.
https://www.dair-institute.org/blog/letter-statement-March2023

🐦🔗: https://twitter.com/timnitGebru/status/1641893027492020224

RT @mapto@twitter.com

Not long ago I presented my experiments on illustrating famous fairy tales with #Midjourney v4 at the #IRCDL23 (http://lacam.di.uniba.it/IRCDL23/) This experimentation is part of the VAST project https://www.vast-project.eu/ that studies values present in different… https://qoto.org/@mapto/110111252142814403

🐦🔗: https://twitter.com/mapto/status/1641354595329875968

Another encountered issue was that it was extremely difficult to force the generator to draw 7 dwarfs. Notoriously, 5 fingers of a hand are also difficult, but the larger the number the smaller hope to get it by chance. A third difficulty I found was the impossibility to generate impossible scenes, typical for fairy tales, such as #RedCap popping out happily from the wolf's belly or #Gretel shoving the witch into the oven. A quick comparison with #dallE showed that @openai's model handles somewhat better two of the above challenges (other than quantities). Yet, it struggles with unwanted #artefacts

The 4-step process is not to be seen as something strict. Rather, it is a suggestion that it would bring little benefit to proceed to a subsequent step, without getting the one before at least approximately right. Moving backwards in the process is not ruled out, yet again, before moving back, try consolidating your progress so far. Although it allowed me to generate the intended number of illustrations, while doing this I also confirmed that specific scenes are close to impossible to generate. For example, trying to illustrate how Cinderella plants a tree at her mother's grave, I couldn't find a way to make #Midjourney generate the scene without a grown tree already present - something that makes the scene absurd

The final fourth step, once we have a roughly satisfactory result, is to try variations that might just fix the remaining issues through randomisation

Once composition is at least roughly right, the third stage to choose a style that helps the efficiency. One that possibly reduces hallucinations, yet eases interpretability by the viewer. For fairytales, "book illustration" is a possibility

In the second stage, considering the outcome of the first step, I aimed to isolate parts of the prompt to be removed, added or replaced to improve the composition of the image, aiming to add important elements and to remove unwanted ones.

The first stage of the process is converting the intended text to a prompt, without deviating from the original vocabulary. This means condensing content into a single phrase, removing words that are not meant to be visualised, e.g. this, here, he, and substituting them with what they refer to.

To initiate such a process, I engaged with an investigation following the principles of action research (https://sonyaterborg.com/2016/02/17/action-research/) - an iterative methodology where research and practice go hand in hand. While a task is being completed, in parallel a reproducible process is being developed, lessons learned are being collected.

Not long ago I presented my experiments on illustrating famous fairy tales with #Midjourney v4 at the #IRCDL23 (http://lacam.di.uniba.it/IRCDL23/) This experimentation is part of the VAST project https://www.vast-project.eu/ that studies values present in different texts, including some fairy tales recorded by the Grimm brothers. My generations were an aside activity, and I attempted to generate 5 illustrations for each of 5 fairy tales: #LittleRedRidingHood, #Cinderella, #LittleSnowWhite, #HanselAndGretel, and #FaithfulJohannes