Martin Ruskov @mapto@qoto.org

The full text has been just made available by CEUR-WS https://ceur-ws.org/Vol-3365/paper6.pdf

RT @PopovaProf@twitter.com

Stumbled upon this 2003 quote from Ukr president Kuchma. Ru had just tried to take over the Ukr island of Tuzla in the Kerch Straight and build a dam to connect it to Ru mainland. Perhaps Pu was already planning his Kerch Bridge, which indeed passes over and is anchored in Tuzla.

🐦🔗: https://twitter.com/PopovaProf/status/1642737144866172928

Изборите в България бяха проведени успешно. Мнозинството избра още от същото негласувайки. Гласуващите също ги подкрепиха

RT @emilymbender@twitter.com

Statement from the listed authors of Stochastic Parrots on the “AI pause” letter

https://www.dair-institute.org/blog/letter-statement-March2023

"Regulatory efforts should focus on transparency, accountability and preventing exploitative labor practices."

w/@timnitGebru @mmitchell_ai@twitter.com and @mcmillan_majora@twitter.com

🐦🔗: https://twitter.com/emilymbender/status/1641891929788149760

RT @timnitGebru@twitter.com

Not only should it always be clear when we are encountering synthetic media, but organizations building these systems should also be required to document and disclose the training data and model architectures.

🐦🔗: https://twitter.com/timnitGebru/status/1641894105084203008

RT @timnitGebru@twitter.com

The onus of creating tools that are safe to use should be on the companies that build and deploy generative systems, which means that builders of these systems should be made accountable for the outputs produced by their products.

🐦🔗: https://twitter.com/timnitGebru/status/1641894106006962176

RT @timnitGebru@twitter.com

Since we've been looking for more things to do, @emilymbender@twitter.com @mmitchell_ai@twitter.com @mcmillan_majora@twitter.com and I wrote a statement about the horrible "letter" on the AI apocalypse, the very first citation of which, was our #StochasticParrots paper.
https://www.dair-institute.org/blog/letter-statement-March2023

🐦🔗: https://twitter.com/timnitGebru/status/1641893027492020224

I tried using #Midjourney to illustrate fairy tales. My intention was to design a process applicable across models and for wider #narratives. I succeeded with the task. It remains to be seen if the process lives up to the expectations https://arxiv.org/abs/2302.08961 @projectVAST@twitter.com

RT @mapto@twitter.com

Not long ago I presented my experiments on illustrating famous fairy tales with #Midjourney v4 at the #IRCDL23 (http://lacam.di.uniba.it/IRCDL23/) This experimentation is part of the VAST project https://www.vast-project.eu/ that studies values present in different… https://qoto.org/@mapto/110111252142814403

🐦🔗: https://twitter.com/mapto/status/1641354595329875968

Now Midjourney v5 is out and it is all about realism - it gets 5 fingers correct. Also, in my experiments I limited myself to not using image prompts and chaos coefficients. So this experimentation definitely could be continued, but certainly I'm done with with commercial closed generators, and certainly we need a more scalable way to evaluate image correctness

Another encountered issue was that it was extremely difficult to force the generator to draw 7 dwarfs. Notoriously, 5 fingers of a hand are also difficult, but the larger the number the smaller hope to get it by chance. A third difficulty I found was the impossibility to generate impossible scenes, typical for fairy tales, such as #RedCap popping out happily from the wolf's belly or #Gretel shoving the witch into the oven. A quick comparison with #dallE showed that @openai's model handles somewhat better two of the above challenges (other than quantities). Yet, it struggles with unwanted #artefacts

The 4-step process is not to be seen as something strict. Rather, it is a suggestion that it would bring little benefit to proceed to a subsequent step, without getting the one before at least approximately right. Moving backwards in the process is not ruled out, yet again, before moving back, try consolidating your progress so far. Although it allowed me to generate the intended number of illustrations, while doing this I also confirmed that specific scenes are close to impossible to generate. For example, trying to illustrate how Cinderella plants a tree at her mother's grave, I couldn't find a way to make #Midjourney generate the scene without a grown tree already present - something that makes the scene absurd

The final fourth step, once we have a roughly satisfactory result, is to try variations that might just fix the remaining issues through randomisation

Once composition is at least roughly right, the third stage to choose a style that helps the efficiency. One that possibly reduces hallucinations, yet eases interpretability by the viewer. For fairytales, "book illustration" is a possibility

In the second stage, considering the outcome of the first step, I aimed to isolate parts of the prompt to be removed, added or replaced to improve the composition of the image, aiming to add important elements and to remove unwanted ones.

The first stage of the process is converting the intended text to a prompt, without deviating from the original vocabulary. This means condensing content into a single phrase, removing words that are not meant to be visualised, e.g. this, here, he, and substituting them with what they refer to.

Other models followed, in November came Midjourney v4 and in December Structured Diffusion Guidance (https://arxiv.org/abs/2212.05032). Better composition was notably easier to achieve. This allowed me to proceed with (and complete) my tasks of illustration of fairy tales. All resulting images can be seen in the paper, but the other important outcome is the definition of a preliminary process for the generation of images aligned to the original story text.

Unlike the widespread #AIart, this task had the constraint to adhere to the original text, thus keep in check some peculiarities of generator models, such as #hallucinations. When working on my task, I quickly discovered the obvious: that #DescriptiveText-s are somewhat easier to generate for. (example RedCap). However, fairy tales contain much less descriptions than one might recall from child memories. So the challenge of the task was to generate illustrations also for #narrative. While it might be too ambitious to try to illustrate for a sequence of events (what would make a narrative), even describing a single event requires an interaction or a scene composition. However, interactions or compositions were notoriously difficult to get right by generative models. Until in mid-2022 Google's Parti (https://arxiv.org/abs/2206.10789) made a notable breakthrough by linking the image generation to (text) transformer models