# Test 1

Let's document this one properly.

## Preparation

Downloaded the video using `youtube-dl`.
Extracted the relevant part of the sound, from the moment it becomes clear (IMO) to when the video cuts to another part of the crowd.

```
ffmpeg -i Rioters\ chant\ \'hang\ Mike\ Pence\'\ as\ they\ breach\ Capitol-ba0UR7gITrU.mp4 -vn -acodec copy chant.aac
```

Created a video out of the sound file with a irrelevant name and the least political picture I could find on short notice (a drawing of a mathematical pun in Polish).

```
ffmpeg -loop 1 -y -i ../kurakLematowskiegoZorna.jpg -i chant.aac -shortest -acodec copy -vcodec libx264 sillyTestVideo.avi
```

Uploaded the result to YT, as of now there are no auto-generated captions present, but the instructions suggest this might take a while.

On second look, if I'm understanding the UI correctly it generated captions already, but they are _empty_. There is a warning it might not generate proper captions if there are multiple people speaking, so maybe that's a problem. That would make the results inconclusive again. Oh well, I can wait just to make sure before declaring that.

Well, that ended up silly. YT managed to autogenerate captions, but not for the chant, but for some barely audible person talking close to the person recording. And all the words it identified were "el bote no". Waiting for Q theories how this proves these were Mexican antifa who entered the capitol by ship and had problems escaping.

At least this is a very clear inconclusive result. I'll continue tomorrow with the other tests, but the odds of me needing to use actual money on this are rising.

# Test 2

Apparently speech-to-text is something only professionals usually do, because the tools I managed to find are not especially easy to use. For now I managed to get julius, followed instructions on its GitHub substituting the file I wanted for the test file. It needed to be converted as follows:

```
ffmpeg -i chant.aac -ar 16000 -map_channel 0.0.0 chantL.wav -ar 16000 -map_channel 0.0.1 chantR.wav
```

The two channels were actually indistingiushable as far as I (and julius btw) can tell. Unfortunately all it recognized was "details had", which means it probably also picked up some random person talking, treating the chant as background noise.

I'll try cutting the file into smaller bits (bit per word, where I think they are most clear), since I will need to do this for further steps anyway, and check whether this helps.

tl;dr Did not help.

The first word is recognized as "oh", the second as "five", the last as "but added". These are so nonsensical (especially the last one) that I believe they provide no evidence one way or another ("five" kinda sounds like "Mike"? pfffft), except for julius being terrible at transcribing chants. _Maaaybe_ this is tiny evidence towards 3., since a chant that's incomprehensible to programs might also be incomprehensible to humans.

I'll try at least one more software of this kind, but at this point I believe mturk will be necessary.

Oh, and I did the cutting from audacity, aiming from the CLI would have been 面倒くさい .

Next I used this: https://github.com/facebookresearch/flashlight/tree/master/flashlight/app/asr/tutorial

It did not detect any words in the first clip and the word "one" in both the other clips. This suggests it again was picking up on noise different from the chant. It also didn't detect anything on the full chant.

Finally I tried Vosk. Did not detect anything on any file.

Welp, MTurk it is. But not today.

There we go, sent both the full chant (without repetitions, I just picked the IMO clearest sounding instance) and single words (cut from the full chant). I requested 20 answers for every data piece, which should be enough for reasonable evidence (unless the responses are atrocious quality). I expect some people will be familiar with the full chant, so answers which correctly identify the given name present there are weaker evidence. With the single words this problem should be somewhat mitigated.

Still hoping I'm not theonly one hearing "Hang my pants!" when ignoring the context.

Oh my this is fast, already almost 1/5th done and I barely finished writing my previous toot!

Well, after the initial surge this took a whole day to complete. I looked a bit at the data and it's pretty disappointing, but I'll analyse it properly in a moment.

So lets start with the predictably most disappointing, the full chant. Four people were clearly familiar with the chant, divided equally between the "standard" interpretations. One more person had an interpretation that was not exactly one of the standard ones, but close enough to make me suspect they were also familiar. Two further people had interpretations that were clearly made through careful listening, somewhat phonetically close to one of the standard interpretations (one each, lol). Ten people had done a terrible job and returned nonsense, wild guesses or just claims it's unintelligible. Three people had tried, but their interpretations are not close to either of the "standard" ones, and phonetic similarities are unclear.

This is relatively strong evidence for 3, and against both 1 and 2.

Before I get to the first word, I need to point out that some people who listened to the full chants were also guessing single words. Among them only one managed to identify that one of the words was part of the full chant and assign the same word as he assigned to it in the full chant. Because of that I decided to not remove these people from the results (including the guy who managed to guess).

So, first word. Here I must confess I did not manage to cut the word off properly, because I was trying very hard not to include the part I percieved as "m" in the clip, and I cut off also the part that I percieved as the "ng" sound (or maybe they were both just the same sound? considering 3. is a real option this might be the case). Anyway here we had more interesting results. Two people heard something beginning with "w", one of them even the "we" sound (although a different word). Nine people heard something starting with the "he" sound (ugh, not "he" as in "he", but "he" as in "hello", so what I meant by the "ha" sound in the original post; ugh, bloody english phonetics), three further people heard things that contained this sound. It's worth mentioning that one of the people claimed to hear the exact word I heard, but followed by another word, which seems like they were guessing, because I see no way in which they could have heard so much in this short clip. They were also one of the people who were aware of the chant (although strangely they claimed they heard the version I _did not_ hear in the full chant?!). And since I'm mentioning this, the person who claimed to have heard the "we" sound was also among the ones familiar with the chant. It is also worth pointing out that many (5/9 or 7/12) of the "he" guesses were variants of "hey" or "hello", towards which people might be biased.

The remaining six people heard mostly nonesense, although three of them also heard variants with some "h" sound.

The wide prevalence of the "h"/"he" sound is moderate evidence against 1, and about equally for 2 and 3.

The second word is apparently the least intelligible. Three people explicitly complained about unintelligibility, six made suggestions that made no sense. Four suggestions contained the "w" sound, but none of them in a way that would be compatible with one of the standard interpretations, and the only one beginning with "w" was someone incorrectly trying to guess which word from the full chant it was (lol). Three people heard the "n" sound – that's not "m", but phonetically very close, and it was always at the beginning of the word. Four people heard the "eye/I" sound as the vowel – interestingly this group is exclusive with the previous one.

This is mostly evidence for 3., but also against 1., since the phonetics align somewhat more with what I heard.

The third word is the strangest. Three people guessed that it was the third word from the chant (I missed them in the original toot where I claimed there was only one, oops... >.<), but they are almost the only people who have noticed the "(t)s" sound at the end – only two more have that sound. There is one explicit unintelligible and two terrible guesses. The remaining 12 people all correctly identified the vowel in the word we expect, but 9 of them also heard some variant of "hey", which should influence the evaluation of the first word (at least the "hey"/"hello" variants might be almost pure bias; note that there were still many others which had different h-words).

Since the "standard" interpretations agreed as to what word this is, people not recognizing it is definitely evidence for 3. On the other hand, this is not really evidence for 3 as applied to the previous two words, they might have been perfectly legible and this word only guessed from context, so it's not very strong evidence for 3 as applied to the problem we are trying to solve... And correctly identifying the vowel seems to be a pattern too...

Anyway, it seems that mostly I should assign much more weight to 3, bloody meatmech really needs an upgrade. On the other hand when the phonetic evidence pointed in a direction it was more in agreement with my interpretation. This is some evidence against me being insane, but it might also be the result of how I cut the chant into pieces?

To finish this up, the raw results I got:

* https://wuatek.tk/~timorl/dlaludzi/inne/transcripts_grouped.csv are the transcripts grouped by me as described in thee above toots.
* https://wuatek.tk/~timorl/dlaludzi/inne/transcripts_random.csv are in random order, went through shuf because I already modified the grouped file when I noticed this version would be useful <_<"

The links inside should be pointing to the files I used for the MTurk questions.

@freemo If you want to look through this yourself, you definitely want the **second** file. I know this is obvious, but stressing it to lower the chances of you accidentally biasing yourself more than needed by first looking at the first one.

Oh, and if you want to help me with all this investigation it might also be useful if you cut the chant into words the way you perceive them, preferably **before** listening to the cuts I made. I have a slight suspicion word boundaries might help me hear your version? I know this would take time though, so no pressure if you have better things to do.

@pettter I am definitely familiar with the general phenomenon (thus point 3 being an option; I was referring to this: https://en.wikipedia.org/wiki/The_dress), although I was not familiar with the specific name for the auditory one. Thanks!

@FailForward Not really, the courts might decide whether there was enough evidence for crimes, perhaps the evidence would even include chants with death threats. However, I believe this might not even be neccessary to establish intent, so I highly doubt they will investigate this video/chant in much detail.

And you are right to an extent, as far as the original discussion purpose goes establishing intent would be crucial to support my argument. What I am trying to do now though, is not continuing that discussion, but investigating how heavily my senses are distorted by my biases. Since the courts likely won't investigate the specific chant for which I have reasons to believe such distortion exists, they won't help me in this matter.

@timorl I am just seeing this thread now. Man that really must have bugged you. It is common people can hear what they want to hear in a distorted audio clip (which of us is guilty of that neither can truly say).

I'll catch up on this thread and see if i have any remarks, but at least your digging, and thats always a good sign when it comes to our biases.

@freemo
It does definitely bug me. I am aware of the general problems with human perception, but I'm usually relatively good at perceiving the alternatives in those strange cases. In this case I tried quite hard and I cannot hear anything close to what you are hearing. This suggests something might be much more broken in my meatmechs input interface than I thought.

And since money is the unit of caring, and I am about to spend actual money on this problem, it appears I really care.

Oh, and just to be clear, I don't think this should be seen as a continuation of our discussion – as explained above the motivations are very personal.

@freemo
And I'll be grateful for suggestions as to how else to test this empirically, if you come up with any. Thanks!

@timorl The problem with human interpritation is, as you point out, the bias. People hear what they want to hear.. So what do people want to hear?

@timorl Can you share the raw results before/along with your interpretation please? I think that would be useful for others to "peer review" your analysis.

@freemo Right, I'll definitely do that, I should've done this before writing my interpretations down, but I did not think of it. >.<

@timorl no worries, it is a lot harder to follow your interpritations or to evaluate them wtihout it is all.

@freemo Yup, I understand I'll write them down and post the file, so maybe don't read them beforehand if you still can? I'll tag you in the file-toot.

@timorl yea once i see the raw data ill read through your interpritation having that to compare with as i see your interpritation, and ill offer any counterpoints or agreements as I see them when i go through it.

@timorl Ok so not a big enough sample for me personally to do much with, but its a start. I would only use the full words.mp3 myself as a person needs context to pull out phrases in general.

How much did this cost you, I might be willing to throw in some money (if its cheap enough) to get a larger sample data on just the words.mp3

@timorl I'd be curious to see the specific responses given, but thats not surprising. When I heard it while it did seem to sound closest to "we want pence" it also wasnt the clearest audio. I noticed this more when i tried to listen for your version of the audio while i couldnt hear what you were hearing i could fool (or not fool as the case may be) my brain into hearing it as unintelligible in certain parts.

Well at least we settled the original issue that it was not a clear cut case of chanting for death.

@freemo Eh, since you brough up the previous discussion I was curious about one more thing, still related to the chant, I hope you don't mind me asking, but if you do maybe stop reading or feel free to ignore.

How does "We want Pence" make sense as a chant? The closest I see in the context is a thinly veiled threat, instead of an explicit one. Do you see something better or is this how you understood it?

Interestingly, one of the interpretations in the file imo makes more sense as a completely non-confrotational chant (and in general as a chant in the context) -- "We want best". It's a bit stretched, but "best" as a kind-of-nickname for Trump who often insisted on being best at many things does make some sense. On the other hand, his supporters rarely call him by anything other than his name, so ehhh... a bit stretched.

@timorl

> Eh, since you brough up the previous discussion I was curious about one more thing, still related to the chant, I hope you don't mind me asking, but if you do maybe stop reading or feel free to ignore.

I dont mind you asking, and in general I dont mind you casually bringing up points from it. I just didnt want to continue a point by point that was unproductive.

> How does "We want Pence" make sense as a chant? The closest I see in the context is a thinly veiled threat, instead of an explicit one. Do you see something better or is this how you understood it?

Hard to say, since as a group they didnt exhibit any violence towards people or politicians (as we covered), and that even when violence was done against them. In fact of the 5 people killed 4 of the were unarmed pro-trump protestors, the 5th was a cop who was actively beating the crowd with his baton when he was struck. So even the data we have from the day shows that they were not violent and were far more the victim than the aggressor in terms of engaging in human violence.

So with that said it paints a picture for me what the crowd likely would have done should Pence have shown up, probably surrounded him and yelled profanities at him until he slinked away out of sight, presuming the cops didn't escalate things, as they tend to do, and start shooting or beating even more people (at which point its hard to say if he would have remained safe).

they clearly didnt like pence though and they clearly wanted to let him know that face to face. I was not implying that it was a chant to show their **support** of pence, I was only claiming that it was not a chant of murder.

> Interestingly, one of the interpretations in the file imo makes more sense as a completely non-confrotational chant (and in general as a chant in the context) -- "We want best".

I personally would dismiss this as an explanation, too stretched.

@freemo Hm, so that's pretty much the same slight-threat interpretation as I had, makes sense.

And I think we still disagree what constitutes aggressive/threatening behaviour (and maybe still a bit about facts, considering the policeman being beaten with the flag(?)pole?), but I guess this is kind of beside the point for this discussion.

@timorl

This is the first time you refered to it as threatening/aggressive behavior rather than violence. That is a big difference in my eyes. There is a huge chasm between being physically violent towards a human being vs breaking a window, one I will call violence, the other I would agree is aggressive and even threatening, but isnt violence.

There is also the matter of considering who strikes a human first in an encounter of mutual violence to determine who is in the right in my mind.

So take the example of the police officer being beaten by the flagpole.. had he at any point earlier int he day beaten people in the crowd with his baton? Every single incident i saw where a police officer was even threatened (such as the one running through the halls as people chassed him while he periodically stopped to beat them a little with his baton)... in every case it can be seen the police officer initiating the violence by beating them with their baton or similar, and only after that (and usually repeated examples of that) did the crowd respond with any level of violence.

Even then those examples are very rare, in one case we have the officer who died who was reported and admitted to be actively beating the crowd with his baton before finally being struck, and in the other case with the flagpole, which we dont have video evidence to know, but seems possible he was doing the exact same thing.

Only other physical violence of any kind I know of from the events were explicitly non-lethal but also in like kind. For example they were being actively tear/pepper gassed by police and one protestor responded by pepper spraying the police in return.

REgardless overall it appears far more violence was done on them then they did in return, as is evidence by the death toll being almost exclusively trump supporters.

@freemo I still think they were violent, I would say trying to enter a building by brute force with people standing in your way is necessarily violent. They weren't violent only when they were not being opposed. The fact that they were aggressive/threatening follows from both the violence and the (explicit or implicit) threats.

Also, I saw you linked an article claiming that Twitter blocked the "Hang Mike Pence" hashtag -- do you happen to know if "We want Pence" was trending at the same time? This seems like relevant evidence as to what was being chanted, if we have the data.

@timorl depends on how they do it.. breaking a window isnt violence against a person, someone standing in your way and you simply walking past them anyway (pushing through them) also isnt violence...

Now if someone stands in your way and you beat them with a stick till they move, then that IS violence.

Ironically the only one (for the most part) being anyone with sticks were the cops.

I would say that they "physically overpowered" the cops, I would also say that a small number of people (the ones breaking glass) were "destructive of property"... but violence towards an individual is not something that is guaranteed as a matter of course simply by walking into a building you arent allowed or walking through a line of people standing in your way, particularly when no one actually struck a cop to do it