Paul Ganssle @pganssle@qoto.org

@benoit If you have a second phone the Google translate app has an AR type mode that might help.

If possible, it would be interesting to collect a bunch of performance stats for my computer use over the course of a few years to see how often I bottleneck on things like "hard drive read/write speed". Not sure if it's worth my time and $100 to replace this SATA with NVME.

I suspect that in almost all cases where I'm reading to or writing from the hard drive, the bottleneck is somewhere else — e.g. network, a drive I'm transferring to/from, speed to parse or serialize whatever data I'm reading/writing.

I imagine my most likely bottlenecks on IO would be during start-up (which is rare and I don't find it especially slow anyway) or if I'm reading/writing some memory-mapped file (can't remember the last time I've done this), but it'd be nice to get real data on this.

Of course, once I took my laptop apart to install the new hard drive I realized that my current SSD is NVME M2 and I don't even have a SATA cable for this... 😅

I ordered one, we'll see if I notice a drop in speed for any of the tasks I do regularly.

Recently got a new 2TB SSD for my laptop and I transferred everything over from my old SSD using a USB 3.0-SATA cable and it took about 30 minutes to transfer ~400 GB with rsync, which I found extremely impressive (especially since it was a lot of little files).

This looks more like what I was looking for at the time: https://codecalamity.com/encoding-uhd-4k-hdr10-videos-with-ffmpeg/

I've written a little Python script wrapping the advice here and it seems to be working very well (though I haven't tried it on any non-HDR and non-x265 content...)

I think this actually boils down to this bug: https://trac.ffmpeg.org/ticket/7037

If I had more time to play with OSS and more experience in this domain I might jump on it. Probably in the meantime I'll try to cobble together a script to mostly-automatically carry over the HDR metadata in transcodes.

Resurrecting this because I finally got around to trying it: unfortunately, this didn't work well for me. I get a lot more opportunity to tweak the color palette, but I haven't gotten one that turns out quite right yet, which makes me think this approach would need about as much tweaking as manually figuring out the parameters to copy over HDR metadata.

Additionally, Tone-mapping requires using `zscale` to make the inputs linear, which requires compiling with `--enable-libzimg`, which Arch linux and most docker containers I've found don't do. I managed to create a docker container that installs `ffmpeg-full` from the AUR, but that was a whole complicated affair, unfortunately, and takes forever.

@benoit Plot twist: HR is full of tax protesters, and they send you a bunch of literature on how you never needed to pay taxes in the first place.

https://en.wikipedia.org/wiki/Tax_protester

@timorl If it's an open instance you could also make an account there and try following someone here.

@freemo I mean, the crowd did kill two of those people... One deliberately, the other accidentally.

@freemo Looks like one of those deaths was trampled by other rioters, one had a heart attack and another had a stroke.

Hard to blame the police for any deaths except the woman who was shot.

@freemo Yeah, should have been multiprocessing.RLock or whatever specific re-entrant lock you are using. I probably shouldn't have even namespaced it 😛

@freemo Hmm, weird. If there is no solution that allows you to have more than 500 RLocks, maybe you can get a larger array by locking "blocks" of the array for your atomic operations. Then you can have 500 locks spread across 20,000 elements.

Python is always pass by reference, so you can do something like this:

locks = [threading.RLock() for _ in range(500)]
locks = locks * int(math.ceil(len(array) / 500))
random.shuffle(locks)
locks = locks[:len(array)]

Each element is randomly assigned a block of locks, so even if the access pattern is non-random the lock distribution will be. At any given time you have 32 processes contending for 500 locks, which seems like good odds.

@freemo Though TBH I'd kinda love to have a problem that admits an embarrassingly parallel solution as an excuse to write something significant in Rust to test out that "fearless concurrency".

Closest I've come is this: https://gitlab.com/pganssle/metadata-backup

There's a big queue that theoretically could be read in parallel, but I think the file system access ends up blocking, because adding multithreading into the mix doesn't seem to have meaningfully sped anything up.

@freemo Weird. If you are doing so much in parallel and it's a big part of your operation (and you think it's worth it to explore this further) it might make sense to try out using Cython or Numba with a function that releases the GIL, then use multithreading instead of multiprocessing.

Running hundreds of processes and serializing / serializing your data probably creates a ton of overhead.

@freemo And yeah, I am far from thinking Python has the best, most ergonomic parallelism story, but I do think Python scales better than many people give it credit for. 🙂