Follow

Very nice! I didn't know about zstd. Trying it out:

Comparison: tar.gz 1.9G 11 min, tar.xz 1.2G 88 min

33 min, 1.3G result, -19 and all cores:
tar -c -I 'zstd -19 -T0' -f cleaned_files.tar.zst cleaned_files/

2 min, 1.7G result, -11 and all cores:
tar -c -I 'zstd -11 -T0' -f cleaned_files11.tar.zst cleaned_files/

Result: -11 compression on the ZSTD was smaller than the gzip compression and also more than 5 times faster. -19 compression was almost as small as the xz compression, and nearly 3x faster.
QT: qoto.org/@worldsendless/108522

(webdev Tory) :emacs:  
I have a directory of 529k json files, totalling 9.5G. Compression: tgz (zip): 1.9G. tar.xz: 1.2G. The 700 mb savings took my machine 1 hour 22 min...
Sign in to participate in the conversation
Qoto Mastodon

QOTO: Question Others to Teach Ourselves
An inclusive, Academic Freedom, instance
All cultures welcome.
Hate speech and harassment strictly forbidden.