Learning from the accident
DO NOT trust Windows explorer's copy function. Use FastCopy and verify your file.
Pause the sync when you keep changing your files, no matter what sync client you use: Google Drive, MountainDuck, or dropbox.
Use something like 7z and rar to provide some abilities to check file integration.
----
Today I got 30+ files with 0 bytes (the file should be 5MB~10GB), and about five files with missing content (should be 20+MB, but only got first 5~6MB synced).
Those are files I copied from Google Drive and pasted to MountainDuck. I don't know how I got those broken files, but I think it's related to Google Drive. I had a similar experience where the file on Google Drive is good, but downloading gives a broken file. And the sync part, the MountainDuck seems like using timestamp only and choosing the newer file, even if the file in the local disk is bigger while the last modified timestamps are the same.
Learning from the accident
@skyblond I've already put my NextCloud on pause on my notes from Obsidian because NC kept complaining about locked files since Obsidian writes way too often to those .md files. But other than that, I've not really had issues with Windows' built in copy. I do, however, always make sure to disconnect the USB devices via software first before I take them out.
Learning from the accident
Using FastCopy, even if Google Drive returned an unexpected EOF, the verify stage would find it and let me know (unless two unexpected TCP RST occurred at the same position and both failed the retry).
With Windows's copy, I just assume all files are properly transferred and delete the original file. With 7z or rar, I can verify them. But with other normal files, like mp3 or flac, I can only notice that when the music stopped too early.
Learning from the accident
@skyblond Ah, yes, I can see that. Mounting remote disks... you mean like WebDAV? I've never felt comfortably enough using that method. I prefer local files and then syncing them 'properly' via tools like NextCloud.
Learning from the accident
@trinsec Yes, I mount the WebDAV using MountainDuck (a paid software that mounts SFTP, S3, etc. as a native disk in Windows and macOS).
It has a builtin cache system, where you write your file into the local disk, and then the software takes care of the sync, just like Google drive. The IO request will be passed to the software and it will make the actual request. I have been using that software for 5 days. The writing experience is... not good. Basic copy and paste are fine, and reading is fine. Writing is not fine, especially when some app writes too many pieces and the software sometimes can't keep up.
So I'm wondering if I should mount them in read-only mode and upload files using SFTP.
Learning from the accident
@trinsec I think the main flaw of Windows's copy is that it doesn't validate file after copy.
Local disk is fine, normally they won't give false data, and the chunks are cached in memory. However, it goes bad when some applications mount the disk, like Google Drive, or MountainDuck. If the network got an unexpected EOF, I assume (based on what I observed, not sure to blame who) the read request would just get an unexpected EOF and call it a day. I don't know if Windows's copy didn't handle the failure well, or if the underlying application decided to give up and return the EOF anyway.
*Normally, the connection to Google Drive should be great, but recently my proxy has been unstable, and there are more and more temporary and short TCP RST.*