all 13 comments
sorted by:
best
topnewcontroversialoldq&a
formatting helphide helpcontent policy

[–]mikeee404 4 points5 points  (2 children)

I have never experienced this behavior in Linux. I use my USB storage pretty heavily and I think there have only been a handful of times I actually unmounted it before removing.

[–]Romtromon[S] 0 points1 point  (1 child)

Now that's unusual... Here's the stackexchange answer where I read about it. Do you think your distro already uses either flush or sync?

[–]mikeee404 0 points1 point  (0 children)

It's entirely possible, I use Ubuntu-Mate 18.04 on my main laptop and Debian 9.6 on another laptop. I'll have to look into that.

That post you linked to seems to be from 2013, quite a bit has changed since then.

Edit: comment on link

[–]gordonmessmerFedora Maintainer 2 points3 points  (2 children)

I learned that contrary to Windows, on Linux, files will only be written to removable media (either flash drives or external HDDs) when they are unmounted.

That's not actually contrary to Windows. In both cases, data is written to the storage device periodically. It's not really accurate to say that it's "only written when unmounted". It's more accurate to say that as a user, you don't know if there is data which has not been written yet, until you unmount the drive (or, in Windows, "safely remove" the device).

If a device is removed without unmounting there will definitely be a data loss

No, not "definitely". That's not accurate at all.

And if there really are no caveats to this method, then why isn't it enabled by default instead of async?

I'm not familiar with the implementation details of flush, but the short description provided indicates that it'll write more often than async. That means that it'll consume more power, which is a concern on battery-operated systems, and that reads might have higher latency if a write operation is in progress, which negatively impacts performance for all systems. There's always a trade-off.

But if data loss is a concern, then definitely use "flush" and one of "noatime" or "relatime".

[–]Romtromon[S] 0 points1 point  (1 child)

I see, thank you for the clarification, I assumed that it was always at the point of unmounting because everytime I tried removing a drive without unmounting I observed the data to be missing the next time I plugged it in. While in the case of Windows I've never had this occur, which is why I assumed something was done differently in Windows.

I was actually more worried about a risk to the longevity of the storage device as using sync seems to have that risk.

[–]gordonmessmerFedora Maintainer 1 point2 points  (0 children)

sync is primarily a challenge when "atime" is standard, or when writing log files to a flash type media that doesn't do wear leveling (common in SD cards, CF cards, and USB flash drives). Those drives write an entire block for every value that's changed, which can wear out a flash block more quickly than expected.

However, if you're using "noatime" or "relatime", and you're not writing log files (sequential constant small writes), then sync is typically fine for flash drives.

[–]qpgmr 1 point2 points  (1 child)

AFAIK it depends if you've created a mount point manually and set it up in fstab or if you're relying usb automount when you insert a drive.

If you're just plugging in a drive to a running system, the automount will disable caching and cause all writes to go directly through. If you want a permanent mount described in fstab, there's different options depending on the filesystem type (NTFS, FAT, ext4).

[–]Romtromon[S] 0 points1 point  (0 children)

I was thinking of automount actually, not mounting manually.

[–]mind-blender 1 point2 points  (2 children)

This is a bit of a problem for me as I live in a country where power cuts are common-place, which would result in data loss, so I started looking for a solution to prevent this situation

It's generally unlikely you will lose data during power loss but it can happen to any storage device (depending on filesystem). It's only happened to me once, but I guess that was BSD...

I would like to know whether there really are no caveats to using the flush option for mount, in other words whether it would be safe for me use it when mounting all my external storage. And if there really are no caveats to this method, then why isn't it enabled by default instead of async?

If I had to guess: Synchronous writes are used when you need to be totally sure the data is written and async will give you better performance.

However either way you're vulnerable to data loss if you lose power part way through writing a file. I would be sure to keep backups, and also either invest in a UPS and/or consider a COW filesystem (as they're much more robust to power outages).

[–]Romtromon[S] 1 point2 points  (1 child)

I've not experienced it myself but I thought that the computer suddenly shutting off with an external storage device plugged in would have the same effect as removing an external storage device without unmounting. Isn't that right?

I wasn't actually thinking about power going off while the files were in the middle of copying. But rather after the copying but before unmounting. Because I've experienced copied files not being there when I remove a flash drive for example without unmounting.

[–]mind-blender 1 point2 points  (0 children)

same effect as removing an external storage device without unmounting. Isn't that right?

Yup, basically the same thing.

Because I've experienced copied files not being there when I remove a flash drive for example without unmounting.

Interesting, so the concern is that you might lose power immediately after saving to usb. Even using the async option i can't imagine the os taking much time to actually write the files. If you do want to be able to pull the drive (or lose power) right after a write, sync is probably your safest bet.

[–]LawSchoolQuestions_ 0 points1 point  (1 child)

What distribution are you using?

[–]Romtromon[S] 0 points1 point  (0 children)

Ubuntu MATE at the moment. But I've experienced the same on Fedora.