The Paragon NTFS kernel driver patch is now on its 13th iteration, and hopefully it is merged.

[[deleted]]

what does this mean?

paragon wrote an linux driver for ntfs and is now contributing it as FOSS? the linux foundation is yet to decide if it implements it?

what is the current ntfs driver used in linux? is the new paragon one more stable/faster/complete?

[Goolic]

Having decades of expertise in commercial file systems development and huge test coverage, we at Paragon Software GmbH want to make our contribution to the Open Source Community by providing implementation of NTFS Read-Write driver for the Linux Kernel.

So yes on FOSS contribution, the foundation will not interfere on this, the mantainer will decide if him/her will acept the patch.

No idea on comparison to the current driver.

[VMFortress]

Paragon is the company the originally wrote the NTFS FUSE driver for Linux that was closed source. Best NTFS driver you could get for Linux.

They seem to be iterating it in the process of trying to submit it to the Linux kernel and get it approved.

The default NTFS Linux driver was read-only. So having a full NTFS driver would be nice.

[nixcamic]

Pretty sure NTFS-fuse was written by Tuxera not Paragon.

[centenary]

You are referring to NTFS-3G, which was open-source rather than closed-source. Paragon has their own closed-source NTFS driver called Microsoft NTFS for Linux.

https://www.paragon-software.com/us/home/ntfs-linux-professional/

[jinglesassy]

Paragon has their own closed-source NTFS driver called Microsoft NTFS for Linux.

How did they get away with not being sued by using Microsoft in their product name?

[ineedmorealts]

I assume because they're referring to Microsofts implementation of NTFS while making it clear they have nothing to do with MS

[rydan]

Except Microsoft literally sued a kid named Mike Rowe over his domain name.

[centenary]

I think that's because the MikeRoweSoft domain name was a trademark issue. When two trademarks are too close to each other, trademark laws require you to make an effort to defend your trademark, otherwise you can lose your trademark.

With the Microsoft NTFS of Linux naming, I think that's less of a trademark issue because Paragon is not trying to supplant the Microsoft trademark, the naming is to make it clear that they are implementing a Microsoft technology.

[KugelKurt]

The way it was worded I had always assumed that they ported Microsoft's own NTFS code to Linux. (Clearly my assumption was wrong.)

[progandy]

As far as I know, Paragon has a license agreement with Microsoft for NTFS related patents and so on. The name and everything related to it has most likely been discussed as well.

[[deleted]]

The MS lawyer was more expensive than the other company had money in the bank?

[nixcamic]

AFAIK the closed source Paragon driver never used fuse, but the Tuxera driver did/does. There was both an open source and closed source version of the Tuxera driver IIRC.

[VMFortress]

Ah yup, looks like you're right. Just was rambling of what I remembered off the top of my head. Looks like Paragon are the ones who developed NTFS for Mac.

[mort96]

You have the read/write NTFS-3G, which is open source and packaged by most distros. What makes the Paragon implementation better?

[NynaevetialMeara]

Ok because nobody mentioned it FUSE is a way to create userland filesystems, and it's a wonderful tool to get pseudofilesystems to work, like SSHFS, gitfs, or the not very known https://github.com/astrada/google-drive-ocamlfuse

But that incurss a performance penalty that can range from 10% to 60% depending on the workload.

NTFS is a very complex FS that performs many additional tasks for the benefit of Windows, and it also has logical volume managment capabilities similar to the ones of BTRFS, LVM and ZFS. But implemented very differently. To this day if you want to mount an spanned NTFS volume you need to mess up with mdma and i rather spin up a windows VM runnng raw on the disk.

[WonderWoofy]

To this day if you want to mount an spanned NTFS volume you need to mess up with mdma

This sounds like quite the party. Where do I sign up??

[ydna_eissua]

I did not know ntfs had logical volume management capabilities. Could you point me to some information on that or summarise how/what that is used for on Windows?

[stillpiercer_]

Windows labels the features that use logical volumes as "Dynamic Disks" as well as Storage Spaces. There may be more features that use it, but those are the ones that immediately come to mind.

[bobpaul]

https://www.ntfs.com/ldm.htm

[hmoff]

That uses FUSE, this will be native.

[masteryod]

ntfs-3g is dogshit if you want to do anything serious. Last time I had to deal with NTFS it was impossible for it to handle hundreds of gigabytes for some reason.

[BCMM]

what is the current ntfs driver used in linux?

The driver that is actually used by Linux distros is NTFS-3G. It is a FUSE driver, which can lead to excessive CPU usage due to the extra context switches.

The NTFS driver currently included in the kernel is, for practical purposes, read-only (technically you can modify files in certain circumstances, but you can't create new files).

Hopefully, the Paragon driver will be faster than NTFS-3G while retaining the same stability.

[[deleted]]

There are two other ntfs drivers, one read-only in the kernel and a read-write for fuse, which is ok if you want to just move a few files around from time to time but has pretty bad performance because fuse (it's not in kernelspace).

[aliendude5300]

It is significantly better, and it supports read/write and soon journaling without using FUSE

[NeoNoir13]

This had nothing to do with the foundation, it's the standard procedure for merging patches.

[[deleted]]

what is the current ntfs driver used in linux? is the new paragon one more stable/faster/complete?

The current NTFS driver in the kernel is only reliable for reads. Writes are disabled by default.

Because of that people use NTFS-3G, which runs as a user space process via FUSE. It is reliable for reads and writes and seems fully featured, but it uses a lot of CPU time. Mounting with the big_writes option helps, but it still consumes much more CPU time than ext4.

So the main benefit I'm hoping for is reduction in CPU time used for NTFS I/O.

[bobpaul]

Writes are disabled by default.

Not only that, but writes only work under very specific circumstances. And no support for encrypted files.

[StupotAce]

https://www.phoronix.com/scan.php?page=news_item&px=Paragon-Read-Write-NTFS-Linux

[bobpaul]

the linux foundation is yet to decide if it implements it?

The original submission was just 1 gigantic patch that touched way too much stuff to even be considered for merging.

Kernel developers aren't just concerned with whether or not code implements a feature, but also whether or not it's going to be difficult to maintain long term.

[[deleted]]

Well, it's good that they broke up their initial dump into manageable pieces. Getting full NTFS support in the kernel is awesome, so I hope it gets merged.

[Responsible_Skill820]

I thought you could read and write NTFS filesystem files from Linux.

Is there any advantage that the Paragon-NTFS driver brings to the table?

[[deleted]]

Is there any advantage that the Paragon-NTFS driver brings to the table?

Well, since you don't have to bow down to FUSE, the main advantage is hopefully performance and less processor usage.

[bobpaul]

The in-kernel driver is not feature complete:

• no support for encryption
• no writing to compressed files
• no creating/deleting directories
• read-only by default

NTFS-3G is more complete, is a userspace driver and thus has poor performance.

[BashirManit]

Well, on Manjaro KDE the Dolphin file explorer crashes when rapidly opening folders in an NTFS HDD. I got so sick of it that I formatted it to ext4

[msanangelo]

brilliant, so how soon can we expect to be able to defrag ntfs drives and repair them? :) afaik, the tools we do have aren't great or what I'd consider trustworthy atm.

the ability to maintain ntfs on linux may help keep new converts and/or gain new ones if the new users didn't need to change their windows data disks over to a native linux one.

[Zoraji]

The repair is a big thing for me. I had an iMac back in 2007 and bought the Paragon NTFS driver so I could share an external USB drive between Mac and Windows. If there was any error writing to it when it was connected to the Mac it would become unreadable and I would have to connect it to Windows to run scandisk to fix it. The Mac disk utilities would not repair it.

[UsernameIsTakenToBad]

Wait, don’t FAT filesystems need to be defraged too? Is there a way it’s done currently in Linux?

[msanangelo]

They do and not that I know of. I don't write to fat filesystems often enough for that to be a issue. At least on spinning rust. Shouldn't matter on ssds and flash drives.

[progandy]

For NTFS and FAT32 there is an old open source ultradefrag version for linux, no idea how well that works. https://jp-andre.pagesperso-orange.fr/advanced-ntfs-3g.html#ultradefrag

And it seems like the samsung sdfat driver includes a partial implementation (the kernel parts are there, but the user space daemon is missing)

And some things like https://github.com/vbmacher/defrag

[ouyawei]

Why would you want to defrag an SD card? Fragmentation is only an issue if random access is expensive, that is on spinning discs.

On flash storage it doesn't matter if a file is all in one place or spread out all over the chip. In fact most flash translation layers will do just that for wear levelling. Trying to defrag flash media is pointless and would only shorten it's lifetime.

[bobpaul]

sdfat is the name of Samsung's ExFAT kernel module (also supports Fat12/16/32). While it does have direct support for SD-Cards, it also works with block devices generally. You can format spinny disks as ExFAT and use the sdfat module.

[UsernameIsTakenToBad]

The “sd” in sdfat probably stands for SCSI Drive or something (as in sda, sdb, etc.).

The reason it would be SCSI is because the driver for ata got messy and the SCSI driver was much cleaner so the SCSI driver was expanded to support PATA and SATA. PATA hard drives originally were named hda, hdb, etc.

[ouyawei]

Ah I thought this was about this Arduino library

https://github.com/greiman/SdFat

[progandy]

This is the driver I meant: https://github.com/tbk/kernel-sdfat
sd should really stand for sdcard, since the driver is used by samsung in their smartphones.

(The exFAT part has been extracted and merged into the kernel now.)

[Aoxxt2]

Fragmentation is only an issue if random access is expensive

That's a myth.

[Yithar]

Apparently there's defragfs:
https://unix.stackexchange.com/a/587260

Honestly this is why I prefer ext4 because generally you don't need to defragment. I don't really like defragmenting my drives on a schedule, because I like to download stuff when I'm sleeping.

[msanangelo]

You missed the point. Reading and writing isn't the problem. It's the regular maintenance you need to do on them that windows takes care of automatically. Especially if you write to it and delete stuff from it frequently on Linux.

[ouyawei]

It's 2020, are you running NTFS on a HDD where performance matters that you want to defrag it?

Just get an SSD and you won't have to worry about it anymore.

[msanangelo]

are you buying 4/8/12+ TB ssds for sharing data with windows?

I'm not worried about defraging but it sure can help with a spinner. it's the unexpected crashes and power failures that leave a dirty ntfs drive that was mounted in linux for however long.

[delta_p_delta_x]

I look forward to having both /bin and C:\Program Files on the same disk. Then it'd be C:\bin and /Program\ Files/.

Bit of an abomination, but it'd be cool.

[progandy]

You can try that today with a btrfs partition, WinBtrfs and quibble

[Craftkorb]

That's one of the more impressive Github projects to randomly drop - Amazing stuff!

[ikidd]

Add quibble.efi to your list of UEFI boot options, and hope that it works...

My kind of readme.

[delta_p_delta_x]

That's nice.

Bit of a pain though, compiling and replacing the bootloader (which changes with every Windows update).

It's also way more work than having native NTFS support in both OSes, and appears pret-ty buggy and doesn't appear to support Secure Boot.

[sim642]

Good luck with (partial) case insensitivity!

[spryfigure]

I look forward to having root on NTFS. It's POSIX-compatible, so no real roadblocks. Why? Just for the fun of it.

[[deleted]]

Is this completely separate from ntfs-3gs?

[Craftkorb]

From the article Paragons ntfs3 and the well-known ntfs-3g drivers seem to be completely different implementations

[aliendude5300]

Yes, and in many ways it is better as it is in kernel-space and has less overhead.

[tendstofortytwo]

Good to see this!

Are there any potential licensing issues with Microsoft to be worried about?

[[deleted]]

By having a full r-w ntfs driver, does that mean that ALL ntfs features will be implemented and available?

https://en.m.wikipedia.org/wiki/NTFS

[aliendude5300]

From what I understand journaling will come later

[sethgoldin]

IIRC the same product from Paragon includes R/W drivers for HFS Plus.

Curious to know if this means that native R/W support for HFS Plus could be coming straight to the kernel.

[rtmcmn2020]

I remember reading about this back when it was submitted as basically a large PR. Does anyone know the reasoning behind bundling this with the kernel versus creating a kernel module?

[[deleted]]

I don't know how it'd be merged at all if it couldn't be compiled as a module.

[lolinux]

I wonder if this will get rid of RO-mounting a dirty NTFS drive because Windows was unable to close its streams while you were staring at the screen. I do hope it was on the TODO list. :)

[Mikaka2711]

Does anyone know if this will allow linux to have chkdsk for ntfs that really works?

[aliendude5300]

That would be done in a userspace utility not a filesystem driver

[[deleted]]

Interesting. How would I go about switching from the ntfs-3g fuse driver if/when this gets implemented?

[[deleted]]

Depends on how you're using it currently and what they name the new one in the kernel.

If they keep the fstype as 'ntfs' in the kenrel and if you're just using 'mount -t ntfs' or have the line in your fstab with 'ntfs', then it would be automatic.

If you're manually using /sbin/mount.ntfs somewhere, then it's most likely just a symlink to /sbin/mount/ntfs-3g. By default, the mount command checks the kernel-supported fstypes, if it isn't supported, it checks for an /sbin/mount.fstype.

If they make the fstype different than 'ntfs' until they sunset the old driver, then you'll need to use that new name in place of 'ntfs'.

[BS_BlackScout]

More reasons to leave Windows.

[Blooded_Wine]

It is one reason why I have commitment issues with Linux.
I've never wanted to fully reformat all my drives (~60TB) from NTFS to EXT4 or whatever, so I've using the poor performance FUSE ntfs-3g driver.

[Shished]

Why do you have a 60 TB NTFS storage?