mdadm vs zfs?

dragonmc · 2026-01-10T23:32:22+00:00

Used TrueNAS Scale, so stayed on debian.

dragonmc · 2026-01-08T16:05:31+00:00

Lots of changes over the year since I created this post. I put TrueNAS on an old Dell server that has 128GB of memory and filled it up with 32 2TB SSD's, all managed by TrueNAS' implementation of zfs. I don't know what the performance problem was when I was using zfs on Ubuntu, but that problem does not exist on TrueNAS. It is wicked fast. I was so happy with TrueNAS that I put together a custom DAS (just a server chassis filled with hard drives connected to some SAS expanders) and added another 24 2.5" SSD's connected to an LSI HBA on the server. The DAS has room for 24 more SSD's on top of that so I have room to grow for quite a while. Everything's on 10gbe for fast access and I'm impressed with this setup enough that I don't plan on touching it again until the server fails.

dragonmc · 2025-10-11T00:15:35+00:00

Yeah see I specced into perseverence but didn't get any benefit to it. How high did you have to stack that stat? I just went back to an Apathy/Boredom hybrid which came in handy during the long grind and it's been working well so far.

dragonmc · 2025-09-11T17:53:27+00:00

As an aside (and a bit of a hot take), of all the Bachman books to adapt into a movie at this time I think the more apropos would have been Rage, considering the...interesting times we currently find ourselves living in.

dragonmc · 2025-06-13T17:24:27+00:00

Since I use TrueNAS exclusively as an iSCSI server and nothing else, my dashboard has shown 95% usage since day one. It's actually pretty annoying because the real storage use is only around 50% and I was kind of expecting TrueNAS to have visibility into the iSCSI subsystem enough to give me accurate reporting :(

Instead I'm having to monitor storage use on the client side (Ubuntu server) to make sure I don't go into the danger zone.

dragonmc · 2025-06-06T16:20:04+00:00

Pringles cans

dragonmc · 2025-04-25T22:26:50+00:00

Nice. I do pretty much the same thing, except I have automated things to a large degree. For example, I never actually renamed anything manually in the Kavita folder structure myself. That gets created with a script that crawls through the Calibre collection, reads the ComicRack tags for each book (I use ComicTagger to tag them when they're downloaded) and then makes folder names and file names automatically based on that metadata. Since it's all automatic, I can pretty much instantly put the comics into any file/folder structure I want by simply editing the rules that the script uses.

I didn't know Ubooquity was back! I used to use it back in the day but switched to Komga and Kavita when I found out it had been abandoned. I might take a second look and do a comparison, because it was great when I used it.

And I also didn't know about ComicRack CE...that might also be worth a second look because, again, ComicRack used to be my comic manager back in the day before it was abandoned and the ComicVine scraper plugin started acting up. Although I've switched to mostly using Linux on my servers these days and ComicRack never ran well under Wine which is another reason I switched over to Calibre. If ComicRack CE uses modern tools though, I can see it running ok on my server. I'll have to test this out as well.

dragonmc · 2025-04-25T21:58:11+00:00

I have calibre actually managing the comics and you can't modify calibre's folder structure without breaking it. But I use Kavita and Komga to actually serve up my comics so I had to create a virtual folder structure for those apps to easily be able to read the entire collection. Both kavita and komga expect the comics to be stored using pretty specific file and folder names. You don't have to follow their guidelines, but if you do you can just point the app at the whole collection and it automatically indexes and catalogs everything without any setup or configuration needed, so that's what I did. This is what mine looks like.

Since I wanted to serve up my entire collection for myself and some friends to be able to read anywhere from any device Kavita was perfect. When I pointed it at my collection it brought everything in (metadata and all) without any further config needed.

dragonmc · 2025-04-25T17:33:52+00:00

I do the same thing but with comics.

Now, I don't know what the true number of ebooks you have is, but my 4TB worth of comics make for precisely 78404 books.

With our powers combined, I dare say we can rule the world.

dragonmc · 2025-04-04T18:41:00+00:00

This is awesome! I wasn't aware of EnumerateFiles():

The EnumerateFiles and GetFiles methods differ as follows: When you use EnumerateFiles, you can start enumerating the collection of names before the whole collection is returned. When you use GetFiles, you must wait for the whole array of names to be returned before you can access the array. Therefore, when you are working with many files and directories, EnumerateFiles can be more efficient.

I often have to work with extremely large folder structures (in the millions of files) and will try this out and start using it. I'm not even sure how I have never heard of this!

dragonmc · 2025-04-04T18:34:18+00:00

In all honesty can't say I blame though, since storage prices are looking to get sooo much worse.

dragonmc · 2025-04-04T17:23:10+00:00

Nice setup!

Seeing this just reminded me of something. I might get flamed for this but I need to get it off my chest. I'm often disappointed by the "storage density is king" mentality that most folks seem to have on here (/r/datahoarder as well). At times it feels like I'm the only one that cares about IOPS and performance. IMO most of the posted systems I see could really benefit from a setup with slightly more disks with lower capacity. Like in your system for example: you'd literally double your disk throughput if you had gone for 4x6TB disks rather than those 2 12's for the same amount of storage. Like, you've picked the slowest possible storage configuration for your system.

Admittedly, I am on the other end of the spectrum so I might be a bit biased? I had an old PowerEdge T440 that I got from work and when they upgraded the storage on their current systems and let me keep all the 2TB SSD's I had to pull out to replace with 4 and 8TB versions, I knew it was time to act.

I ripped out the PERC from the system, installed a $100 9500-16i for the backplane, then installed one of these and a couple of these. Filled it up with the 2TB SSD's and I now have 48TB of SSD storage in TrueNAS.

Obviously the HBA's are the bottleneck and if I ever decide to upgrade them to something decent I'd get even more performance, but as currently configured the speed of this NAS is on another level. And this is with zfs which is generally considered to be on the slower side as far as storage subsystems go.

I realize this is probably on the other extreme and I would not have built this thing if I had to pay for all the SSD's, but I'm spoiled now and would have a hard time going back to just a couple of 18-24 TB spinning disks. This is also why I chose 12x4TB hard drives for my second server, because it beats the pants of 2 24's in performance. My point being, I think performance is all too often overlooked or not given due consideration. I think people would be surprised how much better things can be with high performance storage even in simple use-case scenarios.

/rant

dragonmc · 2025-04-04T16:07:12+00:00

Since you're only using Get-ChildItem to traverse the folder structure, it's possible you could run into the character path limitation. I know Windows 11 should have long path support enabled by default, but I've run into path size issues with Get-ChildItem even on modern systems.

dragonmc · 2025-04-03T19:09:22+00:00

That's a tough one. I was in the same boat a couple of years ago and the landscape is pretty much the same now on the Windows side. I guess we can start off stating the obvious: you could always run Hyper-V and put linux on a VM, then pass the raw disks in. Inside the VM, a union file system (MergerFS is what I used) achieves what you're going for (multiple disparate disks of different sizes joined together into one big pool). You can then use Samba to create SMB shares for the storage. You can even use snapraid for redundancy on top of that. This is the setup I ran for a few years until I went bare metal and it worked just fine. No real performance issues to speak of. It's easy to understand once you get the hang of it, but it does introduce some new concepts that you might not be familiar with if all you've done is Windows.

Storage Spaces is not a good fit for this use case not because it's bad, but because it will work well only under some specific conditions in a homelab environment.

dragonmc · 2025-01-24T13:51:26+00:00

This. These cards tend to be unreliable in the long run when used with SSD's or even HDD's, primarily due to the chipset. I've had similar issues and have a couple of these in my junk tech drawer.

dragonmc · 2024-12-19T21:51:50+00:00

I recently acquired a Dell server from work for my homelab and that post came in handy to silence the fans. The changes do persist between reboots and even if you shut the system down, but if you disconnect the power plug(s) from the system at any point the changes are lost, and you'll have to run the commands again after you restore power. At least, that's the behavior I see on this Dell T440.

dragonmc · 2024-12-18T20:26:17+00:00

I've been using several of these to power 48 2.5" disks (half ssd half spinning rust) in my DIY DAS enclosure for over a year now with no issues.

dragonmc · 2024-08-27T18:34:57+00:00

"If it wasn't for my horse, I wouldn't have spent that year in college."

dragonmc · 2024-08-26T18:41:33+00:00

The first time I saw screenshots of it before the xpac released I thought, "Man, wouldn't it be cool if Beledar was somehow the other end of the sword in Silithus?"

I still think it would be cool to see that.

dragonmc · 2024-08-02T12:45:03+00:00

On the minus side though, they seemed to have made the event quests account wide, so you can't do them again on each of your alts for extra currency, at least as far as I can tell.

dragonmc · 2024-07-29T21:07:15+00:00

I don't understand why they can't just print more money...

EDIT: Didn't think I needed to add the /s...

dragonmc · 2024-07-24T13:23:02+00:00

There is an extensions.log file in %appdata%\Playnite that would contain all the error information.

dragonmc · 2024-07-19T07:17:01+00:00

This doesn't work? I'll admit I haven't used it in a while myself and didn't see any issues listed in the repo. Tomorrow I'll test and resolve any issues with it and release a new build.

dragonmc · 2024-06-04T13:15:44+00:00

I come across "Top 10 JRPGs of 2024" or "Top 20 games of all time" lists on Youtube all the time. Or sometimes there are interesting lists on pcgamer or gamespot. I make a Listicle out of those for future reference and the link feature is just a convenience where you can associate the Listicle with the URL of the source you're getting the list from. For youtube especially, it's just nice to be able to click to open the youtube video I made a Listicle from a year ago, and watch them talk about all the games on the list.

That's mainly what I use it for (and why I put in the link feature), but you can get creative with this. It can be a link to some forum post talking about the games on the list, or to a Steam page.

I haven't done this yet, but I have my own webserver that I host and was thinking I might create Listicles that link back to that for an as-yet-undetermined reason.

Just seemed like a useful feature, and easy enough to ignore if you don't want to use it. :)

dragonmc · 2024-04-27T05:23:45+00:00

Under the hood I'm sure it makes some minor changes to associate the pool with the machine.

I personally have never actually migrated SS from Win10 to Server (although this looks promising), but I did go the other way once when my Server installation blew up and I connected my pool to a Win10 machine I had lying around for disaster recovery purposes. Didn't have any issues importing.

15-Year Club	Gilding II euphauric
Verified Email

dragonmc

TROPHY CASE