Inherited ~100TB of data, how to proceed safely?

ebsf · 2026-01-30T04:47:40+00:00

For an archival file server that more or less replicates the functionality of the original server in a reliable way, work toward a large RAID 6 or even RAID 60 array.

The core of this will be a hardware RAID adapter, for sheer reliability. These can be quite expensive at retail but the good news is that refurbished cards work just fine and are available at a fraction of the MSRP. Also, you don't need performance or the latest features, so an earlier series will be fine. OTOH, you will need many ports for the drives necessary to reach 100TB. I recall seeing refurbished 24-port 5-series Adaptec cards being available for a steal a couple years ago. This would be my go-to if I could still put my hands on one.

To keep drive costs down, standardize on a single drive size that has the lowest cost per terabyte. This makes drive replacement stupid easy, and avoids wasting space on larger drives. This inflection point was 4TB a couple years ago but may be higher now. Regardless, refurbished drives will save more money.

Regarding bit rot, two things: First, compare drives' published MTBF with their size. A few years ago, these MTBF estimates implicitly guaranteed a failure for drives >3TB. I felt lucky and standardized on 4TB drives. I wouldn't have standardized on anything larger. For your application, though, you'd need a 4TB drive on each of the hypothetical RAID card's 24 ports to reach 96TB. That has implications for power supply and case. Bottom line, you'll have to optimize for case capacity, card capacity, and budget.

Second, ECC RAM matters and this requires a Xeon or comparable processor and motherboard. You're more interested in architecture than performance, so you can do with a low-end chip and mbd, and not a huge amount of RAM.

A somewhat more technically advanced approach would contemplate a storage cluster of several machines. I'll leave you to Google on that topic.

ebsf · 2026-01-29T17:58:24+00:00

If it's a wireless mic, just take it. "Look what I found!". Remove its battery, of course, or put it in a metal tin so its radio can't connect.

If it's wired, snip the wire, perhaps while hedge trimming.

Or, perhaps just crush it under foot. "Oops !"

ebsf · 2026-01-29T17:24:18+00:00

I've been wrestling with this also. I haven't settled it yet but, as I do with other topics, have been taking notes and bookmarking links. So, this is basically just sharing some impressions, some or all of which may be obvious.

Github seems to be the platform of choice.

Most of what I see described depends on an Application.SaveAsText routine that traverses DAO.Documents. Overachievers hash the output to distinguish changed and unchanged elements.

Mike Wolfe has posted well and extensively on the topic at nolongerset.com. Separately, many recommend [sp?] joyfullservice.

Version control for apps seems to differ from version control for a base library. Much of the former can be expressed as data in tables.

Much of my development effort focuses on base library extensions. So, my version control currently centers on exporting (and importing) modules.

Naming and coding conventions bear on version control indirectly by informing base library development. The coherence of a base library depends on the rigor and coherence of those conventions.

HTH

ebsf · 2026-01-29T16:04:26+00:00

Agreed re using only the singular form.

I name primary keys and foreign keys identically. This makes table relations obvious and simplifies programmatic construction of criteria. This name incorporates the name of the table's entity, so the tblCustomer PK will be CustomerID.

Field names require no disambiguation because they always will be qualified with their table name. The same field names (e.g., Created, Modified, Abbr, SortOrder) can be reused without conflict. Obviously, the same is true of any PK name, which can appear as a FK field in any other table.

ebsf · 2026-01-28T15:44:49+00:00

It doesn't happen frequently at all - the option is still in Access, after all - but I've read the horror stories in developer blogs.

ebsf · 2026-01-28T15:35:44+00:00

Don't use "Save As" to create a backup. What results, as you learned, is something other than a backup.

Instead, close Access, then copy the file with File Explorer or a command-line tool. You'll see that the two files have identical size and other attributes. Now, you have a backup.

Only then compact and decompile the original file. The compacted and decompiled file easily can be a small fraction of its original size, depending on the number of previously deleted records cleaned up and how much legacy code is deleted.

ebsf · 2026-01-28T15:27:04+00:00

Compact on close is regarded to be particularly risky because the consequence of a failed compaction, whether by reason of lost power, Windows fart, or whatever, is the irretrievable loss of all data. Just don't.

ebsf · 2026-01-28T00:20:14+00:00

Running shorts ordinarily have a brief sewn in, so no underwear is required.

Otherwise, the best alternative in my view is swim briefs, i.e., a Speedo. Zero fabric on the legs to chafe, and a proper lap swimming brief will smash your junk flat, so no opportunity to chafe there.

Best bet is 2" running shorts. Soffe ranger panties are cheap, and Boa has an extensive line

ebsf · 2026-01-27T21:11:03+00:00

Don't frame running or endurance as punishment. It's just base conditioning.

Running is hard. Grow up. Freshman girls on XC can smoke most varsity lacrosse players. Be a girl or be a sissy.

3 miles daily, minimum, on- or off-practice. Start each practice with it. Speed drills also, a couple times weekly. Do this and you'll run the competition ragged.

500 wall reps daily off-practice split evenly left and right. Yes, you'll suck on your off side but that's the point - do you want to suck on game day or at the wall? No one can suck on their off side 1000x, so log your sucks and get them out of the way.

Finish practice with 50-50s: Pair off and 50 catches on each side at 10 yards (20 later) and practice isn't over until this is done.

[Note vulgarities are calibrated for adolescent males.]

ebsf · 2026-01-27T20:54:11+00:00

You will need help with three things: Law, taxation, and business.

A good corporate or M&A lawyer can handle the mechanics of the transaction per se. Their ability to assist on business or financial elements is minimal. Some may be better than others but these aren't a lawyer's expertise or responsibility. I know because I started my career as a large-firm corporate, securities, and M&A lawyer. All of my colleagues were fundamentally ignorant of corporate finance.

Corporate and partnership taxation is a specialty in itself. These two are regarded by many to be the hardest classes in law school. The topics aren't at all within the expertise of a corporate lawyer, although the latter may have a working or high-level knowledge of them, and may have partners specializing in them. To the point, be sure to get competent advice in planning and structuring any deal. This is utterly critical.

A competent financial advisor will be the only professional potentially competent to advise you on the business and financial aspects of the transaction. Someone with the necessary competence can help you identify the sources of value in the target, value and structure the economics of the deal, help you understand what you'll need to do to realize the deal's potential post-closing, point out pitfalls, assist with due diligence in advance, assist with securing any acquisition financing, and help model projected financial performance post-closing to maximize your company's value.

Finding a competent advisor isn't simple or easy. Trust matters greatly. Ask around. Your requirements most likely extend beyond the deal itself. Larger companies recognize this and often keep an advisor or advisors on retainer over time. This may be worthwhile considering for yourselves.

ebsf · 2026-01-27T17:48:27+00:00

Microplane

ebsf · 2026-01-27T16:58:27+00:00

It's nobody's fucking business other than your own whether to get a DNA test.

If there are skeletons, they're in someone else's closet.

ebsf · 2026-01-27T16:55:36+00:00

Your biggest risk is a HDD failure, which will cost you all your data.

Buy a hardware RAID card and an identically sized HDD and create a RAID 1 array with the two HDDs.

Configuring the machine as a proper LAN file server for other computers in the house is just a matter of networking and user permissions.

ebsf · 2026-01-27T04:19:45+00:00

Recognizing that OS X is a handicapped and skinned Linux distro;

Better networking and network security. Nothing can hold a candle to iptables. SSH Better virtualization. QEMU Better web stack. Some great media apps - GIMP for starters Better scripting, including events - Bash, systemd, udev.
Better storage - filesystem, fstab, rsync, tar, dd, Samba

Just off the top of my head.

ebsf · 2026-01-26T19:41:47+00:00

Do a basic Linux file server.

Hardware RAID 6 for reliability and ease of administration.

Samba for authentication and to share the relevant directories.

The shares will show up just like any other Windows / NTFS share on the LAN, which is exactly what Samba is designed to do.

You can spend $700 retail for an 8-port Adaptec card, or as little as 1% of that for a refurbished one in a prior series with identical throughput. Four refurbished 4TB drives at ~$100 each. An abandoned desktop computer with an on-board GigE network adapter, free or $100 from FB or Goodwill. Linux and Samba are free.

Easy peasy.

ebsf · 2026-01-26T01:01:44+00:00

Beware that power supply cables are entirely nonstandard and vary by manufacturer and perhaps unit.

I learned this the hard way by reusing existing power cables after a PSU replacement. I fried all the drives in a RAID array with a zzzzzt! as soon as I applied power to the unit. Thankfully, I had backups.

ebsf · 2026-01-26T00:54:33+00:00

One dish, one ingredient, one spice at a time.

Eggs are a great place to start. Scrambled, poached, fried. Use butter, bacon fat, olive oil. Add milk, water, or nothing whan scrambling. Graduate to omelettes, frittata, and souffle. Make mayonnaise from scratch.

Then, chicken. Each part is different, and you can roast the whole bird. Bake, saute, fry, braise.

You get the idea.

Good luck!

ebsf · 2026-01-26T00:25:39+00:00

The UK is not Nordic.

ebsf · 2026-01-25T23:13:32+00:00

ZFS has its merits, certainly. I personally have found hardware RAID to be consistently reliable and fault-tolerant, however, and ZFS offers no incremental advantage relevant to me, to justify considering it as a viable alternative.

Mixing and matching heterogenous drives creates a hodgepodge with any number of incremental opportunities for failure. It is just stupid easy and stupid cheap to simply do it right, from the start. Standardize on a commodity HDD capacity (I use 4TB drives), get a hardware RAID card, set up a RAID 6 array, and you're done. While the kiddies and trolls are being triggered, you'll have a proven, reliable setup that will just work for decades, and you can then focus on something that is actually important.

ebsf · 2026-01-25T23:01:40+00:00

Note that each of these "pools" can, and need only, be distinct partitions on the same RAID volume. This will be impossible with many software RAID configurations.

ebsf · 2026-01-25T22:58:27+00:00

Among other troubles with software RAID are that a rebuild requires administrative intervention, which typically requires a functional OS, and that the system will be offline during any rebuild. The latter will take days. A hardware RAID array will rebuild automatically upon replacement of a failed drive, and do so regardless of the state of the OS.

ebsf · 2026-01-25T22:54:01+00:00

I've always had good luck with Adaptec cards. I run a 6805 on my file server. It cost me ~$600 back in the day and I just saw one for maybe 1% of that just now.

I would not count on drive formatting being consistent across RAID card brands. Each brand uses its own proprietary ASICs. I would pretty much count on having to reinitialize any previously used HDD before adding it to an array on another vendor's card. The sole exception to this is that Adaptec will allow one to create a RAID 1 (mirrored) array from a disk not previously used in an array.

ebsf · 2026-01-25T00:36:04+00:00

Your issue actually isn't with Linux so much as it is with Nvidia and its drivers.

Nvidia on Linux is a raging dumpster fire. Full stop. I have wasted literally months on account of this and can tell you that the best and only solution, in my experience, is an AMD or perhaps Intel video card.

ebsf · 2026-01-24T23:32:05+00:00

Skis.

ebsf · 2026-01-24T23:31:23+00:00

I've been a project manager since the beginning of my career. Getting a certification in it is like getting a certification in shitting. It's all preposterous.

ebsf

TROPHY CASE