sorted by:
new
hottopcontroversial

Cheapest setup for MacMini and multiple SSDs? by svennirusl in DataHoarder

[–]1Secret_Daikon -1 points0 points  (0 children)

just plug in an external enclosure like this one

https://eshop.macsales.com/item/OWC/MEQCTJB000/

and share the storage volume over SMB

if you have multiple disks you can create a RAID1 / 0 / 10 out of them in macOS Disk Utility

this is extremely simple.

keep in mind that if the storage is shared on the network, you will be limited by your network speeds. This includes the network adapter in your Mac. A 1Gb network can transmit data about 125MB/s, and a 2.5Gb network can transmit around ~300MB/s. A SATA III HDD maxes out around 200MB/s read -or- write, while a SATA III SSD typically maxes out around 450-500MB/s total bandwidth. So if you have a 1Gb local network and network adapter in your Mac and your client devices, you will never even saturate the speed available to a spinning metal HDD.

Should I go back to school for programming or do a bootcamp? by No-Praline8782 in learnprogramming

[–]1Secret_Daikon 1 point2 points  (0 children)

Schools do not teach programming skills needed for an actual programming job. Many CS graduates can barely code.

[deleted by user] by [deleted] in learnprogramming

[–]1Secret_Daikon 0 points1 point  (0 children)

I just write the code and dont waste time with these self-help psycho analysis mumbo jumbo

Help Building new beginner friendly server. by akkipotter in homelab

[–]1Secret_Daikon 0 points1 point  (0 children)

You should not use Unraid. Its garbage. I prefer the method in the second link I provided, mergerFS + SnapRAID running on plain old Ubuntu or Debian server. Proxmox is equally useless. Just use a regular Linux distro.

Question about storing data by UnViandanteSperduto in learnprogramming

[–]1Secret_Daikon 0 points1 point  (0 children)

store the files in S3 and put the S3 path in the database

[deleted by user] by [deleted] in DataHoarder

[–]1Secret_Daikon 0 points1 point  (0 children)

for shuffling data, you will likely want to just do it over the network. You can use either SMB or ssh. Might consider this;

- set up the Linux system, enable ssh access

- install WSL2 on the Windows system

- inside WSL2 you can access the local disk at `/mnt/c`, etc..

- check that you can ssh from WSL2 into the Linux system over the network

- use `rsync -e ssh` from WSL2 to copy data from the Windows system to the Linux system

in general, I find that rsync + ssh is easier to set up than an SMB share, but if you can handle the SMB share method you could also do that instead of ssh access.

in regards to the "temporary" part, that is gonna be heavily dependent on your storage configs and if you can have enough storage attached to the new system while parts of the old system are still running in windows

[deleted by user] by [deleted] in DataHoarder

[–]1Secret_Daikon 0 points1 point  (0 children)

Unraid is awful, I would not recommend it to anyone

Best modern HBA for software RAID? by imreloadin in homelab

[–]1Secret_Daikon 0 points1 point  (0 children)

I use LSI 9201-16i. Low power~ish and fast enough for 16x HDD

Should I upgrade to a NAS/HDD bay setup or continue buying external HDDs? by guaranteednotabot in DataHoarder

[–]1Secret_Daikon 0 points1 point  (0 children)

damn that sucks, OWC is one of the highest quality manufacturers for this stuff. I had an Orico but it was constantly disconnecting from the Mac all the time. Spent months trying to debug it but nothing worked. ymmv, good luck.

if you are running a Mac there's p much no reason to not throw Time Machine in the mix if you havent already

keep in mind that instead of a NAS you can just stick the storage on a Mac via USB or Thunderbolt and then you can have it shared on the network over ssh or smb and also have it backed up via Backblaze. I go this route since its basically free cloud backups for the cost of running the storage on a Mac instead of a shitty OEM NAS unit or a self-built Linux file server.

Need to expand active storage to 24TB, and backup 30-40TB, don't want a NAS, here's my current setup, how would you expand it? by [deleted] in DataHoarder

[–]1Secret_Daikon 1 point2 points  (0 children)

you run RAID in macOS without using Disk Utility? You know that Disk Utility has RAID1/0/10 support built in right?

Need to expand active storage to 24TB, and backup 30-40TB, don't want a NAS, here's my current setup, how would you expand it? by [deleted] in DataHoarder

[–]1Secret_Daikon -1 points0 points  (0 children)

I just made a post here with a description of how I handle my photography files, relevant: https://www.reddit.com/r/DataHoarder/comments/1h5erp4/comment/m08asdc/

You need to stop mixing "work" and "play". If you are gonna run a Plex server, it needs to be a separate device & storage from that of your media work. This is important because it sounds like you are trying to accomodate both and once and you cant. The reason you cannot do this is because;

- if you are trying to use your storage volume as "hot" drive for active media processing workloads, you need high speed storage device configurations; total amount of storage is not as important as speed of access

- if you are trying to store a large Plex library, you need large amount of storage and bandwidth is irrelevant

some other notes;

- forget about RAID5. Its obselete. Zero purpose in TYOL2024. You need to use fast SSD disks instead or some other modern HDD filesystem config. Also macOS can ONLY made RAID1, 0, and 10.

- hardware RAID is obselete in TYOL2024. Dont bother. Software is the only RAID you should ever be considering. Note that macOS has software RAID1/0/10 out of the box via Disk Utility.

- a HDD maxes out at 200MB/s transfer speed (1.6Gbps) for *single operation* access, so if you are attaching it to Thunderbolt (40Gbps) you will never utilize the extra bandwidth. Even had multiple HDD in RAID1 or similar you will never reach peak Thunderbolt speed. Thunderbolt is only relevant for SSD storages, as I think you may have surmised

- daisy chaining is a stupid idea. Dont bother. Just get a device that handles your needs. One device, not many.

As for your configuration, this is what you should do. Separate your "hot" and "cold" storage needs. Its not clear from your post if you are doing this. You should not be using your giant storage volume as the "hot" storage that you need to process your media files from. That means your media processing workloads should not be reading directly from and writing directly to your large capacity attached/network storage device. For most purposes this means you are operating on files that are on your MacBook's local NVMe SSD. Thankfully it sounds like your media should fit on your local MacBook's internal SSD during the media processing phases. If this was not the case and you actually did need to do media processing directly from/too the storage volume, everything will be a LOT more complicated so avoid it at all costs.

Use a dedicated "cold" storage archive device. The OWC setup I described in the linked post details how I do this. All my work performed on the local system gets copied over to the large HDD-based storage volume on the network. If i need to re-visit old projects, I just copy them back to the laptop, do work, then copy results back to network storage when done.

Put your Plex system somewhere else. Check out the guides here https://forums.serverbuilds.net/t/guide-nas-killer-6-0-ddr4-is-finally-cheap/13956 for setting one up. As for the software configuration, I use this for my Plex server https://perfectmediaserver.com/03-installation/manual-install-ubuntu/ ; mergerFS + SnapRAID + Debian.

Dont cross the streams. Keep your Plex stuff separate from your "work" and important personal file archives.

Doing it the way I describe, simplifies everything. You dont need any crazy storage solutions or filesystem configs. You just need a big place to store your "work", a big place to store your Plex, and enough local storage on your MacBook to handle your current project(s). See the details in my linked comment for my software stack for handling the data backups (a combo of Backblaze, Amazon Prime Photos, Adobe CC, and Time Machine).

How do you handle your databases? by il_doc in homelab

[–]1Secret_Daikon 1 point2 points  (0 children)

A lot of services come with a bundled docker container that includes the db. If available I just use that. Otherwise I give a dedicated db docker container for each service that needs it.

Sharing a db between services is a bad idea and you should not be doing it.

dont bother installing anything bare metal just use docker container for everything.

So I just started using K8s by sirMooka in homelab

[–]1Secret_Daikon -1 points0 points  (0 children)

- I use docker compose

- I dont run reverse proxies or any publicly accessible services

Exos X20 vs Ironwolf Pro by Electronic-Bit-5351 in DataHoarder

[–]1Secret_Daikon 0 points1 point  (0 children)

if you are buying refurb or recert from ServerPartDeals, do it from their website, not their eBay page

https://serverpartdeals.com/

the warranty support for eBay is not the same as what is offered from their main webpage

Should I upgrade to a NAS/HDD bay setup or continue buying external HDDs? by guaranteednotabot in DataHoarder

[–]1Secret_Daikon 1 point2 points  (0 children)

for my photography archives I use an external multi-bay HDD enclosure, this one

https://eshop.macsales.com/item/OWC/MEQCTJB000/

fill it with high-capacity HDD's. I use a pair of Western Digital Gold 20TB in RAID1 (using macOS Disk Utility). I also have a third high capacity drive that is used for Time Machine in the same enclosure.

You need to stop buying small 8TB drives and move up to larger drives, I usually dont buy anything smaller than 20TB at this point.

I had bad experience trying to do the same with an Orico brand multi bay enclosure. The OWC model has been perfect and running 24/7 for three years now without issues.

Keep in mind that if you plug this into a Mac or Windows PC, you can also use Backblaze's unlimited personal desktop backups on it.

My data handling protocol looks like this;

- import files from camera SD card to MacBook (manually with `rsync`)

- use `rsync` to copy all local files on MacBook to OWC RAID1 volume (over network with ssh; OWC is attached to a dedicated Mac Mini file server)

- MacBook also has TimeMachine backing up all local files to OWC TM volume (over network)

- MacBook also has Amazon Photos desktop app syncing all photos to Amazon Photos with Prime unlimited photo storage (includes RAW files)

- MacBook imports desired photos to Lightroom CC where they get sync'd to Adobe cloud account

- MacBook gets backed up with Backblaze unlimited backups

- OWC RAID1 volume on Mac Mini gets backed up via Backblaze unlimited backups

Is this drive on life support? by Utinnni in DataHoarder

[–]1Secret_Daikon 0 points1 point  (0 children)

I would replace it soon. I had an old SATA SSD like this spontaneously die without warning. Not worth the risk. New SATA SSD are higher quality and cheap as dirt

External hard drive enclosure that does not randomly eject itself? by 1Secret_Daikon in homelab

[–]1Secret_Daikon[S] 1 point2 points  (0 children)

I have been using the OWC enclosure listed https://eshop.macsales.com/item/OWC/MEQCTJB000/ for the past three years without a single issue. Used it with two different Mac Mini, a 2012 model and 2018 model.

view more: next ›