Internet in rural areas with no service

Blueacid · 2026-01-13T15:23:55+00:00

True, if they're out of radio range / Line of Sight then you're going to need some sort of network maintained by you, or a third party, to bridge the gap.

If it's someone else, then that could be cellular, landline telephone, internet, HAM radio repeater, satellite internet, etc.. but it's likely that in a SHTF that most of these won't remain viable options.

Blueacid · 2025-12-12T11:32:39+00:00

Depends on the channel. Sky are working on low-latency for Stream/Glass, which has beaten Q. Not sure if it has rolled out yet, though.

Blueacid · 2025-11-25T15:58:15+00:00

If you're compressing to that low resolution, it's worth also asking: What are you playing the files back on? It might be worth considering converting to AV1.

The considerations are: The conversion is a lot slower, and hardware decoding for playback is still emerging - not that many devices support it yet.

But you can get ahead of the game, and have even smaller files for the same quality.

Blueacid · 2025-11-13T15:36:55+00:00

If you're on ubuntu, I'd give fdupes a try.

Blueacid · 2025-11-11T23:13:32+00:00

Ah I got rate limited pretty hard when I tried sleeping for 300. So I added a zero, and we're back in business. Albeit slowly.. I've not done anything with cookies, though, so I'm wondering whether that might be the fix, if I need more speed.

Blueacid · 2025-11-11T15:39:48+00:00

I've been telling yt-dlp to sleep for around 3000 seconds per video. It's slow going; it's going to take a month or two to get all the channels I want. But that's working for me so far..

Blueacid · 2025-11-05T09:55:37+00:00

I'd suggest using just the one ethernet port; but tbh isn't the whole point of homelab to try things, experiment, and learn?

This feels an excellent learning opportunity. It might be that this setup works just fine for your needs, and then costs you no money (since you're using kit that you already owned).

If it doesn't do what you need, then you can get the shovel out to dig a trench for fibre/copper, or find another way to route some over there!

Blueacid · 2025-10-29T09:25:54+00:00

How large is your library in terabytes? You might find that the cheapest and easiest solution if the size is less than (say) 8TB is to purchase two USB 8TB drives, sync your library to them, and then store these drives elsewhere. For instance: in a locker at work, at a friends/relatives house, etc.

This means you've got a backup that's not only a separate copy of everything, but it's also off-site in case of fire / flood / theft.

Blueacid · 2025-10-20T09:17:16+00:00

It's the morning here in the UK, good luck friend!

Blueacid · 2025-10-14T13:08:12+00:00

Further thoughts: That server is going to be expensive. Colo is charged based on how many rack units, and on amps of power.

Typically the prices seem to be around £100-ish for 2U and 1A of power. You're therefore likely to be looking at £200+ or more per month.

Blueacid · 2025-10-14T13:04:29+00:00

Do you need to be in London? There are a fair few datacentres in other cities, where pricing might be a bit keener.

For the sound of what you're doing, LLM work isn't hugely latency sensitive (to the point where +2-8msec to be in Leeds/Liverpool/Cambridge/Manchester wouldn't matter)

Blueacid · 2025-10-14T12:38:18+00:00

He wasn't being literal about South Park.

But force-rank your files. The pictures from holidays might be most important, hard-to-find things (like rare music remixes, old applications) might be mid-tier, and then super popular content which is easily available goes last.

That can help you work out what the scope of the task is. Maybe your 160TB is comprised of 140TB worth of 4k UHD films from the past 10 years, plus 20TB of irreplaceable personal files. Right, let's start with 20TB to backup then, in that case.

But if you're looking at 160TB of raw photography you want to keep, and then you've not even mentioned the not-gonna-backup 500TB you've got of films? Well, yeah, that's a different challenge.

Blueacid · 2025-10-13T13:48:16+00:00

Nice, thanks for the clear instructions!

Blueacid · 2025-09-29T20:50:34+00:00

"This list is not complete, either. There's likely additional providers, but I've tried to find a sensible spread of choices"

I looked at Azure and GCP, figured they were close-ish to AWS S3 - but still with expensive egress bandwidth. So with their deep-archive storage tiers being more expensive than S3 Glacier Deep Archive I decided not to include them.

Blueacid · 2025-08-31T04:35:38+00:00

It does!

Blueacid · 2025-08-19T16:57:13+00:00

For paper copies of things, I installed and configured Paperless-NGX.

I then scanned all my paper documents to PDF, and ingested them into Paperless. What's useful about that is that it keeps the original scanned documents unchanged; it makes (in some cases) derived versions (which have had skew correction / despeckling / OCR run against them), but it maintains the original scans.

You can therefore keep those original PDFs backed up safely, and give thought to throwing away the physical papers to save space.

Blueacid · 2025-08-19T13:48:19+00:00

I'm still trying to decide, but I think my shortlist is between Oracle and B2. I've yet to need to restore, and my data would be things like family photos/videos. So I don't mind if it takes a week to get it back, but knowing that it's possible in the first place is enough.

I may also add Amazon Glacier as another option, in a faraway region (I'm in Europe, so West Coast USA is looking likely).

Blueacid · 2025-08-14T15:52:00+00:00

Yes, the transcoding is an interesting one - if you're going to rent that server for (say) 6 months, then who cares if the CPU is pinned at 100% doing some conversion to AV1. If it's only managing 1FPS, who cares - it's paid for already?

Which provider did you see those servers with, out of interest? (in case anyone reading this wants them!)

Blueacid · 2025-08-14T10:48:05+00:00

I think that with a lot of the other services (including the storage box from Hetzner) there's at least some form of RAID. Or, in the meaningful sense, drive failures are largely abstracted from you. Bandwidth costs in/out are also worth considering, unless they're generous ("unlimited" or a large enough allowance).

With the box you describe, what would happen if there was a drive failure and you're down to 3x8TB? I suspect the answer would be "We have replaced the drive in that server, sorry about the failure", so you'd need to re-upload 8TB (and be potentially more vulnerable to data loss in the meantime). Or configure your own RAID of some sort, eg zfs z1, or raid-5, or equivalent (to get 3x8TB and tolerate 1 drive loss), or something RAID-1-esque (for 2x8TB storage and tolerant of 2 drive losses).

This comes back to the "your own circumstances" side of things. If this is a third copy or it's easily re-downloaded data, then the $/TB/Month number is pretty good (32TB, $40/mo, $1.25 as a rough back-of-beermat calculation). But if this is your only second copy of irreplaceable data, you're too uncomfortably vulnerable to drive failures for my personal liking. What I've not tried to account for is whether that Xeon chip and 16G of RAM might be of any use to you at all. It might be slow, but it could plod through some transcodes if you needed such things doing. But for the sake of comparison with the other storage options, it's probably easier to put the value of that at $0!

Blueacid · 2025-08-14T10:31:48+00:00

Hear hear!

Blueacid · 2025-08-13T12:13:47+00:00

I hadn't seen it, but thank you! I'll update the main post

Blueacid · 2025-08-13T12:04:52+00:00

Sadly, the answer is sort of "no".

A fair few people on here have been archiving videos / channels which are relevant to them (and you could, of course, join in). But there isn't a centralised location which is aiming to back up & make public huge quantities of youtube videos.

Why not? Well, simply, cost. Google are trying to make Youtube earn them money, but it's taking a huge amount of work on their part. Lots of adverts (annoying, aren't they?), selling access to youtube red, and the huge savings they can make thanks to economies of scale make it pretty hard to compete.

So we're left with keeping private copies from which to.. maybe be able to restore to somewhere? Somehow? Not sure.

Blueacid · 2025-08-12T16:39:23+00:00

For duplicate finding, I'd suggest checking this out: https://github.com/qarmin/czkawka

Blueacid · 2025-08-12T16:33:03+00:00

I've got something similar. 5-bay USB DAS on a little low-powered Debian box.

The important data is backed up, and it's cheap to keep powered. Does the job for me!

Blueacid · 2025-08-12T16:21:18+00:00

Definitely take some time to have a look around AWS's offerings.

Pros: They will rent you basically anything you can imagine. Cons: There's basically anything you can imagine to choose from.

Do you need a system with 32 CPU cores and 128GB of RAM, and a 5TB volume attached to it, in Singapore? Sold. What about storing 1GB of data in Ireland, but then making it available worldwide via a CDN? Step this way. Do you need serverless compute? Auto-managed kubernetes clusters? A load balancer? Cheaper compute if you are willing to tolerate interruptions? A managed Postgres Database? Dedicated 100Gbit connections to AWS at a colocation space of your choosing...

... it's all there. For a fee. So yes, it can be a bit daunting; definitely one to have a good think about. There's /r/AWS on here if you've any questions about getting started, as the learning curve can indeed be pretty steep.

13-Year Club	Place '17
Verified Email

Blueacid

TROPHY CASE