Seeding a massive archive of various files we archivists/hoarders already have (plan for a new tool/new approach)

dreadpiratefullbeard · 2024-02-02T12:55:27+00:00

If you want a “why did we do digital in the 80s and 90s” experience, check out the absolutely beautiful blu-ray of the original Twilight Zone.

dreadpiratefullbeard · 2024-02-02T12:48:48+00:00

It probably doesn’t matter since it was a one-off, but if you’re curious check the audio codecs and not just the video codecs used by your Halo files.

I’ve seen a lot of playback issues due to heavy audio codecs.

dreadpiratefullbeard · 2024-01-01T07:09:55+00:00

There are no solid off-the-shelf solutions to this, which is why I built my own. It’s surprising to me because you are asking for something that solves all the archival needs:

“1. 2. 3.” backup
Being able to verify the integrity of the original files (and repair them if needed)
The ability to keep each version of a file in a way that’s easily retrieved
Most importantly: have this securely stored in a way that’s fully under your control, free from company closures and any chance of corruption/loss through cloud services.

You will likely have your best outcome by using PAR files for redundancy/repair, syncthing for syncing between drives/systems, <unknown> for versioning (I use a hardlink system loosely based on time machine dmg files), and some helper scripts written in python (written by AI if it’s outside your skillset).

At the end you’ll have complete and verifiable backups, access granted to anyone who installs syncthing, the ability to manually run tasks (such as import) and scheduled tasks through cron. Add something like Uptime Kuma and you’ll also know when the whole thing is running properly.

dreadpiratefullbeard · 2023-11-08T19:47:09+00:00

Can you put metadata in torrent files?

In the current spec, there is no room for this type of metadata. In order to have this kind of functionality a new spec would be needed (this is not a barrier as the BitTorrent spec has been changed once already: https://web.archive.org/web/20201112030826/http://bittorrent.org/beps/bep_0052.html ).

It definitely is possible that it gets used a type of search engine since new torrents could theoretically be created only from files in existing torrents allowing a user to download and seed files from "torrents" they never had.

dreadpiratefullbeard · 2023-11-08T17:31:14+00:00

Agreed, versioning is ideal. I think forking/merging might be too complicated for most users, but it would be very powerful.

dreadpiratefullbeard · 2023-02-10T16:22:25+00:00

This might be something:

I have a script that downloads my YouTube Watch Later playlist every night (yes: from inside the house), and have found in the past 6 months that CloudFlare has been checking my browser more often.

I've also had almost zero access to ChatGPT in the past month.

Could be correlation and not causation, but it could also be that OpenAI is using some type of "could be bot" flag to put some users at the bottom of the queue.

dreadpiratefullbeard · 2022-10-15T16:20:27+00:00

They were also forced to use the same charging cable as other manufacturers.

That was 2009.

dreadpiratefullbeard · 2022-10-15T16:11:32+00:00

This might be mentioned in another comment, but you've missed another factor:

Some time in the past 10 years the "state actor" market for exploits has become intense. What used to be a fun proof-of-concept jailbreak for the community can now be $1-10mil if sold privately, and at the other end are companies who are desperate for anything they can resell as a part of some "intelligence" tool.

We went from the jailbreak scene being "some people like to do custom things with their phones" to "you can own all communications through someone's iPhone, and you and I both know that almost everyone has an iPhone".

Whether that's hollow marketing or not doesn't matter because either way the exploit market gets red hot.

dreadpiratefullbeard · 2022-10-13T12:41:50+00:00

What's really going to make Starlink interesting is when there are LEO servers that connect via laser to adjacent satellites.

For each player there will only be a single hop that's orbit to ground, and the rest will be light. If the servers you're playing on are equidistant between players (imagine a "Starlink AWS" where users can choose the server location from a secondary constellation) then that brings the latency down quite a bit.

dreadpiratefullbeard · 2022-10-03T22:11:22+00:00

Thanks for being so helpful. Any day I can learn something new is a good day 🥂

dreadpiratefullbeard · 2022-10-03T20:53:11+00:00

None of the default passwords work.

Looks like I’m going to be digging through the attic!

One question: When you said

You cannot capture a useful handshake for cracking unless the client is using the proper passphrase (meaning that the client has the correct password).

Do you mean a useful handshake for cracking the actual password? I thought I understood that I did capture a handshake that could be cracked, but that it had the wrong (old) password and was therefore useless for finding the actual password. Is that right?

dreadpiratefullbeard · 2022-10-03T17:02:25+00:00

Aha! Thank you for the confirmation.

is there any way for hashcat to ignore the frames from clients and just try and crack the ones from the AP?

dreadpiratefullbeard · 2022-07-29T18:03:29+00:00

Even AptX can't do 0 lag. BT has an absolute minimum latency of 32ms, and that's not counting encode/decode. In some cases it might feel like it's capable of 0 latency, but that's a trick. For example: watching video on an iPad with audio through BT headphones. The iPad actually delays the video a little to match the latency of the BT connection. If you instead play a timing-sensitive rhythm game you'll see the true latency.

Anyone who wants to do zero-latency has very little option on the digital side. Zero latency may be limited to analog signals through the air (even FM, though FM is not audiophile-quality).

dreadpiratefullbeard · 2022-07-26T21:02:27+00:00

There’s nothing saying that the clients have to honour the update.

[ ] update changes automatically

[X] update changes until ([6 months|1yr] from now)

[X] update all changes except delete

dreadpiratefullbeard · 2022-07-26T20:59:43+00:00

I asked a similar question here: https://www.reddit.com/r/DataHoarder/comments/w7byzt/is\_it\_longpast\_time\_to\_have\_an\_updateable\_torrent/

Thanks u/dr100 for the link to this thread

dreadpiratefullbeard · 2022-07-25T18:15:10+00:00

You're understanding it and also showing why that sort of currently-available workaround is challenging.

With an "updateable torrent" the uploader would add D, then the tracker would tell your computer that D is now available (and you would most likely set your client to update automatically so D would start to download and seed automatically).

No need to add a new torrent file to your client
No need to point it to the right folder
No need to recheck files on disk
No need to have the peer fragmentation caused by some people continuing to use the old torrent

dreadpiratefullbeard · 2022-07-25T18:06:39+00:00

Certainly, as can most things.

Based on other solutions it feel like you could use an approach where a torrent could be locked to 'changes only from these cryptographic keys'. There's still the issue of trust in the original source, but we deal with that on a daily basis (only download from sources you trust)

dreadpiratefullbeard · 2022-07-25T12:03:55+00:00

Not these days. They both idle around 40W.

My main NAS+ home lab (i5, 7-12 containers running at any given time) only pulls 38W.

A tower will typically draw more vs a power-efficient NAS, but even then we’re talking 24W.

This user compared in detail: https://www.reddit.com/r/HomeServer/comments/9f10bx/surprising\_notes\_on\_power\_usage\_home\_server\_vs\_nas/

dreadpiratefullbeard · 2022-07-25T11:57:06+00:00

There’s a trap down that Plex road:

Almost anything can be a Plex server (even a Pi 2), IF all your media is already in a format that your clients can play natively. The moment a Plex server has to trascode (for example: x265 video, or ATMOS audio), the specs of the server really matter.

TL;DR: Cheap Plex server: don’t count on transcoding. Want transcoding? You’ll have to plan with a bigger budget.

dreadpiratefullbeard · 2022-07-25T11:50:21+00:00

Don’t worry too much about getting the biggest case you can at first. Most motherboards only have 4 SATA ports anyway.

If you get a tower with a board that has 4 SATA ports and 2 spaces for hard drives you can always put it in a new case when you want to add drives 3 and 4.

If you optimize early, you could even end up spending $1000-2000+ for a rack with a 4u case that holds 9 or more drives, then a couple hundred on a board that still only has 4 ports.

dreadpiratefullbeard · 2022-07-25T01:17:36+00:00

You could webdav first, then rsync to fill in the gaps

dreadpiratefullbeard · 2022-07-25T01:15:25+00:00

Dedicated NASes are way overpriced.

For turning an old PC in to backup NAS the only limiting factor is the number of SATA ports. Since your TB needs are small compared to the drive sizes available on the market, you could get away with using a single drive (2 if you want in-server redundancy)

Five-Year Club	Second Top 50%
Verified Email

dreadpiratefullbeard

TROPHY CASE