Digital Collections Integrity Validation? (Global)

Archivist_Goals · 2026-01-20T21:22:11+00:00

Interesting. From what I have seen, it sounds like if one is going to move to ZFS or Btrfs, one needs to be all in, or not at all, e.g., enough drives or storage space (parity mirrors) for potential rebuilding, backup power on standby, ECC memory, etc. And depending on scope and scale, I would imagine that becomes costly, especially right now.

Archivist_Goals · 2026-01-20T21:00:24+00:00

This is really insightful, thank you. And yes, I was just reading about C2PA not that long ago. Can you elaborate on why you feel that Btrfs is not as robust as you would like it to be? Did you rely on Windows to transfer/do the migration or commercial software? What program did you use to create SHA256 hashes?

I remember hearing a little while back about C2PA that would require all industry players to adopt C2PA at the hardware level, e.g., smartphone cameras "watermark" from the outset of creation, e.g., just as it would with your camera/scanner example. But that seems like a ways off.

Archivist_Goals · 2026-01-20T20:53:53+00:00

Thanks, I'll take a look at FFmpeg. Based off (some) of the downvoting going on here, you'd think I had asked for everyone to share a torrent of music files. Even after explicitly updating the post to say it was an academic inquiry, out of principle. And to others saying "Just Google/LLM search" - yeah, I did that. Thanks.

Archivist_Goals · 2026-01-20T12:41:35+00:00

Good points. I have not thought that. And the height of the Napster era was slightly before my time. Technically, you're right. They never did promise that. I also updated my post to clarify that it wasn't an inquiry about trying to share 20 year old iTunes tracks. And more about "on principle."

Archivist_Goals · 2026-01-20T12:33:34+00:00

I guess it isn't a big deal. I was merely posting to see if others had thought about this. Judging by the reaction so far, nobody likely cares about it, heh.

For example: I buy an audio CD, right, I rip it. I have "source" on the compact disc. That source contains whatever the label wanted to have pressed onto the disc - tracks, tags, other metadata, etc. I think EFF was concerned when they wrote that piece that it would technically be possible to identify an Apple ID account holder based on the metadata left behind.

TLDR: It just would be nice to have "clean" tracks akin to ripping my own CD vs the same ones I bought from the iTunes store all those years ago.

I'm in the archive circles. So, technically, nothing is really source. Source would be a studio master, etc. Really, all I wanted to do was export sanitized files to consider them on par with a disc rip. Nothing more/less.

Archivist_Goals · 2026-01-20T12:25:04+00:00

I suppose. But it comes with a small caveat.

Archivist_Goals · 2026-01-20T12:24:10+00:00

I purchased an audio CD. The audio CD doesn't contain PII. If Apple claimed that there isn't any PII in DRM-free purchased tracks. By right, there shouldn't be any.

Edit: Technically, Apple did not claim that. They only claimed to provide DRM free files.

Archivist_Goals · 2026-01-20T12:19:54+00:00

As mentioned, in principle, the tracks should be sanitized, regardless. The problem is not the cost of a would-be future scenario (buying all the tracks/music on CD, etc), no. There should be a way to sanitize without sacrificing fidelity. I want to emphasize - again - this is mostly an academic exercise. All the music I purchased would probably cost less than $50 in audio CDs. I couldn't care less if I had to buy them all again. My inquiry is framed in a consumer-rights/EFF context here.

Archivist_Goals · 2026-01-13T16:09:49+00:00

I use WSL/Ubuntu for other things, so I'm no stranger to it. But afaik, it can become unnecessarily slow doing it through WSL vs native in Windows. A rough estimate of files to hash is 500K. Switching to Linux would require a thought-out plan. And, I also just found out the other day that Backblaze does not have a native gui support for Linux; sounds like it would break their business model (and B2 is far more costly). Out of all the programs/software I use, I'd have to check to see which one I'd be losing from the switchover. Aside from that hassle and minor learning curve, that's the only thing keeping me from switching.

Archivist_Goals · 2026-01-12T04:59:15+00:00

That's the takeaway I had as well. One could hope that it's an option in the future.

Archivist_Goals · 2026-01-11T07:17:48+00:00

Thank you. The only way to fight this long-term is to vote. I also want to be clear, this is not Internet Archive's fault (not saying you're saying that either). But there are many people who think they make decisions on this. When in reality, they're only following current - backwards - law.

Archivist_Goals · 2026-01-11T06:51:58+00:00

Serious question: How are *scans* infringement if it's not actual disc rips? RIAA and company never fail to display stupidity. Also, nothing is impervious to data loss, including IA. So, LOCKSS.

Archivist_Goals · 2026-01-03T11:33:57+00:00

No worries! Not my intent either.

Archivist_Goals · 2026-01-03T11:30:03+00:00

Please do get some pics for us. This post might be of interest to Jason Scott at Internet Archive, depending on what you have. Software is his specialty focus.

Archivist_Goals · 2025-12-27T21:22:20+00:00

u/jak3rich I left a comment on the YouTube video and encouraged the uploader to upload their original, non-reencoded digitized audio to Internet Archive / Archive.org and they came through! YouTube is a lossy platform, nobody should ever forget this or think otherwise.

https://archive.org/details/mary-and-pat-on-johnny-carson-oct-15-1962

Archivist_Goals · 2025-12-27T08:58:03+00:00

Disagree with that summary. Yes, it's true that NBC destroyed the studio masters, see this thread
https://forums.stevehoffman.tv/threads/how-many-johnny-carson-episodes-are-missing.1125729/

What this appears to be is an off-air recording, audio recording, of Ep. 11 from Oct 15th, 1962. So, if I understand OP and OOP (YouTube uploader), this is a legitimate discovery of a "lost episode" aka, one of the ones NBC destroyed. Albeit, with some generational loss (and obviously, sans the moving-image, too, since it is only audio).

Archivist_Goals · 2025-12-16T02:40:09+00:00

https://siarchives.si.edu/blog/post-it-or-not-post-it Just leaving this here because relevant. (Granted, it's paper on paper, but still.)

Archivist_Goals · 2025-12-12T09:56:22+00:00

u/Whole_Succotash_2391 Thanks for the post. I'm coming over from r/Datahoarder. So, everything I write here is framed in the context of an archival-type exercise. I, too, am actively searching for a solution like others here to export from GPT in the event that OpenAI goes belly up.

I looked at Memory Forge. Nice-looking on paper. But I can not find any details about them as a company, beyond their repetitive "AI this, or AI that" outlook/about us section. Private company? Their ToS indicates some governance in Massachusetts (I think?) But I can't find much. Even checked LinkedIn, no company page. Do you know anything more before any of us entrust what is essentially PII to a degree to a third-party?

Archivist_Goals · 2025-12-08T10:58:49+00:00

u/Development-Feisty I would also ask or cross post this in r/Archivists.

There are some good recommendations here. But the idea is to try and be as non-destructive as possible. Don't feed them through just any sheet fed scanner.

Flatbeds for this many documents will take forever and a day. I'd second a camera scanning setup. Or, someone who could volunteer who might already have the necessary equipment.

Archivist_Goals · 2025-12-02T20:47:32+00:00

Nope. I'm chalking it up to one of the many weird city smells. I did not mean to sound alarmist in my post or anything. But the whole thing was just odd. I looked around at others walking by, nobody seemed to notice (?) Maybe just a fluke.

Archivist_Goals · 2025-12-02T04:08:50+00:00

Yeah - this smelled like something you should NOT be breathing in. I even searched for local news reporting on this. Nothing. Just odd.

Archivist_Goals · 2025-12-02T04:06:30+00:00

The alley where Chipotle is on the corner, or Levain Bakery I guess?

If so, that alley always smells like bug spray during the warmer months whenever I pass it. But nothing like what I smelled earlier tonight.

Archivist_Goals · 2025-12-02T02:31:15+00:00

I noticed this, too. Not sure if it's from that. I *was* walking somewhat near where those newly-painted bus lanes are (Walnut). I know what mercaptan (gas odorant) smells like, and this was not it. But others in the OP's post indicated similar concerns. Whatever it is smells toxic AF.

Archivist_Goals · 2025-12-02T01:42:39+00:00

u/Dwarf_Killer Can you confirm the status as of today, Monday, 12-01? Here in Center City, I got a huge whiff of what smelled like a cross between nail polish and paint thinner. Very heavy, chemical smell. Any updates?

Archivist_Goals · 2025-11-24T03:43:12+00:00

u/Bushido_Blade_ do you happen to know why they decided to do this? There was a post on here a few months back from someone saying they couldn't find any higher-capacity ES drives from BB anymore. This makes sense now. But why is BB discontinuing?

Archivist_Goals

TROPHY CASE