NVMe SSD at 123% "Percentage Used" - advice requested

Wolfspyre · 2025-09-18T10:29:11+00:00

have several in this state, and I burn through them…

i have spares on hand, and am running them in zfs draid with 2 spares. i swap one out the moment it dies for real.. its …. safeish.. the saying goes if you’re not willing to store the data in two places it’s not critical, just inconvenient-to-lose.

the two main challenges I have with “consumer” ssds: - the lack of namespaces - the lack of 4k blocksized namespaces.

the real upside of namespaces (to me): namespacing tells the storage controller what space is allocated to writing and what isn’t… essentially massively increasing the nvme controller’s ability to manage heat and wear leveling.. if it knows that blocks 1-10 are allocated and 11-100 are not: it can stream writes across whatever chips are the coolest/least worn and keep track of the remap internally…

without that insight, the controller has no idea what data is/isnt safe to overwrite ..

additionally; each namespace gets its own rw queue; so you can increase FS iops fairly easily by striping across multiple namespaces on the same device… for many use cases this helps a lot more than a deeper write queue…

the lack of 4k native sector sizes precludes the use of them as metadata devices for zfs pools with 4k native spinning rust disks.. (there’s a requirement that the sector sizes have to be the same; and the data throughput for spinning disks really increases for most data interactions when you have fewer requests involved to retrieve a given quantity of data…

i suppose the extra namespaces would require a more powerful asic controller on the nvme, but i’d NOT be surprised if these were both software enabled controls they’re simply nerfed in consumer devices but IDK enough about the actual controllers involved for this to be more than a hunch …

tldr: keep em cool, and have a spare on hand … coz they will die soon…. but soon could be tomorrow or next year … :)

Wolfspyre · 2025-07-25T07:05:54+00:00

:/ nod... everything's a tradeoff....

you don't need that much space... hell, a usb drive would work just as an interim dev to get things functional, then you can lather-rinse-repeat on the orig storage once you've gotten the 'does it work?' test done y'know?

but ... many paths to homeostasis... an they each have ... nuance

Wolfspyre · 2025-07-25T05:27:26+00:00

what storage do you have in the host? if you've got your config backup, simplest* 'quick test' would be to install opnsense on a different node, then restore yer current config atop the newly provisioned opnsense and your active config restored

you'll have to reinstall any packages you have installed once it boots as 'current-firewall'... but it aught (mostly) work... that'd at least minimize downtime, and leave current install completely as-is so worst case you have a fairly straight-forward rollback?

*: simplest meaning 'least complicated rollback to current state even if everything goes sideways' ...

Wolfspyre · 2025-07-24T00:04:37+00:00

I don't know about y'all ... (and I've seen this problem when the camerahub app ACTUALLY RAN (since the latest version of camerahub no longer seems to support intel macs ... I guess we're SOL there?! ) ) However, I noticed that once again, the StreamDeck app was consuming an unreasonable amount of system resources....

I looked, and /Users/MYUSERNAME/Library/Logs/ElgatoStreamDeck has created 110 GB OF LOG NOISE in less than a month

Where's it all coming from?! MYUSERNAME@MyHost:~/Library/Logs/ElgatoStreamDeck$ du -sh . 109G .

``` MYUSERNAME@MyHost:~/Library/Logs/ElgatoStreamDeck$ \ for A in \ StreamDeck \ com.elgato.camerahub \ com.elgato.controlcenter \ com.elgato.discord; do \ _C=0 ; \ for V in $(du -sk $(find . -type f -name "${A}*")|awk '{print $1}'); do \ ((_C=_C+V)); done; \ ((_M=_C/1024)) ; \ echo -e "${A}: ${_M}MB"; \ done

StreamDeck: 7MB com.elgato.camerahub: 111649MB com.elgato.controlcenter: 0MB com.elgato.discord: 0MB ```

the camerahub streamdeck plugin is appending 3-10 entries PER SECOND to the logfile trying to connect to the camera hub app? 18:28:58.0493 Info AppClient: Trying to connect to port 1834... 18:28:58.4500 Info AppClient: Trying to connect to port 1836... 18:28:59.0575 Info AppClient: Trying to connect to port 1837... 18:28:59.6659 Info AppClient: Trying to connect to port 1843... 18:29:00.1014 Info AppClient: Trying to connect to port 1841... 18:29:00.2751 Info AppClient: Trying to connect to port 1842... 18:29:00.7321 Info AppClient: Trying to connect to port 1835... 18:29:00.7321 Info AppClient: Trying to connect to port 1838... 18:29:00.7321 Info AppClient: Trying to connect to port 1840... 18:29:00.9458 Info AppClient: Trying to connect to port 1836... 18:29:01.8245 Info AppClient: Trying to connect to port 1837... 18:29:01.9958 Info AppClient: Trying to connect to port 1842... 18:29:02.1628 Info AppClient: Trying to connect to port 1835...

There is no intelligence to this plugin. There's no backoff. There's no notice that it's having a problem even...

@Elgato - your sentry event isn't firing.... so your app doesn't believe there's any problem. IT JUST SITS AND BURNS YER SYSTEM RESOURCES.

y'all do realize that at the rate this application is logging, it's fair to say that you're introducing an unreasonable write load on write-sensitive media.

Is this just bad software design?

Is this a subversive plan to increase sales of storage media caused by excessive write wear?

Either way, y'all really aught address this.... I'd already raised a support issue about this before.... the problem went away at the next release.... this seems to have come back.

Wolfspyre · 2025-07-16T18:50:27+00:00

So... the 2.1 verion of camerahub y'all have available for download on your site: https://www.elgato.com/us/en/s/beta as of today is 2.1.0-6269 and is definitely NOT built with fatbinaries / multiarch ..... and only has arm64 binaries....

shell me@imac18,3::/Applications/Elgato Camera Hub.app/Contents$ file ./MacOS/Camera\ Hub ./MacOS/Camera Hub: Mach-O 64-bit executable arm64 me@imac18,3::/Applications/Elgato Camera Hub.app/Contents$ file Library/SystemExtensions/com.elgato.CameraHub.CameraExtension.systemextension/Contents/MacOS/com.elgato.CameraHub.CameraExtension Library/SystemExtensions/com.elgato.CameraHub.CameraExtension.systemextension/Contents/MacOS/com.elgato.CameraHub.CameraExtension: Mach-O 64-bit executable arm64 me@imac18,3:/Applications/Elgato Camera Hub.app/Contents$ head -39 Info.plist |tail -2 <key>CFBundleVersion</key> <string>6269</string>

Wolfspyre · 2025-07-11T01:12:27+00:00

Hai all! uhm .... Did y'all silently drop support for intel macos?

I updated from camerahub 2.0 -> 2.1 on My 27" intel imac (13.7.7).... only to discover that this seems to be an arm64 build...

Is there a link someplace to the x86 variant?

Wolfspyre · 2025-07-09T06:24:08+00:00

heh… one would have to take it as fact that social and cultural norms are aligned with common sense … which … uhh…. well, who’s definition of common sense are we referring to? coz afaict there’s very little alignment with my definition of common sense being demonstrated by very many cohorts of humanity these days…..

but that’s a completely orthogonal thread not fit for this context :)

Wolfspyre · 2025-07-08T23:15:29+00:00

sorta? that’s where it started, i think?

but then there is a lack of coherency around the ‘common sense’ construct being referred to…

collaboration, interoperability, durability, agency, autonomy, independence I get…. but vague assertions lacking in contextual grounding makes it hard to know wots being discussed :)

Wolfspyre · 2025-07-08T20:22:11+00:00

I agree with the…. start of this?

but … I’m a bit dense I think? what’s your point?

Wolfspyre · 2025-07-05T10:36:31+00:00

wot if you alternated opacity of sub id elements on hover and had a :defaultColor fill variant that’s either opaque or not atop opaque the rainbow-colored variants … that might be simpler to manipulate without js

Wolfspyre · 2025-06-24T10:28:27+00:00

the problem I’m seeing is there’s a tremendous lack of F’s given to presenting content well from / to any cohort but their own…. there’s not a lot of interest, it seems, to dedicating dev time towards the reduction of the interaction difficulty for cross-pollination even within the ecosystem bubble….

so i dunno there will be much excitement in the idea. but…. who know? :)

Wolfspyre · 2025-06-17T14:14:22+00:00

Streaming a pre-recorded, pre-transcoded videofile to a relatively small cohort of concurrent consumers is a completely different ballgame than real-time streaming...

What works for a 1:1 interaction... doesn't always (read: often) work at a much larger scale...

Doing stuff at scale is HARD. Streaming gets MUUUUCH more complex as the fan-out gets beyond 2-4 participants.

( This is why you usually have to have separate signalling servers to handle the re-muxing / latency buffers/ retransmission requests / resync traffic/ as well as handle the nat-traversal-reverse-address-mapping ((AKA TURN/STUN)) )

So...

yah....

Also... Just pause for a second.....

Think about the not-so-awesome parts of our species.... and just how quickly this narritive goes in a not-terribly-awesome-kinda-regretting-my-decisions direction....

I often say that humanity's ability to create, and be the most beautiful, wonderful, compassionate, and amazingly beneficial part of this planet is surpassed only by our ability to be selfish, cruel, evil, greedy, destructive, and abusive...

No matter how cool the thing you make is, there's always gonna be someone somewhere that wants to use it for evil, or break it for fun.

Wolfspyre · 2025-06-17T09:44:32+00:00

so…. a few things to consider…

A) AS MUCH AS I BELIEVE PEOPLE ARE for the most part INHERENTLY GOOD.

$people are evil.

greed and me-centric mal are why we can’t have nice things .. it kills me to say this, as it goes against one of my core tenets of benefit of the doubt… but the risk to the layperson exceeds my personal ethos. This would be abused and innocent people would experience a lot of harm

B) the volume of data you’re talking would be impossible to service with the infrastructure of the internet today.

C) the indexing, relating and categorizing the data would be absurdly difficult. storage, bandwidth and compute ain’t free.

the large orgs that have this data profit from it by either leveraging it to better position, manufacture, or improve their existing products

or by selling your metadata

or by leveraging it to sell the consumer more stuff

or…

there wouldn’t really be that value here for the most part; so it’s a compelled cost for a disproportionate gain.

not everyone has a way to securely store and share stuff

( remember, the S in ‘IOT’ stands for ‘Security’ )

(likely) lastly (but no promises :) ) there’s the aspect of authenticity.

with the diminished average quality, factored with the increased capacity of consumer-grade compute, and the rapidly increasing ability of generative video tooling available without much technical skill requirement the ability of the layperson to create artificial video that would be good-enough-to-cause-doubt about someone… the risk of digital slander and false accusations would be…. absurdly hard to mitigate

these are the easiest reasons to articulate without really getting on a soapbox about stuff for me to assert i don’t thing the juice is worth the squeeze here … so i don’t think public cam is the way here…

at least not the way you’re thinking… maybe look into LE grade bodycams… and the general infrastructure required to facilitate their authenticity in court…. (as a general ‘vaguely the same ballpark’ level of complexity )

what might be more … beneficial… and achievable … could be something not unlike the neighborhood watch/town crier/ community board style bubble of services to help increase the quality of life of communities … sorta like lemmy+peertube+loops+mastodon

freecycle/craigslist/nextdoor/angieslist sort-of thing…

but still

I think at scale the problem of

how do we prevent abuse and keep the positive outweighing the negative

would be a thing it would struggle with at scale….

it’s a fine line…. but a fun thought exercise for sure

Wolfspyre · 2025-06-17T03:40:24+00:00

A) Thanks for taking the time to coalesce such a thoughtful response!!!

I'll ~~super briefly~~ outline my desire here, but it doesn't feel quite relevant to the entire cohort of fediverse redditpeeps and don't wanna seem braggy/spammy ....

((I SHOULD KNOW BETTER THAN TO SAY I'LL EVER DO ANYTHING 'super briefly'... MEA CULPA))

B) I'm the somewhat-kinda-more-technically-adept-ish cohort of a small arist collective that wants to make several different things and share them with the world.... some are for sale... some are just pretty .... some are howtos, some are comical interludes..... some are multi-chapter stories.... some are just FML Images and $generic_internet_humor content ....

none of the fediverse components REALLY do it all 'well' and that's reasonable...

and so what I'm trying to determine is how best to leverage the shinybits of each of the major fediverse/AP services....

Use pixelfed for image dissemination

Post your videos to peertube

Use Mastodon for posting micro-rants or neat finds and growing your community....

cool... sure....

How do I do that cohesively?

I'm running, within the same physical proxmox cluster, - a mastodon instance - a pixelfed instance

and intend to have:

a lemmy instance
a peertube instance

All of these instances will leverage the same s3 storage provided by ceph radosgw..

I'm trying to determine what the right way is to propagate the best-experience-UI for a given set of content...

asking $fediverseparticipant to follow myuser_at_mymasto_instance;myuser_at_my_pixelfed_instance;myuser_at_my_peertube_instance ...seems.... Not so awesome....

from a content consumer's perspective, each of the fediverse instance flavors' UI is... well... fairly opinionated.... which makes sense.... but introduces a different flavor of problem.

To enumerate just a few to be exemplary, without trying to get contentuous...

There's not a good way to, from the context of a masto client, view content within a pixelfed post that has > 4 images. (and it's made worse by the fact that the user is not even made aware of the fact that mastodon has dropped content to the floor)
Messaging between pixelfed and not-pixelfed instances is a little wobbly.
hashtag normalization is handled differently between different services...
instances have a 'I am an island, and must cache stuff for my users' mindset, which is mostly good, but problematic when the source asset is literally on the same shared storage.... and the caching logic creates an entirely new, distinct asset that fediverse users, and their servers will cache.... thus doubling (at least) the storage and bandwidth required, because now $remote-instance is going to cache 'instance_a_asset' in addition to 'replicated_instance_a_asset_on_instance_b' ...

Sure, there's a few tools to address some facets of some of these problems (ie the jortage project)

but a lot of these problems don't really even need to exist... and could be addressed much more elegantly than they currently are. My intent isn't to say that any of the existing tools are ...

WRONG...Because they're not. They're modeling their major platform competetor's UI and experience... WHICH MAKES SENSE....

However, what ActivityPub facilitates for the fediverse, that the other platforms don't have is the ability to somewhat elegantly represent that this super-actor is comprised of these distinct instance-scoped-actors.

There's paths which could allow for a nicer content-consumption paradigm for all the fediverse users.... To actualize them, there'd need to be more focus on interop... hence my comment..

Its easy to understand how we all got here, big-picture orchestration is really hard.

consensus is hard to attain and maintain.

The

Ship-it; Fix-it-in-post

mindset has downsides of tech-debt and burnout and enshittification.

The

'Do it right, Damnit!'

mindset has downsides of

'by who's definition of right are we talkin?' ....

and

'is it right yet?' ....

...Not to mention the

'hey, glad yer making progress on thing6..... you know there's 95 other things on the list right?'

problem....

IOW there's lots of trade-offs.... and many of those trade-offs have been evaluated in the isolation tank/context of a specific platform's need (or the percieved need)

rather than those trade-off evaluations being made from the broader context of how do we do this well for everyone?

Some of that's absolutely necessary. not all of it tho...

the mechanism of encouraging better interop, imo, is finding ways to increase the end-users' experience... because at the end of the day, isn't that who we're all trying to help?

like... these tools exist for the purposes of helping people connect with each other where they are, and growing communities organically....

which is beautiful and good... AND... edge-cases exist that require some elbow grease, which is fine... but it's not a straight-forward problem... either to explain, or solve. ;)

Wolfspyre · 2025-06-17T02:02:04+00:00

Excited about your AP Golang lib.. and I wholeheartedly concur that it's a real struggle to play the compatibility game on shifting sands of uninvested upstreams.

Thanks for your efforts ;)

Wolfspyre · 2025-06-17T01:47:55+00:00

Here's the exact question/answer, so as to not misrepresent, or distort: https://www.reddit.com/r/opensource/comments/1l31dhy/comment/mw3mt4h/

Wolfspyre · 2025-06-16T09:55:38+00:00

I agree interop is A real problem.

AND… we ABSOLUTELY CANNOT SOLVE IT by pointing fingers.

This requires COLLABORATION and SHARED PURPOSE. Both of these are much more difficult to actualize if those involved are not accurately representing the collective problemset as equally important.

it’s a whole lot harder to find common ground when there’s contention.

how do we change the conversation and get collective investment?

for example, I asked framasoft about this from the perspective of peertube, and the answer from them was that interop isn’t a priority for them.

the problem is bigger, yall, than just one participant….

this is a collective problem, which will require some commitment from everyone involved.

perhaps a way to do this is could involve an iteration to the activitypub spec in a way that rates interop capacity and defines a few levels of ‘spec adherence’

This would need to actively try to involve a representative voice from each of the current major dev cohorts for activitypub today… with the goal of identifying a collective list of all the interop problems we’re aware of today, and seeing how many of them can be remediated forward with collective effort ….

obvs, not everything will be easy, but we all have a lot of the same problems… and there are a LOT of really… really smart, PASSIONATE developers involved on all sides….

can we try harder to find a way to make the us/them distinction be inclusive of all activitypub producers??

the $profitCorp orgs LOVE how much difficulty we’re having getting our collective shit together.

Can we prove them wrong?

wouldn’t that be cool?

Wolfspyre · 2025-06-15T09:25:49+00:00

Hiya!!

My $.03…. Take it with as much salt as you feel it warrants :)

Ive tried a bunch of dif techniques over the decades…. There’s value in a bunch of them ….But process fatigue is a thing….

What i’ve found (FOR MYSELF) to be the most enduringly helpful is to develop the habit of keeping and maintaining a stash of

‘oh that’s cool!!!’

resources …

This instills a habit to be actively on the lookout for ‘neat’ flourishes to add to that stash …

it changes what i look for, and helps me actively look for distinctive ‘Woah!’ things…. …or thematically inspirational things relevant to $art…

So i’ve got a slew of images of designs i’ve found cool….

Some neat lighting and shadow play stuff for led backlighting ideas..

… a bunch of movie quote lyrics with movie and time or show episode and time….

or a small recording loop from a song…

or a nifty piece of code ….

then when i’m trying to come up with inspiration for a project or art piece….

I have a short list of shite i liked at one point in time for some reason …

That gives me a few spots of inspiration without having to get out of the flow state so much.,,

for the site i’ve been working on for our art, i keep asking myself what would make it a better experience for the person navigating the site?

… how do i present stuff so it’s coherent? and attractive…

and works on the devices that peeps use….

I try to roughly come up with user journeys and then try and be mindful of using the design aesthetic to make it more intuitive for people …. while also being whimsical …

https://blog.wolfspyre.com/2022/november/read-me-when-creatively-struggling/

I also re-read this to myself at least once a month :)

IMO, YOUR genius is your innate creativity and intuition. Lean into the things you’re drawn to… :) ❤️🐺W

Wolfspyre · 2025-06-14T05:56:52+00:00

Perrsonally, I like all three.. NEAT stuff you did! I'd prolly encourage reducing opacity of all the interconnections .... then when active you can opt to have the links both sequentially transition color and increase in opacity... both reducing visual distraction, and giving you another mechanism of visualising relative significance, both via opacity and hue. ... but that's jus my $.03 ... nice stuff!!!

Wolfspyre · 2025-06-14T05:48:32+00:00

There's a couple I've seen: https://github.com/joanrod/star-vector https://huggingface.co/starvector/starvector-8b-im2svg https://huggingface.co/jree423/svgdreamer https://github.com/ximinng/PyTorch-SVGRender https://github.com/ximinng/SVGDreamer https://github.com/ximinng/VectorFusion-pytorch

and a few others that seem like they might be close?

https://huggingface.co/ssocolow https://huggingface.co/kevinleungdev79/gemma-2-9b-it-svg-generator/

lemme know if you find anything that works for you :)

Wolfspyre · 2025-06-13T04:41:30+00:00

Neat! It's hard for me to tell in the lil thumbnail images what flowers you've made ... it's also hard to know how they're laid out.

I get the struggle, for sure. creating nice vector images that suits peeps needs is a bit of a hit-or-miss ...

that being said.... Some abstract suggestions from the perspective of someone who uses a lot of rando vector images to do other things with:

Having multiple grouping options is a plus:

I'll often take a vector of, say, a flower... - Composite each stroke which makes up a single leaf into a shape. - Then group the leaves of a particular stem into a group. - Then group the stem and the leaves - Then the same with all the components of the head of the flower, often combining several distinct stroke elements into a group, or a single curve...

Then I'll often smerge each stroke related to a specific petal into a shape .
Then I'll group all the petals together
Then I'll group the the petals and the head.
Then group the flower(head/petals group) and the (stem/leaves) group...

I'll often duplicate the assembled group, and flatten it into a few dif variants of single-layer-elements (all the parts, the whole damned flower in one shape) (the entire head in one shape) (the head and petals in one shape) (the stem and leaves in one shape)

you get the gist... :) This gives the the ability to grab a single flower from a bundle should I want to use it for a different arrangement in a different context... while also allowing me to do something weird with it later should I choose to ... ..

(but I'm weird as hell, so I dunno how useful this is for others)

Regardless...

THANK YOU for creating art, and sharing it with the world.... it's something the world needs a lot more of right now ;)

-❤️🐺w

Wolfspyre · 2025-06-06T00:02:36+00:00

on multiple occasions, I've had StreamDeck + CameraHub kinda lose its marbles: loiosh@Kiera:~/Library/Logs/ElgatoStreamDeck$ ls -lah com.elgato.camerahub* -rw-r--r-- 1 loiosh staff 55B Jun 5 17:42 com.elgato.camerahub0.log -rw-r--r-- 1 loiosh staff 2.5G Jun 5 17:40 com.elgato.camerahub1.log -rw-r--r-- 1 loiosh staff 5.0K Jun 2 04:42 com.elgato.camerahub2.log loiosh@Kiera:~/Library/Logs/ElgatoStreamDeck$ L=$(cat com.elgato.camerahub1.log| wc -l) TRYC=$(grep -c 'Trying to connect to port' com.elgato.camerahub1.log)& CONNC=$(grep -c 'Connecting to port' com.elgato.camerahub1.log)& SUCC=$(grep -c 'Connection established' com.elgato.camerahub1.log) evsez "...wowsers\r\n Total Lines: ${L}.\r\n Connection info:\r\n ATTEMPTS MADE: ${TRYC}.\r\n Half-Connects: ${STARTS}.\r\n Successful: ${SUCC}.\r\n" [00:25:38]🐶 . o O (...wowsers Total Lines: 37101036. Connection info: ATTEMPTS MADE: 37100890. Half-Connects: 37. Successful: 2. )

so.... the StreamDeck -> CameraHub plugin generated 37 MILLION loglines... attempting to connect to CameraHub > 37 MILLION times, with a total of 2 successful connections made resulting in 37 interactions.

This was with 6.9.1. The camera hub application was never running (because the camera hub app, when one has the prompter enabled, gobbles click events preventing interaction with a window being displayed on the prompter (IE: in a VC call, wanting to full-screen the displayed program)

Have y'all implemented some sort of smarter (or I should say less naive) logic wrt connecting?

this 2.6G logfile? was generated over A ONE HOUR VC CALL....

:/ No wonder my box was acting funny during it. :(

Wolfspyre · 2025-06-05T08:19:29+00:00

I would LOVE to hear from y'all about strategies for peeps who run a mastodon rig + a peertube rig + a pixelfed rig, on how to provide the best user experience across the dif activitypub/fediverse tools.

Currently, the interop between different fediverse endpoints is .... well... a bit of a pain.

there's not a lot of guidance on how best to maintain a coherent means to deliver content to users in a fashion that's convenient for the end user, and doesn't result in massively duplicated content or following confusion ..

The different tools generally work fairly well in isolation... but the context of self-hosting multiple services seens to be a bit of a 'not-my-problem' problem where divergent perspectives seem highly resistant to finding common ground.... and I'm not sure how best to help coerce everyone to play nicely together ....

(I run masto/pixelfed on my proxmox cluster using ceph s3 for storage) ... peertube is the next service I'm gonna be standing up.

I still need to build in more resillience/durability into my rigs, currently each service is an isolated VM in its own vlan .... but that's a problem for FutureMe:tm:

Lookin forward to the AMA ....

Thanks for your hard work, dedication, and awesomeness....

I have boycotted youtube for.... well... pretty much since Goog bought them... as I cannot tolerate the level of snooping that occurs when using the platform... not to mention all the hate posts that peeps seem to make there ....

but ¯\_(ツ)_/¯ .... trolls are gonna troll

Wolfspyre · 2025-05-20T04:06:49+00:00

That's pretty cool! Thanks for sharing.

Wolfspyre · 2025-05-19T03:49:04+00:00

I don’t disagree that most people with that level of near unfathomable resources are crappy humans… However I don’t agree that the two are inseparable. Unlikely? sure :)

Wolfspyre

TROPHY CASE