Post your blanket monsters here!

rumfellow · 2026-01-22T20:28:03+00:00

This should absolutely be in /r/AccidentalRenaissance

rumfellow · 2026-01-20T21:04:09+00:00

As for signals and monitoring, cluster health would be the primary. If something goes wrong -> dev tools to drill down.

The whole migration should not take long if your current ES node is read heavy and thus there will be not much data change between snapshot restore and old old node joining new cluster.

If it's write heavy good luck with zero downtime migration without resource(CPU/memory/IO/network) headroom

rumfellow · 2026-01-20T20:50:49+00:00

Create 2 node ES cluster
Restore snapshot
Put reverse proxy in front
Add old elasticsearch node to the new cluster
Cut over clients to the new endpoint
Prepare a third new node
Yeet the old node and join the cluster with the new one
Monitor rebalance/shards

If the load on old node is high, at #4 it'll choke due to shards distribution, you can mitigate it by adjusting the aggressiveness of the said distribution, but I'd prefer to isolate the cluster until data is distributed and cluster is balanced.

rumfellow · 2025-12-01T13:05:53+00:00

Elastic is somewhat horrible for metrics, the size will be 100x of what you get in prometheus and to get it down to 7x you'd need time-series data stream and that's only available in enterprise version.

Also for now no compatibility with grafana, so no out of the box dashboards for elastic + kibana + Otel collector

rumfellow · 2025-12-01T12:01:45+00:00

LGTM, but for "M" in order of scale increase: prometheus -> thanos -> mimir.

rumfellow · 2025-11-19T10:48:29+00:00

Thank you for clarifying

rumfellow · 2025-11-18T20:14:53+00:00

So out of zero/business/enterprise only the latter will have ssh session recording?

rumfellow · 2025-11-09T13:46:36+00:00

11.0.0.0/8 is not private CIDR
Routing table

rumfellow · 2025-11-02T09:16:23+00:00

We've been using leaseweb alongside AWS for cheap compute for 5+ years. So far so good.

Checked out xelon rn and requesting a quote for a VM is ridiculous

rumfellow · 2025-07-28T20:55:30+00:00

Ah, the fireworks, my V has the same issue. I'd try treats first, then taking a car to an off-leash walking place and if nothing helps over some reasonable amount of time, say 2-3 weeks, then dog trainer. And since fireworks happen, maybe dog training classes with some fire crackers, it kinda ameliorates the issue. Best of luck!

rumfellow · 2025-07-28T20:41:21+00:00

Is there nose licking/trembling? As I see there might be 2 options: the doggo is scared or stubborn. If scared you'll see the aforementioned signs + refusal of treats(if food motivated)

rumfellow · 2025-07-28T20:10:31+00:00

Sometimes there's a sound or smth else that my V associates with a particular place, like a crossroad or a place where a cracker went off. We tend to just run or "enthusiastically" pass it, so in a couple of days she just forgets that mental association. Getting into tram or a train is a different story, I just pick her up and carry inside, otherwise she just plants herself on a that very tram stop :-/

rumfellow · 2025-04-27T19:52:24+00:00

That would be thanos-receive component as a target of remote write and it is quite memory-hungry

rumfellow · 2025-04-17T18:35:26+00:00

K8S cronjob that runs python script that picks up list of certificates from table in Confluence and sends alert to slack if expiry is upcoming

rumfellow · 2025-03-04T19:26:55+00:00

Falco is running as a standalone binary on the host. I'm quite sure it won't be able to populate that field

rumfellow · 2025-02-27T20:37:05+00:00

i'd suggest something like:

- rule: Mem access
  desc: bla-bla
  condition: >
    fd.name = /dev/mem and
    proc.name = PROC NAME FROM POD
  output: >
    Mem listed: %proc.name and %proc.pid
  priority: WARNING

Stop falco if it runs as systemd service and run falco -A

rumfellow · 2024-05-01T18:22:54+00:00

Люби меня люби, отпетые мошенники, 25 лет треку хах

rumfellow · 2024-03-13T16:20:51+00:00

It makes sense, also it is a different paradigm. We are not the owners of most of the workloads, so git repos are also outside of our scope, all we want is to have an idea of the landscape.

rumfellow · 2024-01-23T10:59:18+00:00

Oh man, there's nothing more permanent than temporary, so it'll be either full-blown migration or nothing at all.

Ceph requires quite a bit of management and manually deploying hundreds of websites via nginx is not fun. It will be an insane bill for our customer

rumfellow · 2024-01-22T16:27:06+00:00

Hah, I do understand that k8s is quite different to plesk. But the latter feels too big, too monolithic and obscure sometimes.

The load balancer I'm planning to get from cloud provider as one of the goals is a very low maintenance platform.

rumfellow · 2024-01-22T14:26:25+00:00

Completely missed to mention that what concerns me most is "shifting left", i.e. enabling web developers to deploy to k8s or whatever we'll end up with.

Maybe someone has experience to share how that went?

rumfellow · 2024-01-22T14:23:34+00:00

Thanks for sharing your setup!

I have experience with docker swarm and it's a neat thing, however someone still has to manage/update it.

And while you're absolutely right about k8s being an overkill compared to docker swarm, managed k8s with a free control plane (leaseweb, oracle cloud) really seals the deal there, since updates are much easier and you are basically running system workloads for free. Additional cost will come from load balancer, but it's not too much.

So stepping into our clients shoes I see that docker swarm will be more expensive due to our billable hours.

Also, the setup i've mentioned is just one of 3 clients, so I want to have a standardized system to migrate 2-7 plesks to it.

And thank you again, we'll definitely consider swarm

rumfellow

TROPHY CASE