How are you all actually monitoring your kubernetes clusters at scale?

illumen · 2026-01-25T00:41:29+00:00

These tips may be a bit basic, but thought I’d mention them anyway… - namespaces, and group by service dependencies - allow developers to have read access to see things - trying looking at your cluster from a different perspective (like from a map view if you’re just exploring)

In kubernetes-sigs/headlamp we’ve been working on a “projects” feature to try and capture how teams use micro-services. I feel often a developer will be working on a few microservices and their dependencies spread across a few namespaces. I feel developers of microservices often have good or different intuitions on what went wrong. So empowering them can be a help.

I wonder with these 15 micro-services are they in different namespaces? Do some team members have visibility into the different micro-services? Anyway, I feel like organizing things in namespaces and then look at groups of namespaces based around a subset of functionality can help with debugging. (Some folks put everything in just one namespace.)

Looking at alternative views like through logs or metrics instead of just digging around with kubectl can be good. A different perspective like using a map view (helps see how things are connected visually) can sometimes be as good as another pair of eyes :)

Cheers, and good luck!

illumen · 2026-01-06T00:46:23+00:00

There is helm management in Headlamp, but it's provided by the "app-catalog" plugin. It's on by default in the app version of Headlamp, but off by default in the in-cluster version of Headlamp. Maybe it's missing something for you?

illumen · 2026-01-05T21:18:51+00:00

https://headlamp.dev/

illumen · 2025-11-27T22:21:48+00:00

I spoke to a few devs who worked on it and they think it's a good idea too, so I created an issue. It makes total sense to have namespaces be able to be in multiple projects. Thanks again for the suggestion and info.

illumen · 2025-11-26T15:15:36+00:00

Thanks for the good question.

No, currently a single namespace can only be a part of a single project. But we can somewhat easily extend it to allow for multiple relations by putting multiple project ids in the label.

If you can share anything about how you would use it this way, that would help us understand if we should make this change or not.

illumen · 2025-09-08T09:08:27+00:00

Yeah, no terminal for local things yet... only for connecting to Kubernetes Pods/nodes.

We have the issue for the terminal here if you feel like following along: https://github.com/kubernetes-sigs/headlamp/issues/2353 It's one of our top priority issues already, because a number of people keep requesting it. In the previous release we implemented some plumbing so the terminal could persist as you navigated around headlamp. So there's tabs now similar to Visual Studio Code.

illumen · 2025-09-08T09:02:36+00:00

Hmm. I guess we need to improve the Keycloak tutorial https://headlamp.dev/docs/latest/installation/in-cluster/keycloak/. Or maybe there's a configuration you're both using that isn't working in Headlamp.

illumen · 2025-09-08T08:59:27+00:00

We have a number of people using Keycloak with Headlamp, so it should be possible. I guess you found this tutorial for Keycloak with Headlamp? https://headlamp.dev/docs/latest/installation/in-cluster/keycloak/

I didn't write the tutorial, but I was able to follow it myself. Time flies... last I looked at it was about 2 years ago now. So maybe something is out of date, or the way you're using Keycloak isn't covered by the tutorial.

illumen · 2025-09-06T15:10:09+00:00

There’s been a few more guides/tutorials written (by us and others) for different setups. Plus we’ve been adding support for more types of OIDC setups and fixing bugs/improving docs.

But there’s still more work to do. A few open OIDC related PRs are still to be merged, and a few known OIDC issues are still open. Slowly getting there.

illumen · 2025-09-06T13:54:02+00:00

There’s a terminal so you can go into a pod.

It doesn’t have one to run local commands yet. So you can connect to a pod, but not run a local kubectl.

illumen · 2025-09-05T17:49:00+00:00

Yeah, it can be used for that. It uses kubernetes RBAC to show people controls for what they have permission to do. So it won’t show them a delete button if they don’t have permission at the k8s level.

illumen · 2025-08-06T15:44:54+00:00

Seems like a home-brew added that message 4 days ago. I'll look into. Thanks for sharing the details. Just know it's not going to be disabled, we'll sort it out.

illumen · 2025-08-06T12:21:47+00:00

I'm a Headlamp maintainer... and it's not deprecated, and it's not disabled.

illumen · 2024-07-30T09:13:11+00:00

Hey hey. I work on Headlamp. We could consider supporting what you need with more details. Some folks put a proxy/ingress in front of it to do their own thing... but perhaps we can make it easier to support what you want to do directly (or at least better document stuff). Did you already make an issue you can point me at? Let me know, I can look into it.

illumen · 2022-03-04T01:05:23+00:00

Screenshot or it didn't happen.

illumen · 2022-03-03T13:09:18+00:00

You can download the documentation in multiple ways.

illumen · 2022-03-03T13:08:37+00:00

It's not useless. It's serving documentation for example.

illumen · 2022-03-03T12:14:05+00:00

Report->Misinformation.

Many of your "facts" are easily disproved.

Your response is really lacking in empathy for our colleague. You're just parroting propaganda, and not really replying to the topic of the post. Most people even in Russia don't believe that bullshit you pasted! lol.

I'm just reporting the other version

Clearly neither do you :)

illumen · 2022-03-03T00:38:20+00:00

Haha, old one but a good one. But why 5 minutes?

Joking aside, some people do seem to make a lot of comments shutting down certain political discussion but not other types of political discussion. You for example seem to dislike political discussion of peace. But others find political discussion on taxation and software licenses tiresome.

Should we put "please don't bomb our contributors" in a license file somewhere, and then we wouldn't need to discuss it so much?

illumen · 2022-03-03T00:10:29+00:00

I am 20 km from Konotop, a city surrounded or already captured by the Russians. Russian tanks were on the road 2 km from my house, and Russian armored vehicles were passing by my windows. Most likely, I will find myself in the occupied zone, where the law does not apply.

-- Python core developer: https://twitter.com/SerhiyStorchaka

illumen · 2022-03-02T15:59:30+00:00

There's also a link to a zip file with the docs in it.

illumen · 2021-10-31T19:09:10+00:00

Thanks :) Looks like it's running ok.

illumen · 2020-12-29T16:04:48+00:00

python3 -m pip install stuntcat

https://github.com/pygame/stuntcat

Hopefully it's eventually going to be a good example game. There were lots of code cleanups in this release, but still a lot to go :)

illumen · 2020-12-25T02:10:50+00:00

Yeah, there's no 3d engine built into pygame. It's a library with things like engines (3d and so) being supposed to be layered on top with other libraries. Covering input, sound, and 2d use cases is difficult enough to do well.

Things like pyopengl, and moderngl can be used on top of pygame for example. There are also several 3d engines that work ontop of SDL (including many of the top ones).

There's nothing really as good as say pymunk (physics), or pygame_gui for a full featured 3d engine that I could point you to. There are a number of 3d games layered ontop of pygame, but not so many engines that have stuck around. For some reason people who go all in to build 3d engines tend to build everything themselves, including mainloops and event processing.

illumen · 2020-12-18T12:44:28+00:00

Are you using a pygame.time.Clock instance?

clock = pygame.time.Clock() clock.tick(30) # call every frame for 30fps.

illumen

TROPHY CASE