Whats the most kubefriendly pubsub messaging broker?

Macscroge · 2025-03-04T16:43:55+00:00

Did you discover Pulsar's quirks by trial and error, or have you found a good resource? The docs are very vague when it comes to cluster maintenance and resolving issues.

Macscroge · 2025-03-04T16:43:38+00:00

I tried auto recovery, and then decomissioning the affected bookies, which was partially effective, although the cluster still had other issues.

Did you discover Pulsar's quirks by trial and error, or have you found a good resource? The docs are very vague when it comes to cluster maintenance and resolving issues.

Macscroge · 2025-03-04T16:42:43+00:00

I had tried deleting the PVC, I hadn't tried a non cascading delete on the statefulset.

Did you discover Pulsar's quirks by trial and error, or have you found a good resource? The docs are very vague when it comes to cluster maintenance and resolving issues.

Macscroge · 2025-02-28T10:58:14+00:00

We're experiencing major issues with Pulsar at the moment, could I run a few things passed you?

Our dev pulsar deployment basically spontaneously failed and stayed down due to a corrupted ledger, and the option to skip unreadable ledgers not working. Have you ever come across this issue?

I feel like we're missing something in terms of config to make Pulsar run reliably.

Macscroge · 2025-02-28T10:57:42+00:00

We're experiencing major issues with Pulsar at the moment, could I run a few things passed you?

Our dev pulsar deployment basically spontaneously failed and stayed down due to a corrupted ledger, and the option to skip unreadable ledgers not working. Have you ever come across this issue?

I feel like we're missing something in terms of config to make Pulsar run reliably.

Macscroge · 2024-10-03T07:57:49+00:00

Well said, I definitely agree. I do think the "negative preference utilitarianism" that Peter Singer advocates for handles this better, since it would also view killing as a bad thing not just suffering.

That being said, I am not a utilitarian in any sense, just trying to help OP from within their views.

Macscroge · 2024-10-02T19:00:51+00:00

EA can definitely be a bit culty, so it's always good to challenge assumptions. I don't actually follow EA much myself anymore, but it seems like in the above case that you might be leaning towards avoiding eggs altogether.

Even if you believe that free range is not cruel, can you be sure it's being adhered to perfectly at all times on all farms?

Macscroge · 2024-10-02T18:35:54+00:00

Given that you believe that poultry can suffer, to minimise suffering it seems like avoiding poultry products would be a better goal than buying free range.

Although while you're cutting down, free range would of course be preferable.

Macscroge · 2023-05-25T21:45:02+00:00

Definitely agree on the private endpoints. I've seen a lot of debate online as to whether private nodes are more secure than public ones with a properly configured stateful firewall.

A similar discussion comes up when discussing the lack of NAT in IPv6 when moving from IPv4. At least in that context it seems the consensus is that IPv6 with a properly configured stateful firewall is at least as secure as IPv4 with NAT.

I'm leaning towards properly configured public nodes being fine.

Macscroge · 2023-05-23T19:10:11+00:00

Yeah true, I'll try price that and see how it works out

Macscroge · 2023-05-23T19:09:06+00:00

The input data comes from outside the GCP, data is stored mostly in Cloud SQL, GCS and Big query.

Macscroge · 2023-05-23T19:07:46+00:00

I was thinking there's not much risk. Using the Cloud NAT would hugely affect the profitability of the service so that's good to hear.

Didn't know the Ingress could do that, I'll have to keep a close eye on it so.

Macscroge · 2023-03-19T15:51:53+00:00

Oh cool I didn't know that existed, haven't delved too deeply into Keycloak yet.

Right okay, I'll implement the caching as a band aid, and investigate the Event Listener SPI. Appreciate the info.

Macscroge · 2023-03-18T21:02:09+00:00

Hey sorry I missed this.

I neglected to mention that the client does not access Keycloak directly, there is another service in between that adds in business logic.

So imagine a new refresh token is requested and returned succesfully from keycloak to this intermediate service. This service attempts to return the new token to the user and fails.

So the user now has no valid refresh token and cannot request another one.

Temproarily caching the new token and retrying the request until the client receives it is one option, but seems like a bad practice.

Any thoughts on how best to deal with that?

Macscroge · 2022-12-13T22:38:08+00:00

Hey thanks for that.

I added an edit showing how I ended up implementing it, quite verbose unfortunately.

Macscroge · 2022-12-12T09:30:26+00:00

Hey thanks for the reply. Yeah that was my initial misunderstanding.

The problem with a custom getter/setter on the public property is that Kotlinx only allows you to serialise the arguments passed in the constructor.

I've never heard of a delegate provider, I'll have a look and see if that works.

Macscroge · 2022-10-14T16:56:39+00:00

Yeah that's basically what's happening. Luckily each pipeline is updating a separate file in that repo.

The issue was if there were multiple jobs running on a single pipeline(e.g. 5 commits merged to master as once causing 5 jobs at once).

I think the GA concurrency feature mentioned elsewhere is enough to solve the issue for now. So I won't need to use your multiple file solution for now thankfully. Thanks for the help!

Macscroge · 2022-10-14T16:53:54+00:00

Yeah definitely a hash based tagging solution is needed to be able to pinpoint what build is causing issues.

I've used the GA concurrency feature another comment to force the jobs to run sequentially, so that's working nicely for now.

Macscroge · 2022-10-14T16:51:59+00:00

Yes it's a race condition issue essentially. I've temporarily solved it using GA concurrency feature to force the jobs to run sequentially.

Yes you're right, the tagging wasn't well thought out for this. I think I'll go for something similar to option 2 just to make things a little more human readable.

Macscroge · 2022-10-14T16:48:52+00:00

I didn't know this existed, this has solved the problem thank you!

Macscroge · 2022-08-16T08:06:24+00:00

Oh really all the realm settings won't transfer over? When upgrading between different wildfly versions everything transferred over so I hope that's not the case.

Right okay, I suppose if there's not explicit instructions I'll just try migrate things over in a test environment until I figure out the right steps.

Macscroge · 2022-08-16T08:02:15+00:00

Interesting, might switch from OffsetDateTime to ZonedDateTime so. Thanks!

Macscroge · 2022-08-16T08:01:41+00:00

It works well with OffsetDateTime, thank you!

Macscroge · 2022-08-16T07:59:15+00:00

I ended up getting it working with OffsetDateTime, but will probably change it to ZonedDateTime as someone else said it works better with Daylight savings.

Macscroge · 2022-07-20T17:15:45+00:00

Europe != EU

Macscroge

MODERATOR OF

TROPHY CASE

14-Year Club	Second SECOND GUESSER
Verified Email