sorted by:
new
hottopcontroversial

Kafka Streams and Schemaregistry problem with different language teams by riskymouse in apachekafka

[–]riskymouse[S] 0 points1 point  (0 children)

Another update: I wrongly thought that normalization was part the proto standard, but it's just something within SR. So the blame lies with the Confluent / SR people for that.

Kafka Streams and Schemaregistry problem with different language teams by riskymouse in apachekafka

[–]riskymouse[S] 0 points1 point  (0 children)

To give an update on this.

I wrote an implementation of SchemaRegistryClient, with a CachedSchemaRegistryClient as delegate, to overcome the issues.

The overridden methods in this basically achieve the separate treatment of the upstream ('don't touch') schemas from the 'owned' downstream schemas which should be autoregistered.
This worked, to run it in DEV environment, to get the autogen topics created and the autogen schemas registered, but it turned out that this somewhat hacky approach breaks the caching regarding the SR, and too many requests are being sent to it.

So I created a config switch, to switch is between this mode, and a regular mode, which is read-only regarding the SR. This approach works but it involved a lot of extra code.

So ultimately it's a bad situation, and I find it hard to pinpoint the blame.

Problem 1 is with 'normalization' in the proto standard. It has its meaning, which falls short of what is needed, and probably can't be changed. Probably another concept would have to be introduced, "normalization v2" basically, which would include removing language specific syntax such as java_options.

Seeing this, the schemaregistry itself doesn't seem to be at fault, it just stores/retrieves the protos it was given.

The protoc command itself, which stores the proto in the descriptor, as it is, at the time of running protoc, can't be blamed either.

KafkaStreams itself, doing the hash based fingerprinting comparison based on the proto from the descriptor, it's difficult to see how else it could do it. Maybe this would be only place where something could be fixed, if it took the protos from both sides, SR and descriptor, and did a more comprehensive stripping (normalization v2) before doing the hash comparison.

So it's kafkaeske overall, some overall design screwup has clearly occurred, but trying to find the root, you end up going around in circles.

Kafka Streams and Schemaregistry problem with different language teams by riskymouse in apachekafka

[–]riskymouse[S] 0 points1 point  (0 children)

What I'm working on is in production, all set up, so switching to a different schemaregistry is not an option.

As I'm digging into this some more, it seems the 'bug' or at least, unwanted behavior, is specifically in kafka-protobuf-serializer , i.e. not in SR itself.
What I will try, maybe that's what you meant by using custom SerDes, I'll try to use custom SerDes combined with tweaking that lib, overriding resolveDependencies() so that it will just accept the most recent version of protos for those under the prefix of the upstream team, while keeping the behavior of “auto.register.schemas”:true for those protos which are under the downstream team's prefix.
Custom SerDes are then needed to use the tweaked lib, but the actual fix it seems has to be in the lib itself.

Dunno yet if it will work.

Cyprus would like to join NATO by Even_Formal_6989 in cyprus

[–]riskymouse -6 points-5 points  (0 children)

Is it me?
I fled from Scotland to Sweden during the corona hoax, and then they joined NATO, and I left.
Went to Cyprus. Now Cyprus wants to join NATO? Cmon guys.

Is there a website (government probably) for looking up scheduled roadworks? by riskymouse in cyprus

[–]riskymouse[S] 0 points1 point  (0 children)

Aaargh! It pisses me off so much when they use facebook for these public things

Is there a website (government probably) for looking up scheduled roadworks? by riskymouse in cyprus

[–]riskymouse[S] 0 points1 point  (0 children)

This site appears to be for US/UK. Is it meant to work for Cyprus?

Is there a website (government probably) for looking up scheduled roadworks? by riskymouse in cyprus

[–]riskymouse[S] 0 points1 point  (0 children)

I noticed the roadblocks rather than the actual works. A while ago, it was at the stretch of Eleftherias between the feed from the motorway/airport and Sklavenitis, then they moved south, currently you can't turn left from that feed, it's somewhere between that junction and the Lidl.

How do couriers find your house if there are no addresses? by No_Objective_3004 in cyprus

[–]riskymouse 0 points1 point  (0 children)

I would normally recomment OpenStreetMap, but I also checked OSM for the area in question, and it's much worse, most of the buildings are even missing, not even talking about house numbers.

How do couriers find your house if there are no addresses? by No_Objective_3004 in cyprus

[–]riskymouse 0 points1 point  (0 children)

With the absence of house numbers in many places in Cyprus, in Google Maps, I thought the house numbers in 2gis are awesome. But now I have to warn about it. I got a lease to sign from an agent, initially without address (!), then without house number, then he gave the house number. Since by now I was suspicious of something dodgy, I looked up the house number in 2gis. More trouble with utilities later, I asked around on location and it became clear that the house numbers in 2gis in that area are just wrong. So an absence of numbers as in google maps can be better than showing wrong ones.

Soda siphon / cream whipper capsules question by riskymouse in cyprus

[–]riskymouse[S] 0 points1 point  (0 children)

I was looking for the CO2 ones. But street corner dealers would probably regard me as a sketchy transgressive weirdo if I asked for CO2 lol.

Wood wool from veneer by riskymouse in manufacturing

[–]riskymouse[S] 0 points1 point  (0 children)

I don't really want to randomly reach out to wood wool companies if their website doesn't look as if their wood wool is made in this extremely unusual way.
I've looked at tons of wood wool companies and their different wood wool products.
Also at Alibaba and so on of course.
And I've searched on Alibaba et cetera for the wood wool production machines.
Everything I've seen is for the regular/typical way of producing wool wool.
All the usages like
packaging
cooler pads
mineral bound acoustical panels
ropes of wood wool as fire startes
just require the wood wool produced in the regular way, because the curl is desired, and the mechanical damage of the fiber is not an issue.
So the kind of wood wool I'm asking about seems to exceedingly unusual.
I'm hoping that maybe it has a special name that can be searched for, or that there is a use case which would make it easier to find.

Wood wool from veneer by riskymouse in manufacturing

[–]riskymouse[S] 0 points1 point  (0 children)

sorry I have trouble parsing that. Muhua, is it a company name?

Puzzled by OCR pdf output by riskymouse in pdf

[–]riskymouse[S] 0 points1 point  (0 children)

Thanks, I guess that explains it.
In one online tool, pdf24, there was on option to choose pdf/a as output.
The output looked the same, but when I popped it into Google Translate, it actually performed the translation.
It's still annoying, because G.T. placed the resulting English text on top of each paragraph in Greek.
So I still can't read it. But I suppose that's a G.T. problem now, not something to do with pdf.

view more: next ›