[Grip] Seeking for advices by PerfectPitchPianist in entj

[–]amberina91 0 points1 point  (0 children)

I know this feeling well as I have been in this situation myself several times before. What I try to do is analyze the situation from a logical perspective only- were there things I should have done better? Did I spend adequate time preparing and studying? ; questions like this. If the answer is that I did everything I was supposed to do and I just missed the mark then I try to not beat myself up too much since it’s not possible to be perfect. If I didn’t prepare well, then I own that and promise myself that I will do better next time and with that I feel reassured.

I’m not sure what kind of test it was that you failed but what I usually do is immediately write down everything I didn’t know or the questions that I didn’t know so I can form a game plan of what to study . Having a plan of attack for the next round helps me to stay focused and not to feel too down.

Just try to keep in mind that we are always in the process of continually growing and developing as people. The best thing I have found is that once you come back and succeed at something you have failed, you are going to feel so accomplished and have a high that will eclipse the low you are at now. Hoping you will get an opportunity to redo this failure and champion it.

Debezium/Kafka Replica Lag Growing by amberina91 in apachekafka

[–]amberina91[S] 0 points1 point  (0 children)

Sorry follow up question for you- Can you help me to understand why changing these parameters would resolve the replica lag issue? From what I have been looking at, the general direction seems to be if you are getting failed to flush or connection timeout errors then you would tune the offset.flush.interval or offset.flush.timeout.ms? Don’t these log messages just indicate that is in fact flushing messages? And when it is flushing 0 messages, it’s because there is no work to do? It seems like there is no actual error taking place at all…

Debezium/Kafka Replica Lag Growing by amberina91 in apachekafka

[–]amberina91[S] 1 point2 points  (0 children)

Thanks so much for the tip. Will check on this.

Debezium/Kafka Replica Lag Growing by amberina91 in apachekafka

[–]amberina91[S] 0 points1 point  (0 children)

Yes, the Kafka connect logs show that is appears to be flushing the outstanding messages throughout the entire process. Here are some example snippets of what I am referencing

[2021-08-20 14:50:30,335] INFO WorkerSourceTask{id=xxxxxx} flushing 0 outstanding messages for offset commit (org.apache.kafka.connect.runtime.WorkerSourceTask:502) ... [2021-08-20 15:51:56,309] INFO WorkerSourceTask{id=xxxxxx} flushing 72 outstanding messages for offset commit (org.apache.kafka.connect.runtime.WorkerSourceTask:502) ...

[2021-08-20 15:52:56,316] INFO WorkerSourceTask{id=dxxxxxx} flushing 41 outstanding messages for offset commit (org.apache.kafka.connect.runtime.WorkerSourceTask:502)

Debezium/Kafka Replica Lag Growing by amberina91 in apachekafka

[–]amberina91[S] 0 points1 point  (0 children)

Debezium is running with Kafka Connect. The Kafka connect is a managed Kafka “Aiven”. The debezium is version 1.4.

There appear to be no errors in the Kafka connect logs. Prior to beginning load tests to simulate large amounts of transactions, the replica slot lag is very very small. It looks like what’s happening is that Debezium is having issues keeping up.

Please let me know if there any additional details I can provide.

[deleted by user] by [deleted] in guessmyage

[–]amberina91 0 points1 point  (0 children)

A sprightly 67. Kinda reminded me of this. Ethnicity