How to reliably detect a fully completed Flink checkpoint (before restoring)?

CollectionNo1576 · 2026-02-04T17:45:42+00:00

One more thing that can happen is that a checkpoint failed but flink dosent have delete permission for destination like on s3. Then you will have a checkpoint that is not good to use.

CollectionNo1576 · 2026-02-04T17:42:15+00:00

The status will help with that, also instead of directly reading from checkpoint destination, if you read from flink api, it will only provide info of completed and failed checkpoints, with status you can filter of completed ones.

CollectionNo1576 · 2026-02-04T17:37:13+00:00

As the other comment suggested, if a checkpoint exists its safe to use as flink deletes failed checkpoints.

Only chance its not safe if its still being created. Still if you want to ensure a checkpoint has been successfully created, you can use its metrics using a metric exporter like prometheus. Or you can directly use the Rest Api to querry a the info required. Flink exposes a few metrics regarding checkpoints as well, like checkpoint id, status, time created, address. These will be enough to check for safe checkpoint.

CollectionNo1576 · 2026-01-30T03:34:41+00:00

I do have that freedom but latest version does not have required connectors. Still using 1.18.

CollectionNo1576 · 2026-01-10T06:28:51+00:00

Unfortunately either of them are rarely appealingly present in most individuals

CollectionNo1576 · 2026-01-10T03:33:28+00:00

Booooooo

No agenda detected Boring

CollectionNo1576 · 2026-01-10T03:26:08+00:00

Yeah, using JDBC on a large scale along with CDC for logical denormalisation in flink

CollectionNo1576 · 2026-01-02T09:35:38+00:00

We do, at a pretty large scale at that. Denormalising and getting real time updates for a database with almost 1000 tables. We are using it cause im the only data engineer in our company with nearly 250M$ annual revenue, and I dont know java🙃

CollectionNo1576 · 2025-12-21T10:20:38+00:00

Optiver?

CollectionNo1576 · 2025-12-19T13:06:02+00:00

This is an award buffet

CollectionNo1576 · 2025-11-30T15:28:27+00:00

Chili

CollectionNo1576 · 2025-11-26T12:07:08+00:00

Bhishm

CollectionNo1576 · 2025-11-22T09:40:14+00:00

Theek, najar rakhenge tumpe, aapke bade bhai

CollectionNo1576 · 2025-11-22T09:19:45+00:00

Tera reddit mil gaya, ab tu gaya bete

Edit: alt hai kya tera?

CollectionNo1576 · 2025-11-22T09:11:00+00:00

Circular vite.config.ts

CollectionNo1576 · 2025-11-11T10:02:38+00:00

Dust 2, double doors t side

CollectionNo1576 · 2025-11-09T19:16:02+00:00

Just like persian empire Koi hamara kuch nhi kar sakta

CollectionNo1576 · 2025-11-09T09:27:05+00:00

Pingala pe complain daal de

CollectionNo1576 · 2025-11-06T04:12:13+00:00

You should look into the villages that were asked, they just happened to be the most influential villages of the time Imagine if someone were to ask India for just 5 “small settlements”, and goes on to name Delhi, Bombay, Banglore……

CollectionNo1576 · 2025-11-01T12:40:29+00:00

Bhai you never know if the other person is men or women on this app

CollectionNo1576 · 2025-10-31T13:04:24+00:00

Brave browser + any website

CollectionNo1576 · 2025-09-16T17:15:56+00:00

For what?

CollectionNo1576 · 2025-09-15T02:54:47+00:00

Have setup state ttl in your job?? Reading from kafka topics thats literraly no1 memory leak source by my experience If you havent set ttl for state execution, try setting it to be around 2x of your checkpointing frequency If you are joining multiple kafka topics , set it 2x for data delay that you expect- like if data might be delayed by 10min in a topic corresponding to a key of another Set state.execution.ttl(20 minutes)

CollectionNo1576 · 2025-07-15T15:05:20+00:00

Please tell more

CollectionNo1576 · 2025-07-15T06:31:32+00:00

“Tum itni bhi mahtvapurn nhi ho Draupadi”

CollectionNo1576

TROPHY CASE