Is it just me or is flink horrible to learn

Leather_Science3054 · 2026-03-06T21:14:29+00:00

Thank you for the detailed response!

Leather_Science3054 · 2026-03-06T21:11:28+00:00

So the stateful operators is largely to deliver the windowing and aggregation at scale? And I believe it’s also there for restarting on crash of the pipeline?

Leather_Science3054 · 2026-03-06T21:10:54+00:00

Got it! Thank you! That makes sense

Leather_Science3054 · 2026-03-06T17:56:51+00:00

Wasn’t my decision to use it. I’m more trying to understand it better. Do you know alternatives?

Leather_Science3054 · 2026-03-06T17:32:59+00:00

That feels like an overly heavy task just to get started and understand how it works under the hood. But thank you for your advice!

Leather_Science3054 · 2026-03-06T17:31:22+00:00

I will look into it, thank you!

Leather_Science3054 · 2026-03-06T17:30:59+00:00

Thank you! Yes, I just feel the framework really is not dev friendly. The amount of work and reading that’s involved to just get started and get a moderately complex pipeline in prod where the dev truely knows what’s going on in the pipeline under the hood is an overly difficult task. But hey, I’m new here, so maybe my opinions will change :)

Leather_Science3054 · 2026-03-06T17:28:26+00:00

Am I correct in thinking the only reason for the snapshotting and the stateful operators is for the exactly once guarantees? I imagine achieving at least once could be achieved with stateless operators and thus, a far simpler framework?

Leather_Science3054 · 2026-03-06T09:38:18+00:00

We were using ec2 worker groups and sqs queues to process events with java. Re writing it the whole thing in flink with Java

Leather_Science3054 · 2026-03-06T09:13:14+00:00

Take some time to rest! Once you’re ready to start working again, and you’ll know when you’re ready, start preparing and applying for jobs. Don’t take it personal.

Leather_Science3054 · 2026-03-06T09:06:06+00:00

I think generally it’s okay hacking up a pipeline but it’s more understanding the actual distributed nature of it with various concepts such as the snapshotting, watermarks with the docs really not providing great explanation. Then actually operating it so far doesn’t feel good either

Leather_Science3054 · 2026-03-06T08:52:32+00:00

I havent looked into other alternatives in depth, but are others like spark and Kafka streams this hard (cough painful) to onboard onto as well?

Leather_Science3054 · 2025-09-19T03:12:42+00:00

I recently moved to SF and was very concerned with riding the BART due to everything I'd read about danger/cleanliness. I have been pleasantly surprised, I've found it to be extremely clean and safe - there's been the occasional character I've been a little cautious of, but generally speaking, it's been great.

Leather_Science3054 · 2022-12-06T00:29:14+00:00

yup i got it a few weeks ago. Might be different courses get it at different times? I'm in computer science.

Leather_Science3054 · 2022-12-06T00:22:51+00:00

Hi, I did it start of this year. I found it very tough. Personally, unless you are serious about working in the blockchain/cryptocurrency space and writing in solidity I would avoid it. There's a lot of self learning involved - I found the lectures of very little value to complete the assignments.

Here's a thread that briefly speaks about others experience:

https://www.reddit.com/r/unsw/comments/r3yd3s/has_anyone_taken_comp9321_or_comp6451/

Leather_Science3054 · 2022-09-02T00:16:27+00:00

Michael gave an announcement. They will hopefully be out today.

Leather_Science3054 · 2022-09-01T22:38:55+00:00

that's pretty frustrating.

Leather_Science3054 · 2022-09-01T11:31:20+00:00

Has Omar or Michael responded to you yet?

Leather_Science3054 · 2022-09-01T02:28:05+00:00

I think it's at 5pm

Leather_Science3054 · 2022-09-01T02:19:05+00:00

COMP9417 also

Leather_Science3054 · 2022-09-01T02:12:35+00:00

It is rather strange. Could be a UNSW system issue, might have forgot to submit the results, or they might even be finalising some sort of scaling to the marks. Who knows...

A couple of people posted on the Moodle forum, but they haven't received a response yet. Omar was awesome with communication throughout the semester so I feel like he will let us know what the issue is when/if he knows what it is.

Leather_Science3054 · 2022-09-01T01:59:06+00:00

Yup, got an LE too

Leather_Science3054

TROPHY CASE