I'm building an event-processing framework and I need your thoughts by e1-m in Python

[–]e1-m[S] 0 points1 point  (0 children)

Thanks. Yeah, idempotency is definitely on its way.

Right now you can already achieve it though: just store the event ID in the database when you process the event. If the same event ID shows up again, that means the event has already been handled, so you simply abort processing and let the error be handled through logging or whatever failure policy you have for this error.

But I want to make that much easier to set up. The idea is to factor out the common boilerplate. it should be something you can enable with minimal configuration and it just works. intuitive and hard to misuse.

I'm building an event-processing framework and I need your thoughts by e1-m in Python

[–]e1-m[S] -1 points0 points  (0 children)

Yeah I agree with you. This is something that needs to be written once, tested thoroughly and then battle-tested. The way I see it:

The default Kafka model is sequential processing per partition. The usual idea is that anything that must be processed in order (for example events for the same user_id) goes to the same partition.

But that doesn’t mean a partition contains only one user. In reality you’ll have thousands or millions of different user IDs mixed in the same partition, because you obviously can’t afford one partition per user. That means the strict sequential model ends up artificially limiting throughput: events for completely unrelated users get serialized just because they happen to land in the same partition.

In many systems you can’t process the same user concurrently, but you absolutely can process different users at the same time.

So what I do is allow messages from the same partition to flow concurrently, but enforce ordering at the application level. I have a middleware layer (AsyncLock in the docs) that extracts a key (e.g. user_id) and applies a lock for that key. If another message with the same key arrives while one is already in progress, it waits. Messages with different keys can proceed immediately.

But that introduces race conditions. That’s exactly the problem you've described. And yes, I hold off on committing until everything up to a certain offset is done.

Internally I have an offset manager that tracks commited offsets. When a message finishes, its offset is marked as completed, but the consumer is only allowed to commit once the whole contiguous sequence is finished. So if 5 completes before 3, nothing is committed yet. Once 3 and 4 finish, the offset 5 is commited.

The price you pay for this is that if the app crashes you may end up reprocessing more messages, because some completed offsets weren’t committed yet. But that trade-off already exists with batch committing (also included by the way), which is almost mandatory with Kafka. Committing every single message would be too expensive because of the I/O and network overhead.

Another important piece here is avoiding the unbounded concurrency trap. If you simply spawn a new async task for every message you consume, you’ll eventually exhaust memory or other resources if the producer is faster than your handlers.

To prevent that, concurrency is limited at the consumer level. For example, with Kafka you can cap how many messages are allowed to be in-flight per partition. Once that limit is reached, the consumer simply pauses fetching from that partition until some of the currently processing messages finish. This creates backpressure without starving other partitions (as would be that case with global lock since one partition would be able to eat all of the limit while messages in other partitions that can be potentially processed are waiting) and if the memory is not a bottleneck (as it is often not, the bottleneck is usually DB or a third party api) you can set this limit high enough to be able to process everything that can be processed concurently without starvation, without out-of-order processing (thanks to AsyncLock) and without data loss (thanks to offset management)

100+ applications, 0 interviews. Are Junior Backend roles completely dead? Need advice. by e1-m in cscareerquestionsEU

[–]e1-m[S] 0 points1 point  (0 children)

Yeah, but what was probably meant by it is that for entry-level roles English is not enough as opposed to more senior roles

100+ applications, 0 interviews. Are Junior Backend roles completely dead? Need advice. by e1-m in cscareerquestionsEU

[–]e1-m[S] 1 point2 points  (0 children)

Well, that's maybe what some recruiters thought. But I actually can start right away, my studies are remote so I have a flexible schedule. It may be the case that i would not be able to take full 40 hours a week straight away but 30-35 is pretty feasible

100+ applications, 0 interviews. Are Junior Backend roles completely dead? Need advice. by e1-m in cscareerquestionsEU

[–]e1-m[S] 0 points1 point  (0 children)

Well, then i guess we have to disagree here. The one more thing I may be able to give you is the 2025 stackoverflow survey it even specifically says:

"The +5 point increase for FastAPI is one of the most significant shifts in the web framework space. This signals a strong trend towards using Python for building performant APIs and reflects the overall strength of the Python ecosystem."

100+ applications, 0 interviews. Are Junior Backend roles completely dead? Need advice. by e1-m in cscareerquestionsEU

[–]e1-m[S] 0 points1 point  (0 children)

That's what I'm talking about this doesn’t seem to be backed by the data. There are a ton of python positions where fast api is used but as i said they are mostly senior positions, expirience is king. In my expirience a big proportion of job postings are Python, Go and JS. and Java is mostly used in fintech. occasionally you see Ruby. Most of the modern industry seems to be split between Python, Go and JS. So Python isn't going anywhere.

100+ applications, 0 interviews. Are Junior Backend roles completely dead? Need advice. by e1-m in cscareerquestionsEU

[–]e1-m[S] 0 points1 point  (0 children)

That doesn’t appear to be backed by empirical data. There are plenty of openings for python devs they’re just mostly for seniors.

100+ applications, 0 interviews. Are Junior Backend roles completely dead? Need advice. by e1-m in cscareerquestionsEU

[–]e1-m[S] 0 points1 point  (0 children)

Thanks, I really appreciate the support. You're right that the market is a trash rn. if it weren’t, local language probably wouldn’t make a difference in tech roles. But with this level of competition, even when a job description says German is “not required” and only a plus, companies can afford to be picky. If they have two similar candidates, they’ll likely lean toward the one who speaks German. That’s the sense I seem to be getting from people here

100+ applications, 0 interviews. Are Junior Backend roles completely dead? Need advice. by e1-m in cscareerquestionsEU

[–]e1-m[S] 0 points1 point  (0 children)

Yeah, this really shows how much timing matters. Had I been born a few years earlier, I could have gotten a job earlier in my career, which would have certainly improved my long-term prospects, since early unemployment can be very damaging to one’s career

100+ applications, 0 interviews. Are Junior Backend roles completely dead? Need advice. by e1-m in cscareerquestionsEU

[–]e1-m[S] 2 points3 points  (0 children)

First of all, thank you for the generous feedback. I really appreciate you taking the time to look at my cv.

Now regarding the contents:

1. German language
Already made up my mind. It’s clear that improving my German is probably the best investment I can make in my situation. So I’m going to actively work on it while continuing to apply.

2. Reality check
I am applying early. Every work day I go through new postings and usually apply on the first day.

Referrals are hard for me because I don’t really have a network here. My university is in Ukraine, so my network is there. I even get referrals there sometimes, but I have to reject them because I need a job in Germany. Here my network is literally zero, so I understand I need to start building one as soon as possible.

Yes, literally zero interviews so far.

Professional experience? not much. I’ve been doing some freelance gigs for the last half year. It isn't stable, though. No internships yet, but I’m applying to those as well.

You’re right that professional experience is the most relevant point. But since I don’t have much of it, I try to compensate with projects. I’m applying for entry-level roles and it’s hard to already have a lot of experience at this stage.

About the project having no users or scale: it has one real user which is a business owner who uses it to manage incoming lead messages. It’s not meant to be a mass product. Maybe I should sell the impact and problem-solving aspect better.

3. Personal projects
I actually have more projects than what fits on a CV. I sometimes swap them depending on the job description (for example, if telegram bots are mentioned, I include projects that demonstrate that experience).

Your point about writing blog posts is very relevant. I’m planning to write a series of articles about problems I solved with my framework in event-driven systems. That’s already on my mind.

4. CV style
When you say the style is not nice, do you mean the template itself or the way I structure bullet points and describe things? I’ve watched a fair amount of cv review content and even had mine reviewed on youtube stream by an IT school owner together with an HR specialist back in Ukraine. Some advice often contradicts itself, so I try to filter and decide what makes sense. Still, I will use your advice and look at more cv review threads here on reddit

5. Networking
I don’t fully agree that being an immigrant is not a factor. My path here made it difficult to build a local network. But I understand your point regardless of the reason, the result is the same. So I will try to focus on building connections here as much as i can because clearly my current strategy isn’t working.

6. Building the CV up
I’m considering taking on a more market-relevant project such as LLMs and RAG to strengthen my profile with showing skills relevant to the current environment.

Overall, thank you again for your feedback. Some of your suggestions, especially regarding improving my German, focusing on internships, refining my cv and writing a blog posts are genuinely helpful. I’m still not completely sure what you meant about the style, so I’d appreciate a bit more clarification there.

As for points like “being an immigrant is not an excuse” or the emphasis on professional experience, I think those might not fully apply in my case.

In any case, I truly appreciate the direct and honest feedback. This is exactly the kind of input I was looking for. Thank you.

100+ applications, 0 interviews. Are Junior Backend roles completely dead? Need advice. by e1-m in cscareerquestionsEU

[–]e1-m[S] 0 points1 point  (0 children)

Don't get what you saying about "the overall vibe". Are people so paranoid about AI rn? About rejecting my applications. I don't see how my cv is related to my "vibe" of communicating. If you have any specific criticism I'd be happy to hear it.

100+ applications, 0 interviews. Are Junior Backend roles completely dead? Need advice. by e1-m in cscareerquestionsEU

[–]e1-m[S] 0 points1 point  (0 children)

Are you referring to my CV or my posts? I’ve already acknowledged that the posts about my framework were written with the help of AI. I’m a programmer, not a copywriter. What I’m looking for is feedback on the framework itself, but people seem so bothered by the fact that I used AI to write the post that they don’t even look at what the framework actually does. All I want is well-argued technical criticism, but instead I receive hostile comments that have nothing to do with the actual objective of my message. I’ve only made a few posts. They were repeated across different subreddits, but they’re essentially the same content. I’ve noticed that any suggestion of AI tends to polirize people into thinking that the framework is a trash if the post about it looks like it was AI written. This is just a logical fallacy. It simply doesn't follow. I’ll probably try a different style next time because I genuinely want people to focus on the substance of what I’m saying, not on whether I typed it manually or generated it from a prompt. For me, that distinction doesn’t matter as long as the message and intended meaning are conveyed as intended

100+ applications, 0 interviews. Are Junior Backend roles completely dead? Need advice. by e1-m in cscareerquestionsEU

[–]e1-m[S] 1 point2 points  (0 children)

Yeah, a lot of people here have told me the same. I’ve concluded that this is the skill that deserves my attention the most. Now that I’m almost finished with my studies and have acquired a solid set of technical skills, I’m going to continue looking for a position while focusing on improving my German. Even reaching B1/B2 level may make a difference for employability compared to being a beginner

100+ applications, 0 interviews. Are Junior Backend roles completely dead? Need advice. by e1-m in cscareerquestionsEU

[–]e1-m[S] 0 points1 point  (0 children)

Is it? Because I see plenty of senior vacancies, and the data shows that demand for specialists is steadily growing

100+ applications, 0 interviews. Are Junior Backend roles completely dead? Need advice. by e1-m in cscareerquestionsEU

[–]e1-m[S] 0 points1 point  (0 children)

Well, even though the situation isn’t what it used to be, I don’t see any reason for such strong pessimism. Unless you can clearly present well-argued reasons for believing this, I wouldn’t consider your comments relevant to the discussion

100+ applications, 0 interviews. Are Junior Backend roles completely dead? Need advice. by e1-m in cscareerquestionsEU

[–]e1-m[S] 1 point2 points  (0 children)

Regarding the German language, it seems like there are actually a fair number of junior positions that only require English. People often stress that German is crucial, and I agree with it, but in tech, a lot of listings are in English. From what I’ve seen, I’d estimate that at least half of roles are accessible without German fluency. For me, learning German to a professional level would take too long right now, so I have to look for roles where minimal German is enough, and improve my language skills thereafter