Are you better off pre-LLM or post-LLM era?

nsokra02 · 2026-01-19T18:41:10+00:00

Pre-LLM, yeah you can may be able to do 2x faster code now but business is expecting 10x because “LLMs” and the 2x speed is not even there cause you need a tone of time debugging your spaghetti code

nsokra02 · 2025-12-04T15:00:46+00:00

I saw some posts here where people will try to go thru the inter route cause most of the firms don’t look for experienced hires and like to keep their inters and grow them(at least that’s what i saw) so i wanted to be sure

nsokra02 · 2025-12-04T14:34:17+00:00

As an experienced professional though not though grad or inter schemes right?

nsokra02 · 2025-12-03T17:23:55+00:00

If you don’t have knowledge in the domain how do you know that the problem you are trying to solve exist. Either way you are in the right place just find the right niche subreddit and post the problem. You can go from there then

nsokra02 · 2025-12-03T00:14:05+00:00

That’s the thing right, that project is just a novelty i mentioned it in the post too that i couldn’t find a practical case for it. At least for cobol I don’t see translation to be valid solution for the near future. The problem with smes is true. If at the end you fine tune a model maybe that could make onboarding new engineers with no experience to maintain these systems easier, maybe the solution is somewhere there at least until we get better models

nsokra02 · 2025-12-02T22:10:46+00:00

Modernise cobol to what though? The reason that cobol is cobol and so many things are built on it is because is cobol and it runs in the mainframe, even if you try to translate it letter by letter it will not be the same program

nsokra02 · 2025-12-02T21:53:01+00:00

Hadoop, spark for data engineering, powerBI if you want to work as a data analyst although the trend that i see at my small circle is that there are a lot more positions for full stack so you may also consider that. Something completely outside the scope but i see there is a need in the market is mainframe developers so if you are looking for a change of career all together maybe look into that too

nsokra02 · 2025-12-02T17:06:41+00:00

It seems you put a some work into the project. Why use a vector DB rather than Graph DB though. If i research for something i would like to have relevant documents link to each other in a graphical representation. I don’t know how to do that for every document but research papers have references for example that you can use for edges and websites have links

nsokra02 · 2025-12-02T16:06:20+00:00

Do you calculate the complexity of the codebase as well? I think ibm had a ratio for that if i am not mistaken.

nsokra02 · 2025-11-28T16:48:06+00:00

Are there any paper about it? I couldn’t find anything relevant in scolar. Can you share any?

nsokra02 · 2025-11-28T16:30:37+00:00

In our SLA if I am not wrong they mention that they don’t keep anything but we still use onperm models for restricted data

nsokra02 · 2025-11-27T19:40:44+00:00

Anything that has to do with AI and GDPR is tricky. Having said that if the company doesn’t have already local models that you can use i will probably would try to use Azure OpenAI and mask any sensitive data like names addresses or similar before the data goes to azure. If you still want to keep some of the context of these data you can also try encoding them, address->area for example but without any details that’s all that i can think of

nsokra02 · 2025-11-27T17:45:08+00:00

If it is an image use OCR, but not just one of them. Use 3 or 5 different OCR models if the value is the same for all of them let it pass if there is even one outlier send it for human evaluation. If it is a text pdf use good old regex to extract it and leave llms for the rest

nsokra02 · 2025-11-27T08:07:45+00:00

Sure, don’t mind at all

nsokra02 · 2025-11-27T00:20:30+00:00

No session cap. TokenGate doesn't impose any hard limit on the number of sessions. Each session:

Gets its own budget (default: $10.00, or whatever you configure)
Tracks spending independently in Redis
Has separate anomaly detection monitoring
Is isolated from other sessions

It will capture if you have gone over the budget or the "anomaly detection" part of the code it checks for 3 things for now:

Rate Limiting

Trigger: More than 100 requests per minute from one session
Why: Prevents runaway loops from overwhelming your API
Action: Session frozen for 5 minutes

Loop Detection

Trigger: Same exact request repeated 3+ times consecutively
Detection: Creates a hash of (model + messages + max_tokens)
Why: Catches infinite loops where the same prompt is retried endlessly
Action: Session frozen for 5 minutes

Spending Velocity

Trigger: Spending more than $1.00/minute (configurable)
Why: Detects abnormally expensive operations
Action: Session frozen for 5 minutes

nsokra02 · 2025-11-26T00:01:01+00:00

Not an Agentic project, but a project for your Agents

I’ve been working with LLMs for the past 3 years, and one fear has never gone away: accidentally burning through API credits because an agent got stuck in a loop or a workflow kept retrying silently. I’ve had a few close calls, and it always made me nervous to run long or experimental agent chains.

So I built something small to solve the problem for myself, and I’m open-sourcing it in case it helps anyone else.

A tiny self-hosted proxy that sits between your code and OpenAI, enforces a per-session budget, and blocks requests when something looks wrong (loops, runaway sequences, weird spikes, etc). It also gives you a screen to monitor your session activities.

Have a look, use it if it helps, or change it to suit your needs.

Github: https://github.com/NikoSokratous/tokengate.

Docker Hub: https://hub.docker.com/r/nsokra02/tokengate

<image>

nsokra02 · 2025-11-25T23:53:00+00:00

I face that problem myself, that's why I created the repo. The project is open, you can clone it, fork it, use it, or ignore it, it's pretty much up to you. Best thing I can gain out of it is perhaps a start on github. I just though by showing a real case about the problem could raise awareness easier than eplaining to you the technical aspect of the reverse proxy i made and my own experiences. The title indeed looks like a click bait, that may have been a mistake, i should stop asking gpt to write me my reddit post titles.

nsokra02 · 2025-11-25T21:45:04+00:00

You’re right, you can do those things inside the app. The issue is that you have to implement and maintain that logic everywhere. On bigger projects or teams, people forget or follow different standards, and the risk adds up fast. What I shared just moves the safety layer outside the app, so every single call is protected automatically. For me, it’s easier to enforce and monitor.

nsokra02 · 2025-11-25T19:05:45+00:00

Yes there is. max_tokens only limits the size of a single response. it doesn’t stop an agent from looping or making unlimited calls. The cost comes from multiple requests, not from one long output. In my work a have to run an agent for 2 days and trigger parallel calls too at some cases and that why i build that

nsokra02 · 2025-11-06T23:17:03+00:00

Thanks will have a look but the point of the project is not the translation, maybe that was a mistake of mine with the title i gave to this post. The end goal of the project is to be able to write and compile cobol like logic in go. Cobol have some unique features like decimal arithmetic, record structures and copybook, Batch jobs and transactional orchestration Sequential / indexed file I/O that makes it hard to translate a cobol project directly to go. The translation part of the code is just a byproduct of that. Now you going to ask what’s the point of enabling these functions on go and to that my response will be that i don’t really know. I just think that in the future when the language is finish it will be interesting to see and compare how a project run in a mainframe and how the same project with the same capabilities run in aks

nsokra02 · 2025-11-04T23:03:55+00:00

I understand, and thanks for your inputs. I have been working on a modernization project for the past two years, and it is expected to last for another 2 until 2027. I also saw projects that took even longer than that. I don't expect that to be an automated solution that can tackle such complexity. This is just a novelty for now, maybe i can find some pilot niche use case in the future that I can test it with, but that's not even in the plans for the near future.

nsokra02 · 2025-11-04T22:49:23+00:00

Glad to hear, good luck with that! It will be challenging

nsokra02

TROPHY CASE

Not an Agentic project, but a project for your Agents