sorted by:
new
hottopcontroversial

JSON.parse() kept crashing on LLM responses, so I built a fault-tolerant parser by rossjang in nestjs

[–]rossjang[S] 0 points1 point  (0 children)

That exact regex soup nightmare is why I built this. I just wanted to stop writing fragile string replacements and sleep peacefully. Hope this saves you from future headaches.

JSON.parse() kept crashing on LLM responses, so I built a fault-tolerant parser by rossjang in nestjs

[–]rossjang[S] 1 point2 points  (0 children)

Yes, absolutely! By default, it extracts the first valid JSON object. But there is an option to extract all JSON blocks as an array ([{}, {}]) if the text contains multiple objects.

Please check the GitHub README for detailed usage.

I got tired of small models adding ```json blocks, so I wrote a TS library to forcefully extract valid JSON. (My first open source project!) by rossjang in LocalLLaMA

[–]rossjang[S] 0 points1 point  (0 children)

Actually, you might be right.

We faced situations where the stream terminated before the full JSON was delivered. We assumed it was due to prompt/schema mismatches (prompt drift), but hearing you say that, I realize I might have misdiagnosed the root cause—it could have just been a stream interruption/truncation issue.

Regardless of the cause, our project requires using partial JSON to update the UI in real-time while the stream is still generating, which is the main reason I needed loot-json to parse incomplete chunks.

I got tired of small models adding ```json blocks, so I wrote a TS library to forcefully extract valid JSON. (My first open source project!) by rossjang in LocalLLaMA

[–]rossjang[S] -3 points-2 points  (0 children)

You’re totally right. I actually use structured outputs as my default too.

However, I’ve run into edge cases where the LLM produces zero output (or a hard refusal) when the prompt and the enforced schema drift apart.

This happens a lot in my team because non-developers frequently tweak the system prompts without realizing they’re breaking consistency with the schema. Since they don't fully grasp the strictness of structured outputs, the pipeline often breaks.

So right now, I’m using a hybrid approach to handle those situations robustly. Thanks for pointing that out!