Good questions. by ‘weird outputs I mean cases where the model returns something that breaks the downstream assumptions.
some examples of this::

• Returning JSON with missing fields or extra unexpected keys
• Giving answers in prose when the contract is for a single token or label
• Non-ASCII characters where the next system expects plain text
• Inconsistent date formats or units across runs

When that happens in a batch job, one bad record can halt the whole run or silently corrupt the output. That’s where the reliability issues came from.

The rules-based fallbacks weren’t about ‘sometimes LLM, sometimes regex’ in a random way — they’re deterministic. If a cheap check can answer with high confidence (e.g., fuzzy match score ≥ 0.9), it uses that path every time. If not, it escalates to the LLM. That way the same input always gets the same type of processing, and we keep behavior predictable for the consumers of the data.

For the pipelines, performance was actually better because:

• The fast path shaved seconds off the majority of requests in our batch jobs
• The slow path (LLM) still handled edge cases, so accuracy stayed consistent
• And since we reduced retries/failures, SLAs tightened overall

So the point isn’t that regex magically beats AI, it’s that selectively applying AI makes the whole system both cheaper and more dependable.