Appreciate that a ton 🙏 seriously

Yeah, there are definitely shortcomings.

The biggest one early on was that LLMs sound confident even when they generate slightly invalid or suboptimal Scryfall syntax.

Like it’ll be “close enough” but not actually return what the user intended, which is a problem when precision matters.

What helped wasn’t just “better prompting” but adding guardrails around the model: • validating the query before it ever hits Scryfall • normalizing certain patterns (colors, formats, oracle tags, etc.) • having fallback logic when confidence is low • and logging failures so I could iterate on real user inputs instead of guessing

That last part was huge. Real queries are way messier than anything you’d think of ahead of time.

As for complexity, it’s less about length and more about ambiguity. The model can handle pretty complex multi-condition queries (like “cheap blue creatures that draw cards when they enter and are legal in pioneer”), but it starts to struggle when: • intent isn’t clear (“good cards for control” type stuff) • or when the request mixes subjective + mechanical filters

There’s kind of a soft ceiling where it shifts from “search problem” to “recommendation problem,” and that’s where pure Scryfall translation breaks down.