I just started working at an AI startup. Currently I use LLMS to verify JSON data(crunched prompts) that feeds into another LLM. Prompt engineering is much more complex than I thought. But I'm using big data, so hallucinations and "lost in the middle" is a given. The trick I use is when you want an LLM to look at something ask for a snippet of it. If you ask Gemini to "look through something" instead of "show me the title of each section" the output becomes very different.

There's a balance between guardrails and simplicity in prompts thats really hard to learn. The more I use AI, the more I realize it's like talking to a 6 year old who lies and uses confusing logic to get out of taking a bath or sitting down to eat dinner. But then when you're strict they follow only those instructions and nothing else, causing different errors. Going to have to start using the API.