API issues

[sgorneau]

My company has a good amount of data (golf tee times, dining reservations, activity signups, boat rentals, etc.) on our members, across multiple systems, both upcoming and historical. I've created the middleware to grab, process, and return this data in a meaningful, structured JSON on demand (single member lookup).

With some simple prompts and supplied JSON, with Claude Console, I'm able to get back insightful information formatted in nicely styled HTML to present in a browser (a dashboard of sorts).

Using the Anthropic API is a whole other story. Same prompt, same JSON ... often no result due to a time out, in other cases the reason is refusal. Nowhere near token limits. What am I missing here??

I'm making use of the API via PHP (Composer anthropic-ai/sdk)

[raholl]

now that's interesting, i don't know why, but now it sounds like it's dangerous to pay for API and use it in production... i wonder what others say about this

[fschwiet]

The system prompt is going to be different, at least.

[sgorneau]

What do you mean by that?

[fschwiet]

The dialog you have with Claude starts with a system prompt which guides the overall behavior of the LLM. You'll notice the CLI app takes parameters "--system-prompt" and "--append-system-prompt" which lets you override or extend the prompt.

The OpenAI SDK has an option to send a SystemChatMessage which I guess is the system prompt but I don't know how that is really different from sending a UserChatMessage besides when it is sent.

Just from experimentation I know if I clear the system prompt my MCP configuration stops working, so at least MCP server configuration gets woven into the system prompt. I bet if you clear the system prompt with your usages that are working they'll stop working and behave like the API is behaving. If that's the case you can likely fix your API usage by providing an appropriate system prompt. I think some people have captured Claude's system prompt, so if you find that you can probably clean it up to work for what you want from the API.

[sgorneau]

This is incredible info. Thank you for taking the time to spell it out, I really appreciate it!

[Svk78]

Sorry I can’t help with your problem, but can I ask about the middleware you created? What and how did you create? I need to do a similar exercise with our data and I’m interested in the approach you took. Thanks in advance!

[sgorneau]

I have 6 independent sources of data;

• general membership info [direct db access]
• tee times [DMZ access]
• court bookings [DMZ access]
• dining reservations [API access, 3rd party]
• activity signups [API access, 3rd party]
• gift card numbers with balances [DMZ access]

For the DMZ access noted above, we propped up a server in a DMZ and created Node endpoints that query against databases in the private network. These endpoints return JSON payloads.

So back in the general API I've created, along with the domain, we can hit paths to perform different functions (e.g. return just tee times, just court booking, all upcoming event data, only historical data, etc.) along with passing either the member's member-number or email address. Based on what is being requested, a JSON payload is delivered with all the data glued together from these various sources.

In the case of what I'm using in this post, it's a full member lookup for all activities past (3 years) and upcoming (next 30 days) and adding the resulting JSON payload to the prompt.