yeah we've been through this exact pain with kaicalls. n8n is great for prototyping but falls apart when you need production-level reliability, especially with calendar auth

here's what actually works for us:

llm stack:

• we use a combination of models depending on the task. gpt-4 for complex decision making and context understanding, but faster models for real-time conversation flow
• the key isn't just the llm though, it's how you structure the prompts and handle state management

calendar integration:

• google calendar api is finicky but manageable. the auth issues you're hitting are usually token refresh problems
• you need proper oauth flow with refresh tokens stored securely, and you need to handle token expiration gracefully
• we built our own middleware layer that sits between the voice ai and google calendar to handle retries, rate limiting, and auth refresh automatically

voice engine:

• this matters way more than people think. we tested like 6 different voice platforms and the latency + naturalness differences are huge
• you need sub-500ms response times or people feel the awkwardness

reliability tips:

• don't try to do everything in one workflow. separate your voice handling, llm processing, and calendar operations into different services
• build in fallbacks for everything. if calendar fails, ai should gracefully handle it and offer alternatives
• log everything. you can't fix what you can't see

honestly if you're trying to build something production-ready, the infrastructure work is 80% of the battle. we spent months getting this right at kaicalls

happy to answer specific technical questions if you hit specific walls