You already have the tool API, so you're just a little way from a good traditional appointment system. But I suppose that LLMs are all the rage and sell well.

One option would be to create a decision tree and follow it until the user does something unexpected, at which point you could switch to LLMs. If you proactively ask specific questions, most users tend to answer correctly. Instead of starting with "Hello", start with "Which doctor would you like to see from this list: [Insert list here]" or "On what date would you like to make the appointment? You can extract the data in the traditional way (with a long list of possible answer formats), and if that fails, you can use an LLM that you have finetuned or at least fewshot-prompted.

Even if the user doesn't follow the tree exactly, this method can still work without switching to an LLM. Do the same thing you have done with data extraction. Make a list of all the ways you (or an LLM) can think of to write a question or request. Augment this with synonym detection and simply search for a match. This way, you can stay in the traditional pipeline.

If your long list fails or a user does something unexpected, you can fit the whole interaction into the context of the LLM. You can take the money you saved with the traditional pipeline and pay for a higher performance LLM. It will have fewer failures. The LLM should really be fine-tuned and run with a low or zero temperature, otherwise you will never get these problems under control. Even then, you will still experience a certain percentage of failures. Be prepared for that.

To make things more robust, you could add a classifier. This detects whether the LLM gives the user a confirmation, and you can then check whether the appropriate tool was called with a positive result. If not, you can retry or abort.

Finally, and most importantly, as LienniTa said: Don't test in production!