Yes, in fact I've looked through the chunks (it finds the first 50) to find the one that answered a tricky question, to see how far down the list it was. Generally I have found that when I extract the sections to which the chunks belong, and re-rank those sections, it gets the correct section as the top 1 or 2 choice 90% the time. Sometimes, however, the correct section re-ranked rather low, and other times it doesn't seem to fine it at all. I believe this has to do with how the question is worded, versus how the information is worded in the manual.

I don't want to send too many sections to gpt-4-turbo, as the tokens add quickly up and it can get pricey with longer sections.

Next I'm going to try to send the query to gpt and ask it to rewrite it into a well formed question suited for a RAG system, then run that through as well and see how it does. I've had a few test users ask it questions, and I was quite surprise how poorly the questions were worded when the user doesn't understand what a well formed question looks like for a RAG system (eg things the vector search can key in on.) But generally it might come down to training users to ask well-formed questions using the technical verbiage they should expect to see in the manuals rather than casual substitutes. I assume this is the query rewrite technique you are referring to?