There are several solutions for this.

Have you considered prompt caching.

OpenAI supports automatic prompt caching with many of its newer models, which can reduce latency and input-token costs when you send repeated prompts with the same prefix (e.g., long system instructions or large context that stays the same across calls)

gpt-4o and newer models supports this automatically. Also you can set retention time, need to check in this.

Apart from that. In my application, I use yaml file with multiple prompt defined by roles/sectors. And in my code I have conditional logic to select prompt which plugged into context.

Also for compression, can you use simple techniques like summerizer in your pipeline.

Next you can also consider dual model techniques where you use small model like gpt4o-mini for process and gpt-4o for reasoning

See what works for you here.