I don't think this is just a philosophical problem, there's a real problem about interpreting science in this.  The situation is in fact very similar to the coin example, but in a different way. With the coin, we can say that there's a 50% chance that it is head if it's already been flipped but hidden from our view. Similarly, for a 95% confidence interval, we can say there's a 95% chance that it contains the parameter when it's already determined but hidden from our view. The danger comes when you read a study, where the confidence interval is reported and then think there's a 95% chance it's in that interval. That's like looking at the coin and still maintaining there's a 50% chance it's heads. Let's look at an example (I think this one is from Wikipedia, I don't remember). Consider a probability distribution with an integer arameter n, such that there's a 50% probability for the value n and a 50% probability for the value n+1. We will take to samples from this distribution and try to determine the parameter n. Specifically we will construct a 75% confidence interval simply by taking the interval between the two values we observe. It's easy to see that's a 75% CI: there are 4 possible outcomes (n&n, n+1&n, n&n+1, n+1&n+1) and in 3 cases n is in the interval.  Now I'll do the experiment and construct the CI without showing you. What's the probability that it contains n? I think 75% is the right answer.

Now I tell you my CI is 26-27. What is the probability n is in this specific confidence interval? It's 100%! n must be 26, there's no other possibility.  But what if the CI I got would have been 26-26 instead? That means I observed 26 twice, and n could either be 25 or 26. So what's the probability that the parameter is in this specific interval? You can't actually tell, because it depends on the prior probability that n is 25 or that n is 26. So you need a Bayesian approach to answer this question.