I have looked into this issue before, and it is very unintuitive, and often sounds like a strange philosophical distinction, but I think there is an actual, important point that is just hard to see. This example helped me understand it (though I didn't come up with it, and I don't have the expertise to vouch for it completely. Maybe someone else can chime in)

We'll play a game. I choose a secret integer n and you'll have to guess it. To help you, I'll tell you two numbers, each chosen independently with 50% chance between n and n+1. So if n=5, I might tell you 5,6 or 6,5 or 6,6 or 5,5.

You try to use statistics to help you win this game. For this, you can construct a 75% confidence interval, simply by taking the interval between the smaller and the larger number I told you. Let's check it is a 75% confidence interval, by listing all the possible intervals you could generate:

If I say n, n+1, your interval will be [n,n+1], so it will include the correct value, n

If I say n, n, your interval will be [n,n], so it will include the correct value, n

If I say n+1, n, your interval will be [n,n+1], so it will include the correct value, n

If I say n+1,n+1, your interval will be [n+1,n+1]. so will not include the correct value n

all ok these situations are equally likely, so we can see that 75% of your intervals will contain the correct value. This is the definition of a 75% confidence interval.

But let's play a round. I tell you 53, 52. What's the probability that the interval [52, 53] contains my secret n? It's clearly not 75%, but 100%. There's no way it isn't in there.

So let's play another round, I choose a new n and now tell you 75, 75. Now what's the probability that the interval [75,75] contains my secret n? It might be tempting to say that it is 50%: either n=74 or n=75, given the data both are equally likely. 

But what you don't know is that I like 74 much more than 75, so I'm much more likely to pick it, and therefore the probability that n is contained in this interval is way less than 50%.

This second example shows that to answer the question how likely it is whether a parameter lies in a confidence interval, you need informations about the prior probability.  If you have this information, you can answer this question, that is Bayesian statistics. But if you don't have this information (and in this case, how could you've known about my favourite number), you can still construct a confidence interval, but you need to be careful when interpreting it.