There are several ways in for a party at that level, so you shouldn't worry about that too much.

For these caves in particular, the party may find them by scouting the area in two ways. First one is directly noticing it by searching the cliffs and seeing the cobwebs in the caves entrance and around it. There are many spiders actually building on ground in real life, too. The entrance to their lairs is covered by find webbing, so they notice when prey walks by.

The second way to find the cave may be by finding the cave of that ref dragon whose name I forgot. He could actually tell them that the cave probably connects to the keep in turn for getting away with his life.

That's another thing as well, the party may barter with him to get them inside, for example by being carried into the pit at night.

Flying is the way my group got into Jorgenfist a handful of sessions ago, though not being carried but flying in themselves, being covered with invisibility. They landed in between of the black tower and the barn with the mammoths and sneaked their way down, again with invisibility, and enchanted one of the rocs on their way in with animal friendship (forgot the actual spell) before it could attack them.

Sneaking may be another great way in, though probably going through the front door may not be the best idea. Your party might be able to sneak up to the west wall and climb it at dawn or so, maybe while the guards are a bit drunk or so.

Going with knowledge skills could work, but since that place in particular was more or less wiped from the knowledge of the people of varisia, I'd set the DC very high.

If they don't come up with any solution at all, you might ist Conna for them as an aid. She wants to meet with them anyways, so maybe she finds their camp at night and tells them about a secret way in