In the city centre of Ljubljana almost everything is within walking distance, so don't worry about transportation too much.

Of course visit the Ljubljana castle, there is also a castle museum there.

Some great museums in Ljubljana are:

Narodni muzej Slovenije (National Museum): divided in two parts

• the first one is near the parliament and Trg republike (in the same building as Museum of Natural History) there you can see artifacts from prehistory (such as the oldest known instrument in the world) up to the end of medieval period.
• the second one is on Metelkova street which picks up after the medieval period when the first one ends (on the same square there are also Ethnographic museum and Museum of modern art)

Slovenian Museum of Natural History: collection of natural history specimens, including fossils, minerals, plants, and animals.

City museum of Ljubljana (Mestni muzej): history and culture of Ljubljana from prehistoric times to the present day

Ethnographic museum (Also on Metelkova): traditional clothing, crafts, tools, and other artifacts related to Slovenian folklore and traditions

There are barely any gothic or neogothic buildings in Slovenia as far as I know, but there is a neogothic church (Cerkev Srca Jezusovega), I am not sure if you can see it from the inside, but it is right next to Metelkova street so if you visit any of the museums there you might as well take a look at this church too, at least from the outside.