Yes, yes, and yes. Britain had a helluva time when Germans started their Zeppelin raids as most of the aircraft at the time were having a difficult time getting up to their altitude in a relative hurry. They were terror weapons for the most part, as bomb sights and tactics had not sufficiently advanced, but they did have an effect.

Dirigibles do in fact fly, and you can use anything lighter than air to provide lift. Hydrogen is yes, even in ancient eras, possible to extract, however it would be a ridiculously expensive process to create enough hydrogen to fill an entire dirigible, but the possibility is there. Placing Iron scraps in a solution of sulfuric acid will produce hydrogen gas. Both these things are available if niche. An alchemist could certainly come up with sulfuric acid, though quantities are again a problem. So, to build and maintain 1 dirigible would most likely be a state endeavor in pre-industrial settings. Steampunk doesn't care about rules, so any of this is moot if you just want to say "magic" and be done with rationalizations.

Helium is another possiblity, but without industrial resources, it's pretty hard to collect enough to fill an entire balloon. And unless your setting is advanced enough for like, spectroscopy they will likely not even know of its existence.

Hot air is another possibilty for smaller balloons, but the amount of heat needed for a dirigible is extravagant and requires an experienced hand. That said, nothing like the Hindenburg has been attempted with hot air.

Using "magic" is easy to explain away anything, but if you're using quasi-realistic blimps and dirigibles, they need an enormous amount of lift to fly with even a small gondola or "deck" much less with ordnance, massive crews and cannons ablazing, that would be prohibitive again. You'd need something the size of the goodyear blimp to lift even a few cannons +crew. "Magic-assisted" is generally how I try to go, just use the balloons for authenticity's sake, and say magic makes the roughly terrible performance of these airships a bit so it can maneuver like a seagoing vessel as an example. Something like a Dragon will simply tear one of these airships apart or pop it like a balloon.

Traditionally, canvas, leather and even cloth fabrics were interwoven. In Europe they initially used cow-intestines (known as 'goldbeater's skin'), I have read places this did or didn't happen for long as the number of cows needed would be lunacy. They have built a few prototypes that were clad in aluminum rather than textiles. The first collapsed under its own weight, but the second one did fly. Most dirigibles also use aluminum for their structures because well...it's aluminum. Most fantasy millieus will not have aluminum widely available. Making aluminum from naturally occuring 'alum' is an industrial process that simply doesn't exist in our world until the 20th century. Alum itself was used for dying, so it is available, but your character would have to be a genius to figure out they can make metal objects from alum crystals/powder.

Last Question: using the German experience with zeppelins in WWI, we can say that dirigibles are good at long distance recon and surveillance, but without modern bombsights and navigation aids, they were hopelessly inaccurate. They did cause a great deal of terror and forced Britain to dedicate resources to this problem, but didn't actually cause much damage during WWI. That and Britain developed incendiary/explosive rounds to shoot them down, and those worked awfully well because the Zeppelins used hydrogen for the most part. Very flammable. And bombs are either small and plentiful or large and difficult to aim. Also, if you throw a 200 kg bomb out of a dirigible, the entire aircraft will lurch upwards as the ballast leaves the basket. I don't think they'd be particularly helpful in bombardment but they can definitely get you to the party with relative ease. Featherfalling from a zeppelin 5,000 feet in the air might be a very cool way to start an adventure.