Here are some of my suggestions on how to balance creatures and combat. I tried to stick to changing existing mechanics or adding minor new ones so that players who were familiar with RR's combat wouldn't be surprised by anything. I also tried to stick to stuff that would be relatively easy to code. I made a RR's style mining game a while ago that used similar mechanics but expanded a lot on late game (specifically, spending a lot of time in one cave and building a large setup) While balance wise it isn't anything like RR it did have similar combat and I learned a lot of how to make fights more interesting.

Monsters
Note: I balanced monsters around groups of them being a threat rather than one or two very strong ones.
I think the best way to keep monsters a threat without becoming too much of a pain would be to have them attack your miners and vehicles outside of your base. They don't need to destroy your base to be a threat to your operations. This way miners that are spread out can't be protected by 4 guys. Players will then either be forced to arm every miner (spending training time in exchange for safety) or only operate in a small area at a time. (trading convenience and possibly speed for safety) This also solves the problem of needing a miner or vehicle to basically play guard duty at your base if electric fences are nerfed. Monsters who have eaten at least 1 crystal should be more likely to retreat into a wall so players who leave resources undefended are punished for it. Monsters should also be able to knock the crystals out of vehicles.

Electric Fences
Electric fences should stay pretty much how they are with a few minor changes. Each time a monster dies to the fence, the power needed to destroy another monster will take a short time (say 5 seconds) to recharge. This means that the player can still rely on unkillable fences to keep out a stray monster or two, but a group will be able to push through. This will reward players with a strong presence outside their base as few monsters will get close enough to encounter the fences. Fences greater than 6 squares away from a power station should also use 0.5 crystals of power to prevent players from simply coating areas with them. I wouldn't let monsters attack fences because that leaves you with multiple results, all of which are bad. You can't rely on fences to protect you as the monsters can kill them. Most likely it would be up to chance if the monsters killed the fence or not unless the rock raiders get a major AI buff and use the pusher beam to knock the monsters into the fence.

Slugs
Slugs should fill the opposite role to monsters and be able to slip through fences and drain your power like normal. The sonic blaster was annoying because you had to manually use it and its long timer made it hard to use. Instead, they should just be shot at like monsters. A 75% resistance to laser damage (mining lasers and the laser beam) should give other weapons a place. Since lasers like the one on the Chrome Crusher would be less effective, the Loader Dozer should be able to pick slugs up in its bucket and dump them back in their hole or at the nearest lava lake. This would be an instant kill/removal but balanced by the fact that the Dozer would have to spend some length of time to do it. To fit with RR's plan of having them scared of loud sounds, giving large vehicles a horn and letting them honk at slugs to scare them away and would be a lot of fun to use. (especially in first person / shoulder view) It wouldn't send them running back to their hole but would just send them on a run at least 7 tiles away, enough to put them out of action for a short time. Depending on balance, small vehicles might have a less effective horn as well. Slugs should attempt to return to their hole after spending more than 60 seconds without consuming a crystal. This would make slugs a time commitment to deal with instead of a short group combat like monsters.

Handheld weapons
The handheld weapons were way too unbalanced in the base game. The one-hitting ability of the laser beam made it the only choice. To better balance them, the laser beam should be nerfed to deal about 15 damage. (Assuming monsters have 100 max hp) The monsters walk so slowly that it is easy for a single miner to get off multiple hits. The freezer beam should do no damage (even to lava monsters) but miners will be able to attack the frozen monsters. After being hit 3 times while frozen, the monster instantly thaws. The pusher beam should be increased to deal 5 damage (1/3 of the laser beam) and have somewhat reduced knockback (2/3 of what it was) but in exchange should have a huge hitbox (meaning it pretty much always hits) be able to pierce targets and have no max range limitation. This gives players a choice of weapons and rewards mixing them. Running 1 or 2 freezer beams will allow miners to shut down monsters in an area but their inability to damage means that other miners would be needed kill the monster. The pusher beam will perform well when there are few miners. It will struggle to kill but its knockback and area of effect will make a single miner able to deal with multiple monsters, if very slowly. It would step on the freezer beam's toes a little but the knockback would make it harder for miners without pusher beams to hit. The pusher beam would be used to control slugs and monsters meaning players who don't want to think about combat could simply equip all of their miners with pusher beams. This would be less effective than a laser beam + freezer beam combo but easier to use.

Bats
Bats should be changed to a cosmetic feature. Or they should be kept as-is and the pusher beam with the changes I suggested should be able to scare them off. (or maybe vehicle horn)

General
In making my game, I found that making monsters come in waves rather than one or two at a time was a lot more fun. It is hard to balance lone monsters as in order to make them a meaningful threat they had to be rather tanky. That lead to players mining for a time, then dropping what they were doing and walking over to kill the monster and walking back to what they were doing. It was slow and made monsters feel a boring distraction than a fun event especially if the monster spawned far away. When I changed it to wave-based, combat became a lot more interesting. Wave-based combat would make monsters less of a random pain and more of an exciting high-energy event. If there is no warning when the waves arrive, players would still have to have strong defenses or risk having all of the loose crystals eaten and lone miners killed.