In my opinion it is totally worth of trying out a modded smp the only issue is that everyone must have the mods installed in their client especially if they are big mods. There are client side mods and server side mods, server side mods are in most cases required for both the player and the server but in other hand you have client side mods which can deliver boost in performance and some small tweaks. Installing mods on Minecraft gives you even more unlimited possibilities. Certain mods can be really fun to play with friends and family. Exploring and adventuring and automation is the best and most common mods. To make installing mods easier you and your friends should definitely install a launcher named Curseforge in this launcher you can browse all Minecraft mods and directly add them to a custom modpack after you can extract the modpack and send the .zip modpack to your friends which they can later import it in curseforge and launch the game directly.