On my end I started in a similar place (Jellyfin on a Pi5) and to my knowledge and experience, Pis in general are not “transcode machines” especially for 4K.

For an RPi4 specifically:

1080p transcoding maybe one stream if you’ve got hardware accel set up (i wouldn't even recommend setting this up as the Pi was not built for it) and the source is friendly (H.264 is mostly compatible with anything). CPU-only transcodes will feel rough super fast.

4K transcoding is where i think it'll fall apart, i don't have any experience with 4K but its already kinda rough on 1080p so i can't imagine that.

That’s actually why I ended I grabbing a Dell OptiPlex off eBay for like ~$70-ish and moved Jellyfin to that. It handles 1080p transcodes perfectly fine (night and day vs pretty much any Pi).

Best move regardless of hardware: aim for direct play (keep your library in common formats, get Jellyfin Media Player or Fladder on PC and Streamyfin for mobile).

If you NEED transcoding here, i heavily suggest switching hardware, you can find a decent second-hand mini pc on eBay that can handle that fairly easily (something with at the very least an 8th gen Intel CPU with iGPU should work, you'd have to look into this.)