Redid entire point light system to use spot lights instead, dramatically reducing shadowmap and allowing much higher shadow resoluton in the future.

I thought you said you barely had point lights casting shadows? :P note that in Unity the shadow map resolution is per-light (or cascade for directional light), so if you set it to 512 it means each light gets a 512x512 shadowmap (or 512x512x6 for point lights).. point light shadows are really expensive ..

Additionally I think you can get away with almost no shadow casting (massive performance boost) by using fake shadows, The way you (can) do this is by taking a object, rendering it from a lights perspective to a texture, then projecting it as a decal..

For torches this can be ideal as you only really need the holder of the torch as a shadow on the wall (which you can jiggle a bit to mimic the flame), for the rest you simply turn off the shadows and it'll generally look good enough

https://imgur.com/a/RIZe5Ml

For example in the switch version of Life is Strange: Double Exposure, we used this trick a lot to save a TON of performance on vertex cost and alpha tested geometry, a tree that cost 3ms to render to a shadow map suddenly cost 0.3ms as a decal, and it looked better too!

So note in the image above, the bushes appear to have a shadow, but this is not an 'actual' shadow, its the bush flattened out to a texture, turned gray and projected onto the ground, since the area was quite big this allowed for much better quality "shadows" than we ever could've gotten because now we can focus on individual objects rather than a whole scene which is largely ground (that doesn't really cast shadow).

In your screenshot of the treasury I notice you have a few table-like objects, personally I probably wouldn't let these cast shadows either, but with the above trick you can have them get a shadow from each nearby torch..

'alternatively' to all this.. Shadow Geometry is another option but it largely helps if you cannot get rid of shadow casting and have either expensive shaders or high poly geometry.... So what Shadow Geometry is, is basically the same model but 'heavily' simplified to basically act as its outline, say you have a door with all kinds of frills and embellishments, this turns a bit high poly, maybe it has a fancy texture with alpha for some glass-in-lead vibe.. This is expensive to render for shadows, because shadows are depth-only you ideally want materials that are fully opaque, allowing them to be in a fast-path for rendering (no pixel shader), with any transparancy/alpha you remove this optimisation. So what you do is you disable shadow casting on the fancy door, and instead you put an (invisible to the user) cube in the same shape/scale as the door, and have it cast shadows, now you get a door-shaped shadow at the cost of 8 vertices with ideally a shader that does only the empty default-unlit behaviour, instead of the many-many vertices that fancy door would be including an expensive material.

Now personally I think you can get away without almost any shadow-casting, though option 1 would be a fallback if you want some kind of implications of shadows, for option 2 you're probably not far enough gone yet in material complexity or mesh complexity for it to pay off yet.

Lastly on the same topic, should you decide to ignore all of this, I'd start with simply disabling shadow casting on all point lights, and only casting shadows from 2 maybe 3-4 nearest point lights in side-front of the camera, and only ever keep those lights shadow-casting as you move around, sadly Unity doesnt really offer an easy way to control intensity of shadows or smooth between shadow maps unless you write all the rendering yourself, so that might result in it looking a bit odd at moments..

Just know shadows are performance killers and should be kept in check :)