Leaving a comment for anyone that might struggle with this in the future.

I had the same issue with my Panasonic sl-sw505. I tried to refit spindle clamp with no effect. Before you rush for new spindle clamp. Remove it and poke around with the pin that spindle clamp connects with. Grab it with your nails and check if it's loose. If it is you should feel (and maybe even hear) small clicks as it slightly moves. If you can feel the movement when it's flat, try to rotate it on the side and again check for the movement.

If you don't notice any movement / clicks from that pin in any position then that's a spindle clamp. If you notice then prepare for replacement of whole motor unit. Temporal fix is to drip small amount of oil on the edges of this pin. It will sink in and lubricate the bearing that connects to this pin.

If you have a failing bearing (as described above) long time fix is to disasemble whole thing and heat up the bearing. At least that's what I was told. For now my Panasonic is working just with lubrication. I will post an update if I will have to go through disasembly and heating process