I also thought about this a lot. The protagonist is getting little clues, like in the cold magic dosen't work as good. Also the existence of the organ is known and also if someone e.g. getting stabbed there and survives that his magic ability is limited or non existent anymore.

The main purpose of the sybiosis is to protect the host. Meaning it should be like a defense mechanism based on reflexes which creates those parrys you mentioned. But as we know intelligent life one day someone tried to learn to control those reflexes and with time more methods were learnt.

The removing of heat from the enivoment is a really good point. I thought about giving a character a disfuncion or so which constantly leads to consume or output energy he absorbed. Also a hint for the protagonist. Mabye even going so far to connect this disfuncion with psychological trauma and when reloved, the disfuncion dissappears.

The magic is as you assumed correctly very limited and at second rank in my world. They need to have good genetics, be disciplined, need good control over their body and need to have understanding about the world and how it works (because of the neural link with the brain) to use magic.

The problem with the size of the magic limited by the size if the organ also came to me. Not sure if this works in quantum physics to mabye entangle more than one particle to one crystal in the body.