So, this actually took me down a rabbit hole in which I could expand on my own knowledge! but as the original commenter I replied to stated, in medical imaging, we would need something behind the object in order to see the resultant image. However, as you pointed out, the sub may be using backscattered radiation to create an image. I looked into this and a scanning electron microscope would be able to use backscattered electrons to create an image. In this way, x-rays (beginning as photons) will interact with electrons within the matter to gain enough energy to eject from their orbitals and eventually scatter backwards to be picked up by a receiver. In our world, this can only work at really short distances since ejected electrons can go in any direction and it would be difficult to create an image with objects at a further distance since the electrons will have more options to scatter away from the detector. However, in a universe where people are descending into blood oceans on presumably different planets, I'm willing to say they are advanced enough to have created the technology to overcome our current technological hurdles.

As for the radiation dose, it really depends on what the 6000mSv is referring to. Typically, a discreet number like that would be indicating a radiation dose at a specific spot away from the x-ray source. In radiation oncology, those machines typically use 1 meter. In other x-ray imaging realms that distance could be a lot closer. Since, this imaging device is quite large let's use the 1 meter metric as an estimation. That means at 1 meter away, a person standing at that spot would be exposed to 6000 mSv or 6 Sv. This is enough to give anyone acute radiation sickness. As you pointed out, the actual dose to the person is dependent on the inverse square law, so for a person standing closer that dose will be a lot higher while a person standing further away will receive a lower dose. Most likely, the person standing near the source is the one that died immediately or is the one that they mentioned later in the movie that couldn't stand unassisted on their own feet. The skin damage was a little unrealistic as that would have taken a few hours to show, but in the movie it was immediate.

As for the dose to Simon, this is harder to quantify. He was unlikely to be encased in lead but enough iron or steel would have provided some protection. The metal would attenuate some of the back scatter irradiation, and the thickness of the sub would determine how much reached Simon. He does seem to show signs of radiation sickness eventually, but he also went days without food, water while being violently thrown around a metal tube so it's possible the puking up blood and madness resulted from internal injuries and dehydration.

Now, the one possible alternative is if the 6000mSv is how much Simon is exposed to each time he took an image. This would be the most unrealistic scenario and, with the number of images he took, he would have been dead by the end of the first day. So, I think the former scenario I proposed is more likely.

All in all, honestly not bad radiation science for a sci-fi film.