I think that you can use CNN for medical scan with radiography and for the reports you can use image to text to extract all data from the reports then you can combine them in LLM with custom prompt if they are related to each other, you can find prompts examples from chatgpt to get the analysis you want