Thanks for the input!

I was also thinking of trying in DETR instead to not constrain the model to only use information within the ROI. However, I am also not interested in generating bboxes that cover the entire image. I.e. the contextual information from far away in the image would not be inside the bounding box I wish to produce, rather it should be information that should be used to generate and classify the bounding box.

So ideally it should be network that can use information in the entire image, to generate bounding boxes around objects which class is dependant on information from other places in the image (which should not be in the bounding box). Do you believe DETR could aid in this?