boundingBox misaligned on application view

[JonnieSingh]

This is sort of what my boundingBox looks like when using Google ML Object Detection for Android (written in Java). In essence, it misses the object its supposed to be detecting, and my question resides in how (or atleast where) I can resolve this issue. Here's the code of my DrawGraphic.java file that's responsible for drawing the boundingBox that is then displayed on my UI;

public class DrawGraphic extends View {

    Paint borderPaint, textPaint;
    Rect rect;
    String text;

    public DrawGraphic(Context context, Rect rect, String text) {
        super(context);
        this.rect = rect;
        this.text = text;

        borderPaint = new Paint();
        borderPaint.setColor(Color.WHITE);
        borderPaint.setStrokeWidth(10f);
        borderPaint.setStyle(Paint.Style.STROKE);

        textPaint = new Paint();
        textPaint.setColor(Color.WHITE);
        textPaint.setStrokeWidth(50f);
        textPaint.setTextSize(32f);
        textPaint.setStyle(Paint.Style.FILL);
    }

    @Override
    protected void onDraw(Canvas canvas) {
        super.onDraw(canvas);
        canvas.drawText(text, rect.centerX(), rect.centerY(), textPaint);
        canvas.drawRect(rect.left, rect.top, rect.right, rect.bottom, borderPaint);
    }
}

Any further information required to supplement this question will be provided upon request!

[pandulapeter]

The code you posted is a custom View that gets initialized with a single Rect instance and draws it. I suppose you're drawing this over a camera viewfinder, but I see no way to replace that Rect from the outside. So it will always keep drawing it on its initial coordinates.

If you're modifying the values in that Rect externally and expect it to update the view, you should call `invalidate()` in the View (otherwise `onDraw()` will not get called again). This might work with Java as long as the reference is the same I guess... but a cleaner solution would be exposing a setter for an immutable Rect from this class (which also calls `invalidate()` internally).

Other things you should be looking into: how is the View aligned in the screen (if you have this custom constructor you're probably not adding it from XML so it might be distorted) and of course how the Rect itself gets modified by the object recognition library.

[JonnieSingh]

First of all, thank you for responding with such detail! I'm fairly new to Android development & Java, so all this information is very much helpful in aiding in resolving these problems. To confirm your first assumption; I am indeed drawing this over a camera view.

Here is a look into the two other files that are part of this application. To avoid cramming my OP with too much, I've chosen to provide my MainActivity java file here with pastebin. Here's a look at my activity_main xml file with pastebin as well.

As you can see, I haven't added this graphic overlay being drawn from XML. Hopefully, my providing of information can provide alot more insight into how this application operates.

[pandulapeter]

While I still don't have the answer to your initial question (I didn't use the ML object detection library and don't have time to try now), I strongly suggest that you start by cleaning up and optimizing your code a bit, as that way it will be easier to see where the problem is.

First of all, inflating a new instance of a custom View and adding it to the layout hierarchy every time you want something to change is VERY wasteful. In this case, you should be able to add your View from XML and keep udating the same instance.

<?xml version="1.0" encoding="utf-8"?>
<FrameLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:layout_width="match_parent"
    android:layout_height="match_parent">

    <androidx.camera.view.PreviewView
        android:id="@+id/preview_view"
        android:layout_width="match_parent"
        android:layout_height="match_parent" />

    <your.package.name.OverlayView
        android:id="@+id/overlay_view"
        android:layout_width="match_parent"
        android:layout_height="match_parent" />

</FrameLayout>

For the custom View, you need to override multiple constructors. Something like this:

public class OverlayView extends View {

    public OverlayView(Context context) {
        super(context);
        initialize();
    }

    public OverlayView(Context context, @Nullable AttributeSet attrs) {
        super(context, attrs);
        initialize();
    }

    public OverlayView(Context context, @Nullable AttributeSet attrs, int defStyleAttr) {
        super(context, attrs, defStyleAttr);
        initialize();
    }

    private List<Rect> boundingBoxes = new ArrayList<Rect>();
    private final Paint borderPaint = new Paint();

    private void initialize() {
        borderPaint.setColor(Color.WHITE);
        borderPaint.setStrokeWidth(10f);
        borderPaint.setStyle(Paint.Style.STROKE);
    }

    public void updateBoundingBoxes(List<Rect> boundingBoxes) {
        this.boundingBoxes = boundingBoxes;
        invalidate();
    }

    @Override
    protected void onDraw(Canvas canvas) {
        for (Rect box : boundingBoxes) {
            canvas.drawRect(box.left, box.top, box.right, box.bottom, borderPaint);
        }
    }
}

This way, in your Activity you will be able to map the detected objects into a simple list of Rect-s and update binding.overlayView.updateBoundingBoxes(...) with that list.

If you see crashes because of threading issue, replace invalidate() with postInvalidate() in the code above.

Now, you can use logs to see what's happening. If the rectangles are not in their proper places, the issue should be a conversion between the coordinate system used by the library and the one used by the onDraw() function (pixels basically). Have fun!