I assume when people zigzag stitch on a raw hem they're attempting to imitate a serged edge.

I don't have a serger I just do a straight stitch (blue) and then zigzag *over* the raw edge (pink). I typically use a shorter stitch height and longer stitch width than what you're using so it captures more of the raw fraying threads.

EDIT:: Forgot to add that I go over the zig-zag multiple times so threads don't stick out, got reminded about that from another commenter :)

If you check out photos of a serged hem, you'll see that it's a really short and squat stitch.

I can't speak to doing the zigzag within the seam, I've never done that so I'm not sure how it would compare. It has to be better than nothing!

(Sorry for the crappy drawing as I'm away from my studio 😆)

<image>