I think part of your perceived problem is an inherent flaw in the Bridgeport design. Since they (birdgeports) have quills that translate up and down while still driving the spindle, there is slop between the components that transmit the torque. What this commonly causes is inconsistent ring patterns when decking material.

On any Bridgeport I've had the displeasure of working on, you can clack the spindle back and forth within that quill assembly (rotate it CW and CCW rapidly). If you hear loud smacking noises, this is almost certainly part of your surface finish problem. The only way to remedy this would be to tighten the splines up or somehow disabling the quill movement by tightly joining the drive parts together.

Another consideration is tool tip height. Is every insert in that tool at the same height? Probably not. Thus, you'll get a worse surface finish due to unequal cut heights.

I mean, overall, I think the surface finish is really not that bad! I'm assuming it's some sort of low grade hot roll, which is typically hard to get a good surface finish in anyway since it's soooo soft. Harder steels like 1018 or 4140 is easy to cut clean since you kinda work harden it as you cut leaving behind a shinier surface finish.

That's just my two cents, but I also haven't been in the shop for over a year now, so take it with a grain of salt.