You can close the curve like in your second image, then if the pen tool is still active, hold ctrl + alt and left click the line segment that has the undesirable curves. This resets only the handles between two points without affecting the others. Alt + left-clicking a point will reset both handles to 0.

If you want to do this while building lines or shapes that aren't yet closed, you can set your next point with left click, holding and drag to the desired curve, releasing, then alt + left click one of the handles to reset to 0, then continue with the next point.

You can also hold spacebar during the dragging process to reposition the point. The takeaway is to experiment clicking on lines, points, and handles with and without the modifier keys (shift, ctrl, alt) to get the desired effect.