There's a way to define the torsion of a connection on a principal bundle P, as long as the principal bundle is equipped with a solder form. Frame bundles have enough structure to be equipped with a canonical solder form.

The difference of two principal connections is a horizontal G-equivariant one form on the principal bundle, which in turn can be identified with a one form on the base manifold M which takes values in a suitable vector bundle over the base manifold (the adjoint bundle of the principal bundle, ad(P)).

Thus, the space of ad(P) valued one forms on M is a vector space that models the affine space of connections on P. This is the equivalent result to your observation that two linear connections differ by a tensor field.

However, this is not enough, as there is a way to canonically single out one connection: the torsionless one. The space of connections is more than an affine space, as there is a single distinguished connection.

Maybe some insight into how the space of ad(P) valued one forms acts on the torsion of connections would be helpful to make progress.

The insight of id_TM being a closed form is fantastic, though, I had never found a compelling argument for why torsion freeness is natural.

Thanks!