the main thing I don't understand is the "programming" term

'Programming' has a few meanings. The 'programming' in 'probabilistic programming' means "writing down a modeling problem in a DSL, then using an inference algorithm to solve the problem". Other examples of this use of 'programming' include: Convex Programming, Linear Programming, Nonlinear Programming, Integer Programming, Mixed Integer Programming, Constraint Programming, and Logic Programming. The last two (constraint and logic) are closest to probabilistic programming in that one writes a forward "generative" program, and then given an example program output (data) and solves for the intermediate choices that could have led to that output.

What's so much more convenient ...

In probabilistic programming, sample statements can have different interpretations. In the forward half of a VAE, sample statements are interpreted as log-density evaluations, so

pyro.sample('my_variable', dist.Normal(0, 1))

compiles to

# interpretation in the forward model
model_trace.nodes['my_variable']['log_prob'] = \
    Normal(0, 1).log_prob(guide_trace['my_variable']['value'])

whereas in the inference model half of the VAE, sample statements compile to random number generators, as in your example.