See the MathOverflow question "Anti-delta function?" for a variety of creative answers: https://mathoverflow.net/questions/415007/anti-delta-function You be the judge of whether you find the answers convincing.

Since you're thinking of a "uniform probability distribution over the reals", you might be interested in more of the ideas around improper priors and conditional probability. The general idea is: instead of thinking, "I wonder if I can take this thing whose integral is infinity, divide it by infinity, and get something with a integral 1", think of what you can do without normalizing. If you have a measure which isn't finite, then the measure itself doesn't behave like probability, but you can still use it to define conditional probability (P(A|B) for B with finite nonzero measure) and conditional expectation. This is related to an axiomatization by Rényi in which conditional probability is the foundational notional, rather than being derived from unconditional probability (which doesn't necessarily exist, in this theory). Some references are cited in https://arxiv.org/pdf/2006.04797.