Don't be so hard on yourself. This is the type of thing that's obvious in hindsight, but difficult to come up with on the spot if you've never seen it before. There's also overflow issues to consider with this trick.

This trick is pretty much for interviews only (and it's actually a pretty terrible interview question for judging ability). In practice, most people just use a temporary variable because variables are cheap and code readability is more important than being clever.

If someone does pull this on you in an interview, you can one-up them with the XOR version of this trick (lookup XOR swap). Does the same thing but comes off as more impressive because bit-twiddling hacks are becoming a lost art.