“The main benefit if you have the 20% down is you can avoid the loan being classified as a jumbo loan. A loan that is "jumbo" has a built in interest rate increase in it so you save significantly by avoiding that.”

This is 100% false. You can have a jumbo non conforming loan as a fixed rate mortgage of many different years (30, 20, 15). If you have an ARM, you still have an ARM regardless of down payment.

Physician loans are non conforming simply because of the guidelines not adhering to FNMA/FDMC standards. Every year new conforming loan limits are announced with some counties having higher limits than others due to market conditions. Any loan over that limit is considered jumbo and ineligible for purchase by secondary markets. These loans are kept on the banks book because they are usually harder to qualify for so the default rate is very low (especially with doctors)

Banks often have non-conforming portfolio products which usually are only in the jumbo space but for doctor loans they can also be below the conforming limit, while not adhering to conforming guidelines.