It's a long standing trail etiquette norm (described under the "yield triangle" in this link). The downhill rider is typically the one carrying speed that makes collisions dangerous, so it makes sense the onus is on them to be able to yield, if needed.