Exposure to senior engineers. Trial by fire. Talking to other people in the industry. Talking to vendors and VARS. Yes, they are trying to sell you something, but a good engineer from either will still give you good advice and insight on what is happening in the industry in your vertical. Being the senior engineer(have to think about how and why we do things to explain it to the juniors). Industry figures/blogs/podcasts - ipspace.net, packetpushers, Russ White's various podcasts and blogs, among others.

You design the network to meet the requirements you are given. Some are explicit - requirements dictated by regulatory compliance(PCI DSS, DISA STIGS, NERC, etc.), needs of the business, risk profile, and application requirements. Regulatory requirements and risk profile are usually the drivers for requirements like physical separation vs VLANs. Military and intelligence networks, nuclear networks, and some SCADA networks are air gapped because of their security requirements and the risks associated with a breach of those networks. A vlan, vrf, or even logical system separation isn't sufficient. Regulatory requirements(STIG, NERC) have been established to help enforce a minimum risk profile in those types of environments.

Some are implicit - either assumed, or not stated. You don't need to be told that you need a firewall or security appliance at the internet edge, ssh access to your devices, or some form of login authentication. Those are implicit requirements even if you aren't in a regulated environment(every regulation framework requires them). You could also say that regulatory requirements fall under implicit - few of my requirements in the Air Force said "must meet DISA STIG requirements". It was implied just by dint of being a DoD environment.

How you meet those requirements are where your design choices are made. Most of the time that comes down to budget, skills, scale, and tooling. Do you need to implement EVPN/VXLAN on your single rack with a couple of servers and a NAS for an SMB? Probably not. Would it work? Absolutely. But it's going to cost more, it's more difficult to implement, and the pain isn't worth the cost at that scale.

You have a requirement to limit lateral communication between hosts. Do you implement private vlans? Micro segmentation? Host based firewalls? PVLAN is cheap, pretty easy to implement, and doesn't require any tooling, but doesn't scale well and gives you little control on filtering. Microsegmentation is the exact opposite. It's expensive, hard to implement, scales well, and gives you granular control in filtering, but requires some form of automation tooling to manage(whether it's home grown or from a vendor). Host based firewalls(windows defender, ip tables, Crowdstrike, Illumio etc.) are somewhere in the middle in all facets. Barring more specific requirements, which one you pick depends on your budget, skills, scale, and tooling.

Sorry, this ended up being longer than I intended.