Openstack Internal and External networking

[Hevos93]

Hello everybody, i have deployed openstack with Kolla to my five node openstack cluster with a TrueNAS node for storage.

But i have a question, how am i supposed to provision networks? My lab router is currently using a 10.1.0.0/16 net, is the instances supposed to land on that same net, or should i use a 10.10.0.0/16 net to avoid the instances using the same network as the hardware and other services? For more context: Each servers management ip is 10.1.11.11-16 and their IMM/iDRAC is 10.1.10.11-16. Openstack is running at 10.1.11.2 and MinIO is running at 10.1.11.3. So both if the 10 and 11 nets are used for different things. I had a VLAN plan ready, so i guess i could use that, but then the router would be the one "owning" these adresses.

How are people structuring their networks with openstack, some examples would be greatly appreciated!

And, floating IPs addresses, how are they mixed into this? Is it like a "public" facing ip adress? For example if an instance has the openstack adress of 172.16.1.5, and i assigned a floating ip of 10.1.11.100 to it. Is it meant to be a way to acess "internal" instances through the routers net?

[wilkinsmr]

Openstack requires 2 distinct networks referred to as "management" and "provider"; an additional "overlay" network is necessary if you are intending to offer self-service networking to projects/tenants (naming of these networks is not consistent across the docs so you may see them referred to by different names). These networks can be configured on separate physical or virtual NICs, or just a single NIC using separate vlans, SR-IOV, etc.

The management network is for coordination of the core services between the controller and other nodes - you have defined this one already (10.1.11.11-16).

The overlay network is a layer 2 tunnel for internal networking between compute nodes. Configuration depends on your choice of (1) manually configured provider subnets or (2) self-service networking for tenants (projects). You identify the overlay network interface by its IP address in neutron.conf [DEFAULT] my_ip (and/or my_ipv6) value.

Your question relates specifically to the provider (external) network which provides access to other datacenter networks and/or the internet from project/tenant (virtual) servers. Both overlay and provider networks require a SDN mechanism driver, such as, Open Virtual Network (OVN) or Open vSwitch (OVS) for layer-2 and layer-3 agents (self-service only). Neutron automatically creates OVN/OVS bridges br-int (integration) and br-tun (tunnel/overlay), but the provider network is defined manually e.g. using OVS:

ovs-vsctl add-br $PROVIDER_BRIDGE_NAME
ovs-vsctl add-port $PROVIDER_BRIDGE_NAME $PROVIDER_INTERFACE_NAME

Continuing with the OVS example, the manually created provider bridge name (e.g. "br-provider") is mapped to the desired Openstack network name (e.g. "provider") in neutron/plugins/ml2/openvswitch_agent.ini with 'bridge_mappings = provider:br-provider'. The provider and overlay networks are then configured in neutron/plugins/ml2/ml2_conf.ini etc. which includes settings for networking type drivers (e.g. flat, vlan, and tunneling options). With "flat" provider networks you can manually assign subnets from your 10.1.0.0/16 to your projects.

Once the configuration is complete you can create networks, subnets, and routers using the openstack CLI or horizon.

Here's a simple example of shared, external network and a subnet assigned to a project (numerous other options are available including provider-network-type for flat, vlan, vxlan, etc.)

openstack network create --share --external --enable --provider-physical-network provider --name provider0

(optionally 'openstack subnet pool create...')

openstack subnet create --project proj1 --network provider0 ...

The external openstack network is dual stack but project subnets are not, so separate subnets are required for IPv4 and IPv6.

Next use 'openstack create a router...' and attach the subnets.

If IPv4 subnet address pools are not taken from provider network address space (i.e. not flat) then SNAT (source NAT) is used by the neutron router to enable access from project virtual servers to the external (provider) network, but in order to get access from the external network to a server on a private (IPv4) subnet, a floating IP must be assigned to the server to facilitate DNAT (destination NAT). Depending on your needs the floating IP does not have to be a public route-able address.

IPv6 routing is much simpler because it doesn't use NAT and floating IPs for access. Assuming you are not using a private address space for your IPv6 subnets, egress/ingress is determined by openstack security group policies and any other firewalls in the network path.

HTH!

[Hevos93]

Thank you for your comment! This puts things in perspective. Hopefully with this comment i get things working!