Network trunking
A common situation is that a instance needs access to several networks, such as a protected intranet and an internet-enabled network. A network trunk allows multiple networks to be connected to a single port (vNIC) on an virtual machine so that multiple networks can be presented to it.
Contents
Every trunk has a parent port and can have any number of subports. The parent port is the port that the trunk is associated with and the port used to launch an instance attached to the trunk.
Additional segmented networks can be dynamically attached to the trunk without disrupting operation of the instance. With a segmented network here means a VLAN-type network.
At a high level, the basic steps to launching an instance on a trunk are:
Create networks and subnets for the trunk and subports
Create the parent port and the trunk
Add subports to the trunk
Launch an instance on the trunk
These steps are described in the following sections.
Networks and subnets
Trunks are closely associated with VLAN, a technique used to control and segment networks. A network trunk is a link between two network devices that carry more than one network. Segregation of the Ethernet frames is achieved by appending a VLAN tag (or ID) to the frames passing through the trunk.
To create a trunk, a VLAN-type network with known ID should be available. The network configuration has to be prepared by a cloud administrator.
When creating a subport, segmentation-id
and segmentation-type
are specified arguments, where segmentation-type
is vlan and segmentation-id
is the VLAN ID by which the network is presented to the instance.
We assume that the trunk will connect to an internal network in the project and a provider network, called <trunked-network>
.
Given a VLAN with ID <vlan-id>
and a prescribed IP address range <trunked-ips>
, create a subnet for the subport with
openstack subnet create --subnet-range <trunked-ips> --dhcp --network <trunked-network> <name>
The internal network <internal-network>
and its corresponding subnet are set up as described in https://pannet.atlassian.net/wiki/spaces/DocEng/pages/1428390705/Basic+networking#Create-network-and-subnet
Create trunk
Create the parent port for the trunk with
openstack port create --network <internal-network> trunk-parent
where the port is called trunk-parent. This is the port that the instance will be launched on. Next, create the trunk resource using --parent-port
to reference this port:
openstack network trunk create --parent-port trunk-parent <trunk-name>
The details of the trunk is listed with
openstack network trunk show <trunk-name>
Add subport
A subport is created on the trunked network with
openstack port create --network <trunked-network> subport1
where the arbitrary subport name is subport1. This subport is added to an existing trunk with the command
openstack network trunk set --subport \ port=subport1,segmentation-type=vlan,segmentation-id=<vlan-id> \ trunk1
Alternatively, an existing subport can be assigned to the trunk already at its creation, with
openstack network trunk create \ --parent-port trunk-parent \ --subport port=subport1,segmentation-type=vlan,segmentation-id=<vlan-id> \ trunk1
Usually, one needs to find the port ID from openstack port list
when working with ports. If security is enabled on the subport (default), a suitable security group needs to be assigned to it with
openstack port set --security-group <security-group> <port-id>
Of course, the security group can be assigned directly when the port is created. A subport is detached from a trunk with
openstack network trunk unset --subport <subport> <trunk>
Launch instance on trunk
When the infrastructure is in place, an instance can be launched directly on the trunk by specifying the trunk parent port ID <port-id>
with
openstack server create --image <image> --flavor <flavor> --security-group <security-group> --key-name <key-name> --nic port-id=<port-id> <server-name>
The instance should launch normally, and it will have the private IP address of the trunk parent port.
Configure server interface
Next, the Ethernet interface on the instance needs to be configured to handle VLAN-tagged frames. At this point, it is not possible to assign a public IP address to the trunk port and access the instance directly through that. It should, however, be possible to access it through a bastion host or another instance with a floating IP using the SSH proxy command:
ssh -i <remote-key> -o ProxyCommand="ssh -i <proxy-key> -W %h:%p ubuntu@<proxy-ip>" ubuntu@<remote-ip>
where the instance having a floating IP is called proxy
and the target is called remote
. Once connected to the remote server, list its interfaces with ip a
. Figure 2 shows the interfaces on a server: the one of interest is the second, ens3
.
This interface will be treated as a trunk interface and configured to support VLAN tagging as long as the VLAN tag matches that of an associated subport. The MAC address of the subport is needed and can be found from
openstack port show <subport-1>
Figure 3 shows the MAC address as an attribute of the port, which is used in the example command below. A VLAN subinterface can be created with the ip link
command or by using netplan
. The ip link commands, with VLAN ID 1217 are
sudo ip link add link ens3 name ens3.1217 type vlan id 1217 sudo ip link set dev ens3.1217 address fa:16:3e:46:fb:42 sudo ip link set ens3.1217 up
The final step is to enable DHCP on the subport with
sudo dhclient ens3.1217
Figure 4 shows the interface before and after an IP address has been assigned to the subinterface.
The new configuration can now be seen with the ip addr
command (Figure 5).
Testing
To test the trunk it is convenient to have access to an instance on each of the networks connected to it. Firstly, the VLAN may not be accessible from the exterior, and secondly, for outgoing packets we need to verify that that the correct network is being used.
From the VLAN, the instance on the trunk should respond to ping on the private IP address of the subport. From the internal network, it should respond to the private IP address of the parent port.
From the instance itself, it should be possible to ping the private IP addresses of each of the servers on the respective network.
Note that the parent port does not have a public IP address, and so external traffic has to pass through an SNAT service or a proxy server. Note that pinging an external address will usually not work even when connected to a proxy server, and then it is better to use curl
for testing as described in https://pannet.atlassian.net/l/c/3YHZUisN
Trunk states
The trunk has a number of possible states, in short:
ACTIVE
- both the logical and physical resources have been created.DOWN
- the state of a trunk without an instance launched on it, or when the instance associated with the trunk has been deleted.DEGRADED
- a temporary failure state during the provisioning process. This includes situations where a subport add or remove operation fails. When in a degraded state, the trunk is still usable and some subports may be usable as well.ERROR
- indicates a conflict or error that cannot be fixed by retrying the request. This can happen when the network is not compatible with the trunk configuration, or the binding process leads to a persistent failure.BUILD
- a temporary state when the resources associated with the trunk are in the process of being provisioned. Once the trunk and all of the subports have been provisioned successfully, the trunk transitions toACTIVE
, otherwise toDEGRADED
orERROR
.
Heat template
Heat can be used to automate the trunk creation and launching an instance on it with a template like the one shown in the end of this section.
The template contains sections for the trunk and its ports, a security group and an instance called server0. A key pair, the networks and subnets are assumed to exist, and these resources are declared in the parameters
section.
There are a few notable details in this template:
Security groups must be assigned to the ports in the template (line 68 and 77 in the template). When the security group assignment declaration is put under the instance itself, the error message “ERROR: Cannot define the following properties at the same time: security_groups, networks/port.” is shown.
The instance must be created after the trunk has been provisioned. This dependency can be declared explicitly, with
depends_on
, or implicitly withget_attr
(line 96 in the template).The MAC addresses of the subports can be set to the MAC address of the parent port (line 74 in the template). In this case, there is no need to set the MAC address of the VLAN subinterface on the instance (see Figure 6).
The stack is created with
openstack stack create --template <template-name> <stack-name>
After successful creation of the stack, the instance needs to be configured as in Section https://pannet.atlassian.net/wiki/spaces/DocEng/pages/2055667717/Network+trunking#Configure-server-interface but without line 2 in the code snippet:
sudo ip link add link ens3 name ens3.<vlan-id> type vlan id <vlan-id> sudo ip link set ens3.<vlan-id> up sudo dhclient ens3.<vlan-id>
The events generated by the Senlin engine - the OpenStack clustering service - are displayed (see Figure 7) with the command
openstack stack event list <stack-name>
Template:
heat_template_version: 2018-08-31 description: Launch an instance with networks trunked over a port parameters: server_name: type: string description: Name of server default: server0 server_flavor: type: string description: Flavor server is based on default: g1.standard-1-1 server_key: type: string description: Key pair used by server default: common server_image: type: string description: Image server is based on default: ubuntu-20.04-x86_64 app_port: type: number description: Port used by the servers default: 80 internal_network: type: string description: Internal network used by server default: internal_network internal_subnet: type: string description: Subnet used by server default: internal_subnet vlan_network: type: string description: VLAN network used by server default: provider-network-vlan-1217 vlan_subnet: type: string description: Subnet used by server default: provider-subnet-1217 vlan_id: type: number default: 1217 resources: sec_group: type: OS::Neutron::SecurityGroup properties: rules: - remote_ip_prefix: 0.0.0.0/0 protocol: tcp port_range_min: { get_param: app_port } port_range_max: { get_param: app_port } - remote_ip_prefix: 0.0.0.0/0 protocol: tcp port_range_min: 22 port_range_max: 22 - remote_ip_prefix: 0.0.0.0/0 protocol: icmp parent_port: type: OS::Neutron::Port properties: network: { get_param: internal_network } security_groups: - default - { get_resource: sec_group } subport0: type: OS::Neutron::Port properties: network: { get_param: vlan_network } mac_address: { get_attr: [parent_port, mac_address] } security_groups: - default - { get_resource: sec_group } trunk0: type: OS::Neutron::Trunk properties: port: { get_resource: parent_port } sub_ports: - { port: { get_resource: subport0 }, segmentation_type: vlan, segmentation_id: { get_param: vlan_id } } instance0: type: OS::Nova::Server properties: name: {get_param: server_name} key_name: { get_param: server_key } flavor: { get_param: server_flavor } image: { get_param: server_image } networks: - { port: { get_attr: [trunk0, port_id] } } outputs: parent_port/name: description: Name of parent port value: { get_attr: [parent_port, name] } parent_port/mac_adress: description: MAC address of parent port value: { get_attr: [parent_port, mac_address] } parent_port/fixed_ips: description: IP address of parent port value: { get_attr: [parent_port, fixed_ips] } subport0/name: description: Name of subport value: { get_attr: [subport0, name] } subport0/mac_address: description: MAC address of subport value: { get_attr: [subport0, mac_address] } subport0/fixed_ips: description: IP address of subport value: { get_attr: [subport0, fixed_ips] }
Horizon trunk interface
There is a Horizon interface for trunk creation as well. This takes a configured parent port, creates a trunk and allows subsequent assignation of subports to it. Most of the configuration still has to be done manually as described in this chapter.
Additional resources
https://docs.openstack.org/neutron/stein/admin/config-trunking.html
https://docs.openstack.org/python-neutronclient/pike/cli/osc/v2/network-trunk.html