Network trunking

A common situation is that a instance needs access to several networks, such as a protected intranet and an internet-enabled network. A network trunk allows multiple networks to be connected to a single port (vNIC) on an virtual machine so that multiple networks can be presented to it.

Contents

Every trunk has a parent port and can have any number of subports. The parent port is the port that the trunk is associated with and the port used to launch an instance attached to the trunk.

Additional segmented networks can be dynamically attached to the trunk without disrupting operation of the instance. With a segmented network here means a VLAN-type network.

Figure 1. Schematic function of a network trunk.

At a high level, the basic steps to launching an instance on a trunk are:

Create networks and subnets for the trunk and subports
Create the parent port and the trunk
Add subports to the trunk
Launch an instance on the trunk

These steps are described in the following sections.

Networks and subnets

Trunks are closely associated with VLAN, a technique used to control and segment networks. A network trunk is a link between two network devices that carry more than one network. Segregation of the Ethernet frames is achieved by appending a VLAN tag (or ID) to the frames passing through the trunk.

To create a trunk, a VLAN-type network with known ID should be available. The network configuration has to be prepared by a cloud administrator.

When creating a subport, segmentation-id and segmentation-type are specified arguments, where segmentation-type is vlan and segmentation-id is the VLAN ID by which the network is presented to the instance.

We assume that the trunk will connect to an internal network in the project and a provider network, called <trunked-network>.

Given a VLAN with ID <vlan-id> and a prescribed IP address range <trunked-ips>, create a subnet for the subport with

openstack subnet create --subnet-range <trunked-ips> --dhcp --network <trunked-network> <name>

The internal network <internal-network> and its corresponding subnet are set up as described in https://pannet.atlassian.net/wiki/spaces/DocEng/pages/1428390705/Basic+networking#Create-network-and-subnet

Create trunk

Create the parent port for the trunk with

openstack port create --network <internal-network> trunk-parent

where the port is called trunk-parent. This is the port that the instance will be launched on. Next, create the trunk resource using --parent-port to reference this port:

openstack network trunk create --parent-port trunk-parent <trunk-name>

The details of the trunk is listed with

openstack network trunk show <trunk-name>

Add subport

A subport is created on the trunked network with

openstack port create --network <trunked-network> subport1

where the arbitrary subport name is subport1. This subport is added to an existing trunk with the command

openstack network trunk set --subport \
  port=subport1,segmentation-type=vlan,segmentation-id=<vlan-id> \
  trunk1

Alternatively, an existing subport can be assigned to the trunk already at its creation, with

openstack network trunk create \
  --parent-port trunk-parent \
  --subport port=subport1,segmentation-type=vlan,segmentation-id=<vlan-id> \
  trunk1

Usually, one needs to find the port ID from openstack port list when working with ports. If security is enabled on the subport (default), a suitable security group needs to be assigned to it with

openstack port set --security-group <security-group> <port-id>

Of course, the security group can be assigned directly when the port is created. A subport is detached from a trunk with

openstack network trunk unset --subport <subport> <trunk>

Launch instance on trunk

When the infrastructure is in place, an instance can be launched directly on the trunk by specifying the trunk parent port ID <port-id> with

openstack server create --image <image> --flavor <flavor> --security-group <security-group> --key-name <key-name> --nic port-id=<port-id> <server-name>

The instance should launch normally, and it will have the private IP address of the trunk parent port.

Configure server interface

Next, the Ethernet interface on the instance needs to be configured to handle VLAN-tagged frames. At this point, it is not possible to assign a public IP address to the trunk port and access the instance directly through that. It should, however, be possible to access it through a bastion host or another instance with a floating IP using the SSH proxy command:

ssh -i <remote-key> -o ProxyCommand="ssh -i <proxy-key> -W %h:%p ubuntu@<proxy-ip>" ubuntu@<remote-ip>

where the instance having a floating IP is called proxy and the target is called remote. Once connected to the remote server, list its interfaces with ip a. Figure 2 shows the interfaces on a server: the one of interest is the second, ens3.

Figure 2. Ethernet interfaces on instance launched on the trunk.

This interface will be treated as a trunk interface and configured to support VLAN tagging as long as the VLAN tag matches that of an associated subport. The MAC address of the subport is needed and can be found from

openstack port show <subport-1>

Figure 3. MAC address of the subport.

Figure 3 shows the MAC address as an attribute of the port, which is used in the example command below. A VLAN subinterface can be created with the ip link command or by using netplan. The ip link commands, with VLAN ID 1217 are

sudo ip link add link ens3 name ens3.1217 type vlan id 1217
sudo ip link set dev ens3.1217 address fa:16:3e:46:fb:42
sudo ip link set ens3.1217 up

The final step is to enable DHCP on the subport with

sudo dhclient ens3.1217

Figure 4 shows the interface before and after an IP address has been assigned to the subinterface.

Figure 4. Enabling DHCP on the subinterface.

The new configuration can now be seen with the ip addr command (Figure 5).

Figure 5. Configured interfaces on instance launched on the trunk.

Testing

To test the trunk it is convenient to have access to an instance on each of the networks connected to it. Firstly, the VLAN may not be accessible from the exterior, and secondly, for outgoing packets we need to verify that that the correct network is being used.

From the VLAN, the instance on the trunk should respond to ping on the private IP address of the subport. From the internal network, it should respond to the private IP address of the parent port.

From the instance itself, it should be possible to ping the private IP addresses of each of the servers on the respective network.

Note that the parent port does not have a public IP address, and so external traffic has to pass through an SNAT service or a proxy server. Note that pinging an external address will usually not work even when connected to a proxy server, and then it is better to use curl for testing as described in https://pannet.atlassian.net/l/c/3YHZUisN

Trunk states

The trunk has a number of possible states, in short:

ACTIVE - both the logical and physical resources have been created.
DOWN - the state of a trunk without an instance launched on it, or when the instance associated with the trunk has been deleted.
DEGRADED - a temporary failure state during the provisioning process. This includes situations where a subport add or remove operation fails. When in a degraded state, the trunk is still usable and some subports may be usable as well.
ERROR - indicates a conflict or error that cannot be fixed by retrying the request. This can happen when the network is not compatible with the trunk configuration, or the binding process leads to a persistent failure.
BUILD - a temporary state when the resources associated with the trunk are in the process of being provisioned. Once the trunk and all of the subports have been provisioned successfully, the trunk transitions to ACTIVE, otherwise to DEGRADED or ERROR.

Heat template

Heat can be used to automate the trunk creation and launching an instance on it with a template like the one shown in the end of this section.

The template contains sections for the trunk and its ports, a security group and an instance called server0. A key pair, the networks and subnets are assumed to exist, and these resources are declared in the parameters section.

There are a few notable details in this template:

Security groups must be assigned to the ports in the template (line 68 and 77 in the template). When the security group assignment declaration is put under the instance itself, the error message “ERROR: Cannot define the following properties at the same time: security_groups, networks/port.” is shown.
The instance must be created after the trunk has been provisioned. This dependency can be declared explicitly, with depends_on, or implicitly with get_attr (line 96 in the template).
The MAC addresses of the subports can be set to the MAC address of the parent port (line 74 in the template). In this case, there is no need to set the MAC address of the VLAN subinterface on the instance (see Figure 6).

Figure 6. Heat output with the same MAC address for the subport and the parent port.

The stack is created with

openstack stack create --template <template-name> <stack-name>

After successful creation of the stack, the instance needs to be configured as in Section https://pannet.atlassian.net/wiki/spaces/DocEng/pages/2055667717/Network+trunking#Configure-server-interface but without line 2 in the code snippet:

sudo ip link add link ens3 name ens3.<vlan-id> type vlan id <vlan-id>
sudo ip link set ens3.<vlan-id> up
sudo dhclient ens3.<vlan-id>

The events generated by the Senlin engine - the OpenStack clustering service - are displayed (see Figure 7) with the command

openstack stack event list <stack-name>

Figure 7. Event record list for the trunk HOT stack.

Template:

heat_template_version: 2018-08-31

description: Launch an instance with networks trunked over a port

parameters:
  server_name:
    type: string
    description: Name of server
    default: server0
  server_flavor:
    type: string
    description: Flavor server is based on
    default: g1.standard-1-1
  server_key:
    type: string
    description: Key pair used by server
    default: common
  server_image:
    type: string
    description: Image server is based on
    default: ubuntu-20.04-x86_64
  app_port:
    type: number
    description: Port used by the servers
    default: 80
  internal_network:
    type: string
    description: Internal network used by server
    default: internal_network
  internal_subnet:
    type: string
    description: Subnet used by server
    default: internal_subnet
  vlan_network:
    type: string
    description: VLAN network used by server
    default: provider-network-vlan-1217
  vlan_subnet:
    type: string
    description: Subnet used by server
    default: provider-subnet-1217
  vlan_id:
    type: number
    default: 1217

resources:
  sec_group:
    type: OS::Neutron::SecurityGroup
    properties:
      rules:
        - remote_ip_prefix: 0.0.0.0/0
          protocol: tcp
          port_range_min: { get_param: app_port }
          port_range_max: { get_param: app_port }
        - remote_ip_prefix: 0.0.0.0/0
          protocol: tcp
          port_range_min: 22
          port_range_max: 22
        - remote_ip_prefix: 0.0.0.0/0
          protocol: icmp

  parent_port:
    type: OS::Neutron::Port
    properties:
      network: { get_param: internal_network }
      security_groups:
        - default
        - { get_resource: sec_group }

  subport0:
    type: OS::Neutron::Port
    properties:
      network: { get_param: vlan_network }
      mac_address: { get_attr: [parent_port, mac_address] }
      security_groups:
        - default
        - { get_resource: sec_group }

  trunk0:
    type: OS::Neutron::Trunk
    properties:
      port: { get_resource: parent_port }
      sub_ports:
        - { port: { get_resource: subport0 },
          segmentation_type: vlan,
          segmentation_id: { get_param: vlan_id } }

  instance0:
    type: OS::Nova::Server
    properties:
      name: {get_param: server_name}
      key_name: { get_param: server_key }
      flavor: { get_param: server_flavor }
      image: { get_param: server_image }
      networks:
        - { port: { get_attr: [trunk0, port_id] } }

outputs:
  parent_port/name:
    description: Name of parent port
    value: { get_attr: [parent_port, name] }
  parent_port/mac_adress:
    description: MAC address of parent port
    value: { get_attr: [parent_port, mac_address] }
  parent_port/fixed_ips:
    description: IP address of parent port
    value: { get_attr: [parent_port, fixed_ips] }
  subport0/name:
    description: Name of subport
    value: { get_attr: [subport0, name] }
  subport0/mac_address:
    description: MAC address of subport
    value: { get_attr: [subport0, mac_address] }
  subport0/fixed_ips:
    description: IP address of subport
    value: { get_attr: [subport0, fixed_ips] }

Horizon trunk interface

There is a Horizon interface for trunk creation as well. This takes a configured parent port, creates a trunk and allows subsequent assignation of subports to it. Most of the configuration still has to be done manually as described in this chapter.

Additional resources

https://docs.openstack.org/neutron/stein/admin/config-trunking.html

https://docs.openstack.org/python-neutronclient/pike/cli/osc/v2/network-trunk.html