Deploy web server

The tutorial describes the steps to create a virtual machine with Heat and on it deploy Apache web server using Ansible.

Contents

Prerequisites

The following resources required for the web server have to be available in the tenant

  • One unassociated floating IP address

  • Compute resources for the chosen instance flavor and a new security group

  • Existing external network, internal network and subnet, see Quick Start

The internal network and subnet are called <internal_network> and <internal_subnet>. In addition, a key file needs to be created (here called <my_key> as follows:

openstack keypair create my_key > my_key.pem
chmod 600 my_key.pem

Create VM with OpenStack Heat

The OpenStack client provides many commands to create and manage virtual machines and other objects. For larger projects, such repetitive operations become tedious and prone to errors and orchestration with OpenStack Heat greatly facilitates infrastructure deployment.

Heat is a template-based orchestration utility, where templates are text files that specifies a deployment which are parsed and translated into the appropriate OpenStack API calls. The Heat Orchestration Templates (HOT) are written in YAML or JSON format; Heat is used primarily to manage infrastructure, but the templates can also be integrated with software configuration tools, such as Ansible.

Install client

OpenStack Heat is part of the client suite, and is installed along with the other OpenStack modules. After loading the tenant environment variables, the command

openstack stack list

should print an empty line, which also shows that the heat client is installed. Otherwise, it can easily be installed with

pip install python-heatclient

Templates

In this chapter, the templates are written in YAML format, which is sensitive to indentation and other syntax issues. The supported keywords and syntax is dependent on the template version, which is specified in the parameter heat_template_version. The supported template versions can be found with

openstack orchestration template version list

which produces a list (Figure 1) of versions that can be used.

The the functions supported by a template version is listed with

openstack orchestration template function list <template-version>

where <template-version> can be for example heat_template_version.rocky.

The following simple template for VM deployment specifies resource type (OS::Nova::Server) and its properties that correspond to OpenStack CLI arguments.

heat_template_version: 2018-08-31

description: Simple template for server deployment

resources:
  server:
    type: OS::Nova::Server
    properties:
      name: my_server
      key_name: my_key
      image: ubuntu-18.04-x86_64
      flavor: g1.standard-1-1
      availability_zone: az1
      networks: 
        - network: internal_network
      security_groups: [ssh_only]

After saving the HOT with the property parameters filled in as, say, create_server.yml, it is used with the command (from the same directory where the template was saved)

openstack stack create --template create_server.yml my_stack

Create a stack

The term stack is used for objects created by Heat, referring to the set of components that operate together to support an application. A stack is created by

openstack stack create --template <template> <stack-name>

from a template <template> and with some given name <stack-name>. The template must have a valid version after heat_template_version. The version can be a key date or the code name of the Heat release, which specifies both template format and supported features. If the format or a feature are unsupported by the given version, an error message is returned.

The template uses the resource type OS::Nova::Server to create an instance called my_instance with specified flavor, image, and key, and connecting to the existing network my_network. The description field are used for single-line comments (for multi-line comments description: > is used).

The creation output looks similar to Figure 2, showing some key data and creation status.

The templates can be made more general by accepting optional arguments overriding the default values in the template as shown in the code snippet.

heat_template_version: 2018-08-31

description: Template for deploying test server

parameters:
  server_name:
    type: string
    description: Name of server
    default: piglet

  app_port:
    type: number
    description: Port used by the servers
    default: 80

  key_pair:
    type: string
    description: Key pair used by server
    default: my_key

  internal_network:
    type: string
    description: Network used by server
    default: internal_network

  internal_subnet:
    type: string
    description: Subnet used by server
    default: internal_subnet

  external_network:
    type: string
    description: External internet provider
    default: external_internet_provider

  server_image:
    type: string
    description: Image server is based on
    default: ubuntu-20.04-x86_64

  server_flavor:
    type: string
    description: Flavor server is based on
    default: g1.standard-1-1

resources:
  sec_group:
    type: OS::Neutron::SecurityGroup
    properties:
      rules:
        - remote_ip_prefix: 0.0.0.0/0
          protocol: tcp
          port_range_min: { get_param: app_port }
          port_range_max: { get_param: app_port }
        - remote_ip_prefix: 0.0.0.0/0
          protocol: tcp
          port_range_min: 22
          port_range_max: 22
        - remote_ip_prefix: 0.0.0.0/0
          protocol: icmp

  server:
    type: OS::Nova::Server
    properties:
      name: {get_param: server_name}
      key_name: {get_param: key_pair}
      image: {get_param: server_image}
      flavor: {get_param: server_flavor}
      networks: 
        - network: {get_param: internal_network}
      security_groups: [{get_resource: sec_group}]

  floating_ip:
    type: OS::Neutron::FloatingIP
    properties:
      floating_network: { get_param: external_network }

  server_ip_assoc:
    type: OS::Neutron::FloatingIPAssociation
    properties:
      floatingip_id: { get_resource: floating_ip }
      port_id: {get_attr: [server, addresses, {get_param: internal_network}, 0, port]}

outputs:
  floating_ip:
    description: IP Address of the server instance
    value: { get_attr: [ floating_ip, floating_ip_address ]}

A parameter is defined with its name, a type (usually string), a default value and optionally, label and description.

The intrinsic function get_param copies a user-specified parameter value and uses it in the resource specification. In the example, the parameter values for image, flavor and key pair can now be set by adding --parameter key_name=<keypair-name> to set another key pair than default.

The arguments can be placed in the parameters section, in which each parameter is declared with a name, type, default value and an optional description. In the body, parameter values, resources attributes and resource identities are accessed with the functions:

  • get_param - references an input parameter of a template and resolves to the value provided for this input parameter at runtime.

  • get_attr - references an attribute of a resource which is resolved at runtime from the resource instance created from the respective resource definition.

  • get_resource - references another resource within the same template. At runtime, it is resolved to reference the ID of the referenced resource, which is resource type specific.

Default parameter values specified in the template can be overwritten in the creation command with the argument --parameter, for example,

openstack stack create --parameter "server_name=server2" --template create_server.yml <stack-name>

The argument is given in quotation marks and multiple parameters are separated with semi-colon.

The reader is encouraged to visit some of the extensive on-line documentation available for more details on techniques and Heat templates.

Verify deployment

At any time, the status of the deployment can be inspected with

openstack stack list

The creation process goes through the stage CREATE_IN_PROGRESS before reaching CREATE_COMPLETE, when the VM is ready to use. Should the creation process fail, this is indicated by CREATE_FAILED.

The OpenStack CLI provides some details related to the cause of failure in the creation report, which is fetched with

openstack stack show <stack-name>

In case the stack creation failed, it should be deleted and the cause of error needs to be corrected before attempting re-deployment. Using the template above, the public IP address <ip-address> is presented in the output from the show command (Figure 3).

Now, the server should be fully operational and respond to ping requests.

Another useful command for troubleshooting is

openstack stack event list <stack-name>

The output is a list of event records generated by the Senlin engine, the OpenStack clustering service.

Delete stack

Deleting a stack reverses the creation process and removes all dependent resources created in the stack. The command is

openstack stack delete <stack-name>

The entire stack should be deleted in this way rather than removing the resources manually. The deletion process go through DELETE_IN_PROGRESS status, so it is advisable to wait until a previous version of a stack has been fully deleted and purged from the tenant before attempting re-creation of the stack.

Establish SSH connection

In the template, the same key file is used for all servers by default, but it can be overridden by a command-line argument. To avoid having to refer to the key files explicitly, they can preferably be added to the SSH client with

ssh-add my_key.pem

for all key files used. Now, the servers should be accessible over SSH with

ssh ubuntu@<ip-address>

Install Apache with Ansible

Ansible is an open source orchestration tool, which will be used to install and configure a basic Apache web server on a VM by performing a set of tasks and using variable and directives from a set of templates and configuration files. The top level instructions are specified in a playbook file.

Install Ansible

Ansible can be installed with pip or the package manager of the distribution used on a Linux client. In the former case, the installation is performed with

python -m pip install --user ansible

To install through the Ubuntu packet manger, first the Personal Package Archive (PPA) repository has to be added to the APT packet management tool

sudo apt-add-repository ppa:ansible/ansible
sudo apt-get update -y
sudo apt-get install -y ansible

The installation can be verified with

ansible --version

which prints the Ansible version and dependencies (Figure 4)

Create role

An Ansible role can be seen as a project and uses a predefined directory structure which contains eight main standard directories. The Ansible roles are conveniently stored in a dedicated directory called roles/ relative to the playbook file.

First create the directory and move into it:

mkdir roles && cd roles

The role, here called apache, and its directory structure is created with

ansible-galaxy init apache

The command tree (which requires the utility tree installed) shows the created directory structure

├── defaults
│   └── main.yml
├── files
├── handlers
│   └── main.yml
├── meta
│   └── main.yml
├── README.md
├── tasks
│   └── main.yml
├── templates
├── tests
│   ├── inventory
│   └── test.yml
└── vars
    └── main.yml 

The roles can be listed with

ansible-galaxy role list

By adding instructions in YAML format to the directory structure, the role is built to perform various tasks, such as

  • Installing apache2 for Ubuntu

  • Creating a document root folder for the Apache VirtualHost and set up a test page

  • Enabling the Apache VirtualHost

Create inventory

The inventory file contains the basic server access details used by Ansible. Under the group header webservers, all web servers that will be affected are listed.

[webservers]
188.125.27.152   ansible_user=ubuntu
# 188.125.27.199  ansible_user=ubuntu

Note that there is no space surrounding the equality sign. Saving the file as inventory.ini, a connectivity test can be performed with

ansible webservers -m ping -i inventory.ini

which produces an output similar to Figure 5.

The default inventory file is /etc/ansible/hosts. To remove the need to specify the inventory in the command, either the path to the used inventory can be set in the configuration file, or the contents of inventory.inin copied to /etc/ansible/hosts. After updating the hosts file, there is no need to specify the inventory explicitly (Figure 6)

Configuration and log files

The path to the Ansible configuration file is /etc/ansible/ansible.cfg. In the same directory is the file hosts which is the default inventory file. The configuration contains, for example, default file paths and privilege escalation settings (needed for sudo operations).

Open the file for editing with

sudo nano /etc/ansible/ansible.cfg

and uncomment the lines for privilege escalation (Figure 7)

For larger projects, it is strongly recommended to enable logging. In ansible.cfg, uncomment the line

log_path = /var/log/ansible.log

to enable the logging function (Figure 8).

Ansible may not have permissions to create the log file, which is done manually by

sudo touch /var/log/ansible.log
sudo chmod 0777 /var/log/ansible.log 

Specify tasks

Just as in manual installation, we begin with updating the APT repository cache with sudo apt update. To perform this action, the playbook contains the group name of the servers to be updated (webservers), the necessary access elevation through become and the actual task calling the apt utility.

---
- hosts: webservers
  become: yes
  become_method: sudo
  tasks:
  - name: "Update Repository cache"
    apt:
      update_cache: yes
      cache_valid_time: 3600
      force_apt_get: yes

Saving the code as, say, apt_update.yml under the roles directory, all web servers can be updated by executing

ansible-playbook apt_update.yml -i inventory.ini

A successful run produces the terminal output shown in Figure 9.

Defaults

In the file defaults/main.yml, global variables which are referenced from other template files are defined. Note the http_port which must correspond to the port set in the Heat template for the security group (app_port in the HOT). The scr_dir contains site files that are copied to the remote hosts, in this example the file index.html.

The Boolean variable disable_default is used to control the disabling of the default server configuration (yes/no).

# defaults/main.yml

http_host_dir: "mytestsite.com"
http_conf_file: "myapache.conf"
http_port: 80
src_dir: "files/"
disable_default: yes

Files

Under the sub-directory files/ are files that are to be uploaded to the remote web server by Ansible. In this case only index.html with the content shown below.

<!DOCTYPE html>
<html lang="en">
  <title> </title>
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <link rel="stylesheet" href="https://unpkg.com/tachyons/css/tachyons.min.css">
  <body>
    <article class="vh-100 dt w-100 bg-dark-pink">
      <div class="dtc v-mid tc white ph3 ph4-l">
        <h1 class="f6 f2-m f-subheadline-l fw6 tc">Hello World!</h1>
      </div>
    </article>
  </body>
</html>

Handlers

The handlers define control commands that we wish to execute during the installation, in this case the Apache restart and reload operations.

# handlers/main.yml

- name: "restart apache2"
  service:
    name: apache2
    state: restarted

- name: "reload apache2"
  service:
    name: apache2
    state: reloaded

Tasks

Under the tasks directory, all tasks related to the Apache installation are specified in the file called apache_ubuntu.yml and shown below. It contains all steps from updating the repository cache, creating the document root, copying artifacts and configuring the virtual host.

# tasks/apache_ubuntu.yml

- name: "Install Apache web server on Ubuntu"
  apt:
    name: "apache2"
    update_cache: yes
    state: latest

- name: "Create document root"
  file:
    path: "/var/www/{{ http_host_dir }}"
    state: directory
    mode: '0755'

- name: "Copy source code artifacts"
  copy:
    src: "{{src_dir}}"
    dest: "/var/www/{{ http_host_dir }}"

- name: "Set up Apache virtual host"
  template:
    src: "apache.conf.j2"
    dest: "/etc/apache2/sites-available/{{ http_conf_file }}"

- name: "Enable new web site"
  shell: /usr/sbin/a2ensite {{ http_conf_file }}
  notify: "reload apache2"

- name: "Disable default web site"
  shell: /usr/sbin/a2dissite 000-default.conf
  when: disable_default
  notify: "reload apache2"

- name: "Add UFW rule: allow HTTP on port {{ http_port }}"
  ufw:
    rule: allow
    port: "{{ http_port }}"
    proto: tcp

- name: "Enable httpd"
  service:
    name: "apache2"
    enabled: yes

The main task file tasks/main.yml only contains a reference to the previously described detailed file

# tasks/main.yml

- name: "Install Apache on Ubuntu"
  import_tasks: "apache_ubuntu.yml"

Templates

The file templates/apache.conf.j2 with the content shown below contains the virtual host modifications of the Apache server configuration. Using the Jinja2 template syntax, the variable values defined in defaults/main.yml are used.

<VirtualHost *:{{ http_port }}>
DocumentRoot "/var/www/{{ http_host_dir }}"
</VirtualHost>

Playbook

The playbook contains the instructions and file references used by Ansible. All such instructions have be specified in the role, so the playbook - called webservers.yml - only contains the server group header and a reference to role apache.

---
- hosts: webservers
  roles:
    - role: 'apache'

Deploy web server

In the roles/ directory, the file structure looks like in Figure 10. Note that not all template files are used - only the ones described in the previous section.

To deploy the Apache web server as specified, simply run the Ansible playbook with

ansible-playbook webservers.yml

The output (Figure 11) shows all tasks and their results in a compact form.

Testing

Apache has a utility to verify the configuration. On the remote host, run

sudo apachectl configtest

which reveals any configuration errors. After Apache deployment with Ansible, the result from the test is likely to be as shown in Figure 12.

This is not critical for the operation of the web server, but it can easily be fixed by a dding a line to the Apache configuration file. Open the file for editing with

sudo nano /etc/apache2/apache2.conf

and add the line

ServerName <internal-ip-address>

as shown in Figure 13.

where <internal-ip-address> is the private IP address of the server. Save and close the file, and do

sudo systemctl reload apache2

to restart the server with the new configuration. The reason for this missing configuration item is that Apache needs both ServerName and an IP address to bind to. An IP address can be found from DNS, but in case no such record exists, the information has to be entered manually.

Verify that the server is running with

systemctl status apache2

which should show active (running) in green font (Figure 14).

The server response can be tested with

curl 188.125.27.152

or by typing the IP address in a browser (Figure 15).

The Apache server can be restarted (when already running) with

$ sudo systemctl restart apache2

where restart can be replaced with stop or start as desired.