Nginx server configurations
Nginx is an easily configurable web server that can be used to implement load balancer, reverse proxy and test servers.
Contents
In the configurations described below, the first step is to create an compute instance for the server following the instructions in Create and manage virtual machines
Back-end server
To test a network, it is often necessary to have a simple web server that can respond to HTTP requests. This is easily done with Nginx. The Nginx server is powerful and can be used for complex services, but here it is used mainly in the context of network configuration and testing since it is easy to implement and configure.
After creating a compute instance, log in to the server through SSH, and install Nginx (after an update) with
sudo apt update && sudo apt install nginx
For testing purposes, it is sufficient that the web server returns some sort of identifier of the instance hosting it. For this purpose we can create an extremely simple landing page with a text string with
mkdir www
nano www/index.html
In the text editor, enter some simple string, for example "Back-end server 1". The page does not have to be in HTML - plain text will do for testing purposes. Save the file and close nano.
To set the document root to ~/www, copy and edit the default server configuration (saving it as <name>), and replace the symbolic link in the directory of enabled sites through the following steps:
sudo cp /etc/nginx/sites-available/default /etc/nginx/sites-available/<name>
sudo nano /etc/nginx/sites-available/<name>
Change the line beginning with “root” to root /home/ubuntu/www; Save and close nano.
sudo ln -s /etc/nginx/sites-available/<name> /etc/nginx/sites-enabled/<name>
sudo rm /etc/nginx/sites-enabled/default
Repeat this on the second back-end server, using some other text for the index.html, like and "Back-end server 2".
Initially, the nginx server is started automatically. By default, nginx is enabled to start automatically after system boot time. Should the server be disabled, this is set by
sudo systemctl enable nginx
After changing the configuration, restart nginx with a graceful shut-down of any active worker processes with
sudo systemctl reload nginx
The reload command is preferable to restart for this worker process handling. Test the HTTP response with
curl 127.0.0.1
which now should show the custom message.
Configure with Ansible
Ansible uses SSH to perform tasks specified in a template on multiple servers. Nginx installation and basic configuration of back-end servers can be orchestrated with Ansible, including the tasks
Updating the server package index
Installing Nginx
Modifying the configuration file
Uploading a specific index.html file
Reloading/restarting Nginx
For this purpose, four files are created: an inventory file, the Ansible playbook, the configuration modification and the index.html to be uploaded. Ansible needs to be installed on the client from which the orchestration is launched.
Inventory
The inventory file contains the addresses to remote servers, which can be grouped into classes where only servers in a specified class are affected by the Ansible execution. The default inventory file is /etc/ansible/hosts, but it can also be specified in a separate file, here called inventory.ini, included with the -i option.
The servers can be identified by IP address, domain name, or server nickname as specified in ~/.ssh/config, see https://pannet.atlassian.net/l/c/1j17uEyb. When the inventory file is written in INI format, the group headings are given in square brackets, for example
[webservers] server0 server1
The connection can now be tested with
ansible -i inventory.ini webservers -u ubuntu -m ping
which when successful produces output similar to Figure 1.
In the Ansible command, the inventory file above is used and the ping operation is applied to the servers listed under the group webservers.
Configuration
The server configuration to be used is stored in a separate file called site.conf.j2. This uses the Jinja2 template syntax with variable values specified in the Ansible playbook.
server {
listen 80;
listen [::]:80;
server_name {{ domain }};
root /home/{{ ansible_user }}/www;
location / {
try_files $uri $uri/ =404;
}
}
Site content
Common site content are copied to the remote servers by executing the Ansible task synchronize. On the client, such files including index.html are located in a dedicated subdirectory called site/.
Playbook
The Ansible playbook contains sections specifying the group of remote servers affected with hosts: webservers, the login name remote_user: ubuntu, and elevation to root access with become: true. The variables domain and ansible_user are used throughout the templates, and the task section specifies the order and details of operations carried out on the servers.
The playbook is here written in YAML format. Saving it as configure_nginx.yaml, it is executed with
ansible-playbook -i inventory.ini configure_nginx.yaml
---
- hosts: webservers
remote_user: ubuntu
become: true
vars:
domain: mytestsite.com
ansible_user: ubuntu
tasks:
- name: "apt-get update"
apt:
update_cache: yes
cache_valid_time: 3600
- name: "install nginx"
apt:
name: ['nginx']
state: latest
- name: "create www directory"
file:
path: /home/{{ ansible_user }}/www
state: directory
mode: '0775'
owner: "{{ ansible_user }}"
group: "{{ ansible_user }}"
- name: "synchronize site content"
synchronize:
src: site/
dest: /home/{{ ansible_user }}/www
archive: no
checksum: yes
recursive: yes
delete: yes
- name: "delete default nginx site"
file:
path: /etc/nginx/sites-enabled/default
state: absent
notify: restart nginx
- name: "copy nginx site config"
template:
src: site.conf.j2
dest: /etc/nginx/sites-enabled/{{ domain }}
owner: root
group: root
mode: '0644'
notify: restart nginx
handlers:
- name: restart nginx
service:
name: nginx
state: restarted
As the playbook is executed, the progress is reported as the tasks are carried out (Figure 2).
Load balancer on Nginx
A simple load balancer can implemented based on the Nginx open source web server. For the setup, we need an instance as reverse proxy and at least two back-end servers. Resources on the tenant for three small servers are needed for this configuration.This load balancer consists of a single compute node and should not be used in production environments.
Log in to a created server through SSH, and install Nginx (after an update) with
sudo apt update && sudo apt install nginx
The load balancer is deployed by modifying the configuration file /etc/nginx/nginx.conf as follows.
Open the file for editing with
sudo nano /etc/nginx/nginx.conf
and before the last closing curly bracket, add
upstream my_load_balancer {
server <internal-ip-1>:<port>;
server <internal-ip-2>:<port>;
...
}
server {
listen 80;
location / {
proxy_pass http://my_loadbalancer;
}
}
where the internal IP addresses and ports for the back-end servers need to be specified, and the server pool is identified by a string, here my_loadbalancer. Note that this string is used directly with the protocol prefix after proxy_pass.
Save the file and exit the text editor. A configuration test can be performed with
sudo nginx -t
or, alternatively
service nginx configtest
which, if correctly configured, gives the result (Figure 3)
Restart the web server with
sudo systemctl restart nginx
or
service nginx restart
Reverse proxy
Just like bastion hosts gives advantages in terms of security and and management of remote login (SSH) on virtual machines, a reverse proxy can be used for a similar role handling HTTP traffic. Having a floating IP assigned to it, the reverse proxy passes traffic to back-end servers, and can thereby act as a secure gateway.
In addition to load balancing, the purpose of a reverse proxy is to act as a front-end to application servers and improving performance and operational efficiency. When incoming requests are handled at a single host, it can be used to implement
Central logging
Improved security
TLS termination
Caching and compression
Central logging
Logging is essential to the operation of any complex service. With several back-end servers, it is often desirable to collect access logs on a the reverse proxy which contains all traffic rather than separate logs on the back-end servers.
Nginx contains a number of directives to control logging, where a directive refers to a module or a set of general rules. The to most prominent directives are error_log and access_log.
The error_log directive is configured in /etc/nginx/nginx.conf and uses the syntax
error_log <log-file> [<log-level>];
where <log-file> is the absolute path to the file where errors will be logged, and the optional <log-level> is a keyword specifying the level of priority of the error messages of interest and thereby the amount of information. The error messages generated by the software are tagged with log level, which is used by the directive to filter out the messages to be logged. Note that if the log file does not exist, it will be created automatically.
Log levels
The more commonly used log levels are:
crit - shows critical faults leading to emergency situations where the system is in an unusable state
error - shows that an error has occurred, such as an unsuccessful operation
warn - shows that something out of the ordinary has happened that may need attention, but that the operation was successful
info - shows information that might be good to know in additions to errors and warnings
debug - the most detailed log level, including any information that can be useful to find where where a problem lies
The default log file is /var/log/nginx/error.log. The default log level is error for the main system, and crit for the HTTP and server modules.
The levels higher on the list are considered a higher priority. When a log level is specified, the log will contain that level and any level higher than the specified level. The proxy can be instructed to listen to event to its back-end servers by specifying them as
events {
debug_connection 192.168.1.1;
debug_connection 192.168.10.0/24;
}
To switch off error logging the output must be sent to /dev/null, like this
error_log /dev/null crit;
In general, error logging should not be switched off.
Access logs
The access_log directive is used to configure access logs which contain details of service requests reaching the proxy. It allows detailed configuration of the information and format to be stored. The general syntax is
access_log <log-file> [<log-format> <buffer-size>];
The optional argument <log-format> is the name of a directive log_format specifying how log entries should be printed in the log, with the rules expressed in plain text and variables. The format combined is predefined in Nginx and is used in many servers. As an illustration of how the log_format directive can be used, the definition of combined is given below
log_format combined '$remote_addr - $remote_user [$time_local] '
'"$request" $status $body_bytes_sent '
'"$http_referer" "$http_user_agent"';
where words after a dollar sign ($) are variables and other characters are printed literally (including the square brackets). The variables available are described in the Nginx documentation.
Since the format combined is already defined, it can be used directly with the access_log directive, like
access_log <log-file> combined;
The default access log file is /var/log/nginx/access.log. If either the buffer or the gzip parameter is used, the log data will be buffered. The optional <buffer-size> is the maximum size of data that Nginx will hold in memory before writing it all to the log. The default size is 64K, expressed as buffer=64k. The minimum value is the Linux size of an atomic write, or 4K (4096 bytes).
It is also possible to specify compression of the log file by adding gzip to the definition:
access_log <log-file> combined gzip;
Access logging can be turned off simply by setting
access_log off;
which can be useful to prevent duplicate access logging on back-end servers.
Log rotation
As the log files accumulate data, they fill up disk space and need to be managed. A common solution to this is log rotation, where older log files are replaced successively, possibly combined with archiving old file for a limited amount of time.
The logrotate utility is a simple program to manage log rotation. It is installed on Ubuntu by default, and Nginx on Ubuntu comes with a custom logrotate script. The script can be opened for inspection and modification by typing
sudo nano /etc/logrotate.d/nginx
which is shown in Figure 4.
On Ubuntu, logrotate is pre-configured to run daily and to log its own actions to syslog (which is also rotating). Its pre-configured cron job is located at /etc/cron.daily/logrotate. In most cases, no manual configuration of logrotate is required.
Security
Security features can be implemented on the reverse proxy similarly to the bastion host to protect application servers - UFW firewall rules and Fail2Ban intrusion protection, for example. For details on how to enable these utilities, see Configure bastion host
TLS termination
The reverse proxy can store CA certificates, act as TLS termination and perform port translation for communication with back-end servers.
Note that usually TLS require a domain name be associated with the IP address, that is most Certificate Authorities do not accept IP addresses without any domain name. Once a domain name has been registered and a DNS record set up, the TLS termination can be implemented on the proxy, for example by installing the popular software client certbot by Let’s Encrypt.
Please note that the security groups and firewall settings need to be complemented with rules to accept HTTPS traffic as described in https://pannet.atlassian.net/l/c/hA20v19K
Performance improvement
Nginx can be configured to improve server performance with content caching and compression.
Caching
Caching is enabled by adding the an expiry to different file types, in particular multimedia files. For example, by adding expires 6d to various file types, the content is cached for 6 days after it is first served before it is re-loaded.
To enable this, open the server configuration to modify with
sudo nano /etc/nginx/sites-available/<site-name>
and within the server {…} section, add (as an example)
server {
...
location ~* \favicon.ico$ {
expires 6d;
}
location ~ \.css {
root /var/www/html;
add_header Content-Type text/css;
expires 6d;
}
location ~* \.js {
root /var/www/html;
add_header Content-Type application/x-javascript;
expires 6d;
}
location ~* \.png|jpeg|jpg {
root /var/www/html;
add_header Content-Type image/png;
add_header Content-Type image/jpeg;
add_header Content-Type image/jpg;
expires 6d;
}
location ~* \.svg {
root /var/www/html;
add_header Content-Type image/svg+xml;
expires 6d;
}
}
Compression
Performance can also be improved compression, which saves bandwidth. Gzip compression is enabled by adding the following directives either to a server configuration under the http {…} section, or as global directives in /etc/nginx/nginx.conf or as a separate configuration file stored as/etc/nginx/conf.d/gzip.conf.
gzip on; gzip_vary on; gzip_min_length 512; gzip_proxied any; gzip_comp_level 6; gzip_buffers 16 8k; gzip_http_version 1.1; gzip_types text/css text/javascript text/xml text/plain text/x-component application/javascript application/json application/xml application/rss+xml font/truetype font/opentype application/vnd.ms-fontobject image/svg+xml;
The parameters are used as follows:
gzip on; – enable gzip compression
gzip_vary on; – enable (or disable with off) inserting the “Vary: Accept-Encoding” response header field if the directives.
gzip_min_length <size>; – file size limit: do not compress anything smaller than the stated size
gzip_proxied <response>; – instruct Nginx to compress data for clients connecting through proxies. With <response> set to any, it enables compression to response headers including “expired”, “no-cache”, “no-store”, “private”, and “Authorization” parameters.
gzip_comp_level <level>; - gzip level of compression (1-9), where 1 is the fastest and lowest, and 9 is the highest level of compression. The default level is 6.
gzip_buffers <number> <size>; Sets the <number> and <size> of buffers used to compress a response. By default, the buffer size is equal to one memory page.
gzip_http_version <http-version>; – the minimum version of the HTTP protocol of a client request needed to compress the response from the server.
gzip_types <file-types>; – list of file types (MIME types) that should be compressed. The default MIME types are listed in /etc/nginx/mime.types.