Manage volumes

There are two types of storage - block storage and object storage. A volume (also known as a Cinder volume) is a block storage resource of fixed size. Such a device can be moved between virtual machines - similar to a USB hard drive.

Contents

A volume, like a USB hard drive, can only be attached to one server at a time, and retains any data on it even when not attached to any server. It is characterized by its size and is created from an existing image. The advantages of using separate volumes include:

  • Scale server storage independently from compute resources

  • Flexibility to attach different types of storage as needed

  • Facilitate service migration by the separation of data and runtime environment

  • Use as secure data storage, recovery repository or disaster management

Note: This section described the necessary steps from the command-line interface, but volumes can be created and attached from the Horizon interface as well.

Volume status

The volume can be in one operational status out of four possible, which determines what can be done with it. The status of all volumes are shown in Horizon under Projects/Volumes and in the list or show output from OpenStack client:

  • Building – the volume is in the process of being created

  • Available – the volume has been created, but is not attached to any server

  • In-use – the volume is attached to a server

  • Deleting – the volume is in the process of being deleted

Create and configure volume

The OpenStack commands described can take a number of optional arguments, most of which have been omitted in the text. For more details on command arguments, please refer to the OpenStack documentation.

Create blank volume

A volume is created by the command

openstack volume create [--image <image-id>] --size <size> <name>

where size is the storage size in gigabytes and <name> is a name of choice for the volume. The main option is whether to include an image or not on the volume. Including the --image parameter makes the volume bootable and allows creation of an instance from it at a later stage.

Passing this (optional) argument along with a size and suitable name for the volume to the command above will create it. To verify that the volume has been created successfully, do

openstack volume list

After the command completes successfully, the status of the volume is available.

Attach to server

An available volume need to be attached to a server to be accessed. The command takes server identity, volume identity and device directory as arguments:

openstack server add volume <server-id> <volume-id> --device <device-dir>

Like for other resources, server and volume identities are found by

openstack server list

openstack volume list

The device is a directory name - in conformance with practice something like /dev/vdx, where x is the letter b for the first separate volume, c for the second, etc. The device name /dev/vda is used for the internal server storage. To see the mounted partitions (Figure 1), connect to the virtual machine by SSH and run df -h.

Figure 1. Output from df -h before mounting the volume.

By convention, the device name for the first external volume uses the first free letter and is attached to the server by the command (here using vdb)
openstack server add volume <server-id> <volume-id> --device /dev/vdb

The status of the volume is now in-use. More details about the volume are listed with

openstack volume show <volume-id>

Build file system

The volume needs to be formatted (if created without an image) and mounted before it can be used. This is done with standard Linux tools. The procedure described here contains a minimum of options - the reader is encouraged to visit the man pages of these commands for further details.

Please note that if the volume has been created with an image, partitions are automatically created for the operating system. The empty partition is then /dev/vdb1 rather than /dev/vdb.

To format the device and create a file system, login to your server by SSH and check if the unmounted raw volume is attached with

lsblk /dev/vdb

Figure 2 shows a volume created without any image, and Figure 3 a volume based on an image.

Figure 2. Output from lsblk on a volume created without image.
Figure 3. Output from lsblk on a volume created with an image.

When the volume has been created with an image, no formatting is needed and it can be mounted directly. A pure block storage device, however, needs to be formatted as follows. Note that the device name is the same as the one passed to the OpenStack command attaching the volume. In the code examples, vdb has been used throughout.

When formatting the volume, a number of tools and options can be chosen. For simplicity, we use

sudo mkfs.xfs /dev/vdb

The terminal output from the file system creation (mkfs) is shown in Figure 4.

Figure 4. Output from mkfs.

Mount device

A device is mounted with

sudo mount <device-dir> <mount-point>

The mount point is a directory that usually needs to be created in a suitable location, such as /media/disk. To mount the volume automatically whenever re-attached to the server, the fstab file is updated as in step 2 of the mount procedure:

sudo mkdir /media/disk
echo "/dev/vdb /media/disk xfs defaults 0 0" | sudo tee -a /etc/fstab
sudo mount -a

View current mount map by df -h (Figure 5), now showing the volume near the end of the list.

Figure 5. Mount map showing external volume as vdb.

The volume is now attached, mounted and prepared for use. Now we can create a directory for data storage, say /data, and change the permissions on the disk with

sudo chown ubuntu:ubuntu /media/disk/data

To use the volume efficiently, it is convenient to configure an FTP client to move files between your client and the volume. This is described in Configure client .

Block storage backup

An entire volume can be copied either by creating a snapshot for use within a tenant, or an image, which can be downloaded and moved to another tenant.

Depending on the service running on the server, it might first be necessary to stop it to halt all operations on the volume.

Create volume snapshot

If a volume is detached (status: available) from its server, a snapshot is created by

openstack volume snapshot create --volume <volume-id> <snapshot-name>

Either the name or the identification string can be used as parameter <volume-id>. Even if the volume is attached to a server (status: in-use), a snapshot can still be created by using the argument --force. However, data consistency between the original volume and the snapshot is not guaranteed.

The snapshots created are listed with

openstack volume snapshot list

Note that its size is listed as well. A new volume can then be created from the snapshot by

openstack volume create --snapshot <snapshot-id> --size <volume-size> <volume-name>

We assume that the size of the new volume is at least that of the snapshot.

The snapshot can be deleted with

openstack volume snapshot delete <snapshot-id>

The command also accepts the argument --force to delete the snapshot regardless of the state it is in.

Resize volume

Given that storage resources are available, it is possible to increase the size of a volume. To be able to resize a volume, it has to be detached from any server. In the resizing command, the new size must be strictly greater than the existing size,

openstack volume set <volume-id> --size <new-size>

The command does not print any information of the new configuration.

Migrate volume

One of the advantages of using volumes for data storage is the ease with which the data can be moved between servers within a tenant or between tenants.

Currently, the OpenStack command migrate has not been implemented. An equivalent migration procedure is described in this section.

Migrate within a tenant

From the OpenStack CLI, a volume is attached to a server with

openstack server add volume <server-id> <volume-id> --device <device-dir>

and detached from a server with

openstack server remove volume <server-id> <volume-id>

After attaching the volume to a server it also needs to be mounted before it can be used. If added to a server where a free mount point already exists (if the volume was detached without being unmounted first), it is either mounted automatically or by connecting over SSH to the server and mounting it manually with

sudo mount -a

When added to a server without any free mount point, a few more steps need to be done, see https://pannet.atlassian.net/wiki/spaces/DocEng/pages/501022851/Manage+volumes#Mount-device

Migrate between tenants

An image created from a volume can be used to copy data to another DT Cloud Services tenant than it was created in. How to create an image from a volume is described in Manage images

After uploading the image with

openstack image create --container-format bare --disk-format raw --file <image-name>.raw <new-image-name>

a new volume can then be created from this image with

openstack volume create --image <new-image-name> --size <size> --availability-zone <zone-id> <new-volume-name>

If the server has an available mount point, the newly created volume can be attached to it with

openstack server add volume <server-id> <new-volume-name> --device <mount-point>

and should be mounted automatically. If not, set up an SSH connection to the server, create a mount point as described in Migrate within a tenant, and mount manually with

sudo mount -a

The server may need to be rebooted with

openstack server reboot <server-id>

Delete volume

A detached volume that no longer needed is irretrievably deleted with

openstack volume delete <volume-id>

While the deletion process is ongoing, the volume enters into status deleting. It permanently disappears, when it has been fully deleted in the tenant.

Transfer volume

In a project with multiple users, all resources created by a user are visible to other users apart from key pairs and volumes. Access rights (or ownership) of a volume can be transferred to another user by its creator.

Access rights to a volume are granted to another user who can present a transfer identity and an authorization key, generated by the owner of the volume.

Create transfer request

First the owner identifies the volume to be transferred from the listing

openstack volume list

For the volume to be possible to transfer, it needs to be in available state, that is, detached from any server. The two access codes are generated with

openstack volume transfer request create <volume>

where <volume> is the name or ID of the volume (Figure 5).

Figure 5. Create transfer request.

The pending transfer is listed with

openstack volume transfer request list

showing (Figure 6) the status of the transfer request. The transfer request ID is shown to the requester only as long as the transfer request has not yet been accepted by the second party.

Figure 6. Status of a transfer request.

There is also the command

openstack volume transfer request show <id>

with the transaction request ID as argument, but note that the authorization key is not shown in the output.

Accept transfer request

The volume transfer request is not associated with any particular OpenStack user. It suffices to forward the transfer request ID and authorization key to designated user, for example by e-mail, and the receiver accepts the transfer with

openstack volume transfer request accept --auth-key <auth-key> <id>

which gives the user full access to the volume including permission to delete it. The command output is shown in Figure 7.

Figure 7. Accepting the transfer request.

After completion of this, the volume is visible to both the original owner and the receiver of the request, the request itself is removed and the associated credentials are invalid for further use.

Delete transfer request

Before a transfer request has been accepted, it can be deleted with

openstack volume transfer request delete <id>

with the transaction request ID as argument, which cancels the entire operation.

Using Heat

To create a volume in Heat only requires a few lines. To attach it to a server automatically, however, requires running a script on the server and considering a few details.

The intricacy in the script relies on the fact that Cinder/Nova do not communicate with the udev (dynamic device naming support) within the instance. It can in general assign the volume any device name - similarly to how the operating system would handle a detachable USB storage.

The template below creates three resources: a server, a volume and a volume attachment. Although Nova cannot fetch the volume device name, it can use the volume_id which - truncated to 20 characters - is also used in the by-id identifier on the server as /dev/disk/by-id/virtio-<truncated-volume_id>.

An instance booting in OpenStack can be passed a configuration or script as cloud-config. The line #cloud-config has to be included at the top of the script file. The script takes the variable vol1_id which is assigned the Cinder ID of the volume created. The variable name is surrounded by a special character (percent) to make it unique and thereby avoid false matches.

Based on this, the device name vol1_dev is created, and this is used to format and mount the volume at the created mount point. The script also adds the volume to /etc/fstab for future use. The script is saved as /tmp/disk-config, which can be inspected after the server has been created (Figure 8).

Figure 8. Inspecting script and mounted volume on server.

Template:

heat_template_version: 2018-08-31

description: Single compute instance with volume attachment

resources:
  server1:
    type: OS::Nova::Server
    properties:
      key_name: <key-name>
      image: <image>
      flavor: <flavor>
      networks:
        - network: <network>
      user_data_format: RAW
      user_data:
        str_replace:
          template: |
            #cloud-config
            write_files:
              - content: |
                  #!/bin/bash
                  vol1_id="%vol1_id%"
                  vol1_dev="/dev/disk/by-id/virtio-$(echo ${vol1_id} | cut -c -20)"
                  mkfs.ext4 ${vol1_dev}
                  mkdir -pv /media/disk
                  echo "${vol1_dev} /media/disk ext4 defaults 1 2" >> /etc/fstab
                  mount /media/disk
                path: /tmp/disk-config
                permissions: '0700'
            runcmd:
              - /tmp/disk-config
          params:
            "%vol1_id%": { get_resource: server1_vol1 }
            
  server1_vol1:
    type: OS::Cinder::Volume
    properties:
      size: 10

  server1_vol1_attach:
    type: OS::Cinder::VolumeAttachment
    properties:
      instance_uuid: { get_resource: server1 }
      volume_id: { get_resource: server1_vol1 }

Additional resources

OpenStack Block Storage (Cinder) documentation