Telos Alliance Container Backup and Restore Tool - Beta

Overview

The Telos Alliance Container Backup and Restore Tool is a set of utilities that eases backup and restore of containers in a Docker environment.

Prerequisites

Debian or RHEL derivative Linux (apt or yum package managers)
wget or curl packages
Docker Engine
Quay.io account

Tested Platforms

Telos Alliance AP-3000
Telos Infinity VIP Server
AWS EC2
Bare metal servers

Tested Operating Systems

Tested Container Runtimes

Docker is the only officially supported runtime at the time of writing, with support the following network modes:

Host networking
Macvlan (telosmacvlan) networking
Dual-macvlan (telosmacvlan and ext1macvlan) networking

Installation

Online Install

Run the following command to install the latest production version:

curl -Ssf https://telos-public.s3.amazonaws.com/devops/ta-container-backup/install-beta.sh | sudo bash

Offline Install

Download the install package and transfer it to the target system:

curl -Ssf https://telos-public.s3.amazonaws.com/devops/ta-container-backup/container-backup-and-restore-beta.tgz -o container-backup-and-restore-beta.tgz

Run the installation script:

tar -xf container-backup-and-restore-beta.tgz
cd container-backup-and-restore
sudo ./install.sh

Environment Configuration

The /etc/telos/tabr-backup.env file contains configuration options for backup behavior.

Update these values as needed and run tabr-backup to apply the changes immediately, otherwise they will be applied by the next scheduled execution.

Sample:

### After updating any environment values, run "tabr-backup" to apply the setting. ###

# Enter the retention unit that local backups in "/opt/backup-telos/snapshots" should use. 
# "1" = days, "2" = files. Default "2" files. 
LOCAL_RETENTION_UNIT=2

# Enter the number of days or files (see LOCAL_RETENTION_UNIT) 
# for which local backups in "/opt/backup-telos/snapshots" should be retained. Default "7".
LOCAL_RETENTION_NUM=7

# Enter the frequency in which automatic backups are taken. 
# "2" = hourly, "3" = daily, "4" = weekly. Default "3" daily.
BACKUP_FREQUENCY=3

# Enter the name of the S3 Bucket dedicated to telos container backups for this host.
# Sub folders are supported as long as the syntax <bucket/path/to/folder> is used.
S3_BUCKET=

# Enable tabr-restore to pull and install the latest S3 backup if "S3_BUCKET" is configured.
# Fallback to latest local backup if "S3_BUCKET" is not configured.
# "1" = enabled. Leave blank to disable feature.
LATEST_AT_STARTUP=

### After updating any environment values, run "tabr-backup" to apply the setting. ###

Utilities

To invoke the utility, simply type in the shell one of the following commands, depending on the goal.

All the utilities included as part of this tool follow the naming scheme of tabr-funciton.

Currently, the tool comprises the following:

tabr-version
tabr-backup
tabr-restore
tabr-delete
tabr-volumes
tabr-startup

Please find below an in-depth explanation of each utility.

Backup Utility: `tabr-backup`

Creates a timestamped backup tarball in /opt/backup-telos of the installed containers, Telos Alliance environment files and some nginx files.
If configured in /etc/telos/tabr-backup.env, uploads backups to an S3 bucket.
Can be run manually or through a systemd service and timer. More on the systemd usage here.

Command:

Writes timestamped backup "Snapshot" to /opt/backup-telos/snapshots and latest. Contents of the snapshots directory are rotated based on the settings in /etc/telos/tabr-backup.env.

tabr-backup

Writes timestamped backup to /opt/backup-telos/archive. Contents of this directory are not rotated. This feature is useful for backups that you want to store long term.

tabr-backup --archive

Writes timestamped backup, with an identifier you specify to /opt/backup-telos/archive. Contents of this directory are not rotated. This feature is useful for backups that you want to store long term. Replace <identifier> with a name you'd like to add to the front of the filename.

tabr-backup --archive <identifier>

Restore Utility: `tabr-restore`

Runs tabr-delete to clear the existing container state and configuration.
Prompts the user to proceed.
Restores a backup from a local file, URL or S3 URI.
Automatically handles container states (running or stopped) during restoration.

Note: When restoring to a system where the Docker images of the containers are not downloaded, you will be prompted to log into Quay to pull them again. Should this fail, you may log in with the command below before running the tabr-restore utility.

sudo docker login quay.io

Commands:

Restore from a local file:
```
tabr-restore path/to/file.tgz
```
Restore from an object URL:
```
tabr-restore https://path/to/file.tgz
```

Restore from an S3 URI:

tabr-restore s3://bucket/path/to/file.tgz

Restore the latest backup:
```
tabr-restore --latest
```

NOTE: When using the tabr-restore --latest feature, if an S3 bucket is configured in /etc/telos/tabr-backup.env, tabr-restore will pull and restore the latest from S3, ignoring the local backup.

Delete Utility: `tabr-delete`

Removes all containers and Telos Alliance-specific container configurations from the system.

Command:

tabr-delete

Version Utility: `tabr-version`

Displays the currently installed version of the tool.

Command:

tabr-version

Startup Utility: `tabr-startup`

tabr-startup is run at startup by tabr-startup.service. If LATEST_AT_STARTUP=1 in /etc/telos/tabr-backup.env, then tabr-startup will restore the latest s3 or local file. This setting is mostly to support use cases where S3 backup are utilized.

Advanced Usage

Startup Systemd Service: `tabr-startup.service`

Located at: /etc/systemd/system/tabr-startup.service
Manages the automatic execution of tabr-startup based on the LATEST_AT_STARTUP variable in /etc/telos/tabr-backup.env.
When enabled, will load the latest local or S3 (if configured) backup at system boot.
Very useful for failover workflows.

Backup Systemd Timer: `tabr-backup.timer`

Located at: /etc/systemd/system/tabr-backup.timer.
Manages the automatic execution of tabr-backup based on the BACKUP_FREQUENCY variable in /etc/telos/tabr-backup.env.

Backup Systemd Service: `tabr-backup.service`

Located at: /etc/systemd/system/tabr-backup.service.
When called by tabr-backup.timer, executes the tabr-backup utility.

S3 Integration

Backups can be uploaded to an S3 bucket using the AWS CLI.
Configure the S3 bucket in /etc/telos/tabr-backup.env. It should follow the following syntax: S3_BUCKET=bucketname/path/to/folder/
For AWS EC2 deployments, configure the S3 permissions by attaching an S3 role to the EC2 instance.
For non-EC2 instances, run sudo aws configure.

Sample IAM Policy:

Replace bucket with your S3 Bucket name:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Telos-Container-Backup-and-Restore",
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:PutObject",
                "s3:GetObject"
            ],
            "Resource": "arn:aws:s3:::bucket/*"
        }
    ]
}

Volume Management: `tabr-volumes`

tabr-volumes is managed by the tabr-backup and tabr-restore utilities. Direct interaction is optional but available for advanced use cases.

Advanced usage:

Export a volume to a gzip tarball:
```
sudo tabr-volumes export VOLUME FILE
```
Import a gzip tarball into a volume:
```
sudo tabr-volumes import FILE VOLUME
```
Save volume contents to a BusyBox image in /volume-data:
```
sudo tabr-volumes save VOLUME IMAGE
```
Load contents from a BusyBox image into a volume:
```
sudo tabr-volumes load IMAGE VOLUME
```