Telos Alliance Container Backup and Restore Tool - Beta

Overview

The Telos Alliance Container Backup and Restore Tool is a set of utilities that eases backup and restore of containers in a Docker environment.

Prerequisites

• Debian or RHEL derivative Linux (apt or yum package managers)

• wget or curl packages

• Docker Engine

• Quay.io account

Tested Platforms

• Telos Alliance AP-3000

• Telos Infinity VIP Server

• AWS EC2

• Bare metal servers

Tested Operating Systems

• Ubuntu Server 20.04 LTS

• Ubuntu Server 22.04 LTS

• Ubuntu Server 24.04 LTS

• AWS Linux 2023

Tested Container Runtimes

Docker is the only officially supported runtime at the time of writing, with support the following network modes:

• Host networking

• Macvlan (telosmacvlan) networking

• Dual-macvlan (telosmacvlan and ext1macvlan) networking

Installation

Online Install

1. Run the following command to install the latest production version:

curl -Ssf https://telos-public.s3.amazonaws.com/devops/ta-container-backup/install-beta.sh | sudo bash

Offline Install

1. Download the install package and transfer it to the target system:

   curl -Ssf https://telos-public.s3.amazonaws.com/devops/ta-container-backup/container-backup-and-restore-beta.tgz -o container-backup-and-restore-beta.tgz

2. Run the installation script:

   tar -xf container-backup-and-restore-beta.tgz
   cd container-backup-and-restore
   sudo ./install.sh

Environment Configuration

The /etc/telos/backup-tac.env file contains configuration options for backup behavior.

Update these values as needed and run backup-tac to apply the changes immediately, otherwise they will be applied by the next scheduled execution.

Sample:

### After updating any environment values, run "backup-tac" to apply the setting. ###

# Enter the retention unit that local backups in "/opt/backup-telos/snapshots" should use. 
# "1" = days, "2" = files. Default "2" files. 
LOCAL_RETENTION_UNIT=2

# Enter the number of days or files (see LOCAL_RETENTION_UNIT) 
# for which local backups in "/opt/backup-telos/snapshots" should be retained. Default "7".
LOCAL_RETENTION_NUM=7

# Enter the frequency in which automatic backups are taken. 
# "1" = minutely, "2" = hourly, "3" = daily, "4" = weekly. Default "3" daily.
BACKUP_FREQUENCY=3

# Enter the name of the S3 Bucket dedicated to telos container backups for this host.
# Sub folders are supported as long as the syntax <bucket/path/to/folder> is used.
S3_BUCKET=

# Enable restore-tac to pull and install the latest S3 backup if "S3_BUCKET" is configured.
# Fallback to latest local backup if "S3_BUCKET" is not configured.
# "1" = enabled. Leave blank to disable feature.
LATEST_AT_STARTUP=

### After updating any environment values, run "backup-tac" to apply the setting. ###

Utilities

To invoke the utility, simply type in the shell one of the following commands, depending on the goal.

All the utilities included as part of this tool follow the naming scheme of function-tac.

Currently, the tool comprises the following:

• version-tac

• backup-tac

• restore-tac

• delete-tac

• volumes-tac

• startup-tac

Please find below an in-depth explanation of each utility.

Backup Utility: backup-tac

• Creates a timestamped backup tarball in /opt/backup-telos of the installed containers, Telos Alliance environment files and some nginx files.

• If configured in /etc/telos/backup-tac.env, uploads backups to an S3 bucket.

• Can be run manually or through a systemd service and timer. More on the systemd usage here.

Command:

• Writes timestamped backup "Snapshot" to /opt/backup-telos/snapshots and latest. Contents of the snapshots directory are rotated based on the settings in /etc/telos/backup-tac.env.

backup-tac

• Writes timestampted backup to /opt/backup-telos/archive. Contents of this directory are not rotated. This feature is useful for backups that you want to store long term.

backup-tac --archive

• Writes timestamped backup, with an identifier you specify to /opt/backup-telos/archive. Contents of this directory are not rotated. This feature is useful for backups that you want to store long term. Replace <identifier> with a name you'd like to add to the front of the filename.

backup-tac --archive <identifier>

Restore Utility: restore-tac

• Runs delete-tac to clear the existing container state and configuration.

• Prompts the user to proceed.

• Restores a backup from a local file, URL or S3 URI.

• Automatically handles container states (running or stopped) during restoration.

Note: When restoring to a system where the Docker images of the containers are not downloaded, you will be prompted to log into Quay to pull them again. Should this fail, you may log in with the command below before running the restore-tac utility.

sudo docker login quay.io

Commands:

• Restore from a local file:

  restore-tac path/to/file.tgz

• Restore from an object URL:

  restore-tac https://path/to/file.tgz

• Restore from an S3 URI:

  restore-tac s3://bucket/path/to/file.tgz

• Restore the latest backup:

  restore-tac --latest

NOTE: When using the restore-tac --latest feature, if an S3 bucket is configured in /etc/telos/backup-tac.env, restore-tac will pull and restore the latest from S3, ignoring the local backup.

Delete Utility: delete-tac

• Removes all containers and Telos Alliance-specific container configurations from the system.

Command:

delete-tac

Version Utility: version-tac

• Displays the currently installed version of the tool.

Command:

version-tac

Startup Utility: startup-tac

• startup-tac is run at startup by startup-tac.service. If LATEST_AT_STARTUP=1 in /etc/telos/backup-tac.env, then startup-tac will restore the latest s3 or local file. This setting is mostly to support use cases where S3 backup are utilized.

Advanced Usage

Startup Systemd Service: startup-tac.service

• Located at: /etc/systemd/system/startup-tac.service

• Manages the automatic execution of startup-tac based on the LATEST_AT_STARTUP variable in /etc/telos/backup-tac.env.

• When enabled, will load the latest local or S3 (if configured) backup at system boot.

• Very useful for failover workflows.

Backup Systemd Timer: backup-tac.timer

• Located at: /etc/systemd/system/backup-tac.timer.

• Manages the automatic execution of backup-tac based on the BACKUP_FREQUENCY variable in /etc/telos/backup-tac.env.

Backup Systemd Service: backup-tac.service

• Located at: /etc/systemd/system/backup-tac.service.

• When called by backup-tac.timer, executes the backup-tac utility.

S3 Integration

• Backups can be uploaded to an S3 bucket using the AWS CLI.

• Configure the S3 bucket in /etc/telos/backup-tac.env. It should follow the following syntax: S3_BUCKET=bucketname/path/to/folder/

• For AWS EC2 deployments, configure the S3 permissions by attaching an S3 role to the EC2 instance.

• For non-EC2 instances, run sudo aws configure.

Sample IAM Policy:

Replace bucket with your S3 Bucket name:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Telos-Container-Backup-and-Restore",
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:PutObject",
                "s3:GetObject"
            ],
            "Resource": "arn:aws:s3:::bucket/*"
        }
    ]
}

Volume Management: volumes-tac

volumes-tac is managed by the backup-tac and restore-tac utilities. Direct interaction is optional but available for advanced use cases.

Advanced usage:

• Export a volume to a gzip tarball:

  sudo volumes-tac export VOLUME FILE

• Import a gzip tarball into a volume:

  sudo volumes-tac import FILE VOLUME

• Save volume contents to a BusyBox image in /volume-data:

  sudo volumes-tac save VOLUME IMAGE

• Load contents from a BusyBox image into a volume:

  sudo volumes-tac load IMAGE VOLUME