Modernize repository: Fix shell scripts, update documentation, and improve configurability

.gitignore

@@ -1,2 +1,4 @@
 b2_env.sh
 b2_pw.txt
+.DS_Store
+**/.DS_Store

LICENSE

@@ -1,7 +1,7 @@
 restic-systemd-automatic-backup -  My restic backup solution using Backblaze B2 storage, systemd timers (or cron) and email notifications on failure.
 
 
-Copyright (c) 2018, see commit log for auhtors
+Copyright (c) 2018, see commit log for authors
 All rights reserved.
 
 Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

README.md

@@ -1,105 +1,138 @@
-## Directadmin Restic Backup using S3 Block Storage
+## DirectAdmin Restic Backup using S3 Block Storage
 
 # Requirements
-* `restic >=v0.9.6`
-* `zstd:  for mysql backups`
+* `restic` (latest stable version recommended, minimum v0.9.6)
+* `zstd` for MySQL backups
+* `git` for installation
 
 ## Required: Install Restic
 
 [restic](https://restic.net/) is a command-line tool for making backups.
 
-Ubuntu:
+Ubuntu/Debian:
 ```bash
-$ apt-get install restic && apt-get install git
-
-sudo apt-get update && sudo apt-get install -y software-properties-common && sudo add-apt-repository -y ppa:copart/restic && sudo apt-get update && sudo apt-get install -y restic git
-````
+sudo apt-get update
+sudo apt-get install -y restic zstd git
+```
 
-CentOS:
+CentOS/RHEL:
 ```bash
-$ yum install yum-plugin-copr && yum copr enable copart/restic && yum install restic && yum install git
+sudo yum install -y epel-release
+sudo yum install -y restic zstd git
+```
 
-sudo apt-get update && sudo apt-get install -y software-properties-common && sudo add-apt-repository -y ppa:copart/restic && sudo apt-get update && sudo apt-get install -y restic git
-````
+For the latest version, you can download from the [official releases page](https://github.com/restic/restic/releases).
 
-## Installguide Directadmin VPS Backup
+## Installation Guide for DirectAdmin VPS Backup
 
-Tip: The steps in this section will instruct you to copy files from this repo to system directories.
+The following steps will install the backup scripts and configuration files to system directories.
 
 ```bash
-$ git clone https://github.com/payrequestio/directadmin-vps-backup.git
-$ cd directadmin-vps-backup
-$ sudo make install
-````
+git clone https://github.com/payrequestio/directadmin-restic-backup.git
+cd directadmin-restic-backup
+sudo make install
+```
 
 
 ### 1. Configure S3 credentials
-Put these files in `/etc/restic/`:
-* `env.sh`: Fill this file out with your S3 bucket settings. The reason for putting these in a separate file is that it can be used also for you to simply source, when you want to issue some restic commands. For example:
+Edit `/etc/restic/env.sh` with your S3 bucket settings. The installation will create this file from the template if it doesn't exist.
+
+Required configuration:
+* `AWS_ACCESS_KEY_ID`: Your S3 access key
+* `AWS_SECRET_ACCESS_KEY`: Your S3 secret key
+* `RESTIC_PASSWORD`: Password for encrypting backups
+* `RESTIC_REPOSITORY`: S3 repository URL (e.g., `s3:https://s3.amazonaws.com/your-bucket-name`)
+
+Optional configuration:
+* `BACKUP_EMAIL`: Email address for backup notifications (used by restic_backup.sh)
+* `DISCORD_WEBHOOK`: Discord webhook URL for notifications (used by directadmin-vps-backup.sh)
+
+Once configured, you can source the file to use restic commands easily:
 ```bash
-$ source /etc/restic/env.sh
-$ restic snapshots    # You don't have to supply all parameters like --repo, as they are now in your environment!
-````
+source /etc/restic/env.sh
+restic snapshots    # You don't have to supply all parameters like --repo, as they are now in your environment!
+```
 
 ### 2. Initialize remote repo
 Now we must initialize the repository on the remote end:
 ```bash
 source /etc/restic/env.sh && restic init
 ```
 
-### 3. Script for doing the backup
-Put this file in `/usr/local/sbin`:
-* `directadmin-vps-backup.sh`: A script that defines how to run the backup. Edit this file to respect your needs in terms of backup which paths to backup, retention (number of backups to save), etc.
+### 3. Configure backup settings
+The installation places these files:
+* `/usr/local/sbin/directadmin-vps-backup.sh`: Main backup script. Edit this to customize:
+  - Backup paths (default: `/`, `/boot`, `/home`)
+  - Retention policy (default: 7 days, 4 weeks, 3 months, 0 years)
+  - Discord notifications
+* `/etc/restic/backup_exclude`: File patterns to exclude from backups (cache, logs, temporary files, etc.)
 
-Put this file in `/`:
-* `.backup_exclude`: A list of file pattern paths to exclude from you backups, files that just occupy storage space, backup-time, network and money.
+You can also create per-user exclusion files at `$HOME/.backup_exclude` for each user.
 
 
-### 4. Make first backup & verify
-Now see if the backup itself works, by running
+### 4. Run first backup & verify
+Test the backup to ensure everything works:
 
 ```bash
-$ /usr/local/sbin/directadmin-vps-backup.sh
-$ restic snapshots
-````
-
-### 5. Backup automatically; systemd service + timer
-Now we can do the modern version of a cron-job, a systemd service + timer, to run the backup every day!
+/usr/local/sbin/directadmin-vps-backup.sh
+source /etc/restic/env.sh
+restic snapshots
+```
 
+### 5. Enable automatic backups with systemd
+The installation includes systemd service and timer units:
+* `directadmin-vps-backup.service`: Service that executes the backup script
+* `directadmin-vps-backup.timer`: Timer that runs backups daily
+* `directadmin-vps-backup-check.service`: Service for checking backup integrity
+* `directadmin-vps-backup-check.timer`: Timer that runs checks monthly
 
-Put these files in `/etc/systemd/system/`:
-* `directadmin-vps-backup.service`: A service that calls the backup script.
-* `directadmin-vps-backup.timer`: A timer that starts the backup every day.
+Enable and start the backup timer:
+```bash
+systemctl daemon-reload
+systemctl enable --now directadmin-vps-backup.timer
+systemctl enable --now directadmin-vps-backup-check.timer
+```
 
+### 6. Managing backups
 
-Now simply enable the timer with:
+View scheduled backup times:
 ```bash
-$ systemctl start directadmin-vps-backup.timer
-$ systemctl enable directadmin-vps-backup.timer
-````
+systemctl list-timers | grep directadmin-vps-backup
+```
 
-You can see when your next backup is scheduled to run with
+Check backup status:
 ```bash
-$ systemctl list-timers | grep directadmin-vps-backup
+systemctl status directadmin-vps-backup
 ```
 
-and see the status of a currently running backup with
-
+Start a backup manually:
 ```bash
-$ systemctl status directadmin-vps-backup
+systemctl start directadmin-vps-backup
 ```
 
-or start a backup manually
+View backup logs in real-time:
+```bash
+journalctl -f -u directadmin-vps-backup.service
+```
 
+View all backup logs:
 ```bash
-$ systemctl start directadmin-vps-backup
+journalctl -u directadmin-vps-backup.service
 ```
 
-You can follow the backup stdout output live as backup is running with:
+## Additional Scripts
 
-```bash
-$ journalctl -f -u directadmin-vps-backup.service
-````
+This repository includes additional helper scripts:
+* `restic_backup.sh`: Alternative backup script with email notifications
+* `restic_check.sh`: Repository integrity check script
+* `mysql.sh`: MySQL database backup script (dumps to `/home/mysqlbackups` with zstd compression)
+
+## Troubleshooting
 
-(skip `-f` to see all backups that has run)
+If you encounter issues:
+1. Check that restic is properly installed: `restic version`
+2. Verify your S3 credentials in `/etc/restic/env.sh`
+3. Test repository access: `source /etc/restic/env.sh && restic snapshots`
+4. Review service logs: `journalctl -u directadmin-vps-backup.service`
+5. Ensure sufficient disk space for MySQL dumps and temporary files
 

etc/restic/env.sh.template

@@ -4,3 +4,12 @@ export AWS_ACCESS_KEY_ID="IDhere"
 export AWS_SECRET_ACCESS_KEY="Keyhere"
 export RESTIC_PASSWORD="passwordhere"
 export RESTIC_REPOSITORY="s3:https://s3.amazonaws.com/bucketname"
+
+# Optional: Email notifications (used by restic_backup.sh)
+# export BACKUP_EMAIL="admin@example.com"
+
+# Optional: Systemd failure notification email (used by status-email-user@.service)
+# export SYSTEMD_EMAIL="admin@example.com"
+
+# Optional: Discord webhook for notifications (used by directadmin-vps-backup.sh)
+# export DISCORD_WEBHOOK="https://discord.com/api/webhooks/your-webhook-url"

etc/systemd/system/status-email-user@.service

@@ -1,11 +1,14 @@
 # Source: https://serverfault.com/questions/876233/how-to-send-an-email-if-a-systemd-service-is-restarted
 # Source: https://wiki.archlinux.org/index.php/Systemd/Timers#MAILTO
+#
+# NOTE: Set SYSTEMD_EMAIL environment variable in /etc/restic/env.sh
+# or the default admin@example.com will be used
 
 [Unit]
 Description=Send status email for %i to user
 
 [Service]
 Type=oneshot
-ExecStart=/usr/local/sbin/systemd-email abc@gmail.com %i
+ExecStart=/usr/local/sbin/systemd-email-wrapper.sh %i
 User=root
 Group=systemd-journal

usr/local/sbin/directadmin-vps-backup.sh

@@ -1,15 +1,16 @@
 #!/usr/bin/env bash
 # Make backup my system with restic
 
-# Run first Directadmin Cleaner
-bash /usr/local/sbin/directadmin-cleaner.sh &
-wait $!
+# Run first Directadmin Cleaner (optional - create this script if you need cleanup before backup)
+if [ -f /usr/local/sbin/directadmin-cleaner.sh ]; then
+    bash /usr/local/sbin/directadmin-cleaner.sh &
+    wait $!
+fi
 
-## add your discord channel webhook
-discord="URL"
+## add your discord channel webhook (or set DISCORD_WEBHOOK in env.sh)
+discord="${DISCORD_WEBHOOK:-URL}"
 
 # Check available disk space
-total_space=$(df -H | awk '{if($NF=="/") print $2}' | tr -d 'G')
 free_space=$(df -H | awk '{if($NF=="/") print $4}' | tr -d 'G')
 required_space=$(du -sh /var/lib/mysql | tr -d 'G' | awk '{print $1}')
 if (( $(echo "$free_space < $required_space" | bc -l) )); then
@@ -105,10 +106,8 @@ wait $!
 #restic check &
 #wait $!
 
-RESTICSNAPSHOTS="restic snapshots --no-lock --json --repo ${RESTIC_REPOSITORY}"
-RESTICOUTPUT=$(eval "$RESTICSNAPSHOTS" | grep -oP '"short_id":"\K[0-9a-f]+|"time":"\K[^"]+' | paste -d' ' - - | sed 's/T/ /; s/\.\(.*\)Z/\1/' )
-COUNT=$(restic snapshots --compact --repo ${RESTIC_REPOSITORY} | awk -F '\t' '{print $1}' | wc -l)
-HOSTNAME=`hostname`
+COUNT=$(restic snapshots --compact --repo "${RESTIC_REPOSITORY}" | awk -F '\t' '{print $1}' | wc -l)
+HOSTNAME=$(hostname)
 
 
 

usr/local/sbin/mysql.sh

@@ -1,10 +1,10 @@
 #!/bin/bash
 
 USER="da_admin"
-PASSWORD="`grep passwd= /usr/local/directadmin/conf/mysql.conf | cut -d\= -f2`"
+PASSWORD="$(grep passwd= /usr/local/directadmin/conf/mysql.conf | cut -d= -f2)"
 BACKUP_RETAIN_DAYS=0   ## Number of days to keep local backup copy
 DB_BACKUP_PATH='/home/mysqlbackups'
-TODAY=`date +"%Y%m%d"`
+TODAY=$(date +"%Y%m%d")
 
 #OUTPUT="/Users/rabino/DBs"
 
@@ -16,13 +16,12 @@ fi
 
 find ${DB_BACKUP_PATH} -name '*.zst' -type f -mtime +${BACKUP_RETAIN_DAYS} -exec rm -f {} \;
 
-databases=`mysql -u $USER -p$PASSWORD -e "SHOW DATABASES;" | tr -d "| " | grep -v Database`
+databases=$(mysql -u "$USER" -p"$PASSWORD" -e "SHOW DATABASES;" | tr -d "| " | grep -v Database)
 
 for db in $databases; do
     if [[ "$db" != "information_schema" ]] && [[ "$db" != "performance_schema" ]] && [[ "$db" != "mysql" ]] && [[ "$db" != _* ]] ; then
         echo "Dumping database: $db"
-        #mysqldump -u $USER -p$PASSWORD --databases $db | gzip > ${DB_BACKUP_PATH}/${TODAY}.$db.sql.gz
-        mysqldump -u $USER -p$PASSWORD --databases $db | zstd > ${DB_BACKUP_PATH}/${TODAY}.$db.sql.zst
-       # gzip $OUTPUT/`date +%Y%m%d`.$db.sql
+        #mysqldump -u "$USER" -p"$PASSWORD" --databases "$db" | gzip > "${DB_BACKUP_PATH}/${TODAY}.${db}.sql.gz"
+        mysqldump -u "$USER" -p"$PASSWORD" --databases "$db" | zstd > "${DB_BACKUP_PATH}/${TODAY}.${db}.sql.zst"
     fi
 done

usr/local/sbin/restic_backup.sh

@@ -1,3 +1,6 @@
+#!/usr/bin/env bash
+# Backup script using restic
+# This script is typically run by: /etc/systemd/system/directadmin-vps-backup.{service,timer}
 
 # Set all environment variables like
 # B2_ACCOUNT_ID, B2_ACCOUNT_KEY, RESTIC_REPOSITORY etc.
@@ -23,7 +26,7 @@ wait $!
 restic backup \
         --verbose \
         --one-file-system \
-        --tag $BACKUP_TAG \
+        --tag "$BACKUP_TAG" \
         $BACKUP_EXCLUDES \
         $BACKUP_PATHS &
 wait $!
@@ -33,20 +36,24 @@ wait $!
 # --group-by only the tag and path, and not by hostname. This is because I create a B2 Bucket per host, and if this hostname accidentially change some time, there would now be multiple backup sets.
 restic forget \
         --verbose \
-        --tag $BACKUP_TAG \
+        --tag "$BACKUP_TAG" \
         --prune \
         --group-by "paths,tags" \
-        --keep-daily $RETENTION_DAYS \
-        --keep-weekly $RETENTION_WEEKS \
-        --keep-monthly $RETENTION_MONTHS \
-        --keep-yearly $RETENTION_YEARS &
+        --keep-daily "$RETENTION_DAYS" \
+        --keep-weekly "$RETENTION_WEEKS" \
+        --keep-monthly "$RETENTION_MONTHS" \
+        --keep-yearly "$RETENTION_YEARS" &
 wait $!
 
 # Check repository for errors.
 # NOTE this takes much time (and data transfer from remote repo?), do this in a separate systemd.timer which is run less often.
 #restic check &
 #wait $!
 
-RESTICOUTPUT=`restic snapshots --repo ${RESTIC_REPOSITORY}`
-HOSTNAME=`hostname`
-echo "Backup ${HOSTNAME} has finished, we keep ${RETENTION_DAYS} each day. \n ${RESTICOUTPUT}" | mail -s "Backup done ${HOSTNAME}" geertjan@hostingwalk.com
+RESTICOUTPUT=$(restic snapshots --repo "${RESTIC_REPOSITORY}")
+HOSTNAME=$(hostname)
+# Configure email recipient in env.sh by setting BACKUP_EMAIL variable
+# Example: export BACKUP_EMAIL="admin@example.com"
+if [ -n "${BACKUP_EMAIL:-}" ]; then
+    printf "Backup %s has finished, we keep %s each day.\n\n%s" "${HOSTNAME}" "${RETENTION_DAYS}" "${RESTICOUTPUT}" | mail -s "Backup done ${HOSTNAME}" "${BACKUP_EMAIL}"
+fi

usr/local/sbin/restic_check.sh

@@ -16,7 +16,7 @@ exit_hook() {
 trap exit_hook INT TERM
 
 
-source /etc/restic/b2_env.sh
+source /etc/restic/env.sh
 
 # How many network connections to set up to B2. Default is 5.
 B2_CONNECTIONS=50

usr/local/sbin/systemd-email-wrapper.sh

@@ -0,0 +1,21 @@
+#!/usr/bin/env bash
+# Wrapper script for systemd-email to handle environment variables
+# This ensures SYSTEMD_EMAIL is properly set before sending notifications
+
+# Validate required argument
+if [ -z "$1" ]; then
+    echo "Error: Unit name required" >&2
+    exit 1
+fi
+
+# Source environment if available
+if [ -f /etc/restic/env.sh ]; then
+    . /etc/restic/env.sh
+fi
+
+# Use SYSTEMD_EMAIL from environment, or fall back to default
+EMAIL="${SYSTEMD_EMAIL:-admin@example.com}"
+UNIT="$1"
+
+# Call the actual systemd-email script
+exec /usr/local/sbin/systemd-email "$EMAIL" "$UNIT"