added ansible script
This commit is contained in:
liph
420
ansible/BACKUP_RESTORE.md
Normal file
420
ansible/BACKUP_RESTORE.md
Normal file
@@ -0,0 +1,420 @@
|
||||
# Backup and Restore Guide
|
||||
|
||||
Complete guide for backing up and restoring your Nextcloud Stack.
|
||||
|
||||
## Automated Backups
|
||||
|
||||
### What Gets Backed Up
|
||||
|
||||
- **Daily (3:00 AM)**:
|
||||
- PostgreSQL database dump
|
||||
|
||||
- **Weekly (Sundays, 3:00 AM)**:
|
||||
- Nextcloud data volume
|
||||
- Configuration files
|
||||
|
||||
- **Retention**: 30 days
|
||||
|
||||
### Backup Locations
|
||||
|
||||
```
|
||||
/opt/nextcloud-stack/backups/
|
||||
├── database/
|
||||
│ └── nextcloud_db_YYYYMMDD_HHMMSS.sql.gz
|
||||
├── volumes/
|
||||
│ ├── nextcloud_data_YYYYMMDD_HHMMSS.tar.gz
|
||||
│ └── configs_YYYYMMDD_HHMMSS.tar.gz
|
||||
└── backup.log
|
||||
```
|
||||
|
||||
### Manual Backup
|
||||
|
||||
```bash
|
||||
ssh user@server
|
||||
/opt/nextcloud-stack/backup.sh
|
||||
```
|
||||
|
||||
### Check Backup Status
|
||||
|
||||
```bash
|
||||
# View backup log
|
||||
tail -f /opt/nextcloud-stack/backups/backup.log
|
||||
|
||||
# List backups
|
||||
ls -lh /opt/nextcloud-stack/backups/database/
|
||||
ls -lh /opt/nextcloud-stack/backups/volumes/
|
||||
|
||||
# Check disk usage
|
||||
du -sh /opt/nextcloud-stack/backups/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Restore Procedures
|
||||
|
||||
### 1. Restore Database Only
|
||||
|
||||
**When to use:**
|
||||
- Database corruption
|
||||
- Accidental data deletion
|
||||
- Need to revert to earlier state
|
||||
|
||||
**Steps:**
|
||||
|
||||
```bash
|
||||
# 1. Stop Nextcloud (keep database running)
|
||||
cd /opt/nextcloud-stack
|
||||
docker compose stop next
|
||||
|
||||
# 2. List available backups
|
||||
ls -lh backups/database/
|
||||
|
||||
# 3. Choose backup to restore
|
||||
BACKUP_FILE="backups/database/nextcloud_db_20260216_030000.sql.gz"
|
||||
|
||||
# 4. Drop existing database (CAUTION!)
|
||||
docker exec next-db psql -U nextcloud -c "DROP DATABASE nextcloud;"
|
||||
docker exec next-db psql -U nextcloud -c "CREATE DATABASE nextcloud;"
|
||||
|
||||
# 5. Restore from backup
|
||||
gunzip < $BACKUP_FILE | docker exec -i next-db psql -U nextcloud -d nextcloud
|
||||
|
||||
# 6. Restart Nextcloud
|
||||
docker compose start next
|
||||
|
||||
# 7. Run Nextcloud upgrade (if needed)
|
||||
docker exec -u www-data next php occ upgrade
|
||||
|
||||
# 8. Disable maintenance mode
|
||||
docker exec -u www-data next php occ maintenance:mode --off
|
||||
```
|
||||
|
||||
### 2. Restore Data Volumes
|
||||
|
||||
**When to use:**
|
||||
- Lost files
|
||||
- Corrupted data directory
|
||||
- Complete system failure
|
||||
|
||||
**Steps:**
|
||||
|
||||
```bash
|
||||
# 1. Stop all services
|
||||
cd /opt/nextcloud-stack
|
||||
docker compose down
|
||||
|
||||
# 2. List available backups
|
||||
ls -lh backups/volumes/
|
||||
|
||||
# 3. Choose backup
|
||||
BACKUP_FILE="backups/volumes/nextcloud_data_20260216_030000.tar.gz"
|
||||
|
||||
# 4. Restore volume
|
||||
# Note: This requires accessing the Docker volume directory
|
||||
sudo tar -xzf $BACKUP_FILE -C /var/lib/docker/volumes/nextcloud_nextcloud_data/_data/
|
||||
|
||||
# 5. Restore configs (optional)
|
||||
CONFIG_BACKUP="backups/volumes/configs_20260216_030000.tar.gz"
|
||||
sudo tar -xzf $CONFIG_BACKUP -C /opt/nextcloud-stack/configs/
|
||||
|
||||
# 6. Fix permissions
|
||||
sudo chown -R www-data:www-data /var/lib/docker/volumes/nextcloud_nextcloud_data/_data/
|
||||
|
||||
# 7. Start services
|
||||
docker compose up -d
|
||||
|
||||
# 8. Run Nextcloud file scan
|
||||
docker exec -u www-data next php occ files:scan --all
|
||||
```
|
||||
|
||||
### 3. Complete System Restore
|
||||
|
||||
**When to use:**
|
||||
- Disaster recovery
|
||||
- Migration to new server
|
||||
- Complete system failure
|
||||
|
||||
**Steps:**
|
||||
|
||||
```bash
|
||||
# 1. On new server, install base system
|
||||
# Run Ansible playbooks 01-04 (up to Tailscale)
|
||||
ansible-playbook playbooks/01-preflight-checks.yml --ask-vault-pass
|
||||
ansible-playbook playbooks/02-system-setup.yml --ask-vault-pass
|
||||
ansible-playbook playbooks/03-docker-setup.yml --ask-vault-pass
|
||||
ansible-playbook playbooks/04-tailscale-setup.yml --ask-vault-pass
|
||||
|
||||
# 2. Create deployment directory structure
|
||||
ssh user@new-server
|
||||
sudo mkdir -p /opt/nextcloud-stack/backups/{database,volumes}
|
||||
|
||||
# 3. Copy backups from old server to new server
|
||||
# On your local machine:
|
||||
scp -r old-server:/opt/nextcloud-stack/backups/* user@new-server:/opt/nextcloud-stack/backups/
|
||||
|
||||
# 4. Deploy stack (creates volumes and containers)
|
||||
ansible-playbook playbooks/05-deploy-stack.yml --ask-vault-pass
|
||||
|
||||
# 5. Stop services on new server
|
||||
ssh user@new-server
|
||||
cd /opt/nextcloud-stack
|
||||
docker compose down
|
||||
|
||||
# 6. Restore database
|
||||
BACKUP_FILE="backups/database/nextcloud_db_20260216_030000.sql.gz"
|
||||
docker compose up -d next-db next-redis
|
||||
sleep 10
|
||||
gunzip < $BACKUP_FILE | docker exec -i next-db psql -U nextcloud -d nextcloud
|
||||
|
||||
# 7. Restore data volumes
|
||||
VOLUME_BACKUP="backups/volumes/nextcloud_data_20260216_030000.tar.gz"
|
||||
sudo tar -xzf $VOLUME_BACKUP -C /var/lib/docker/volumes/nextcloud_nextcloud_data/_data/
|
||||
|
||||
# 8. Restore configs
|
||||
CONFIG_BACKUP="backups/volumes/configs_20260216_030000.tar.gz"
|
||||
sudo tar -xzf $CONFIG_BACKUP -C /opt/nextcloud-stack/configs/
|
||||
|
||||
# 9. Fix permissions
|
||||
sudo chown -R www-data:www-data /var/lib/docker/volumes/nextcloud_nextcloud_data/_data/
|
||||
|
||||
# 10. Start all services
|
||||
docker compose up -d
|
||||
|
||||
# 11. Run Nextcloud maintenance
|
||||
docker exec -u www-data next php occ maintenance:mode --off
|
||||
docker exec -u www-data next php occ files:scan --all
|
||||
docker exec -u www-data next php occ db:add-missing-indices
|
||||
|
||||
# 12. Continue with remaining playbooks
|
||||
exit
|
||||
ansible-playbook playbooks/06-configure-caddy.yml --ask-vault-pass
|
||||
ansible-playbook playbooks/07-setup-backups.yml --ask-vault-pass
|
||||
ansible-playbook playbooks/08-post-deployment.yml --ask-vault-pass
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Off-Site Backups with rclone
|
||||
|
||||
### Initial Setup
|
||||
|
||||
```bash
|
||||
ssh user@server
|
||||
|
||||
# Configure rclone
|
||||
rclone config
|
||||
|
||||
# Example: Setup Backblaze B2
|
||||
# Follow prompts:
|
||||
# - Choose: New remote
|
||||
# - Name: b2backup
|
||||
# - Storage: Backblaze B2
|
||||
# - Enter account ID and application key
|
||||
```
|
||||
|
||||
### Sync Backups to Remote
|
||||
|
||||
```bash
|
||||
# Test sync (dry run)
|
||||
rclone sync /opt/nextcloud-stack/backups/ b2backup:nextcloud-backups --dry-run
|
||||
|
||||
# Actual sync
|
||||
rclone sync /opt/nextcloud-stack/backups/ b2backup:nextcloud-backups
|
||||
```
|
||||
|
||||
### Automated Off-Site Backups
|
||||
|
||||
Add to `/opt/nextcloud-stack/backup.sh` (after the cleanup section):
|
||||
|
||||
```bash
|
||||
# Off-site backup (if rclone configured)
|
||||
if command -v rclone &> /dev/null && rclone listremotes | grep -q "b2backup"; then
|
||||
log "Syncing backups to off-site storage..."
|
||||
rclone sync $BACKUP_DIR b2backup:nextcloud-backups --log-file=$LOG_FILE
|
||||
log "Off-site sync completed"
|
||||
fi
|
||||
```
|
||||
|
||||
### Restore from Off-Site Backup
|
||||
|
||||
```bash
|
||||
# List remote backups
|
||||
rclone ls b2backup:nextcloud-backups/database/
|
||||
|
||||
# Download specific backup
|
||||
rclone copy b2backup:nextcloud-backups/database/nextcloud_db_20260216_030000.sql.gz /opt/nextcloud-stack/backups/database/
|
||||
|
||||
# Download all backups
|
||||
rclone sync b2backup:nextcloud-backups /opt/nextcloud-stack/backups/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Backup Verification
|
||||
|
||||
### Test Database Backup Integrity
|
||||
|
||||
```bash
|
||||
# 1. Extract backup
|
||||
gunzip -c backups/database/nextcloud_db_20260216_030000.sql.gz > /tmp/test_restore.sql
|
||||
|
||||
# 2. Check for errors
|
||||
grep -i "error" /tmp/test_restore.sql
|
||||
|
||||
# 3. Verify size (should be reasonable)
|
||||
ls -lh /tmp/test_restore.sql
|
||||
|
||||
# 4. Cleanup
|
||||
rm /tmp/test_restore.sql
|
||||
```
|
||||
|
||||
### Test Volume Backup Integrity
|
||||
|
||||
```bash
|
||||
# Test tar archive
|
||||
tar -tzf backups/volumes/nextcloud_data_20260216_030000.tar.gz | head -20
|
||||
|
||||
# Check for errors
|
||||
tar -tzf backups/volumes/nextcloud_data_20260216_030000.tar.gz > /dev/null && echo "Archive OK"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Migration to New Server
|
||||
|
||||
**Complete migration guide:**
|
||||
|
||||
1. **Prepare new server:**
|
||||
- Same or newer Ubuntu version
|
||||
- Run Ansible setup playbooks (01-04)
|
||||
|
||||
2. **Backup old server:**
|
||||
```bash
|
||||
/opt/nextcloud-stack/backup.sh
|
||||
```
|
||||
|
||||
3. **Copy backups to new server:**
|
||||
```bash
|
||||
rsync -avz old-server:/opt/nextcloud-stack/backups/ new-server:/opt/nextcloud-stack/backups/
|
||||
```
|
||||
|
||||
4. **Update DNS:**
|
||||
- Point all domains to new server IP
|
||||
- Wait for propagation
|
||||
|
||||
5. **Restore on new server:**
|
||||
- Follow "Complete System Restore" steps above
|
||||
|
||||
6. **Verify:**
|
||||
- Test all services
|
||||
- Check Nextcloud functionality
|
||||
- Verify SSL certificates
|
||||
|
||||
7. **Decommission old server:**
|
||||
- Only after confirming new server works
|
||||
- Keep final backup from old server
|
||||
|
||||
---
|
||||
|
||||
## Backup Best Practices
|
||||
|
||||
### DO:
|
||||
- ✅ Test restores regularly (monthly)
|
||||
- ✅ Keep backups off-site (rclone to cloud storage)
|
||||
- ✅ Verify backup integrity
|
||||
- ✅ Monitor backup log for errors
|
||||
- ✅ Document restore procedures
|
||||
- ✅ Keep multiple backup generations
|
||||
|
||||
### DON'T:
|
||||
- ❌ Store backups only on same server
|
||||
- ❌ Assume backups work without testing
|
||||
- ❌ Delete old backups without verification
|
||||
- ❌ Skip database backups
|
||||
- ❌ Ignore backup failure notifications
|
||||
|
||||
---
|
||||
|
||||
## Backup Schedule Customization
|
||||
|
||||
Edit `/opt/nextcloud-stack/backup.sh` to change:
|
||||
|
||||
- Backup frequency
|
||||
- Retention period
|
||||
- What gets backed up
|
||||
- Compression settings
|
||||
|
||||
Edit cron job:
|
||||
```bash
|
||||
sudo crontab -e
|
||||
|
||||
# Change from 3:00 AM to 2:00 AM
|
||||
0 2 * * * /opt/nextcloud-stack/backup.sh >> /opt/nextcloud-stack/backups/backup.log 2>&1
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting Backups
|
||||
|
||||
### Backup script fails
|
||||
|
||||
**Check logs:**
|
||||
```bash
|
||||
tail -100 /opt/nextcloud-stack/backups/backup.log
|
||||
```
|
||||
|
||||
**Common issues:**
|
||||
- Disk space full
|
||||
- Docker container not running
|
||||
- Permission denied
|
||||
|
||||
**Solutions:**
|
||||
```bash
|
||||
# Check disk space
|
||||
df -h
|
||||
|
||||
# Check containers
|
||||
docker ps
|
||||
|
||||
# Fix permissions
|
||||
sudo chown -R root:root /opt/nextcloud-stack/backup.sh
|
||||
sudo chmod +x /opt/nextcloud-stack/backup.sh
|
||||
```
|
||||
|
||||
### Database backup empty
|
||||
|
||||
**Verify database is running:**
|
||||
```bash
|
||||
docker exec next-db pg_isready -U nextcloud
|
||||
```
|
||||
|
||||
**Test manual backup:**
|
||||
```bash
|
||||
docker exec next-db pg_dump -U nextcloud nextcloud > /tmp/test.sql
|
||||
ls -lh /tmp/test.sql
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Emergency Recovery
|
||||
|
||||
If all backups are lost and you need to start fresh:
|
||||
|
||||
```bash
|
||||
# 1. Remove everything
|
||||
ansible-playbook playbooks/99-rollback.yml --ask-vault-pass
|
||||
cd /opt/nextcloud-stack
|
||||
docker compose down -v
|
||||
|
||||
# 2. Redeploy from scratch
|
||||
ansible-playbook playbooks/site.yml --ask-vault-pass
|
||||
|
||||
# 3. Reconfigure Nextcloud manually
|
||||
# Login and set up users, apps, etc.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
**Remember:** Backups are worthless unless you've tested restoring from them!
|
||||
|
||||
**Last Updated:** 2026-02-16
|
||||
Reference in New Issue
Block a user