ClickHouse DB::Exception: Backup already exists

The "DB::Exception: Backup already exists" error is raised by ClickHouse when you attempt to create a backup whose name matches one that already exists at the specified destination. The BACKUP_ALREADY_EXISTS error code signals that ClickHouse will not silently overwrite a previous backup, protecting you from accidental data loss.

Impact

When this error fires, the BACKUP statement fails immediately and no new backup is written. This can disrupt automated backup schedules if your scripts do not account for pre-existing backups. If left unaddressed, your backup pipeline may stall and you could miss your recovery point objective (RPO) window.

Common Causes

1. Re-running a backup command without changing the name -- executing the same BACKUP ... TO Disk('backups', 'daily_2024-01-15') command twice produces this error on the second run.
2. Scheduled jobs using a non-unique naming pattern -- a cron job that always writes to the same backup path instead of generating a timestamped name.
3. Partial failure on a previous attempt -- an earlier backup may have failed partway through but left enough metadata at the destination for ClickHouse to consider it "existing."
4. Multiple ClickHouse nodes targeting the same shared storage path -- in clustered setups, two nodes might race to create a backup with the same name on shared S3 or NFS storage.

Troubleshooting and Resolution Steps

1. Check existing backups. List what is already at the destination:

   SELECT * FROM system.backups ORDER BY start_time DESC;
   

2. Use a unique backup name. Include a timestamp or UUID to avoid collisions:

   BACKUP TABLE my_db.my_table
   TO Disk('backups', 'my_table_2024-01-15_1705312000')
   

3. Remove the old backup if it is no longer needed. Delete the backup directory from the underlying storage (disk, S3, etc.) and then retry:

   # For local disk backups
   rm -rf /var/lib/clickhouse/backups/daily_2024-01-15
   

4. Use the allow_backup_overwriting setting if your workflow intentionally replaces backups:

   BACKUP TABLE my_db.my_table
   TO Disk('backups', 'latest')
   SETTINGS allow_backup_overwriting = 1
   

5. Fix naming in automated scripts. Ensure your cron or orchestration tool generates unique names:

   BACKUP_NAME="daily_$(date +%Y%m%d_%H%M%S)"
   clickhouse-client --query "BACKUP DATABASE my_db TO Disk('backups', '${BACKUP_NAME}')"
   

6. Handle the error gracefully in pipelines. Wrap your backup command in logic that checks for the error and either renames or removes the conflicting backup before retrying.

Best Practices

• Always include a timestamp or monotonically increasing identifier in backup names.
• Implement a retention policy that cleans up old backups before new ones run.
• Use system.backups to verify backup state before and after operations.
• In clustered environments, coordinate backup names through a shared naming convention or a single orchestrator node.
• Test your backup-and-restore workflow in a staging environment to catch naming conflicts early.

Frequently Asked Questions

Q: Can I force ClickHouse to overwrite an existing backup?
A: Yes. Pass SETTINGS allow_backup_overwriting = 1 in your BACKUP statement. This tells ClickHouse to replace the existing backup at that path.

Q: Does a failed backup count as "already existing"?
A: It can. If enough metadata was written before the failure, ClickHouse may treat the destination as occupied. You will need to manually remove the incomplete backup files before retrying.

Q: How do I list all backups on disk?
A: Query SELECT * FROM system.backups to see backups ClickHouse knows about. For S3 or local disk, you can also list the storage directory directly to find backup folders.

Q: Is this error specific to a particular backup destination type?
A: No. The BACKUP_ALREADY_EXISTS error applies to all supported destinations -- local Disk, S3, and any other configured backup storage.