ClickHouse DB::Exception: Atomic rename failed

The "DB::Exception: Atomic rename failed" error in ClickHouse occurs when an atomic rename operation on a table or database cannot be completed. The ATOMIC_RENAME_FAIL error code is raised when the filesystem-level or metadata-level rename that ClickHouse attempts as a single atomic action does not succeed.

Impact

The RENAME TABLE or EXCHANGE TABLES statement fails, and neither the source nor the target object is modified — the operation is atomic, so it either succeeds entirely or not at all. Ongoing queries against the existing table are not affected, but any workflow depending on the rename (such as a blue-green deployment pattern) will be blocked.

Common Causes

1. The target table or database name already exists and conflicts with the rename
2. A filesystem error prevents the metadata file from being renamed (permissions, disk full, or a read-only mount)
3. The source and target reside on different databases that use incompatible engines (for example, renaming across Atomic and Ordinary database engines)
4. ZooKeeper or ClickHouse Keeper is unreachable, blocking rename operations on replicated tables
5. Another concurrent DDL operation holds a lock on the table being renamed
6. The table has active dependencies (materialized views, dictionaries) that prevent the rename

Troubleshooting and Resolution Steps

1. Check the exact error message in the server log for additional context:

   grep -i 'ATOMIC_RENAME_FAIL\|rename.*fail' /var/log/clickhouse-server/clickhouse-server.log | tail -20
   

2. Verify that the target name does not already exist:

   SHOW TABLES FROM my_database LIKE 'target_name';
   

3. If renaming across databases, ensure both use the Atomic engine:

   SELECT name, engine FROM system.databases;
   

   Renaming between an Atomic and Ordinary database is not supported. Consider recreating the table in the target database instead.

4. Check for concurrent DDL operations that might hold locks:

   SELECT query_id, query, elapsed
   FROM system.processes
   WHERE query LIKE '%RENAME%' OR query LIKE '%ALTER%' OR query LIKE '%DROP%';
   

5. Verify filesystem health and permissions on the metadata directory:

   ls -la /var/lib/clickhouse/metadata/my_database/
   df -h /var/lib/clickhouse/
   

6. For replicated tables, ensure ZooKeeper or ClickHouse Keeper is healthy:

   SELECT * FROM system.zookeeper WHERE path = '/clickhouse';
   

7. If the table has dependent materialized views, you may need to drop or detach them first, perform the rename, then recreate the dependencies.

Best Practices

• Use the Atomic database engine for all databases to enable safe, atomic rename and exchange operations.
• Avoid renaming tables across databases with different engines; instead, use CREATE TABLE ... AS SELECT to move data.
• Ensure adequate free disk space on the metadata volume, as even metadata-only operations need some space for temporary files.
• Coordinate DDL operations in multi-user environments to avoid locking conflicts during renames.
• Use EXCHANGE TABLES instead of a rename chain when swapping two tables to reduce the window of unavailability.

Frequently Asked Questions

Q: What is the difference between RENAME TABLE and EXCHANGE TABLES?
A: RENAME TABLE a TO b moves a table's name. EXCHANGE TABLES a AND b swaps two tables' names atomically. Both require the Atomic database engine for true atomicity.

Q: Can I rename a table to a different database?
A: Yes, but only if both databases use the Atomic engine. Cross-engine renames are not supported and will produce the ATOMIC_RENAME_FAIL error.

Q: Is there any risk of data loss from a failed rename?
A: No. Atomic renames are all-or-nothing. If the rename fails, the original table remains intact and accessible under its original name.