The DB::Exception: Replica already exists error (code REPLICA_ALREADY_EXISTS) is thrown when ClickHouse tries to create a replicated table but discovers that a ZooKeeper node with the same replica path already exists. This means another replica -- or a leftover from a previous instance -- has already claimed that identity.
Impact
The CREATE TABLE statement fails, and the table is not created on this node. The existing replica in ZooKeeper continues to function normally. Until the conflict is resolved, the affected node cannot participate in replication for this table, which may reduce cluster redundancy and query distribution capacity.
Common Causes
- Re-creating a table without cleaning up ZooKeeper -- the table was dropped on this node (or the node was rebuilt), but the corresponding ZooKeeper metadata was not removed.
- Duplicate replica macro -- two servers resolve
{replica}to the same value, causing a naming collision. - Restored backup on a new node -- a backup was restored that includes the original replica identity, conflicting with the still-registered original.
- Cluster scaling misconfiguration -- a new node was added with a replica name already in use.
Troubleshooting and Resolution Steps
Confirm the conflicting path in ZooKeeper Check what replicas are registered for the table:
SELECT name FROM system.zookeeper WHERE path = '/clickhouse/tables/01/my_table/replicas';Look for the replica name that matches your current node's
{replica}macro.Verify macro uniqueness across all nodes On each server, run:
SELECT * FROM system.macros;Ensure no two nodes share the same
{replica}value for the same shard.Clean up stale ZooKeeper entries If the old replica is genuinely gone (decommissioned, rebuilt), remove it from ZooKeeper:
SYSTEM DROP REPLICA 'old_replica_name' FROM TABLE db.my_table;This command removes the replica metadata from ZooKeeper without affecting other replicas.
Drop the table with ZooKeeper cleanup If you intended to recreate the table from scratch:
DROP TABLE IF EXISTS db.my_table SYNC;If the table was already dropped but ZooKeeper entries remain, use the
SYSTEM DROP REPLICAcommand from step 3.Assign a unique replica name If both replicas are legitimate, change the
{replica}macro on the conflicting node to a unique value:<macros> <replica>clickhouse-node-3</replica> </macros>Restart ClickHouse and retry the
CREATE TABLEstatement.Use DNS or hostname-based macros To avoid future collisions, derive the replica name from the hostname automatically:
<macros> <replica>{hostname}</replica> </macros>
Best Practices
- Use
SYSTEM DROP REPLICAto clean up metadata when decommissioning nodes, rather than relying on ZooKeeper session expiry alone. - Derive replica names from hostnames or stable, unique identifiers to prevent collisions.
- Document your naming convention for shards and replicas and enforce it through configuration management.
- Before restoring a backup onto a new node, update the replica name in the macros configuration.
- Audit ZooKeeper paths periodically to identify orphaned replica entries.
Frequently Asked Questions
Q: Can I just delete the replica path directly in ZooKeeper?
A: While possible, it is safer to use SYSTEM DROP REPLICA from within ClickHouse, as it ensures all related metadata is cleaned up correctly.
Q: Will dropping the replica from ZooKeeper affect the data on other replicas?
A: No. SYSTEM DROP REPLICA only removes the metadata for the specified replica. Other replicas and their data remain untouched.
Q: What if I want to reuse the same replica name after rebuilding a node?
A: You need to first clean up the old ZooKeeper entries using SYSTEM DROP REPLICA, then create the table on the new node with the same name.
Q: Does this error occur with ClickHouse Keeper as well?
A: Yes. ClickHouse Keeper is a drop-in replacement for ZooKeeper, and replica registration works the same way.