Removing Empty Parts in ClickHouse

An empty part in ClickHouse is an active data part that contains zero rows. They are a normal by-product of TTL deletes, mutations, and collapsing merges that remove every row in a part's range. Modern ClickHouse cleans these up automatically, but the behavior has a sharp edge: the first time a server learns how to remove empty parts — typically right after an upgrade — it tries to remove all of them at once, which on a busy replicated cluster can cause replicas to block each other.

This guide explains why empty parts form, how to find them, how the automatic cleanup works, and how to perform the one-time post-upgrade cleanup safely. It is distinct from detached parts (files under the detached/ directory) — see ClickHouse Detached Parts cleanup for those.

What Is an Empty Part?

A part is "empty" when it is active (visible to queries, counted in system.parts WHERE active) but has rows = 0. This is different from:

• Detached parts — parts moved to the detached/ subdirectory, no longer active. Covered in Detached Parts cleanup.
• Inactive (outdated) parts — parts that were merged away and are awaiting deletion after old_parts_lifetime. These still hold data; they are simply superseded.

An empty part occupies almost no disk space, but it still counts toward the part total for its partition, shows up in metadata, and participates in merges and replication. A partition made up entirely of empty parts is effectively dead weight.

Why Empty Parts Form

Empty parts are created when an operation removes all rows from a part but leaves the part itself in place:

• TTL DELETE — a TTL ... DELETE expression that ages out every row in a part. With merge_with_ttl_timeout controlling how often TTL merges run, parts can be reduced to zero rows.
• Mutations — ALTER TABLE ... DELETE WHERE ... that matches every row in a part. See Mutations performance impact.
• Collapsing / VersionedCollapsing merges — a CollapsingMergeTree merge where all sign rows cancel out.
• Replication and recovery flows — empty parts can be fetched or created when a replica reconstructs a partition's state.

Empty-part removal was introduced in ClickHouse 20.12 (earlier versions left 0-row parts behind after TTL pruning). On any reasonably recent 24.x/25.x release the cleanup is built in and runs continuously.

How Automatic Cleanup Works

The relevant MergeTree setting is remove_empty_parts, which the official documentation describes as: "Remove empty parts after they were pruned by TTL, mutation, or collapsing merge algorithm." It is enabled by default. With it on, ClickHouse replaces a fully-pruned part with an empty part and then drops the empty part as part of normal background maintenance.

Once an empty part becomes inactive, it follows the same retention path as any outdated part: it is physically deleted by a background task after old_parts_lifetime seconds. The documented default is 480 seconds (8 minutes), chosen to protect against data loss during spontaneous reboots. So even after cleanup kicks in, expect a short delay before disk and metadata fully reflect the removal.

On steady-state clusters you rarely need to do anything: TTL prunes rows, remove_empty_parts removes the resulting empties, and old_parts_lifetime governs final deletion. The manual work is almost always tied to the first run after an upgrade.

Identifying Empty Parts

Find partitions where every active part is empty — these are the safe, high-value cleanup targets:

SELECT
    database,
    table,
    partition_id,
    count() AS parts
FROM system.parts
WHERE active
GROUP BY database, table, partition_id
HAVING count() = countIf(rows = 0)
ORDER BY parts DESC;

To generate ready-to-run DROP PARTITION statements for those fully-empty partitions:

SELECT concat(
    'ALTER TABLE ', database, '.', table,
    ' DROP PARTITION ID ''', partition_id, ''';')
FROM system.parts
WHERE active
GROUP BY database, table, partition_id
HAVING count() = countIf(rows = 0);

To inspect individual empty parts (for example, to see whether they are isolated or cluster in a few partitions):

SELECT database, table, partition_id, name, rows, active
FROM system.parts
WHERE active AND rows = 0
ORDER BY database, table, partition_id;

The Post-Upgrade Cleanup and the Replica Deadlock

When you upgrade from a version that left empty parts behind to one that removes them, the server notices the accumulated empties and tries to remove all of them at once. This is a one-time burst that runs right after the upgrade; once it completes, TTL and remove_empty_parts handle empties incrementally on their own.

On a ReplicatedMergeTree cluster this burst is risky. If multiple replicas of the same table start removing the same empty parts simultaneously, they can block each other (a known coordination bug). The cleanup operations contend in ClickHouse Keeper / ZooKeeper, the replication queue stalls, and the upgrade appears to hang. To avoid this, do the cleanup deliberately rather than letting every replica race.

Safe Upgrade Procedure

1. Pre-upgrade: drop fully-empty partitions. Run the DROP PARTITION generator query above before upgrading to shrink the number of empty parts the new version has to clean up. Fewer empties means a shorter, lower-contention cleanup burst.

2. Upgrade one replica per shard at a time. Restart a single replica, then wait for its replication queue to drain before touching the next one. If only one replica is cleaning empty parts at a time, there is no deadlock from replicas waiting on each other.

3. Watch the replication queue between restarts:

   SELECT database, table, count() AS tasks
   FROM system.replication_queue
   GROUP BY database, table
   ORDER BY tasks DESC;
   

   Wait for this to return to its baseline before restarting the next replica.

Disabling Cleanup During a Risky Upgrade

If you want full control over timing, disable automatic empty-part removal before the upgrade by adding remove_empty_parts=0 to the default profile, then re-enable it afterward and clean up manually:

<!-- /etc/clickhouse-server/users.d/remove_empty_parts.xml -->
<clickhouse>
    <profiles>
        <default>
            <remove_empty_parts>0</remove_empty_parts>
        </default>
    </profiles>
</clickhouse>

With removal disabled, no replica will start the cleanup burst on restart. Once the cluster is fully upgraded and healthy, remove this override (or set it back to 1) and drop the empty partitions with the generator query at a controlled pace.

Empty Parts Stuck in the Replication Queue

A related failure mode is an empty (or now-nonexistent) partition referenced by a GET_PART task that retries forever in system.replication_queue. The part can never be fetched because it holds no data anywhere, so the queue keeps growing. This overlaps with empty-part cleanup but is handled with a queue-aware query that cross-references aged GET_PART entries against partitions with zero active parts.

That exact recipe — generating DROP PARTITION statements for empty partitions stuck in the queue across all replicas — lives in Remove Empty Partitions from the Replication Queue. For broader queue troubleshooting, see the Replication Queue guide.

Best Practices

1. Leave remove_empty_parts enabled in steady state. The default is correct for normal operation. Only override it as a deliberate, temporary step around a risky upgrade.

2. Drop empty partitions before upgrading. Pre-cleaning shrinks the one-time post-upgrade burst and reduces the chance of replica contention.

3. Roll upgrades one replica per shard. Wait for the replication queue to drain between restarts so only one node cleans empties at a time.

4. Don't fight old_parts_lifetime. After cleanup, parts are physically deleted only after the retention window (default 480s). A short lag is expected, not a bug. Lowering it server-wide trades reboot-safety for faster deletion.

5. Target whole-empty partitions, not scattered rows. DROP PARTITION is cheap and safe when every active part in the partition has rows = 0. Verify with the HAVING count() = countIf(rows = 0) check before dropping.

Common Issues

• Upgrade appears to hang on a replicated cluster. Multiple replicas are racing to remove the same empty parts and blocking each other. Restart/upgrade replicas one at a time, or disable remove_empty_parts during the upgrade window.

• Empty parts persist after dropping rows. TTL merges run on a cadence governed by merge_with_ttl_timeout; the empty part may not have been produced yet, or old_parts_lifetime hasn't elapsed since it became inactive. Re-check after the retention window.

• Confusing empty parts with detached parts. They are unrelated. rows = 0 AND active is an empty part; anything under detached/ is a detached part — see Detached Parts cleanup.

How Pulse Helps

Pulse continuously inspects system.parts, system.replication_queue, and merge activity across every node in a ClickHouse cluster, so empty-part accumulation and post-upgrade cleanup bursts surface as concrete findings rather than a stalled upgrade you discover by accident. It flags partitions made entirely of empty parts, watches for the replica-contention pattern that blocks the replication queue during an upgrade, and tracks whether old_parts_lifetime retention is keeping pace with deletions. If you run ClickHouse and want this kind of monitoring without building it yourself, see Pulse.

Frequently Asked Questions

Q: What is an empty part in ClickHouse? A: An active data part with zero rows (active = 1 AND rows = 0 in system.parts). It is created when TTL deletes, mutations, or collapsing merges remove every row from a part's range but leave the part itself in place.

Q: How do I remove empty parts? A: In normal operation you don't have to — remove_empty_parts (on by default) drops them automatically, and old_parts_lifetime (default 480s) governs final deletion. To clean them up manually, drop the partitions whose parts are all empty using the HAVING count() = countIf(rows = 0) query in this guide.

Q: Why did empty parts suddenly appear after an upgrade? A: Versions before ClickHouse 20.12 left 0-row parts behind. After upgrading to a version that removes them, the server tries to clean up all accumulated empties at once. On replicated clusters this burst can cause replicas to block each other, so upgrade one replica at a time.

Q: Will removing empty parts lose data? A: No, by definition empty parts contain no rows. The risk is operational, not data loss: dropping the wrong partition, or letting all replicas race the cleanup at once. Always verify a partition is fully empty before dropping it.

Q: How is this different from cleaning up detached parts? A: Detached parts live in the detached/ directory and are no longer active; they may still hold recoverable data and need allow_drop_detached to remove. Empty parts are active, hold no data, and are removed via normal DROP PARTITION or automatic cleanup. See Detached Parts cleanup.

Q: Can I stop ClickHouse from removing empty parts? A: Yes, set remove_empty_parts=0 in the default profile (a users.d/ XML override). This is mainly useful as a temporary measure to control timing around a risky upgrade; re-enable it for steady-state operation.