Install Standalone ZooKeeper on Ubuntu/Debian

ClickHouse uses ZooKeeper (or its drop-in replacement, ClickHouse Keeper) to coordinate ReplicatedMergeTree tables, distributed DDL, and cross-replica locks. This guide walks through installing a standalone, single-node ZooKeeper on Ubuntu or Debian: choosing a package, creating the user and directory layout, writing a zoo.cfg, and wiring up a systemd service.

A standalone install is correct for evaluation, development, and CI. It is not suitable for production — a single ZooKeeper node is a single point of failure, and any replicated ClickHouse cluster that depends on it stops accepting inserts the moment that node goes down. For production sizing, ensemble layout, and tuning, see the ClickHouse ZooKeeper Configuration Guide. And before you commit to ZooKeeper at all, read What Is ClickHouse Keeper — for new deployments, Keeper is usually the better starting point.

Should You Install ZooKeeper or Keeper?

ClickHouse Keeper has been GA since version 22.3 and is the recommended coordination service for new clusters. It speaks the ZooKeeper wire protocol, so ClickHouse talks to it exactly as it would to ZooKeeper, but it has no JVM, no GC pauses, and a smaller memory footprint.

Install ZooKeeper when you need parity with an existing ZooKeeper-based environment or are following a runbook that assumes it. Otherwise, prefer Keeper. The rest of this guide covers ZooKeeper specifically.

Choosing a Package

There are two practical ways to get ZooKeeper onto Ubuntu/Debian:

1. Distribution package (apt install zookeeper / zookeeperd). Simple, but distro packages are often several minor versions behind and lay files out differently from upstream (/usr/share/zookeeper, /etc/zookeeper/conf). Acceptable for quick tests.
2. Apache binary tarball (recommended). Gives you a known, current version and a predictable layout. ClickHouse works well with ZooKeeper 3.5+; 3.6.x or later is a sensible choice for a fresh install. This guide uses the tarball approach because it matches the directory structure assumed by most ClickHouse runbooks.

Either way, ZooKeeper is a JVM application and needs a Java runtime.

Step 1: Install Java

sudo apt-get update
sudo apt-get install -y default-jre netcat

default-jre pulls a supported OpenJDK runtime. netcat is used later to send ZooKeeper's four-letter health-check commands. Verify Java:

java -version

Step 2: Create the User and Directories

Run ZooKeeper under a dedicated, non-login system user — never as root. Keep the data directory and the transaction-log directory separate so you can later place the log on its own fast disk.

# Dedicated system user and group
sudo groupadd -r zookeeper
sudo useradd -r -g zookeeper --home-dir=/var/lib/zookeeper --shell=/bin/false zookeeper

# Data, transaction log, config, and server log directories
sudo mkdir -p /var/lib/zookeeper/data \
             /var/lib/zookeeper/logs \
             /etc/zookeeper \
             /var/log/zookeeper \
             /opt

The split here is deliberate:

• /var/lib/zookeeper/data — snapshots (dataDir).
• /var/lib/zookeeper/logs — the transaction log (dataLogDir). On a real deployment this belongs on a dedicated low-latency SSD, because fsync latency on this directory is the single biggest performance factor for ClickHouse coordination.
• /etc/zookeeper — configuration.
• /var/log/zookeeper — the ZooKeeper process's own application log.

Step 3: Download and Extract ZooKeeper

export ZOOKEEPER_VERSION=3.6.3
wget "https://dlcdn.apache.org/zookeeper/zookeeper-${ZOOKEEPER_VERSION}/apache-zookeeper-${ZOOKEEPER_VERSION}-bin.tar.gz" \
     -O /tmp/apache-zookeeper-${ZOOKEEPER_VERSION}-bin.tar.gz

sudo tar -xvf /tmp/apache-zookeeper-${ZOOKEEPER_VERSION}-bin.tar.gz -C /opt
sudo ln -s /opt/apache-zookeeper-${ZOOKEEPER_VERSION}-bin /opt/zookeeper

The symlink /opt/zookeeper lets you upgrade later by re-pointing it at a new extracted version without rewriting your systemd unit. Set ownership on everything ZooKeeper touches:

sudo chown -R zookeeper:zookeeper \
    /var/lib/zookeeper \
    /var/log/zookeeper \
    /etc/zookeeper \
    /opt/apache-zookeeper-${ZOOKEEPER_VERSION}-bin

Always pick the current version string from the Apache ZooKeeper download page and verify the checksum/signature; mirrors retire older releases.

Step 4: Write zoo.cfg

Create /etc/zookeeper/zoo.cfg. These values are tuned for ClickHouse coordination rather than ZooKeeper's generic defaults:

tickTime=2000
initLimit=20
syncLimit=10
maxSessionTimeout=60000000
maxClientCnxns=2000
preAllocSize=131072
snapCount=3000000
dataDir=/var/lib/zookeeper/data
dataLogDir=/var/lib/zookeeper/logs
clientPort=2181
autopurge.snapRetainCount=10
autopurge.purgeInterval=1
4lw.commands.whitelist=*

What matters here:

• dataDir vs dataLogDir — keeping snapshots and the transaction log on separate directories (ideally separate disks) is the most important operational choice.
• autopurge.purgeInterval=1 — ZooKeeper does not clean up old snapshots and logs by default. Without autopurge, the data disk fills and the node dies. This enables hourly cleanup, retaining the last 10 snapshots.
• maxClientCnxns=2000 — the upstream default (60) is far too low for a busy ClickHouse cluster, which opens many connections.
• 4lw.commands.whitelist=* — enables the four-letter monitoring commands (ruok, mntr, stat). Restrict this to specific commands in security-sensitive environments.

For what each of these parameters does in depth, plus production sizing, see the ClickHouse ZooKeeper Configuration Guide.

The myid file (single node)

A standalone server still benefits from a node ID. Create it for completeness so the same layout extends cleanly to an ensemble later:

echo "1" | sudo tee /var/lib/zookeeper/data/myid
sudo chown zookeeper:zookeeper /var/lib/zookeeper/data/myid

myid lives in dataDir and contains a single integer (1–255), unique per server. In an ensemble it must match the server.N= line in zoo.cfg.

Step 5: Create the systemd Service

Create /etc/systemd/system/zookeeper.service:

[Unit]
Description=ZooKeeper Daemon
Documentation=http://zookeeper.apache.org
Requires=network.target
After=network.target

[Service]
Type=forking
WorkingDirectory=/var/lib/zookeeper
User=zookeeper
Group=zookeeper
Environment=ZK_SERVER_HEAP=1536
Environment=SERVER_JVMFLAGS="-Xms1536m -XX:+AlwaysPreTouch -Djute.maxbuffer=8388608 -XX:MaxGCPauseMillis=50"
Environment=ZOO_LOG_DIR=/var/log/zookeeper
ExecStart=/opt/zookeeper/bin/zkServer.sh start /etc/zookeeper/zoo.cfg
ExecStop=/opt/zookeeper/bin/zkServer.sh stop /etc/zookeeper/zoo.cfg
ExecReload=/opt/zookeeper/bin/zkServer.sh restart /etc/zookeeper/zoo.cfg
TimeoutSec=30
Restart=on-failure

[Install]
WantedBy=default.target

Notes on the JVM settings:

• ZK_SERVER_HEAP=1536 sets a 1.5 GB max heap (megabytes). Size this to roughly 80–90% of RAM on a dedicated box, but heaps above ~8 GB rarely help — the OS page cache holding the snapshot matters more than a large heap. On a small test VM, 1536 is fine.
• -Djute.maxbuffer=8388608 (8 MB) raises the maximum znode/packet size. This value must match the jute.maxbuffer on every ClickHouse server, or large mutations trigger dropped sessions. See ClickHouse ZooKeeper Session Expired.
• Equal -Xms/heap and AlwaysPreTouch avoid heap-resize stalls that can cause session timeouts.

Enable and start it:

sudo systemctl daemon-reload
sudo systemctl enable --now zookeeper.service
sudo systemctl status zookeeper.service

Step 6: Verify ZooKeeper Is Healthy

ZooKeeper listens on clientPort 2181. Use the four-letter commands:

# Liveness — expect "imok"
echo ruok | nc localhost 2181

# Server status and mode (should report "standalone")
echo stat | nc localhost 2181

# Detailed metrics
echo mntr | nc localhost 2181

If ruok returns imok and stat reports Mode: standalone, the server is up. If nc returns nothing, confirm 4lw.commands.whitelist=* is set and the service is running. Check the application log at /var/log/zookeeper/ for stack traces.

Step 7: Point ClickHouse at ZooKeeper

Add a config drop-in so ClickHouse can find the server. Create /etc/clickhouse-server/config.d/zookeeper.xml:

<clickhouse>
    <zookeeper>
        <node>
            <host>127.0.0.1</host>
            <port>2181</port>
        </node>
        <session_timeout_ms>30000</session_timeout_ms>
        <operation_timeout_ms>30000</operation_timeout_ms>
        <root>/clickhouse</root>
    </zookeeper>
</clickhouse>

Restart clickhouse-server, then confirm the connection from inside ClickHouse:

SELECT name, value FROM system.zookeeper WHERE path = '/';

A result set (rather than an error) means ClickHouse reached ZooKeeper. You can now create ReplicatedMergeTree tables. For the full coordination story, see What Is ClickHouse ReplicatedMergeTree and the ClickHouse Replication guide.

Best Practices

1. Do not co-locate ZooKeeper with clickhouse-server. A heavy merge starves the JVM and triggers session expirations. Use a separate host (or at minimum cgroup-pinned CPU/IO) even in test setups that you intend to grow.
2. Never run ZooKeeper for production on a single node. Standalone has no fault tolerance. Move to a 3-node ensemble before going live — see the configuration guide.
3. Keep dataLogDir on a dedicated low-latency disk. Transaction-log fsync latency directly caps ClickHouse insert throughput.
4. Enable autopurge. Without it, snapshots and logs accumulate until the disk fills.
5. Disable swap. A swapping JVM stalls during GC and never recovers within the session timeout.
6. Match jute.maxbuffer everywhere. A mismatch between ZooKeeper and ClickHouse is a classic cause of dropped sessions.
7. Don't expose 2181 publicly. ZooKeeper has no meaningful authentication by default; firewall it to the ClickHouse subnet.

Common Issues

• nc to port 2181 returns nothing. The four-letter commands are blocked. Confirm 4lw.commands.whitelist=* (or the specific command) in zoo.cfg and restart.
• Service fails with "JAVA_HOME is not set" or no Java found. default-jre isn't installed or zkServer.sh can't locate it; install the JRE and confirm java -version.
• stat reports the wrong mode or the node won't form a quorum. For a single node you want Mode: standalone. If you later add servers, every node needs a unique myid matching its server.N= line.
• ClickHouse logs Session expired or Coordination::Exception. Usually GC pauses, an undersized jute.maxbuffer, or co-location contention. See ClickHouse ZooKeeper Session Expired and ZooKeeper/Keeper Coordination Bottlenecks.
• Disk slowly fills up. Autopurge is disabled or purgeInterval=0. Set autopurge.purgeInterval=1.

How Pulse Helps

Coordination problems rarely announce themselves cleanly — they show up as stalled merges, replication lag, or intermittent Session expired errors long after the ZooKeeper node started degrading. Pulse continuously monitors ClickHouse alongside its ZooKeeper or Keeper ensemble, surfacing rising request latency, growing outstanding requests, session churn, and quorum changes before they cascade into ingest failures. When something does break, Pulse ties the ClickHouse-side symptom back to the coordination-layer cause, so you spend less time guessing whether the problem is in the table or the ensemble underneath it.

Frequently Asked Questions

Q: Can I use this standalone install in production? No. A single ZooKeeper node has no fault tolerance — if it fails, every ReplicatedMergeTree table stops accepting writes. Use it for development, testing, and CI only, and deploy a 3-node ensemble (or ClickHouse Keeper) for production.

Q: Should I install ZooKeeper or ClickHouse Keeper? For new clusters, prefer Keeper: it ships with ClickHouse, needs no JVM, and avoids GC-related session expirations. Install ZooKeeper when you need parity with an existing ZooKeeper estate. See What Is ClickHouse Keeper.

Q: Which ZooKeeper version should I use with ClickHouse? ClickHouse works with ZooKeeper 3.5+; 3.6.x or newer is a good choice for a fresh install. Always download the current release from the official Apache page and verify its signature rather than relying on a potentially stale distro package.

Q: Why split dataDir and dataLogDir? The transaction log (dataLogDir) is written synchronously on every coordination operation, so its fsync latency directly limits ClickHouse. Keeping it on a separate, idle, low-latency disk from the snapshots (dataDir) is the single most impactful storage decision.

Q: How do I confirm ClickHouse is actually using ZooKeeper? Run SELECT * FROM system.zookeeper WHERE path = '/' in ClickHouse. A successful result means the connection works. You can also query system.zookeeper_log (21.11+) for OperationTimeout or ConnectionLoss events.

Q: Do I need the myid file on a single node? ZooKeeper can run standalone without an ensemble, but creating myid (containing 1) keeps your layout consistent so you can later add server.N= entries and grow into an ensemble without restructuring directories.