ClickHouse Logging Configuration: Levels, Rotation, and System Tables

Q: How do I change the log level without restarting ClickHouse?

Edit the XML and run SYSTEM RELOAD CONFIG . For a single query, use SET send_logs_level='debug' .

Q: Where are the log files by default?

/var/log/clickhouse-server/clickhouse-server.log and clickhouse-server.err.log on Linux package installs.

Q: Can I send ClickHouse logs to syslog?

Yes. Configure inside with address , facility , and format sub-elements, or pipe stdout to your collector when running in a container.

ClickHouse has two parallel logging surfaces: the text log file written by the server process and a set of system tables (query_log, part_log, trace_log, text_log, and others) that capture structured events you can query with SQL. They are configured independently. This article covers both, plus the common problems people hit with file rotation and disk space.

The main server log file

Server log output is controlled by the <logger> section in config.xml. Typical defaults look like this:

<logger>
    <level>information</level>
    <log>/var/log/clickhouse-server/clickhouse-server.log</log>
    <errorlog>/var/log/clickhouse-server/clickhouse-server.err.log</errorlog>
    <size>1000M</size>
    <count>10</count>
</logger>

level is one of trace, debug, information, warning, error, fatal, or none. information is the default and the right choice for production. debug is loud; trace is firehose.
log and errorlog are file paths. The error log gets only warning+ messages.
size is the rotation threshold (1000M = 1 GB).
count is how many rotated files to keep. Older files get deleted.

To override only the level, drop a file into config.d/:

<clickhouse>
    <logger>
        <level>debug</level>
    </logger>
</clickhouse>

A live SQL way to change the level without restart:

SYSTEM RELOAD CONFIG;

after editing the XML, or for transient debugging:

SET send_logs_level = 'trace';

which streams server logs of that query back to the client.

Console logging

For container deployments, write logs to stdout instead of disk:

<logger>
    <console>true</console>
</logger>

When <console> is true, ClickHouse writes to stdout/stderr regardless of whether <log> is also defined. Most containerised setups disable file logging entirely:

<logger>
    <console>true</console>
    <log remove="remove"/>
    <errorlog remove="remove"/>
</logger>

JSON-formatted logs

Since ClickHouse 22.8, the log output can be emitted as JSON instead of the default plain-text format:

<logger>
    <formatting>
        <type>json</type>
    </formatting>
</logger>

Each line becomes a JSON object with fields like date_time, thread_name, thread_id, level, query_id, logger_name, message, and source_file. This is much easier to ship into a log pipeline than parsing the text format. See the companion article on transforming ClickHouse logs to ndjson with Vector.dev for a full pipeline.

System tables for structured logs

ClickHouse writes structured events to MergeTree tables under the system database. They all share the same shape: enabled by config block, async flush, configurable retention.

Table	What it captures
`system.query_log`	One row per query (start, finish, exception). Includes duration, memory, rows/bytes read, user, query text.
`system.query_thread_log`	Per-thread breakdown for each query. Off by default in recent versions.
`system.part_log`	Part lifecycle events: create, merge, mutate, remove, download.
`system.trace_log`	Stack-trace samples for CPU profiling, memory profiling, and real-time tracing.
`system.text_log`	Server log messages, queryable via SQL. Enabled by default since 24.8; off in older versions.
`system.metric_log`	Periodic snapshot of `system.metrics` and `system.events`.
`system.asynchronous_metric_log`	Periodic snapshot of `system.asynchronous_metrics`.
`system.crash_log`	One row per server crash with stack trace.
`system.opentelemetry_span_log`	OpenTelemetry trace spans.

Each is configured with a block in config.xml. The shape is the same for all of them:

<query_log>
    <database>system</database>
    <table>query_log</table>
    <partition_by>toYYYYMM(event_date)</partition_by>
    <ttl>event_date + INTERVAL 30 DAY DELETE</ttl>
    <flush_interval_milliseconds>7500</flush_interval_milliseconds>
</query_log>

To disable a system log entirely, set remove="1" on the element:

<query_thread_log remove="1"/>

Enabling text_log

system.text_log is enabled by default since 24.8 and was off in earlier versions because it duplicates the file log. To configure it explicitly or enable it on an older server:

<text_log>
    <database>system</database>
    <table>text_log</table>
    <flush_interval_milliseconds>7500</flush_interval_milliseconds>
    <level>information</level>
</text_log>

Querying it:

SELECT event_time, level, query_id, message
FROM system.text_log
WHERE event_time > now() - INTERVAL 10 MINUTE
  AND level <= 'Warning'
ORDER BY event_time DESC
LIMIT 50;

OpenTelemetry

ClickHouse supports OpenTelemetry trace context propagation. Spans are stored in system.opentelemetry_span_log and can be exported to any OTLP-compatible backend through the standard OpenTelemetry collector. Enable the span log:

<opentelemetry_span_log>
    <database>system</database>
    <table>opentelemetry_span_log</table>
    <flush_interval_milliseconds>7500</flush_interval_milliseconds>
</opentelemetry_span_log>

Clients then need to pass W3C trace context headers (traceparent) on the HTTP interface for spans to be linked into existing traces.

Disk and rotation problems

The most common log-related failure is the server complaining about missing rotated files:

File not found: /var/log/clickhouse-server/clickhouse-server.log.0
File not found: /var/log/clickhouse-server/clickhouse-server.log.8.gz

This is almost always one of three things:

Permissions on /var/log/clickhouse-server/ changed and clickhouse cannot read its own rotated files. Check with ls -la /var/log/clickhouse-server/.

Disk full or inodes exhausted. Check both:

df -Th /var/log/clickhouse-server
df -Thi /var/log/clickhouse-server

External logrotate is fighting with the built-in rotation. ClickHouse's logger renames files itself, so if the OS logrotate config also touches the directory, you get confused state. Either remove the external rule or disable ClickHouse's internal rotation by setting size very large.

Common Pitfalls

Running with level=trace in production. Disk fills in hours, not days.
Forgetting that query_log is disabled with log_queries = 0. Setting it server-wide silently kills the most useful diagnostic table.
Using JSON formatting and an external regex parser that expects the old format. Pick one and stick to it.
Putting /var/log/clickhouse-server/ on the same filesystem as /var/lib/clickhouse/. If logs fill the disk, the database stops accepting writes.

Frequently Asked Questions

Q: How do I change the log level without restarting ClickHouse? A: Edit the XML and run SYSTEM RELOAD CONFIG. For a single query, use SET send_logs_level='debug'.

Q: Where are the log files by default? A: /var/log/clickhouse-server/clickhouse-server.log and clickhouse-server.err.log on Linux package installs.

Q: How long are query_log rows kept? A: Whatever TTL you configure on the system table. The default in modern versions is 30 days. Set <ttl>event_date + INTERVAL N DAY DELETE</ttl> in config.xml to change it.

Q: Why is query_thread_log so big? A: It writes one row per thread per query. For a heavy server it can be the largest system table. Disable it unless you need per-thread profiling.

Q: Can I send ClickHouse logs to syslog? A: Yes. Configure <syslog> inside <logger> with address, facility, and format sub-elements, or pipe stdout to your collector when running in a container.