ClickHouse DB::Exception: Cannot create timer

The "DB::Exception: Cannot create timer" error in ClickHouse indicates that the timer_create() system call failed. ClickHouse uses POSIX timers for internal scheduling, profiling, and timeout management. The CANNOT_CREATE_TIMER error means the operating system could not allocate a new timer, typically due to resource limits being reached.

Impact

This error can affect ClickHouse's ability to enforce query timeouts, perform internal profiling, or schedule periodic tasks. Depending on which component requested the timer, the impact can range from a single query failure to degraded server functionality. If timers are unavailable for timeout enforcement, queries may run longer than expected without being cancelled.

Common Causes

1. The per-process timer limit has been reached (POSIX defines a minimum of 32 timers per process, Linux typically allows more)
2. Insufficient system resources (memory or kernel objects) to create new timers
3. A resource leak causing timers to accumulate without being released
4. Container or cgroup restrictions limiting kernel resource allocation
5. The kernel's signal queue is full, preventing timer signal delivery setup

Troubleshooting and Resolution Steps

1. Check the error log for context:

   grep -i "CANNOT_CREATE_TIMER\|Cannot create timer" /var/log/clickhouse-server/clickhouse-server.err.log | tail -10
   

2. Check the system signal queue limits:

   cat /proc/$(pidof clickhouse-server)/limits | grep "pending signals"
   

   POSIX timers use signals for notification, so signal queue limits matter.

3. Check system resource usage:

   cat /proc/$(pidof clickhouse-server)/status | grep -E "Threads|SigQ"
   

   The SigQ line shows current/max pending signals.

4. Increase signal queue limits if needed:

   sudo systemctl edit clickhouse-server
   # Add:
   # [Service]
   # LimitSIGPENDING=65536
   sudo systemctl restart clickhouse-server
   

5. Check overall system health:

   free -h
   cat /proc/sys/kernel/threads-max
   

6. Review ClickHouse settings that affect timer usage:

   SELECT name, value FROM system.settings
   WHERE name IN ('max_execution_time', 'os_thread_priority');
   

7. Restart ClickHouse to release any leaked timers:

   sudo systemctl restart clickhouse-server
   

Best Practices

• Ensure the pending signals limit is set high enough for the ClickHouse process, especially on busy servers with many concurrent queries.
• Monitor the ClickHouse error log for CANNOT_CREATE_TIMER warnings, as they may indicate a gradual resource leak.
• In containerized deployments, verify that kernel resource limits are not overly restrictive.
• Keep ClickHouse updated, as timer management improvements are included in newer releases.

Frequently Asked Questions

Q: What are POSIX timers used for in ClickHouse?
A: ClickHouse uses POSIX timers for enforcing query execution timeouts, internal profiling (such as the query profiler), and scheduling periodic maintenance tasks. They enable the server to interrupt long-running operations when time limits are exceeded.

Q: Can I disable the feature that needs timers to avoid this error?
A: You can disable the query profiler by setting query_profiler_real_time_period_ns and query_profiler_cpu_time_period_ns to 0, which reduces timer usage. However, timeouts and other core functionality still require timers.

Q: Is this error more common in containers?
A: It can be. Some container runtimes apply restrictive limits on kernel resources including signal queues and timers. If you see this error in containerized ClickHouse, check the container's resource configuration.

Q: Will this error cause data loss?
A: No. CANNOT_CREATE_TIMER does not affect data integrity. It may cause individual queries to fail or timeouts to not be enforced, but stored data remains safe.