ClickHouse DB::Exception: Cannot create file

The "DB::Exception: Cannot create file" error in ClickHouse is triggered when the server attempts to create a new file and the operating system denies the request. The CANNOT_CREATE_FILE error code covers situations where file creation fails -- whether because of missing directories, permission problems, or exhausted disk resources. ClickHouse creates files frequently during inserts, merges, and temporary data processing, so this error can disrupt a wide range of operations.

Impact

File creation failures affect ClickHouse in the following ways:

• Insert operations fail because new data parts cannot be written
• Merges are blocked since the output part cannot be created
• Temporary files needed for sorting, joining, or aggregation cannot be allocated
• Table creation may fail if the metadata file cannot be written
• The server may become effectively unable to process any write workload

Common Causes

1. The target directory does not exist (perhaps it was never created or was accidentally deleted)
2. Insufficient permissions for the ClickHouse user on the data directory
3. Disk space or inode exhaustion
4. The filesystem is mounted read-only
5. SELinux or AppArmor denying file creation
6. File descriptor limit reached, preventing new file handles
7. Disk quota exceeded for the ClickHouse user

Troubleshooting and Resolution Steps

1. Check the error log for the file path:

   grep "Cannot create file" /var/log/clickhouse-server/clickhouse-server.err.log | tail -5
   

2. Verify the parent directory exists:

   ls -la $(dirname /path/to/file/from/error)
   

   If it doesn't exist, recreate it with proper ownership:

   sudo mkdir -p /path/to/parent/directory
   sudo chown clickhouse:clickhouse /path/to/parent/directory
   

3. Check disk space and inodes:

   df -h /var/lib/clickhouse
   df -i /var/lib/clickhouse
   

   Free up space or expand the volume if needed.

4. Verify permissions:

   sudo -u clickhouse touch /var/lib/clickhouse/test_create && rm /var/lib/clickhouse/test_create
   

   If the touch fails, fix ownership:

   sudo chown -R clickhouse:clickhouse /var/lib/clickhouse
   

5. Check filesystem mount mode:

   mount | grep $(df /var/lib/clickhouse --output=source | tail -1)
   

6. Review file descriptor limits:

   cat /proc/$(pidof clickhouse-server)/limits | grep "open files"
   

   Raise the limit if it is too low.

7. Check security policies:

   sudo ausearch -m avc -ts recent
   sudo journalctl | grep apparmor | tail -10
   

Best Practices

• Ensure all directories in the ClickHouse data path exist and are owned by the clickhouse user before starting the server
• Set file descriptor limits to at least 100,000 for production
• Monitor disk space and inode usage with alerts that fire well before exhaustion
• Use separate storage volumes for ClickHouse data to prevent other applications from consuming disk space
• Validate permissions and directory structure after OS upgrades or storage changes
• Run ClickHouse with appropriate SELinux or AppArmor profiles rather than disabling security entirely

Frequently Asked Questions

Q: Can running out of inodes cause this error even with free disk space?
A: Yes. Each file on the filesystem consumes an inode. If all inodes are allocated (check with df -i), no new files can be created even though disk space is available. This is common on filesystems with many small files.

Q: Should I create ClickHouse data directories manually?
A: ClickHouse normally creates its directory structure on startup. However, if you configure custom storage paths, you must ensure those directories exist and have correct ownership before starting the server.

Q: Could this error be caused by a container misconfiguration?
A: Yes. In containerized deployments, the mounted volume must be writable by the ClickHouse user inside the container. Check that the volume mount permissions match the expected UID/GID for the clickhouse user.

Q: How many files does a typical ClickHouse table create?
A: Each data part contains multiple files (one per column plus index and metadata files). A table with many columns and frequent inserts can accumulate thousands of files. Plan your inode allocation accordingly.