ClickHouse DB::Exception: Cap'n Proto bad type

The CAPN_PROTO_BAD_TYPE error indicates that a Cap'n Proto schema contains a field type that ClickHouse does not know how to handle. While ClickHouse supports many common Cap'n Proto types, some advanced or less common types are not implemented in the ClickHouse Cap'n Proto reader/writer. When such a type is encountered, the operation fails with this error.

Impact

The entire query using the CapnProto format will fail. This affects both reading (INSERT from Cap'n Proto data) and writing (SELECT into Cap'n Proto format). If the unsupported type is in a field that your query does not actually need, you still cannot proceed without modifying the schema, because ClickHouse parses the full schema definition before processing data.

Common Causes

Using Cap'n Proto AnyPointer type -- This generic pointer type is not supported by ClickHouse.
Complex union types -- Certain unnamed or deeply nested union constructs may not be parsable.
Interface types -- Cap'n Proto interfaces (for RPC) are not relevant to data serialization and are not supported.
Generic types or type parameters -- Parameterized struct types in Cap'n Proto are not handled.
Outdated ClickHouse version -- Older ClickHouse versions support fewer Cap'n Proto types. A type that works in a newer release may fail in an older one.

Troubleshooting and Resolution Steps

Identify the unsupported type: The error message will indicate which field or type ClickHouse cannot process. Note the type name from the exception text.
Review your .capnp schema:
```
cat /path/to/schema.capnp
```
Look for types that fall outside the commonly supported set: Void, Bool, Int8-Int64, UInt8-UInt64, Float32, Float64, Text, Data, List, Enum, and simple struct types.
Simplify the schema for ClickHouse: Create a separate .capnp schema specifically for ClickHouse that omits unsupported types. Replace the unsupported field with a compatible alternative:
```
struct MyRecord {
    id @0 :UInt64;
    name @1 :Text;
    # Removed unsupported AnyPointer field
    payload @2 :Data;  # Use Data (bytes) instead
}
```
Upgrade ClickHouse: Check whether a newer ClickHouse version adds support for the type you need:
```
clickhouse-server --version
```
Consult the ClickHouse changelog for CapnProto-related improvements.
Use an alternative serialization format: If the Cap'n Proto schema cannot be simplified, consider using Protobuf, Avro, or JSON as an intermediate format that supports your data model more naturally.
Validate the schema compiles cleanly:
```
capnp compile schema.capnp
```
A schema that does not compile will also cause issues in ClickHouse.

Best Practices

Keep the Cap'n Proto schemas used with ClickHouse as simple as possible -- stick to primitive types, structs, lists, and enums.
Maintain a separate ClickHouse-specific .capnp file if your canonical schema uses advanced features not supported by ClickHouse.
Test schema compatibility with a small query before wiring up production pipelines.
Monitor ClickHouse release notes for expanded Cap'n Proto type support when planning upgrades.

Frequently Asked Questions

Q: Which Cap'n Proto types does ClickHouse support?
A: ClickHouse supports Void, Bool, Int8 through Int64, UInt8 through UInt64, Float32, Float64, Text, Data, List, Enum, and struct types. Nested structs are also supported. Types like AnyPointer and interfaces are not.

Q: Can I skip unsupported fields in the schema?
A: ClickHouse processes the entire schema definition, but you only need to map fields that exist as columns in your table. However, if the schema itself contains an unsupported type definition, the parser may reject it even if you do not select that field. The safest approach is to remove the unsupported field from the schema file used with ClickHouse.

Q: Is there a way to convert Cap'n Proto data to a supported format before loading?
A: Yes. You can write a small program using the Cap'n Proto library in your preferred language to read the data and output it as JSON, CSV, or Protobuf, which ClickHouse can then consume directly.

Q: Does this error occur at query time or at schema load time?
A: It occurs at query time when ClickHouse parses the .capnp schema file referenced by the format_schema setting. The schema is parsed fresh for each query unless internal caching applies.