The jump from Elasticsearch 7.x to 8.x is the most disruptive major version upgrade since the 2.x to 5.x transition. Security is now on by default, mapping types are gone, the transport client is removed, and Java 17 is the minimum runtime. Most failed upgrades trace back to skipping the pre-upgrade deprecation checks or not understanding what "security enabled by default" actually means in practice.
You must be on 7.17.x before upgrading to any 8.x release. There is no supported path from 7.15 or 7.16 directly to 8.x. Run the Upgrade Assistant in Kibana 7.17 and resolve every critical issue it reports. Do not skip this step - indices created with deprecated settings in 7.x can prevent 8.x nodes from starting, and at that point you cannot downgrade because the data directory layout has already changed.
Security Enabled by Default
This is the change that catches the most teams off guard. Elasticsearch 8.0 enables security automatically on fresh installs: TLS on both the transport and HTTP layers, a generated password for the elastic superuser, and enrollment tokens for adding new nodes. For upgrades from 7.x, the behavior depends on whether you already had security configured.
If your 7.x cluster ran with xpack.security.enabled: false (common in development and some production setups), the upgrade to 8.x will force security on. Your existing clients, Logstash pipelines, and Beats agents will fail to connect until you either configure credentials and TLS or explicitly disable security in elasticsearch.yml. The setting still works - xpack.security.enabled: false - but you need to set it before starting the 8.x node.
Clusters that already had security enabled face a different problem. The .security-7 internal index gets reindexed to .security-7-reindexed-for-8 during the upgrade. If your cluster was originally created on 5.x and still has the security-index-template-v6 template, this reindexing will fail with a mapping error about allow_restricted_indices. Delete the legacy template before upgrading.
Removed Mapping Types and API Changes
Elasticsearch 8 completes the removal of mapping types that was deprecated in 7.x. The _type field no longer exists in documents, and APIs that accepted type parameters now reject them. If your application still passes _doc as a type in index or search requests, most 8.x versions tolerate it for backward compatibility, but any custom type names will fail.
The practical impact depends on your client library version. The Java Transport Client is gone entirely - there is no 8.x version. You must migrate to the Java API Client, which uses a REST-based transport layer rather than the binary transport protocol. The High Level REST Client still works in early 8.x releases but is deprecated; plan to migrate to the new Java API Client.
// 7.x style (with type) - fails in 8.x
PUT /my-index/_doc/1?type=my_type
{ "field": "value" }
// 8.x style (no type)
PUT /my-index/_doc/1
{ "field": "value" }
Check your index templates too. Templates that include _default_ mappings or explicit type definitions need updating. The include_type_name parameter, which existed as a transition aid in 7.x, is removed in 8.x.
The Deprecation API and Pre-Upgrade Checks
The _migration/deprecations API is the single most useful tool for planning your upgrade. Call it on your 7.17.x cluster and it returns every deprecated setting, mapping, and configuration grouped into critical and warning levels.
GET /_migration/deprecations
The response breaks down into cluster_settings, node_settings, index_settings, and ml_settings. Critical items block the upgrade - the 8.x node will not start until you fix them. Warnings indicate deprecated functionality that still works in 8.x but will be removed in a future release.
Common critical deprecations include: camelCase date format names like dateOptionalTime (must become date_optional_time), translog retention settings (index.translog.retention.size and index.translog.retention.age are no longer used), and legacy template formats. Run this API repeatedly as you fix issues until the critical list is empty.
Rolling Upgrade Procedure
A rolling upgrade lets you move from 7.17.x to 8.x with zero downtime. The order matters: upgrade non-master-eligible nodes first, then data nodes tier by tier (frozen, cold, warm, hot), then master-eligible nodes last.
For each node:
# 1. Disable shard allocation
PUT _cluster/settings
{ "persistent": { "cluster.routing.allocation.enable": "primaries" } }
# 2. Stop indexing and perform a synced flush (optional, speeds recovery)
POST _flush/synced
# 3. Stop the node, install 8.x, start the node
# 4. Re-enable allocation
PUT _cluster/settings
{ "persistent": { "cluster.routing.allocation.enable": null } }
# 5. Wait for green status before moving to the next node
GET _cluster/health?wait_for_status=green&timeout=5m
Do not set cluster.initial_master_nodes in 8.x config during a rolling upgrade - the cluster is already bootstrapped. Set discovery.seed_hosts to point to the existing master nodes. Java 17 is required; the bundled JDK in 8.x includes it, but if you use a custom JDK path, verify the version.
Common Post-Upgrade Issues
Index compatibility is the first thing to verify. Elasticsearch 8.x can read indices created in 7.x, but not indices created in 6.x. If your cluster contains indices originally created before 7.0 that were never reindexed, the 8.x node will refuse to start. Check creation versions with GET /my-index/_settings?filter_path=**.version.created before upgrading.
Watch for null values in alias is_write_index parameters. Elasticsearch 8 rejects null where 7.x silently accepted it - set the field to true or false explicitly. ILM policies that reference removed settings (like translog retention) will also fail to execute.
Client connection failures are the most visible post-upgrade issue. If security was newly enabled, every integration needs updated connection strings with credentials and TLS configuration. Check your Beats, Logstash, and application clients. For Logstash, the Elasticsearch output plugin needs ssl => true, the CA certificate, and valid credentials. Beats need the same under output.elasticsearch. Test each integration before declaring the upgrade complete.
