Removing a node from an Elasticsearch cluster may be necessary in various scenarios, such as:
- Decommissioning old or underperforming hardware
- Scaling down the cluster due to reduced workload
- Replacing a faulty node
- Performing maintenance on a specific node
- Rebalancing the cluster for optimal performance
Steps to remove a node from the cluster
Prepare the cluster:
- Ensure the cluster is in a healthy state (green)
- Verify that removing the node won't impact the cluster's availability
Disable shard allocation:
PUT _cluster/settings { "persistent": { "cluster.routing.allocation.enable": "none" } }
Stop indexing and perform a synced flush (optional but recommended):
POST _flush/synced
Shut down the node:
- Stop the Elasticsearch service on the node you want to remove
Remove the node from the cluster configuration:
- Update the
elasticsearch.yml
file on all remaining nodes to remove references to the decommissioned node
- Update the
Re-enable shard allocation:
PUT _cluster/settings { "persistent": { "cluster.routing.allocation.enable": null } }
Monitor cluster health:
- Watch the cluster health and shard reallocation process
GET _cluster/health GET _cat/shards
Update your client applications (if necessary):
- Remove the decommissioned node from client configurations
Best Practices and Additional Information
- Always maintain enough nodes to store at least one replica of each shard
- Perform node removals during off-peak hours to minimize impact on performance
- Ensure proper backups are in place before making any cluster changes
- If removing multiple nodes, do so one at a time to maintain cluster stability
- Consider using the Cluster Update Settings API to exclude the node from shard allocation before shutting it down:
PUT _cluster/settings { "transient" : { "cluster.routing.allocation.exclude._name" : "node_to_remove" } }
Frequently Asked Questions
Q: How long does it take to remove a node from the cluster?
A: The time varies depending on cluster size, data volume, and hardware. It can take minutes to hours. Monitor the cluster health and shard allocation to track progress.
Q: Can I remove a master-eligible node?
A: Yes, but ensure you have enough remaining master-eligible nodes to form a quorum. It's recommended to have at least three master-eligible nodes in a production cluster.
Q: What happens to the data on the removed node?
A: Data from the removed node will be redistributed to the remaining nodes in the cluster. Ensure you have enough storage capacity on other nodes before removal.
Q: Will removing a node affect my cluster's performance?
A: Temporarily, yes. The cluster will need to reallocate shards and may experience increased load. Plan the removal during off-peak hours to minimize impact.
Q: How can I ensure I'm not removing a critical node?
A: Check the cluster health, shard allocation, and node roles before removal. Ensure you have enough nodes to maintain the required number of replicas and master-eligible nodes.