---
title: Recover a crashed node
description: 'Safely recover an InfluxDB 3 Enterprise node that crashed or was stopped without a graceful stop node: restart it with the same node ID, let it drain its write-ahead log (WAL), stop it gracefully, and then remove it.'
url: https://docs.influxdata.com/influxdb3/enterprise/admin/recover-node/
estimated_tokens: 1070
product: InfluxDB 3 Enterprise
version: enterprise
publisher: InfluxData
canonical: https://docs.influxdata.com/influxdb3/enterprise/admin/recover-node/
date: '2026-07-29T11:41:56-05:00'
lastmod: '2026-07-29T11:41:56-05:00'
---

Use this procedure to recover a node that stopped **ungracefully** — for
example, the process crashed, was killed (`kill -9`), or its container was
force-stopped.
It also applies to a node wedged in the `stopping` state.

> [!Note]
> This procedure applies to both the Parquet engine and the upgraded storage
> engine (the default for new clusters in InfluxDB 3 Enterprise 3.11+).

## Why ungraceful stops put writes at risk

A node stores recently acknowledged writes in its write-ahead log (WAL) until it
captures them in a snapshot.
These buffered writes are the node’s [*WAL tail*](/influxdb3/enterprise/reference/internals/durability/#wal-tail).
Run the [`influxdb3 stop node`](/influxdb3/enterprise/reference/cli/influxdb3/stop/node/)command against a running node to save the WAL tail before the node reports a`stopped` state.

A node that dies without a graceful stop skips that drain:

* Running `stop node` against the dead process cannot drain anything.
* [Removing the node](/influxdb3/enterprise/reference/cli/influxdb3/remove/node/)purges its object-store file paths, permanently deleting any acknowledged
  writes not covered by its last snapshot.
* Sending the process a bare `SIGTERM` (without calling `stop node`) does not
  force a WAL snapshot on the upgraded storage engine, so a plain process
  shutdown can also leave a WAL tail behind.

On clusters that have fully adopted the upgraded storage engine,`remove node` refuses (HTTP 409) to remove a `stopped`node that still has a WAL tail unless you pass `--force-finalize`.
Parquet clusters and clusters still mid-upgrade do **not** have this safeguard.
On those clusters, completing this recovery procedure before removal is the only
protection against losing the tail.

## Recover the node

1. **Restart a server process with the same `--node-id`** (environment
   variable: `INFLUXDB3_NODE_ID`) and the same object store configuration.
   This is functionally the same as restarting the original node: on startup, WAL recovery replays the WAL files that were not drained, so the
   acknowledged writes are safe again.

   ```
   influxdb3 serve \
     --node-id NODE_ID \
     --cluster-id CLUSTER_ID \
     # ...same object store configuration as the original node
   ```

2. **Stop the node gracefully** and wait for it to reach `stopped`.
   This drains the WAL tail (Parquet: WAL flush; upgraded engine: WAL
   snapshot):

   ```
   influxdb3 stop node --node-id NODE_ID
   ```

   The command waits (up to `--timeout`, default `5m`) for the node to
   confirm the stop.

3. **Remove the node** if you intend to permanently remove it from the
   cluster.
   If you are recovering the node to keep using it, skip this step—after a
   graceful stop, you can start it again at any time.

   ```
   influxdb3 remove node --node-id NODE_ID
   ```

## If the node cannot be brought back

If the node cannot be restarted (for example, its data is
unrecoverable or the hardware is gone) and you accept the possible loss of
writes not yet captured in a snapshot, use the[`--force-finalize` option](/influxdb3/enterprise/reference/cli/influxdb3/remove/node/#force-removal-of-a-node-that-did-not-shut-down-cleanly)to force the removal:

```bash
influxdb3 remove node --node-id NODE_ID --force-finalize
```

> [!Warning]
> `--force-finalize` **can lose data**: it removes the node using
> whatever snapshot exists in object storage and permanently deletes any
> acknowledged writes not covered by that snapshot.
> If the node had finished snapshotting and only failed to report a clean
> stop, nothing is lost.
> Always attempt the [recovery procedure](#recover-the-node) first.

## Verify node status

At any point, check node states with[`influxdb3 show nodes`](/influxdb3/enterprise/reference/cli/influxdb3/show/nodes/):

```bash
influxdb3 show nodes
```

#### Related

* [influxdb3 stop node](/influxdb3/enterprise/reference/cli/influxdb3/stop/node/)
* [influxdb3 remove node](/influxdb3/enterprise/reference/cli/influxdb3/remove/node/)
* [influxdb3 show nodes](/influxdb3/enterprise/reference/cli/influxdb3/show/nodes/)

[clustering](/influxdb3/enterprise/tags/clustering/)[nodes](/influxdb3/enterprise/tags/nodes/)[recovery](/influxdb3/enterprise/tags/recovery/)[wal](/influxdb3/enterprise/tags/wal/)