hydra-head-troubleshooter

• Hydra Head doesn't start or no head observed
• Head exists but doesn't make progress
• Peers out of sync or disconnected
• Log messages: PeerConnected, AckSn, LogicOutcome errors

• Confirm network and hydra-node version first
• Request logs from ALL participants for same time window
• Never request key contents (file paths OK)
• Output: (1) Root cause (2) Fix steps (3) Verification

```bash

# Check hydra-node version

hydra-node --version

# Check API health

curl -s localhost:4001/health

# Check peers

curl -s localhost:4001/peers

# Check head status

curl -s localhost:4001/status

# Check metrics (if enabled)

curl -s localhost:6001/metrics | grep hydra

```

A) "No head is observed from the chain"

Symptoms:

• Head never appears in logs
• No Init/Commit/Open progression
• Client shows no head state

Check 1: Cardano connection

```bash

# Verify cardano-node is ready

cardano-cli query tip --testnet-magic 1

# Should show current slot, not error

# Check socket exists

ls -la $CARDANO_NODE_SOCKET_PATH

```

Fix: Wait for cardano-node sync, verify socket path and network magic

Check 2: Scripts tx id

```bash

# Verify you're using correct scripts tx id for network

# Get from hydra-node release notes

# In hydra-node logs, look for:

grep -i "script" hydra-node.log | head -20

```

Fix: Use correct --hydra-scripts-tx-id for your network

Check 3: Key mismatch

```bash

# Verify cardano.sk matches what peers have as your vkey

cardano-cli key verification-key \

--signing-key-file cardano.sk \

--verification-key-file check.vkey

# Compare check.vkey with what you distributed

```

Fix: Re-exchange verification keys with all peers

---

B) "Head does not make progress"

Symptoms:

• Head exists but stuck at Init or Commit
• Snapshots not confirmed
• Commands hang or timeout

Check 1: Peer connectivity

```bash

# In logs, look for PeerConnected

grep "PeerConnected" hydra-node.log

# Check metrics

curl -s localhost:6001/metrics | grep peers_connected

```

Fix: Verify --peer host:port is correct and ports are reachable

Check 2: Hydra key mismatch

```bash

# Look for AckSn issues

grep -E "AckSn|LogicOutcome" hydra-node.log

# Verify hydra keys match

hydra-node verify-keys \

--hydra-signing-key hydra.sk \

--hydra-verification-key peer-hydra.vk

```

Fix: Re-exchange hydra verification keys

Check 3: Version mismatch

```bash

# All nodes must run compatible versions

hydra-node --version

# Compare across all participants

```

Fix: Align hydra-node versions

---

C) "Head stuck: peer out of sync"

Symptoms:

• One node accepts txs, others reject
• Snapshots stop being signed
• Ledger state diverged

Cause: Different ledger views due to config drift or missed txs

Fix: Snapshot side-loading

```bash

# 1. Get latest snapshot from healthy node

curl -s http://healthy-node:4001/snapshot > snapshot.json

# 2. Post to out-of-sync node

curl -X POST http://out-of-sync-node:4001/snapshot \

-H "Content-Type: application/json" \

-d @snapshot.json

```

Verify: Snapshots resume, all parties sign again

---

D) Mirror nodes / HA issues

Symptoms:

• SnapshotAlreadySigned in logs
• etcd quorum issues

Reality:

• SnapshotAlreadySigned is transient and harmless with mirrors
• Keep mirror count < floor(n/2) for quorum stability

Fix: Reduce mirror count or accept transient warnings

---

| Pattern | Meaning |

|---------|---------|

| PeerConnected | Peer connected successfully |

| PeerDisconnected | Peer lost connection |

| AckSn | Snapshot acknowledgment |

| LogicOutcome | Head logic result (check for errors) |

| SnapshotConfirmed | Snapshot finalized |

| HeadIsOpen | Head successfully opened |

| HeadIsClosed | Head closed, contestation started |

After any fix:

• [ ] All nodes same hydra-node version
• [ ] Scripts tx id correct for network
• [ ] cardano-node ready (query tip works)
• [ ] PeerConnected for all peers
• [ ] AckSn messages flowing
• [ ] No LogicOutcome errors

Example: Diagnose stuck head

User request: "My Hydra head won't open—stuck at commit"

Response:

```bash

# 1. Check all peers connected

grep "PeerConnected" hydra-node.log | tail -10

# Should see PeerConnected for each peer

# 2. If missing peers, check network

nc -zv peer-host 5001

# Should show "Connection succeeded"

# 3. Check all parties committed

curl -s localhost:4001/status | jq .

# Look for parties and their commit status

# 4. Check for key issues

grep -E "InvalidSignature|WrongParty" hydra-node.log

# Common fix: restart with correct peer addresses

# Verify --peer host:port matches actual peer locations

```

• hydra-head (operational guidance)
• hydra-head-operator (execution)
• reference/sources.md (doc provenance)
• [hydra.family troubleshooting](https://hydra.family/head-protocol/docs/how-to/operating-hydra)