Cosmovisor
Cosmovisor is the standard Cosmos SDK tool for automated binary upgrades. It monitors for governance-approved upgrade proposals and swaps the monod binary at the correct block height, minimizing downtime during chain upgrades.
Monarch CLI provides equivalent upgrade functionality via OxidePM with monarch upgrade. Cosmovisor is documented here for operators who prefer standard Cosmos SDK tooling or need to integrate with existing infrastructure that expects Cosmovisor. See the comparison table below.
When to Use Cosmovisor
| Scenario | Recommended Tool |
|---|---|
| Single validator, standard operations | Monarch CLI (monarch upgrade) |
| Existing Cosmos infrastructure or tooling | Cosmovisor |
| Operators familiar with Cosmos SDK conventions | Cosmovisor |
| Need TUI monitoring, health checks, relay management | Monarch CLI |
| Multi-chain operator with unified Cosmovisor setup | Cosmovisor |
| New to Monolythium | Monarch CLI |
Installation
go install cosmossdk.io/tools/cosmovisor/cmd/cosmovisor@latest
Verify the installation:
cosmovisor version
Cosmovisor requires Go 1.21 or later. If you do not have Go installed, see Installation for setup instructions.
Directory Structure
Cosmovisor expects a specific directory layout inside your node home directory:
~/.mono/cosmovisor/
├── genesis/
│ └── bin/
│ └── monod # Initial binary (current version)
└── upgrades/
└── <upgrade-name>/
└── bin/
└── monod # New binary for this upgrade
Initial Setup
Create the directory structure and place the current binary:
mkdir -p ~/.mono/cosmovisor/genesis/bin
mkdir -p ~/.mono/cosmovisor/upgrades
# Copy current monod binary
cp $(which monod) ~/.mono/cosmovisor/genesis/bin/monod
# Verify
~/.mono/cosmovisor/genesis/bin/monod version
Environment Variables
Cosmovisor is configured entirely through environment variables:
| Variable | Value | Description |
|---|---|---|
DAEMON_NAME | monod | Name of the binary |
DAEMON_HOME | ~/.mono | Node home directory |
DAEMON_ALLOW_DOWNLOAD_BINARIES | false | Whether to auto-download binaries from upgrade proposals |
DAEMON_RESTART_AFTER_UPGRADE | true | Automatically restart the node after binary swap |
DAEMON_POLL_INTERVAL | 300ms | How often to check for upgrade height |
DAEMON_DATA_BACKUP_DIR | (optional) | Directory for pre-upgrade data backups |
DAEMON_PREUPGRADE_MAX_RETRIES | 0 | Max retries for pre-upgrade hooks |
UNSAFE_SKIP_BACKUP | true | Skip data backup before upgrades (set false if disk space allows) |
Setting DAEMON_ALLOW_DOWNLOAD_BINARIES=true allows Cosmovisor to download binaries from URLs specified in governance proposals. This is a security risk because a malicious proposal could point to a compromised binary. Always set this to false and place verified binaries manually.
systemd Service
Create a systemd unit file that wraps Cosmovisor instead of running monod directly:
sudo tee /etc/systemd/system/monod.service > /dev/null <<'EOF'
[Unit]
Description=Monolythium Node (Cosmovisor)
After=network-online.target
Wants=network-online.target
[Service]
User=mono
Group=mono
Type=simple
WorkingDirectory=/home/mono
Environment="DAEMON_NAME=monod"
Environment="DAEMON_HOME=/home/mono/.mono"
Environment="DAEMON_ALLOW_DOWNLOAD_BINARIES=false"
Environment="DAEMON_RESTART_AFTER_UPGRADE=true"
Environment="DAEMON_POLL_INTERVAL=300ms"
Environment="UNSAFE_SKIP_BACKUP=true"
ExecStart=/home/mono/go/bin/cosmovisor run start --home /home/mono/.mono
Restart=on-failure
RestartSec=5
LimitNOFILE=65535
StandardOutput=journal
StandardError=journal
SyslogIdentifier=monod
[Install]
WantedBy=multi-user.target
EOF
Enable and start:
sudo systemctl daemon-reload
sudo systemctl enable monod
sudo systemctl start monod
Check status:
sudo systemctl status monod
sudo journalctl -u monod -f
Preparing for Upgrades
When a chain upgrade is approved through governance, you need to place the new binary before the upgrade height is reached.
1. Check the Upgrade Plan
monod query upgrade plan
This returns the upgrade name and target block height. If no upgrade is scheduled, it returns "no upgrade scheduled."
2. Build or Download the New Binary
# Clone and build from source
git clone https://github.com/monolythium/mono-chain.git
cd mono-chain
git checkout <release-tag>
make install
# Verify the new version
$(go env GOPATH)/bin/monod version
3. Place the Binary
The directory name must exactly match the upgrade name from the governance proposal:
mkdir -p ~/.mono/cosmovisor/upgrades/<upgrade-name>/bin
cp $(go env GOPATH)/bin/monod ~/.mono/cosmovisor/upgrades/<upgrade-name>/bin/monod
# Verify
~/.mono/cosmovisor/upgrades/<upgrade-name>/bin/monod version
4. Wait
Cosmovisor handles the rest. At the upgrade height:
- The running
monodprocess halts withUPGRADE "<upgrade-name>" NEEDED at height: XXXXXX - Cosmovisor detects the halt and finds the matching upgrade binary
- Cosmovisor swaps the binary and restarts the node
- The node resumes with the new version
Monitoring the Upgrade
# Watch logs during upgrade
sudo journalctl -u monod -f
# After upgrade, verify version
curl -s localhost:26657/status | jq '.result.node_info.version'
Cosmovisor vs Monarch CLI
| Feature | Cosmovisor | Monarch CLI |
|---|---|---|
| Binary swap at upgrade height | Yes | Yes (monarch upgrade) |
| Auto-download binaries | Yes (not recommended) | No (manual or verified download) |
| Process management | Relies on systemd | OxidePM (built-in, health-checked) |
| Health monitoring | No | Yes (automatic health checks) |
| TUI dashboard | No | Yes (monarch status) |
| Relay management | No | Yes (monarch relay) |
| Pre-upgrade hooks | Yes | No |
| Data backup before upgrade | Yes (optional) | No (manual) |
| Cosmos SDK standard | Yes | No (Monolythium-specific) |
| Multi-chain support | Yes (one instance per chain) | No (Monolythium only) |
| Log aggregation | Via journalctl | Built-in with monarch logs |
| Automatic restart | Via systemd + DAEMON_RESTART_AFTER_UPGRADE | Via OxidePM (always) |
| Configuration | Environment variables | monarch commands |
| Installation | Go install | Single install script |
You do not need to run both Cosmovisor and Monarch CLI. Choose one approach for managing your node process and upgrades. Running both will cause conflicts over process ownership.
Troubleshooting
Cosmovisor Cannot Find the Upgrade Binary
UPGRADE "<name>" NEEDED at height: XXXXXX: upgrade binary not found
The upgrade directory name must match the proposal name exactly (case-sensitive). Verify:
ls -la ~/.mono/cosmovisor/upgrades/
Node Fails to Start After Upgrade
Check logs for the specific error:
sudo journalctl -u monod --since "5 minutes ago"
Common causes:
- Wrong binary version or architecture
- Missing library dependencies
- Database migration required (check release notes)
Reverting an Upgrade
Monolythium does not support chain state reversions. Recovery is always forward-only: HALT, PATCH, UPGRADE, RESTART. If an upgrade fails, wait for a patched release from the team.
FAQ
Can I switch from Cosmovisor to Monarch CLI (or vice versa)?
Yes. Stop the Cosmovisor-managed service, then use monarch start to take over process management. Or stop Monarch's OxidePM process and switch to a Cosmovisor systemd unit. Make sure only one tool manages the monod process at a time.
Does Cosmovisor handle non-upgrade restarts?
With Restart=on-failure in the systemd unit, the service restarts if monod crashes. Cosmovisor itself does not manage restarts outside of upgrade scenarios -- that is handled by systemd.
What if I miss the upgrade height?
Your node halts and waits. Place the correct binary in the upgrades directory and restart the service. The node will catch up to the network after the binary swap.
Can I use Cosmovisor with remote signing (TMKMS)?
Yes. Cosmovisor manages the monod binary lifecycle. Remote signing via TMKMS operates independently through the validator's priv_validator_laddr configuration. The two are fully compatible.
Related
- Upgrades - Upgrade process and procedures
- Installation - Building monod from source
- Monarch CLI - Monarch CLI reference and OxidePM
- Monitoring - Monitoring your node