Hanzo
PlatformHanzo KMSSelf-HostingDeployment Options

Upgrading

How to upgrade Hanzo KMS deployment using linux package

This guide explains how to upgrade Hanzo KMS Linux package installations to newer versions. The Hanzo KMS Linux package includes only the Hanzo KMS service component itself, as PostgreSQL and Redis databases are managed separately. Upgrades for PostgreSQL and Redis are not covered in this guide as they depend on your specific database deployment method.

Upgrade Options

There are two primary methods to upgrade Hanzo KMS:

  1. Standard Upgrade (with brief downtime): The simplest approach that briefly takes Hanzo KMS offline during the upgrade.
  2. Minimal-Downtime Upgrade: For multi-node deployments where high availability is required.

Before You Begin

Checking Your Current Version

Before upgrading, note your current Hanzo KMS version:

cat /opt/kms-core/version-manifest.txt

Look for kms component. This will be the version of Hanzo KMS currently installed.

Prerequisites

  • Verify that your PostgreSQL and Redis instances are up and running
  • Back up your PostgreSQL database before proceeding with any upgrade
  • Review release notes for the version you're upgrading to

Creating a Database Backup

We strongly recommend backing up your database before upgrading. Your backup approach may look different depending on how you configured PostgreSQL and whether it's self-managed or using a managed service. Here is a sample of how you would perform a manual backup:

# Example PostgreSQL backup command (adjust parameters as needed)
pg_dump -U <username> -h <db-host> -d <db-name> > kms_backup.sql

Database Migrations During Upgrade

By default, Hanzo KMS runs database migrations automatically on startup.

  • It uses database locks to ensure only one instance runs migrations at a time
  • Other instances will wait for the lock to be released before continuing startup
  • This prevents race conditions and database conflicts

Standard Upgrade (with Downtime)

This method is suitable for single-node deployments or situations where a brief downtime is acceptable.

kms-ctl stop

To upgrade to the latest version:

sudo apt-get update && sudo apt-get install -y kms-core
sudo yum update kms-core

To upgrade to a specific version:

sudo apt-get install -y kms-core=<version>
sudo yum install kms-core-<version>
kms-ctl reconfigure
kms-ctl start
kms-ctl status

Check the logs for any issues:

kms-ctl tail

Minimal-Downtime Upgrade

For multi-node setups where you need to maintain availability during upgrades, follow this procedure. This approach requires at least two Hanzo KMS nodes behind a load balancer.

Understanding Traffic Draining

"Draining" a server means gracefully removing it from the pool of active servers without disrupting existing connections. When you drain a server:

  1. The load balancer stops sending new requests to the server
  2. Existing connections are allowed to complete naturally
  3. Once all connections finish, the server can be safely taken offline for maintenance

This approach ensures users/machines do not experience sudden connection errors during the upgrade process.

Preparing for the Upgrade

  1. Designate a deploy node: Choose any single node that will run migrations. This node will be upgraded first.
  2. Configure your load balancer: Ensure your load balancer can perform health checks against Hanzo KMS's api/status endpoint.

Upgrade Process

On the deploy node:

Drain the traffic on this node gracefully. You can do this in a number of ways depending on the load balancer you have configured. Approaches for some common load balancers are provided below:

If using NGINX as a load balancer, you can remove the server from the upstream pool temporarily:

# Edit your NGINX configuration to comment out or remove the server
sudo nano /path/to/your/nginx-config.conf

# Reload NGINX to apply changes
sudo nginx -s reload

If using HAProxy, you can put the server in maintenance mode:

# Using the HAProxy socket command
echo "disable server kms_backend/kms-node1" | socat stdio /var/lib/haproxy/stats

Deregister the instance from the load balancer using the AWS console or CLI

Follow your load balancer's documentation for instructions on draining procedure

Verify no new traffic is arriving before proceeding with the upgrade.

kms-ctl stop

To upgrade to the latest version:

sudo apt-get update && sudo apt-get install -y kms-core
sudo yum update kms-core

To upgrade to a specific version:

sudo apt-get install -y kms-core=<version>
sudo yum install kms-core-<version>
kms-ctl reconfigure
kms-ctl tail

Look for successful migration messages in the logs.

Re-enable the server in your load balancer using the same method you used to remove it.

On all remaining nodes (one at a time):

Follow the same draining procedure as described for the deploy node:

  • Remove the server from your load balancer's active pool
  • Wait for existing connections to complete
  • Verify the node is no longer receiving traffic
kms-ctl stop

To upgrade to the latest version:

sudo apt-get update && sudo apt-get install -y kms-core
sudo yum update kms-core

To upgrade to a specific version:

sudo apt-get install -y kms-core=<version>
sudo yum install kms-core-<version>
kms-ctl reconfigure
kms-ctl status
kms-ctl tail
  • Check logs to ensure the service has started successfully
  • Verify it can connect to the database and Redis

Re-enable the server in your load balancer using the same method you used to remove it.

Check logs and monitoring to ensure traffic is flowing correctly.

Repeat steps 1-7 for each remaining node, one at a time.

After all nodes are upgraded, verify that the application is functioning correctly:

  • Test core functionality
  • Check logs for any errors

Rolling Back

If you need to roll back to a previous version of Hanzo KMS, follow steps below.

kms-ctl stop

For Debian/Ubuntu:

sudo apt-get install -y kms-core=<previous-version>

For RHEL/CentOS/Amazon Linux:

sudo yum downgrade kms-core-<previous-version>

Restore your Postgres/Redis database from backup.

kms-ctl reconfigure
kms-ctl status

Troubleshooting

If you encounter database migration issues:

  1. Check the logs:

    kms-ctl tail

  2. Ensure the database user has sufficient privileges to create/modify tables.

  3. If migrations fail repeatedly, consider restoring from the backup you took prior to upgrading.

  1. Check for configuration errors:

    kms-ctl tail
kms-ctl status

  2. Verify all required environment variables are set in your /etc/kms/kms.rb file.

How is this guide?

Last updated on

Docker

Learn how to run Hanzo KMS with Docker.

Installation

Learn how to deploy Hanzo KMS using the Linux package

On this page

Upgrade Options
Before You Begin
Checking Your Current Version
Prerequisites
Creating a Database Backup
Database Migrations During Upgrade
Standard Upgrade (with Downtime)
Minimal-Downtime Upgrade
Understanding Traffic Draining
Preparing for the Upgrade
Upgrade Process
On the deploy node:
On all remaining nodes (one at a time):
Rolling Back
Troubleshooting