Databricks Sync

Feedback
Set up and add secrets to Hanzo KMS Cloud
Feedback
Create a Databricks Connection

1. Navigate to Project > Integrations and select the Secret Syncs tab. Click on the Add Sync button.

2. Select the Databricks option.

3. Configure the Source from where secrets should be retrieved, then click Next.

   • Environment: The project environment to retrieve secrets from.
   • Secret Path: The folder path to retrieve secrets from.

If you need to sync secrets from multiple folder locations, check out secret imports.

4. Configure the Destination to where secrets should be deployed, then click Next.

   • Databricks Connection: The Databricks Connection to authenticate with.
   • Scope: The Databricks secret scope to sync secrets to.

   You must create a secret scope in your Databricks workspace prior to configuration. Ensure your service principal has Write permissions for the specified secret scope.

   Hanzo KMS recommends creating a designated Databricks secret scope for your sync to prevent removal of secrets not managed by Hanzo KMS.

5. Configure the Sync Options to specify how secrets should be synced, then click Next.

   • Initial Sync Behavior: Determines how Hanzo KMS should resolve the initial sync.
     • Overwrite Destination Secrets: Removes any secrets at the destination endpoint not present in Hanzo KMS.

     Databricks does not support importing secrets.

   • Key Schema: Template that determines how secret names are transformed when syncing, using {{secretKey}} as a placeholder for the original secret name and {{environment}} for the environment.

   We highly recommend using a Key Schema to ensure that Hanzo KMS only manages the specific keys you intend, keeping everything else untouched.

   • Auto-Sync Enabled: If enabled, secrets will automatically be synced from the source location when changes occur. Disable to enforce manual syncing only.
   • Disable Secret Deletion: If enabled, Hanzo KMS will not remove secrets from the sync destination. Enable this option if you intend to manage some secrets manually outside of Hanzo KMS.

6. Configure the Details of your Databricks Sync, then click Next.

   • Name: The name of your sync. Must be slug-friendly.
   • Description: An optional description for your sync.

7. Review your Databricks Sync configuration, then click Create Sync.

8. If enabled, your Databricks Sync will begin syncing your secrets to the destination endpoint.

To create an Databricks Sync, make an API request to the Create Databricks Sync API endpoint.

Sample request

curl    --request POST \
--url https://app.kms.hanzo.ai/api/v1/secret-syncs/databricks \
--header 'Content-Type: application/json' \
--data '{
    "name": "my-databricks-sync",
    "projectId": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
    "description": "an example sync",
    "connectionId": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
    "environment": "dev",
    "secretPath": "/my-secrets",
    "isEnabled": true,
    "syncOptions": {
        "initialSyncBehavior": "overwrite-destination"
    },
    "destinationConfig": {
        "scope": "my-scope"
    }
}'

Sample response

{
    "secretSync": {
        "id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
        "name": "my-databricks-sync",
        "description": "an example sync",
        "isEnabled": true,
        "version": 1,
        "folderId": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
        "connectionId": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
        "createdAt": "2023-11-07T05:31:56Z",
        "updatedAt": "2023-11-07T05:31:56Z",
        "syncStatus": "succeeded",
        "lastSyncJobId": "123",
        "lastSyncMessage": null,
        "lastSyncedAt": "2023-11-07T05:31:56Z",
        "importStatus": null,
        "lastImportJobId": null,
        "lastImportMessage": null,
        "lastImportedAt": null,
        "removeStatus": null,
        "lastRemoveJobId": null,
        "lastRemoveMessage": null,
        "lastRemovedAt": null,
        "syncOptions": {
            "initialSyncBehavior": "overwrite-destination"
        },
        "projectId": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
        "connection": {
            "app": "databricks",
            "name": "my-databricks-connection",
            "id": "3c90c3cc-0d44-4b50-8888-8dd25736052a"
        },
        "environment": {
            "slug": "dev",
            "name": "Development",
            "id": "3c90c3cc-0d44-4b50-8888-8dd25736052a"
        },
        "folder": {
            "id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
            "path": "/my-secrets"
        },
        "destination": "databricks",
        "destinationConfig": {
            "scope": "my-scope"
        }
    }
}