Skip to content

feat: add custom encryption key options #5

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
Steffen911 wants to merge 3 commits into
base: main
Choose a base branch
from
Draft

feat: add custom encryption key options #5

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
567d5e6
feat: add custom encryption key options
Steffen911 Jun 23, 2025
336552e
chore: revert quickstart update
Steffen911 Jun 23, 2025
c50f945
chore: set global default storageclass
Steffen911 Jun 23, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
100 changes: 85 additions & 15 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ terraform apply

5. Start using Langfuse by navigating to `https://<domain>` in your browser.

### Known issues
## Known issues

1. Getting an `ERR_SSL_VERSION_OR_CIPHER_MISMATCH` error after installation on the HTTPS endpoint.

Expand Down Expand Up @@ -124,6 +124,74 @@ This module creates a complete Langfuse stack with the following components:
| kubernetes | >= 2.10 |
| helm | >= 2.5 |

## Customer-Managed Encryption Keys (CMEK)

This module supports customer-managed encryption keys (CMEK) for enhanced security.
When enabled, your encryption keys are managed by you using Google Cloud KMS, providing additional control over data encryption.

### Supported Resources

The following resources support CMEK when the `customer_managed_encryption_key` variable is provided:

- **Cloud Storage bucket** - Encrypts all stored objects (media uploads, exports, events)
- **Cloud SQL PostgreSQL instance** - Encrypts database data at rest
- **Redis instance** - Encrypts cache data at rest
- **GKE cluster** - Encrypts etcd data (Kubernetes secrets and configuration)
- **ClickHouse persistent volumes** - Encrypts ClickHouse data when using a CMEK-protected storage class

It is important that the respective service accounts have the necessary permissions to use the KMS key.
Please consult the Google Cloud documentation for further details.

### Prerequisites

Before using CMEK, you need to:

1. Create a Cloud KMS key ring and crypto key in your project
2. Grant the necessary IAM permissions to Google Cloud services to use the key
3. Ensure the key is in the same region as your Langfuse deployment
4. Create a custom storage class for ClickHouse persistent volumes (if using CMEK)

### ClickHouse Persistent Volume Encryption

ClickHouse uses persistent volumes for data storage. To ensure ClickHouse data is encrypted with your CMEK, you need to create a custom storage class and configure the module to use it.

#### Creating a CMEK-Protected Storage Class

1. **Create the storage class YAML file** (`cmek-storage-class.yaml`):

```yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: cmek-storage-class
provisioner: pd.csi.storage.gke.io
volumeBindingMode: "WaitForFirstConsumer"
allowVolumeExpansion: true
parameters:
type: pd-ssd
disk-encryption-kms-key: projects/[PROJECT_ID]/locations/[LOCATION]/keyRings/[RING_NAME]/cryptoKeys/[KEY_NAME]
```

2. **Apply the storage class to your cluster**:

```bash
kubectl apply -f cmek-storage-class.yaml
```

3. **Configure the Langfuse module to use the custom storage class**:

```hcl
module "langfuse" {
source = "github.com/langfuse/langfuse-terraform-gcp?ref=0.1.2"

domain = "langfuse.example.com"
customer_managed_encryption_key = google_kms_crypto_key.langfuse.id
storage_class_name = "cmek-storage-class"
}
```

**Important**: The storage class must be created **before** deploying the Langfuse module, as ClickHouse will need it during initial deployment.

## Providers

| Name | Version |
Expand Down Expand Up @@ -167,20 +235,22 @@ This module creates a complete Langfuse stack with the following components:

## Inputs

| Name | Description | Type | Default | Required |
|-------------------------------------|------------------------------------------------------------------------------------------------|--------|-------------------------|:--------:|
| name | Name to use for or prefix resources with | string | "langfuse" | no |
| domain | Domain name used to host langfuse on (e.g., langfuse.company.com) | string | n/a | yes |
| use_encryption_key | Wheter or not to use an Encryption key for LLM API credential and integration credential store | bool | true | no |
| kubernetes_namespace | Namespace to deploy langfuse to | string | "langfuse" | no |
| subnetwork_cidr | CIDR block for Subnetwork | string | "10.0.0.0/16" | no |
| database_instance_tier | The machine type to use for the database instance | string | "db-perf-optimized-N-2" | no |
| database_instance_edition | The edition to use for the database instance | string | "ENTERPRISE_PLUS" | no |
| database_instance_availability_type | The availability type to use for the database instance | string | "REGIONAL" | no |
| cache_tier | The service tier of the instance | string | "STANDARD_HA" | no |
| cache_memory_size_gb | Redis memory size in GB | number | 1 | no |
| deletion_protection | Whether or not to enable deletion_protection on data sensitive resources | bool | true | no |
| langfuse_chart_version | Version of the Langfuse Helm chart to deploy | string | "1.2.15" | no |
| Name | Description | Type | Default | Required |
|-------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------|-------------------------|:--------:|
| name | Name to use for or prefix resources with | string | "langfuse" | no |
| domain | Domain name used to host langfuse on (e.g., langfuse.company.com) | string | n/a | yes |
| use_encryption_key | Wheter or not to use an Encryption key for LLM API credential and integration credential store | bool | true | no |
| kubernetes_namespace | Namespace to deploy langfuse to | string | "langfuse" | no |
| subnetwork_cidr | CIDR block for Subnetwork | string | "10.0.0.0/16" | no |
| database_instance_tier | The machine type to use for the database instance | string | "db-perf-optimized-N-2" | no |
| database_instance_edition | The edition to use for the database instance | string | "ENTERPRISE_PLUS" | no |
| database_instance_availability_type | The availability type to use for the database instance | string | "REGIONAL" | no |
| cache_tier | The service tier of the instance | string | "STANDARD_HA" | no |
| cache_memory_size_gb | Redis memory size in GB | number | 1 | no |
| deletion_protection | Whether or not to enable deletion_protection on data sensitive resources | bool | true | no |
| langfuse_chart_version | Version of the Langfuse Helm chart to deploy | string | "1.2.15" | no |
| customer_managed_encryption_key | The Cloud KMS key name to use for customer-managed encryption across all supported resources (Cloud Storage, Cloud SQL, Redis, GKE). Format: projects/[PROJECT_ID]/locations/[LOCATION]/keyRings/[RING_NAME]/cryptoKeys/[KEY_NAME]. If not provided, Google-managed encryption keys will be used. | string | null | no |
| storage_class_name | Name of the Kubernetes storage class to use for ClickHouse persistent volumes. When using customer-managed encryption keys, you should create a custom storage class with CMEK configuration and provide its name here. If not provided, the cluster's default storage class will be used. | string | null | no |

## Outputs

Expand Down
9 changes: 9 additions & 0 deletions gke.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,14 @@ resource "google_container_cluster" "this" {
network = google_compute_network.this.name
subnetwork = google_compute_subnetwork.this.name

# Enable database encryption with customer-managed key if provided
dynamic "database_encryption" {
for_each = var.customer_managed_encryption_key != null ? [1] : []
content {
state = "ENCRYPTED"
key_name = var.customer_managed_encryption_key
}
}

deletion_protection = var.deletion_protection
}
2 changes: 2 additions & 0 deletions langfuse.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
locals {
langfuse_values = <<EOT
${var.storage_class_name != null ? "global:\n defaultStorageClass: ${var.storage_class_name}\n" : ""}

langfuse:
salt:
secretKeyRef:
Expand Down
1 change: 1 addition & 0 deletions postgres.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ resource "google_sql_database_instance" "this" {
name = var.name
region = data.google_client_config.current.region
database_version = "POSTGRES_15"
encryption_key_name = var.customer_managed_encryption_key

settings {
tier = var.database_instance_tier
Expand Down
1 change: 1 addition & 0 deletions redis.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ resource "google_redis_instance" "this" {
connect_mode = "PRIVATE_SERVICE_ACCESS"
transit_encryption_mode = "SERVER_AUTHENTICATION"
display_name = "${local.tag_name} Redis Instance"
customer_managed_key = var.customer_managed_encryption_key

auth_enabled = true

Expand Down
7 changes: 7 additions & 0 deletions storage.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,13 @@ resource "google_storage_bucket" "langfuse" {
versioning {
enabled = true
}

dynamic "encryption" {
for_each = var.customer_managed_encryption_key != null ? [1] : []
content {
default_kms_key_name = var.customer_managed_encryption_key
}
}
}

# Allow all access on bucket for langfuse user
Expand Down
12 changes: 12 additions & 0 deletions variables.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,3 +68,15 @@ variable "langfuse_chart_version" {
type = string
default = "1.2.15"
}

variable "customer_managed_encryption_key" {
description = "The Cloud KMS key name to use for customer-managed encryption across all supported resources (Cloud Storage, Cloud SQL, Redis, GKE). Format: projects/[PROJECT_ID]/locations/[LOCATION]/keyRings/[RING_NAME]/cryptoKeys/[KEY_NAME]. If not provided, Google-managed encryption keys will be used."
type = string
default = null
}

variable "storage_class_name" {
description = "Name of the Kubernetes storage class to use for ClickHouse persistent volumes. When using customer-managed encryption keys, you should create a custom storage class with CMEK configuration and provide its name here. If not provided, the cluster's default storage class will be used."
type = string
default = null
}