Database & Workflow Modules
These modules handle database initialization and workflow orchestration for DocuStack applications.
DB Init Lambda
Lambda function for automated PostgreSQL database and user initialization in VPC-connected RDS instances.
Why This Module?
When RDS is deployed, it creates an empty PostgreSQL instance. Applications like Temporal require specific databases and users to be created before they can start. This Lambda:
- Automates database creation - No manual SQL required
- Creates users with proper permissions - Least privilege access
- Is idempotent - Safe to run multiple times
- Runs in VPC - Accesses private RDS instances securely
Architecture
┌─────────────────────────────────────────────────────────────┐
│ DB Init Lambda Architecture │
├─────────────────────────────────────────────────────────────┤
│ Manual Invoke / RDS Deployment │
│ │ │
│ │ trigger │
│ v │
│ ┌──────────────────────┐ │
│ │ DB Init Lambda │ │
│ │ (VPC-connected) │ │
│ │ │ │
│ │ 1. Get credentials │◄────── Secrets Manager │
│ │ 2. Connect to RDS │ │
│ │ 3. CREATE DATABASE │ │
│ │ 4. CREATE USER │ │
│ │ 5. GRANT privileges │ │
│ └──────────┬───────────┘ │
│ │ │
│ │ PostgreSQL (port 5432) │
│ v │
│ ┌──────────────────────┐ │
│ │ RDS PostgreSQL │ │
│ │ (Private subnet) │ │
│ │ │ │
│ │ Databases: │ │
│ │ - temporal │ │
│ │ - temporal_visibility│ │
│ └──────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
Usage
module "db_init_lambda" {
source = "git::git@github.com:docustackapp/docustack-infrastructure-modules.git//modules/db-init-lambda?ref=v1.0.0"
function_name = "docustack-dev-db-init"
vpc_id = module.vpc.vpc_id
subnet_ids = module.vpc.private_subnet_ids
rds_endpoint = module.rds.endpoint
rds_secret_arn = module.rds.secret_arn
rds_security_group_id = module.rds.security_group_id
database_name = "temporal"
database_user = "temporal"
lambda_source_dir = "${path.root}/../../../docustack-mono/services/lambdas/db-init"
kms_key_arn = aws_kms_key.cloudwatch.arn
}
Invoking the Lambda
# Create temporal database
aws lambda invoke \
--function-name docustack-dev-db-init \
--payload '{"database_name":"temporal","database_user":"temporal"}' \
/tmp/response.json
# Create temporal_visibility database
aws lambda invoke \
--function-name docustack-dev-db-init \
--payload '{"database_name":"temporal_visibility","database_user":"temporal"}' \
/tmp/response.json
What Gets Created
For each invocation, the Lambda executes:
-- Create database (if not exists)
CREATE DATABASE temporal;
-- Create user (if not exists)
CREATE USER temporal WITH PASSWORD 'generated-password';
-- Grant privileges
GRANT ALL PRIVILEGES ON DATABASE temporal TO temporal;
GRANT ALL ON SCHEMA public TO temporal;
Verification
# Check Lambda logs
aws logs tail /aws/lambda/docustack-dev-db-init --since 10m
# Connect to RDS and verify
psql -h your-rds-endpoint -U postgres
\l -- List databases
\du -- List users
Lambda Code Location
Source: docustack-mono/services/lambdas/db-init/
The Lambda includes pre-bundled psycopg2 for PostgreSQL connectivity.
Temporal
Deploys self-hosted Temporal workflow orchestration platform on AWS ECS Fargate.
Why This Module?
Temporal provides durable workflow execution for complex business processes:
- Fault tolerance - Workflows survive failures and restarts
- Visibility - Web UI for monitoring workflow state
- Scalability - Handles millions of concurrent workflows
- Developer experience - Write workflows as regular code
Architecture
┌─────────────────────────────────────────────────────────────────┐
│ Temporal on ECS │
├─────────────────────────────────────────────────────────────────┤
│ Internet │
│ │ │
│ ├──────────► ALB (Web UI) ──────────┐ │
│ │ Port 8080 │ │
│ │ │ │
│ └──────────► NLB (gRPC) ────────────┤ │
│ Port 7233 │ │
│ │ │
│ v │
│ ┌───────────────────┐ │
│ │ ECS Service │ │
│ │ (Fargate Spot) │ │
│ │ │ │
│ │ Temporal Server │ │
│ │ auto-setup:1.24.2│ │
│ └─────────┬─────────┘ │
│ │ │
│ ┌────────────┴────────────┐ │
│ │ │ │
│ v v │
│ ┌─────────────┐ ┌─────────────────┐ │
│ │ temporal │ │ temporal_ │ │
│ │ database │ │ visibility │ │
│ └─────────────┘ └─────────────────┘ │
│ │ │ │
│ └────────────┬────────────┘ │
│ v │
│ ┌──────────────────────┐ │
│ │ RDS PostgreSQL │ │
│ │ (Shared Instance) │ │
│ └──────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
Prerequisites
Before deploying Temporal:
- VPC with private subnets
- ECS cluster deployed
- RDS PostgreSQL available
- Databases created via db-init-lambda:
temporal- Main workflow storagetemporal_visibility- Search and query optimization
Usage
module "temporal" {
source = "../../modules/temporal"
name = "docustack-dev"
environment = "dev"
vpc_id = module.vpc.vpc_id
private_subnets = module.vpc.private_subnets
public_subnets = module.vpc.public_subnets
ecs_cluster_id = module.ecs_cluster.cluster_id
ecs_cluster_name = module.ecs_cluster.cluster_name
db_host = module.rds.endpoint
db_port = 5432
db_password_secret_arn = module.rds.master_password_secret_arn
# Use ECR pull-through cache to avoid Docker Hub rate limits
temporal_image = "${data.aws_caller_identity.current.account_id}.dkr.ecr.us-east-1.amazonaws.com/docker-hub/temporalio/auto-setup:1.24.2"
cpu = 1024 # 1 vCPU
memory = 2048 # 2 GB
use_fargate_spot = true
}
ECR Pull-Through Cache
Docker Hub enforces rate limits (100-200 pulls per 6 hours). The module uses ECR pull-through cache:
# Store Docker Hub credentials
aws secretsmanager create-secret \
--name ecr-pullthroughcache/docker-hub \
--secret-string '{"username":"YOUR_USERNAME","secret":"YOUR_ACCESS_TOKEN"}'
Images are then pulled via ECR:
123456789012.dkr.ecr.us-east-1.amazonaws.com/docker-hub/temporalio/auto-setup:1.24.2
RDS TLS Configuration
Temporal's auto-setup image doesn't easily support TLS. For dev environments:
# Disable TLS (dev only)
aws rds modify-db-parameter-group \
--db-parameter-group-name docustack-dev-pg \
--parameters "ParameterName=rds.force_ssl,ParameterValue=0,ApplyMethod=immediate"
aws rds reboot-db-instance \
--db-instance-identifier docustack-dev-postgres
Production must use TLS for HIPAA compliance.
Deployment
# 1. Ensure databases exist
aws lambda invoke --function-name docustack-dev-db-init /tmp/response.json
# 2. Deploy Temporal
cd infrastructure/dev/us-east-1/temporal
terragrunt apply # Takes ~8-10 minutes
# 3. Verify
aws ecs describe-services \
--cluster docustack-dev \
--services temporal-dev \
--query 'services[0].{Status:status,Running:runningCount}'
Accessing Temporal
# Get Web UI URL
aws elbv2 describe-load-balancers \
--names temporal-alb-dev \
--query 'LoadBalancers[0].DNSName' \
--output text
# Get gRPC endpoint
aws elbv2 describe-load-balancers \
--names temporal-nlb-dev \
--query 'LoadBalancers[0].DNSName' \
--output text
Cost Breakdown
| Component | 24/7 | With Nightly Scheduler |
|---|---|---|
| ECS Fargate (1 task, Spot) | ~$9/month | ~$3/month |
| ALB | ~$16/month | ~$16/month |
| NLB | ~$16/month | ~$16/month |
| Total | ~$41/month | ~$35/month |
Note: Load balancers cannot be stopped. For maximum savings, use full teardown on weekends.
Troubleshooting
Database Permission Errors
ERROR: permission denied for schema public
Solution: Re-run db-init-lambda:
aws lambda invoke --function-name docustack-dev-db-init /tmp/response.json
TLS Connection Errors
FATAL: no pg_hba.conf entry for host "10.0.x.x", user "temporal", database "temporal", no encryption
Solution: Disable TLS for dev (see above) or configure TLS certificates for production.
Docker Hub Rate Limiting
Error response from daemon: toomanyrequests
Solution: ECR pull-through cache should handle this. Force new deployment:
aws ecs update-service --cluster docustack-dev --service temporal-dev --force-new-deployment