OpenSearch Domain

Cost-optimized OpenSearch domain for development and testing environments.

Overview

OpenSearch Domain provides a managed OpenSearch cluster with predictable costs and full control over instance sizing. This is the recommended option for dev/staging environments where 96% cost savings over serverless justify the operational overhead.

Key Features

Cost-optimized - $31/month vs $691/month for serverless (96% savings)
Full-text search - Traditional keyword-based search with analyzers
Vector search (k-NN) - Semantic similarity search using embeddings
VPC-based - Private subnet deployment, no public access
HIPAA compliant - Encryption at rest/transit, fine-grained access control
OpenSearch 3.3 - Latest stable version with all features

Use Cases

Development environments - Cost-effective search for local development
Staging environments - Pre-production testing with production-like setup
Small workloads - Predictable, low-volume search requirements
Learning/experimentation - Full OpenSearch features at minimal cost

When to Use Domain vs Serverless

Factor	Domain (t3.small)	Serverless
Cost	$31/month (fixed)	$691/month (minimum 4 OCUs)
Best for	Dev, staging, small workloads	Production, variable workloads
Scaling	Manual (change instance type)	Automatic
Availability	Single-AZ (dev) or Multi-AZ	Multi-AZ by default
Management	Requires version upgrades	Fully managed
Startup time	10-15 minutes	Instant

Recommendation:

Dev/Staging: Use OpenSearch Domain (this module)
Production: Use OpenSearch Serverless for auto-scaling and high availability

Architecture

┌──────────────────────────────────────────────────────────────────┐
│                  OpenSearch Domain Architecture                   │
├──────────────────────────────────────────────────────────────────┤
│                                                                   │
│  Developer Laptop                                                │
│       │                                                           │
│       │ aws ssm start-session (port forward)                     │
│       v                                                           │
│  ┌─────────────────┐                                             │
│  │  Bastion Host   │                                             │
│  │  (Private)      │                                             │
│  └────────┬────────┘                                             │
│           │                                                       │
│           │ HTTPS (443)                                           │
│           v                                                       │
│  ┌─────────────────────────┐        ┌──────────────────┐        │
│  │  OpenSearch Domain      │───────►│  Security        │        │
│  │  (Private Subnet)       │        │  - Fine-grained  │        │
│  │                         │        │    access control│        │
│  │  - t3.small.search      │        │  - Security group│        │
│  │  - 50GB gp3 storage     │        │  - Resource      │        │
│  │  - OpenSearch 3.3       │        │    policy        │        │
│  └────────┬────────────────┘        └──────────────────┘        │
│           │                                                       │
│           │                          ┌──────────────────┐        │
│           └─────────────────────────►│  KMS Encryption  │        │
│                                      │  (AWS-managed)   │        │
│                                      └──────────────────┘        │
│                                                                   │
│  ┌─────────────────────────┐        ┌──────────────────┐        │
│  │  SSM Parameter Store    │        │  CloudWatch      │        │
│  │  - Endpoint             │        │  - Metrics       │        │
│  │  - Master password      │        │  - Logs          │        │
│  │  - Security group ID    │        │  - Alarms        │        │
│  └─────────────────────────┘        └──────────────────┘        │
│                                                                   │
└──────────────────────────────────────────────────────────────────┘

Prerequisites

Before deploying OpenSearch Domain:

VPC Module - Deployed with private subnets
Bastion Orchestrator - For local development access
AWS CLI - With Session Manager plugin
SSM Parameter Store - For credential management

Deployment

Step 1: Deploy OpenSearch Domain

The module is already configured in infrastructure-live/_envcommon/opensearch-domain.hcl:

terraform {
  source = "git::git@github.com:docustackapp/docustack-infrastructure-modules.git//modules/opensearch-domain?ref=v1.3.1"
}

inputs = {
  project_name = local.project_name
  environment  = local.environment
  
  # Instance configuration
  instance_type  = "t3.small.search"  # 2 vCPU, 2GB RAM
  instance_count = 1                   # Single node for dev
  
  # Storage configuration
  ebs_volume_size = 50                 # GB
  ebs_volume_type = "gp3"
  ebs_iops        = 3000
  ebs_throughput  = 125                # MB/s
  
  # OpenSearch version
  engine_version = "OpenSearch_3.3"
  
  # Network configuration
  vpc_id            = dependency.vpc.outputs.vpc_id
  subnet_ids        = [dependency.vpc.outputs.private_subnet_ids[0]]  # Single AZ
  allowed_cidr_blocks = [dependency.vpc.outputs.vpc_cidr_block]
  
  # Security
  create_master_user = true
  master_username    = "admin"
}

Deploy to dev:

cd infrastructure-live/dev/us-east-1/opensearch-domain
terragrunt plan
terragrunt apply

Deployment time: ~10-15 minutes

Step 2: Retrieve Credentials

The master password is stored in SSM Parameter Store:

aws ssm get-parameter \
  --name /docustack/dev/opensearch/master-password \
  --with-decryption \
  --region us-east-1 \
  --query 'Parameter.Value' \
  --output text

Step 3: Access Locally

See OpenSearch Local Access for complete guide.

Quick start:

# 1. Create bastion via Slack
/infra bastion create

# 2. Port forward to OpenSearch
aws ssm start-session \
  --target i-BASTION_ID \
  --document-name AWS-StartPortForwardingSessionToRemoteHost \
  --parameters '{"host":["OPENSEARCH_ENDPOINT"],"portNumber":["443"],"localPortNumber":["9200"]}' \
  --region us-east-1

# 3. Access cluster
curl -k -u admin:PASSWORD https://localhost:9200/_cluster/health

Configuration

Instance Types

Instance Type	vCPU	RAM	Storage	Cost/Month	Use Case
t3.small.search	2	2GB	EBS	$31	Dev, small workloads
t3.medium.search	2	4GB	EBS	$62	Staging, medium workloads
r5.large.search	2	16GB	EBS	$146	Production, memory-intensive

Storage Options

Volume Type	IOPS	Throughput	Cost/GB/Month	Use Case
gp3	3,000-16,000	125-1,000 MB/s	$0.08	General purpose (recommended)
gp2	3 IOPS/GB	250 MB/s	$0.10	Legacy
io1	Up to 64,000	1,000 MB/s	$0.125 + IOPS	High performance

Recommendation: Use gp3 for cost-effectiveness and performance.

Multi-AZ Deployment

For staging/production, enable multi-AZ:

inputs = {
  instance_count = 3  # Minimum for multi-AZ
  subnet_ids     = dependency.vpc.outputs.private_subnet_ids  # All AZs
  
  zone_awareness_enabled = true
  availability_zone_count = 3
}

Cost impact: 3x instance costs ($93/month for 3x t3.small)

Security

Fine-Grained Access Control

The module enables fine-grained access control with:

Master user: admin (password in SSM)
Internal database: OpenSearch's built-in user database
Role-based access: Create additional users/roles in Dashboards

Network Security

VPC-only: No public endpoint
Security group: Allows HTTPS (443) from VPC CIDR only
Resource policy: Allows all ES actions from within VPC
Encryption in transit: TLS 1.2+ enforced

Encryption

At rest: AWS-managed KMS key
In transit: Node-to-node encryption enabled
HTTPS: Required for all API calls

Access Patterns

Access Method	Authentication	Use Case
Port forwarding via bastion	Master user (admin)	Local development
VPC endpoint (future)	IAM roles	Application access
OpenSearch Dashboards	Master user or IAM	Administration

Operations

Monitoring

CloudWatch metrics available:

Cluster health: Green/Yellow/Red status
CPU utilization: Instance CPU usage
JVM memory pressure: Heap usage
Storage: Free space, IOPS, throughput
Search performance: Query latency, indexing rate

Backups

Automated snapshots:

inputs = {
  automated_snapshot_start_hour = 3  # 3 AM UTC
}

Snapshots stored in AWS-managed S3 bucket (free).

Upgrades

OpenSearch version upgrades:

inputs = {
  engine_version = "OpenSearch_3.4"  # Update version
}

Important: Test upgrades in dev before applying to staging/prod.

Scaling

Vertical Scaling (Instance Type)

inputs = {
  instance_type = "t3.medium.search"  # Upgrade from t3.small
}

Downtime: ~10-15 minutes during instance replacement.

Horizontal Scaling (Node Count)

inputs = {
  instance_count = 3  # Scale from 1 to 3 nodes
}

Downtime: None (rolling deployment).

Storage Scaling

inputs = {
  ebs_volume_size = 100  # Increase from 50GB
}

Downtime: None (online resize).

Cost Analysis

Dev Environment (Current)

Resource	Specification	Cost/Month
OpenSearch instance	t3.small.search	$31.00
EBS storage	50GB gp3	$4.00
Total		$35.00

Comparison with Serverless

Metric	Domain (t3.small)	Serverless
Monthly cost	$35	$691
Savings	$656/month	-
Savings %	95%	-
Annual savings	$7,872	-

Scaling Costs

Configuration	Monthly Cost	Use Case
1x t3.small (dev)	$35	Development
1x t3.medium (staging)	$66	Staging
3x t3.medium (multi-AZ)	$198	Production (small)
3x r5.large (multi-AZ)	$438	Production (large)

Still cheaper than serverless for predictable workloads.

Troubleshooting

Cluster Status Yellow

Cause: Unassigned replica shards (expected for single-node clusters).

Solution: For dev, this is normal. For production, use multi-AZ with 3+ nodes.

Out of Memory Errors

Cause: JVM heap pressure too high.

Solution:

Check CloudWatch metric JVMMemoryPressure
If consistently >75%, upgrade instance type
Consider reducing shard count or index size

Slow Queries

Cause: Insufficient resources or inefficient queries.

Solution:

Check CloudWatch metric SearchLatency
Review slow query logs in CloudWatch Logs
Optimize queries or upgrade instance type

Cannot Connect via Port Forwarding

Cause: Security group or bastion issues.

Solution: See OpenSearch Local Access - Troubleshooting

Migration from Serverless

If migrating from OpenSearch Serverless:

Step 1: Export Data

# Export indices from serverless collection
elasticdump \
  --input=https://SERVERLESS_ENDPOINT/my-index \
  --output=my-index.json \
  --type=data

Step 2: Deploy Domain

cd infrastructure-live/dev/us-east-1/opensearch-domain
terragrunt apply

Step 3: Import Data

# Import to domain
elasticdump \
  --input=my-index.json \
  --output=https://localhost:9200/my-index \
  --type=data

Step 4: Update Application

Update application to use new endpoint from SSM:

aws ssm get-parameter \
  --name /docustack/dev/opensearch/endpoint \
  --query 'Parameter.Value' \
  --output text

Step 5: Destroy Serverless

cd infrastructure-live/dev/us-east-1/opensearch-serverless
terragrunt destroy

SSM Parameters

The module creates these SSM parameters for cross-module reference:

Parameter	Description	Example
`/docustack/dev/opensearch/endpoint`	Domain endpoint	`vpc-docustack-dev-search-*.es.amazonaws.com`
`/docustack/dev/opensearch/domain-name`	Domain name	`docustack-dev-search`
`/docustack/dev/opensearch/master-password`	Master user password	(SecureString)
`/docustack/dev/opensearch/kibana-endpoint`	Dashboards URL	`https://vpc-docustack-dev-search-*/_dashboards`
`/docustack/dev/opensearch/security-group-id`	Security group ID	`sg-*`

OpenSearch Local Access - How to connect locally
Bastion Orchestrator - On-demand bastion hosts
OpenSearch Serverless - Alternative for production
AWS OpenSearch Service - Official AWS docs

Module Source

docustack-infrastructure-modules/modules/opensearch-domain/
├── main.tf                 # Domain resource
├── variables.tf            # Input variables
├── outputs.tf              # Outputs
├── data.tf                 # Data sources
├── locals.tf               # Local values
├── versions.tf             # Provider requirements
├── security-groups.tf      # Security group
├── ssm-parameters.tf       # SSM parameters
├── README.md               # Module documentation
└── examples/
    └── basic/
        └── terragrunt.hcl  # Example usage

Current version: v1.3.1

Repository: docustack-infrastructure-modules

Overview​

Key Features​

Use Cases​

When to Use Domain vs Serverless​

Architecture​

Prerequisites​

Deployment​

Step 1: Deploy OpenSearch Domain​

Step 2: Retrieve Credentials​

Step 3: Access Locally​

Configuration​

Instance Types​

Storage Options​

Multi-AZ Deployment​

Security​

Fine-Grained Access Control​

Network Security​

Encryption​

Access Patterns​

Operations​

Monitoring​

Backups​

Upgrades​

Scaling​

Vertical Scaling (Instance Type)​

Horizontal Scaling (Node Count)​

Storage Scaling​

Cost Analysis​

Dev Environment (Current)​

Comparison with Serverless​

Scaling Costs​

Troubleshooting​

Cluster Status Yellow​

Out of Memory Errors​

Slow Queries​

Cannot Connect via Port Forwarding​

Migration from Serverless​

Step 1: Export Data​

Step 2: Deploy Domain​

Step 3: Import Data​

Step 4: Update Application​

Step 5: Destroy Serverless​

SSM Parameters​

Related Documentation​

Module Source​