Bastion Orchestrator

Lambda function for managing on-demand bastion instances with auto-termination.

Why This Exists

Traditional bastion hosts have problems:

• Security risk: Long-lived instances with SSH keys
• Cost waste: Running 24/7 even when unused
• No audit trail: Hard to track who accessed what
• Key management: SSH keys need rotation and distribution

The bastion orchestrator solves these by providing on-demand, SSM-only bastions that:

• Launch in seconds via Slack
• Auto-terminate after 3 hours
• Use IAM for authentication (no SSH keys)
• Log all sessions to CloudWatch

Cost per session: ~$0.001 (3 hours on t4g.nano)

How It Works

Slack Bot → bastion-orchestrator → EC2 + EventBridge
                                 ↓
                          auto_terminate Lambda
                                 ↓
                          Slack Notification

1. User runs /infra bastion create in Slack
2. Lambda launches t4g.nano instance in private subnet
3. EventBridge schedules warning (2.5h) and termination (3h)
4. User connects via SSM Session Manager
5. Instance auto-terminates, user gets Slack notification

Actions

Create

Launch a new bastion instance with user attribution and auto-termination.

{
  "action": "create",
  "user_id": "U12345",
  "username": "john.doe",
  "display_name": "John Doe",
  "vpc_id": "vpc-xxx",
  "subnet_id": "subnet-xxx",
  "rds_security_group_id": "sg-xxx",
  "rds_endpoint": "xxx.rds.amazonaws.com"
}

Response:

{
  "success": true,
  "instance_id": "i-xxx",
  "private_ip": "10.0.1.100",
  "created_at": "2025-12-18T12:00:00+00:00",
  "expires_at": "2025-12-18T15:00:00+00:00",
  "connection_instructions": "aws ssm start-session..."
}

Status

Get current status and metadata of a bastion instance.

{
  "action": "status",
  "instance_id": "i-xxx"
}

Extend

Extend bastion lifetime by 2 hours.

{
  "action": "extend",
  "instance_id": "i-xxx"
}

Destroy

Terminate an existing bastion instance and clean up EventBridge rules.

{
  "action": "destroy",
  "instance_id": "i-xxx"
}

Connecting to the Bastion

Users receive SSM connection instructions in Slack:

Terminal Session

aws ssm start-session --target i-INSTANCE_ID

Port Forward to RDS

# Start port forwarding
aws ssm start-session --target i-INSTANCE_ID \
  --document-name AWS-StartPortForwardingSessionToRemoteHost \
  --parameters '{"host":["RDS_ENDPOINT"],"portNumber":["5432"],"localPortNumber":["5432"]}'

# In another terminal, connect to database
psql -h localhost -p 5432 -U postgres -d temporal

Auto-Termination

The auto_terminate.py handler is triggered by EventBridge rules:

Event	Timing	Action
Warning	30 min before expiry	Slack notification to user
Termination	At expiry (3 hours)	Terminate instance, notify user

Users can extend the lifetime by 2 hours via Slack button or /infra bastion extend.

Security Features

Feature	Benefit
No SSH access	SSM Session Manager only - no key management
No public IP	Private subnet deployment
Encrypted volumes	EBS encryption enabled
IMDSv2 only	Metadata service v2 required
Audit logging	All SSM sessions logged to CloudWatch
User attribution	EC2 tags track who requested the bastion
Auto-termination	Prevents forgotten instances

HIPAA Compliance

• All sessions logged to CloudWatch (90-day retention)
• Encrypted EBS volumes
• No long-lived credentials (SSM uses IAM roles)
• Security group least privilege
• User attribution for audit trail

Environment Variables

Variable	Description	Required
PROJECT_NAME	Project name for resource naming	Yes
ENVIRONMENT	Environment (dev/staging/prod)	Yes
INSTANCE_TYPE	EC2 instance type	No (default: t4g.nano)
AMI_ID	Amazon Linux 2023 ARM64 AMI ID	Yes
IAM_INSTANCE_PROFILE	IAM instance profile name	Yes
BASTION_LIFETIME_HOURS	Hours before auto-termination	No (default: 3)
AUTO_TERMINATE_LAMBDA_ARN	ARN of auto-terminate Lambda	Yes
WARNING_LAMBDA_ARN	ARN of warning Lambda	Yes
SLACK_NOTIFIER_LAMBDA_ARN	ARN of Slack notifier Lambda	Yes

Deployment

Deployed via Terraform module:

module "bastion" {
  source = "../../modules/bastion"

  project_name = "docustack"
  environment  = "dev"
  
  vpc_id     = module.vpc.vpc_id
  subnet_ids = module.vpc.private_subnet_ids
  
  rds_security_group_id = module.rds.security_group_id
  rds_endpoint          = module.rds.endpoint
}

Development Workflow

Local Testing

cd docustack-mono/services/lambdas/bastion-orchestrator

# Install dependencies
pip install boto3

# Set environment variables
export PROJECT_NAME=docustack
export ENVIRONMENT=dev
export AMI_ID=ami-xxx
export IAM_INSTANCE_PROFILE=bastion-profile
export LOG_LEVEL=DEBUG

# Test locally (requires AWS credentials)
python -c "
from handler import lambda_handler
result = lambda_handler({
    'action': 'status',
    'instance_id': 'i-xxx'
}, None)
print(result)
"

Testing in AWS

# Create bastion
aws lambda invoke \
  --function-name docustack-dev-bastion-orchestrator \
  --payload '{"action":"create","user_id":"U123","username":"test"}' \
  /tmp/response.json

cat /tmp/response.json

# Check logs
aws logs tail /aws/lambda/docustack-dev-bastion-orchestrator --since 10m

Cost Optimization

Factor	Value
Instance type	t4g.nano (ARM-based, cheapest)
Monthly cost if 24/7	~$3/month
Cost per 3-hour session	~$0.001
Auto-termination	Prevents forgotten instances

Troubleshooting

Bastion not launching

1. Check CloudWatch logs for errors
2. Verify AMI ID is valid for the region
3. Check IAM instance profile exists
4. Verify subnet has route to NAT Gateway (for SSM)

Cannot connect via SSM

1. Verify instance is running: aws ec2 describe-instances --instance-ids i-xxx
2. Check SSM agent is running (should auto-start on Amazon Linux 2023)
3. Verify IAM role has SSM permissions
4. Check VPC endpoints or NAT Gateway for SSM connectivity

Auto-termination not working

1. Check EventBridge rules were created
2. Verify auto-terminate Lambda has EC2 permissions
3. Review CloudWatch logs for the auto-terminate function

Code Location

docustack-mono/services/lambdas/bastion-orchestrator/
├── handler.py           # Main orchestrator logic
├── auto_terminate.py    # Auto-termination handler
└── README.md