ChatOps Modules
These modules enable Slack-based infrastructure management, providing a secure and auditable way to perform operations without direct AWS console access.
Overview
DocuStack's ChatOps system consists of:
Slack Workspace
│
│ /infra, /bastion, /whitelist commands
v
┌──────────────────┐
│ Slack Bot │ (ECS Fargate, Socket Mode)
│ - Routes commands
│ - Access control
└────────┬─────────┘
│
┌────┴────┬────────────┐
v v v
┌───────┐ ┌───────┐ ┌──────────┐
│ Infra │ │Bastion│ │ IP │
│Orch. │ │ Orch. │ │Whitelist │
└───────┘ └───────┘ └──────────┘
Slack Bot
ECS Fargate-based Slack bot using Socket Mode for persistent WebSocket connections.
Why This Module?
Traditional Slack bots require public endpoints, which:
- Expose attack surface
- Require SSL certificates
- Need load balancers
Socket Mode solves this:
- No public endpoints - Outbound WebSocket only
- Firewall-friendly - Works in private subnets
- Simpler architecture - No ALB required
Architecture
┌─────────────────────────────────────────────────────────────┐
│ Slack Bot Architecture │
├─────────────────────────────────────────────────────────────┤
│ Slack Workspace │
│ │ │
│ │ WebSocket (Socket Mode) │
│ v │
│ ┌──────────────────────┐ │
│ │ ECS Fargate Task │ │
│ │ (Private subnet) │ │
│ │ │ │
│ │ Slack Bot Container │ │
│ │ - Socket Mode │ │
│ │ - Health endpoint │ │
│ └──────────┬───────────┘ │
│ │ │
│ │ invoke Lambda │
│ v │
│ ┌──────────────────────┐ ┌──────────────────┐ │
│ │ Infra Orchestrator │ │ Bastion │ │
│ │ Lambda │ │ Orchestrator │ │
│ └──────────────────────┘ └──────────────────┘ │
│ │
│ │ ┌──────────────────┐ │
│ └─────────────────────►│ IP Whitelist │ │
│ │ Manager │ │
│ └──────────────────┘ │
└─────────────────────────────────────────────────────────────┘
Slack App Setup
1. Create Slack App
- Go to https://api.slack.com/apps
- Click "Create New App" → "From scratch"
- Name: "Infrastructure Bot"
- Select your workspace
2. Enable Socket Mode
- Go to "Socket Mode" in sidebar
- Enable Socket Mode
- Generate app-level token with
connections:writescope - Save token (starts with
xapp-)
3. Configure Bot Permissions
- Go to "OAuth & Permissions"
- Add Bot Token Scopes:
chat:write- Send messagescommands- Respond to slash commandsapp_mentions:read- Read mentions
- Install app to workspace
- Save Bot Token (starts with
xoxb-)
4. Create Slash Commands
Go to "Slash Commands" and create:
/infra- Infrastructure operations/bastion- Bastion management/whitelist- IP whitelist management
5. Store Tokens
aws secretsmanager create-secret \
--name slack/bot-tokens \
--secret-string '{
"bot_token": "xoxb-your-bot-token",
"app_token": "xapp-your-app-token"
}'
Usage
module "slack_bot" {
source = "git::git@github.com:docustackapp/docustack-infrastructure-modules.git//modules/slack-bot?ref=v1.0.0"
name = "docustack-dev"
environment = "dev"
# Lambda integrations
orchestrator_lambda_arn = module.infra_orchestrator.lambda_arn
ip_whitelist_lambda_arn = module.ip_whitelist.lambda_arn
bastion_orchestrator_lambda_arn = module.bastion_orchestrator.orchestrator_lambda_arn
# Slack configuration
slack_tokens_secret_arn = aws_secretsmanager_secret.slack_tokens.arn
# Network
vpc_id = module.vpc.vpc_id
subnet_ids = module.vpc.private_subnet_ids
# ECS
ecs_cluster_arn = module.ecs_cluster.cluster_arn
container_image = "${module.ecr.repository_url}:latest"
# Resources
task_cpu = 256
task_memory = 512
# Access control
allowed_slack_channels = ["C01234567"] # Restrict to specific channels
allowed_users = ["U01234567"] # Restrict to specific users
use_fargate_spot = true
}
Available Commands
Infrastructure Commands
/infra stop dev # Stop ECS/RDS (Tier 1)
/infra start dev # Start ECS/RDS (Tier 1)
/infra teardown dev # Full teardown (Tier 2)
/infra spinup dev # Full spinup (Tier 2)
/infra status dev # Check resource status
Bastion Commands
/bastion create dev # Create bastion (3 hour TTL)
/bastion status dev # Check bastion status
/bastion extend dev # Extend by 3 hours
/bastion destroy dev # Terminate bastion
IP Whitelist Commands
/whitelist add 1.2.3.4 "John office IP" # Add IP
/whitelist remove 1.2.3.4 # Remove IP
/whitelist list dev # List all IPs
Cost
| Configuration | Monthly Cost |
|---|---|
| Fargate (256/512) | ~$10.50 |
| Fargate Spot (256/512) | ~$3.20 |
| CloudWatch Logs | ~$0.50 |
| Secrets Manager | ~$0.40 |
| Total (Spot) | ~$4.10 |
IP Whitelist
DynamoDB-backed IP whitelist management with automatic security group synchronization.
Why This Module?
Managing IP whitelists manually is error-prone:
- Forgetting to remove old IPs
- Inconsistent security group rules
- No audit trail
This module provides:
- Centralized storage in DynamoDB
- Automatic TTL - IPs expire automatically
- Real-time sync - Security groups updated via DynamoDB Streams
- Full audit trail - All changes logged
Architecture
┌─────────────────────────────────────────────────────────────┐
│ IP Whitelist Architecture │
├─────────────────────────────────────────────────────────────┤
│ Slack Bot / API │
│ │ │
│ │ /whitelist add 1.2.3.4 │
│ v │
│ ┌──────────────────────┐ │
│ │ DynamoDB Table │ │
│ │ (IP Whitelist) │ │
│ │ │ │
│ │ PK: ip_address │ │
│ │ SK: environment │ │
│ │ TTL: expires_at │ │
│ └──────────┬───────────┘ │
│ │ │
│ │ DynamoDB Stream (INSERT/REMOVE) │
│ v │
│ ┌──────────────────────┐ │
│ │ Lambda Function │ │
│ │ (Stream Handler) │ │
│ │ │ │
│ │ 1. Detect change │ │
│ │ 2. Find SGs │ │
│ │ 3. Update rules │ │
│ │ 4. Send notification│ │
│ └──────────┬───────────┘ │
│ │ │
│ │ authorize/revoke ingress │
│ v │
│ ┌──────────────────────┐ │
│ │ Security Groups │ │
│ │ (Tagged) │ │
│ │ │ │
│ │ Tag: │ │
│ │ WhitelistManaged=true│ │
│ └──────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
Usage
module "ip_whitelist" {
source = "git::git@github.com:docustackapp/docustack-infrastructure-modules.git//modules/ip-whitelist?ref=v1.0.0"
name = "docustack-dev"
environment = "dev"
vpc_id = module.vpc.vpc_id
private_subnet_ids = module.vpc.private_subnet_ids
# Tag-based discovery
enable_sg_discovery = true
sg_discovery_tag_key = "WhitelistManaged"
sg_discovery_tag_value = "true"
lambda_source_dir = "${path.root}/../../../docustack-mono/services/lambdas/ip-whitelist-manager"
kms_key_arn = aws_kms_key.cloudwatch.arn
default_ttl_days = 7
}
Tag security groups for automatic management:
resource "aws_ec2_tag" "alb_whitelist" {
resource_id = module.alb.security_group_id
key = "WhitelistManaged"
value = "true"
}
DynamoDB Schema
Table: docustack-dev-ip-whitelist
Primary Key:
- ip_address (String, Hash Key)
- environment (String, Range Key)
Attributes:
- description (String)
- added_by (String)
- added_at (Number, Unix timestamp)
- expires_at (Number, Unix timestamp, TTL)
TTL: expires_at (automatic deletion)
Streams: NEW_AND_OLD_IMAGES
Manual Operations
# Add IP via AWS CLI
aws dynamodb put-item \
--table-name docustack-dev-ip-whitelist \
--item '{
"ip_address": {"S": "1.2.3.4/32"},
"environment": {"S": "dev"},
"description": {"S": "John office IP"},
"added_by": {"S": "john.doe"},
"added_at": {"N": "1704067200"},
"expires_at": {"N": "1704672000"}
}'
# List IPs
aws dynamodb query \
--table-name docustack-dev-ip-whitelist \
--index-name environment-index \
--key-condition-expression "environment = :env" \
--expression-attribute-values '{":env":{"S":"dev"}}'
# Remove IP
aws dynamodb delete-item \
--table-name docustack-dev-ip-whitelist \
--key '{"ip_address":{"S":"1.2.3.4/32"},"environment":{"S":"dev"}}'
Automatic Sync Flow
- IP Added → DynamoDB Stream event (INSERT)
- Lambda Triggered → Processes stream event
- Find Security Groups → Via tags or explicit list
- Add Ingress Rule →
authorize-security-group-ingress - Send Notification → Slack webhook
When TTL expires:
- DynamoDB deletes item → Stream event (REMOVE)
- Lambda removes rule →
revoke-security-group-ingress - Notification sent → IP expired
Lambda Code Location
Source: docustack-mono/services/lambdas/ip-whitelist-manager/
Cost
| Component | Monthly Cost |
|---|---|
| DynamoDB (on-demand) | < $0.10 |
| Lambda | < $0.01 |
| Total | < $0.11 |
Security Best Practices
Slack Bot
- Use private subnets - Never deploy in public subnets
- Restrict access - Use
allowed_slack_channelsandallowed_users - Rotate tokens - Regularly rotate Slack tokens
- Enable Container Insights - Monitor for anomalies
- Review logs - Audit command usage regularly
IP Whitelist
- Use CIDR notation - Always specify /32 for single IPs
- Set appropriate TTLs - Don't leave IPs whitelisted indefinitely
- Tag security groups - Use consistent tagging for discovery
- Monitor additions - Review CloudWatch logs regularly
- Enable Slack notifications - Stay informed of changes
Troubleshooting
Bot Not Responding
# Check ECS service status
aws ecs describe-services \
--cluster docustack-dev \
--services docustack-dev-slack-bot \
--query 'services[0].{Status:status,Running:runningCount}'
# Check container logs
aws logs tail /ecs/docustack-dev-slack-bot --since 10m
Common Causes:
- ECS task not running
- Invalid Slack tokens
- Network connectivity issues
Socket Mode Connection Errors
# Check logs for connection errors
aws logs filter-log-events \
--log-group-name /ecs/docustack-dev-slack-bot \
--filter-pattern "socket mode"
Solutions:
- Verify Socket Mode is enabled in Slack app
- Check app token has
connections:writescope - Ensure ECS task has internet access (NAT Gateway)
IP Whitelist Not Syncing
# Check stream status
aws dynamodb describe-table \
--table-name docustack-dev-ip-whitelist \
--query 'Table.StreamSpecification'
# Check event source mapping
aws lambda list-event-source-mappings \
--function-name docustack-dev-ip-whitelist-manager
# Check Lambda logs
aws logs tail /aws/lambda/docustack-dev-ip-whitelist-manager --since 10m
Common Causes:
- DynamoDB Streams not enabled
- Event source mapping disabled
- Lambda missing EC2 permissions