terraform-aws-s3-backend/README.md

Terraform AWS S3 Backend Module

Overview

This Terraform module creates a secure S3 backend for Terraform state management with DynamoDB state locking. It provides encrypted state storage, version control, and concurrent access protection using AWS best practices.

Features

• S3 bucket with versioning enabled
• KMS encryption for state files
• DynamoDB table for state locking
• IAM role for secure access
• Public access blocking
• Resource grouping for easy management
• Configurable force destroy option
• Least privilege IAM policies

Resources Created

Storage

• S3 Bucket with:
  • Versioning enabled
  • KMS encryption
  • Public access blocked
  • Unique naming with random suffix

State Locking

• DynamoDB Table with:
  • Pay-per-request billing
  • LockID hash key

Security

• KMS Key for encryption
• IAM Role for state access
• IAM Policy with minimal permissions
• Resource Group for organization

Usage

Basic Example

module "terraform_backend" {
  source = "git@github.com:webuildyourcloud/terraform-aws-s3-backend.git"

  namespace           = "myorg-prod"
  force_destroy_state = false
}

With Custom IAM Principals

module "terraform_backend" {
  source = "git@github.com:webuildyourcloud/terraform-aws-s3-backend.git"

  namespace           = "myorg-dev"
  force_destroy_state = true

  principle_arns = [
    "arn:aws:iam::123456789012:user/terraform",
    "arn:aws:iam::123456789012:role/TerraformExecutionRole"
  ]
}

For CI/CD Pipeline

module "terraform_backend" {
  source = "git@github.com:webuildyourcloud/terraform-aws-s3-backend.git"

  namespace           = "cicd-terraform"
  force_destroy_state = false

  principle_arns = [
    "arn:aws:iam::123456789012:role/GitHubActionsRole",
    "arn:aws:iam::123456789012:role/JenkinsRole"
  ]
}

# Output the backend configuration
output "terraform_backend_config" {
  value = module.terraform_backend.config
}

Variables

Name	Description	Type	Default	Required
namespace	Project namespace for unique resource naming	string	"s3backend"	no
principle_arns	List of principal ARNs allowed to assume the IAM role	list(string)	null	no
force_destroy_state	Force destroy the S3 bucket containing state files	bool	true	no

Outputs

Name	Description
config	Complete backend configuration object containing bucket, region, role_arn, and dynamodb_table

The config output provides:

{
  bucket         = "namespace-randomstring-state-bucket"
  region         = "current-region"
  role_arn       = "arn:aws:iam::account-id:role/namespace-randomstring-tf-assume-role"
  dynamodb_table = "namespace-randomstring-state-lock"
}

Requirements

Name	Version
terraform	>= 0.15
aws	~> 3.28
random	~> 3.0

Configuring Terraform Backend

After creating the backend, configure Terraform to use it:

Step 1: Create the Backend Resources

terraform init
terraform apply

Step 2: Note the Outputs

terraform output config

Step 3: Configure Your Terraform Backend

Create or update backend.tf:

terraform {
  backend "s3" {
    bucket         = "myorg-prod-xyz123-state-bucket"
    key            = "path/to/statefile.tfstate"
    region         = "us-east-1"
    dynamodb_table = "myorg-prod-xyz123-state-lock"
    encrypt        = true
    role_arn       = "arn:aws:iam::123456789012:role/myorg-prod-xyz123-tf-assume-role"
  }
}

Step 4: Migrate Existing State (if applicable)

terraform init -migrate-state

IAM Permissions

The module creates an IAM role with the following permissions:

S3 Permissions

• s3:ListBucket - List bucket contents
• s3:GetObject - Read state files
• s3:PutObject - Write state files
• s3:DeleteObject - Delete old state versions

DynamoDB Permissions

• dynamodb:GetItem - Read lock status
• dynamodb:PutItem - Acquire lock
• dynamodb:DeleteItem - Release lock

Security Features

Encryption at Rest

All state files are encrypted using AWS KMS with a dedicated encryption key.

Encryption in Transit

All S3 API calls use HTTPS (TLS/SSL).

Public Access Blocked

The module explicitly blocks:

• Public ACLs
• Public bucket policies
• Public object access
• Public bucket access

Versioning

S3 bucket versioning is enabled to:

• Protect against accidental deletion
• Allow state recovery
• Maintain state history

State Locking

DynamoDB table prevents:

• Concurrent state modifications
• State corruption
• Race conditions

Important Notes

1. Unique Naming: The module generates unique bucket names using random strings
2. Bootstrapping: This module must be deployed without a backend initially
3. State Migration: After creation, migrate your state to the new backend
4. Force Destroy: Set force_destroy_state = false for production
5. IAM Principals: If not specified, defaults to the current caller's ARN
6. KMS Costs: KMS key incurs monthly charges
7. Region Locked: Backend resources are created in the current region

Best Practices

1. Separate Backends: Use different backends for different environments
2. Least Privilege: Only grant access to necessary IAM principals
3. State File Paths: Use descriptive key paths (e.g., env/prod/vpc/terraform.tfstate)
4. Backup: Enable S3 Cross-Region Replication for critical state files
5. Monitoring: Set up CloudWatch alarms for bucket access
6. Lifecycle Policies: Configure S3 lifecycle policies for old versions
7. Resource Groups: Use the created resource group for cost tracking
8. Documentation: Document backend configuration for team members

Example: Complete Setup

# Step 1: Create backend infrastructure
module "backend" {
  source = "git@github.com:webuildyourcloud/terraform-aws-s3-backend.git"

  namespace           = "myapp-prod"
  force_destroy_state = false

  principle_arns = [
    "arn:aws:iam::123456789012:role/TerraformRole"
  ]
}

# Step 2: Output configuration for easy reference
output "backend_bucket" {
  value       = module.backend.config.bucket
  description = "S3 bucket name for Terraform state"
}

output "backend_dynamodb_table" {
  value       = module.backend.config.dynamodb_table
  description = "DynamoDB table name for state locking"
}

output "backend_role_arn" {
  value       = module.backend.config.role_arn
  description = "IAM role ARN for backend access"
}

output "backend_region" {
  value       = module.backend.config.region
  description = "AWS region for backend resources"
}

Troubleshooting

Cannot access state file

• Verify IAM role has correct permissions
• Check if role_arn is correctly specified in backend configuration
• Ensure principal is allowed in principle_arns

State locking errors

• Verify DynamoDB table exists
• Check for orphaned locks in DynamoDB
• Ensure sufficient DynamoDB capacity (should not be an issue with PAY_PER_REQUEST)

Backend initialization fails

• Verify bucket and table names are correct
• Check AWS credentials have appropriate permissions
• Ensure region matches where resources were created

Bucket name conflicts

• The module uses random suffixes to prevent conflicts
• If conflicts occur, destroy and recreate with a new namespace

State File Organization

Recommended key structure:

<environment>/<project>/<component>/terraform.tfstate

Examples:
prod/networking/vpc/terraform.tfstate
prod/compute/eks/terraform.tfstate
prod/data/rds/terraform.tfstate
staging/networking/vpc/terraform.tfstate
dev/compute/eks/terraform.tfstate

Migration Guide

From Local State

# 1. Create backend
terraform apply -target=module.backend

# 2. Add backend configuration to your code
# 3. Initialize with migration
terraform init -migrate-state

# 4. Verify
terraform state list

From Another S3 Backend

Update backend configuration and run:

terraform init -migrate-state -reconfigure

License

This module is provided as-is for use within your organization.