terraform-aws-s3-backend/README.md

# Terraform AWS S3 Backend Module

## Overview

This Terraform module creates a secure S3 backend for Terraform state management with DynamoDB state locking. It provides encrypted state storage, version control, and concurrent access protection using AWS best practices.

## Features

- S3 bucket with versioning enabled
- KMS encryption for state files
- DynamoDB table for state locking
- IAM role for secure access
- Public access blocking
- Resource grouping for easy management
- Configurable force destroy option
- Least privilege IAM policies

## Resources Created

### Storage
- S3 Bucket with:
  - Versioning enabled
  - KMS encryption
  - Public access blocked
  - Unique naming with random suffix

### State Locking
- DynamoDB Table with:
  - Pay-per-request billing
  - LockID hash key

### Security
- KMS Key for encryption
- IAM Role for state access
- IAM Policy with minimal permissions
- Resource Group for organization

## Usage

### Basic Example

```hcl
module "terraform_backend" {
  source = "git@github.com:webuildyourcloud/terraform-aws-s3-backend.git"

  namespace           = "myorg-prod"
  force_destroy_state = false
}
```

### With Custom IAM Principals

```hcl
module "terraform_backend" {
  source = "git@github.com:webuildyourcloud/terraform-aws-s3-backend.git"

  namespace           = "myorg-dev"
  force_destroy_state = true

  principle_arns = [
    "arn:aws:iam::123456789012:user/terraform",
    "arn:aws:iam::123456789012:role/TerraformExecutionRole"
  ]
}
```

### For CI/CD Pipeline

```hcl
module "terraform_backend" {
  source = "git@github.com:webuildyourcloud/terraform-aws-s3-backend.git"

  namespace           = "cicd-terraform"
  force_destroy_state = false

  principle_arns = [
    "arn:aws:iam::123456789012:role/GitHubActionsRole",
    "arn:aws:iam::123456789012:role/JenkinsRole"
  ]
}

# Output the backend configuration
output "terraform_backend_config" {
  value = module.terraform_backend.config
}
```

## Variables

| Name | Description | Type | Default | Required |
|------|-------------|------|---------|----------|
| namespace | Project namespace for unique resource naming | `string` | `"s3backend"` | no |
| principle_arns | List of principal ARNs allowed to assume the IAM role | `list(string)` | `null` | no |
| force_destroy_state | Force destroy the S3 bucket containing state files | `bool` | `true` | no |

## Outputs

| Name | Description |
|------|-------------|
| config | Complete backend configuration object containing bucket, region, role_arn, and dynamodb_table |

The `config` output provides:
```hcl
{
  bucket         = "namespace-randomstring-state-bucket"
  region         = "current-region"
  role_arn       = "arn:aws:iam::account-id:role/namespace-randomstring-tf-assume-role"
  dynamodb_table = "namespace-randomstring-state-lock"
}
```

## Requirements

| Name | Version |
|------|---------|
| terraform | >= 0.15 |
| aws | ~> 3.28 |
| random | ~> 3.0 |

## Configuring Terraform Backend

After creating the backend, configure Terraform to use it:

### Step 1: Create the Backend Resources

```bash
terraform init
terraform apply
```

### Step 2: Note the Outputs

```bash
terraform output config
```

### Step 3: Configure Your Terraform Backend

Create or update `backend.tf`:

```hcl
terraform {
  backend "s3" {
    bucket         = "myorg-prod-xyz123-state-bucket"
    key            = "path/to/statefile.tfstate"
    region         = "us-east-1"
    dynamodb_table = "myorg-prod-xyz123-state-lock"
    encrypt        = true
    role_arn       = "arn:aws:iam::123456789012:role/myorg-prod-xyz123-tf-assume-role"
  }
}
```

### Step 4: Migrate Existing State (if applicable)

```bash
terraform init -migrate-state
```

## IAM Permissions

The module creates an IAM role with the following permissions:

### S3 Permissions
- `s3:ListBucket` - List bucket contents
- `s3:GetObject` - Read state files
- `s3:PutObject` - Write state files
- `s3:DeleteObject` - Delete old state versions

### DynamoDB Permissions
- `dynamodb:GetItem` - Read lock status
- `dynamodb:PutItem` - Acquire lock
- `dynamodb:DeleteItem` - Release lock

## Security Features

### Encryption at Rest
All state files are encrypted using AWS KMS with a dedicated encryption key.

### Encryption in Transit
All S3 API calls use HTTPS (TLS/SSL).

### Public Access Blocked
The module explicitly blocks:
- Public ACLs
- Public bucket policies
- Public object access
- Public bucket access

### Versioning
S3 bucket versioning is enabled to:
- Protect against accidental deletion
- Allow state recovery
- Maintain state history

### State Locking
DynamoDB table prevents:
- Concurrent state modifications
- State corruption
- Race conditions

## Important Notes

1. **Unique Naming**: The module generates unique bucket names using random strings
2. **Bootstrapping**: This module must be deployed without a backend initially
3. **State Migration**: After creation, migrate your state to the new backend
4. **Force Destroy**: Set `force_destroy_state = false` for production
5. **IAM Principals**: If not specified, defaults to the current caller's ARN
6. **KMS Costs**: KMS key incurs monthly charges
7. **Region Locked**: Backend resources are created in the current region

## Best Practices

1. **Separate Backends**: Use different backends for different environments
2. **Least Privilege**: Only grant access to necessary IAM principals
3. **State File Paths**: Use descriptive key paths (e.g., `env/prod/vpc/terraform.tfstate`)
4. **Backup**: Enable S3 Cross-Region Replication for critical state files
5. **Monitoring**: Set up CloudWatch alarms for bucket access
6. **Lifecycle Policies**: Configure S3 lifecycle policies for old versions
7. **Resource Groups**: Use the created resource group for cost tracking
8. **Documentation**: Document backend configuration for team members

## Example: Complete Setup

```hcl
# Step 1: Create backend infrastructure
module "backend" {
  source = "git@github.com:webuildyourcloud/terraform-aws-s3-backend.git"

  namespace           = "myapp-prod"
  force_destroy_state = false

  principle_arns = [
    "arn:aws:iam::123456789012:role/TerraformRole"
  ]
}

# Step 2: Output configuration for easy reference
output "backend_bucket" {
  value       = module.backend.config.bucket
  description = "S3 bucket name for Terraform state"
}

output "backend_dynamodb_table" {
  value       = module.backend.config.dynamodb_table
  description = "DynamoDB table name for state locking"
}

output "backend_role_arn" {
  value       = module.backend.config.role_arn
  description = "IAM role ARN for backend access"
}

output "backend_region" {
  value       = module.backend.config.region
  description = "AWS region for backend resources"
}
```

## Troubleshooting

### Cannot access state file
- Verify IAM role has correct permissions
- Check if `role_arn` is correctly specified in backend configuration
- Ensure principal is allowed in `principle_arns`

### State locking errors
- Verify DynamoDB table exists
- Check for orphaned locks in DynamoDB
- Ensure sufficient DynamoDB capacity (should not be an issue with PAY_PER_REQUEST)

### Backend initialization fails
- Verify bucket and table names are correct
- Check AWS credentials have appropriate permissions
- Ensure region matches where resources were created

### Bucket name conflicts
- The module uses random suffixes to prevent conflicts
- If conflicts occur, destroy and recreate with a new namespace

## State File Organization

Recommended key structure:

```
<environment>/<project>/<component>/terraform.tfstate

Examples:
prod/networking/vpc/terraform.tfstate
prod/compute/eks/terraform.tfstate
prod/data/rds/terraform.tfstate
staging/networking/vpc/terraform.tfstate
dev/compute/eks/terraform.tfstate
```

## Migration Guide

### From Local State

```bash
# 1. Create backend
terraform apply -target=module.backend

# 2. Add backend configuration to your code
# 3. Initialize with migration
terraform init -migrate-state

# 4. Verify
terraform state list
```

### From Another S3 Backend

Update backend configuration and run:
```bash
terraform init -migrate-state -reconfigure
```

## License

This module is provided as-is for use within your organization.