wbyc/terraform-aws-eks-cluster
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
terraform-aws-eks-cluster/README.yaml
Patrick de Ruiter 6053b9e348
first commit
2020-06-25 09:46:10 +02:00

359 lines
18 KiB
YAML
Raw Blame History

---
#
# This is the canonical configuration for the `README.md`
# Run `make readme` to rebuild the `README.md`
#
# Name of this project
name: terraform-aws-eks-cluster
# Logo for this project
#logo: docs/logo.png
# License of this project
license: "APACHE2"
# Canonical GitHub repo
github_repo: cloudposse/terraform-aws-eks-cluster
# Badges to display
badges:
- name: "Codefresh Build Status"
image: "https://g.codefresh.io/api/badges/pipeline/cloudposse/terraform-modules%2Fterraform-aws-eks-cluster?type=cf-1"
url: "https://g.codefresh.io/public/accounts/cloudposse/pipelines/5d8cd583941e46a098d3992d"
- name: "Latest Release"
image: "https://img.shields.io/github/release/cloudposse/terraform-aws-eks-cluster.svg"
url: "https://github.com/cloudposse/terraform-aws-eks-cluster/releases/latest"
- name: "Slack Community"
image: "https://slack.cloudposse.com/badge.svg"
url: "https://slack.cloudposse.com"
related:
- name: "terraform-aws-eks-workers"
description: "Terraform module to provision an AWS AutoScaling Group, IAM Role, and Security Group for EKS Workers"
url: "https://github.com/cloudposse/terraform-aws-eks-workers"
- name: "terraform-aws-ec2-autoscale-group"
description: "Terraform module to provision Auto Scaling Group and Launch Template on AWS"
url: "https://github.com/cloudposse/terraform-aws-ec2-autoscale-group"
- name: "terraform-aws-ecs-container-definition"
description: "Terraform module to generate well-formed JSON documents (container definitions) that are passed to the aws_ecs_task_definition Terraform resource"
url: "https://github.com/cloudposse/terraform-aws-ecs-container-definition"
- name: "terraform-aws-ecs-alb-service-task"
description: "Terraform module which implements an ECS service which exposes a web service via ALB"
url: "https://github.com/cloudposse/terraform-aws-ecs-alb-service-task"
- name: "terraform-aws-ecs-web-app"
description: "Terraform module that implements a web app on ECS and supports autoscaling, CI/CD, monitoring, ALB integration, and much more"
url: "https://github.com/cloudposse/terraform-aws-ecs-web-app"
- name: "terraform-aws-ecs-codepipeline"
description: "Terraform module for CI/CD with AWS Code Pipeline and Code Build for ECS"
url: "https://github.com/cloudposse/terraform-aws-ecs-codepipeline"
- name: "terraform-aws-ecs-cloudwatch-autoscaling"
description: "Terraform module to autoscale ECS Service based on CloudWatch metrics"
url: "https://github.com/cloudposse/terraform-aws-ecs-cloudwatch-autoscaling"
- name: "terraform-aws-ecs-cloudwatch-sns-alarms"
description: "Terraform module to create CloudWatch Alarms on ECS Service level metrics"
url: "https://github.com/cloudposse/terraform-aws-ecs-cloudwatch-sns-alarms"
- name: "terraform-aws-ec2-instance"
description: "Terraform module for providing a general purpose EC2 instance"
url: "https://github.com/cloudposse/terraform-aws-ec2-instance"
- name: "terraform-aws-ec2-instance-group"
description: "Terraform module for provisioning multiple general purpose EC2 hosts for stateful applications"
url: "https://github.com/cloudposse/terraform-aws-ec2-instance-group"
# Short description of this project
description: |-
Terraform module to provision an [EKS](https://aws.amazon.com/eks/) cluster on AWS.
introduction: |-
The module provisions the following resources:
- EKS cluster of master nodes that can be used together with the [terraform-aws-eks-workers](https://github.com/cloudposse/terraform-aws-eks-workers) module to create a full-blown cluster
- IAM Role to allow the cluster to access other AWS services
- Security Group which is used by EKS workers to connect to the cluster and kubelets and pods to receive communication from the cluster control plane (see [terraform-aws-eks-workers](https://github.com/cloudposse/terraform-aws-eks-workers))
- The module creates and automatically applies (via `kubectl apply`) an authentication ConfigMap to allow the wrokers nodes to join the cluster and to add additional users/roles/accounts
### Works with [Terraform Cloud](https://www.terraform.io/docs/cloud/index.html)
To run on Terraform Cloud, set the following variables:
```hcl
install_aws_cli = true
install_kubectl = true
external_packages_install_path = "~/.terraform/bin"
kubeconfig_path = "~/.kube/config"
configmap_auth_file = "/home/terraform/.terraform/configmap-auth.yaml"
# Optional
aws_eks_update_kubeconfig_additional_arguments = "--verbose"
aws_cli_assume_role_arn = "arn:aws:iam::xxxxxxxxxxx:role/OrganizationAccountAccessRole"
aws_cli_assume_role_session_name = "eks_cluster_example_session"
```
Terraform Cloud executes `terraform plan/apply` on workers running Ubuntu.
For the module to provision the authentication ConfigMap (to allow the EKS worker nodes to join the EKS cluster and to add additional users/roles/accounts),
AWS CLI and `kubectl` need to be installed on Terraform Cloud workers.
To install the required external packages, set the variables `install_aws_cli` and `install_kubectl` to `true` and specify `external_packages_install_path`, `kubeconfig_path` and `configmap_auth_file`.
See [auth.tf](auth.tf) and [Installing Software in the Run Environment](https://www.terraform.io/docs/cloud/run/install-software.html) for more details.
In a multi-account architecture, we might have a separate identity account where we provision all IAM users, and other accounts (e.g. `prod`, `staging`, `dev`, `audit`, `testing`)
where all other AWS resources are provisioned. The IAM Users from the identity account can assume IAM roles to access the other accounts.
In this case, we provide Terraform Cloud with access keys (`AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`) for an IAM User from the identity account
and allow it to assume an IAM Role into the AWS account where the module gets provisioned.
To support this, the module can assume an IAM role before executing the command `aws eks update-kubeconfig` when applying the auth ConfigMap.
Set variable `aws_cli_assume_role_arn` to the Amazon Resource Name (ARN) of the role to assume and variable `aws_cli_assume_role_session_name` to the identifier for the assumed role session.
See [auth.tf](auth.tf) and [assume-role](https://docs.aws.amazon.com/cli/latest/reference/sts/assume-role.html) for more details.
# How to use this project
usage: |-
For a complete example, see [examples/complete](examples/complete).
For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](test).
Other examples:
- [terraform-root-modules/eks](https://github.com/cloudposse/terraform-root-modules/tree/master/aws/eks) - Cloud Posse's service catalog of "root module" invocations for provisioning reference architectures
- [terraform-root-modules/eks-backing-services-peering](https://github.com/cloudposse/terraform-root-modules/tree/master/aws/eks-backing-services-peering) - example of VPC peering between the EKS VPC and backing services VPC
```hcl
provider "aws" {
region = var.region
}
module "label" {
source = "git::https://github.com/cloudposse/terraform-null-label.git?ref=master"
namespace = var.namespace
name = var.name
stage = var.stage
delimiter = var.delimiter
attributes = compact(concat(var.attributes, list("cluster")))
tags = var.tags
}
locals {
# The usage of the specific kubernetes.io/cluster/* resource tags below are required
# for EKS and Kubernetes to discover and manage networking resources
# https://www.terraform.io/docs/providers/aws/guides/eks-getting-started.html#base-vpc-networking
tags = merge(var.tags, map("kubernetes.io/cluster/${module.label.id}", "shared"))
# Unfortunately, most_recent (https://github.com/cloudposse/terraform-aws-eks-workers/blob/34a43c25624a6efb3ba5d2770a601d7cb3c0d391/main.tf#L141)
# variable does not work as expected, if you are not going to use custom AMI you should
# enforce usage of eks_worker_ami_name_filter variable to set the right kubernetes version for EKS workers,
# otherwise the first version of Kubernetes supported by AWS (v1.11) for EKS workers will be used, but
# EKS control plane will use the version specified by kubernetes_version variable.
eks_worker_ami_name_filter = "amazon-eks-node-${var.kubernetes_version}*"
}
module "vpc" {
source = "git::https://github.com/cloudposse/terraform-aws-vpc.git?ref=master"
namespace = var.namespace
stage = var.stage
name = var.name
attributes = var.attributes
cidr_block = "172.16.0.0/16"
tags = local.tags
}
module "subnets" {
source = "git::https://github.com/cloudposse/terraform-aws-dynamic-subnets.git?ref=master"
availability_zones = var.availability_zones
namespace = var.namespace
stage = var.stage
name = var.name
attributes = var.attributes
vpc_id = module.vpc.vpc_id
igw_id = module.vpc.igw_id
cidr_block = module.vpc.vpc_cidr_block
nat_gateway_enabled = false
nat_instance_enabled = false
tags = local.tags
}
module "eks_workers" {
source = "git::https://github.com/cloudposse/terraform-aws-eks-workers.git?ref=master"
namespace = var.namespace
stage = var.stage
name = var.name
attributes = var.attributes
tags = var.tags
instance_type = var.instance_type
eks_worker_ami_name_filter = local.eks_worker_ami_name_filter
vpc_id = module.vpc.vpc_id
subnet_ids = module.subnets.public_subnet_ids
health_check_type = var.health_check_type
min_size = var.min_size
max_size = var.max_size
wait_for_capacity_timeout = var.wait_for_capacity_timeout
cluster_name = module.label.id
cluster_endpoint = module.eks_cluster.eks_cluster_endpoint
cluster_certificate_authority_data = module.eks_cluster.eks_cluster_certificate_authority_data
cluster_security_group_id = module.eks_cluster.security_group_id
# Auto-scaling policies and CloudWatch metric alarms
autoscaling_policies_enabled = var.autoscaling_policies_enabled
cpu_utilization_high_threshold_percent = var.cpu_utilization_high_threshold_percent
cpu_utilization_low_threshold_percent = var.cpu_utilization_low_threshold_percent
}
module "eks_cluster" {
source = "git::https://github.com/cloudposse/terraform-aws-eks-cluster.git?ref=master"
namespace = var.namespace
stage = var.stage
name = var.name
attributes = var.attributes
tags = var.tags
vpc_id = module.vpc.vpc_id
subnet_ids = module.subnets.public_subnet_ids
kubernetes_version = var.kubernetes_version
kubeconfig_path = var.kubeconfig_path
oidc_provider_enabled = false
workers_security_group_ids = [module.eks_workers.security_group_id]
workers_role_arns = [module.eks_workers.workers_role_arn]
}
```
Module usage with two worker groups:
```hcl
module "eks_workers" {
source = "git::https://github.com/cloudposse/terraform-aws-eks-workers.git?ref=master"
namespace = var.namespace
stage = var.stage
name = "small"
attributes = var.attributes
tags = var.tags
instance_type = "t3.small"
vpc_id = module.vpc.vpc_id
subnet_ids = module.subnets.public_subnet_ids
health_check_type = var.health_check_type
min_size = var.min_size
max_size = var.max_size
wait_for_capacity_timeout = var.wait_for_capacity_timeout
cluster_name = module.label.id
cluster_endpoint = module.eks_cluster.eks_cluster_endpoint
cluster_certificate_authority_data = module.eks_cluster.eks_cluster_certificate_authority_data
cluster_security_group_id = module.eks_cluster.security_group_id
# Auto-scaling policies and CloudWatch metric alarms
autoscaling_policies_enabled = var.autoscaling_policies_enabled
cpu_utilization_high_threshold_percent = var.cpu_utilization_high_threshold_percent
cpu_utilization_low_threshold_percent = var.cpu_utilization_low_threshold_percent
}
module "eks_workers_2" {
source = "git::https://github.com/cloudposse/terraform-aws-eks-workers.git?ref=master"
namespace = var.namespace
stage = var.stage
name = "medium"
attributes = var.attributes
tags = var.tags
instance_type = "t3.medium"
vpc_id = module.vpc.vpc_id
subnet_ids = module.subnets.public_subnet_ids
health_check_type = var.health_check_type
min_size = var.min_size
max_size = var.max_size
wait_for_capacity_timeout = var.wait_for_capacity_timeout
cluster_name = module.label.id
cluster_endpoint = module.eks_cluster.eks_cluster_endpoint
cluster_certificate_authority_data = module.eks_cluster.eks_cluster_certificate_authority_data
cluster_security_group_id = module.eks_cluster.security_group_id
# Auto-scaling policies and CloudWatch metric alarms
autoscaling_policies_enabled = var.autoscaling_policies_enabled
cpu_utilization_high_threshold_percent = var.cpu_utilization_high_threshold_percent
cpu_utilization_low_threshold_percent = var.cpu_utilization_low_threshold_percent
}
module "eks_cluster" {
source = "git::https://github.com/cloudposse/terraform-aws-eks-cluster.git?ref=master"
namespace = var.namespace
stage = var.stage
name = var.name
attributes = var.attributes
tags = var.tags
vpc_id = module.vpc.vpc_id
subnet_ids = module.subnets.public_subnet_ids
kubernetes_version = var.kubernetes_version
kubeconfig_path = var.kubeconfig_path
oidc_provider_enabled = false
workers_role_arns = [module.eks_workers.workers_role_arn, module.eks_workers_2.workers_role_arn]
workers_security_group_ids = [module.eks_workers.security_group_id, module.eks_workers_2.security_group_id]
}
```
Module usage on [Terraform Cloud](https://www.terraform.io/docs/cloud/index.html):
```hcl
provider "aws" {
region = "us-east-2"
assume_role {
role_arn = "arn:aws:iam::xxxxxxxxxxx:role/OrganizationAccountAccessRole"
}
}
module "eks_cluster" {
source = "git::https://github.com/cloudposse/terraform-aws-eks-cluster.git?ref=master"
namespace = var.namespace
stage = var.stage
name = var.name
attributes = var.attributes
tags = var.tags
region = "us-east-2"
vpc_id = module.vpc.vpc_id
subnet_ids = module.subnets.public_subnet_ids
local_exec_interpreter = "/bin/bash"
kubernetes_version = "1.14"
workers_role_arns = [module.eks_workers.workers_role_arn]
workers_security_group_ids = [module.eks_workers.security_group_id]
# Terraform Cloud configurations
kubeconfig_path = "~/.kube/config"
configmap_auth_file = "/home/terraform/.terraform/configmap-auth.yaml"
install_aws_cli = true
install_kubectl = true
external_packages_install_path = "~/.terraform/bin"
aws_eks_update_kubeconfig_additional_arguments = "--verbose"
aws_cli_assume_role_arn = "arn:aws:iam::xxxxxxxxxxx:role/OrganizationAccountAccessRole"
aws_cli_assume_role_session_name = "eks_cluster_example_session"
}
```
include:
- "docs/targets.md"
- "docs/terraform.md"
# Contributors to this project
contributors:
- name: "Erik Osterman"
homepage: "https://github.com/osterman"
avatar: "http://s.gravatar.com/avatar/88c480d4f73b813904e00a5695a454cb?s=144"
github: "osterman"
- name: "Andriy Knysh"
homepage: "https://github.com/aknysh/"
avatar: "https://avatars0.githubusercontent.com/u/7356997?v=4&u=ed9ce1c9151d552d985bdf5546772e14ef7ab617&s=144"
github: "aknysh"
- name: "Igor Rodionov"
homepage: "https://github.com/goruha/"
avatar: "http://s.gravatar.com/avatar/bc70834d32ed4517568a1feb0b9be7e2?s=144"
github: "goruha"
- name: "Oscar"
homepage: "https://github.com/osulli/"
avatar: "https://avatars1.githubusercontent.com/u/46930728?v=4&s=144"
github: "osulli"
Reference in New Issue View Git Blame Copy Permalink