[![README Header][readme_header_img]][readme_header_link]
[![Cloud Posse][logo]](https://cpco.io/homepage)
# terraform-aws-eks-cluster [](https://g.codefresh.io/public/accounts/cloudposse/pipelines/5d8cd583941e46a098d3992d) [](https://github.com/cloudposse/terraform-aws-eks-cluster/releases/latest) [](https://slack.cloudposse.com)
Terraform module to provision an [EKS](https://aws.amazon.com/eks/) cluster on AWS.
---
This project is part of our comprehensive ["SweetOps"](https://cpco.io/sweetops) approach towards DevOps.
[
][share_email]
[
][share_googleplus]
[
][share_facebook]
[
][share_reddit]
[
][share_linkedin]
[
][share_twitter]
[][terraform_modules]
It's 100% Open Source and licensed under the [APACHE2](LICENSE).
We literally have [*hundreds of terraform modules*][terraform_modules] that are Open Source and well-maintained. Check them out!
## Introduction
The module provisions the following resources:
- EKS cluster of master nodes that can be used together with the [terraform-aws-eks-workers](https://github.com/cloudposse/terraform-aws-eks-workers) module to create a full-blown cluster
- IAM Role to allow the cluster to access other AWS services
- Security Group which is used by EKS workers to connect to the cluster and kubelets and pods to receive communication from the cluster control plane (see [terraform-aws-eks-workers](https://github.com/cloudposse/terraform-aws-eks-workers))
- The module creates and automatically applies (via `kubectl apply`) an authentication ConfigMap to allow the wrokers nodes to join the cluster and to add additional users/roles/accounts
### Works with [Terraform Cloud](https://www.terraform.io/docs/cloud/index.html)
To run on Terraform Cloud, set the following variables:
```hcl
install_aws_cli = true
install_kubectl = true
external_packages_install_path = "~/.terraform/bin"
kubeconfig_path = "~/.kube/config"
configmap_auth_file = "/home/terraform/.terraform/configmap-auth.yaml"
# Optional
aws_eks_update_kubeconfig_additional_arguments = "--verbose"
aws_cli_assume_role_arn = "arn:aws:iam::xxxxxxxxxxx:role/OrganizationAccountAccessRole"
aws_cli_assume_role_session_name = "eks_cluster_example_session"
```
Terraform Cloud executes `terraform plan/apply` on workers running Ubuntu.
For the module to provision the authentication ConfigMap (to allow the EKS worker nodes to join the EKS cluster and to add additional users/roles/accounts),
AWS CLI and `kubectl` need to be installed on Terraform Cloud workers.
To install the required external packages, set the variables `install_aws_cli` and `install_kubectl` to `true` and specify `external_packages_install_path`, `kubeconfig_path` and `configmap_auth_file`.
See [auth.tf](auth.tf) and [Installing Software in the Run Environment](https://www.terraform.io/docs/cloud/run/install-software.html) for more details.
In a multi-account architecture, we might have a separate identity account where we provision all IAM users, and other accounts (e.g. `prod`, `staging`, `dev`, `audit`, `testing`)
where all other AWS resources are provisioned. The IAM Users from the identity account can assume IAM roles to access the other accounts.
In this case, we provide Terraform Cloud with access keys (`AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`) for an IAM User from the identity account
and allow it to assume an IAM Role into the AWS account where the module gets provisioned.
To support this, the module can assume an IAM role before executing the command `aws eks update-kubeconfig` when applying the auth ConfigMap.
Set variable `aws_cli_assume_role_arn` to the Amazon Resource Name (ARN) of the role to assume and variable `aws_cli_assume_role_session_name` to the identifier for the assumed role session.
See [auth.tf](auth.tf) and [assume-role](https://docs.aws.amazon.com/cli/latest/reference/sts/assume-role.html) for more details.
## Usage
**IMPORTANT:** The `master` branch is used in `source` just as an example. In your code, do not pin to `master` because there may be breaking changes between releases.
Instead pin to the release tag (e.g. `?ref=tags/x.y.z`) of one of our [latest releases](https://github.com/cloudposse/terraform-aws-eks-cluster/releases).
For a complete example, see [examples/complete](examples/complete).
For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](test).
Other examples:
- [terraform-root-modules/eks](https://github.com/cloudposse/terraform-root-modules/tree/master/aws/eks) - Cloud Posse's service catalog of "root module" invocations for provisioning reference architectures
- [terraform-root-modules/eks-backing-services-peering](https://github.com/cloudposse/terraform-root-modules/tree/master/aws/eks-backing-services-peering) - example of VPC peering between the EKS VPC and backing services VPC
```hcl
provider "aws" {
region = var.region
}
module "label" {
source = "git::https://github.com/cloudposse/terraform-null-label.git?ref=master"
namespace = var.namespace
name = var.name
stage = var.stage
delimiter = var.delimiter
attributes = compact(concat(var.attributes, list("cluster")))
tags = var.tags
}
locals {
# The usage of the specific kubernetes.io/cluster/* resource tags below are required
# for EKS and Kubernetes to discover and manage networking resources
# https://www.terraform.io/docs/providers/aws/guides/eks-getting-started.html#base-vpc-networking
tags = merge(var.tags, map("kubernetes.io/cluster/${module.label.id}", "shared"))
# Unfortunately, most_recent (https://github.com/cloudposse/terraform-aws-eks-workers/blob/34a43c25624a6efb3ba5d2770a601d7cb3c0d391/main.tf#L141)
# variable does not work as expected, if you are not going to use custom AMI you should
# enforce usage of eks_worker_ami_name_filter variable to set the right kubernetes version for EKS workers,
# otherwise the first version of Kubernetes supported by AWS (v1.11) for EKS workers will be used, but
# EKS control plane will use the version specified by kubernetes_version variable.
eks_worker_ami_name_filter = "amazon-eks-node-${var.kubernetes_version}*"
}
module "vpc" {
source = "git::https://github.com/cloudposse/terraform-aws-vpc.git?ref=master"
namespace = var.namespace
stage = var.stage
name = var.name
attributes = var.attributes
cidr_block = "172.16.0.0/16"
tags = local.tags
}
module "subnets" {
source = "git::https://github.com/cloudposse/terraform-aws-dynamic-subnets.git?ref=master"
availability_zones = var.availability_zones
namespace = var.namespace
stage = var.stage
name = var.name
attributes = var.attributes
vpc_id = module.vpc.vpc_id
igw_id = module.vpc.igw_id
cidr_block = module.vpc.vpc_cidr_block
nat_gateway_enabled = false
nat_instance_enabled = false
tags = local.tags
}
module "eks_workers" {
source = "git::https://github.com/cloudposse/terraform-aws-eks-workers.git?ref=master"
namespace = var.namespace
stage = var.stage
name = var.name
attributes = var.attributes
tags = var.tags
instance_type = var.instance_type
eks_worker_ami_name_filter = local.eks_worker_ami_name_filter
vpc_id = module.vpc.vpc_id
subnet_ids = module.subnets.public_subnet_ids
health_check_type = var.health_check_type
min_size = var.min_size
max_size = var.max_size
wait_for_capacity_timeout = var.wait_for_capacity_timeout
cluster_name = module.label.id
cluster_endpoint = module.eks_cluster.eks_cluster_endpoint
cluster_certificate_authority_data = module.eks_cluster.eks_cluster_certificate_authority_data
cluster_security_group_id = module.eks_cluster.security_group_id
# Auto-scaling policies and CloudWatch metric alarms
autoscaling_policies_enabled = var.autoscaling_policies_enabled
cpu_utilization_high_threshold_percent = var.cpu_utilization_high_threshold_percent
cpu_utilization_low_threshold_percent = var.cpu_utilization_low_threshold_percent
}
module "eks_cluster" {
source = "git::https://github.com/cloudposse/terraform-aws-eks-cluster.git?ref=master"
namespace = var.namespace
stage = var.stage
name = var.name
attributes = var.attributes
tags = var.tags
vpc_id = module.vpc.vpc_id
subnet_ids = module.subnets.public_subnet_ids
kubernetes_version = var.kubernetes_version
kubeconfig_path = var.kubeconfig_path
oidc_provider_enabled = false
workers_security_group_ids = [module.eks_workers.security_group_id]
workers_role_arns = [module.eks_workers.workers_role_arn]
}
```
Module usage with two worker groups:
```hcl
module "eks_workers" {
source = "git::https://github.com/cloudposse/terraform-aws-eks-workers.git?ref=master"
namespace = var.namespace
stage = var.stage
name = "small"
attributes = var.attributes
tags = var.tags
instance_type = "t3.small"
vpc_id = module.vpc.vpc_id
subnet_ids = module.subnets.public_subnet_ids
health_check_type = var.health_check_type
min_size = var.min_size
max_size = var.max_size
wait_for_capacity_timeout = var.wait_for_capacity_timeout
cluster_name = module.label.id
cluster_endpoint = module.eks_cluster.eks_cluster_endpoint
cluster_certificate_authority_data = module.eks_cluster.eks_cluster_certificate_authority_data
cluster_security_group_id = module.eks_cluster.security_group_id
# Auto-scaling policies and CloudWatch metric alarms
autoscaling_policies_enabled = var.autoscaling_policies_enabled
cpu_utilization_high_threshold_percent = var.cpu_utilization_high_threshold_percent
cpu_utilization_low_threshold_percent = var.cpu_utilization_low_threshold_percent
}
module "eks_workers_2" {
source = "git::https://github.com/cloudposse/terraform-aws-eks-workers.git?ref=master"
namespace = var.namespace
stage = var.stage
name = "medium"
attributes = var.attributes
tags = var.tags
instance_type = "t3.medium"
vpc_id = module.vpc.vpc_id
subnet_ids = module.subnets.public_subnet_ids
health_check_type = var.health_check_type
min_size = var.min_size
max_size = var.max_size
wait_for_capacity_timeout = var.wait_for_capacity_timeout
cluster_name = module.label.id
cluster_endpoint = module.eks_cluster.eks_cluster_endpoint
cluster_certificate_authority_data = module.eks_cluster.eks_cluster_certificate_authority_data
cluster_security_group_id = module.eks_cluster.security_group_id
# Auto-scaling policies and CloudWatch metric alarms
autoscaling_policies_enabled = var.autoscaling_policies_enabled
cpu_utilization_high_threshold_percent = var.cpu_utilization_high_threshold_percent
cpu_utilization_low_threshold_percent = var.cpu_utilization_low_threshold_percent
}
module "eks_cluster" {
source = "git::https://github.com/cloudposse/terraform-aws-eks-cluster.git?ref=master"
namespace = var.namespace
stage = var.stage
name = var.name
attributes = var.attributes
tags = var.tags
vpc_id = module.vpc.vpc_id
subnet_ids = module.subnets.public_subnet_ids
kubernetes_version = var.kubernetes_version
kubeconfig_path = var.kubeconfig_path
oidc_provider_enabled = false
workers_role_arns = [module.eks_workers.workers_role_arn, module.eks_workers_2.workers_role_arn]
workers_security_group_ids = [module.eks_workers.security_group_id, module.eks_workers_2.security_group_id]
}
```
Module usage on [Terraform Cloud](https://www.terraform.io/docs/cloud/index.html):
```hcl
provider "aws" {
region = "us-east-2"
assume_role {
role_arn = "arn:aws:iam::xxxxxxxxxxx:role/OrganizationAccountAccessRole"
}
}
module "eks_cluster" {
source = "git::https://github.com/cloudposse/terraform-aws-eks-cluster.git?ref=master"
namespace = var.namespace
stage = var.stage
name = var.name
attributes = var.attributes
tags = var.tags
region = "us-east-2"
vpc_id = module.vpc.vpc_id
subnet_ids = module.subnets.public_subnet_ids
local_exec_interpreter = "/bin/bash"
kubernetes_version = "1.14"
workers_role_arns = [module.eks_workers.workers_role_arn]
workers_security_group_ids = [module.eks_workers.security_group_id]
# Terraform Cloud configurations
kubeconfig_path = "~/.kube/config"
configmap_auth_file = "/home/terraform/.terraform/configmap-auth.yaml"
install_aws_cli = true
install_kubectl = true
external_packages_install_path = "~/.terraform/bin"
aws_eks_update_kubeconfig_additional_arguments = "--verbose"
aws_cli_assume_role_arn = "arn:aws:iam::xxxxxxxxxxx:role/OrganizationAccountAccessRole"
aws_cli_assume_role_session_name = "eks_cluster_example_session"
}
```
## Makefile Targets
```
Available targets:
help Help screen
help/all Display help for all targets
help/short This help short screen
lint Lint terraform code
```
## Inputs
| Name | Description | Type | Default | Required |
|------|-------------|:----:|:-----:|:-----:|
| allowed_cidr_blocks | List of CIDR blocks to be allowed to connect to the EKS cluster | list(string) | `` | no |
| allowed_security_groups | List of Security Group IDs to be allowed to connect to the EKS cluster | list(string) | `` | no |
| apply_config_map_aws_auth | Whether to generate local files from `kubeconfig` and `config-map-aws-auth` templates and perform `kubectl apply` to apply the ConfigMap to allow worker nodes to join the EKS cluster | bool | `true` | no |
| associate_public_ip_address | Associate a public IP address with an instance in a VPC | bool | `true` | no |
| attributes | Additional attributes (e.g. `1`) | list(string) | `` | no |
| aws_cli_assume_role_arn | IAM Role ARN for AWS CLI to assume before calling `aws eks` to update `kubeconfig` | string | `` | no |
| aws_cli_assume_role_session_name | An identifier for the assumed role session when assuming the IAM Role for AWS CLI before calling `aws eks` to update `kubeconfig` | string | `` | no |
| aws_eks_update_kubeconfig_additional_arguments | Additional arguments for `aws eks update-kubeconfig` command, e.g. `--role-arn xxxxxxxxx`. For more info, see https://docs.aws.amazon.com/cli/latest/reference/eks/update-kubeconfig.html | string | `` | no |
| configmap_auth_file | Path to `configmap_auth_file` | string | `` | no |
| configmap_auth_template_file | Path to `config_auth_template_file` | string | `` | no |
| delimiter | Delimiter to be used between `name`, `namespace`, `stage`, etc. | string | `-` | no |
| enabled | Whether to create the resources. Set to `false` to prevent the module from creating any resources | bool | `true` | no |
| enabled_cluster_log_types | A list of the desired control plane logging to enable. For more information, see https://docs.aws.amazon.com/en_us/eks/latest/userguide/control-plane-logs.html. Possible values [`api`, `audit`, `authenticator`, `controllerManager`, `scheduler`] | list(string) | `` | no |
| endpoint_private_access | Indicates whether or not the Amazon EKS private API server endpoint is enabled. Default to AWS EKS resource and it is false | bool | `false` | no |
| endpoint_public_access | Indicates whether or not the Amazon EKS public API server endpoint is enabled. Default to AWS EKS resource and it is true | bool | `true` | no |
| external_packages_install_path | Path to install external packages, e.g. AWS CLI and `kubectl`. Used when the module is provisioned on workstations where the external packages are not installed by default, e.g. Terraform Cloud workers | string | `` | no |
| install_aws_cli | Set to `true` to install AWS CLI if the module is provisioned on workstations where AWS CLI is not installed by default, e.g. Terraform Cloud workers | bool | `false` | no |
| install_kubectl | Set to `true` to install `kubectl` if the module is provisioned on workstations where `kubectl` is not installed by default, e.g. Terraform Cloud workers | bool | `false` | no |
| jq_version | Version of `jq` to download to extract temporaly credentials after running `aws sts assume-role` if AWS CLI needs to assume role to access the cluster (if variable `aws_cli_assume_role_arn` is set) | string | `1.6` | no |
| kubeconfig_path | The path to `kubeconfig` file | string | `~/.kube/config` | no |
| kubectl_version | `kubectl` version to install. If not specified, the latest version will be used | string | `` | no |
| kubernetes_version | Desired Kubernetes master version. If you do not specify a value, the latest available version is used | string | `1.14` | no |
| local_exec_interpreter | shell to use for local exec | string | `/bin/bash` | no |
| map_additional_aws_accounts | Additional AWS account numbers to add to `config-map-aws-auth` ConfigMap | list(string) | `` | no |
| map_additional_iam_roles | Additional IAM roles to add to `config-map-aws-auth` ConfigMap | object | `` | no |
| map_additional_iam_users | Additional IAM users to add to `config-map-aws-auth` ConfigMap | object | `` | no |
| name | Solution name, e.g. 'app' or 'cluster' | string | - | yes |
| namespace | Namespace, which could be your organization name, e.g. 'eg' or 'cp' | string | `` | no |
| oidc_provider_enabled | Create an IAM OIDC identity provider for the cluster, then you can create IAM roles to associate with a service account in the cluster, instead of using kiam or kube2iam. For more information, see https://docs.aws.amazon.com/eks/latest/userguide/enable-iam-roles-for-service-accounts.html | bool | `false` | no |
| region | AWS Region | string | - | yes |
| stage | Stage, e.g. 'prod', 'staging', 'dev', or 'test' | string | `` | no |
| subnet_ids | A list of subnet IDs to launch the cluster in | list(string) | - | yes |
| tags | Additional tags (e.g. `map('BusinessUnit`,`XYZ`) | map(string) | `