Introduction

Containerization has gained popularity as a method for deploying and managing applications with Kubernetes, which is a leading container orchestration platform. Many customers choose Amazon Elastic Kubernetes Service (Amazon EKS) for its performance, scalability, availability, and integration with other AWS services and security.

Enterprises across the industry opt for private container repositories, such as JFrog Artifactory, to enhance security, maintain compliance, and protect intellectual property. They secure those repositories with certificates issued by Private Certificate Authority, which helps to optimize cost and to tailor certificate to their unique needs. These enterprises also want to utilize Amazon EKS to host their applications and securely retrieve images from private repositories.

This post guides you through the process of configuring Amazon EKS worker nodes to securely use a private container image repository. We’ll use JFrog Artifactory as the private container image repository for our example. You may choose any other repository management software available on the market as well.

Solution overview

Figure 1. Amazon EKS cluster securely accessing private image repository

1. The client certificate is stored in an Amazon S3 bucket, which is encrypted with Amazon KMS customer managed key.
2. Amazon EKS nodes, which get access via the attached AWS IAM role, copy the client certificate and install on each Amazon EKS node.
3. Nodes with the installed certificate are able to securely connect to the container image repository and pull container images.

Walkthrough

1. Create a private certificate authority (CA) (using AWS Private Certificate Authority)
2. Issue an end-entity certificate (using AWS Certificate Manager)
3. Install and configure a private container image repository (e.g., JFrog Artifactory and secure with the end-entity certificate)
4. Create private hosted zone using Amazon Route 53 to support business friendly domain name (optional)
5. Upload Root CA certificate to Amazon Simple Storage Service (Amazon S3)
6. Create an Amazon EKS cluster, which automates the installation of a Root CA certificate for cluster worker nodes
7. Securely connect to and deploy a container image from a JFrog Artifactory repository

Note: Following the steps describe in this post will incur cost.

Prerequisites

• An AWS Account
• Install eksctl
• Install AWS Command Line Interface (AWS CLI)

1. Create and install Root and subordinate CA

To begin, let’s create the certificate authority using the AWS Private Certificate Authority service. In contrast to the public certificate, private certificate is used only internally. It is recommended to follow the best practice of creating a certificate authority hierarchy and issuing an end-entity certificate from a subordinate certificate authority. For detailed information on designing a CA hierarchy, please refer to the AWS documentation. In this post, we’ll demonstrate the creation of a Root and a subordinate certificate.

• Sign into your AWS account, navigate to the AWS Private Certificate Authority service Console and create Root certificate authority.

Figure 2. Create Root certificate authority

Under Subject distinguished name options, configure the subject name of your private CA. You must enter a value for at least one of the following options. (Note: we’ll use myca.local domain name throughout this post):

• • Organization (O) – For example, a company name
  • Organization Unit (OU) – For example, a division within a company
  • Country name (C) – A two-letter country code
  • State or province name – Full name of a state or province
  • Locality name – The name of a city
  • Common Name (CN) – myca.local

• Install a root CA certificate.
• Create a subordinate CA. We’ll use myca.local as Common Name (CN).

Figure 3. Create Subordinate certificate authority

• Install a certificate for your AWS Private CA hosted subordinate CA.
• Once the subordinate CA is active, export the Root CA certificate and save it in a .pem file (e.g., client.pem). That’s the certificate that we’ll install later on Amazon EKS cluster worker nodes.

2. Issue end-entity certificate

The next step is to issue an end-entity certificate using the AWS Certificate Manager service. An end-entity certificate is digitally signed statement issued by CA to a person or a system. It is used to validate the identity of an entity such as website, business, or person. Select the previously created Subordinate CA as the certificate authority. Additionally, choose a fully qualified domain name (FQDN) for your certificate, such as repo.sub.myca.local. This certificate will be used to enable TLS for JFrog Artifactory.

• Request Private Certificate

Figure 4. Create private certificate

After the end-entity certificate is issued, you need to export the certificate bundle for further use to enable Transport Layer Security (TLS) on the JFrog Artifactory server. The export includes three key elements (i.e., .pem files): the primary TLS certificate, the Certificate Chain, and the private key.

3. Install and configure a private container image repository

Next, install and configure JFrog Artifactory on an Amazon Elastic Compute Cloud (Amazon EC2) instance. Please refer to be JFrog installation guide for further detail. After the installation, use the certificate bundle to enable TLS for the JFrog Artifactory server.

Once JFrog Artifactory is installed and configured, build and push a sample docker image to the repository that we will be using for testing.

4. Create Amazon Route53 private hosted zone for user-friendly domain name (optional)

The FQDN for the certificate is repo.sub.myca.local. We need to assign the same domain name to the container image repository. To manage the Domain Name System (DNS), we’ll use Amazon Route53 service. Create a private hosted zone in Amazon Route53. Once the private hosted zone is available, create a DNS record of type A to map repo.sub.myca.local to the repository server’s IP address:

Figure 5. Create DNS record in Amazon Route 53

5. Upload Root CA certificate to Amazon S3

Amazon EKS worker nodes need access to the Root CA certificate during bootstrapping. We can use Amazon Simple Storage Service (Amazon S3) service as a storage solution for the certificate. To store the certificate, you can use an existing Amazon S3 bucket or create a new one. Use Amazon S3 security best practices to protect the content of your bucket. For better tracking, we recommend to create a dedicated Amazon S3 bucket with server access log enabled. Then, upload client certificate client.pem, we retrieved earlier:

6. Create an Amazon EKS cluster and automate installation of Root CA certificate

We’re prepared to provision an Amazon EKS cluster and dynamically install the Root CA certificate. The Root CA certificate is used to established the chain of trust with server certificate installed in image repository (e.g., JFrog Artifactory) and establish TLS connection. To simplify the process, we utilize the eksctl tool for provisioning Amazon EKS clusters and worker nodes. During the launch of each worker node, the certificate is fetched from the specified Amazon S3 bucket and automatically installed on the nodes.

Create an configuration file, called cluster.yaml and copy-paste the following content:

Execute the following command to deploy an Amazon EKS cluster and a managed NodeGroup.

Note: When you create Amazon EKS-managed node groups, the standard Amazon EKS optimized Amazon Linux 2 machine image is used by default

eksctl creates two AWS CloudFormation stacks with all required AWS resources. You can review them in AWS CloudFormation Console. Let’s go over some of the settings in cluster.yaml file that are relevant to this post:

• IAM section under managedNodeGroups is translated into an AWS Identity and Access Management (AWS IAM) policy that allows the Amazon EKS nodes to read files from the Amazon S3 bucket where we uploaded the client certificate. The policy is included in the AWS IAM role along with other required policies and attached to an AWS Launch Template.
• preBootstrapCommands is added as user data to the launch template. The commands copy the client certificate and update consolidated and dynamic configuration of Root CA certificates and associated trust on each node.

The same configuration works for unmanaged node groups as well, to use unmanaged node groups, replace managedNodeGroups with nodeGroups in cluster.yaml.

Validate certificate installation

To verify the certificate has been installed on the nodes, proceed with the following instructions.

Get node instance ids first:

Replace <node instance id> with the value from the previous command’s output:

The last two commands execute trust list command on the node and return the output, you should see the following:

Note that by default, eksctl creates a separate Virtual Private Cloud (VPC) in your account. If you’d like to use an existing VPC, then you need to supply VPC configuration. Also, eksctl configures kubectl for us.

7. Securely connect to and deploy image from secure JFrog Artifactory repository

Our container image repository is password protected so a pod needs to use a Secret to pull an image. Create a json file with the secret (e.g., secret.json).

You can generate base64 encoded credentials by running the following command:

The user and password credentials of the container image repository account are allowed to pull images. You can set any user name and password of your choice.

Now, create secret manifest file (e.g., secret.yaml).

The .dockerconfigjson is a base64 encoded string of secret.json. Generate it using the following command:

Now, deploy the secret:

Next, create pod manifest file (e.g., pod.yaml):

Final step is to deploy the pod:

Validate image pull and pod creation

To check whether the pod is successfully created, run the following command:

Cleaning up

To avoid incurring further charges, delete the Amazon EKS cluster using the following command:

Additionally, use console or command line to clean-up and remove the following resources:

• Amazon S3 bucket containing the certificate
• Amazon Route 53 private hosted zone
• Root and Subordinate CAs
• Amazon EC2 instance hosting container image repository

Conclusion

In this post, we showed you how you can configure your Amazon EKS cluster, generate and install the necessary certificates, and deploy nodes that can successfully pull container images from the private registry. This enables seamless integration of private registry images with your Amazon EKS cluster. Enabling Amazon EKS clusters to support pulling container images from private registries secured with private certificates is crucial for secure and efficient application deployments, and for adherence to the compliance need of your organization.