We build and support Cloud Infrastructure

We build and support Cloud Infrastructure

© 2024. All rights reserved.

Programmatically accessing AWS EKS with Ruby

20 Jan 2019

Lately I have been doing some research on how to programmatically access k8s API hosted on AWS EKS. This took some time, therefore I decided to share the experience. First of all, I would like to mention this great article which is a great help: Kubernetes Client Authentication on Amazon EKS. It gives overview of Kubernetes authentication and authorization, as well as AWS EKS specifics.

You can read the full article to get detailed information, but briefly summing it up, Amazon uses a tool called aws-iam-authenticator.

On a client-side you have to use it to generate a Kubernetes API authentication token. On server-side Kubernetes passes this request to aws-iam-authenticator server via a webhook. In case you need to access Kubernetes API from one of the pods within the cluster, then the process is much easier - authentication token and CA are mounted to pods by default to: /var/run/secrets/kubernetes.io/serviceaccount/

Manual walk-through

Lets manually go through this process to get understanding on what is happening. You have to have an existing AWS EKS cluster to do this.

Get the aws-iam-authenticator client tool:

In case you encounter problems, refer to original manual from AWS: Installing aws-iam-authenticator

The tool uses the same credential providers to talk to AWS API as AWS CLI does. You have to set up credentials using one of available methods, for example using environment variables:

See this manual to find available options: Configuring the AWS CLI

Once you have credentials set, try to run the tool to generate the token:

aws-iam-authenticator token -i <eks_cluster_name>

If all was set up correctly, you should receive a json with token data.

{
  "kind":"ExecCredential",
  "apiVersion":"client.authentication.k8s.io/v1alpha1",
  "spec":{},
  "status":{
    "token":"k8s-aws-v1.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"}
}

Refer to aws-iam-authenticator Troubleshooting section if not.

This token can be used to Authenticate requests to Kubernetes API, but we still need some extra information to succeed. We need to have the CA certificate of Kubernetes API server. It is automatically generated during EKS cluster creation. It is possible to retrieve it from AWS EKS console, or via cli command:

aws eks describe-cluster --name <eks_cluster_name> --query 'cluster.certificateAuthority.data'

It is a base64-encoded certificate of our API endpoint. Decode it to a certificate file:

aws eks describe-cluster --name <eks_cluster_name> --query 'cluster.certificateAuthority.data' --output text | base64 --decode > ca.crt

Now we are all set to make a manual query to Kubernetes API. You can retrieve the API endpoint either from AWS Console or using AWS CLI:

aws eks describe-cluster --name <eks_cluster_name> --query 'cluster.endpoint' --output text

Include the token and CA to the request:

TOKEN=<token provided by aws-iam-authenticator>
HOST=<eks cluster endpoint url>
CURL_CA_BUNDLE=ca.crt curl -H "Authorization: Bearer $TOKEN" -X GET $HOST/apiv1/namespaces/

Hey! You should get a json response listing all your cluster namespaces. Not bad!

Doing it in Ruby

If you would like to use programmatic access, you still need to use aws-iam-authenticator to generate the token. Make sure to have it available on your PATH. Also keep in mind that it will also need AWS access keys (Configuring the AWS CLI).

I decided to use the official Kubernetes Ruby gem. You can add it to Gemfile like this:

gem 'kubernetes', :git => 'https://github.com/kubernetes-client/ruby.git'

Below you can find the complete script showing the approach of generating access token and communicating with Kubernetes API.

# This script shows how to initiate a connection to AWS EKS programmatically

# Dependencies
# - aws-iam-authenticator

require 'logger'
require 'optparse'
require 'kubernetes'
require 'kubernetes/utils'

@logger = Logger.new(STDOUT)
@logger.level = Logger::DEBUG

@options = {}

OptionParser.new do |opts|
  opts.banner = "Initiate a connection to AWS EKS programmatically. Usage: #{$0} [options]"
  
  opts.on("--eks-cluster-name CLUSTER_NAME", "Name of EKS cluster") { |o| @options[:eks_cluster_name] = o }
  opts.on("--eks-cluster-ca-file CLUSTER_CA_FILE", "Path to EKS Cluster CA file") { |o| @options[:eks_cluster_ca_file] = o }
  opts.on("--eks-cluster-endpoint CLUSTER_ENDPOINT", "Provide a cluster endpoint for connection") { |o| @options[:eks_cluster_endpoint] = o }
end.parse!

# Sanity checks
if [:eks_cluster_name, :eks_cluster_ca_file, :eks_cluster_endpoint].any? { |s| @options[s].nil? }
  @logger.error("Mandatory arguments missing. Use -h for help.")
  raise OptionParser::MissingArgument 
end

%x(which aws-iam-authenticator)
if $? != 0
  @logger.error("aws-iam-authenticator not found. Please make sure it is available in PATH.")
  raise RuntimeError
end

# Functions
# this function depends on aws-iam-authenticator tool
def auth_token()
  @logger.debug("Generating auth token with aws-iam-authenticator tool...")
  token_res = %x(aws-iam-authenticator token -i #{@options[:eks_cluster_name]})
  if $? != 0
    @logger.error("Unable to generate token using aws-iam-authenticator. Make sure the tool is available in PATH and correct AWS credentials exist.")
    raise RuntimeError
  end
  JSON.parse(token_res)
end 

def configure_kubernetes_connection()
  Kubernetes.configure do |c|
    c.api_key_prefix['authorization'] = 'Bearer'
    c.api_key['authorization'] = auth_token['status']['token']
    c.host = @options[:eks_cluster_endpoint]
    c.ssl_ca_cert = @options[:eks_cluster_ca_file]
    c.logger = @logger
  end
end

# main
configure_kubernetes_connection()

api_instance = Kubernetes::CoreV1Api.new

opts = { 
  limit: 10, 
  timeout_seconds: 10 
}

begin
  result = api_instance.list_namespace(opts)
  p result
rescue Kubernetes::ApiError => e
  @logger.error("Exception when calling CoreV1Api->list_namespace: #{e}")
end

Main focus of this post was to give some hints on how to Authenticate to AWS EKS Kubernetes API programmatically, so I will leave exploring Kubernetes API up to you. The script is just a reference, it probably needs some retry logic as well.

That’s it, happy coding =)

Related Posts