Farewell Static Access Keys

Blog Header Image

Introduction

CI/CD pipelines frequently require integration with external cloud services to support a wide range of use cases. This may include uploading artifacts to object storage solutions (e.g. Amazon S3, Azure Blob Storage), publishing container images to artifact repositories (e.g. AWS Elastic Container Registry, Google Artifact Registry), or deploying applications to container orchestration platforms (e.g. Amazon ECS, Kubernetes, Azure AKS).

For example, in AWS, the common practice involves creating an IAM user, assigning the necessary permissions, and generating AWS access keys to facilitate this interaction.

These keys are then typically stored manually in the CI/CD platform, such as GitHub Secrets, CircleCI Contexts, or similar tools. However, this approach presents several challenges.

Storing static secrets in third-party platforms poses a significant risk of unauthorized access and potential exposure.
High-profile breaches, such as the CircleCI breach and the Codecov supply chain attack, highlight the appeal of CI/CD services as prime targets for attackers.
With these platforms' critical role in modern development pipelines, the question isn't "if" but "when" the next breach will occur.

If not handled perfectly, rotating these keys frequently can result in unexpected downtimes or operational challenges. Even small errors during the rotation process can disrupt workflows and cause delays. Moreover, the process of key rotation demands a significant time investment, diverting valuable engineering resources from other critical tasks.
Given the sensitive nature of CI/CD pipelines, it is evident that relying on such fragile and resource-intensive practices is a considerable risk.

Luckily, there is a better approach. OpenID Connect (OIDC) offers a solution by replacing static credentials with short-lived tokens.
These tokens are dynamically issued during each workflow run, removing the need for long-lived secrets in CI/CD platforms and significantly enhancing security and operational efficiency.

In this blog, I’ll demonstrate how I set up GitHub Actions to securely interact with AWS resources using OIDC.
My specific use case involves uploading files to a website hosted on S3 and clearing the CloudFront cache to ensure the new content is served.

By the end of this blog, you’ll gain a clear understanding of how OIDC overcomes the challenges posed by static secrets and learn how to implement this approach effectively in your own environments.

Before diving into the technical details of this setup, let’s take a step back to explore what OIDC is and how it works so that we can lay a solid foundation for the implementation.

Understanding OIDC

OpenID Connect (OIDC) is a protocol built on top of OAuth 2.0 that simplifies verifying the identity of users or services. Instead of relying on static credentials like passwords or API keys, OIDC uses tokens to provide secure access via short-lived credentials.

Using an OIDC Identity Provider allows you to manage external identities, such as users or services, to access your internal resources without the need to create and maintain separate internal user accounts.

To illustrate this, consider the analogy of a pass for a convention or conference. The event spans multiple days with various sessions, but access to specific sessions requires an appropriate pass. This pass contains your name (identity), your registration type (role or scope), and a unique QR code (short-lived credentials) to grant you access to the sessions. The pass is only valid for the event and time it’s issued, ensuring controlled, temporary access.

If we extend this analogy to our use case, GitHub workflows represent the attendees needing access to the specific event sessions (AWS resources).
To gain access, the workflows request a pass from the GitHub OIDC provider, which issues an OIDC token in the form of a JSON Web Token (JWT). This token includes key details, called claims, about the specific request.

Lets focus on the main ones:

  • iss: The issuer of the token (GitHub OIDC provider). This claim, along with other header parameters such as alg(algorithm) and kid(key ID), helps AWS validate the token's origin and verify its authenticity.
  • sub: The subject of the token. Represents the unique workflow making the request. It includes metadata like the repository, branch, and triggering event.
  • aud: The audience of the token. Specifies who the token is intended for (default to the URL of the repository owner). In my case, because I use the official aws-actions/configure-aws-credentials action in my workflow, it must be set to sts.amazonaws.com. More on that later.

AWS then validates the OIDC token’s details against its predefined trust configuration. If the token’s claims match, AWS generates a temporary set of credentials. These credentials provide the workflow with secure, time-limited access to the specified resources.

Putting it All Together

Now with the basic understanding of what OIDC is, it’s time to piece everything together.
By building these components step by step, we’ll create a secure and seamless integration that allows GitHub Actions workflows to interact with AWS resources without relying on static credentials.

Let’s dive into the implementation. I will use Terraform, my preferred tool for provisioning infrastructure. However, the same setup can be achieved using the AWS Management Console, the AWS Command Line Interface (CLI), or Tools for Windows PowerShell.
Detailed instructions can be found in the official AWS documentation.

It is important to note that support for custom claims for GitHub OIDC is unavailable in AWS.

Building The Trust

The first step is to add the GitHub OIDC Provider as a trusted identity Provider (IdP) in AWS. This is done by creating an OIDC provider entity in AWS IAM.
By doing so, we set the foundations to establish trust between our AWS account and GitHub’s OIDC provider. 


    resource "aws_iam_openid_connect_provider" "github_actions_oidc_provider" {
        url   = "https://token.actions.githubusercontent.com"

        client_id_list = [
            "sts.amazonaws.com",
        ]

        thumbprint_list = ["33e4e80807204c2b6182a3a14b591acd25b5f0db"] # Thumbprint of the GitHub OIDC provider's Intermediate Certificate
    }

The OIDC provider resource has three key components that AWS uses during the validation process:

  • url: The OIDC provider URL, which corresponds to the iss (Issuer) claim in the token.
  • client_id_list: These correspond to the aud (Audience) claim. In our case, it is set to sts.amazonaws.com.
  • thumbprint_list: A list of certificate thumbprints that AWS uses to validate the OIDC provider’s SSL certificate. This ensures that the OIDC provider is trusted.
    You can follow this procedure to obtain the thumbprint.

Creating a Role and Trust Policy

Once the OIDC provider is configured in AWS, the next step is to create an IAM role.
Unlike a user, an IAM role does not have long-term credentials. Instead, it is a temporary identity assigned to a federated user, such as a GitHub workflow, after successful authentication by the Identity Provider (IdP).

The role defines the trust relationship, allowing the workflow to request temporary security credentials from AWS Security Token Service (STS) using a token issued by the Identity Provider (IdP). These credentials grant access to AWS resources, with the scope of access governed by the policies attached to the role.

It is best practice to restrict which entities can assume the role by defining conditions in the trust policy.
For example, you can use the token.actions.githubusercontent.com:sub condition key with string operators to limit access to a specific GitHub organization, repository, or branch.
This ensures that only trusted repositories can request tokens for your resources. Additionally, in our use case, IAM enforces that the token.actions.githubusercontent.com:sub condition key is included and prohibits the use of wildcard (* or ?) or null values. It is to prevent unauthorized access to the role. If not set correctly, the request will fail. 


    resource "aws_iam_role" "github_actions_oidc_role" {
        name = "github-actions-oidc-role"

        assume_role_policy = jsonencode({
            "Version": "2012-10-17",
            "Statement": [
            {
                "Effect": "Allow",
                "Principal": {
                    "Federated": "${aws_iam_openid_connect_provider.github_actions_oidc_provider.arn}"
                },
                "Action": "sts:AssumeRoleWithWebIdentity",
                "Condition": {
                    "StringEquals": {
                        "token.actions.githubusercontent.com:aud": "sts.amazonaws.com"
                    },
                    "StringLike": {
                        "token.actions.githubusercontent.com:sub": "repo:GitHubOrg/GitHubRepo:ref:refs/heads/GitHubBranch" # Restrict access to a specific GitHub repository and branch
                    }
                }
            }
            ]
        })
    }

Some Tips

Using Locals

Here’s an approach to restrict access to workflows from specific GitHub repositories, regardless of the branch, by leveraging Terraform locals. 



    # Environment-specific variable for repository list
    variable "github_repos" {
        type = list(string)
        default = [
          "repo1",
          "repo2",
          "repo3"
        ]
      }
      
      locals {
        repo_sub_list = [for repo in var.github_repos : "repo:GitHubUser/${repo}:*"]
      }
      
      # Use the local in the trust policy condition
        "StringLike": {
                  "token.actions.githubusercontent.com:sub": local.repo_sub_list
         }
  • github_repos: A variable defining the list of GitHub repositories you want to allow.
  • repo_sub_list: A local variable that dynamically generate the sub claim conditions for each repository.
  • Condition: The StringLike operator ensures the role is limited to workflows from the specified repositories, regardless of the branch.

This approach becomes especially handy for managing resource access across multiple environments. It allows you to bind specific repositories to environments based on the required access.
It also makes it more readable and maintainable.

You can explore this page to see some examples of how you might use these conditions in the GitHub OIDC trust policy.

Using ABAC

In larger environments, Attribute-Based Access Control (ABAC) can greatly simplify access management by utilizing tags in IAM policies.
Tags allow you to logically group resources, eliminating the need to explicitly list individual resources in your policies.

In fact, in a previous blog, I demonstrated how to control Lambda access to ECR repositories based on their tags.

To learn more, check out the AWS documentation on ABAC and access control with tags.

Creating The Policies

Once the role and its trust policy are in place, the next step is to create your policies.

In my use case, as described earlier, I need the workflow to upload files to an S3 bucket and create a CloudFront invalidation to clear the cache.

Here is a sample policy that allows the workflow to perform these actions: 


    {
          "Version": "2012-10-17",
          "Statement": [
            {
              "Effect": "Allow",
              "Action": [
                "s3:GetObject",
                "s3:PutObject",
                "s3:DeleteObject"
              ],
              "Resource": [
                "arn:aws:s3:::BucketName/*",
                "arn:aws:s3:::BucketName"
              ]
            },
            {
              "Effect": "Allow",
              "Action": [
                "cloudfront:CreateInvalidation"
              ],
              "Resource": "arn:aws:cloudfront::AccountID:distribution/DistributionID"
            }
          ]
    }

Configuring GitHub Workflows for OIDC Integration

Great! We’re making steady progress.

With the AWS setup complete, we’ve configured it to validate OIDC tokens and issue short-lived credentials. Now, it’s time to update our GitHub workflows to request these tokens and utilize the generated credentials within the workflow jobs.

Defining Permissions

The first step is to define the necessary permissions in the workflow’s YAML configuration file. 


    permissions:
        id-token: write
        contents: read

The id-token: write permission allows the workflow to request an OIDC token from the GitHub OIDC provider.
The contents: read is needed if your workflow uses the actions/checkout action to access repository contents.

Requesting Access

With the permissions configured, the next step is to utilize the OIDC token within the workflow job. The aws-actions/configure-aws-credentials action simplifies this process by retrieving the JWT from GitHub’s OIDC provider and exchanging it for temporary credentials from AWS.
You can find more details about this action in the official documentation.

Here is an example of how its implemented in the workflow file: 


    - name: Configure AWS Credentials
      uses: aws-actions/configure-aws-credentials@v3
      with:
        role-to-assume: arn:aws:iam::${{ env.AWS_ACCOUNT_ID }}:role/github-actions-oidc-role
        aws-region: ${{ env.AWS_REGION }}

    - name: Sync to S3
      run: aws s3 sync . s3://${{ env.S3_BUCKET_NAME }} --delete


    - name: CloudFront Cache Invalidation
      run: aws cloudfront create-invalidation --distribution-id ${{ env.CLOUDFRONT_DISTRIBUTION_ID }} --paths "/*"

Configure AWS Credentials

  • Assumes the specified IAM role (github-actions-oidc-role) using the OIDC token.
  • Retrieves short-lived credentials valid for the duration of the workflow run.

Sync to S3

  • Uses the temporary credentials to upload files to the S3 bucket.

CloudFront Cache Invalidation

  • Uses the temporary credentials to clear the CloudFront cache.

And That’s It!

By combining OpenID Connect (OIDC) with GitHub Actions and AWS, we’ve built a secure, seamless workflow that eliminates the need for static credentials. Instead, we’re using dynamically generated, short-lived tokens to ensure secure access to AWS resources.

Following these steps, both improves the security of your workflows and simplifies credential management. Whether your workflows are deploying applications, managing infrastructure, or updating content like in my use case, this setup ensures that sensitive credentials remain out of reach.

Oh, and don’t forget to delete those static credentials from your CI/CD platform, you won’t need them anymore! 😉

Thank you for reading. I hope you found it helpful and enjoyable!