Secure AWS Credentials with credential_process

Jan 2, 2026 · 6 min read · AWS AWS CLI Security Credentials Best Practices ·

Share on:

Overview

Managing AWS credentials securely is a fundamental challenge for developers. Storing plain text access keys in ~/.aws/credentials creates significant security risks, especially when backing up dotfiles to version control systems. This post introduces credential_process, a powerful AWS CLI feature that allows you to source credentials from external processes, enabling encrypted credential storage while maintaining seamless AWS access.

The Problem with Plain Text Credentials

The traditional approach stores AWS credentials in ~/.aws/credentials:

1[default]
2aws_access_key_id = AKIAIOSFODNN7EXAMPLE
3aws_secret_access_key = wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

This approach has several drawbacks:

Security Risk: Plain text credentials can be exposed if the file is accidentally committed or shared
Backup Challenges: Cannot safely backup dotfiles to cloud storage or version control
Credential Rotation: Manual updates required across multiple machines
Audit Difficulty: No visibility into when and how credentials are accessed

credential_process: A Better Approach

The credential_process configuration option, introduced in AWS CLI v1.14.0 (November 2017) and botocore 1.8.0, allows the CLI and SDKs to retrieve credentials by executing an external command. This enables sophisticated credential management patterns including encryption, hardware security modules, and custom authentication flows.

How It Works

Instead of storing credentials directly, you specify a command in your ~/.aws/config file:

1[profile secure-profile]
2region = us-west-2
3credential_process = /path/to/your/credential-script.sh profile-name

When the AWS CLI or SDK needs credentials for this profile, it executes the specified command and expects a JSON response on stdout.

Required JSON Output Format

The external process must output valid JSON with the following structure:

1{
2  "Version": 1,
3  "AccessKeyId": "AKIAIOSFODNN7EXAMPLE",
4  "SecretAccessKey": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
5  "SessionToken": "optional-session-token",
6  "Expiration": "2025-01-01T12:00:00Z"
7}

Field	Required	Description
`Version`	Yes	Must be `1`
`AccessKeyId`	Yes	AWS access key ID
`SecretAccessKey`	Yes	AWS secret access key
`SessionToken`	No	Required for temporary credentials
`Expiration`	No	ISO8601 timestamp; triggers auto-refresh

Practical Implementation: Encrypted Credentials

Here's a practical implementation that stores credentials in an encrypted file, enabling safe backup of your dotfiles while keeping credentials secure.

Step 1: Encrypt Your Credentials

First, create a credentials file in the standard format and encrypt it:

 1# Create a strong encryption key
 2mkdir -p ~/.secrets
 3openssl rand -base64 32 > ~/.secrets/aws-creds.key
 4chmod 600 ~/.secrets/aws-creds.key
 5
 6# Encrypt your credentials file
 7openssl enc -aes-256-cbc -pbkdf2 \
 8  -in ~/.aws/credentials \
 9  -out ~/.aws/static_credentials.enc \
10  -pass file:~/.secrets/aws-creds.key
11
12# Remove the plain text file
13rm ~/.aws/credentials

Step 2: Create the Credential Retrieval Script

Create a script at ~/.aws/get_creds.sh:

 1#!/bin/bash
 2ENC_FILE="$HOME/.aws/static_credentials.enc"
 3KEY_FILE="$HOME/.secrets/aws-creds.key"
 4PROFILE="${1:-default}"
 5
 6if [ ! -f "$KEY_FILE" ]; then
 7  echo "Key file not found: $KEY_FILE" >&2
 8  exit 1
 9fi
10
11CONTENT=$(openssl enc -aes-256-cbc -d -pbkdf2 \
12  -in "$ENC_FILE" \
13  -pass file:"$KEY_FILE" 2>/dev/null)
14
15if [ $? -ne 0 ]; then
16  echo "Failed to decrypt credentials" >&2
17  exit 1
18fi
19
20AK=$(echo "$CONTENT" | sed -n "/^\[$PROFILE\]/,/^\[/p" | \
21     grep aws_access_key_id | cut -d= -f2 | tr -d ' ')
22SK=$(echo "$CONTENT" | sed -n "/^\[$PROFILE\]/,/^\[/p" | \
23     grep aws_secret_access_key | cut -d= -f2 | tr -d ' ')
24
25if [ -z "$AK" ] || [ -z "$SK" ]; then
26  echo "Profile '$PROFILE' not found" >&2
27  exit 1
28fi
29
30cat <<EOF
31{"Version":1,"AccessKeyId":"$AK","SecretAccessKey":"$SK"}
32EOF

Make it executable:

1chmod +x ~/.aws/get_creds.sh

Step 3: Configure AWS CLI Profiles

Update your ~/.aws/config:

1[profile dev]
2region = us-west-2
3cli_pager =
4credential_process = sh -c '$HOME/.aws/get_creds.sh dev'
5
6[profile prod]
7region = us-east-1
8cli_pager =
9credential_process = sh -c '$HOME/.aws/get_creds.sh prod'

Step 4: Safe Dotfiles Backup

Now you can safely backup your AWS configuration:

1# Files safe to backup (no plain text credentials)
2~/.aws/config                    # Profile configurations
3~/.aws/static_credentials.enc    # Encrypted credentials
4~/.aws/get_creds.sh             # Retrieval script
5
6# File to keep separate and secure
7~/.secrets/aws-creds.key         # Encryption key - NEVER backup to public repos

Supporting Temporary Credentials (STS)

For profiles that use temporary credentials from STS, create an extended script:

 1#!/bin/bash
 2PROFILE="${1:-default}"
 3CRED_FILE="$HOME/.aws/credential_$PROFILE"
 4FALLBACK_FILE="$HOME/.aws/credentials"
 5
 6if [ -f "$CRED_FILE" ]; then
 7  FILE="$CRED_FILE"
 8else
 9  FILE="$FALLBACK_FILE"
10fi
11
12if [ ! -f "$FILE" ]; then
13  echo "Credential file not found" >&2
14  exit 1
15fi
16
17AK=$(sed -n "/^\[$PROFILE\]/,/^\[/p" "$FILE" | \
18     grep aws_access_key_id | cut -d= -f2 | tr -d ' ')
19SK=$(sed -n "/^\[$PROFILE\]/,/^\[/p" "$FILE" | \
20     grep aws_secret_access_key | cut -d= -f2 | tr -d ' ')
21ST=$(sed -n "/^\[$PROFILE\]/,/^\[/p" "$FILE" | \
22     grep aws_session_token | cut -d= -f2 | tr -d ' ')
23
24if [ -z "$AK" ] || [ -z "$SK" ]; then
25  echo "Profile '$PROFILE' not found" >&2
26  exit 1
27fi
28
29if [ -n "$ST" ]; then
30  cat <<EOF
31{"Version":1,"AccessKeyId":"$AK","SecretAccessKey":"$SK","SessionToken":"$ST"}
32EOF
33else
34  cat <<EOF
35{"Version":1,"AccessKeyId":"$AK","SecretAccessKey":"$SK"}
36EOF
37fi

SDK and Tool Compatibility

The credential_process feature is supported across the AWS ecosystem:

Tool/SDK	Minimum Version	Notes
AWS CLI v1	1.14.0	November 2017
AWS CLI v2	All versions	Built-in support
botocore	1.8.0	Python SDK foundation
boto3	1.5.0	Python SDK
AWS SDK for Go	v1.15.0	Go SDK
AWS SDK for Java	2.x	Java SDK v2
AWS SDK for JavaScript	v3	Node.js SDK v3

Security Best Practices

When implementing credential_process, follow these guidelines:

Protect the encryption key: Store it separately from encrypted credentials
Use strong encryption: AES-256 with PBKDF2 key derivation
Set proper permissions: chmod 600 for sensitive files
Never log secrets: Avoid writing credentials to stderr
Handle errors gracefully: Return non-zero exit codes for failures
Consider hardware security: For high-security environments, integrate with HSMs or TPMs

Advanced Use Cases

Integration with Password Managers

1#!/bin/bash
2# Retrieve credentials from 1Password
3PROFILE="${1:-default}"
4op item get "AWS-$PROFILE" --format json | \
5  jq '{Version:1, AccessKeyId:.fields[0].value, SecretAccessKey:.fields[1].value}'

Integration with HashiCorp Vault

For organizations using HashiCorp Vault for secrets management:

 1#!/bin/bash
 2# Retrieve credentials from HashiCorp Vault
 3VAULT_PATH="${1:-secret/data/aws/credentials}"
 4
 5SECRET=$(vault kv get -format=json "$VAULT_PATH" 2>/dev/null)
 6
 7if [ $? -ne 0 ]; then
 8  echo "Failed to retrieve secret from Vault" >&2
 9  exit 1
10fi
11
12echo "$SECRET" | jq '{
13  Version: 1,
14  AccessKeyId: .data.data.access_key_id,
15  SecretAccessKey: .data.data.secret_access_key
16}'

Alternative: AWS IAM Identity Center (SSO)

For environments using AWS IAM Identity Center (formerly AWS SSO), note that SSO provides a native alternative to credential_process rather than being combined with it. SSO handles temporary credential generation automatically:

1[profile sso-profile]
2sso_session = my-sso
3sso_account_id = 123456789012
4sso_role_name = DeveloperAccess
5
6[sso-session my-sso]
7sso_start_url = https://my-org.awsapps.com/start
8sso_region = us-east-1
9sso_registration_scopes = sso:account:access

When to use each approach:

Approach	Use Case
`credential_process`	Long-term IAM credentials, custom auth systems, secrets managers
AWS SSO	Federated identity, temporary credentials, enterprise SSO integration
Both (rare)	Legacy systems requiring IAM credentials alongside SSO migration

Conclusion

The credential_process feature provides a flexible and secure approach to AWS credential management. By storing credentials in encrypted form and retrieving them through external processes, you can:

Safely backup your AWS configuration to version control
Implement custom authentication flows
Integrate with enterprise security tools
Maintain credential hygiene across multiple machines

For more AWS CLI tips and tricks, check out my post on Awesome AWS CLI.