AWS Credentials Configuration Guide

Method 1: Using AWS CLI (Recommended)

Install AWS CLI (if not installed)

# Ubuntu/Debian
sudo apt-get update
sudo apt-get install awscli

# macOS
brew install awscli

# Or download from AWS
# https://aws.amazon.com/cli/

Configure Credentials

aws configure

You’ll be prompted for:

AWS Access Key ID [None]: AKIAIOSFODNN7EXAMPLE
AWS Secret Access Key [None]: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
Default region name [None]: us-east-1
Default output format [None]: json

Verify Configuration

aws sts get-caller-identity

Output should show:

{
    "UserId": "AIDACKCEVSQ6C2EXAMPLE",
    "Account": "123456789012",
    "Arn": "arn:aws:iam::123456789012:user/your-username"
}

Method 2: Environment Variables

Set temporarily in your shell:

export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
export AWS_DEFAULT_REGION=us-east-1

Or add to ~/.bashrc or ~/.zshrc for persistence:

echo 'export AWS_ACCESS_KEY_ID=your_key' >> ~/.bashrc
echo 'export AWS_SECRET_ACCESS_KEY=your_secret' >> ~/.bashrc
source ~/.bashrc

Method 3: Credentials File

Edit ~/.aws/credentials:

[default]
aws_access_key_id = AKIAIOSFODNN7EXAMPLE
aws_secret_access_key = wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

[work]
aws_access_key_id = AKIAI44QH8DHBEXAMPLE
aws_secret_access_key = je7MtGbClwBF/2Zp9Utk/h3yCo8nvbEXAMPLEKEY

[personal]
aws_access_key_id = AKIAIOSFODNN7EXAMPLE
aws_secret_access_key = wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

Edit ~/.aws/config for profiles:

[default]
region = us-east-1
output = json

[profile work]
region = us-west-2
output = json

[profile personal]
region = eu-west-1
output = yaml

Using Profiles

# Set default profile
export AWS_PROFILE=work

# Or use with aws command
aws s3 ls --profile work

# Or in Python/Polars
import polars as pl
df = pl.read_parquet("s3://bucket/file.parquet", storage_options={"profile": "work"})

Method 4: IAM Role (EC2/ECS/Lambda)

If running on AWS infrastructure, no configuration needed!

import polars as pl

# Automatically uses instance role
df = pl.read_parquet("s3://my-private-bucket/data.parquet")

Method 5: Session Token (Temporary Credentials)

For temporary credentials (MFA, assumed roles):

export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
export AWS_SESSION_TOKEN=IQoJb3JpZ2luX2VjEHYaCXVzLWVhc3QtMSJ...

Getting Your AWS Credentials

1. From AWS Console:

1. Sign in to AWS Console
2. Click your name (top right) → “Security Credentials”
3. Click “Access keys” → “Create access key”
4. Download the .csv file (you won’t see the secret key again!)

2. Using AWS CLI:

# Create access key
aws iam create-access-key --user-name your-username

# List access keys
aws iam list-access-keys --user-name your-username

3. For IAM Roles (Assumed Roles):

# Assume a role
aws sts assume-role \
  --role-arn arn:aws:iam::123456789012:role/MyRole \
  --role-session-name MySession \
  --duration-seconds 3600

# Set the temporary credentials from the output
export AWS_ACCESS_KEY_ID=...
export AWS_SECRET_ACCESS_KEY=...
export AWS_SESSION_TOKEN=...

Testing Your Credentials

Test 1: AWS CLI

# List S3 buckets
aws s3 ls

# List specific bucket
aws s3 ls s3://my-bucket-name/

Test 2: Python with Boto3

import boto3

# Using default credentials
s3 = boto3.client('s3')
response = s3.list_buckets()
print([bucket['Name'] for bucket in response['Buckets']])

Test 3: Python with Polars

import polars as pl

# Test reading from private bucket
df = pl.read_parquet("s3://my-bucket/data.parquet")
print(df.head())

Security Best Practices

✅ DO:

• Use IAM roles when possible (EC2/ECS/Lambda)
• Rotate access keys regularly
• Use least-privilege permissions
• Enable MFA for root and IAM users
• Use environment variables for CI/CD
• Store credentials in AWS Secrets Manager

❌ DON’T:

• Hardcode credentials in code
• Commit credentials to Git
• Share credentials between users
• Use root account access keys
• Store keys in plain text files

Troubleshooting

Issue: “Unable to locate credentials”

# Check if credentials file exists
ls -la ~/.aws/credentials

# Check environment variables
env | grep AWS

Issue: “Access Denied”

# Verify identity
aws sts get-caller-identity

# Check permissions
aws iam get-user

# Test specific bucket
aws s3api head-bucket --bucket my-bucket-name

Issue: “Invalid access key”

# Reconfigure
aws configure

# Or delete and recreate
aws iam delete-access-key --access-key-id OLD_KEY --user-name your-username
aws iam create-access-key --user-name your-username

Quick Reference

Method	Best For	Persistence
aws configure	Local development	Permanent
Environment vars	CI/CD, scripts	Session only
Credentials file	Multiple profiles	Permanent
IAM roles	AWS infrastructure	Automatic
Session tokens	MFA, temp access	Temporary

Next Steps

Once configured, you can:

# Download the OSM dataset
aws s3 cp s3://daylight-openstreetmap/parquet/osm_features/ ./osm_data/ --recursive

# Or access private buckets
aws s3 ls s3://my-private-bucket/

# Or use with Polars
python3 -c "import polars as pl; print(pl.read_parquet('s3://daylight-openstreetmap/parquet/osm_features/release=v1.58/type=way/20241112_191814_00139_grr7u_fea4d477-4748-4e7d-9aed-90290d792f01').head())"