using non-default and multiple aws profiles with boto3

I was recently working on a Python script that needed to access AWS resources. Fortunately, that’s exactly what the library boto3 is for (boto3 is the name of the AWS SDK for Python).

In my particular case, however, I wanted to use a non-default profile for AWS.

Setting the Profile is now possible thanks to this 2015 MR which highlights two usecases:

1. Setting a profile other than the default as the one for boto3 to use
2. Cycling through multiple profiles (as may be the case if you have to perform similar functions in various environments).

The API is actually quite nice. Imagine the following profiles:

[personal]
aws_access_key_id = personal_access_key
aws_secret_access_key = personal_secret_key
[default]
aws_access_key_id = default_access_key
aws_secret_access_key = default_secret_key
[alt]
aws_access_key_id = alt_access_key
aws_secret_access_key = alt_secret_key

Now, let’s run through the two situations!

Setting Boto To A Non-Default Profile

For the first case, we’ll say that this is a personal project, so instead of the default profile, we want to set it to personal:

import boto3

boto3.setup_default_session(profile_name='personal')

s3 = boto3.resource('s3')
for bucket in s3.buckets.all():
    print(f'personal buckets are -->{bucket.name}')

Using Multiple Profiles Simultaneously

If the situation were slightly different and instead of different domains (work, personal), the AWS profiles referenced different environments (dev, qa, prod) , we can imagine doing the following:

dev = boto3.session.Session(profile_name='dev')
qa = boto3.session.Session(profile_name='qa')
prod = boto3.session.Session(profile_name='prod')

for session in [dev, qa, prod]:
    s3 = session.resource('s3')
    for bucket in s3.buckets.all():
        print(f'{session} bucket is named --> {bucket.name})