using non-default and multiple aws profiles with boto3



~2 min read


249 words

I was recently working on a Python script that needed to access AWS resources. Fortunately, that’s exactly what the library boto3 is for (boto3 is the name of the AWS SDK for Python).

In my particular case, however, I wanted to use a non-default profile for AWS.

Setting the Profile is now possible thanks to this 2015 MR which highlights two usecases:

  1. Setting a profile other than the default as the one for boto3 to use
  2. Cycling through multiple profiles (as may be the case if you have to perform similar functions in various environments).

The API is actually quite nice. Imagine the following profiles:

aws_access_key_id = personal_access_key
aws_secret_access_key = personal_secret_key
aws_access_key_id = default_access_key
aws_secret_access_key = default_secret_key
aws_access_key_id = alt_access_key
aws_secret_access_key = alt_secret_key

Now, let’s run through the two situations!

Setting Boto To A Non-Default Profile

For the first case, we’ll say that this is a personal project, so instead of the default profile, we want to set it to personal:
import boto3


s3 = boto3.resource('s3')
for bucket in s3.buckets.all():
    print(f'personal buckets are -->{}')

Using Multiple Profiles Simultaneously

If the situation were slightly different and instead of different domains (work, personal), the AWS profiles referenced different environments (dev, qa, prod) , we can imagine doing the following:
dev = boto3.session.Session(profile_name='dev')
qa = boto3.session.Session(profile_name='qa')
prod = boto3.session.Session(profile_name='prod')

for session in [dev, qa, prod]:
    s3 = session.resource('s3')
    for bucket in s3.buckets.all():
        print(f'{session} bucket is named --> {})

Hi there and thanks for reading! My name's Stephen. I live in Chicago with my wife, Kate, and dog, Finn. Want more? See about and get in touch!