using non-default and multiple aws profiles with boto3



~2 min read


249 words

I was recently working on a Python script that needed to access AWS resources. Fortunately, that’s exactly what the library boto3 is for (boto3 is the name of the AWS SDK for Python).

In my particular case, however, I wanted to use a non-default profile for AWS.

Setting the Profile is now possible thanks to this 2015 MR which highlights two usecases:

  1. Setting a profile other than the default as the one for boto3 to use
  2. Cycling through multiple profiles (as may be the case if you have to perform similar functions in various environments).

The API is actually quite nice. Imagine the following profiles: `shell:title=~/.aws/credentials [personal] awsaccesskeyid = personalaccesskey awssecretaccesskey = personalsecretkey [default] awsaccesskeyid = defaultaccesskey awssecretaccesskey = defaultsecretkey [alt] awsaccesskeyid = altaccesskey awssecretaccesskey = altsecretkey

Now, let's run through the two situations!

## Setting Boto To A Non-Default Profile
For the first case, we'll say that this is a personal project, so instead of the `default` profile, we want to set it to `personal`:

import boto3


s3 = boto3.resource('s3')
for bucket in s3.buckets.all():
    print(f'personal buckets are -->{}')

Using Multiple Profiles Simultaneously

If the situation were slightly different and instead of different domains (work, personal), the AWS profiles referenced different environments (dev, qa, prod) , we can imagine doing the following:
dev = boto3.session.Session(profile_name='dev')
qa = boto3.session.Session(profile_name='qa')
prod = boto3.session.Session(profile_name='prod')

for session in [dev, qa, prod]:
    s3 = session.resource('s3')
    for bucket in s3.buckets.all():
        print(f'{session} bucket is named --> {})

Hi there and thanks for reading! My name's Stephen. I live in Chicago with my wife, Kate, and dog, Finn. Want more? See about and get in touch!