Multiple sets of S3 credentials / profiles

Getting Help
1

First off, I’d like to express my gratitude to the devs and contributors for the incredible piece of software that they made available for everyone to use for free. I compared some of the available backup options and decided to go with Restic and have absolutely no regrets, best decision I ever made.

I do have a question about the possibility of using multiple sets of S3-compatible credentials at the same time. Reason for this is I’m backing up both to AWS as well as Backblaze B2 (using its S3 API) so in order to prep for this I did the following:

  • configured AWS with an IAM role and bucket policy to allow proper access for said IAM role

  • created the ~/.aws/credentials file in my backup client home folder per documentation at AWS regarding CLI authentication. This works nicely and I’m able to avoid having to export env variables for improved security (restic uses the “aws_access_key_id” and “aws_secret_access_key” parameters from the credentials file - neat!)

  • created a B2 bucket and keyID / accessKey with proper permissions for said B2 bucket. Again this works nicely with B2 if I replace the “aws_access_key_id” and “aws_secret_access_key” in the ~/.aws/credentials file for the B2 keyID / accessKey

But if I need to backup both, I must edit the credentials file before each restic run, so the keys match the provider / repository / bucket being accessed :frowning: I looked up the options in the ~/.aws/credentials and it appears it accepts multiple “profiles” (ie - default and user1) but no way to indicate this to restic at runtime…

Am I missing something obvious? Any insights on how this can be used efficiently?

TIA

1 Like
2

Hi there,
i have no experince with two S3 Instances. And i have no clue what exactly the “~/.aws/credentials” file is neede for. But i can tell you that I perhaps have a solution that should work for you.

You could set up a “config file” lets say: ~/.restic_s3_aws.config

# credentials for: aws
export AWS_ACCESS_KEY_ID=aaabbbcccdddeeefff
export AWS_SECRET_ACCESS_KEY=121212121212121212121212121212
export RESTIC_PASSWORD=558585858585858585858
export RESTIC_REPOSITORY="s3:s3.eu-central-1.wasabisys.com/Workstation1-bucket"

(of course you should change the pararmeters for your needs)

finally you start the backup with:

source ~/.restic_s3_aws.config; restic backup $HOME

Voila, you just need to set up a second “.config file” and source the second one to make the second backup.

As i told: if have no clue what the “~/.aws/credentials” file is good for. Perhaps you have to ‘source’ that file after everything got backuped twice. You should have an eye on this.

3

Thanks for replying! AWS credentials is part of security configuration, and I’m by no means an expert but it appears restic is able to use the same credentials as the AWS CLI. Basically a file named /home/username/.aws/credentials can be used to hold the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY values (and probably others too) and restic uses them if found there.

Your solution could work too, but for the time being I went with creating this file as a symbolic link to two separate files (credentials.aws and credentials.b2) and update the link as part of the backup script

4

It looks like setting AWS_PROFILE should do it.

The wrapper approach might be a little cleaner.

5

Setting the AWS_PROFILE looks like a more elegant way to me, but I don’t know if I can set (and then change) that within the same script - ie without source-ing or spawning a new shell. I’m using a single backup script that runs restic repeatedly and targeting a few different repositories (S3 and B2 being two of them)

Re: the wrapper approach - if I understand it correctly it means (re)defining / (re)exporting env variables, but does that work as part of a single script?

Thanks!

6

Replying to myself here… What I ended up doing was "wrapping the restic command in the backup script as follows, which achieves exactly what I wanted (ie leaves no env variable set behind, so the default profile isn’t switched away from):

...
export AWS_PROFILE=backblaze
restic ...
unset AWS_PROFILE
...

Thanks to @dsissitka for suggesting the AWS_PROFILE idea

7

An even simpler syntax would be the following:

AWS_PROFILE=backblaze restic ...

That way the environment variable is only set for this specific call to restic.

1 Like