I just want to start over... (repo lost, can't init new)

Getting Help
1

OK, I’ve done some admittedly dumb things while getting acquainted with Restic, and I just want to start over.

I (was) backing up to a minio s3 bucket, and the disk where the bucket was stored was compromised, losing the bucket. I have recreated a completely new bucket on a new disk. I do have the original aws keys as well as the restic password, the repo doesn’t exist. I have to use new aws keys for the new bucket.

All I want to do is init on a new repo, but all I get is:

Fatal: create key in repository at s3:https:///bucket failed: repository master key and config already initialized

Tried with the same bucket name as well as a new bucket name. Where is this stored? How can I just forget the prior repo and start over?

Thanks

2

Please provide the full command you run and all of its output, when you get that message.

3

Thanks for the reply.

First I initialize variables:
export AWS_DEFAULT_REGION=“region”
export RESTIC_REPOSITORY=“s3:https://destination.net/backups2”
export AWS_ACCESS_KEY_ID=“mykey”
export AWS_SECRET_ACCESS_KEY=“mysecretkey”
export RESTIC_PASSWORD=“myresticpassword”

and then:

restic init

Fatal: create key in repository at s3:https://location.net/backups2 failed: repository master key and config already initialized

Any other restic command (restic backup, restic check, etc.) results in:
Fatal: wrong password or no key found

I have also removed the cache directory somewhere along the way.

4

Hm, interesting! So…

  • On the one hand you said that you created a completely new bucket on a new disk. This ought to mean that (assuming you try to create a new repository in that new bucket) there should be no previous restic related files there already.

  • On the other hand you say that you get this error message, which suggests that there is a config file in that bucket, seeing as the only occurrence of that message is here which should only show if the file exists.

  • Also, the other error messages also strongly indicates that restic does find a repository in the place it is looking.

I noticed in your information above that you get location.net in the output, but in your environment variables you have destination.net. Can you make sure to check the env vars so that they really are what you think? I mean, the simplest explanation to what you are seeing is that your restic init is still getting env vars that directs it to the old bucket somehow (although I don’t know if it’s even possible).

Can you use an S3 client to look into the bucket and see if you can find any files there? It would make sense if you could find a config file there.

Otherwise, can you please run DEBUG_LOG=debug.log restic init instead and paste the log file here (unless you spot something that explains it all in it yourself)?

5

Thank you, good catch my inconsistency. I was redacting the servername in the URL but was inconsistent. I double-checked my names and also the public/private keys and they’re good.

The minio web UI and mc command-line shows nothing in the bucket. I also can traverse the directory on the server, and it’s empty, even of hidden files (ls -al).

This is an odd one to me, hence my attempt to “just start over” is even getting difficult.

Pasting the debug.log in a separate post to keep it easier to read.

6

2023/03/09 18:46:28 restic/main.go:93 main.main 1 main []string{“restic”, “init”}
2023/03/09 18:46:28 restic/main.go:94 main.main 1 restic 0.15.0 compiled with go1.19.5 on linux/arm64
2023/03/09 18:46:28 restic/global.go:771 main.create 1 parsing location s3:https://deanlester.duckdns.org:2589/backups2
2023/03/09 18:46:28 restic/global.go:594 main.parseConfig 1 opening s3 repository at s3.Config{Endpoint:“deanlester.duckdns.org:2589”, UseHTTP:false, KeyID:“4WWrdhUyupVHhcNh”, Secret:“redacted”, Bucket:“backups2”, Prefix:“”, Layout:“”, StorageClass:“”, Connections:0x5, MaxRetries:0x0, Region:“homes”, BucketLookup:“”, ListObjectsV1:false}
2023/03/09 18:46:28 s3/s3.go:40 s3.open 1 open, config s3.Config{Endpoint:“deanlester.duckdns .org:2589”, UseHTTP:false, KeyID:“4WWrdhUyupVHhcNh”, Secret:“redacted”, Bucket:“backups2”, Prefix:“”, Layout:“”, StorageClass:“”, Connections:0x5, MaxRetries:0x0, Region:“homes”, BucketLookup:“”, ListObjectsV1:false}
2023/03/09 18:46:28 layout/layout.go:139 layout.ParseLayout 1 parse layout string “” for backend at
2023/03/09 18:46:28 layout/layout.go:99 layout.DetectLayout 1 detect layout at
2023/03/09 18:46:28 s3/s3.go:200 s3.(*Backend).ReadDir 1 ReadDir(keys)
2023/03/09 18:46:28 s3/s3.go:210 s3.(*Backend).ReadDir 1 using ListObjectsV1(false)
2023/03/09 18:46:28 debug/round_tripper.go:93 debug.loggingRoundTripper.RoundTrip 9 ------------ HTTP REQUEST -----------
GET /backups2/?delimiter=%2F&encoding-type=url&fetch-owner=true&list-type=2&prefix=keys%2F HTTP/1.1
Host: deanlester.duckdns .org:2589
User-Agent: MinIO (linux; arm64) minio-go/v7.0.46
Authorization: redacted
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20230309T234628Z
Accept-Encoding: gzip

2023/03/09 18:46:29 debug/round_tripper.go:110 debug.loggingRoundTripper.RoundTrip 9 ------------ HTTP RESPONSE ----------
HTTP/1.1 200 OK
Content-Length: 300
Accept-Ranges: bytes
Connection: keep-alive
Content-Security-Policy: block-all-mixed-content
Content-Type: application/xml
Date: Thu, 09 Mar 2023 23:46:29 GMT
Server: nginx/1.23.3
Strict-Transport-Security: max-age=31536000; includeSubDomains
Vary: Origin
Vary: Accept-Encoding
X-Amz-Id-2: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Request-Id: 174AE4FDA4525E02
X-Content-Type-Options: nosniff
X-Xss-Protection: 1; mode=block

2023/03/09 18:46:29 s3/s3.go:200 s3.(*Backend).ReadDir 1 ReadDir(key)
2023/03/09 18:46:29 s3/s3.go:210 s3.(*Backend).ReadDir 1 using ListObjectsV1(false)
2023/03/09 18:46:29 debug/round_tripper.go:93 debug.loggingRoundTripper.RoundTrip 39 ------------ HTTP REQUEST -----------
GET /backups2/?delimiter=%2F&encoding-type=url&fetch-owner=true&list-type=2&prefix=key%2F HTTP/1.1
Host: deanlester.duckdns .org:2589
User-Agent: MinIO (linux; arm64) minio-go/v7.0.46
Authorization: redacted
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20230309T234629Z
Accept-Encoding: gzip

2023/03/09 18:46:29 debug/round_tripper.go:110 debug.loggingRoundTripper.RoundTrip 39 ------------ HTTP RESPONSE ----------
HTTP/1.1 200 OK
Content-Length: 299
Accept-Ranges: bytes
Connection: keep-alive
Content-Security-Policy: block-all-mixed-content
Content-Type: application/xml
Date: Thu, 09 Mar 2023 23:46:29 GMT
Server: nginx/1.23.3
Strict-Transport-Security: max-age=31536000; includeSubDomains
Vary: Origin
Vary: Accept-Encoding
X-Amz-Id-2: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Request-Id: 174AE4FDA4AB1C48
X-Content-Type-Options: nosniff
X-Xss-Protection: 1; mode=block

2023/03/09 18:46:29 layout/layout.go:132 layout.DetectLayout 1 layout detection failed
2023/03/09 18:46:29 layout/layout.go:156 layout.ParseLayout 1 error: auto-detecting the filesystem layout failed, use default layout default
2023/03/09 18:46:29 layout/layout.go:139 layout.ParseLayout 1 parse layout string “default” for backend at
2023/03/09 18:46:29 debug/round_tripper.go:93 debug.loggingRoundTripper.RoundTrip 1 ------------ HTTP REQUEST -----------
HEAD /backups2/ HTTP/1.1
Host: deanlester.duckdns .org:2589
User-Agent: MinIO (linux; arm64) minio-go/v7.0.46
Authorization: redacted
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20230309T234629Z

2023/03/09 18:46:29 debug/round_tripper.go:110 debug.loggingRoundTripper.RoundTrip 1 ------------ HTTP RESPONSE ----------
HTTP/1.1 200 OK
Accept-Ranges: bytes
Connection: keep-alive
Content-Security-Policy: block-all-mixed-content
Content-Type: application/xml
Date: Thu, 09 Mar 2023 23:46:29 GMT
Server: nginx/1.23.3
Strict-Transport-Security: max-age=31536000; includeSubDomains
Vary: Origin
Vary: Accept-Encoding
X-Amz-Id-2: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Request-Id: 174AE4FDA4E7A4AB
X-Content-Type-Options: nosniff
X-Xss-Protection: 1; mode=block
Content-Length: 0

2023/03/09 18:46:29 index/index.go:389 index.(*Index).Finalize 1 finalizing index
2023/03/09 18:46:29 s3/s3.go:359 s3.(*Backend).Stat 1
2023/03/09 18:46:29 debug/round_tripper.go:93 debug.loggingRoundTripper.RoundTrip 65 ------------ HTTP REQUEST -----------
HEAD /backups2/config HTTP/1.1
Host: deanlester.duckdns .org:2589
User-Agent: MinIO (linux; arm64) minio-go/v7.0.46
Authorization: redacted
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20230309T234629Z

2023/03/09 18:46:29 debug/round_tripper.go:110 debug.loggingRoundTripper.RoundTrip 65 ------------ HTTP RESPONSE ----------
HTTP/1.1 200 OK
Content-Length: 155
Accept-Ranges: bytes
Connection: keep-alive
Content-Security-Policy: block-all-mixed-content
Content-Type: application/octet-stream
Date: Thu, 09 Mar 2023 23:46:29 GMT
Etag: “df1a665091a61b9e5ae21e1b89b9fb80”
Last-Modified: Thu, 09 Feb 2023 20:08:27 GMT
Server: nginx/1.23.3
Strict-Transport-Security: max-age=31536000; includeSubDomains
Vary: Origin
Vary: Accept-Encoding
X-Amz-Id-2: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Request-Id: 174AE4FDA524305E
X-Amz-Version-Id: a01898f7-10b9-4981-ae8c-70d1891dc568
X-Content-Type-Options: nosniff
X-Xss-Protection: 1; mode=block

7

forum is restricting my ability to include links in posts. I added a space before .org in all occurrences to break the link syntax.

8

Looks like there is a config file here and minio returns it. I assume you have single instance of minio only, serving from that port. In order to prevent env variable confusions I’d suggest working on the same terminal.
Also could you send mc ls --debug output for that bucket? You can send preformatted/code inside triple backticks “`” or just pressing ctrl-e here.

9

So I have a resolution, but I’m confused about the root cause.

One additional fact I didn’t think was relevant: I had been (and intend to continue) replicating this bucket to another site on another minio server (one way). When I recreated the new bucket, I immediately configured the one-way replication from source to target, and then went to set up the restic repo, which instantly failed.

Interestingly, disabling the minio replication rule allows the repo to initialize. Re-enabling the replication rule, the repo is still usable for backups.

I wouldn’t have expected one-way minio bucket replication to have anything to do with restic’s config files in the bucket. Is there something i’m overlooking?