New issue with backblaze B2: `RoundTrip()` error in debug log

Hello,

I have been a very happy restic + B2 backup user for a couple years now. But for maybe the last week (I would guess after an OS update, but I am not sure), I am no longer able to connect to my B2 bucket.

First, my restic version is restic 0.17.3 compiled with go1.23.3 on linux/amd64. To access my bucket, I have the following workflow:

I have /etc/restic/env, which looks like

export B2_ACCOUNT_ID="$(pash s restic/b2accid)"
export B2_ACCOUNT_KEY="$(pash s restic/b2acckey)"
export RESTIC_REPOSITORY="b2:[bucket name]"
export RESTIC_PASSWORD="$(pash s restic/[bucket name])"
export DEBUG_LOG=/home/cg/Downloads/restic.log # new addition

And then I have programs /usr/local/bin/erestic as

#!/bin/bash

source /etc/restic/env
restic "$@"

and finally my actual backup script of /usr/local/bin/full_restic_backup, given by

#!/bin/bash

dryflag="${1:invalid}"

/usr/local/bin/promptconf "Run a full system backup with restic?" || exit 0

case "$dryflag" in
  "dry" )
    echo "Running dry-run backup..."
    /usr/local/bin/erestic backup --dry-run --exclude-file /home/cg/.freshbox/restic_exclude.txt /home/cg/ ;;
  "run" )
    echo "Running backup..."
    /usr/local/bin/erestic -vv backup --exclude-file /home/cg/.freshbox/restic_exclude.txt /home/cg/ ;;
  * )
    echo "Invalid input. Please say "dry" for dry run, and "run" for actual backup." ;;
esac

I run my backups with

full_restic_backup run

which has been my go-to for a couple years at this point.

Once it started hanging, I tried a few things.

  1. Based on a few posts here, I tried changing from b2 to s3 by generating a new account key and checking that extra box that is necessary for s3 and changing the line in /etc/restic/env that specifies my bucket:
export RESTIC_REPOSITORY="s3:s3.us-east-005.backblazeb2.com/[bucket name]"

But when I then run

erestic -vv snapshots

it still hangs.

  1. I have also tried creating a debug log file. Switching back to the b2 access method, my restic.log looks like this (with many repeated sections about TLS issues removed)
2025/01/26 17:54:48 restic/main.go:137	main.main	1	main []string{"restic", "-vv", "snapshots"}
2025/01/26 17:54:48 restic/main.go:138	main.main	1	restic 0.17.3 compiled with go1.23.3 on linux/amd64
2025/01/26 17:54:48 restic/global.go:583	main.innerOpen	1	parsing location b2:[bucket name]
2025/01/26 17:54:48 restic/global.go:578	main.parseConfig	1	opening b2 repository at &b2.Config{AccountID:[AccountID], Key:"**redacted**", Bucket:"[bucket name]", Prefix:"", Connections:0x5}
2025/01/26 17:54:48 b2/b2.go:92	b2.Open	1	cfg b2.Config{AccountID:[AccountID], Key:"**redacted**", Bucket:"[bucket name]", Prefix:"", Connections:0x5}
2025/01/26 17:54:48 debug/round_tripper.go:93	debug.loggingRoundTripper.RoundTrip	1	------------  HTTP REQUEST -----------
GET /b2api/v1/b2_authorize_account HTTP/1.1
Host: api.backblazeb2.com
User-Agent: blazer/0.6.1
Authorization: **redacted**
X-Blazer-Method: b2_authorize_account
X-Blazer-Request-Id: 1
Accept-Encoding: gzip


2025/01/26 17:54:49 debug/round_tripper.go:110	debug.loggingRoundTripper.RoundTrip	1	------------  HTTP RESPONSE ----------
HTTP/1.1 200 
Content-Length: 1017
Cache-Control: max-age=0, no-cache, no-store
Connection: keep-alive
Content-Type: application/json;charset=UTF-8
Date: Sun, 26 Jan 2025 23:51:37 GMT
Server: nginx
Strict-Transport-Security: max-age=63072000


2025/01/26 17:54:49 debug/round_tripper.go:93	debug.loggingRoundTripper.RoundTrip	1	------------  HTTP REQUEST -----------
POST /b2api/v1/b2_list_buckets HTTP/1.1
Host: api005.backblazeb2.com
User-Agent: blazer/0.6.1
Content-Length: 98
Authorization: **redacted**
X-Blazer-Method: b2_list_buckets
X-Blazer-Request-Id: 2
Accept-Encoding: gzip


2025/01/26 17:54:49 debug/round_tripper.go:110	debug.loggingRoundTripper.RoundTrip	1	------------  HTTP RESPONSE ----------
HTTP/1.1 200 
Content-Length: 861
Cache-Control: max-age=0, no-cache, no-store
Connection: keep-alive
Content-Type: application/json;charset=UTF-8
Date: Sun, 26 Jan 2025 23:51:37 GMT
Server: nginx
Strict-Transport-Security: max-age=63072000


2025/01/26 17:54:49 logger/log.go:52	logger.(*Backend).Stat	1	Stat(<config/>)
2025/01/26 17:54:49 sema/semaphore.go:27	sema.semaphore.GetToken	1	acquired token
2025/01/26 17:54:49 debug/round_tripper.go:93	debug.loggingRoundTripper.RoundTrip	1	------------  HTTP REQUEST -----------
HEAD /file/[bucket name]/config HTTP/1.1
Host: f005.backblazeb2.com
User-Agent: blazer/0.6.1
Authorization: **redacted**
X-Blazer-Method: b2_download_file_by_name
X-Blazer-Request-Id: 3


2025/01/26 17:54:49 debug/round_tripper.go:100	debug.loggingRoundTripper.RoundTrip	1	RoundTrip() returned error: tls: first record does not look like a TLS handshake
2025/01/26 17:54:50 debug/round_tripper.go:93	debug.loggingRoundTripper.RoundTrip	1	------------  HTTP REQUEST -----------
HEAD /file/[bucket name]/config HTTP/1.1
Host: f005.backblazeb2.com
User-Agent: blazer/0.6.1
Authorization: **redacted**
X-Blazer-Method: b2_download_file_by_name
X-Blazer-Request-Id: 4


2025/01/26 17:54:50 debug/round_tripper.go:100	debug.loggingRoundTripper.RoundTrip	1	RoundTrip() returned error: tls: first record does not look like a TLS handshake
2025/01/26 17:54:51 debug/round_tripper.go:93	debug.loggingRoundTripper.RoundTrip	1	------------  HTTP REQUEST -----------
HEAD /file/[bucket name]/config HTTP/1.1
Host: f005.backblazeb2.com
User-Agent: blazer/0.6.1
Authorization: **redacted**
X-Blazer-Method: b2_download_file_by_name
X-Blazer-Request-Id: 5


2025/01/26 17:54:52 debug/round_tripper.go:100	debug.loggingRoundTripper.RoundTrip	1	RoundTrip() returned error: tls: first record does not look like a TLS handshake
2025/01/26 17:54:53 debug/round_tripper.go:93	debug.loggingRoundTripper.RoundTrip	1	------------  HTTP REQUEST -----------
HEAD /file/[bucket name]/config HTTP/1.1
Host: f005.backblazeb2.com
User-Agent: blazer/0.6.1
Authorization: **redacted**
X-Blazer-Method: b2_download_file_by_name
X-Blazer-Request-Id: 6


2025/01/26 17:54:53 debug/round_tripper.go:100	debug.loggingRoundTripper.RoundTrip	1	RoundTrip() returned error: tls: first record does not look like a TLS handshake
2025/01/26 17:54:54 debug/round_tripper.go:93	debug.loggingRoundTripper.RoundTrip	1	------------  HTTP REQUEST -----------
HEAD /file/[bucket name]/config HTTP/1.1
Host: f005.backblazeb2.com
User-Agent: blazer/0.6.1
Authorization: **redacted**
X-Blazer-Method: b2_download_file_by_name
X-Blazer-Request-Id: 7


[...]

2025/01/26 17:56:12 debug/round_tripper.go:100	debug.loggingRoundTripper.RoundTrip	1	RoundTrip() returned error: tls: first record does not look like a TLS handshake
2025/01/26 17:56:12 restic/cleanup.go:25	main.cleanupHandler	54	signal interrupt received, cleaning up
2025/01/26 17:56:12 restic/global.go:254	main.Warnf	54	e[2Ksignal interrupt received, cleaning up
2025/01/26 17:56:12 logger/log.go:54	logger.(*Backend).Stat	1	  stat err Stat: context canceled
2025/01/26 17:56:12 logger/log.go:24	logger.(*Backend).IsNotExist	1	IsNotExist(*errors.withStack, Stat: context canceled, false)
2025/01/26 17:56:12 restic/cleanup.go:39	main.Exit	1	exiting with status code 1

To be honest, I do not know how to interpret the error message from RoundTrip(), and when I google it I don’t see anything that seems obviously relevant.

  1. I have also looked into whether Backblaze recently updated its SSL certificate (?), something I do not understand but see from earlier forum posts here has at least been relevant in the past. It doesn’t seem like there is any chatter on forums about this recently, though, and so it seems unlikely to be the problem here.

If anybody has any thoughts or suggestions, I would greatly appreciate hearing them! I would not be surprised at all to learn that this is a basic mistake about something I really shouldn’t be so ignorant about. And please accept my apologies if this question is a dupe of an earlier forum post that I either missed or read and didn’t understand to be the same problem.

Thank you for taking time to provide detailed information!

You could use tcpdump to inspect the network traffic - if the response isn’t valid TLS then you might be able to spot what it is instead.

This could be an issue with your router or ISP - what are they? Can you try the connection over a different Internet connection, router and ISP?

At the end of the day, it’s rather likely that it is a problem between your client and B2, considering the same thing happens both with the B2 and S3 protocols. Here’s one and two examples of that.

Finally, can you try to access (e.g. list contents of) your bucket rclone, using both B2 and S3?

@rawtaz, thanks very much for your response and tips for next steps in investigating! I’ll have to learn about a few things to do those investigations, so it may take a few days or a week but I will be back with some follow-up and hopefully answers!

1 Like

Backblaze would definitely have noticed if they broke some TLS certificate, so the problem is most likely in your local setup or your ISP. The debug log shows that accessing f005.backblazeb2.com yields an error, so most likely it is broken due to some DNS block list that thought it is a good idea to once again blocklist some central Backblaze server (we had a few such cases over the years).

The simplest way to debug this is probably to manually try to access the same file as restic did:

curl https://f005.backblazeb2.com/file/[bucket name]/config

This should yield an error that the request is not authenticated. However, in your case you should see a TLS error.

What does host f005.backblazeb2.com return? I get the following:

❯ host f005.backblazeb2.com                                                
f005.backblazeb2.com has address 149.137.136.16
❯ host 149.137.136.16      
16.136.137.149.in-addr.arpa domain name pointer f005.backblazeb2.com.
1 Like

@MichaelEischer, thanks very much! You and @rawtaz were exactly right—it clearly is an ISP problem. I just tried backing up at the office and everything worked as usual. My ISP is Spectrum, so I guess I’ll have to start googling around and see what kind of firewall settings I can toggle. I’ll update the thread again when I find the right switch to flip in the app or on the phone with them.

Thank you both so much again for your help with this! I am very relieved to have my backups functional again and will try to provide the most useful postmortem I can for future users shortly.

2 Likes