perpetually slow B2 uploads with 10 gig fiber

[peter.conheim]

I'm entirely unable to improve a very slow upload situation to my Backblaze B2, no matter how many Mac OS apps and command line GUIs I try (i.e. rclone). On my 10 gig connection, I've never gotten above 100 MB/s, and if it starts at that level, it inevitably plunges to 30-40 MB/s after a few hours of continuous upload. Most consistent client has been Syncovery; Cyberduck is ridiculously slow for me, Duplicati is buggy, Transmit often dumps mid-transfer.

Backblaze claims they do "no throttling" but I think it is more complicated and sneaky than that. In any event, has anyone had any success with any particular client or method that can at least remotely take advantage of the line speed? Because I find the situation absurd. I routinely have multi-terabyte backups or uploads in a given month.

[ngufra]

100MB/s is 1 Gbps. Not 10 but still not really "slow".
Have you read https://www.backblaze.com/computer-back ... er-uploads in particular:

If you have a faster internet connection and you want to maximize Backblaze's utilization of your bandwidth, you can disable the automatic threading/throttle setting and slide the manual throttle setting to the right to back up faster.
If you set the threads too high, it can use all available bandwidth of the network and can cause timeouts and other connection issues across the network. Start with fewer connections and go up accordingly.

[peter.conheim]

100MB/s is 1 Gbps. Not 10 but still not really "slow".
Have you read https://www.backblaze.com/computer-back ... er-uploads in particular:

If you have a faster internet connection and you want to maximize Backblaze's utilization of your bandwidth, you can disable the automatic threading/throttle setting and slide the manual throttle setting to the right to back up faster.
If you set the threads too high, it can use all available bandwidth of the network and can cause timeouts and other connection issues across the network. Start with fewer connections and go up accordingly.

That's not relevant to B2. That's only relevant to their other storage platform.

1Gbps is still very slow compared to the line speed, and when you are dealing with uploads over 1Tb, it makes quite the difference. So still very much looking for options to speed this up.

[ngufra]

Is the data from memory or read from a hard drive/ssd?
How fast is your hard drive?
Try to copy data from a SSD see if it makes a difference

[ngufra]

I have fond memories of a time when hard drives went finally faster that 1MB per second. Mind you the drive capacity was probably 200MB

[peter.conheim]

Is the data from memory or read from a hard drive/ssd?
How fast is your hard drive?
Try to copy data from a SSD see if it makes a difference

Excellent question. On the slowest end, I am backing up hard drives which won't read much faster than 175 MB/s. But with the SSDs I use, the sky is the limit. So the issue shouldn't be with my machine's throughput.

With more testing, I am getting consistent 100-ish using web-based rclone interface, but it's clunky and I can't upload folders, only individual files, so not very helpful to me. (And of course, that's still quite a bit slower than I would want.)

[ngufra]

Have you tried to ask Backblaze?
Maybe on their side it's writing to a single hard drive
Maybe there is something on their end they can change for free or for a price.

Maybe Sonic (ASN 46375) and Backblaze (ASN 40401) can answer abourt specific peering between their networks.
They seem to both be interconnected at Equinix San Jose https://www.peeringdb.com/fac/6

[peter.conheim]

Have you tried to ask Backblaze?
Maybe on their side it's writing to a single hard drive
Maybe there is something on their end they can change for free or for a price.

Maybe Sonic (ASN 46375) and Backblaze (ASN 40401) can answer abourt specific peering between their networks.
They seem to both be interconnected at Equinix San Jose https://www.peeringdb.com/fac/6

I've been back and forth with Backblaze on this with no answers. Different Sonic folks made different suggestions, including one saying to try another DNS (so I switched to OpenDNS... no discernible improvement).

[ngufra]

Some reading you may have already found:
https://www.reddit.com/r/backblaze/comm ... kblaze_b2/
https://www.backblaze.com/blog/2023-per ... rovements/

So make sure you use multiple connections to saturate your pipe.

[smbrannon]

With more testing, I am getting consistent 100-ish using web-based rclone interface, but it's clunky and I can't upload folders, only individual files, so not very helpful to me. (And of course, that's still quite a bit slower than I would want.)

I suggest you become familiar with rclone's CLI. I use it to sync to OneDrive, and have found it pretty robust. You can definitely sync directories from the CLI.

However, I think perhaps your expectations might be a bit high for what you'll get across the Internet. There's a lot between you and BackBlaze to cause slow downs.