Backing Up Virtualmin Server Using S3 to Minio Via Nginx Gives Error

Wow that’s a long title. Nevertheless, I’ve just battled this problem and putting out here partially for my own reasons when I inevitably come across this again and partially in case it helps anyone else.

The Scenario

You have a server, somewhere in the world, running Webmin / Virtualmin. You wish to back that server up every so often (always a good idea). You use MinIO to do provide the S3 storage capacity on a different server somewhere else. This lets you use the Virtualmin backup system with Amazon S3 buckets.

Now, the Virtualmin backup system to Amazon S3 works out of the box if you’re backing up to Amazon themselves. But if you’re using MinIO you’ll find you cannot configure the Virtualmin backup plugin to allow a port number in the Amazon S3 Cloud Storage Provider settings screen.

This is why you’ll need an NGINX reverse proxy as well. That way, you can put the NGINX server on Port 443, proxy_pass the requests through to port 9000 for your MinIO server.

I’m not going into how to do that – it’s available in the MinIO Documentation if you need to know how to set that part up.

It Used To Work, And Now It’s Broken

So I migrated my MinIO from a desktop computer to a rather nice Dell R710 server that I acquired for my Homelab. The server runs Proxmox VE 7.0 and is excellent for my usage. Actually it’s brilliant for my uses – but I digress. I’ll write about my experiences there in another article at another time.

Nevertheless, on the desktop machine I had an NGinx reverse proxy running in front of the MinIO server which all ran in Docker containers. I now have them running in LXC containers instead – for better separation and control over processor and RAM usage. It is an enterprise server it’s running on after all.

And somewhere along the way, the whole setup stopped allowing Virtualmin to backup.

The First Problem – Uploads Are Too Big

This one is fairly trivial. You need to set the client_max_body_size 0 ; parameter in the Nginx configuration file. The MinIO documentation will tell you that.

The Second Problem – Failed To Upload Information File

This is the bit I spent hours resolving. And there’s no answer on the internet despite MULTIPLE bug requests and forum questions. I would answer these specific questions on the actual forums I visited except for 2 reasons;

• I can’t remember all the places I found the questions at now
• Almost every single bug report or forum automatically closes the thread when it becomes inactive. Even if it remains unanswered. Whose dumb idea is that?

Anyway, the answer is a simple one line change to your Nginx configuration for the reverse proxy.

Update Your NGINX Proxy_Pass statement

This article from Atlassian had absolutely no link to MinIO but I stumbled across it and it gave me inspiration to try the solution they suggested for a different, but similarly sounding problem.

I tried it and it worked.

Update your Nginx reverse proxy configuration thusly;

proxy_pass http://192.168.0.xxx:9000$request_uri ;

Note that you’ll need, of course, to replace the IP address with the address of your MinIO server. And then note the addition of the $request_uri parameter on the end.

This will resolve the problem for you, which is that for some reason, without this, Nginx will generate a 403 error for the tar.gz.info and tar.gz.dom files.

If you’re having the same problem as I was, you should see errors in your nginx_domain_access.log – they’ll likely be similar to this;

xx.xx.xx.xx - - [22/Aug/2021:15:10:31 +0000] "PUT /redacted/test%2Fsomedomain%2Eco%2Euk%2Etar%2Egz%2Einfo HTTP/1.1" 403 451 "-" "libwww-perl/6.43"

The 403 is the problem – Nginx is forbidding the request – without even passing it to the MinIO server. The fix above should sort it.

Summary

When Virtualmin backups fail with the error upload failed! Failed to upload information file : The request signature we calculated does not match the signature you provided. Check your key and signing method and you’re using an Nginx server reverse proxy as a front end for MinIO, then you may need to adjust the proxy_pass directive as shown above.

If this has been helpful for you please drop me a comment below, or if you have any other questions please feel free to ask.

If it’s been really helpful for me, and you’re using Brave Browser, please consider a small BAT donation using the Brave Rewards button in your browser bar. Thanks!

Dell R710 Server for Homelab Use

I’m just going to add a quick bit on the end here, although I’ll probably do a full article later in case anyone is interested in the Dell R710 Server. I run Proxmox VE 7 on it currently and it’s handling absolutely everything I’ve thrown at it, including streaming 4 separate movies, across the internet, requiring transcoding – which it has to do in software.

Do be aware, if you’re wanting to run any hardware passthrough it doesn’t look like the R710 BIOS supports this properly and later Linux kernels will disable the IOMMU because it’s unstable. I’ve yet to find a solution – which is annoying because I bought a GPU card specifically for the job and then discovered this. But everything else the R710 is brilliant for.

If you’re in the UK, the links to Dell R710 server in this article will take you to Contitel’s eBay store, which is who I bought mine through. Contitel were excellent and I’d highly recommend them. The links are affiliate links so if you purchase as a result, the site may generate a small commission for us, for which we thank you.