Amanda 3.4.1 and AWS S3 Signature Version 4 (AWS4-HMAC-SHA256)

[Rustam]

Hello,

I successfully configured amanda backup with Signature Version 2 in Amazon. Now I'm trying backup with Signature Version 4 because I need backup in another zone which supports only Version 4. And unsuccessful. The only difference in my configuration is an added param 'STORAGE_API' which I set to 'AWS4'.

The service error is


slot 1: While creating new S3 bucket: The request signature we calculated does not match the signature you provided. Check your key and signing method. (SignatureDoesNotMatch) (HTTP 403)


/etc/amanda/amanda.conf


org "my_backup"

infofile "/var/lib/amanda/state/curinfo"
logdir "/var/lib/amanda/state/log"
indexdir "/var/lib/amanda/state/index"
dumpuser "amandabackup"

device_property "S3_ACCESS_KEY" "XXXXXXXXXXXXXXXXXXXXX"
device_property "S3_SECRET_KEY" "SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS"
device_property "S3_SSL" "YES"

device_property "S3_HOST" "s3.amazonaws.com:443"
device_property "STORAGE_API" "AWS4"
device_property "VERBOSE" "YES"

define changer "S3" {
comment "S3 changer"
changerfile "s3-statefile"
tpchanger "chg-multi:s3:myrar-backup/test_backup/slot-{01,02,03,04,05,06,07,08,09,10}"
}

tpchanger "S3"

autolabel "$c-$2s" volume-error empty

tapecycle 7
dumpcycle 5

tapetype "S3"
define tapetype S3 {
comment "S3 Backup Bucket"
length 10240 gbytes
}

define dumptype client-fast-gnutar-ssh {
auth "ssh"
ssh_keys "/var/lib/amanda/.ssh/amandabackup_id_rsa"
compress client fast
program "GNUTAR"
}

holdingdisk hd1 {
directory "/var/lib/amanda/holding"
use 500 mbytes
chunksize 1 mbyte
}


Tested on Centos 7.2. As it worked "STORAGE_API" "S3" I suppose that amanda configuration and amazon permissions are correct.

Any thoughts how to solve this problem?

[martineau]

Try to emove the ":443" in the s3-host device_property.

[Rustam]

I have a suspicions that AWS4 doesn't work at all.

I tried with unset parameters S3_HOST, S3_BUCKET_LOCATION and STORAGE_API and it works. By default it should be AWS4. Or not? From doc:


STORAGE_API
(read-write) Which API to use for the cloud:

S3 Amazon S3 AWS Signature Version 2
AWS4 Amazon S3 AWS Signature Version 4
SWIFT-1.0 Openstack swift v1.0
SWIFT-2.0 Openstack swift v2.0
OAUTH2 Google
CASTOR Caringo CAStor

The default is AWS4 if S3-HOST end with '.amazonaws.com', otherwise it is S3.


But when I set STORAGE_API to "AWS4" (with unset S3_HOST which by default is s3.amazonaws.com) I have SignatureDoesNotMatch error.
I attached taper logs with successful and unsuccessful dump. Both supposed to use AWS4, but I can see that they are different.

Also seems the doc is not quite correct, at least for S3_HOST parameter


S3_HOST
(read-write) The host name to connect, in the form "hostname:port" or "ip:port", default is "s3.amazonaws.com"

As I understood it doesn't work for AWS4. Is it correct?

[martineau]

The problem is because of the / character in the prefix

tpchanger "chg-multi:s3:myrar-backup/test_backup/slot-{01,02,03,04,05,06,07,08,09,10}"

The / between test_backup and slot...

The attached patch fix this issue

hunk 1,2,4,5: Fix when a port is set in the S3-HOSTproperty
hunk 3: Add debugging statement
hunk 6: Fix for / in the prefix

[martineau]

STORAGE_API
The default is AWS4 if S3-HOST end with '.amazonaws.com', otherwise it is S3.




It should be rewritten as:


STORAGE_API
The default is AWS4 if S3-HOST is set in the configuration file and end with '.amazonaws.com', otherwise it is S3.

[Rustam]

Thank you for answers. When the patch will be released? I can't do patches for packages due restrictions in my company.