PDA

View Full Version : backup using amazon works but so slow using zmanda



ntenz
March 22nd, 2010, 02:05 AM
Hi there,

I have setup a device and diskpool on Amazon S3. I am able to backup to amazon just fine. But the thing is that its extreme slow.

I am transfering around 2 GB and i takes around 2 hours. I have mounted the amazon bucket directly on the server and moved a 1 GB file to the bucket which took around 3 minutes so I guess the problem is zmanda?

When I browse my bucket on amazon I can see that the transfered files are divided into 2MB tape files?


---------- 1 root root 2097152 2010-03-19 12:45 amabond-tape1f00000001-b00000000000002c2.data
---------- 1 root root 2097152 2010-03-19 12:45 amabond-tape1f00000001-b00000000000002c3.data
---------- 1 root root 2097152 2010-03-19 12:45 amabond-tape1f00000001-b00000000000002c4.data
---------- 1 root root 2097152 2010-03-19 12:45 amabond-tape1f00000001-b00000000000002c5.data
---------- 1 root root 2097152 2010-03-19 12:45 amabond-tape1f00000001-b00000000000002c6.data
---------- 1 root root 492431 2010-03-19 12:45 amabond-tape1f00000001-b00000000000002c7.data

/dev/sdd1 100G 261M 94G 1% /diskpool2
s3fs 256T 0 256T 0% /mnt/S3amazon

paddy
March 22nd, 2010, 10:08 AM
The size of tape files can be increased by adjusting the block size. Please see Zmanda Knowledgebase for the article that describes how to change the value of block size for
amanda devices in Amanda Enterprise.

It is not clear what the bottleneck is without looking at the Amanda reports. Please ask
Zmanda support for help to see if the bottleneck is the transfer to S3 or something else.

Paddy

ntenz
March 23rd, 2010, 08:32 AM
I tried to change

device_property "BLOCK_SIZE" "2097152"


but that didnt work

paddy
March 23rd, 2010, 02:27 PM
Can you provide more information where you added device_property line? If you need help, please open a support case.

Paddy

ntenz
March 24th, 2010, 01:46 AM
Hi Paddy,

I changed this file: /etc/amanda/BACKUPSET/s3-2-amabond.conf but that didnt help.

ntenz
March 24th, 2010, 02:53 AM
# DO NOT EDIT. This file is generated automatically from *.yml

So i instead configured it in the amabond.profile file.

MIN_BLOCK_SIZE_comment: '262144'
BLOCK_SIZE: '52428800'

But is still writes 2MB block to amazon

---------- 1 root root 2097152 2010-03-24 10:52 amabond-tape1f00000001-b000000000000005c.data
---------- 1 root root 2097152 2010-03-24 10:52 amabond-tape1f00000001-b000000000000005d.data
---------- 1 root root 2097152 2010-03-24 10:52 amabond-tape1f00000001-b000000000000005e.data
---------- 1 root root 2097152 2010-03-24 10:53 amabond-tape1f00000001-b000000000000005f.data
---------- 1 root root 2097152 2010-03-24 10:53 amabond-tape1f00000001-b0000000000000060.data
---------- 1 root root 2097152 2010-03-24 10:53 amabond-tape1f00000001-b0000000000000061.data

paddy
March 24th, 2010, 03:58 PM
Can you please open a support case? We will need your configuration and log files after the block size change.