Results 1 to 5 of 5

Thread: Slow streaming throughput for a very large filesystems

Threaded View

  1. #1

    Default Slow streaming throughput for a very large filesystem

    Hi,

    I'm trying to use Amanda to backup 1.4 Terabytes of data with an HP 1/8 Ultrium 960 Autoloader over the network. The network maximum throughput is 100Mb/s. It is measured by copying a 20GB tar file from the host client to the backup server. The host client has a SATA raid 5 array holding the 1.4TB data and a single SATA as its system drive. The backup server has a holding disk of 230GB on a SATA drive. I don't use any software nor hardware compression.

    I want to backup the data of 1.4TB in 48 hours which is not unreasonable. However, so far it takes more than 4 days to backup 1.2TB of data. The log statistics is at the bottom of this message. It backups at around 3MB/s.

    I do a backup test of 86GB of data. It can backup at round 8MB/s.

    I'm using SUSE Linux 9.3 for the host client and the backup server.

    Could you please help me explaining the cause of this problem? What can I do to solve this problem? Am I doomed?

    I understand that backing up more than 1.4TB is a challenge, but it should be possible to do it in less than 60 hours. Please see my amanda statistics log below.

    -- Log for 1.2TB backup.

    These dumps were to tapes WeeklySet-4-01, WeeklySet-4-02, WeeklySet-4-03, WeeklySet-4-04.
    The next 4 tapes Amanda expects to use are: WeeklySet-1-02, WeeklySet-1-03, WeeklySet-2-01, WeeklySet-2-02.
    The next new tape already labelled is: WeeklySet-2-03.

    STATISTICS:
    Total Full Incr.
    -------- -------- --------
    Estimate Time (hrs:min) 0:00
    Run Time (hrs:min) 110:12
    Dump Time (hrs:min) 110:11 110:11 0:00
    Output Size (meg) 1128387.4 1128387.4 0.0
    Original Size (meg) 1124861.2 1124861.2 0.0
    Avg Compressed Size (%) -- -- --
    Filesystems Dumped 1 1 0
    Avg Dump Rate (k/s) 2912.9 2912.9 --

    Tape Time (hrs:min) 110:01 110:01 0:00
    Tape Size (meg) 1128387.4 1128387.4 0.0
    Tape Used (%) 292.3 292.3 0.0
    Filesystems Taped 1 1 0

    Chunks Taped 112839 112839 0
    Avg Tp Write Rate (k/s) 2917.4 2917.4 --

    USAGE BY TAPE:
    Label Time Size % Nb Nc
    WeeklySet-4-01 35:04 367222592k 92.9 0 35974
    WeeklySet-4-02 35:03 367222592k 92.9 0 35974
    WeeklySet-4-03 35:05 367222592k 92.9 0 35974
    WeeklySet-4-04 4:47 50190144k 12.7 1 4917

    NOTES:
    planner: Adding new disk host_client:/data1/tiff.
    driver: WARNING: This is not the first amdump run today. Enable the usetimestamps option in the configuration file if you want to run amdump more than once per calendar day.
    taper: no split_diskbuffer specified: using fallback split size of 10240kb to buffer host_client:/data1/tiff.0 in-memory
    taper: tape WeeklySet-4-01 kb 368373792 fm 35975 writing file: No space left on device
    taper: continuing host_client:/data1/tiff.0 on new tape from 368373760kb mark: [writing file: No space left on device]
    taper: tape WeeklySet-4-02 kb 368373792 fm 35975 writing file: No space left on device
    taper: continuing host_client:/data1/tiff.0 on new tape from 736747520kb mark: [writing file: No space left on device]
    taper: tape WeeklySet-4-03 kb 368373792 fm 35975 writing file: No space left on device
    taper: continuing host_client:/data1/tiff.0 on new tape from 1105121280kb mark: [writing file: No space left on device]
    taper: tape WeeklySet-4-04 kb 50347488 fm 4917 [OK]

    DUMP SUMMARY:
    DUMPER STATS TAPER STATS
    HOSTNAME DISK L ORIG-kB OUT-kB COMP% MMM:SS KB/s MMM:SS KB/s
    -------------------------- ------------------------------------- -------------
    host_client -data1/tiff 0 1151857910 1155468736 -- 6611:13 2903.8 6600:5 2917.4

    (brought to you by Amanda version 2.5.1p3)

    -- Log for 86GB backup.

    These dumps were to tape WeeklySet-3-04.
    The next 4 tapes Amanda expects to use are: WeeklySet-4-01, WeeklySet-4-02, WeeklySet-4-03, WeeklySet-4-04.
    The next 2 new tapes already labelled are: WeeklySet-2-04, WeeklySet-2-03.

    STATISTICS:
    Total Full Incr.
    -------- -------- --------
    Estimate Time (hrs:min) 0:00
    Run Time (hrs:min) 3:16
    Dump Time (hrs:min) 2:35 2:35 0:00
    Output Size (meg) 87281.8 87281.8 0.0
    Original Size (meg) 87281.8 87281.8 0.0
    Avg Compressed Size (%) -- -- --
    Filesystems Dumped 1 1 0
    Avg Dump Rate (k/s) 9582.1 9582.1 --

    Tape Time (hrs:min) 0:36 0:36 0:00
    Tape Size (meg) 87284.4 87284.4 0.0
    Tape Used (%) 22.6 22.6 0.0
    Filesystems Taped 1 1 0

    Chunks Taped 86 86 0
    Avg Tp Write Rate (k/s) 41463.8 41463.8 --

    USAGE BY TAPE:
    Label Time Size % Nb Nc
    WeeklySet-3-04 0:36 89376544k 22.6 1 86

    NOTES:
    planner: Adding new disk host_client:/data1/dat.
    driver: WARNING: This is not the first amdump run today. Enable the usetimestamps option in the configuration file if you want to run amdump more than once per calendar day.
    taper: tape WeeklySet-3-04 kb 89379296 fm 86 [OK]

    DUMP SUMMARY:
    DUMPER STATS TAPER STATS
    HOSTNAME DISK L ORIG-kB OUT-kB COMP% MMM:SS KB/s MMM:SS KB/s
    -------------------------- ------------------------------------- -------------
    host_client -/dat 0 89376520 89379264 -- 155:27 9582.1 35:56 41463.8

    (brought to you by Amanda version 2.5.1p3)

    Puthick
    Last edited by puthick; April 19th, 2007 at 08:01 PM.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •