Results 1 to 10 of 10

Thread: Failure dump summary

  1. #1
    Join Date
    Oct 2012
    Location
    USA
    Posts
    21

    Question Failure dump summary

    I need some help troubleshooting some failures. I'm having a hard time figuring out what's going on. I'm still new to this system so I'm trying to learn as I go.

    I'm getting emails, as I should be, but they are showing failures. Below are two that I've gotten.

    Code:
    Hostname: h1p-backup1
    Org     : DailySet1
    Config  : DailySet1
    Date    : November 11, 2012
    
    These dumps were to tapes DailySet1-4, DailySet1-5.
    The next 4 tapes Amanda expects to use are: 4 new tapes.
    The next 4 new tapes already labelled are: DailySet1-6, DailySet1-7, DailySet1-8, DailySet1-9
    FAILURE DUMP SUMMARY:
      fs1 "F:/NAS1-BU" lev 1  FAILED [data timeout]
      fs1 "F:/NAS1-BU" lev 1  partial taper: dumper failed
      fs1 "F:/NAS1-BU" lev 1  was successfully retried
    
    
    
    STATISTICS:
                              Total       Full      Incr.   Level:#
                            --------   --------   --------  --------
    Estimate Time (hrs:min)     0:49
    Run Time (hrs:min)         34:34
    Dump Time (hrs:min)        33:13      32:20       0:53
    Output Size (meg)      1148002.5  1147960.5       42.0
    Original Size (meg)    1575238.2  1574771.3      466.9
    Avg Compressed Size (%)     72.9       72.9        9.0
    DLEs Dumped                   13          1         12  1:12
    Avg Dump Rate (k/s)       9830.5    10100.4       13.5
    
    Tape Time (hrs:min)        33:44      32:19       1:25
    Tape Size (meg)        1148002.5  1147960.5       42.0
    Tape Used (%)              109.5      109.5        0.0
    DLEs Taped                    14          1         13  1:13
    Parts Taped                   15          2         13  1:13
    Avg Tp Write Rate (k/s)   9681.2    10103.8        8.5
    
    USAGE BY TAPE:
      Label               Time         Size      %  DLEs Parts
      DailySet1-4        29:43        1024G  100.0    14    14
      DailySet1-5         4:01          97G    9.5     0     1
    
    FAILED DUMP DETAILS:
      /-- fs1 "F:/NAS1-BU" lev 1 FAILED [data timeout]
      sendbackup: start [fs1:"F:/NAS1-BU" level 1]
      sendbackup: info BACKUP=pkzip
      sendbackup: info RECOVER_CMD=Extract with zmanda windows client or unzip program
      sendbackup: info end
      \--------
    
    NOTES:
      planner: Full dump of fs1:"F:/FileServer" promoted from 5 days ahead.
      taper: Slot 3 with label DailySet1-3 is not reusable
      taper: Slot 4 with label DailySet1-4 is usable
      taper: tape DailySet1-4 kb 1073735498 fm 14 [OK]
      taper: Slot 5 with label DailySet1-5 is usable
      taper: tape DailySet1-5 kb 101819090 fm 1 [OK]
    
    
    DUMP SUMMARY:
                                                                                    DUMPER STATS               TAPER STATS
    HOSTNAME        DISK                                             L ORIG-GB  OUT-GB  COMP%  MMM:SS    KB/s MMM:SS    KB/s
    ------------------------------------------------------------------ -------------------------------------- --------------
    dc2             "C:/GP-BU"                                       1       0       0   17.0   27:12    14.7  27:11    14.7
    dc2             "C:/Windows/SYSVOL/sysvol/HOMENET.local/SCRIPTS" 1       0       0  200.0    0:06     0.5   0:05     0.4
    dhcp1           "C:/Windows/System32/dhcp/backup"                1       0       0    7.4    0:03    77.5   0:02   115.5
    fs1             "F:/FileServer"                                  0    1538    1121   72.9 1939:42 10100.4 1939:0 10103.8
    fs1             "F:/NAS1-BU"                                     1       0       0    7.6   21:40     4.6  21:30     4.6
    h1p-wds1        "C:/DeploymentShare"                             1       0       0  200.0    0:19     0.2   0:18     0.1
    h1p-wsus1       "C:/WSUS/DB"                                     1       0       0  200.0    0:02     1.3   0:01     2.0
    h2p-dns1        "C:/ProgramData/JH Software/Simple DNS Plus"     1       0       0   62.5    0:02     3.0   0:12     0.4
    h2p-dns2        "C:/ProgramData/JH Software/Simple DNS Plus"     1       0       0   66.7    0:02     2.4   0:01     4.0
    iolserver       "E:/Applications"                                1       0       0    4.6    3:31    44.5   3:30    44.7
    netmon1         "C:/Program Files (x86)/CatTools3"               1       0       0    3.8    0:08   201.4   0:07   243.3
    netmon1         "C:/_Backup"                                     1       0       0  200.0    0:04     0.8   0:03     0.7
    source-control1 "C:/Backups"                                     1       0       0   25.1    0:11   157.2   0:10   168.3
    
    (brought to you by Amanda version 3.3.2)
    Code:
    Hostname: h1p-backup1
    Org     : DailySet1
    Config  : DailySet1
    Date    : November 13, 2012
    
    These dumps were to tape DailySet1-6.
    The next 4 tapes Amanda expects to use are: 4 new tapes.
    The next 4 new tapes already labelled are: DailySet1-7, DailySet1-8, DailySet1-9, DailySet1-10
    FAILURE DUMP SUMMARY:
      fs1 "F:/FileServer" lev 0  FAILED [Some estimate timeout on fs1, using server estimate if possible]
      fs1 "F:/NAS1-BU" lev 0  FAILED [Some estimate timeout on fs1, using server estimate if possible]
    
    
    
    STATISTICS:
                              Total       Full      Incr.   Level:#
                            --------   --------   --------  --------
    Estimate Time (hrs:min)     1:00
    Run Time (hrs:min)          1:24
    Dump Time (hrs:min)         0:24       0:20       0:04
    Output Size (meg)        13155.7    13146.9        8.8
    Original Size (meg)      40606.8    40419.8      187.0
    Avg Compressed Size (%)     32.4       32.5        4.7
    DLEs Dumped                   11          9          2  1:2
    Avg Dump Rate (k/s)       9553.3    11351.4       40.2
    
    Tape Time (hrs:min)         0:23       0:20       0:04
    Tape Size (meg)          13155.7    13146.9        8.8
    Tape Used (%)                1.3        1.3        0.0
    DLEs Taped                    11          9          2  1:2
    Parts Taped                   11          9          2  1:2
    Avg Tp Write Rate (k/s)   9629.4    11438.0       40.6
    
    USAGE BY TAPE:
      Label               Time         Size      %  DLEs Parts
      DailySet1-6         0:23          13G    1.3    11    11
    
    NOTES:
      planner: Full dump of dc2:"C:/GP-BU" promoted from 4 days ahead.
      planner: Full dump of netmon1:"C:/Program Files (x86)/CatTools3" promoted from 4 days ahead.
      planner: Full dump of h1p-wsus1:"C:/WSUS/DB" promoted from 3 days ahead.
      planner: Full dump of source-control1:"C:/Backups" promoted from 4 days ahead.
      planner: Full dump of dhcp1:"C:/Windows/System32/dhcp/backup" promoted from 4 days ahead.
      planner: Full dump of h2p-dns1:"C:/ProgramData/JH Software/Simple DNS Plus" promoted from 4 days ahead.
      planner: Full dump of h2p-dns2:"C:/ProgramData/JH Software/Simple DNS Plus" promoted from 4 days ahead.
      planner: Full dump of netmon1:"C:/_Backup" promoted from 4 days ahead.
      planner: Full dump of dc2:"C:/Windows/SYSVOL/sysvol/HOMENET.local/SCRIPTS" promoted from 4 days ahead.
      taper: Slot 5 with label DailySet1-5 is not reusable
      taper: Slot 6 with label DailySet1-6 is usable
      taper: tape DailySet1-6 kb 13471484 fm 11 [OK]
    
    
    DUMP SUMMARY:
                                                                                    DUMPER STATS               TAPER STATS
    HOSTNAME        DISK                                             L ORIG-GB  OUT-GB  COMP%  MMM:SS    KB/s MMM:SS    KB/s
    ------------------------------------------------------------------ -------------------------------------- --------------
    dc2             "C:/GP-BU"                                       0       0       0   17.0    1:34   571.1   1:33   579.9
    dc2             "C:/Windows/SYSVOL/sysvol/HOMENET.local/SCRIPTS" 0       0       0  125.0    0:06     1.0   0:05     1.0
    dhcp1           "C:/Windows/System32/dhcp/backup"                0       0       0    7.2    0:03    50.7   0:02    74.5
    fs1             "F:/FileServer"                                                               FAILED
    fs1             "F:/NAS1-BU"                                                                  FAILED
    h1p-wds1        "C:/DeploymentShare"                             1       0       0  200.0    0:18     0.2   0:17     0.1
    h1p-wsus1       "C:/WSUS/DB"                                     0       2       1   31.4    1:21  8254.0   1:21  8287.9
    h2p-dns1        "C:/ProgramData/JH Software/Simple DNS Plus"     0       0       0   26.5    0:03    74.2   0:02   102.5
    h2p-dns2        "C:/ProgramData/JH Software/Simple DNS Plus"     0       0       0    9.3    0:04    62.2   0:03    79.0
    iolserver       "E:/Applications"                                1       0       0    4.7    3:26    43.6   3:25    43.9
    netmon1         "C:/Program Files (x86)/CatTools3"               0       0       0   14.5    0:21  1035.1   0:20  1094.8
    netmon1         "C:/_Backup"                                     0       0       0   34.8    0:05     8.8   0:03    13.3
    source-control1 "C:/Backups"                                     0      37      12   32.8   16:09 13124.5  16:08 13135.0
    
    (brought to you by Amanda version 3.3.2)
    As near as I can tell, they are complaining about timeouts but I don't understand how or why they are timing out. Perhaps there's a more verbose log somewhere or a switch to run amreport with to get more details but I don't know how.

    I have it scheduled, daily, via cron but over the weekend, I got an email with what you see below. Does this mean that, when the job kicks off, and is not finished within 24 hours, I'll see these errors?

    Code:
    *** THE DUMPS DID NOT FINISH PROPERLY!
    
    The next 4 tapes Amanda expects to use are: 4 new tapes.
    The next 4 new tapes already labelled are: DailySet1-3, DailySet1-4, DailySet1-5, DailySet1-6
    FAILURE DUMP SUMMARY:
      amdump: ERROR An Amanda process is already running - please run amcleanup manually
    
    
    
    STATISTICS:
                              Total       Full      Incr.   Level:#
                            --------   --------   --------  --------
    Estimate Time (hrs:min)     0:00
    Run Time (hrs:min)          0:00
    Dump Time (hrs:min)         0:00       0:00       0:00
    Output Size (meg)            0.0        0.0        0.0
    Original Size (meg)          0.0        0.0        0.0
    Avg Compressed Size (%)      --         --         --
    DLEs Dumped                    0          0          0
    Avg Dump Rate (k/s)          --         --         --
    
    Tape Time (hrs:min)         0:00       0:00       0:00
    Tape Size (meg)              0.0        0.0        0.0
    Tape Used (%)                0.0        0.0        0.0
    DLEs Taped                     0          0          0
    Parts Taped                    0          0          0
    Avg Tp Write Rate (k/s)      --         --         --
    
    
    DUMP SUMMARY:
                                           DUMPER STATS               TAPER STATS
    HOSTNAME     DISK        L ORIG-GB  OUT-GB  COMP%  MMM:SS   KB/s MMM:SS   KB/s
    -------------------------- ------------------------------------- -------------
    
    (brought to you by Amanda version 3.3.2)

  2. #2
    Join Date
    Oct 2012
    Location
    USA
    Posts
    21

    Default

    I think I figured out why the FS1 DLEs were giving timeout errors. The ZWC Service was not running. It had crashed at some point and needed to be restarted. I'm now testing this via amdump DailySet1 fs1 F:/NAS1-BU but what will happen to my vtape rotation? Will the next scheduled job, tonight @ 20:00, use up space on the same vtape or will it run on another one altogether? If the latter, how does that affect my total number of vtapes (30)?

  3. #3
    Join Date
    Oct 2012
    Location
    USA
    Posts
    21

    Default

    Still having a timeout issue. Where can I get more details on what's going on below?

    Code:
    Hostname: h1p-backup1
    Org     : DailySet1
    Config  : DailySet1
    Date    : November 13, 2012
    
    The next 4 tapes Amanda expects to use are: 4 new tapes.
    The next 4 new tapes already labelled are: DailySet1-11, DailySet1-12, DailySet1-13, DailySet1-14
    FAILURE DUMP SUMMARY:
      fs1 "F:/FileServer" lev 0  FAILED [Some estimate timeout on fs1, using server estimate if possible]
    
    
    
    STATISTICS:
                              Total       Full      Incr.   Level:#
                            --------   --------   --------  --------
    Estimate Time (hrs:min)     0:30
    Run Time (hrs:min)          0:30
    Dump Time (hrs:min)         0:00       0:00       0:00
    Output Size (meg)            0.0        0.0        0.0
    Original Size (meg)          0.0        0.0        0.0
    Avg Compressed Size (%)      --         --         --
    DLEs Dumped                    0          0          0
    Avg Dump Rate (k/s)          --         --         --
    
    Tape Time (hrs:min)         0:00       0:00       0:00
    Tape Size (meg)              0.0        0.0        0.0
    Tape Used (%)                0.0        0.0        0.0
    DLEs Taped                     0          0          0
    Parts Taped                    0          0          0
    Avg Tp Write Rate (k/s)      --         --         --
    
    NOTES:
      planner: Last full dump of fs1:"F:/FileServer" on tape DailySet1-4 overwritten in 5 runs.
      driver: WARNING: got empty schedule from planner
      taper: Slot 10 with label DailySet1-10 is not reusable
      taper: Slot 11 with label DailySet1-11 is usable
    
    
    DUMP SUMMARY:
                                               DUMPER STATS               TAPER STATS
    HOSTNAME     DISK            L ORIG-GB  OUT-GB  COMP%  MMM:SS   KB/s MMM:SS   KB/s
    ------------------------------ ------------------------------------- -------------
    fs1          "F:/FileServer"                              FAILED
    
    (brought to you by Amanda version 3.3.2)

  4. #4
    Join Date
    Aug 2012
    Posts
    81

    Default

    Yes. I will be considered as a standard amdump run.

  5. #5
    Join Date
    Aug 2012
    Posts
    81

    Default

    What does the log say?

  6. #6
    Join Date
    Oct 2012
    Location
    USA
    Posts
    21

    Default

    Quote Originally Posted by xHan View Post
    What does the log say?
    I don't know where the logs are.

  7. #7
    Join Date
    Aug 2012
    Posts
    81

    Default

    /var/log/amanda/server/your-set-name

  8. #8
    Join Date
    Oct 2012
    Location
    USA
    Posts
    21

    Default

    Quote Originally Posted by xHan View Post
    /var/log/amanda/server/your-set-name
    Okay, which log(s) are you after? I see some for DUMPER, PLANNER, TAPER, DRIVER, etc.

  9. #9
    Join Date
    Oct 2012
    Location
    USA
    Posts
    21

    Default

    Another failure last night. Still complaining about a timeout.

    Error:
    Code:
    Hostname: h1p-backup1
    Org     : DailySet1
    Config  : DailySet1
    Date    : November 13, 2012
    
    These dumps were to tape DailySet1-11.
    The next 4 tapes Amanda expects to use are: 4 new tapes.
    The next 4 new tapes already labelled are: DailySet1-12, DailySet1-13, DailySet1-14, DailySet1-15
    FAILURE DUMP SUMMARY:
      fs1 "F:/FileServer" lev 1  FAILED [data timeout]
      fs1 "F:/FileServer" lev 1  partial taper: dumper failed
      fs1 "F:/FileServer" lev 1  FAILED [data timeout]
      fs1 "F:/FileServer" lev 1  partial taper: dumper failed
      fs1 "F:/NAS1-BU" lev 1  FAILED [data timeout]
      fs1 "F:/NAS1-BU" lev 1  partial taper: dumper failed
      fs1 "F:/NAS1-BU" lev 1  was successfully retried
    
    
    
    STATISTICS:
                              Total       Full      Incr.   Level:#
                            --------   --------   --------  --------
    Estimate Time (hrs:min)     0:55
    Run Time (hrs:min)          7:49
    Dump Time (hrs:min)         3:42       3:24       0:18
    Output Size (meg)       115400.7   115393.9        6.8
    Original Size (meg)     276765.0   276687.7       77.3
    Avg Compressed Size (%)     41.7       41.7        8.8
    DLEs Dumped                   12         11          1  1:1
    Avg Dump Rate (k/s)       8877.9     9654.6        6.5
    
    Tape Time (hrs:min)         6:54       3:24       3:30
    Tape Size (meg)         115400.7   115393.9        6.8
    Tape Used (%)               11.0       11.0        0.0
    DLEs Taped                    15         11          4  1:4
    Parts Taped                   15         11          4  1:4
    Avg Tp Write Rate (k/s)   4761.3     9655.4        0.6
    
    USAGE BY TAPE:
      Label               Time         Size      %  DLEs Parts
      DailySet1-11        6:53         113G   11.1    15    15
    
    FAILED DUMP DETAILS:
      /-- fs1 "F:/FileServer" lev 1 FAILED [data timeout]
      sendbackup: start [fs1:"F:/FileServer" level 1]
      sendbackup: info BACKUP=pkzip
      sendbackup: info RECOVER_CMD=Extract with zmanda windows client or unzip program
      sendbackup: info end
      \--------
      /-- fs1 "F:/FileServer" lev 1 FAILED [data timeout]
      sendbackup: start [fs1:"F:/FileServer" level 1]
      sendbackup: info BACKUP=pkzip
      sendbackup: info RECOVER_CMD=Extract with zmanda windows client or unzip program
      sendbackup: info end
      \--------
      /-- fs1 "F:/NAS1-BU" lev 1 FAILED [data timeout]
      sendbackup: start [fs1:"F:/NAS1-BU" level 1]
      sendbackup: info BACKUP=pkzip
      sendbackup: info RECOVER_CMD=Extract with zmanda windows client or unzip program
      sendbackup: info end
      \--------
    
    NOTES:
      planner: Last full dump of h1p-wds1:"C:/DeploymentShare" on tape DailySet1-1 overwritten in 5 runs.
      planner: Last full dump of fs1:"F:/FileServer" on tape DailySet1-4 overwritten in 5 runs.
      planner: Last full dump of fs1:"F:/NAS1-BU" on tape DailySet1-2 overwritten in 5 runs.
      planner: Last full dump of iolserver:"E:/Applications" on tape DailySet1-2 overwritten in 5 runs.
      planner: Full dump of dc2:"C:/GP-BU" promoted from 7 days ahead.
      planner: Full dump of netmon1:"C:/Program Files (x86)/CatTools3" promoted from 7 days ahead.
      planner: Full dump of h1p-wds1:"C:/DeploymentShare" promoted from 2 days ahead.
      planner: Full dump of iolserver:"E:/Applications" promoted from 3 days ahead.
      planner: Full dump of source-control1:"C:/Backups" promoted from 7 days ahead.
      planner: Full dump of dhcp1:"C:/Windows/System32/dhcp/backup" promoted from 7 days ahead.
      planner: Full dump of h2p-dns1:"C:/ProgramData/JH Software/Simple DNS Plus" promoted from 7 days ahead.
      planner: Full dump of h2p-dns2:"C:/ProgramData/JH Software/Simple DNS Plus" promoted from 7 days ahead.
      planner: Full dump of h1p-wsus1:"C:/WSUS/DB" promoted from 7 days ahead.
      planner: Full dump of netmon1:"C:/_Backup" promoted from 7 days ahead.
      planner: Full dump of dc2:"C:/Windows/SYSVOL/sysvol/HOMENET.local/SCRIPTS" promoted from 7 days ahead.
      taper: Slot 11 with label DailySet1-11 is usable
      taper: tape DailySet1-11 kb 118170340 fm 15 [OK]
    
    
    DUMP SUMMARY:
                                                                                    DUMPER STATS               TAPER STATS
    HOSTNAME        DISK                                             L ORIG-GB  OUT-GB  COMP%  MMM:SS    KB/s MMM:SS    KB/s
    ------------------------------------------------------------------ -------------------------------------- --------------
    dc2             "C:/GP-BU"                                       0       0       0   17.0    1:33   643.3   1:32   651.3
    dc2             "C:/Windows/SYSVOL/sysvol/HOMENET.local/SCRIPTS" 0       0       0  125.0    0:06     1.0   0:11     0.5
    dhcp1           "C:/Windows/System32/dhcp/backup"                0       0       0    7.4    0:04    63.3   0:03    77.0
    fs1             "F:/FileServer"                                  1                     --     PARTIAL      76:48     0.0 PARTIAL
    fs1             "F:/NAS1-BU"                                     1       0       0    8.8   17:52     6.5  17:50     6.5
    h1p-wds1        "C:/DeploymentShare"                             0      51      49   96.5   93:59  9167.3  93:59  9167.9
    h1p-wsus1       "C:/WSUS/DB"                                     0       2       1   31.5    1:31  7421.0   1:30  7466.5
    h2p-dns1        "C:/ProgramData/JH Software/Simple DNS Plus"     0       0       0   26.5    0:02    96.8   0:02   102.5
    h2p-dns2        "C:/ProgramData/JH Software/Simple DNS Plus"     0       0       0    9.3    0:04    63.4   0:03    79.0
    iolserver       "E:/Applications"                                0     180      51   28.1   89:08  9908.9  89:07  9910.9
    netmon1         "C:/Program Files (x86)/CatTools3"               0       0       0   14.5    0:25   882.9   0:24   902.2
    netmon1         "C:/_Backup"                                     0       0       0   34.8    0:04    11.6   0:02    20.0
    source-control1 "C:/Backups"                                     0      37      12   32.8   17:04 12416.6  17:05 12407.1
    
    (brought to you by Amanda version 3.3.2)
    I'm not sure what debug file to open. As you can see, I have plenty to choose from: [URL="http://pastebin.com/ULrfGRd7"]http://pastebin.com/ULrfGRd7[/URL]
    Last edited by homenet; November 14th, 2012 at 03:30 AM.

  10. #10
    Join Date
    Oct 2012
    Location
    USA
    Posts
    21

    Default

    It seems to have something to do with compression or maybe the estimation part of the compression job. If I disable compression altogether, it seems to work. If I set it to compress server fast or compress client fast, it fails. Playing with the estimating system (estimate client/calcsize/server) doesn't seem to help either.

    Hasn't been a whole lot of movement on this thread. Did I place it in the wrong area of the forum?
    Last edited by homenet; November 15th, 2012 at 02:22 AM.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •