PDA

View Full Version : Faulting application ZWCService.exe



tomek
June 26th, 2009, 10:30 AM
Hi,

I am new to amanda backup and I would like to get it to work. I have set up my test environment and I have a moderate success so far with it. I am running 2.6.1p1 on Centos 5.3 (installed from RPM) and a few windows machines (XPSP3, 2003SP1)

My XP experience was not good. I could not backup anything however all the tests were successful.
On first 2003 backup works great!
On my second 2003 system; the backup fails. Exactly the agent stops, generating this error in the event viewer:

Faulting application ZWCService.exe, version 0.0.0.0, faulting module msvcr80.dll, version 8.0.50727.1433, fault address 0x00008aa0.

The agent crashes upon backup request. Service was running all night and died as soon as I requested backup.

I suspect that there is a conflict with StorageCraft agent running on this system
Also, XP system runs sonicwall’s CDP agent.

I wonder if the amanda agent can co-exist with any other backup on the system?
Is there a way to get it to work with StorageCraft agent installed?
Are there issues with the latest version of the client?

Answers will be greatly appreciated!

harsha
June 28th, 2009, 08:22 PM
Hi,

To debug the Service crash problem pl. follow the following steps
1. Run setup-debug.bat file (residing in bin folder of community installation)
(Running this script will generate the memory dump after the ZWCService crash which will eventually help to debug the problem)
2. Set the log level to 5 by setting following registry key
HKEY_LOCAL_MACHINE\SOFTWARE\ZMANDA\ZWC\1.0\Debug\L ogLevel
3. Restart ZWCService
4. Reproduce the crash
5. Collect the logs using zwc-support.bat (in bin folder)(This script collects the required information from the system and store it in .cab file) and send the ZWC*.cab file to us.

Is ZWCService crashing on XP as well ?
ZWC can co-exist with other backup products provided the backups are not running at the same time.
BTW, windows 2003 with Service Pack2 is expected for ZWC.

tomek
June 30th, 2009, 10:05 AM
Hi,

Thank you for the prompt reply. Here is what happened...

I reproduced the crash without any problems. The only difference was that I used; setup-debug-env.bat instead of setup-debug.bat as the last one was not in the /bin directory.

I do have a CAB file, produced as the result but I am not sure how to send it to you. The file is a little biger than 2MB so I can send it as an email attachment. However I would need some address.

Please let me know how do you want me to transfer the file.

Thank you again!

gokul
June 30th, 2009, 09:48 PM
You can use "http://www.yousendit.com/" to send us the CAB file. (It stores the attachment at a defined URL for 7 days or so.)

Gokul.

tomek
July 2nd, 2009, 09:56 AM
It still asks me for the email address of "us" which I do not have.
Can I assume "support@amanda.org" as a safe bet?
I understand that you want to avoid the spammers hence you do not want to expose your email addresses.

paddy
July 2nd, 2009, 09:58 AM
It still asks me for the email address of "us" which I do not have.
Can I assume "support@amanda.org" as a safe bet?
I understand that you want to avoid the spammers hence you do not want to expose your email addresses.

Please send it to community at zmanda dot com

tomek
July 2nd, 2009, 11:04 AM
I have email the log file. Also, I got email form StorageCraft confirming that there are no issues with running amanda/zmanda and StorageCraft ShadowProtect V3.3 server edition.
I hope the logs will help you to figure out what causes the crash.
Thank you for your help!

tomek
July 2nd, 2009, 11:32 AM
Opps.. This is what I got in my email

Your mail to 'Community' with the subject

Faulting application ZWCService.exe - logs

Is being held until the list moderator can review it for approval.

The reason it is being held:

Message body is too big: 3983226 bytes with a limit of 40 KB

paddy
July 2nd, 2009, 01:26 PM
We have received the logs. We will post a reply after reviewing the logs.

harsha
July 3rd, 2009, 01:15 AM
From the application event logs it seems that NTBackup is running at the same time and its backup is also failing.

From ZWC logs it seems that ZWCService is crashing while creating a snashot. If the NTBackup is runing at the same time, ZWC backup should go ahead without creating the snapshot, but instead its crashing.

As the memory dump (.dmp) file is not generated at the time of crash, I could not debug the issue
(setup-debug-env.bat is expected to help to generate this file).

Will you pl. confirm if there is a user.dmp in c:\. If yes pl. send it to us.

Will you be able to upgrade the service pack to 2.

Make sure that NTBackup not running at the same time ZWC backup run.
And then try to re-run the backup.

tomek
July 7th, 2009, 07:47 AM
Yes you are right! I had NTBackup running and I forgot about it. I have removed all scheduled jobs and I ran Amanda backup. At first it failed but then I had restarted the amanda service and the full backup took place. I got a nice report via email.
Then I setup a cron job for the amanda process to run at 1AM in the moring Monday to Friday. To my disappointment I got the email stating that the backup failed. I was away on Monday so I checked today (after receiving another email stating errors) and the amanda service was down again.
I guess we are at square one with the troubleshooting. I will run the debug scripts again and I will send you email with the results.

tomek
July 8th, 2009, 09:24 AM
I have not seen any memory DUMP files on the C:\ drive.
Also, I have stopped all services related to storagecraft. Including StorageCraft Volume Shadow Provider. Actually I had to disable this service. It caused some DCOM related errors but the backup process ran as expected.
At this point I am sure that the agent is competing about something related to VSS as the VSS is listed as a dependency under the SC Volume Shadow Provider.

Volume Shadow Copy Service error: Error creating the Shadow Copy Provider COM class with CLSID {e4eb5095-f587-4159-a1d8-2710692fd243} [0x80070422].

Backup has finished although some domino files (mailboxes) were skipped.

tomek
July 9th, 2009, 09:50 AM
:( I guess this is it...

gokul
July 9th, 2009, 11:09 PM
Hello User,
I guess you shut down NTBackup, before running a ZWC backup this time. So is ZWC still crashing ? If yes, then we need the crash dump file from you if we are to come up with any kind of investigation.

To obtain the dump file
1 . Run ZWC Setup Debug Environment
2 . Run the backup and wait for the crash.
3 . Run ZWC Support Script tp collect the logs and send it to us.

If ZWC is not crashing, then we still need the logfiles to see why some files are getting skipped ?

Also, you might want to restart "Volume Shadow Copy" service and make sure writers are in stable states before running a backup (might require a reboot).

Gokul.

tomek
July 29th, 2009, 05:22 AM
I lost hope that anyone will reply. I will run the support tools again. BTW I have emailed dump some time ago... but I guess it got lost.

tomek
September 2nd, 2009, 12:59 PM
I have emailed a ZIP file which was generated by your debug script. It waits for the approval.

harsha
September 3rd, 2009, 10:01 PM
It is still crashing after creating a snapshot. We have never seen this before.

Few recommandations though

1. Pl. upgrade to Service pack 2, it contains many VSS related fixes. VSS is a volume Snapshot Service which ZWC uses to create a snapshot for backup.

2. Make sure that you have enough disk space availble on the drives to take the snapshots. I am asking for this because I seen following message in system event log

Event Type: Information
Event Source: VolSnap
Event Category: None
Event ID: 33
Date: 8/29/2009
Time: 8:37:04 AM
User: N/A
Computer: MARS
Description:
The oldest shadow copy of volume D: was deleted to keep disk space usage for
shadow copies of volume D: below the user defined limit.

For more information, see Help and Support Center at
http://go.microsoft.com/fwlink/events.asp.
Data:
0000: 00 00 00 00 02 00 58 00 ......X.
0008: 00 00 00 00 21 00 06 40 ....!..@
0010: 02 00 00 00 00 00 00 00 ........
0018: 1e 00 00 00 00 00 00 00 ........
0020: 00 00 00 00 00 00 00 00 ........

3. There are lot of errors in application event log from "Sophas Antivirus". I
guess its running at the same time as backup might be causing the problem.

tomek
September 15th, 2009, 06:19 AM
Thank you for you post. We applied SP2 and all other patches needed so the server is as current as possible. The problem occurred again.
I looked at the SOPHOS AV but its scheduled scan ended at 11:21PM long before Amanda started. There are errors reported by SOPHOS but they are irrelevant as we run only a scheduled scans on the server and no on-access scans.
In terms of free space:
C: is 30GB 10GB free
D: is 39 15GB free
E: is 80GB 1.6GB free

We only run backup on the D drive and just for few files (about 11 to 16GB in total).

I do not know what to think about it. I have noticed that sometimes the backup works, 1 out of 20 or so. I can not tell why but I have see it working.
Also, we have updated StorageCraft to the latest version 3.4 it works great. The SC technical support told us that the new version of SC works with Amanda.

I would like to stop StrorageCraft Volume Shadow Provider before running Amanda. Is there a way to run a script as Amanda’s agent starts?

prashant
September 15th, 2009, 10:16 PM
You can use the pre-backup and post-backup scripts to stop and start "StrorageCraft Volume Shadow Provider" before and after the backups resp.

To do so, you will have to use the ZWCConfig utility in Start Menu > Programs > Zmanda > ZWC Config utility, to create a template of what you want to backup. Once the template is saved, open the template.txt located in "Zmanda_installation_dir\misc\template.txt". In the section for the template you have configured, enter the following tags:
<PRE_BACKUP_SCRIPT>script1location</PRE_BACKUP_SCRIPT>
<POST_BACKUP_SCRIPT>script2location</POST_BACKUP_SCRIPT>

eg:
<PRE_BACKUP_SCRIPT>C:\stopscript.bat</PRE_BACKUP_SCRIPT>
<POST_BACKUP_SCRIPT>C:\startscript.bat</POST_BACKUP_SCRIPT>

Please let us know if this works.

Regards,
Prashant

gokul
September 15th, 2009, 10:25 PM
You need to insert

<PRE_BACKUP_SCRIPT>script_path</PRE_BACKUP_SCRIPT> (and likewise for POST_ BACKUP_SCRIPT ) between

<DLE_Template> and </DLE_Template> tags for that particular template.

gokul
October 20th, 2009, 02:18 AM
Regarding the pre-backup and post-backup script problem :

Can you copy your scripts to the <INSTALLDIR>\bin directory, and change the XM tags to only include the script name.

For .e.g : <pre_backup_script>test.bat</pre_backup_script>.

Full paths in backup scripts are only supported with the upcoming release.

Gokul.

gokul
October 20th, 2009, 02:23 AM
Regarding the issue of StorageCraft Provider:

We have found out about issues concerning ZWC's ability to work with the StorageCraft Provider. These issues have been fixed, and should be shipped in the upcoming release of Zmanda Community Windows Client.

Gokul.