PDA

View Full Version : amcheck successful but amdump fails with 'No route to host'



sahar
November 9th, 2011, 11:09 AM
Hi,

First, I need to thank amanda team for their great backup solution. I am currently running backups successfully from 4 windows clients on a linux server. However the 5th machine doesn't seem to be lucky :(

Server: Ubuntu 9.04
Client: Windows 7
Amanda version: 3.2.3

-amcheck is successful.
-amservice also ends successfully:

$ amservice 129.128.208.163 bsdtcp noop </dev/null
OPTIONS features=ff7fffff9cfeffffd3cf1300;
-however when amdump is run, an error code is returned and no dump is stored in the tapes.
-In one of the dumper debug files, I can see the following error:

Wed Nov 9 12:00:41 2011: dumper: connect_port: Try port 516: available - Success
Wed Nov 9 12:00:41 2011: dumper: connect_portrange: Connect from 0.0.0.0.516 failed: No route to host
Wed Nov 9 12:00:41 2011: dumper: connect_portrange: connect to 129.128.208.163.10080 failed: No route to host
Wed Nov 9 12:00:41 2011: dumper: stream_client: Could not bind to port in range 512-1023.
Wed Nov 9 12:00:41 2011: dumper: security_seterror(handle=0x17a9da0, driver=0x7fb9fccc9900 (BSDTCP) error=No route to host)
Wed Nov 9 12:00:41 2011: dumper: security_close(handle=0x17a9da0, driver=0x7fb9fccc9900 (BSDTCP))


-I checked the firewall inbound rules on windows 7 machine. Everything looks fine. The ZWC service is added there and all connections is allowed.
-also, netstat on windows 7 shows that the 10080 and 10081 ports are LISTENING
-I have attached more log file.

I appreciate your help with this
Sahar

jfarschman
November 17th, 2011, 01:27 PM
Sahar,

I gave up trying to get the ZWCSevice wor work properly. I found that it put an undue burden on my windows boxes, building giant mySQL database files and consuming lots of resources. For instance, on one system it needed a spare 8GB to build a database and it crushed other processes.

Instead, I mount the appropriate drives on the Linux backup server.

Still, if I had to guess, I'd say it's one of the following:

1) firewall issue - but you checked this.
2) Too many files - One of our servers is filled with millions of tiny files. Not good for the windows client.
3) Not enough room for the mySQL DB.

That probably doesn't help, but it's the best I can offer. I gave up on the windows client about 18 months ago and everything is happy now.

sahar
November 18th, 2011, 11:02 AM
Hi,

Thank you very much for your reply.
I just didn't understand what you mean by processing burden and the memory space needed for ZWC. I have just installed the client version on the windows machine and the server instance is on a Linux. So, why should be big database files on windows? Where can I see how much process and disk usage ZWC consumes on the windows machines?
At the moment, I don't have problem with the other 4 windows clients which also have ZWC. I only see some log files stored but no database files or anything else.

You said you are not using ZWC on windows any more? So, how do you get the windows boxes backed up?

Thank again. I appreciate your time and help,
Sahar

sahar
November 24th, 2011, 03:13 PM
I have noticed that the error initiates in the chunker. Here is the chunker's log file:

Thu Nov 24 03:15:05 2011: chunker: pid 5559 ruid 1011 euid 1011 version 3.2.3: start at Thu Nov 24 03:15:05 2011
Thu Nov 24 03:15:05 2011: chunker: pid 5559 ruid 1011 euid 1011 version 3.2.3: rename at Thu Nov 24 03:15:05 2011
Thu Nov 24 03:15:05 2011: chunker: getcmd: START 20111124031501
Thu Nov 24 03:15:05 2011: chunker: getcmd: PORT-WRITE 00-00001 /MAgroup-Backup/holding/20111124031501/129.128.208.163.C__Users_Masoud_Documents_Students .0 129.128.208.163 ff7ffff
f9cfeffffd3cf130000 "C:/Users/Masoud/Documents/Students" 0 1970:1:1:0:0:0 1024 DUMP 40800 |;auth=bsdtcp;index;
Thu Nov 24 03:15:05 2011: chunker: stream_server opening socket with family 2 (requested family was 2)
Thu Nov 24 03:15:05 2011: chunker: try_socksize: receive buffer size is 65536
Thu Nov 24 03:15:05 2011: chunker: bind_portrange2: Try port 27360: Available - Success
Thu Nov 24 03:15:05 2011: chunker: stream_server: waiting for connection: 0.0.0.0.27360
Thu Nov 24 03:15:05 2011: chunker: stream_server opening socket with family 2 (requested family was 2)
Thu Nov 24 03:15:05 2011: chunker: try_socksize: receive buffer size is 65536
Thu Nov 24 03:15:05 2011: chunker: bind_portrange2: Try port 27360: Available - Address already in use
Thu Nov 24 03:15:05 2011: chunker: bind_portrange2: Try port 27361: Available - Success
Thu Nov 24 03:15:05 2011: chunker: stream_server: waiting for connection: 0.0.0.0.27361
Thu Nov 24 03:15:05 2011: chunker: putresult: 23 PORT
Thu Nov 24 03:15:05 2011: chunker: stream_accept: connection from 127.0.0.1.1024
Thu Nov 24 03:15:05 2011: chunker: try_socksize: receive buffer size is 65536
Thu Nov 24 03:15:15 2011: chunker: putresult: 10 FAILED
Thu Nov 24 03:15:15 2011: chunker: pid 5559 finish time Thu Nov 24 03:15:15 2011

What could be the reason for the chunker fail? Please help me if you have any idea.
Thanks,
Sahar

sahar
November 25th, 2011, 11:01 AM
Hi,

I also attached the client (windows) side logs. I can see some errors:

- "sql specific error". Which I also can see it in the log files of other windows clients, whose backup is performed successfully. So, I doubt this might be the cause of the failure for this client.

- "ZWC Failure error code: 2" which I also see on the rest of the clients!

I don't know the reason for the above errors in the ZWC logs on the windows clients. However, both of them are present in the successfully backed up clients, as well.

Any ideas please,
Sahar

jfarschman
December 13th, 2011, 09:32 AM
I just didn't understand what you mean by processing burden and the memory space needed for ZWC.

Sorry for the delay in responding. I don't check in here very often and I did not subscribe to this thread. Amanda backups generally have the client do a bunch of the backup work. For instance, the client can be asked to to the compression before the files are moved to the backup server. This will save bandwidth. What I have noticed on some systems is the compression work is too much for the client to do.

The second problem I found was with the mySQL db that the client builds. if you check your services (Start | Run | Services.msc) you'll see two services at the very bottom of the list one is a mysql server. One of my file servers ended up with a 8GB mysql DB and it was writing this to the c:\Program Files\ This ran my C drive out of space and resulted in SQL errors. I added another paritition specially for the mySQL db and still had issues from problem #1 above. Basically, it was just too slow.

Finally, I just stopped using the ZWC on that system. Instead, on the backup server, I mount the drives I'd like to backup and do a localhost backup of the mounted data. Because my backup server is more powerful this worked great for me. I can tell you more about this process if you need.

About your specific problem. I don't have a clue. Sorry.

sahar
December 15th, 2011, 08:29 AM
Hi,

Thank you so much for the reply and the explanation. What you did seems to be a good substitute solution. I would appreciate if you could give me more details on that. How can I mount the windows drives on the linux server?

Thanks,
Sahar

jfarschman
December 15th, 2011, 09:30 AM
Sahar,

I'm happy I can help. I do not have very many notes on this process, but basically, you mount the windows drives CIFS using the fstab file. I mount it with the amandabackup user and I use a special AD domain (msdomain).

# /etc/fstab (the line below should not wrap)
//HOME3/USERS/ /mnt/home3_users cifs nodev,nosuid,username=amandabackup,password=secret password,msdomain,uid=16777222,gid=16777218 0 0

The disk list is even easier:
# /etc/amanda/daily/disklist
localhost /mnt/home3_users cifs-noest # 51GB

Finally, I'm not sure if I had to add the cifs-noest to my amanda.conf files, but if you need to add it, mine looks like this:

# Don't estimate, but otherwise a comp-user-tar
define dumptype cifs-noest {
user-tar
estimate server
compress server fast
exclude list "/var/lib/amanda/exclude_list"
}