Incremental backups mostly work but fail sometimes

[phishee]

I have a backup set which I currently run hourly incrementals and nightly fulls with and it seems like I have been getting inconsistent results.

It backs up the DB successfully most of the time but there have been 7 days in the past month where it fails completely.

I have "verbose" logging set and the only error I see in the log file are entries like the following:



Could not copy bin log file /var/lib/mysql-zrm/PayIt/20100301190102/mysqld-bin.001039
Could not copy bin log file /var/lib/mysql-zrm/PayIt/20100301190102/mysqld-bin.001039
Could not copy bin log file /var/lib/mysql-zrm/PayIt/20100301190102/mysqld-bin.001040
Could not copy bin log file /var/lib/mysql-zrm/PayIt/20100301190102/mysqld-bin.001041
Could not copy bin log file /var/lib/mysql-zrm/PayIt/20100301190102/mysqld-bin.001042

Any ideas on what could be causing this issue? Could it be a network issue at the time of copying?

Thanks in advance for any and all thoughts on this matter - Brandon (phishee)

[paddy]

Are you using socket copy plugin? Can you check socket-server.log in /var/log/mysql-zrm directory on the MySQL server for errors?

Paddy

[paddy]

I think you have opened a support case with Zmanda for this problem and the problem is being resolved.

Paddy

[pyeatman]

It turns out that the reason for this in this case is because backups of the same client are occurring at the same time. If a backup is in progress for a particular client which uses the Socket Copy Plugin, the copy for any additional backups requiring the Socket Copy Plugin will fail if started for the same client. This is because the Socket Copy Plugin, which runs as the mysql-zrm-socket-server network service on the client, allows only one connection at a time. This in turn is because of the following value for "instances" in its xinetd file /etc/xinetd.d/mysql-zrm-socket-server

instances = 1

This value is set to "1" because there would be a problem with multiple connections if simultaneous backups of different MySQL servers occurred on the same client at the same time. If you are not backing up more than one MySQL server on a client, you can safely increase the allowed number of connection instances to a number greater than "1" to allow simultaneous backups on a client for a single MySQL server.