PDA

View Full Version : Copy-plugin errors on incremental and full backups



psyferre
April 4th, 2011, 06:44 PM
Hi Folks,

I recently had to downgrade from the enterprise version back down to the community version. It was supposed to be a simple process, leaving all config intact, but once I uninstalled enterprise and installed the community version package my backups started failing.

No configuration had been changed on the servers, but suddenly I had what appeared to be permissions issues with the ssh copy plugin. I have two backup sets, we'll call them mydb and mydb2.

The mydb2 backup set is running normally, now that I have gone through and run all the commands I have recorded over the last year for troubleshooting. The mydb set fails on both incremental and full backups despite having run the same commands.

On incremental backups, the error is:

mydb_backup:backup:INFO: Getting list of binary logs using command "/usr/bin"/mysql --user="root" --password="*****" --host="mydb" --port="3306" -e "show master logs"
mydb_backup:backup:ERROR: Could not copy file "/var/log/mysql/mysql-bin.009098"
mydb_backup:backup:ERROR: copy-plugin exited with error 512
mydb_backup:backup:ERROR: Could not copy bin log file /db_backups/mydb_backup/20110404210002/mysql-bin.009098
... for each binary log in the directory.

For full backups the error is:

mydb_backup:backup:INFO: getting device details using command /usr/share/mysql-zrm/plugins/lvm-snapshot.pl --action get-vm-device-details --directory /var/lib/mysql/. --sname zrmbvSXB1rmKv --host mydb 2>/tmp/AHYFRN0LyF
mydb_backup:backup:ERROR: Command /usr/share/mysql-zrm/plugins/lvm-snapshot.pl --action get-vm-device-details --directory /var/lib/mysql/. --sname zrmbvSXB1rmKv --host mydb 2>/tmp/AHYFRN0LyF returned error
mydb_backup:backup:WARNING: Error getting device details for /var/lib/mysql/.
mydb_backup:backup:WARNING: Output of command: 'Getting Device Details' is {
copy-plugin returned error

}

Things I've checked:
All binary logs are owned by mysql:mysql.
The uid and gid for the mysql user match on both zrm and mysql server.
The mysql user is in sudoers with full permissions.
The mysql user and root user on the zrm server can login to the mysql server without using a password.

:confused:

Any help woudl be greatly appreciated!

Full log for an incremental run:

mydb_backup:backup:INFO: START OF BACKUP
mydb_backup:backup:INFO: PHASE START: Initialization
mydb_backup:backup:WARNING: The lvm-snapshot option is deprecated. Use snapshot-size instead
mydb_backup:backup:INFO: Mail address: myemail@address is ok
mydb_backup:backup:INFO: ZRM Temporary configuration file = /etc/mysql-zrm/mydb_backup/tmp06lX7.conf
mydb_backup:backup:INFO: {
mydb_backup:backup:INFO: verbose=1
mydb_backup:backup:INFO: retention-policy=2D
mydb_backup:backup:INFO: mysql-binpath=/usr/bin
mydb_backup:backup:INFO: tmpdir=/tmp
mydb_backup:backup:INFO: destination=/db_backups
mydb_backup:backup:INFO: all-databases=1
mydb_backup:backup:INFO: snapshot-plugin=/usr/share/mysql-zrm/plugins/lvm-snapshot.pl
mydb_backup:backup:INFO: password=******
mydb_backup:backup:INFO: backup-mode=raw
mydb_backup:backup:INFO: backup-type=regular
mydb_backup:backup:INFO: rss-header-location=/etc/mysql-zrm/
mydb_backup:backup:INFO: user=root
mydb_backup:backup:INFO: copy-plugin=/usr/share/mysql-zrm/plugins/socket-copy.pl
mydb_backup:backup:INFO: quiet=0
mydb_backup:backup:INFO: backup-level=1
mydb_backup:backup:INFO: lvm-snapshot=5G
mydb_backup:backup:INFO: mailto=myemail@address
mydb_backup:backup:INFO: html-reports=backup-status-info
mydb_backup:backup:INFO: port=3306
mydb_backup:backup:INFO: host=mydb
mydb_backup:backup:INFO: mail-policy=always
mydb_backup:backup:INFO: snapshot-size=6G
mydb_backup:backup:INFO: comment=mydb backup.
mydb_backup:backup:INFO: compress=
mydb_backup:backup:INFO: mysql-binlog-path=/var/log/mysql
mydb_backup:backup:INFO: }
mydb_backup:backup:INFO: Getting mysql variables
mydb_backup:backup:INFO: "/usr/bin"/mysqladmin --user="root" --password="*****" --host="mydb" --port="3306" variables
mydb_backup:backup:INFO: datadir is /var/lib/mysql/
mydb_backup:backup:INFO: mysql_version is 5.1.37-1ubuntu5.4-log
mydb_backup:backup:INFO: InnoDB data file are /var/lib/mysql/ibdata1
mydb_backup:backup:INFO: InnoDB log dir is /var/lib/mysql/.
mydb_backup:backup:INFO: backup set being used is mydb_backup
mydb_backup:backup:INFO: backup-set=mydb_backup
mydb_backup:backup:INFO: backup-date=20110404210002
mydb_backup:backup:INFO: mysql-server-os=Linux/Unix
mydb_backup:backup:INFO: backup-type=regular
mydb_backup:backup:INFO: host=mydb
mydb_backup:backup:INFO: backup-date-epoch=1301965202
mydb_backup:backup:INFO: retention-policy=2D
mydb_backup:backup:INFO: mysql-zrm-version=ZRM for MySQL Community Edition - version 2.2.0
mydb_backup:backup:INFO: mysql-version=5.1.37-1ubuntu5.4-log
mydb_backup:backup:INFO: backup-directory=/db_backups/mydb_backup/20110404210002
mydb_backup:backup:INFO: comment=mydb backup.
mydb_backup:backup:INFO: backup-level=1
mydb_backup:backup:INFO: PHASE END: Initialization
mydb_backup:backup:INFO: PHASE START: Running pre backup plugin
mydb_backup:backup:INFO: Executing pre-backup-plugin
mydb_backup:backup:INFO: PHASE END: Running pre backup plugin
mydb_backup:backup:INFO: PHASE START: Flushing logs
mydb_backup:backup:INFO: Flushing the logs
mydb_backup:backup:INFO: "/usr/bin"/mysqladmin --user="root" --password="*****" --host="mydb" --port="3306" flush-logs
mydb_backup:backup:INFO: Getting master logname using command "/usr/bin"/mysql --user="root" --password="*****" --host="mydb" --port="3306" -e "show master status"
mydb_backup:backup:INFO: PHASE END: Flushing logs
mydb_backup:backup:INFO: PHASE START: Creating incremental backup
mydb_backup:backup:INFO: Getting list of binary logs using command "/usr/bin"/mysql --user="root" --password="*****" --host="mydb" --port="3306" -e "show master logs"
mydb_backup:backup:ERROR: Could not copy file "/var/log/mysql/mysql-bin.009098"
mydb_backup:backup:ERROR: copy-plugin exited with error 512
mydb_backup:backup:ERROR: Could not copy bin log file /db_backups/mydb_backup/20110404210002/mysql-bin.009098
mydb_backup:backup:ERROR: Could not copy file "/var/log/mysql/mysql-bin.009099"
mydb_backup:backup:ERROR: copy-plugin exited with error 512
mydb_backup:backup:ERROR: Could not copy bin log file /db_backups/mydb_backup/20110404210002/mysql-bin.009099

... and on and on for all binary log files....

mydb_backup:backup:INFO: incremental=mysql-bin.[0-9]*
mydb_backup:backup:INFO: PHASE END: Creating incremental backup
mydb_backup:backup:INFO: PHASE START: Calculating backup size & checksums
mydb_backup:backup:INFO: next-binlog=mysql-bin.009127
mydb_backup:backup:INFO: last-backup=/db_backups/mydb_backup/20110403213002
mydb_backup:backup:INFO: backup-size=0.00 MB
mydb_backup:backup:INFO: PHASE END: Calculating backup size & checksums
mydb_backup:backup:INFO: PHASE START: Compression/Encryption
mydb_backup:backup:INFO: Compressing backup
mydb_backup:backup:INFO: Command used is 'tar --same-owner -cpsC "/db_backups/mydb_backup/20110404210002" --exclude=backup-data --exclude=index --exclude=zrm_checksum --exclude=backup-sql . 2>/tmp/9FBAJjxuae | gzip 2>/tmp/cdCXbGrtKX > "/db_backups/mydb_backup/20110404210002/backup-data" 2>/tmp/x2fT0LpzR2'
mydb_backup:backup:INFO: Output of command: 'tar' is {
tar: .: file changed as we read it
}
mydb_backup:backup:INFO: compress=
mydb_backup:backup:INFO: backup-size-compressed=0.00 MB
mydb_backup:backup:INFO: Removing all of the uncompressed/unencrypted data
mydb_backup:backup:INFO: PHASE END: Compression/Encryption
mydb_backup:backup:INFO: read-locks-time=00:00:00
mydb_backup:backup:INFO: flush-logs-time=00:00:00
mydb_backup:backup:INFO: compress-encrypt-time=00:00:00
mydb_backup:backup:INFO: backup-time=00:00:08
mydb_backup:backup:INFO: backup-status=Backup done but with errors
mydb_backup:backup:INFO: Backup done but with errors
mydb_backup:backup:INFO: PHASE START: Running post backup plugin
mydb_backup:backup:INFO: Executing post-backup-plugin
mydb_backup:backup:INFO: PHASE END: Running post backup plugin
mydb_backup:backup:INFO: PHASE START: Mailing backup report
mydb_backup:backup:INFO: mailing file /tmp/oLWQTJvBtn
mydb_backup:backup:INFO: mail command is cat "/tmp/oLWQTJvBtn"|mail -s "[ZRM for MySQL Report] backup-set mydb_backup" myemail@address
mydb_backup:backup:INFO: PHASE END: Mailing backup report
mydb_backup:backup:INFO: PHASE START: Cleanup
mydb_backup:backup:INFO: mailing file /tmp/iPPQbGNUSD
mydb_backup:backup:INFO: mail command is cat "/tmp/iPPQbGNUSD"|mail -s "[ZRM for MySQL Report] ERROR during backup of backup-set mydb_backup" myemail@address
mydb_backup:backup:INFO: PHASE END: Cleanup
mydb_backup:backup:INFO: END OF BACKUP

paddy
April 5th, 2011, 04:52 PM
Check to see if mysql user can read /var/log/mysql/mysql-bin.009098 log file.

Are you running socket copy plugin (see /etc/xinetd.d/mysql-zrm-socket-server) as
mysql user on the MySQL server

psyferre
April 6th, 2011, 09:59 PM
Odd. When you asked me to confirm the user for the socket-copy plugin I did a double-take... We've always used the ssh-copy plugin. Thanks for pointing out my problem! :D

I checked the config and found that the line activating the ssh-copy plugin was missing (not commented out... actually gone) from the default config file. I added it back in and now all my backups are working just fine!

Can you give any hints as to where the enterprise version stores config data? I'd like to see if there were any other options apparently set by the enterprise version that I might be lacking in the community config files.... I had thought they were the same but I guess I was mistaken.

Thanks again!

paddy
April 7th, 2011, 12:54 PM
Enterprise version also stores configuration information in /etc/mysql-zrm directory.