PDA

View Full Version : Incremental backup fail



Johan Barelds
December 30th, 2007, 05:19 AM
Hi,

I do have a very odd problem.
I make backups from several remote Mysql dbase servers over the internet to a central backup server
The incremental backup of one specific server always fails (full backup works and incremental backups of other servers are working just fine):
---------------------------
shedevil:backup:INFO: backup-level=1
shedevil:backup:INFO: Getting list of binary logs using command mysql --user="phpmyadmin" --password="*****" --host="shedevil" -e "show master logs"

gzip: stdin: unexpected end of file
/bin/tar: Child returned status 1
/bin/tar: Error exit delayed from previous errors
close of pipe failed
shedevil:backup:ERROR: Could not copy file /var/log/mysql/mysql-bin.000352
shedevil:backup:ERROR: copy-plugin exited with error 512
shedevil:backup:ERROR: Could not copy bin log file mysql-bin.000352
--------------------------------

Anyone any clues?

Thanks!

Gr. Johan Barelds

kkg
December 30th, 2007, 09:47 PM
Hi,

I do have a very odd problem.
I make backups from several remote Mysql dbase servers over the internet to a central backup server
The incremental backup of one specific server always fails (full backup works and incremental backups of other servers are working just fine):
---------------------------
shedevil:backup:INFO: backup-level=1
shedevil:backup:INFO: Getting list of binary logs using command mysql --user="phpmyadmin" --password="*****" --host="shedevil" -e "show master logs"

gzip: stdin: unexpected end of file
/bin/tar: Child returned status 1
/bin/tar: Error exit delayed from previous errors
close of pipe failed
shedevil:backup:ERROR: Could not copy file /var/log/mysql/mysql-bin.000352
shedevil:backup:ERROR: copy-plugin exited with error 512
shedevil:backup:ERROR: Could not copy bin log file mysql-bin.000352
--------------------------------

Anyone any clues?

Thanks!

Gr. Johan Barelds


Could you provide the version details of ZRM you are using? Are you using 1.1.4? I remember the issue that you had when using 1.1.4. The solution for that was to add the following in socket-server.pl
Could you add the following line to socket-server.pl

delete $ENV{"ENV"};

after the line
my $MYSQL_BINPATH = "/usr/bin";

This was fixed sometime back and in the post 1.1.4 releases you will find the following statement in socket-server.pl

delete @ENV{'IFS', 'CDPATH', 'ENV', 'BASH_ENV'};

If you are using the latest version of ZRM and if the above does not work, please do the following and let me know what happens

go to the directory where socket-server.pl is installed and run it.
./socket-server.pl

Then give the following inputs in the following order. Press return after each input.

1.4
copy from
/var/lib/mysql/mysql-bin.000352
/tmp
/usr/bin

Once these 5 inputs have been given, can you let me know if you got any output.

--kkg

Johan Barelds
December 31st, 2007, 04:35 PM
Hi kkg,

Thanks for your reply.
See my comments below.


Could you provide the version details of ZRM you are using? Are you using 1.1.4? I remember the issue that you had when using 1.1.4. The solution for that was to add the following in socket-server.pl....

I am using the latest version for the server and client atm (1.2.1-1).
I did also recall that it happened before and tried everything already we discussed in the threat then..



Once these 5 inputs have been given, can you let me know if you got any output.....
--kkg

I tried it and the result is that it starts to flow data after i entered the inputs you mentioned.
So the client part seems to be ok. Maybe the problem is on the server side?
Any clues?

Thanks!
Gr. Johan

kkg
December 31st, 2007, 11:58 PM
Hi kkg,

Thanks for your reply.
See my comments below.



I am using the latest version for the server and client atm (1.2.1-1).
I did also recall that it happened before and tried everything already we discussed in the threat then..



I tried it and the result is that it starts to flow data after i entered the inputs you mentioned.
So the client part seems to be ok. Maybe the problem is on the server side?
Any clues?

Thanks!
Gr. Johan

That is strange. I dont think it is the server since you say that all the other mysql servers get backed up correctly.

BTW did you run this as the mysql user on the particulart mysql server machine?

--kkg

Johan Barelds
January 1st, 2008, 09:38 PM
That is strange. I dont think it is the server since you say that all the other mysql servers get backed up correctly.

BTW did you run this as the mysql user on the particulart mysql server machine?

--kkg

Nope. just as root.
After i discovered this problem i upgraded the backup server to version 1.2.1-1 just to be sure they a re all running on the same version.
Right now i discovered that all other servers are having the same problem with the same error message. So all incremental backup end with errors atm...

Gr. Johan

sunil
January 2nd, 2008, 03:01 AM
In my case it is succeeding:

Please check the following :

1. Run the backup as user mysql:mysql
2. uid ( user id) and gid ( group id) of mysql user on server and client side should be same.
eg;
in my case id if mysql user is

[mysql@localhost ~]$ id mysql
uid=500(mysql) gid=500(mysql) groups=500(mysql)

3. Post the verbose output of your backup as shown below:


[mysql@localhost ~]$ mysql-zrm --action backup --backup-set abc --host 192.168.15.110 --copy-plugin=/usr/share/mysql-zrm/plugins/socket-copy.pl --backup-level 1 --verbose
backup:INFO: ZRM for MySQL Community Edition - version 1.2.1
backup:INFO: Reading options from file /etc/mysql-zrm/mysql-zrm.conf
abc:backup:INFO: Using /usr/share/mysql-zrm/plugins/lvm-snapshot.pl as the default plugin since snapshot-plugin has not been specified
abc:backup:INFO: ZRM Temporary configuration file = /etc/mysql-zrm/abc/tmpf2Vdb.conf
abc:backup:INFO: {
abc:backup:INFO: backup-level=1
abc:backup:INFO: snapshot-plugin=/usr/share/mysql-zrm/plugins/lvm-snapshot.pl
abc:backup:INFO: backup-mode=raw
abc:backup:INFO: host=192.168.15.110
abc:backup:INFO: copy-plugin=/usr/share/mysql-zrm/plugins/socket-copy.pl
abc:backup:INFO: }
abc:backup:INFO: Getting mysql variables
abc:backup:INFO: mysqladmin --host="192.168.15.110" variables
abc:backup:INFO: datadir is /var/lib/mysql/
abc:backup:INFO: mysql_version is 5.0.27-log
abc:backup:INFO: log_bin=ON
abc:backup:INFO: InnoDB data file are /var/lib/mysql/ibdata1
abc:backup:INFO: InnoDB log dir is /var/lib/mysql/.
abc:backup:INFO: backup set being used is abc
abc:backup:INFO: backup-set=abc
abc:backup:INFO: backup-date=20080104090917
abc:backup:INFO: host=192.168.15.110
abc:backup:INFO: backup-date-epoch=1199455757
abc:backup:INFO: mysql-zrm-version=ZRM for MySQL Community Edition - version 1.2.1
abc:backup:INFO: mysql-version=5.0.27-log
abc:backup:INFO: backup-directory=/var/lib/mysql-zrm/abc/20080104090917
abc:backup:INFO: Executing pre-backup-plugin
abc:backup:INFO: Flushing the logs
abc:backup:INFO: flush-logs
abc:backup:INFO: Getting master logname using command mysql --host="192.168.15.110" -e "show master status"
abc:backup:INFO: backup-level=1
abc:backup:INFO: Getting list of binary logs using command mysql --host="192.168.15.110" -e "show master logs"
/bin/tar: localhost-bin.000003: time stamp 2008-01-07 22:04:41 is 305724 s in the future
abc:backup:INFO: incremental=localhost-bin.[0-9]*
abc:backup:INFO: next-binlog=localhost-bin.000004
abc:backup:INFO: last-backup=/var/lib/mysql-zrm/abc/20080104080940
abc:backup:INFO: /var/lib/mysql-zrm/abc/20080104090917/localhost-bin.000003=568d481c98490ece56139aaca8968a91
abc:backup:INFO: backup-size=0.00 MB
abc:backup:INFO: read-locks-time=00:00:00
abc:backup:INFO: flush-logs-time=00:00:00
abc:backup:INFO: backup-time=00:00:01
abc:backup:INFO: backup-status=Backup succeeded
abc:backup:INFO: Backup succeeded
abc:backup:INFO: Executing post-backup-plugin

Johan Barelds
January 2nd, 2008, 10:43 AM
Checked the mysql uid's and guid's but still no luck.
See the output below.
Any other clues?

Gr. Johan

-------------------------------------------------------------------------------------------------
mysql@7ofnine:~> /usr/bin/mysql-zrm-scheduler --now --backup-set shedevil --backup-level 1
schedule:INFO: ZRM for MySQL Community Edition - version 1.2.1
Logging to /var/log/mysql-zrm/mysql-zrm-scheduler.log
backup:INFO: ZRM for MySQL Community Edition - version 1.2.1
shedevil:backup:INFO: Mail address: technischbeheer@pith-it.com is ok
shedevil:backup:INFO: Using /usr/share/mysql-zrm/plugins/lvm-snapshot.pl as the default plugin since snapshot-plugin has not been specified
shedevil:backup:INFO: ZRM Temporary configuration file = /etc/mysql-zrm/shedevil/tmp3M6uj.conf
shedevil:backup:INFO: {
shedevil:backup:INFO: verbose=1
shedevil:backup:INFO: retention-policy=2W
shedevil:backup:INFO: destination=/backup/mysql-zrm
shedevil:backup:INFO: snapshot-plugin=/usr/share/mysql-zrm/plugins/lvm-snapshot.pl
shedevil:backup:INFO: password=******
shedevil:backup:INFO: backup-mode=raw
shedevil:backup:INFO: user=phpmyadmin
shedevil:backup:INFO: copy-plugin=/usr/share/mysql-zrm/plugins/socket-copy.pl
shedevil:backup:INFO: backup-level=1
shedevil:backup:INFO: quiet=0
shedevil:backup:INFO: mailto=technischbeheer@pith-it.com
shedevil:backup:INFO: databases=zarafa
shedevil:backup:INFO: html-reports=backup-status-info,backup-method-info,backup-performance-info,backup-app-performance-info
shedevil:backup:INFO: host=shedevil
shedevil:backup:INFO: html-report-directory=/srv/www/htdocs/mysql-zrm/reports/
shedevil:backup:INFO: compress=
shedevil:backup:INFO: mysql-binlog-path=/var/log/mysql
shedevil:backup:INFO: }
shedevil:backup:INFO: Getting mysql variables
shedevil:backup:INFO: mysqladmin --user="phpmyadmin" --password="*****" --host="shedevil" variables
shedevil:backup:INFO: datadir is /var/lib/mysql/
shedevil:backup:INFO: mysql_version is 5.0.26-log
shedevil:backup:INFO: log_bin=ON
shedevil:backup:INFO: InnoDB data file are /var/lib/mysql/ibdata1
shedevil:backup:INFO: InnoDB log dir is /var/lib/mysql/.
shedevil:backup:INFO: backup set being used is shedevil
shedevil:backup:INFO: backup-set=shedevil
shedevil:backup:INFO: backup-date=20080102203649
shedevil:backup:INFO: host=shedevil
shedevil:backup:INFO: backup-date-epoch=1199302609
shedevil:backup:INFO: retention-policy=2W
shedevil:backup:INFO: mysql-zrm-version=ZRM for MySQL Community Edition - version 1.2.1
shedevil:backup:INFO: mysql-version=5.0.26-log
shedevil:backup:INFO: backup-directory=/backup/mysql-zrm/shedevil/20080102203649
shedevil:backup:INFO: Executing pre-backup-plugin
shedevil:backup:INFO: Flushing the logs
shedevil:backup:INFO: flush-logs
shedevil:backup:INFO: Getting master logname using command mysql --user="phpmyadmin" --password="*****" --host="shedevil" -e "show master status"
shedevil:backup:INFO: backup-level=1
shedevil:backup:INFO: Getting list of binary logs using command mysql --user="phpmyadmin" --password="*****" --host="shedevil" -e "show master logs"

gzip: stdin: unexpected end of file
/bin/tar: Child returned status 1
/bin/tar: Uitgestelde afbreking na eerdere fouten
close of pipe failed
shedevil:backup:ERROR: Could not copy file /var/log/mysql/mysql-bin.000359
shedevil:backup:ERROR: copy-plugin exited with error 512
shedevil:backup:ERROR: Could not copy bin log file mysql-bin.000359
shedevil:backup:INFO: incremental=mysql-bin.[0-9]*
shedevil:backup:INFO: next-binlog=mysql-bin.000360
shedevil:backup:INFO: last-backup=/backup/mysql-zrm/shedevil/20080102203449
shedevil:backup:INFO: backup-size=0.00 MB
shedevil:backup:INFO: Compressing backup
shedevil:backup:INFO: Command used is 'tar --same-owner -cpsC "/backup/mysql-zrm/shedevil/20080102203649" --exclude=backup-data --exclude=index . 2>/tmp/CwHcwCsgst | gzip 2>/tmp/Bibv72QqxC > "/backup/mysql-zrm/shedevil/20080102203649/backup-data" 2>/tmp/DXZ9VoJDNA'
shedevil:backup:INFO: compress=
shedevil:backup:INFO: backup-size-compressed=0.00 MB
shedevil:backup:INFO: Removing all of the uncompressed/unencrypted data
shedevil:backup:INFO: read-locks-time=00:00:00
shedevil:backup:INFO: flush-logs-time=00:00:00
shedevil:backup:INFO: compress-encrypt-time=00:00:00
shedevil:backup:INFO: backup-time=00:00:01
shedevil:backup:INFO: backup-status=Backup done but with errors
shedevil:backup:INFO: Backup done but with errors
shedevil:backup:INFO: Executing post-backup-plugin
shedevil:backup:INFO: mailing file /tmp/OaP60fn5l5
shedevil:backup:INFO: mail command is cat "/tmp/OaP60fn5l5"|mail -s "[ZRM for MySQL Report] backup-set shedevil" technischbeheer@pith-it.com
shedevil:backup:INFO: mailing file /tmp/equDLK6YwJ
shedevil:backup:INFO: mail command is cat "/tmp/equDLK6YwJ"|mail -s "[ZRM for MySQL Report] ERROR during backup of backup-set shedevil" technischbeheer@pith-it.com
ERROR: /usr/bin/mysql-zrm did not finish successfully
-------------------------------------------------------------------------------
/etc/mysql-zrm/shedevil/mysql-zrm.conf
------
backup-mode=raw
destination=/backup/mysql-zrm
retention-policy=2W
compress=1
databases=zarafa
user="********"
password="***********"
host="shedevil"
mysql-binlog-path="/var/log/mysql"
verbose=1
mailto="*********************"
html-reports=backup-status-info,backup-method-info,backup-performance-info,backup-app-performance-info
html-report-directory=/srv/www/htdocs/mysql-zrm/reports/
copy-plugin=/usr/share/mysql-zrm/plugins/socket-copy.pl

Johan Barelds
January 2nd, 2008, 12:42 PM
Hi guys,

I fixed the problems.

First problem is a bug in socket version 1.2.1-1:
If you install this rpm it doesn't make the directory /var/log/mysql-zrm.
If this directory isn't available the backup via the socket-server will fail.

Second problem was a typo in my mysql-zrm.conf file:

mysql-binlog-path="/var/log/mysql"
instead of
mysql-binlog-path="/var/lib/mysql"
Then the binary logs can't be found..:-)

Thanks for the support anyway.
Gr. Johan Barelds

Jayadevan
January 31st, 2008, 12:56 AM
Hi,
I am getting the following error for incremental back-ups.
dailyrun:backup:INFO: Getting master logname using command "/usr/local/mysql-5.0.45-linux-i686-glibc23/bin/"/mysql --user="backup-user" --password="*****" --host="*.*.*.* --port="3306" -e "show master status"
dailyrun:backup:INFO: backup-level=1
dailyrun:backup:INFO: Getting list of binary logs using command "/usr/local/mysql-5.0.45-linux-i686-glibc23/bin/"/mysql --user="backup-user" --password="*****" --host="*.*.*.*" --port="3306" -e "show master logs"
Out of memory!

gzip: stdin: unexpected end of file
/bin/tar: Child returned status 1
/bin/tar: Error exit delayed from previous errors
dailyrun:backup:ERROR: Could not copy file /opt/mysql/data/binlog.000001
dailyrun:backup:ERROR: copy-plugin exited with error 256
dailyrun:backup:ERROR: Could not copy bin log file binlog.000001
Out of memory!

gzip: stdin: unexpected end of file
/bin/tar: Child returned status 1
/bin/tar: Error exit delayed from previous errors
dailyrun:backup:ERROR: Could not copy file /opt/mysql/data/binlog.000002
dailyrun:backup:ERROR: copy-plugin exited with error 256
dailyrun:backup:ERROR: Could not copy bin log file binlog.000002
Out of memory!

I tred ./socket-server.pl with parameters as you mentioned on the machine where MySQL server is running and I got output like
M````````````````````````````````````````````````` ```````````
M````````````````````````````````````````````````` ```````````
M````````````````````````````````````````````````` ```````````



Regards,
Jayadevan