Results 1 to 10 of 10

Thread: compression weirdness

  1. #1

    Default compression weirdness

    Hello all,

    I have got Zmanda working correctly on Solaris x86 ( sunfire x4140 fwiw ) using zfs snapshots with the coolstack implementation of MySQL , Daily full backups with hourly incrementals.

    Everything cushty here...

    mysql-zrm.conf :

    mysql-binpath="/opt/coolstack/mysql/bin"
    host="localhost"
    databases="dash_production"
    password="******"
    user="root"
    snapshot-plugin="/usr/share/mysql-zrm/plugins/zfs-snapshot.pl"
    destination=/data/backup/dash_production

    Tue Jul 29 13:00:00 2008: dashboard-backup:backup:INFO: START OF BACKUP
    Tue Jul 29 13:00:00 2008: dashboard-backup:backup:INFO: PHASE START: Initialization
    Tue Jul 29 13:00:00 2008: dashboard-backup:backup:INFO: backup-set=dashboard-backup
    Tue Jul 29 13:00:00 2008: dashboard-backup:backup:INFO: backup-date=20080729130000
    Tue Jul 29 13:00:00 2008: dashboard-backup:backup:INFO: mysql-server-os=Linux/Unix
    Tue Jul 29 13:00:00 2008: dashboard-backup:backup:INFO: host=localhost
    Tue Jul 29 13:00:00 2008: dashboard-backup:backup:INFO: backup-date-epoch=1217332800
    Tue Jul 29 13:00:00 2008: dashboard-backup:backup:INFO: mysql-zrm-version=ZRM for MySQL Community Edition - version 2.0
    Tue Jul 29 13:00:00 2008: dashboard-backup:backup:INFO: mysql-version=5.1.25-rc-log
    Tue Jul 29 13:00:00 2008: dashboard-backup:backup:INFO: backup-directory=/data/backup/dash_production/dashboard-backup/20080729130000
    Tue Jul 29 13:00:00 2008: dashboard-backup:backup:INFO: backup-level=1
    Tue Jul 29 13:00:00 2008: dashboard-backup:backup:INFO: PHASE END: Initialization
    Tue Jul 29 13:00:00 2008: dashboard-backup:backup:INFO: PHASE START: Running pre backup plugin
    Tue Jul 29 13:00:00 2008: dashboard-backup:backup:INFO: PHASE END: Running pre backup plugin
    Tue Jul 29 13:00:00 2008: dashboard-backup:backup:INFO: PHASE START: Flushing logs
    Tue Jul 29 13:00:00 2008: dashboard-backup:backup:INFO: PHASE END: Flushing logs
    Tue Jul 29 13:00:00 2008: dashboard-backup:backup:INFO: PHASE START: Creating incremental backup
    Tue Jul 29 13:00:00 2008: dashboard-backup:backup:INFO: incremental=mysql-bin.[0-9]*
    Tue Jul 29 13:00:00 2008: dashboard-backup:backup:INFO: PHASE END: Creating incremental backup
    Tue Jul 29 13:00:00 2008: dashboard-backup:backup:INFO: PHASE START: Calculating backup size & checksums
    Tue Jul 29 13:00:00 2008: dashboard-backup:backup:INFO: next-binlog=mysql-bin.000051
    Tue Jul 29 13:00:00 2008: dashboard-backup:backup:INFO: last-backup=/data/backup/dash_production/dashboard-backup/20080729120000
    Tue Jul 29 13:00:00 2008: dashboard-backup:backup:INFO: backup-size=0.00 MB
    Tue Jul 29 13:00:00 2008: dashboard-backup:backup:INFO: PHASE END: Calculating backup size & checksums
    Tue Jul 29 13:00:00 2008: dashboard-backup:backup:INFO: read-locks-time=00:00:00
    Tue Jul 29 13:00:00 2008: dashboard-backup:backup:INFO: flush-logs-time=00:00:00
    Tue Jul 29 13:00:00 2008: dashboard-backup:backup:INFO: backup-time=00:00:00
    Tue Jul 29 13:00:00 2008: dashboard-backup:backup:INFO: backup-status=Backup succeeded
    Tue Jul 29 13:00:00 2008: dashboard-backup:backup:INFO: Backup succeeded
    Tue Jul 29 13:00:00 2008: dashboard-backup:backup:INFO: PHASE START: Running post backup plugin
    Tue Jul 29 13:00:00 2008: dashboard-backup:backup:INFO: PHASE END: Running post backup plugin
    Tue Jul 29 13:00:00 2008: dashboard-backup:backup:INFO: PHASE START: Mailing backup report
    Tue Jul 29 13:00:00 2008: dashboard-backup:backup:INFO: PHASE END: Mailing backup report
    Tue Jul 29 13:00:00 2008: dashboard-backup:backup:INFO: PHASE START: Cleanup
    Tue Jul 29 13:00:00 2008: dashboard-backup:backup:INFO: PHASE END: Cleanup
    Tue Jul 29 13:00:00 2008: dashboard-backup:backup:INFO: END OF BACKUP

    Everything looks fine from here until I enable compressed backups, then for *FULL* backups only

    Tue Jul 29 16:00:01 2008: dashboard-backup:backup:INFO: PHASE START: Initialization
    Tue Jul 29 16:00:01 2008: dashboard-backup:backup:INFO: backup-set=dashboard-backup
    Tue Jul 29 16:00:01 2008: dashboard-backup:backup:INFO: backup-date=20080729160001
    Tue Jul 29 16:00:01 2008: dashboard-backup:backup:INFO: mysql-server-os=Linux/Unix
    Tue Jul 29 16:00:01 2008: dashboard-backup:backup:INFO: host=localhost
    Tue Jul 29 16:00:01 2008: dashboard-backup:backup:INFO: backup-date-epoch=1217343601
    Tue Jul 29 16:00:01 2008: dashboard-backup:backup:INFO: mysql-zrm-version=ZRM for MySQL Community Edition - version 2.0
    Tue Jul 29 16:00:01 2008: dashboard-backup:backup:INFO: mysql-version=5.1.25-rc-log
    Tue Jul 29 16:00:01 2008: dashboard-backup:backup:INFO: backup-directory=/data/backup/dash_production/dashboard-backup/20080729160001
    Tue Jul 29 16:00:01 2008: dashboard-backup:backup:INFO: backup-level=0
    Tue Jul 29 16:00:01 2008: dashboard-backup:backup:INFO: backup-mode=raw
    Tue Jul 29 16:00:01 2008: dashboard-backup:backup:INFO: PHASE END: Initialization
    Tue Jul 29 16:00:01 2008: dashboard-backup:backup:INFO: PHASE START: Running pre backup plugin
    Tue Jul 29 16:00:01 2008: dashboard-backup:backup:INFO: PHASE END: Running pre backup plugin
    Tue Jul 29 16:00:01 2008: dashboard-backup:backup:INFO: PHASE START: Flushing logs
    Tue Jul 29 16:00:01 2008: dashboard-backup:backup:INFO: PHASE END: Flushing logs
    Tue Jul 29 16:00:01 2008: dashboard-backup:backup:INFO: PHASE START: Creating snapshot based backup
    Tue Jul 29 16:00:01 2008: dashboard-backup:backup:INFO: File Position Binlog_Do_DB Binlog_Ignore_DB
    mysql-bin.000055 106
    Tue Jul 29 16:00:26 2008: dashboard-backup:backup:INFO: innodb-data=/data/mysql/data/ibdata2;/data/mysql/data/ibdata1;
    Tue Jul 29 16:00:26 2008: dashboard-backup:backup:INFO: innodb-logs=/logs/mysql/dash-live-db1//ib_logfile*
    Tue Jul 29 16:00:26 2008: dashboard-backup:backup:INFO: raw-databases-snapshot=dash_production
    Tue Jul 29 16:00:26 2008: dashboard-backup:backup:INFO: PHASE END: Creating snapshot based backup
    Tue Jul 29 16:00:26 2008: dashboard-backup:backup:INFO: PHASE START: Find table type
    Tue Jul 29 16:00:26 2008: dashboard-backup:backup:INFO: PHASE END: Find table type
    Tue Jul 29 16:00:26 2008: dashboard-backup:backup:INFO: PHASE START: Calculating backup size & checksums
    Tue Jul 29 16:00:27 2008: dashboard-backup:backup:INFO: next-binlog=mysql-bin.000055
    Tue Jul 29 16:00:27 2008: dashboard-backup:backup:INFO: last-backup=/data/backup/dash_production/dashboard-backup/20080729150001
    Tue Jul 29 16:00:27 2008: dashboard-backup:backup:INFO: backup-size=2.96 GB
    Tue Jul 29 16:00:27 2008: dashboard-backup:backup:INFO: PHASE END: Calculating backup size & checksums
    Tue Jul 29 16:00:27 2008: dashboard-backup:backup:INFO: PHASE START: Compression/Encryption
    Tue Jul 29 16:01:03 2008: dashboard-backup:backup:INFO: compress=/usr/bin/gzip
    Tue Jul 29 16:01:03 2008: dashboard-backup:backup:INFO: backup-size-compressed=2.98 MB
    Tue Jul 29 16:01:03 2008: dashboard-backup:backup:ERROR: could not delete /data/backup/dash_production/dashboard-backup/20080729160001/logs/mysql/dash-live-db1
    Tue Jul 29 16:01:03 2008: dashboard-backup:backup:ERROR: could not delete /data/backup/dash_production/dashboard-backup/20080729160001/logs/mysql
    Tue Jul 29 16:01:03 2008: dashboard-backup:backup:ERROR: could not delete /data/backup/dash_production/dashboard-backup/20080729160001/logs
    Tue Jul 29 16:01:03 2008: dashboard-backup:backup:ERROR: could not delete /data/backup/dash_production/dashboard-backup/20080729160001/data/mysql/data
    Tue Jul 29 16:01:03 2008: dashboard-backup:backup:ERROR: could not delete /data/backup/dash_production/dashboard-backup/20080729160001/data/mysql
    Tue Jul 29 16:01:03 2008: dashboard-backup:backup:ERROR: could not delete /data/backup/dash_production/dashboard-backup/20080729160001/data
    Tue Jul 29 16:01:03 2008: dashboard-backup:backup:ERROR: could not delete /data/backup/dash_production/dashboard-backup/20080729160001/dash_production
    Tue Jul 29 16:01:03 2008: dashboard-backup:backup:INFO: PHASE END: Compression/Encryption
    Tue Jul 29 16:01:03 2008: dashboard-backup:backup:INFO: read-locks-time=00:00:00
    Tue Jul 29 16:01:03 2008: dashboard-backup:backup:INFO: flush-logs-time=00:00:00
    Tue Jul 29 16:01:03 2008: dashboard-backup:backup:INFO: compress-encrypt-time=00:06:00
    Tue Jul 29 16:01:03 2008: dashboard-backup:backup:INFO: backup-time=00:01:02
    Tue Jul 29 16:01:03 2008: dashboard-backup:backup:INFO: backup-status=Backup done but with errors
    Tue Jul 29 16:01:03 2008: dashboard-backup:backup:INFO: Backup done but with errors
    Tue Jul 29 16:01:03 2008: dashboard-backup:backup:INFO: PHASE START: Running post backup plugin
    Tue Jul 29 16:01:03 2008: dashboard-backup:backup:INFO: PHASE END: Running post backup plugin
    Tue Jul 29 16:01:03 2008: dashboard-backup:backup:INFO: PHASE START: Mailing backup report
    Tue Jul 29 16:01:04 2008: dashboard-backup:backup:INFO: PHASE END: Mailing backup report
    Tue Jul 29 16:01:04 2008: dashboard-backup:backup:INFO: PHASE START: Cleanup
    Tue Jul 29 16:01:04 2008: dashboard-backup:backup:INFO: PHASE END: Cleanup
    Tue Jul 29 16:01:04 2008: dashboard-backup:backup:INFO: END OF BACKUP

    Looking in the directory that the backup created ....

    drwxr-xr-x 3 root root 3 Jul 29 16:00 logs
    drwxr-xr-x 3 root root 3 Jul 29 16:00 data
    drwx------ 2 mysql mysql 2 Jul 29 16:01 dash_production

    since the zrm-scheduler created those files , I can't see how it's failing to delete them ...


    any clues ?


    TIA
    Steve

  2. #2
    Join Date
    Oct 2006
    Posts
    199

    Default

    The data and log dirs seem to be owned by root. That is why ZRM is unable to delete these.

    Is the datadir, the innodb shared data dir & log dir and its contents owned by root?

    --kkg

  3. #3

    Default

    Please make sure they are owned by "mysql" with the group "mysql" and not root.

  4. #4

    Default

    I have created a complete new config , now the data and logs dir are being created as mysql:mysql

    The innodb data , logs and datadir are all owned by mysql.

    It's still failing on the deletion of the directories , they are however all empty, so the contents have been removed.

  5. #5

    Default

    And if you su to the mysql user on this system and try to remove those directories, what are the results?

    Tue Jul 29 16:01:03 2008: dashboard-backup:backup:ERROR: could not delete /data/backup/dash_production/dashboard-backup/20080729160001/logs/mysql/dash-live-db1
    Tue Jul 29 16:01:03 2008: dashboard-backup:backup:ERROR: could not delete /data/backup/dash_production/dashboard-backup/20080729160001/logs/mysql
    Tue Jul 29 16:01:03 2008: dashboard-backup:backup:ERROR: could not delete /data/backup/dash_production/dashboard-backup/20080729160001/logs
    Tue Jul 29 16:01:03 2008: dashboard-backup:backup:ERROR: could not delete /data/backup/dash_production/dashboard-backup/20080729160001/data/mysql/data
    Tue Jul 29 16:01:03 2008: dashboard-backup:backup:ERROR: could not delete /data/backup/dash_production/dashboard-backup/20080729160001/data/mysql
    Tue Jul 29 16:01:03 2008: dashboard-backup:backup:ERROR: could not delete /data/backup/dash_production/dashboard-backup/20080729160001/data
    Tue Jul 29 16:01:03 2008: dashboard-backup:backup:ERROR: could not delete /data/backup/dash_production/dashboard-backup/20080729160001/dash_production

  6. #6

    Default

    Works perfectly.....

    [email protected][/data/backup/production/production/20080803213920] id
    uid=102(mysql) gid=100(mysql) groups=100(mysql)
    [email protected][/data/backup/production/production/20080803213920] cd /data/backup/dash_production/dashboard-backup/20080729160001
    [email protected][/data/backup/dash_production/dashboard-backup/20080729160001] ls -l
    total 3083
    -rw-r--r-- 1 mysql mysql 3123700 Jul 29 16:01 backup-data
    drwx------ 2 mysql mysql 2 Jul 29 16:01 dash_production
    drwxr-xr-x 3 mysql mysql 3 Jul 29 16:00 data
    -rw-r--r-- 1 mysql mysql 785 Jul 29 16:01 index
    drwxr-xr-x 3 mysql mysql 3 Jul 29 16:00 logs
    -rw-r--r-- 1 mysql mysql 184 Jul 29 16:01 zrm_checksum
    [email protected][/data/backup/dash_production/dashboard-backup/20080729160001] rm -R logs data
    [email protected][/data/backup/dash_production/dashboard-backup/20080729160001] ls -l
    total 3080
    -rw-r--r-- 1 mysql mysql 3123700 Jul 29 16:01 backup-data
    drwx------ 2 mysql mysql 2 Jul 29 16:01 dash_production
    -rw-r--r-- 1 mysql mysql 785 Jul 29 16:01 index
    -rw-r--r-- 1 mysql mysql 184 Jul 29 16:01 zrm_checksum

  7. #7
    Join Date
    Jun 2007
    Posts
    60

    Default

    I remember that there seems to be a bug in version 2.0.

    Do you have a trailing slash at the end for innodb data dir in my.cnf ? Can you please post the contents of my.cnf ?

  8. #8

    Default

    Looks like it could be a winner......

    innodb_data_home_dir = /data/mysql/data/
    innodb_data_file_path = ibdata1:2000M;ibdata2:10M:autoextend
    innodb_log_group_home_dir = /logs/mysql/dash-live-db1/


    ..... but not *quite* . Removing the trailing slash hash got the errors down to 1 now though....




    Tue Aug 5 13:45:02 2008: production:backup:INFO: backup-size-compressed=0.10 MB
    Tue Aug 5 13:45:02 2008: production:backup:ERROR: could not delete /data/backup/production/production/20080805134500/dash_production
    Tue Aug 5 13:45:02 2008: production:backup:INFO: PHASE END: Compression/Encryption
    Tue Aug 5 13:45:02 2008: production:backup:INFO: read-locks-time=00:00:00
    Tue Aug 5 13:45:02 2008: production:backup:INFO: flush-logs-time=00:00:00
    Tue Aug 5 13:45:02 2008: production:backup:INFO: compress-encrypt-time=00:00:10
    Tue Aug 5 13:45:02 2008: production:backup:INFO: backup-time=00:00:02
    Tue Aug 5 13:45:02 2008: production:backup:INFO: backup-status=Backup done but with errors
    Tue Aug 5 13:45:02 2008: production:backup:INFO: Backup done but with errors


    You can still rm -R the dash_production directory with the mysql user, no errors....

    Steve
    Last edited by -stevem-; August 5th, 2008 at 04:53 AM.

  9. #9
    Join Date
    Jun 2007
    Posts
    60

    Default

    Quote Originally Posted by -stevem- View Post
    Looks like it could be a winner......

    innodb_data_home_dir = /data/mysql/data/
    innodb_data_file_path = ibdata1:2000M;ibdata2:10M:autoextend
    innodb_log_group_home_dir = /logs/mysql/dash-live-db1/


    ..... but not *quite* . Removing the trailing slash hash got the errors down to 1 now though....




    Tue Aug 5 13:45:02 2008: production:backup:INFO: backup-size-compressed=0.10 MB
    Tue Aug 5 13:45:02 2008: production:backup:ERROR: could not delete /data/backup/production/production/20080805134500/dash_production
    Tue Aug 5 13:45:02 2008: production:backup:INFO: PHASE END: Compression/Encryption
    Tue Aug 5 13:45:02 2008: production:backup:INFO: read-locks-time=00:00:00
    Tue Aug 5 13:45:02 2008: production:backup:INFO: flush-logs-time=00:00:00
    Tue Aug 5 13:45:02 2008: production:backup:INFO: compress-encrypt-time=00:00:10
    Tue Aug 5 13:45:02 2008: production:backup:INFO: backup-time=00:00:02
    Tue Aug 5 13:45:02 2008: production:backup:INFO: backup-status=Backup done but with errors
    Tue Aug 5 13:45:02 2008: production:backup:INFO: Backup done but with errors


    You can still rm -R the dash_production directory with the mysql user, no errors....

    Steve
    Have you specified the trailing slash anywhere else ? Remove all the trailing slashes and then perform the backup.

  10. #10

    Default

    destination=/data/backup/dash_production

    in the mysql-zrm.conf is the only place that it's defined, and as you can see that's never had a trailing slash...

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •