PDA

View Full Version : How do I configure zrm to perform a mixture of full and incremental backups?



motin
May 22nd, 2007, 02:08 AM
When using zrm it seems to me (thanks to the point in time restore feature) that it should be enough to perform a weekly full logical backup and then hourly incremental ones. This makes it possible to perform point in time restorations for every week that the full backup and the incremental ones are saved.

My current setup is one "hourlyrun" performing incremental backups and one weeklyrun with full backups according to these crontab rows:

0 2 * * 0 /usr/bin/zrm-pre-scheduler --action backup --backup-set weeklyrun --backup-level 0 --interval weekly
30 * * * * /usr/bin/mysql-zrm-scheduler --now --backup-set hourlyrun --backup-level 1

But it doesn't seem possible to use these backup sets together since they are not based on one another meaning I can never delete a backup in the hourlyrun directory if I want to be able to perform a point in time restore...

Am I just confused by the name of the default backup set - dailyrun?

Should I instead use a configuration like this? :
0 2 * * 0 /usr/bin/zrm-pre-scheduler --action backup --backup-set weeklycollection --backup-level 0 --interval weekly
30 * * * * /usr/bin/mysql-zrm-scheduler --now --backup-set weeklycollection --backup-level 1

Will the incremental backup then see if the last performed backup is a full one and if so base the incremental data on this full backup - making it possible for me to move away all backups older than the last full one to an offline archive and still now that I will be able to perform a full point in time restoration to any time from this full weekly backup and forwards in time?

kkg
May 23rd, 2007, 01:39 AM
Should I instead use a configuration like this? :
0 2 * * 0 /usr/bin/zrm-pre-scheduler --action backup --backup-set weeklycollection --backup-level 0 --interval weekly
30 * * * * /usr/bin/mysql-zrm-scheduler --now --backup-set weeklycollection --backup-level 1



The following is probably all that you need to do (unless I have misunderstood what your requirement is).

0 2 * * 0 /usr/bin/zrm-pre-scheduler --action backup --backup-set weeklycollection --backup-level 0 --interval weekly
30 * * * * /usr/bin/mysql-pre-scheduler --action backup --backup-set weeklycollection --backup-level 0 --interval daily

motin
May 29th, 2007, 05:20 AM
Your suggestion does not include hourly backups.

I have this in crontab now:



# Old backup strategy
#0 3 * * * /usr/bin/zrm-pre-scheduler --action backup --backup-set dailyrun --backup-level 0 --interval daily
#0 2 * * 0 /usr/bin/zrm-pre-scheduler --action backup --backup-set weeklyrun --backup-level 0 --interval weekly
#30 * * * * /usr/bin/mysql-zrm-scheduler --now --backup-set hourlyrun --backup-level 1

# Weekly full backups
0 2 * * 0 /usr/bin/zrm-pre-scheduler --action backup --backup-set weeklycollection --backup-level 0 --interval weekly
0 4 * * * /usr/bin/mysql-zrm --action purge

# Incremental backups every hour
30 * * * * /usr/bin/mysql-zrm-scheduler --now --backup-set weeklycollection --backup-level 1


The main question is: If I have a full backup from say May 13th and then hourly incremental backups up to say May 16th - can I then do a point in time restore to say May 15th 14:37 ?

kkg
May 29th, 2007, 08:08 PM
Your suggestion does not include hourly backups.


My typo, I actually wanted to illustrate that you do not need to have different backup sets for full and incremental.



I have this in crontab now:



# Old backup strategy
#0 3 * * * /usr/bin/zrm-pre-scheduler --action backup --backup-set dailyrun --backup-level 0 --interval daily
#0 2 * * 0 /usr/bin/zrm-pre-scheduler --action backup --backup-set weeklyrun --backup-level 0 --interval weekly
#30 * * * * /usr/bin/mysql-zrm-scheduler --now --backup-set hourlyrun --backup-level 1

# Weekly full backups
0 2 * * 0 /usr/bin/zrm-pre-scheduler --action backup --backup-set weeklycollection --backup-level 0 --interval weekly
0 4 * * * /usr/bin/mysql-zrm --action purge

# Incremental backups every hour
30 * * * * /usr/bin/mysql-zrm-scheduler --now --backup-set weeklycollection --backup-level 1


The main question is: If I have a full backup from say May 13th and then hourly incremental backups up to say May 16th - can I then do a point in time restore to say May 15th 14:37 ?

Yes you can.

If I was setting up the schedule I would make a small tweak as following


# Weekly full backups
0 2 * * 0 /usr/bin/zrm-pre-scheduler --action backup --backup-set weeklycollection --backup-level 0 --interval weekly
0 4 * * * /usr/bin/mysql-zrm --action purge

# Incremental backups every hour
30 * * * * /usr/bin/mysql-pre-scheduler --action backup --backup-set weeklycollection --backup-level 1 --interval daily



The only reason for this is for enabling me to use the pre-scheduler plugin to decide if you do not want a particular incremental backup not to happen based on some of my site policies.

motin
May 31st, 2007, 10:43 AM
When I configure site policies I will probably use the scheduler as well. I didn't know that it has possible to do hourly backups with the scheduler. It is a bit strange that one should use the "daily" interval for this as well, but that is a later issue.

Thank you kkg! You've helped me configure a secure, space and resource-efficient backup strategy which allows point in time restores. Amanda ZRM must be the best backup tool for mysql databases there is!

Now I only have to configure the perfect restoration script (http://forums.zmanda.com/showthread.php?p=2175#post2175) :)

motin
May 31st, 2007, 12:45 PM
As you can see in the other thread, I am not creating this backup script after all. Too much manual and thus unsecure operations.

I am delighted that you have confirmed that restoration is possible through many incremental backups.

However, reading http://www.howtoforge.com/point_in_time_restoration_mysql_zrm only one incremental backup after the full backup is used for point in time restoration.

According to http://mysqlbackup.zmanda.com/index.php/Complete_Restoration_of_Full_%26_Incremental_Backu ps it should be enough to point to a source directory of an incremental backup for a full recovery. Apparently zrm will go backwards in time and find the latest full backup?

If this is true then the procedure of making a point in time restore of a particular database, once I have decided a specific date/time to be say 20070530140710, as easy as (I have a incremental backup from 20070530143002):


mysql-zrm --action restore --backup-set weeklycollection \
--source-directory /var/lib/mysql-zrm/weeklycollection/20070530143002 \
--stop-datetime "20070530140710" \
--databases "databasename"

Am I correct?

kkg
May 31st, 2007, 08:42 PM
As you can see in the other thread, I am not creating this backup script after all. Too much manual and thus unsecure operations.

I am delighted that you have confirmed that restoration is possible through many incremental backups.

However, reading http://www.howtoforge.com/point_in_time_restoration_mysql_zrm only one incremental backup after the full backup is used for point in time restoration.

According to http://mysqlbackup.zmanda.com/index.php/Complete_Restoration_of_Full_%26_Incremental_Backu ps it should be enough to point to a source directory of an incremental backup for a full recovery. Apparently zrm will go backwards in time and find the latest full backup?

If this is true then the procedure of making a point in time restore of a particular database, once I have decided a specific date/time to be say 20070530140710, as easy as (I have a incremental backup from 20070530143002):


mysql-zrm --action restore --backup-set weeklycollection \
--source-directory /var/lib/mysql-zrm/weeklycollection/20070530143002 \
--stop-datetime "20070530140710" \
--databases "databasename"

Am I correct?


As of today the --stop-datetime is only for within what you haqve specified. It will not look outside of that.
So if you specify the --source-directory then the --stop-datetime is used to stop at the given point in time in that particular backup only.
But in case you use the --bin-logs and give it a list of binary log files, then it will look at all of the binary logs you have specified and stop at that given point in time.

So I would suggest you do the following for restores.
a) First restore from the full backup.
b) Restart the mysql server if needed.
c) Find out the list of incremental backups you need to restore using the mysql-zrm-reporter utility. For example suppose the list is
/var/lib/mysql-zrm/20070528143002
/var/lib/mysql-zrm/20070529143002
/var/lib/mysql-zrm/20070530143002
d) Finally use the following command to do the full restore


mysql-zrm --action restore --backup-set weeklycollection \
--bin-logs "/var/lib/mysql-zrm/20070528143002/bin-log.[0-9]* /var/lib/mysql-zrm/20070529143002/bin-log.[0-9]* /var/lib/mysql-zrm/20070530143002/bin-log.[0-9]*"\
--stop-datetime "20070530140710" \
--databases "databasename"

So this is a little more involved as of today.

Doing the one command restore where you just specify the stop-datetime is on the roadmap and you will be seeing this feature in the product in the next few months.

--kkg