Bacula 1.29 User's Guide Chapter 10
Back
Catalog Maintenance
Index
Index
Next
Using AutoChangers

Automatic Volume Recycling

Normally, Bacula will write on a volume, and once the tape is written, it may append to the volume, but it will never overwrite the data thus destroying it. When we speak of recycling volumes, we mean that Bacula can write over the previous contents of a volume. Thus all previous data will be lost.

If you are like me, you may not want Bacula to automatically recycle (reuse) tapes. This requires a large number of tapes, and at some point, I will manually recycle them. However, many people prefer to have a Pool of tapes that are used for daily backups and recycled once a week, another Pool of tapes that are used for Full backups once a week and recycled monthly, and finally a Pool of tapes that are used once a month and recycled after a year or two. With a scheme like this, your pool of tapes remains constant.

By properly defining your Volume Pools with appropriate Retention periods, Bacula can manage the recycling (such as defined above) automatically.

Automatic recycling of Volumes is controlled by three records in the Pool resource definition in the Director's configuration file. These three records are:

  • AutoPrune = yes
  • VolumeRetention = <time>
  • Recycle = yes
Automatic recycling of Volumes is performed by Bacula only when it wants a new Volume and no appendable Volumes are available in the Pool. It will then search the Pool for any Volumes with the Recycle flag set and whose Volume Status is Full. At that point, the recycling occurs in two steps. The first is that a Volume must be purged of all Jobs and Files, and the second step is the actual recycling of the Volume. The Volume will be purged if the VolumeRetention period has expired. If no volumes can be recycled for any of the reasons stated above, Bacula will request operator intervention (i.e. it will ask you to label a new volume).

Automatic Pruning

By setting AutoPrune to yes you will permit Bacula to automatically prune all Volumes in the Pool when a Job needs another Volume. When a Job requests another volume and there are no Volumes with Volume Status Append available, Bacula will begin volume pruning. This means that all Jobs that are older than the VolumeRetention period will be pruned from every Volume that has Volume Status Full and has Recycle set to yes. When all files are pruned from a Volume, it will be marked as Purged implying that no Jobs remain on the volume. The records that control the pruning are described below.
AutoPrune = <yes/no>
If AutoPrune is set to yes (default), Bacula (version 1.20 or greater) will automatically apply the Volume retention period when it needs a new Volume and no appendable volumes are available. The default is yes.
Volume Retention = <time-period-specification>
The Volume Retention record defines the length of time that Bacula will guarantee that the Volume is not reused counting from the time the last job stored on the Volume terminated.

When this time period expires, and if AutoPrune is set to yes, and a new Volume is needed, but no appendable Volume is available, Bacula will prune (remove) Job records that are older than the specified Volume Retention period.

The Volume Retention period takes precedence over any Job Retention period you have specified in the Client resource. It should also be noted, that the Volume Retention period is obtained by reading the Catalog Database Media record rather than the Pool resource record. This means that if you change the VolumeRetention in the Pool resource record, you must ensure that the corresponding change is made in the catalog.

Retention periods are specified in seconds, but as a convenience, there are a number of modifiers that permit easy specification in terms of minutes, hours, days, weeks, months, quarters, or years on the record. See the Configuration chapter of this manual for additional details of modifier specification.

The default is 1 year.

Recycle = <yes/no>
If Recycle is set to no (the default), then even if Bacula prunes all the Jobs on the volume, it will not consider the tape for recycling. If Recycle is set to yes and all Jobs have been pruned, the volume status will be set to purged and the volume may then be reused when another volume is needed. If the volume is reused, it is relabeled with the same Volume Name and all previous data will be lost.

Recycling

After all Volumes of a Pool have been pruned (as mentioned above, this happens when a Job needs a new Volume and no appendable Volumes are available), Bacula will look for the oldest Volume that is Purged (all Jobs and Files expired), and if the Recycle flag is on (Recycle=yes) for that Volume, Bacula will relabel it and write new data on it.

Recycle Status

Each Volume inherits the Recycle status (yes or no) from the Pool resource record when the Media record is created (normally when the Volume is labeled). This Recycle status is stored in the Media record of the Catalog. Using the the Console program, you may subsequently change the Recycle status for each Volume. For example in the following output from list volumes:
+------------+-----------+-----------+----------+------------------+----------+-------+
| VolumeName | MediaType | VolStatus | VolBytes | LastWritten      | VolReten | Recyc |
+------------+-----------+-----------+----------+------------------+----------+-------+
| File0001   | File      | Full      | 4190055  | 2002-05-25 18:42 | 14400    | 1     |
| File0002   | File      | Full      | 1896460  | 2002-05-26 18:05 | 14400    | 1     |
| File0003   | File      | Full      | 1896460  | 2002-05-26 20:05 | 14400    | 1     |
| File0004   | File      | Full      | 1896460  | 2002-05-26 21:35 | 14400    | 1     |
| File0005   | File      | Full      | 1896460  | 2002-05-26 22:05 | 14400    | 1     |
| File0006   | File      | Full      | 1896460  | 2002-05-26 19:35 | 14400    | 1     |
| File0007   | File      | Purged    | 1896466  | 2002-05-26 18:05 | 14400    | 1     |
+------------+-----------+-----------+----------+------------------+----------+-------+
all the volumes are marked as recyclable, and the last Volume, File0007 has been purged, so it may be immediately recycled. The other volumes are all marked recyclable and when their Volume Retention period (14400 seconds or 4 hours) expires, they will be eligible for pruning, and possible recycling. Even though Volume File0007 has been purged, all the data on the Volume is still recoverable. A purged Volume simply means that there are no entries in the Catalog. Even if the Volume Status is changed to Recycle, the data on the Volume will be recoverable. The data is lost only when the Volume is re-labeled and re-written.

To modify Volume File0001 so that it cannot be recycled, you use the update volume pool=File command in the console program, or simply update and Bacula will prompt you for the information.

+------------+-----------+-----------+----------+------------------+----------+-------+
| VolumeName | MediaType | VolStatus | VolBytes | LastWritten      | VolReten | Recyc |
+------------+-----------+-----------+----------+------------------+----------+-------+
| File0001   | File      | Full      | 4190055  | 2002-05-25 18:42 | 14400    | 0     |
| File0002   | File      | Full      | 1897236  | 2002-05-26 23:35 | 14400    | 1     |
| File0003   | File      | Full      | 1896460  | 2002-05-26 20:05 | 14400    | 1     |
| File0004   | File      | Full      | 1896460  | 2002-05-26 21:35 | 14400    | 1     |
| File0005   | File      | Full      | 1896460  | 2002-05-26 22:05 | 14400    | 1     |
| File0006   | File      | Full      | 1896460  | 2002-05-26 19:35 | 14400    | 1     |
| File0007   | File      | Purged    | 1896466  | 2002-05-26 18:05 | 14400    | 1     |
+------------+-----------+-----------+----------+------------------+----------+-------+
In this case, File0001 will never be automatically recycled. The same effect can be achieved by setting the Volume Status to Read-Only.

Making Bacula Use a Single Tape

Most people will want Bacula to fill a tape and when it is full, a new tape will be mounted, and so on. However, as an extreme example, it is possible for Bacula to write on a single tape, and every night to rewrite it. To get this to work, you must do two things: first, set the VolumeRetention to less than your save period (one day), and the second item is to make Bacula mark the tape as full after using it once. This is done using UseVolumeOnce = yes. If this latter record is not used and the tape is not full after the first time it is written, Bacula will simply append to the tape and eventually request another volume. Using the tape only once, forces the tape to be marked Full after each use, and the next time Bacula runs, it will recycle the tape.

An example Pool resource that does this is:

Pool {
  Name = DDS-4
  Use Volume Once = yes
  Pool Type = Backup
  AutoPrune = yes
  VolumeRetention = 12h # expire after 12 hours
  Recycle = yes
}

A Daily, Weekly, Monthly Tape Usage Example

This example is meant to show you how one could define a fixed set of volumes that Bacula will rotate through on a regular schedule. There are an infinite number of such schemes, all of which have various advantages and disadvantages.

We start with the following assumptions:

  • A single tape has more than enough capacity to do a full save.
  • There are 10 tapes that are used on a daily basis for incremental backups. They are prelabled Daily1 ... Daily10.
  • There are 4 tapes that are used on a weekly basis for full backups. They are labeled Week1 ... Week4.
  • There are 12 tapes that are used on a monthly basis for full backups. They are numbered Month1 ... Month12
  • A full backup is done every Saturday evening (tape inserted Friday evening before leaving work).
  • No backups are done over the weekend (this is easy to change).
  • The first Friday of each month, a Monthly tape is used for the Full backup.
  • Incremental backups are done Monday - Friday (actually Tue-Fri mornings).
We start the system by doing a Full save to one of the weekly volumes or one of the monthly volumes. The next morning, we remove the tape and insert a Daily tape. Friday evening, we remove the Daily tape and insert the next tape in the Weekly series. Monday, we remove the Weekly tape and re-insert the Daily tape. On the first Friday of the next month, we insert the next Monthly tape in the series rather than a Weekly tape, then continue. When a Daily tape finally fills up, Bacula will request the next one in the series, and the next day when you notice the email message, you will mount it and Bacula will finish the unfinished incremental backup.

What does this give? Well, at any point, you will have a the last complete Full save plus several Incremental saves. For any given file your want to recover (or your whole system), you will have a copy of that file every day for at least the last 14 days. For older versions, you will have at least 3 and probably 4 Friday full saves of that file, and going back further, you will have a copy of that file made on the beginning of the month for at least a year.

So you have copies of any file (or your whole system) for at least a year, but as you go back in time, the time between copies increases from daily to weekly to monthly.

What would the Bacula configuration look like to implement such a scheme?

Schedule {
  Name = "NightlySave"
  Run = Level=Full Pool=FullSaves sat at 03:05
  Run = Level=Incremental Pool=Daily tue-fri at 03:05
}

Job {
  Name = "NightlySave"
  Type = Backup
  Level = Full
  Client=XXXXXXXXXX
  FileSet="File Set"
  Messages = Standard
  Storage = DDS-4
  Pool = FullSaves
  Schedule = "NightlySave"
}

Storage {
  Name = DDS-4
  Address = XXXXXXXXXXX
  SDPort = 9103
  Password = XXXXXXXXXXXXX
  Device = FileStorage
  Media Type = 8mm
}

FileSet {
  Name = "File Set"
  Include = signature=MD5 {
    fffffffffffffffff
  }
  Exclude = { *.o }
}

Pool {
  Name = Daily
  Pool Type = Backup
  AutoPrune = yes
  VolumeRetention = 10d   # recycle in 10 days
  Maximum Volumes = 10
  Recycle = yes
}

Pool {
  Name = FullSaves
  Use Volume Once = yes
  Pool Type = Backup
  AutoPrune = yes
  VolumeRetention = 30d  # recycle in 30 days (default)
  Recycle = yes
}
The FullSaves Pool will contain both your weekly and your monthly tapes. However, after labeling your Monthly tapes, you will need to manually set the VolumeRetention period for each tape to 1 year.

In a future version of Bacula, the Schedule resource will be enhanced to have 1sat, 2sat, ... to represent the first, second, etc saturday of the month. When that is available, you could separate the Weekly and the Monthly tapes into two different Pools.

Automatic Pruning and Recycling Example

Perhaps the best way to understand the various resource records that come into play during automatic pruning and recycling is to run a Job that goes through the whole cycle. If you add the following resources to your Director's configuration file:
Schedule {
  Name = "30 minute cycle"
  Run = Level=Full Pool=File Messages=Standard Storage=File hourly at 0:05
  Run = Level=Full Pool=File Messages=Standard Storage=File hourly at 0:35
}

Job {
  Name = "Filetest"
  Type = Backup
  Level = Full
  Client=XXXXXXXXXX
  FileSet="Test Files"
  Messages = Standard
  Storage = File
  Pool = File
  Schedule = "30 minute cycle"
}

Storage {
  Name = File
  Address = XXXXXXXXXXX
  SDPort = 9103
  Password = XXXXXXXXXXXXX
  Device = FileStorage
  Media Type = File
}

FileSet {
  Name = "Test Files"
  Include = signature=MD5 {
    fffffffffffffffff
  }
  Exclude = { *.o }
}

Pool {
  Name = File
  Use Volume Once = yes
  Pool Type = Backup
  LabelFormat = File
  AutoPrune = yes
  VolumeRetention = 4h
  Maximum Volumes = 12
  Recycle = yes
}
Where you will need to replace the ffffffffff's by the appropriate files to be saved for your configuration. For the FileSet Include, choose a directory that has one or two megabytes maximum since there will probably be approximately 8 copies of the directory that Bacula will cycle through.

In addition, you will need to add the following to your Storage daemon's configuration file:

Device {
  Name = FileStorage
  Media Type = File
  Archive Device = /tmp
  LabelMedia = yes;                   # lets Bacula label unlabeled media
  Random Access = Yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
}

With the above resources, Bacula will start a Job every half hour that saves a copy of the directory you chose to /tmp/File0001 ... /tmp/File0012. After 4 hours, Bacula will start recycling the backup Volumes (/tmp/File0001 ...). You should see this happening in the output produced. Bacula will automatically create the Volumes (Files) the first time it uses them.

To turn it off, either delete all the resources you've added, or simply comment out the Schedule record in the Job resource.


Back
Catalog Maintenance
Index
Index
Next
Using AutoChangers
Bacula 1.29 User's Guide
The Network Backup Solution
Copyright © 2000-2003
Kern Sibbald and John Walker