Bacula 1.29 User's Guide Chapter 14
Back
Tips and Suggestions
Index
Index
Next
When Bacula crashes (Kaboom)

Volume Utility Tools

This document is describes the utility programs written to aid Bacula users and developers in dealing with Volumes external to Bacula.

Specifying the Configuration File

Starting with version 1.27, each of the following programs requires a valid Storage daemon configuration file (actually, the only part of the configuration file that these programs need is the Device resource definitions). This permits the programs to find the configuration parameters for your archive device (generally a tape drive). By default, they read bacula-sd.conf in the current directory, but you may specify a different configuration file using the -c option.

Specifying a Device Name For a Tape

Each of these programs require a device-name where the Volume can be found. In the case of a tape, this is the physical device name such as /dev/nst0 or /dev/rmt/0ubn depending on your system. For the program to work, it must find the identical name in the Device resource of the configuration file. In the case of the tape drive you are not required to specify the volume name. You simply mount the tape and the program will read it. However, if you have multiple volumes or you want to specify the volume name (a good idea), you do so by using a bootstrap file.

Specifying a Device Name For a File

If you are attempting to read or write an archive file rather than a tape, the device-name should be the full path to the archive location including the filename. The filename (last part of the specification) will be stripped and used as the Volume name, and the path (first part before the filename) must have the same entry in the configuration file. So, the path is equivalent to the archive device name, and the filename is equivalent to the volume name.

Specifying Multiple Volumes

If you want to use a Bacula archive that spans two Volumes, and the Volumes are on tape, you will need to specify a bootstrap file in addition to the device name. For example, suppose you want to read tapes tape1 and tape2. First construct a bootstrap file named say, list.bsr which contains:
Volume=test1|test2
where each Volume is separated by a vertical bar. Then simply use:
./bls -b list.bsr /dev/nst0
In the case of Bacula Volumes that are on files, you may simply append volumes as follows:
./bls /tmp/test1\|test2
where the backslash (\) was necessary as a shell escape to permit entering the vertical bar (|).

bls

bls can be used to do an ls type listing of a Bacula tape or file. It is called:
Usage: bls [-d debug_level] <device-name>
       -b <file>       specify a bootstrap file
       -c <file>       specify a configuration file
       -d <level>       specify a debug level
       -e <file>       exclude list
       -i <file>       include list
       -j              list jobs
       -k              list blocks
       -L              list tape label
    (none of above)    list saved files
       -t              use default tape device
       -v              be verbose
       -?              print this message
For example, to list the contents of a tape:
./bls /dev/nst0
Or to list the contents of a file:
./bls /tmp/xxx
Note that, in the case of a file, the Volume name becomes the filename, so in the above example, you will replace the xxx with the name of the volume (file) you wrote.

Normally if no options are specified, bls will produce the equivalent output to the ls -l command for each file on the tape. Using other options listed above, it is possible to display only the Job records, only the tape blocks, etc. For example:

 
./bls /tmp/File002
bls: butil.c:148 Using device: /tmp
drwxrwxr-x   3 kern  kern     4096 2002-10-19 21:08  /home/kern/bacula/k/src/dird/
drwxrwxr-x   2 kern  kern     4096 2002-10-10 18:59  /home/kern/bacula/k/src/dird/CVS/
-rw-rw-r--   1 kern  kern       54 2002-07-06 18:02  /home/kern/bacula/k/src/dird/CVS/Root
-rw-rw-r--   1 kern  kern       16 2002-07-06 18:02  /home/kern/bacula/k/src/dird/CVS/Repository
-rw-rw-r--   1 kern  kern     1783 2002-10-10 18:59  /home/kern/bacula/k/src/dird/CVS/Entries
-rw-rw-r--   1 kern  kern    97506 2002-10-18 21:07  /home/kern/bacula/k/src/dird/Makefile
-rw-r--r--   1 kern  kern     3513 2002-10-18 21:02  /home/kern/bacula/k/src/dird/Makefile.in
-rw-rw-r--   1 kern  kern     4669 2002-07-06 18:02  /home/kern/bacula/k/src/dird/README-config
-rw-r--r--   1 kern  kern     4391 2002-09-14 16:51  /home/kern/bacula/k/src/dird/authenticate.c
-rw-r--r--   1 kern  kern     3609 2002-07-07 16:41  /home/kern/bacula/k/src/dird/autoprune.c
-rw-rw-r--   1 kern  kern     4418 2002-10-18 21:03  /home/kern/bacula/k/src/dird/bacula-dir.conf
...
-rw-rw-r--   1 kern  kern       83 2002-08-31 19:19  /home/kern/bacula/k/src/dird/.cvsignore
bls: Got EOF on device /tmp
84 files found.

Listing Bacula Jobs

If you are listing a Volume to determine what you Jobs to restore, normally the -j option provides you with most of what you will need as long as you don't have multiple clients. For example,
./bls -j /tmp/test1
Volume Record: SessId=2 SessTime=1033762386 JobId=0 DataLen=144
Begin Session Record: SessId=2 SessTime=1033762386 JobId=1 Level=F Type=B
End Session Record: SessId=2 SessTime=1033762386 JobId=1 Level=F Type=B
Begin Session Record: SessId=3 SessTime=1033762386 JobId=2 Level=I Type=B
End Session Record: SessId=3 SessTime=1033762386 JobId=2 Level=I Type=B
Begin Session Record: SessId=4 SessTime=1033762386 JobId=3 Level=I Type=B
End Session Record: SessId=4 SessTime=1033762386 JobId=3 Level=I Type=B
bls: Got EOF on device /tmp
shows a full save followed by two incremental saves.

Adding the -v option will display virtually all information that is available for each record:

Listing Bacula Blocks

Normally except for debug purposes, you will not need to list Bacula blocks (the "primitive" unit of Bacula data on the Volume). However, you can do so with:
./bls -k /tmp/File002
bls: butil.c:148 Using device: /tmp
Block: 1 size=64512
Block: 2 size=64512
...
Block: 65 size=64512
Block: 66 size=19195
bls: Got EOF on device /tmp
End of File on device
By adding the -v option, you can get more information, which can be useful in knowing what sessions were written to the volume:
./bls -k -v /tmp/File002

Volume Label:
Id                : Bacula 0.9 mortal
VerNo             : 10
VolName           : File002
PrevVolName       :
VolFile           : 0
LabelType         : VOL_LABEL
LabelSize         : 147
PoolName          : Default
MediaType         : File
PoolType          : Backup
HostName          :
Date label written: 2002-10-19 at 21:16

Block: 1 blen=64512 First rec FI=VOL_LABEL SessId=1 SessTim=1035062102 Strm=0 rlen=147
Block: 2 blen=64512 First rec FI=6 SessId=1 SessTim=1035062102 Strm=DATA rlen=4087
Block: 3 blen=64512 First rec FI=12 SessId=1 SessTim=1035062102 Strm=DATA rlen=5902
Block: 4 blen=64512 First rec FI=19 SessId=1 SessTim=1035062102 Strm=DATA rlen=28382
...
Block: 65 blen=64512 First rec FI=83 SessId=1 SessTim=1035062102 Strm=DATA rlen=1873
Block: 66 blen=19195 First rec FI=83 SessId=1 SessTim=1035062102 Strm=DATA rlen=2973
bls: Got EOF on device /tmp
End of File on device

Armed with the SessionId and the SessionTime, you can extract just about anything.

If you want to know even more, add a second -v to the command line to get a dump of every record in every block.

./bls -k -v -v /tmp/File002
bls: block.c:79 Dump block  80f8ad0: size=64512 BlkNum=1
               Hdrcksum=b1bdfd6d cksum=b1bdfd6d
bls: block.c:92    Rec: VId=1 VT=1035062102 FI=VOL_LABEL Strm=0 len=147 p=80f8b40
bls: block.c:92    Rec: VId=1 VT=1035062102 FI=SOS_LABEL Strm=-7 len=122 p=80f8be7
bls: block.c:92    Rec: VId=1 VT=1035062102 FI=1 Strm=UATTR len=86 p=80f8c75
bls: block.c:92    Rec: VId=1 VT=1035062102 FI=2 Strm=UATTR len=90 p=80f8cdf
bls: block.c:92    Rec: VId=1 VT=1035062102 FI=3 Strm=UATTR len=92 p=80f8d4d
bls: block.c:92    Rec: VId=1 VT=1035062102 FI=3 Strm=DATA len=54 p=80f8dbd
bls: block.c:92    Rec: VId=1 VT=1035062102 FI=3 Strm=MD5 len=16 p=80f8e07
bls: block.c:92    Rec: VId=1 VT=1035062102 FI=4 Strm=UATTR len=98 p=80f8e2b
bls: block.c:92    Rec: VId=1 VT=1035062102 FI=4 Strm=DATA len=16 p=80f8ea1
bls: block.c:92    Rec: VId=1 VT=1035062102 FI=4 Strm=MD5 len=16 p=80f8ec5
bls: block.c:92    Rec: VId=1 VT=1035062102 FI=5 Strm=UATTR len=96 p=80f8ee9
bls: block.c:92    Rec: VId=1 VT=1035062102 FI=5 Strm=DATA len=1783 p=80f8f5d
bls: block.c:92    Rec: VId=1 VT=1035062102 FI=5 Strm=MD5 len=16 p=80f9668
bls: block.c:92    Rec: VId=1 VT=1035062102 FI=6 Strm=UATTR len=95 p=80f968c
bls: block.c:92    Rec: VId=1 VT=1035062102 FI=6 Strm=DATA len=32768 p=80f96ff
bls: block.c:92    Rec: VId=1 VT=1035062102 FI=6 Strm=DATA len=32768 p=8101713
bls: block.c:79 Dump block  80f8ad0: size=64512 BlkNum=2
               Hdrcksum=9acc1e7f cksum=9acc1e7f
bls: block.c:92    Rec: VId=1 VT=1035062102 FI=6 Strm=contDATA len=4087 p=80f8b40
bls: block.c:92    Rec: VId=1 VT=1035062102 FI=6 Strm=DATA len=31970 p=80f9b4b
bls: block.c:92    Rec: VId=1 VT=1035062102 FI=6 Strm=MD5 len=16 p=8101841
...

bextract

Normally, you will restore files by running a Restore Job from the Console program. However, bextract can be used to extract a single file or a list of files from a Bacula tape or file. In fact, bextract can be a useful tool to restore files to an empty system assuming you are able to boot, you have statically linked bextract and you have an appropriate bootstrap file.

It is called:

 
Usage: bextract [-d debug_level] <device-name> <directory-to-store-files>
       -b        specify a bootstrap file
       -dnn            set debug level to nn
       -e <file>       exclude list
       -i <file>       include list
       -?              print this message
where device-name is the Archive Device (raw device name or full filename) of the device to be read, and directory-to-store-files is a path prefix to prepend to all the files restored.

NOTE: On Windows systems, if you specify a prefix of say d:/tmp, any file that would have been restored to c:/My Documents will be restored to d:/tmp/My Documents. That is the original drive specification will be stripped. If no prefix is specified, the file will be restored to the original drive.

Extracting with Include or Exclude Lists

Using the -e option, you can specify a file containing a list of files to be excluded. Wildcards can be used in the exclusion list. This option will normally be used in conjunction with the -i option (see below). Both the -e and the -i options may be specified at the same time as the -b option. The bootstrap filters will be applied first, then the include list, then the exclude list.

Likewise, and probably more importantly, with the -i option, you can specify a file that contains a list (one file per line) of files and directories to include for restoration. The list must contain the full filename with the path. If you specify a path name only, it should be terminated with a slash, and all files and subdirectories of that path will be restored. If you specify a line containing only the filename (e.g. my-file.txt) it probably will not be extracted because you have not specified the full path.

For example, if the file include-list contains:

/home/kern/bacula
/usr/local/bin
N.B. Please do not include a trailing slash on directory names. This confuses Bacula.

Then the command:

./bextract -i include-list /dev/nst0 /tmp
will restore from the Bacula archive /dev/nst0 all files and directories in the backup from /home/kern/bacula and from /usr/local/bin. The restored files will be placed in a file of the original name under the directory /tmp (i.e. /tmp/home/kern/bacula/... and /tmp/usr/local/bin/...).

Extracting With a Bootstrap File

The -b option is used to specify a bootstrap file containing the information needed to restore precisely the files you want. Specifying a bootstrap file is optional but recommended because it gives you the most control over which files will be restored. For more details on the bootstrap file, please see Restoring Files with the Bootstrap File chapter of this document. Note, you may also use a bootstrap file produced by the restore command. For example:
./bextract -b bootstrap-file /dev/nst0 /tmp
The bootstrap file allows detailed specification of what files you want restored (extracted). You may specify a bootstrap file and include and/or exclude files at the same time. The bootstrap conditions will first be applied, and then each file record seen will be compared to the include and exclude lists.

Extracting From Multiple Volumes

If you wish to extract files that span several Volumes, you can specify the Volume names in the bootstrap file or you may specify the Volume names on the command line by separating them with a vertical bar. See the section above under the bls program entitled Listing Multiple Volumes for more information. The same techniques apply equally well to the bextract program.

bscan

The bscan program can be used to re-create a database (catalog) from the backup information written to one or more Volumes. With some care, it can also be used to synchronize your catalog with a Volume (not currently recommended as it is untested).

After the loss of a hard disk, if you do not have a valid bootstrap file for reloading your system, you may want to consider using bscan to re-create your database, which can then be used to restore your system to its previous state.

It is called:

 
Usage: bscan [options] 
       -b bootstrap      specify a bootstrap file
       -dnn              set debug level to nn
       -m                update media info in database
       -n name           specify the database name (default bacula)
       -u user           specify database user name (default bacula)
       -p password       specify database password (default none)
       -r                list records
       -s                synchronize or store in database
       -v                verbose
       -w dir            specify working directory (default /tmp)
       -?                print this message
If you are using SQLite as your database, you must supply the -w option and provide the path to the "working-directory" which contains the database (normally bacula.db).

If you are using MySQL, there is no need to supply a working directory since MySQL knows where its databases are.

If you have provided security on your MySQL database, you may also need to supply either the database name (-b option), the user name (-u option), and/or the password (-p) options.

Using bscan to Compare a Volume to an existing Catalog

If you wish to compare the contents of a Volume to an existing catalog without changing the catalog, you can safely do so if and only if you do not specify either the -m or the -s options. However, at this time (Bacula version 1.26), the comparison routines are not as good or as thorough as they should be, so we don't particularly recommend this mode other than for testing.

Using bscan to Create a new Catalog from a Volume

This is the mode for which bscan is most useful. Normally, you should start wit a freshly created catalog that contains no data. For example, use the ./drop_bacula_tables and ./make_bacula_tables before running bscan.

PLEASE TAKE CARE NOT TO DELETE A VALID DATABASE IN USING ./drop_bacula_tables.

Starting with a fresh database and say a single Volume named TestVolume1, you would create a bootstrap file (bootstrap.bsr) that contains:

Volume="TestVolume1"
If there are multiple volumes, add them to the first line of the bootstrap file separated by vertical bar symbols (|). Then you would run a command such as:
./bscan -b bootstrap.bsr -v -s -w /usr/bin/bacula/working /dev/nst0
In the above, the bootstrap file (-b) was specified to define the volume or volumes to scan, the -v option for verbose output (this can be omitted if desired), the -s option that tells bscan to store information in the database, the location of your working-directory (-w option) as you previously defined it, and finally the physical device name /dev/nst0.

For example, after having done a full backup of a directory, then two incrementals, I reinitialized the SQLite database as described above, and using the bootstrap.bsr file noted above, I entered the following command:

./bscan -b bootstrap.bsr -v -s -w /home/kern/bacula/working /dev/nst0
which produced the following output:
bscan: bscan.c:182 Using Database: bacula, User: bacula
bscan: bscan.c:673 Created Pool record for Pool: Default
bscan: bscan.c:271 Pool type "Backup" is OK.
bscan: bscan.c:632 Created Media record for Volume: TestVolume1
bscan: bscan.c:298 Media type "DDS-4" is OK.
bscan: bscan.c:307 VOL_LABEL: OK for Volume: TestVolume1
bscan: bscan.c:693 Created Client record for Client: Rufus
bscan: bscan.c:769 Created new JobId=1 record for original JobId=2
bscan: bscan.c:717 Created FileSet record "Kerns Files"
bscan: bscan.c:819 Updated Job termination record for new JobId=1
bscan: bscan.c:905 Created JobMedia record JobId 1, MediaId 1
bscan: Got EOF on device /dev/nst0
bscan: bscan.c:693 Created Client record for Client: Rufus
bscan: bscan.c:769 Created new JobId=2 record for original JobId=3
bscan: bscan.c:708 Fileset "Kerns Files" already exists.
bscan: bscan.c:819 Updated Job termination record for new JobId=2
bscan: bscan.c:905 Created JobMedia record JobId 2, MediaId 1
bscan: Got EOF on device /dev/nst0
bscan: bscan.c:693 Created Client record for Client: Rufus
bscan: bscan.c:769 Created new JobId=3 record for original JobId=4
bscan: bscan.c:708 Fileset "Kerns Files" already exists.
bscan: bscan.c:819 Updated Job termination record for new JobId=3
bscan: bscan.c:905 Created JobMedia record JobId 3, MediaId 1
bscan: Got EOF on device /dev/nst0
bscan: bscan.c:652 Updated Media record at end of Volume: TestVolume1
bscan: bscan.c:428 End of Volume. VolFiles=3 VolBlocks=57 VolBytes=10,027,437
The key points to note are that bscan prints a line when each major record is created. Due to the volume of output, it does not print a line for each file record unless you supply the -v option twice or more on the command line.

In the case of a Job record, the new JobId will not normally be the same as the original Jobid. For example, for the first JobId above, the new JobId is 1 but the original JobId is 2. This is nothing to be concerned about as it is the normal nature of databases. bscan will keep everything straight.

Although bscan claims that it created a Client record for Client: Rufus three times, it was actually only created the first time. This is normal.

You will also notice that it read an end of file after each Job (Got EOF on device ...). Finally the last line gives the total statistics for the bscan.

If you had added a second -v option to the command line, Bacula would have been even more verbose, dumping virtually all the details of each Job record it encountered.

Now if you start Bacula and enter a list jobs command to the console program, you will get:


+-------+----------+------------------+------+-----+----------+----------+---------+
| JobId | Name     | StartTime        | Type | Lvl | JobFiles | JobBytes | JobStat |
+-------+----------+------------------+------+-----+----------+----------+---------+
| 1     | kernsave | 2002-10-07 14:59 | B    | F   | 84       | 4180207  | T       |
| 2     | kernsave | 2002-10-07 15:00 | B    | I   | 15       | 2170314  | T       |
| 3     | kernsave | 2002-10-07 15:01 | B    | I   | 33       | 3662184  | T       |
+-------+----------+------------------+------+-----+----------+----------+---------+

which corresponds virtually identically with what the database contained before it was re-initialized and restored with bscan. All the Jobs and Files found on the tape are restore including most of the Media record. The Volume (Media) records restored will be marked as Full so that they cannot be rewritten without operator intervention.

It should be noted that bscan cannot restore a database to the exact condition it was in previously because a lot of the less important information contained in the database is not saved to the tape. Nevertheless, the reconstruction is sufficiently complete that you can run restore against it and get valid results.

Using bscan to Correct the Volume File Count

If the Storage daemon crashes during a backup Job, the catalog will no be properly updated for the Volume being used at the time of the crash. This means that the Storage daemon will have written say 20 files on the tape, but the catalog record for the Volume indicates only 19 files.

Currently, Bacula simply complains about this situation but in a future version (later than 1.26), Bacula will refuse to write on a tape that contains a different number of files from what is in the catalog. To correct this situation, you may run a bscan with the -m option (but without the -s option) to update only the final Media record for the Volumes read. This feature is implemented but has not yet been tested.

bcopy

The bcopy program can be used to copy one Bacula archive file to another. For example, you may copy a tape to a file, a file to a tape, a file to a file, or a tape to a tape. For tape to tape, you will need two tape drives. (a later version is planned that will buffer it to disk).

bcopy Command Options

Usage: bcopy [-d debug_level]  
       -b bootstrap      specify a bootstrap file
       -c          specify configuration file
       -dnn              set debug level to nn
       -v                verbose
       -w dir            specify working directory (default /tmp)
       -?                print this message

By using a bootstrap file, you can copy parts of a Bacula archive file to another archive.

One of the objectives of this program is to be able to recover as much data as possible from a damaged tape. However, the current version does not yet have this feature.

As this is a new program, any feedback on its use would be appreciated.

btape

This program permits a number of elementary tape operations via a tty command interface. The test command, described below, can be very useful for testing older tape drive compatibility problems. Aside from initial testing of tape drive compatibility with Bacula, btape will be mostly used by developers writing new tape drivers.

btape can be dangerous to use with existing Bacula tapes because it will relabel a tape or write on the tape if so requested regardless that the tape may contain valuable data, so please be careful and use it only on blank tapes.

To work properly, btape needs to read the Storage daemon's configuration file. As a default, it will look for bacula-sd.conf in the current directory. If your configuration file is elsewhere, please use the -c option to specify where.

The physical device name must be specified on the command line, and that this same device name must be present in the Storage daemon's configuration file read by btape

Usage: btape [-c config_file] [-d debug_level] [device_name]
       -c <file>   set configuration file to file
       -dnn        set debug level to nn
       -s          turn off signals
       -t          open the default tape device
       -?          print this message.

Using btape to Verify your Tape Drive

An important reason for this program is to ensure that a Storage daemon configuration file is defined so that Bacula will correctly read and write tapes.

It is highly recommended that you run the test command before running your first Bacula job to ensure that the parameters you have defined for your storage device (tape drive) will permit Bacula to function properly. You only need to mount a blank tape, enter the command, and the output should be reasonably self explanatory. For example:

(ensure that Bacula is not running)
./btape -c /usr/bin/bacula/bacula-sd.conf /dev/nst0
The output will be:
Tape block granularity is 1024 bytes.
btape: btape.c:376 Using device: /dev/nst0
*
Enter the test command:
test
The output produced should be something similar to the following:
btape: btape.c:652 Append files test.

I'm going to write one record  in file 0,
                   two records in file 1,
             and three records in file 2

btape: btape.c:410 Rewound /dev/nst0
btape: btape.c:782 Wrote one record of 32768 bytes.
btape: btape.c:790 Wrote block to device.
btape: btape.c:440 Wrote EOF to /dev/nst0
btape: btape.c:782 Wrote one record of 32768 bytes.
btape: btape.c:790 Wrote block to device.
btape: btape.c:782 Wrote one record of 32768 bytes.
btape: btape.c:790 Wrote block to device.
btape: btape.c:440 Wrote EOF to /dev/nst0
btape: btape.c:782 Wrote one record of 32768 bytes.
btape: btape.c:790 Wrote block to device.
btape: btape.c:782 Wrote one record of 32768 bytes.
btape: btape.c:790 Wrote block to device.
btape: btape.c:782 Wrote one record of 32768 bytes.
btape: btape.c:790 Wrote block to device.
btape: btape.c:440 Wrote EOF to /dev/nst0
btape: btape.c:410 Rewound /dev/nst0
btape: btape.c:668 Now moving to end of media.
btape: btape.c:530 Moved to end of media
btape: btape.c:670
We should be in file 3. I am at file 3. This is correct! <========

btape: btape.c:673
Now I am going to attempt to append to the tape.
btape: btape.c:782 Wrote one record of 32768 bytes.
btape: btape.c:790 Wrote block to device.
btape: btape.c:440 Wrote EOF to /dev/nst0
btape: btape.c:410 Rewound /dev/nst0
1 block of 64512 bytes in file 0                         <========
End of File mark.
2 blocks of 64512 bytes in file 1                        <========
End of File mark.
3 blocks of 64512 bytes in file 2                        <========
End of File mark.
1 block of 64512 bytes in file 3                         <========
End of File mark.
End of File mark.
End of tape
Total files=4, blocks=7, bytes = 451584                  <========
btape: btape.c:679 The above scan should have four files of:
One record, two records, three records, and one record respectively.

btape: btape.c:683 Append block test.

I'm going to write a block, an EOF, rewind, go to EOM,
then backspace over the EOF and attempt to append a second block in the first file.

btape: btape.c:410 Rewound /dev/nst0
btape: btape.c:782 Wrote one record of 32768 bytes.
btape: btape.c:790 Wrote block to device.
btape: btape.c:440 Wrote EOF to /dev/nst0
btape: btape.c:410 Rewound /dev/nst0
btape: btape.c:530 Moved to end of media
btape: btape.c:693 We should be at file 1. I am at EOM File=1.
                   This is correct! <======
btape: btape.c:695 Doing backspace file.
btape: btape.c:556 Back spaced one file.
btape: btape.c:697 Write second block, hoping to append to first file.
btape: btape.c:782 Wrote one record of 32768 bytes.
btape: btape.c:790 Wrote block to device.
btape: btape.c:440 Wrote EOF to /dev/nst0
btape: btape.c:410 Rewound /dev/nst0
btape: btape.c:701 Done writing, scanning results
2 blocks of 64512 bytes in file 0
End of File mark.
End of File mark.
End of tape
Total files=1, blocks=2, bytes = 129024                  <========
btape: btape.c:703 The above should have one file of two blocks.

If the output put is not essentially identical to the above, please resolve the problem(s) before attempting to use Bacula. I have manually indicated the most important lines with <======, but please review the output carefully.

Tips for Resolving Problems

Incorrect File Number

When Bacula moves to the end of the medium, it normally uses the ioctl(MTEOM) function. Then Bacula uses the ioctl(MTIOCGET) function to retrieve the current file position from the mt_fileno field. Some SCSI tape drivers will use a fast means of seeking to the end of the medium and in doing so, they will not know the current file position and hence return a -1. As a consequence, if you get "This is NOT correct!" in the positioning tests, this may be the cause. You must correct this condition in order for Bacula to work.

There are two possible solutions to the above problem of incorrect file number:

  • Figure out how to configure your SCSI driver to keep track of the file position during the MTEOM request. This is the preferred solution.
  • Modify the Device resource of your bacula-sd.conf file to include:
    Hardware End of File = no
    This will cause Bacula to use the MTFSF request to seek to the end of the medium, and Bacula will keep track of the file number itself.

Incorrect Number of Blocks

Bacula's preferred method of working with tape drives (sequential devices) is to run in variable block mode. All modern tape drives support this mode, but some older drives (in particular the QIC drives) as well as the ATAPI ide-scsi driver run only in fixed block mode.

Even in variable block mode, with the exception of the first record on the second or subsequent volume of a multi-volume backup, Bacula will write blocks of a fixed size. However, in reading a tape, Bacula will assume that for each read request, exactly one block from the tape will be transferred. This the most common way that tape drives work and is well supported by Bacula.

Drives that run in fixed block mode can cause serious problems for Bacula if the drive's block size does not correspond exactly to Bacula's block size. In fixed block size mode, drivers may transmit a partial block or multiple blocks for a single read request. From Bacula's point of view, this destroys the concept of tape blocks. In order for Bacula to run in fixed block mode, you must include the following records in the Storage daemon's Device resource definition:

Minimum Block Size = nnn
Maximum Block Size = nnn
where nnn must be the same for both records and must be identical to the driver's fixed block size.

We recommend that you avoid this configuration if at all possible. In any case, as of version 1.27, it is not at all clear that Bacula's handling of these fixed block drivers really works.

Ensuring that the Tape Modes Are Properly Set

If you have a modern SCSI tape drive and you are having problems with the test command as noted above, it may be that some program has set one or more of the your SCSI driver's options to non-default values. For example, if your driver is set to work in SysV manner, Bacula will not work correctly because it expects BSD behavior. To reset your tape drive to the default values, you can try the following:
become super user
mt -f /dev/nst0 rewind
mt -f /dev/nst0 stoptions buffer-writes async-writes read-ahead
The above command will clear all options and then set those specified. None of the specified options are required by Bacula, but a number of other options such as SysV behavior must not be set.

Using btape to Simulate Bacula Filling a Tape

Because there are often problems with certain tape drives or systems when end of tape conditions occur, btape has a special command fill that causes it to write random data to a tape until the tape fills. It then writes at least one more Bacula block to a second tape. Finally, it reads back both tapes to ensure that the data has been written in a way that Bacula can recover it.

This can be an extremely time consuming process (here is is about 6 hours), and you must have two blank tapes available. As this command is completely new, it is not well tested, and has considerable room for improvement, especially during the error checking in the read-back phase.

To begin this test, you enter the fill command and follow the instructions.

btape Commands

The full list of commands are:
  Command    Description
  =======    ===========
  bsf        backspace file
  bsr        backspace record
  cap        list device capabilities
  clear      clear tape errors
  eod        go to end of Bacula data for append
  test       General test Bacula tape functions
  eom        go to the physical end of medium
  fill       fill tape, write onto second volume
  unfill     read filled tape
  fsf        forward space a file
  fsr        forward space a record
  help       print this command
  label      write a Bacula label to the tape
  load       load a tape
  quit       quit btape
  rd         read tape
  readlabel  read and print the Bacula tape label
  rectest    test record handling functions
  rewind     rewind the tape
  scan       read tape block by block to EOT and report
  status     print tape status
  weof       write an EOF on the tape
  wr         write a single record of 2048 bytes
The most useful commands are:
  • test -- test writing records and EOF marks and reading them back.
  • fill -- completely fill a volume with records, then write a few records on a second volume, and finally, both volumes will be read back.
  • readlabel -- read and dump the label on a Bacula tape.
  • cap -- list the device capabilities as defined in the configuration file and as perceived by the Storage daemon.
The readlabel command can be used to display the details of a Bacula tape label. This can be useful if the physical tape label was lost or damaged.

In the event that you want to relabel a Bacula, you can simply use the label command which will write over any existing label. However, please note for labeling tapes, we recommend that you use the label command in the Console program since it will never overwrite a valid Bacula tape.

Other Programs

The following programs are general utility programs and in general do not need a configuration file nor a device name.

smtp

smtp is a simple mail transport program that permits more flexibility than the standard mail programs typically found on Unix systems. It can even be used on Windows machines.

It is called:

Usage: smtp [-f from] [-h mailhost] [-s subject] [-c copy] [recipient ...]
       -c          set the Cc: field
       -dnn        set debug level to nn
       -f          set the From: field
       -h          use mailhost:port as the SMTP server
       -s          set the Subject: field
       -?          print this message.
If the -f option is not specified, smtp will use your userid. If the <-h> option is not specified smtp will use the value in the environment variable SMTPSERVER or if there is none localhost. By default the port 25 is used.

recipients is a space separated list of email recipients.

The body of the email message is read from standard input.

An example of the use of smtp would be to put the following statement in the Messages resource of your bacula-dir.conf file. Note, these commands should appear on a single line each.

  mailcommand = "/home/bacula/bin/smtp -h mail.domain.com -f \"Bacula <%r>\"
                 -s \"Bacula: %t %e of %c %l\" %r"
  operatorcommand = "/home/bacula/bin/smtp -h mail.domain.com -f \"Bacula <%r>\"
                    -s \"Bacula: Intervention needed for %j\" %r"
Where you replace /home/bacula/bin with the path to your Bacula binary directory, and you replace mail.domain.com with the fully qualified name of your SMTP (email) server, which normally listens on port 25. For more details on the substitution characters (e.g. %r) used in the above line, please see the documentation of the MailCommand in the Messages Resource chapter of this manual.

It is highly recommended that you test one or two cases by hand to make sure that the mailhost that you specified is correct and that it will accept your email requests. Since smtp always uses a TCP connection rather than writing in the spool file, you may find that your from address is being rejected because it does not contain a valid domain, or because your message is caught in your spam filtering rules. Generally, you should specify a fully qualified domain name in the from field.

dbcheck

dbcheck is a simple program that will search for inconsistencies in your database, and optionally fix them. The dbcheck program can be found in the <bacula-source>/src/tools directory of the source distribution. Though it is built with the make process, it is not normally "installed".

It is called:

Usage: dbcheck [-d debug_level] <working-directory>
                  <bacula-database> <user> <password>
       -b               batch mode
       -dnn            set debug level to nn
       -f              fix inconsistencies
       -v              verbose
       -?              print this message
If the -f option is specified, dbcheck will repair (fix) the inconsistencies it finds. Otherwise, it will report only.

If the -b option is specified, dbcheck will run in batch mode, and it will proceed to examine and fix (if -f is set) all programmed inconsistency checks. If the -b option is not specified, dbcheck will enter interactive mode and prompt with the following:

Hello, this is the database check/correct program.
Please select the function you want to perform.

     1) Toggle modify database flag
     2) Toggle verbose flag
     3) Eliminate duplicate Filename records
     4) Eliminate duplicate Path records
     5) Eliminate orphaned Jobmedia records
     6) Eliminate orphaned File records
     7) Eliminate orphaned Path records
     8) Eliminate orphaned Filename records
     9) Eliminate orphaned FileSet records
    10) All (3-9)
    11) Quit
Select function number:
By entering 1 or 2, you can toggle the modify database flag (-f option) and the verbose flag (-v). It can be helpful and reassuring to turn off the modify database flag, then select one or more of the consistency checks (items 3 through 9) to see what will be done, then toggle the modify flag on and re-run the check.

The inconsistencies examined are the following:

  • Duplicate filename records. This can happen if you accidentally run two copies of Bacula at the same time, and they are both adding filenames simultaneously. It is a rare occurrence, but will create an inconsistent database. If this is the case, you will receive error messages during Jobs warning of duplicate database records. If you are not getting these error messages, there is no reason to run this check.
  • Duplicate path records. This can happen if you accidentally run two copies of Bacula at the same time, and they are both adding filenames simultaneously. It is a rare occurrence, but will create an inconsistent database. See the item above for why this occurs and how you know it is happening.
  • Orphaned JobMedia records. This happens when a Job record is deleted (perhaps by a user issued SQL statement), but the corresponding JobMedia record (one for each Volume used in the Job) was not deleted. Normally, this should not happen, and even if it does, these records generally do not take much space in your database. However, by running this check, you can eliminate any such orphans.
  • Orphaned File records. This happens when a Job record is deleted (perhaps by a user issued SQL statement), but the corresponding File record (one for each Volume used in the Job) was not deleted. Note, searching for these records can be very time consuming (i.e. it may take hours) for a large database. Normally this should not happen as Bacula takes care to prevent it. Just the same, this check can remove any orphaned File records. It is recommended that you run this once a year since orphaned File records can take a large amount of space in your database.
  • Orphaned Path records. This condition happens any time a directory is deleted from your system and all associated Job records have been purged. During standard purging (or pruning) of Job records, Bacula does not check for orphaned Path records. As a consequence, over a period of time, old unused Path records will tend to accumulate and use space in your database. This check will eliminate them. It is strongly recommended that you run this check at least once a year.
  • Orphaned Filename records. This condition happens any time a file is deleted from your system and all associated Job records have been purged. This can happen quite frequently as there are quite a large number of files that are created and then deleted. In addition, if you do a system update or delete an entire directory, there can be a very large number of Filename records that remain in the catalog but are no longer used.

    During standard purging (or pruning) of Job records, Bacula does not check for orphaned Filename records. As a consequence, over a period of time, old unused Filename records will to accumulate and use space in your database. This check will eliminate them. It is strongly recommended that you run this check at least once a year, and for large database (more than 200 Megabytes), it is probably better to run this once every 6 months.

testfind

testfind permits listing of files using the same search engine that is used for the Include resource in Job resources. The original use was to ensure that the search engine was correct and to print some statistics on file name and path length. However, you may find it useful to see what bacula would do with a given Include resource. testfind program can be found in the <bacula-source>/src/tools directory of the source distribution. Though it is built with the make process, it is not normally "installed".

It is called:

Usage: testfind [-d debug_level] [-] [pattern1 ...]
       -a          print extended attributes (Win32 debug)
       -dnn        set debug level to nn
       -           read pattern(s) from stdin
       -?          print this message.

Patterns are file inclusion -- normally directories.
Debug level>= 1 prints each file found.
Debug level>= 10 prints path/file for catalog.
Errors always printed.
Files/paths truncated is number with len> 255.
Truncation is only in catalog.
Where a pattern is any filename specification that is valid within an Include resource definition. If none is specified, / (the root directory) is assumed. For example:
./testfind /bin
Would print the following:
Dir: /bin
Reg: /bin/bash
Lnk: /bin/bash2 -> bash
Lnk: /bin/sh -> bash
Reg: /bin/cpio
Reg: /bin/ed
Lnk: /bin/red -> ed
Reg: /bin/chgrp
...
Reg: /bin/ipcalc
Reg: /bin/usleep
Reg: /bin/aumix-minimal
Reg: /bin/mt
Lnka: /bin/gawk-3.1.0 -> /bin/gawk
Reg: /bin/pgawk
Total files    : 85
Max file length: 13
Max path length: 5
Files truncated: 0
Paths truncated: 0
Even though testfind uses the same search engine as Bacula, each directory to be listed, must be entered as a separate command line entry or entered one line at a time to standard input if the - option was specified.

Specifying a debug level of one (i.e. -d1) on the command line will cause testfind to print the raw filenames without showing the Bacula internal file type, or the link (if any). Debug levels of 10 or greater cause the filename and the path to be separated using the same algorithm that is used when putting filenames into the Catalog database.



Back
Tips and Suggestions
Index
Index
Next
When Bacula crashes (Kaboom)
Bacula 1.29 User's Guide
The Network Backup Solution
Copyright © 2000-2003
Kern Sibbald and John Walker