mosaicing

mosaicing — build image mosaics

Stability Level

Stable, unless otherwise indicated

Synopsis

#include <vips/vips.h>

int                 im_lrmerge                          (VipsImage *ref,
                                                         VipsImage *sec,
                                                         VipsImage *out,
                                                         int dx,
                                                         int dy,
                                                         int mwidth);
int                 im_tbmerge                          (VipsImage *ref,
                                                         VipsImage *sec,
                                                         VipsImage *out,
                                                         int dx,
                                                         int dy,
                                                         int mwidth);
int                 im_lrmerge1                         (VipsImage *ref,
                                                         VipsImage *sec,
                                                         VipsImage *out,
                                                         int xr1,
                                                         int yr1,
                                                         int xs1,
                                                         int ys1,
                                                         int xr2,
                                                         int yr2,
                                                         int xs2,
                                                         int ys2,
                                                         int mwidth);
int                 im_tbmerge1                         (VipsImage *ref,
                                                         VipsImage *sec,
                                                         VipsImage *out,
                                                         int xr1,
                                                         int yr1,
                                                         int xs1,
                                                         int ys1,
                                                         int xr2,
                                                         int yr2,
                                                         int xs2,
                                                         int ys2,
                                                         int mwidth);
int                 im_lrmosaic                         (VipsImage *ref,
                                                         VipsImage *sec,
                                                         VipsImage *out,
                                                         int bandno,
                                                         int xref,
                                                         int yref,
                                                         int xsec,
                                                         int ysec,
                                                         int hwindowsize,
                                                         int hsearchsize,
                                                         int balancetype,
                                                         int mwidth);
int                 im_tbmosaic                         (VipsImage *ref,
                                                         VipsImage *sec,
                                                         VipsImage *out,
                                                         int bandno,
                                                         int xref,
                                                         int yref,
                                                         int xsec,
                                                         int ysec,
                                                         int hwindowsize,
                                                         int hsearchsize,
                                                         int balancetype,
                                                         int mwidth);
int                 im_lrmosaic1                        (VipsImage *ref,
                                                         VipsImage *sec,
                                                         VipsImage *out,
                                                         int bandno,
                                                         int xr1,
                                                         int yr1,
                                                         int xs1,
                                                         int ys1,
                                                         int xr2,
                                                         int yr2,
                                                         int xs2,
                                                         int ys2,
                                                         int hwindowsize,
                                                         int hsearchsize,
                                                         int balancetype,
                                                         int mwidth);
int                 im_tbmosaic1                        (VipsImage *ref,
                                                         VipsImage *sec,
                                                         VipsImage *out,
                                                         int bandno,
                                                         int xr1,
                                                         int yr1,
                                                         int xs1,
                                                         int ys1,
                                                         int xr2,
                                                         int yr2,
                                                         int xs2,
                                                         int ys2,
                                                         int hwindowsize,
                                                         int hsearchsize,
                                                         int balancetype,
                                                         int mwidth);
int                 im_global_balance                   (VipsImage *in,
                                                         VipsImage *out,
                                                         double gamma);
int                 im_global_balancef                  (VipsImage *in,
                                                         VipsImage *out,
                                                         double gamma);
int                 im_correl                           (VipsImage *ref,
                                                         VipsImage *sec,
                                                         int xref,
                                                         int yref,
                                                         int xsec,
                                                         int ysec,
                                                         int hwindowsize,
                                                         int hsearchsize,
                                                         double *correlation,
                                                         int *x,
                                                         int *y);
int                 im_remosaic                         (VipsImage *in,
                                                         VipsImage *out,
                                                         const char *old_str,
                                                         const char *new_str);
int                 im_align_bands                      (VipsImage *in,
                                                         VipsImage *out);
int                 im_maxpos_subpel                    (VipsImage *in,
                                                         double *x,
                                                         double *y);

Description

These functions are useful for joining many small images together to make one large image. They can cope with unstable contrast and arbitary sub-image layout, but will not do any geometric correction. Geometric errors should be removed before using these functions.

The mosaicing functions can be grouped into layers:

The lowest level functions are im_correl(), im_lrmerge() and im_tbmerge(). im_correl() searches a large image for a small sub-image, returning the position of the best sub-image match. im_lrmerge() and im_tbmerge() join two images together left-right or up-down with a smooth seam.

Next, im_lrmosaic() and im_tbmosaic() use the search function plus the two low-level merge operations to join two images given just an approximate overlap as a start point.

The functions im_lrmosaic1() and im_tbmosaic1() are first-order analogues of the basic mosaic functions: they take two approximate tie-points and use them to rotate and scale the right-hand or bottom image before starting to join.

Finally, im_global_balance() can be used to remove contrast differences in a mosaic which has been assembled with these functions. It takes the mosaic apart, measures image contrast differences along the seams, finds a set of correction factors which will minimise these differences, and reassembles the mosaic. im_remosaic() uses the same techniques, but will reassemble the image from a different set of source images.

Details

im_lrmerge ()

int                 im_lrmerge                          (VipsImage *ref,
                                                         VipsImage *sec,
                                                         VipsImage *out,
                                                         int dx,
                                                         int dy,
                                                         int mwidth);

This operation joins two images left-right (with ref on the left) with a smooth seam.

If the number of bands differs, one of the images must have one band. In this case, an n-band image is formed from the one-band image by joining n copies of the one-band image together, and then the two n-band images are operated upon.

The two input images are cast up to the smallest common type (see table Smallest common format in arithmetic).

dx and dy give the displacement of sec relative to ref, in other words, the vector to get from the origin of sec to the origin of ref, in other words, dx will generally be a negative number.

mwidth limits the maximum width of the blend area. A value of "-1" means "unlimited". The two images are blended with a raised cosine.

Pixels with all bands equal to zero are "transparent", that is, zero pixels in the overlap area do not contribute to the merge. This makes it possible to join non-rectangular images.

See also: im_lrmosaic(), im_tbmerge(), im_match_linear(), im_insert().

ref :

reference image

sec :

secondary image

out :

output image

dx :

displacement of ref from sec

dy :

displacement of ref from sec

mwidth :

maximum seam width

Returns :

0 on success, -1 on error

im_tbmerge ()

int                 im_tbmerge                          (VipsImage *ref,
                                                         VipsImage *sec,
                                                         VipsImage *out,
                                                         int dx,
                                                         int dy,
                                                         int mwidth);

This operation joins two images top-bottom (with ref on the top) with a smooth seam.

If the number of bands differs, one of the images must have one band. In this case, an n-band image is formed from the one-band image by joining n copies of the one-band image together, and then the two n-band images are operated upon.

The two input images are cast up to the smallest common type (see table Smallest common format in arithmetic).

dx and dy give the displacement of sec relative to ref, in other words, the vector to get from the origin of sec to the origin of ref, in other words, dx will generally be a negative number.

mwidth limits the maximum height of the blend area. A value of "-1" means "unlimited". The two images are blended with a raised cosine.

Pixels with all bands equal to zero are "transparent", that is, zero pixels in the overlap area do not contribute to the merge. This makes it possible to join non-rectangular images.

See also: im_lrmosaic(), im_lrmerge(), im_match_linear(), im_insert().

ref :

reference image

sec :

secondary image

out :

output image

dx :

displacement of ref from sec

dy :

displacement of ref from sec

mwidth :

maximum seam width

Returns :

0 on success, -1 on error

im_lrmerge1 ()

int                 im_lrmerge1                         (VipsImage *ref,
                                                         VipsImage *sec,
                                                         VipsImage *out,
                                                         int xr1,
                                                         int yr1,
                                                         int xs1,
                                                         int ys1,
                                                         int xr2,
                                                         int yr2,
                                                         int xs2,
                                                         int ys2,
                                                         int mwidth);

This operation joins two images left-right (with ref on the left) given a pair of tie-points. sec is scaled and rotated as necessary before the join.

mwidth limits the maximum width of the blend area. A value of "-1" means "unlimited". The two images are blended with a raised cosine.

Pixels with all bands equal to zero are "transparent", that is, zero pixels in the overlap area do not contribute to the merge. This makes it possible to join non-rectangular images.

If the number of bands differs, one of the images must have one band. In this case, an n-band image is formed from the one-band image by joining n copies of the one-band image together, and then the two n-band images are operated upon.

The two input images are cast up to the smallest common type (see table Smallest common format in arithmetic).

See also: im_tbmerge1(), im_lrmerge(), im_insert(), im_global_balance().

ref :

reference image

sec :

secondary image

out :

output image

xr1 :

first reference tie-point

yr1 :

first reference tie-point

xs1 :

first secondary tie-point

ys1 :

first secondary tie-point

xr2 :

second reference tie-point

yr2 :

second reference tie-point

xs2 :

second secondary tie-point

ys2 :

second secondary tie-point

mwidth :

maximum blend width

Returns :

0 on success, -1 on error

im_tbmerge1 ()

int                 im_tbmerge1                         (VipsImage *ref,
                                                         VipsImage *sec,
                                                         VipsImage *out,
                                                         int xr1,
                                                         int yr1,
                                                         int xs1,
                                                         int ys1,
                                                         int xr2,
                                                         int yr2,
                                                         int xs2,
                                                         int ys2,
                                                         int mwidth);

This operation joins two images top-bottom (with ref on the top) given a pair of tie-points. sec is scaled and rotated as necessary before the join.

mwidth limits the maximum height of the blend area. A value of "-1" means "unlimited". The two images are blended with a raised cosine.

Pixels with all bands equal to zero are "transparent", that is, zero pixels in the overlap area do not contribute to the merge. This makes it possible to join non-rectangular images.

If the number of bands differs, one of the images must have one band. In this case, an n-band image is formed from the one-band image by joining n copies of the one-band image together, and then the two n-band images are operated upon.

The two input images are cast up to the smallest common type (see table Smallest common format in arithmetic).

See also: im_lrmerge1(), im_tbmerge(), im_insert(), im_global_balance().

ref :

reference image

sec :

secondary image

out :

output image

xr1 :

first reference tie-point

yr1 :

first reference tie-point

xs1 :

first secondary tie-point

ys1 :

first secondary tie-point

xr2 :

second reference tie-point

yr2 :

second reference tie-point

xs2 :

second secondary tie-point

ys2 :

second secondary tie-point

mwidth :

maximum blend width

Returns :

0 on success, -1 on error

im_lrmosaic ()

int                 im_lrmosaic                         (VipsImage *ref,
                                                         VipsImage *sec,
                                                         VipsImage *out,
                                                         int bandno,
                                                         int xref,
                                                         int yref,
                                                         int xsec,
                                                         int ysec,
                                                         int hwindowsize,
                                                         int hsearchsize,
                                                         int balancetype,
                                                         int mwidth);

This operation joins two images left-right (with ref on the left) given an approximate overlap.

sec is positioned so that the pixel (xsec, ysec) lies on top of the pixel in ref at (xref, yref). The overlap area is divided into three sections, 20 high-contrast points in band bandno of image ref are found in each, and each high-contrast point is searched for in sec using hwindowsize and hsearchsize (see im_correl()).

A linear model is fitted to the 60 tie-points, points a long way from the fit are discarded, and the model refitted until either too few points remain or the model reaches good agreement.

The detected displacement is used with im_lrmerge() to join the two images together.

mwidth limits the maximum width of the blend area. A value of "-1" means "unlimited". The two images are blended with a raised cosine.

Pixels with all bands equal to zero are "transparent", that is, zero pixels in the overlap area do not contribute to the merge. This makes it possible to join non-rectangular images.

If the number of bands differs, one of the images must have one band. In this case, an n-band image is formed from the one-band image by joining n copies of the one-band image together, and then the two n-band images are operated upon.

The two input images are cast up to the smallest common type (see table Smallest common format in arithmetic).

See also: im_lrmerge(), im_tbmosaic(), im_insert(), im_global_balance().

ref :

reference image

sec :

secondary image

out :

output image

bandno :

band to search for features

xref :

position in reference image

yref :

position in reference image

xsec :

position in secondary image

ysec :

position in secondary image

hwindowsize :

half window size

hsearchsize :

half search size

balancetype :

no longer used

mwidth :

maximum blend width

Returns :

0 on success, -1 on error

im_tbmosaic ()

int                 im_tbmosaic                         (VipsImage *ref,
                                                         VipsImage *sec,
                                                         VipsImage *out,
                                                         int bandno,
                                                         int xref,
                                                         int yref,
                                                         int xsec,
                                                         int ysec,
                                                         int hwindowsize,
                                                         int hsearchsize,
                                                         int balancetype,
                                                         int mwidth);

This operation joins two images top-bottom (with ref on the top) given an approximate overlap.

sec is positioned so that the pixel (xsec, ysec) lies on top of the pixel in ref at (xref, yref). The overlap area is divided into three sections, 20 high-contrast points in band bandno of image ref are found in each, and each high-contrast point is searched for in sec using hwindowsize and hsearchsize (see im_correl()).

A linear model is fitted to the 60 tie-points, points a long way from the fit are discarded, and the model refitted until either too few points remain or the model reaches good agreement.

The detected displacement is used with im_tbmerge() to join the two images together.

mwidth limits the maximum height of the blend area. A value of "-1" means "unlimited". The two images are blended with a raised cosine.

Pixels with all bands equal to zero are "transparent", that is, zero pixels in the overlap area do not contribute to the merge. This makes it possible to join non-rectangular images.

If the number of bands differs, one of the images must have one band. In this case, an n-band image is formed from the one-band image by joining n copies of the one-band image together, and then the two n-band images are operated upon.

The two input images are cast up to the smallest common type (see table Smallest common format in arithmetic).

See also: im_tbmerge(), im_lrmosaic(), im_insert(), im_global_balance().

ref :

reference image

sec :

secondary image

out :

output image

bandno :

band to search for features

xref :

position in reference image

yref :

position in reference image

xsec :

position in secondary image

ysec :

position in secondary image

hwindowsize :

half window size

hsearchsize :

half search size

balancetype :

no longer used

mwidth :

maximum blend width

Returns :

0 on success, -1 on error

im_lrmosaic1 ()

int                 im_lrmosaic1                        (VipsImage *ref,
                                                         VipsImage *sec,
                                                         VipsImage *out,
                                                         int bandno,
                                                         int xr1,
                                                         int yr1,
                                                         int xs1,
                                                         int ys1,
                                                         int xr2,
                                                         int yr2,
                                                         int xs2,
                                                         int ys2,
                                                         int hwindowsize,
                                                         int hsearchsize,
                                                         int balancetype,
                                                         int mwidth);

This operation joins two images left-right (with ref on the left) given an approximate pair of tie-points. sec is scaled and rotated as necessary before the join.

Before performing the transformation, the tie-points are improved by searching band bandno in an area of sec of size hsearchsize for a match of size hwindowsize to ref.

mwidth limits the maximum width of the blend area. A value of "-1" means "unlimited". The two images are blended with a raised cosine.

Pixels with all bands equal to zero are "transparent", that is, zero pixels in the overlap area do not contribute to the merge. This makes it possible to join non-rectangular images.

If the number of bands differs, one of the images must have one band. In this case, an n-band image is formed from the one-band image by joining n copies of the one-band image together, and then the two n-band images are operated upon.

The two input images are cast up to the smallest common type (see table Smallest common format in arithmetic).

See also: im_tbmosaic1(), im_lrmerge(), im_insert(), im_global_balance().

ref :

reference image

sec :

secondary image

out :

output image

bandno :

band to search for features

xr1 :

first reference tie-point

yr1 :

first reference tie-point

xs1 :

first secondary tie-point

ys1 :

first secondary tie-point

xr2 :

second reference tie-point

yr2 :

second reference tie-point

xs2 :

second secondary tie-point

ys2 :

second secondary tie-point

hwindowsize :

half window size

hsearchsize :

half search size

balancetype :

no longer used

mwidth :

maximum blend width

Returns :

0 on success, -1 on error

im_tbmosaic1 ()

int                 im_tbmosaic1                        (VipsImage *ref,
                                                         VipsImage *sec,
                                                         VipsImage *out,
                                                         int bandno,
                                                         int xr1,
                                                         int yr1,
                                                         int xs1,
                                                         int ys1,
                                                         int xr2,
                                                         int yr2,
                                                         int xs2,
                                                         int ys2,
                                                         int hwindowsize,
                                                         int hsearchsize,
                                                         int balancetype,
                                                         int mwidth);

This operation joins two images top-bottom (with ref on the top) given an approximate pair of tie-points. sec is scaled and rotated as necessary before the join.

Before performing the transformation, the tie-points are improved by searching band bandno in an area of sec of size hsearchsize for a match of size hwindowsize to ref.

mwidth limits the maximum height of the blend area. A value of "-1" means "unlimited". The two images are blended with a raised cosine.

Pixels with all bands equal to zero are "transparent", that is, zero pixels in the overlap area do not contribute to the merge. This makes it possible to join non-rectangular images.

If the number of bands differs, one of the images must have one band. In this case, an n-band image is formed from the one-band image by joining n copies of the one-band image together, and then the two n-band images are operated upon.

The two input images are cast up to the smallest common type (see table Smallest common format in arithmetic).

See also: im_lrmosaic1(), im_tbmerge(), im_insert(), im_global_balance().

ref :

reference image

sec :

secondary image

out :

output image

bandno :

band to search for features

xr1 :

first reference tie-point

yr1 :

first reference tie-point

xs1 :

first secondary tie-point

ys1 :

first secondary tie-point

xr2 :

second reference tie-point

yr2 :

second reference tie-point

xs2 :

second secondary tie-point

ys2 :

second secondary tie-point

hwindowsize :

half window size

hsearchsize :

half search size

balancetype :

no longer used

mwidth :

maximum blend width

Returns :

0 on success, -1 on error

im_global_balance ()

int                 im_global_balance                   (VipsImage *in,
                                                         VipsImage *out,
                                                         double gamma);

im_global_balance() can be used to remove contrast differences in an assembled mosaic.

It reads the History field attached to in and builds a list of the source images that were used to make the mosaic and the position that each ended up at in the final image.

It opens each of the source images in turn and extracts all parts which overlap with any of the other images. It finds the average values in the overlap areas and uses least-mean-square to find a set of correction factors which will minimise overlap differences. It uses gamma to gamma-correct the source images before calculating the factors. A value of 1.0 will stop this.

Each of the source images is transformed with the appropriate correction factor, then the mosaic is reassembled. out always has the same BandFmt as in. Use im_global_balancef() to get float output and avoid clipping.

There are some conditions that must be met before this operation can work: the source images must all be present under the filenames recorded in the history on in, and the mosaic must have been built using only operations in this package.

See also: im_global_balancef(), im_remosaic().

in :

mosaic to rebuild

out :

output image

gamma :

gamma of source images

Returns :

0 on success, -1 on error

im_global_balancef ()

int                 im_global_balancef                  (VipsImage *in,
                                                         VipsImage *out,
                                                         double gamma);

Just as im_global_balance(), but the output image is always float. This stops overflow or underflow in the case of an extremely unbalanced image mosaic.

See also: im_global_balance(), im_remosaic().

in :

mosaic to rebuild

out :

output image

gamma :

gamma of source images

Returns :

0 on success, -1 on error

im_correl ()

int                 im_correl                           (VipsImage *ref,
                                                         VipsImage *sec,
                                                         int xref,
                                                         int yref,
                                                         int xsec,
                                                         int ysec,
                                                         int hwindowsize,
                                                         int hsearchsize,
                                                         double *correlation,
                                                         int *x,
                                                         int *y);

This operation finds the position of sec within ref.

The area around (xsec, ysec) is searched for the best match to the area around (xref, yref). It searches an area of size hsearchsize for a match of size hwindowsize. The position of the best match is returned, together with the correlation at that point.

Only the first band of each image is correlated. ref and sec may be very large --- the function extracts and generates just the parts needed. Correlation is done with im_spcor(); the position of the maximum is found with im_maxpos().

See also: im_match_linear(), im_match_linear_search(), im_lrmosaic().

ref :

reference image

sec :

secondary image

xref :

position in reference image

yref :

position in reference image

xsec :

position in secondary image

ysec :

position in secondary image

hwindowsize :

half window size

hsearchsize :

half search size

correlation :

return detected correlation

x :

return found position

y :

return found position

Returns :

0 on success, -1 on error

im_remosaic ()

int                 im_remosaic                         (VipsImage *in,
                                                         VipsImage *out,
                                                         const char *old_str,
                                                         const char *new_str);

im_remosaic() works rather as im_global_balance(). It takes apart the mosaiced image in and rebuilds it, substituting images.

Unlike im_global_balance(), images are substituted based on their file‐ names. The rightmost occurence of the string old_str is swapped for new_str, that file is opened, and that image substituted for the old image.

It's convenient for multispectral images. You can mosaic one band, then use that mosaic as a template for mosaicing the others automatically.

See also: im_lrmosaic(), im_global_balance().

in :

mosaic to rebuild

out :

output image

old_str :

gamma of source images

new_str :

gamma of source images

Returns :

0 on success, -1 on error

im_align_bands ()

int                 im_align_bands                      (VipsImage *in,
                                                         VipsImage *out);

This operation uses im_phasecor_fft() to find an integer displacement to align all image bands band 0. It is very slow and not very accurate.

Use im_estpar() in preference: it's fast and accurate.

See also: im_global_balancef(), im_remosaic().

in :

image to align

out :

output image

Returns :

0 on success, -1 on error

im_maxpos_subpel ()

int                 im_maxpos_subpel                    (VipsImage *in,
                                                         double *x,
                                                         double *y);

This function implements:

"Extension of Phase Correlation to Subpixel Registration" by H. Foroosh, from IEEE trans. Im. Proc. 11(3), 2002.

If the best three matches in the correlation are aranged:

02 or 01 1 2

then we return a subpixel match using the ratio of correlations in the vertical and horizontal dimension.

( xs[0], ys[0] ) is the best integer alignment ( xs[ use_x ], ys[ use_x ] ) is equal in y and (+/-)1 off in x ( xs[ use_y ], ys[ use_y ] ) is equal in x and (+/-)1 off in y

Alternatively if the best four matches in the correlation are aranged in a square:

01 or 03 or 02 or 03 32 12 31 21

then we return a subpixel match weighting with the sum the two on each side over the sum of all four, but only if all four of them are very close to the best, and the fifth is nowhere near.

This alternative method is not described by Foroosh, but is often the case where the match is close to n-and-a-half pixels in both dimensions.

See also: im_maxpos(), im_min(), im_stats().

in :

input image

x :

output position of maximum

y :

output position of maximum

Returns :

0 on success, -1 on error