Morphological Image Processing

o o

Basic concepts Fundamental operations Erosion and dilation Compound operations References

Binary images may contain numerous imperfections. In particular, the binary regions produced by simple thresholding are distorted by noise and texture. Morphological image processing pursues the goals of remo ing these imperfections by accounting for the form and structure of the image. !hese techni"ues can be extended to greyscale images.

Basic concepts
Morphological image processing is a collection of non#linear operations related to the shape or morphology of features in an image. $ccording to %i&ipedia, morphological operations rely only on the relati e ordering of pixel alues, not on their numerical alues, and therefore are especially suited to the processing of binary images. Morphological operations can also be applied to greyscale images such that their light transfer functions are un&no'n and therefore their absolute pixel alues are of no or minor interest. Morphological techni"ues probe an image 'ith a small shape or template called a structuring element. !he structuring element is positioned at all possible locations in the image and it is compared 'ith the corresponding neighbourhood of pixels. (ome operations test 'hether the element )fits) 'ithin the neighbourhood, 'hile others test 'hether it )hits) or intersects the neighbourhood*

+robing of an image 'ith a structuring element ,'hite and grey pixels ha e -ero and non#-ero alues, respecti ely..

$ morphological operation on a binary image creates a ne' binary image in 'hich the pixel has a non# -ero alue only if the test is successful at that location in the input image. !he structuring element is a small binary image, i.e. a small matrix of pixels, each 'ith a alue of -ero or one*

!he matrix dimensions specify the size of the structuring element. !he pattern of ones and -eros specifies the shape of the structuring element. $n origin of the structuring element is usually one of its pixels, although generally the origin can be outside the structuring element.

Examples of simple structuring elements.

$ common practice is to ha e odd dimensions of the structuring matrix and the origin defined as the centre of the matrix. (tucturing elements play in moprphological image processing the same role as con olution &ernels in linear image filtering. %hen a structuring element is placed in a binary image, each of its pixels is associated 'ith the corresponding pixel of the neighbourhood under the structuring element. !he structuring element is said to fit the image if, for each of its pixels set to /, the corresponding image pixel is also /. (imilarly, a structuring element is said to hit, or intersect, an image if, at least for one of its pixels set to / the corresponding image pixel is also /.

Fitting and hitting of a binary image 'ith structuring elements s/ and s0.

1ero# alued pixels of the structuring element are ignored, i.e. indicate points 'here the corresponding image alue is irrele ant.

Fundamental operations
More formal descriptions and examples of ho' basic morphological operations 'or& are gi en in the 2ypermedia Image +rocessing Reference ,2I+R. de eloped by 3r. R. Fisher et al. at the 3epartment of $rtificial Intelligence in the 4ni ersity of Edinburgh, (cotland, 45.

Erosion and dilation

!he erosion of a binary image f by a structuring element s ,denoted f s. produces a ne' binary image g 6 f s 'ith ones in all locations , x,y. of a structuring element7s origin at 'hich that structuring element s fits the input image f, i.e. g(x,y) 6 / is s fits f and 8 other'ise, repeating for all pixel coordinates ,x,y..

9reyscale image

Binary image by thresholding

Erosion* a 0:0 s"uare structuring element

Erosion 'ith small ,e.g. 0:0 # ;:;. s"uare structuring elements shrin&s an image by stripping a'ay a

layer of pixels from both the inner and outer boundaries of regions. !he holes and gaps bet'een different regions become larger, and small details are eliminated*

Erosion* a <:< s"uare structuring element ,'''.cs.princeton.edu=>pshilane=class=mosaic=..

?arger structuring elements ha e a more pronounced effect, the result of erosion 'ith a large structuring element being similar to the result obtained by iterated erosion using a smaller structuring element of the same shape. If s/ and s0 are a pair of structuring elements identical in shape, 'ith s0 t'ice the si-e of s/, then f s2 (f s1) s1. Erosion remo es small#scale details from a binary image but simultaneously reduces the si-e of regions of interest, too. By subtracting the eroded image from the original image, boundaries of each region can be found* b 6 f (f s ) 'here f is an image of the regions, s is a <:< structuring element, and b is an image of the region boundaries. !he dilation of an image f by a structuring element s ,denoted f s. produces a ne' binary image g 6 f s 'ith ones in all locations ,x,y. of a structuring element7s orogin at 'hich that structuring element s hits the the input image f, i.e. g(x,y) 6 / if s hits f and 8 other'ise, repeating for all pixel coordinates ,x,y.. 3ilation has the opposite effect to erosion ## it adds a layer of pixels to both the inner and outer boundaries of regions.

Binary image

3ilation* a 0:0 s"uare structuring element

!he holes enclosed by a single region and gaps bet'een different regions become smaller, and small intrusions into boundaries of a region are filled in*

3ilation* a <:< s"uare structuring element ,'''.cs.princeton.edu=>pshilane=class=mosaic=..

Results of dilation or erosion are influenced both by the si-e and shape of a structuring element. 3ilation and erosion are dual operations in that they ha e opposite effects. ?et f c denote the complement of an image f, i.e., the image produced by replacing / 'ith 8 and ice ersa. Formally, the duality is 'ritten as f s 6 f c srot 'here srot is the structuring element s rotated by /@8 . If a structuring element is symmetrical 'ith respect to rotation, then srot does not differ from s. If a binary image is considered to be a collection of connected

regions of pixels set to / on a bac&ground of pixels set to 8, then erosion is the fitting of a structuring element to these regions and dilation is the fitting of a structuring element ,rotated if necessary. into the bac&ground, follo'ed by in ersion of the result.

Compound operations
Many morphological operations are represented as combinations of erosion, dilation, and simple set# theoretic operations such as the complement of a binary image* f c(x,y) 6 / if f(x,y) 6 8, and f c(x,y) 6 8 if f(x,y) 6 /, the intersection h = f g of t'o binary images f and g* h(x,y) 6 / if f(x,y) 6 / and g(x,y) 6 /, and h(x,y) 6 8 other'ise, and the union h = f g of t'o binary images f and g* h(x,y) 6 / if f(x,y) 6 / or g(x,y) 6 /, and h(x,y) 6 8 other'ise*

(et operations on binary images* from left to right* a binary image f, a binary image g, the complement f c of f, the intersection f g, and the union f g.

!he opening of an image f by a structuring element s ,denoted by f s. is an erosion follo'ed by a dilation* f s 6 , f s. s

Binary image

Apening* a 0:0 s"uare structuring element

Apening is so called because it can open up a gap bet'een obBects connected by a thin bridge of pixels. $ny regions that ha e sur i ed the erosion are restored to their original si-e by the dilation*

Binary image f f s ,;:; s"uare. f s ,C:C s"uare. Results of opening 'ith a s"uare structuring element ,'''.mmorph.com=html=morph=mmopen.html=..

Apening is an idempotent operation* once an image has been opened, subse"uent openings 'ith the same structuring element ha e no further effect on that image* ,f s) s. 6 f s. pening

Common !ames" Apening Brief #escription Apening and closing are t'o important operators from mathematical morphology. !hey are both deri ed from the fundamental operations of erosion and dilation. ?i&e those operators they are normally applied to binary images, although there are also grayle el ersions. !he basic effect of an opening is some'hat li&e erosion in that it tends to remo e some of the foreground ,bright. pixels from the edges of regions of foreground pixels. 2o'e er it is less destructi e than erosion in general. $s 'ith other morphological operators, the exact operation is determined by a structuring element. !he effect of the operator is to preser e foreground regions that ha e a similar shape to this structuring element, or that can completely contain the structuring element, 'hile eliminating all other regions of foreground pixels. $o% It &or's Dery simply, an opening is defined as an erosion follo'ed by a dilation using the same structuring element for both operations. (ee the sections on erosion and dilation for details of the indi idual steps. !he opening operator therefore re"uires t'o inputs* an image to be opened, and a structuring element. 9rayle el opening consists simply of a grayle el erosion follo'ed by a grayle el dilation. Apening is the dual of closing, i.e. opening the foreground pixels 'ith a particular structuring element is e"ui alent to closing the bac&ground pixels 'ith the same element. (uidelines for )se %hile erosion can be used to eliminate small clumps of undesirable foreground pixels, e.g. Esalt noise7, "uite effecti ely, it has the big disad antage that it 'ill affect all regions of foreground pixels indiscriminately. Apening gets around this by performing both an erosion and a dilation on the image. !he effect of opening can be "uite easily isuali-ed. Imagine ta&ing the structuring element and sliding it around inside each foreground region, 'ithout changing its orientation. $ll pixels 'hich can be co ered by the structuring element 'ith the structuring element being entirely 'ithin the foreground region 'ill be preser ed. 2o'e er, all foreground pixels 'hich cannot be reached by the structuring element 'ithout parts of it mo ing out of the foreground region 'ill be eroded a'ay. $fter the opening has been carried out, the ne' boundaries of foreground regions 'ill all be such that the structuring element fits inside them, and so further openings 'ith the same element ha e no effect. !he property is &no'n as idempotence. !he effect of an opening on a binary image using a <:< s"uare structuring element is illustrated in Figure /.

Figure 1 Effect of opening using a <:< s"uare structuring element $s 'ith erosion and dilation, it is ery common to use this <:< structuring element. !he effect in the abo e figure is rather subtle since the structuring element is "uite compact and so it fits into the foreground boundaries "uite 'ell e en before the opening operation. !o increase the effect, multiple erosions are often performed 'ith this element follo'ed by the same number of dilations. !his effecti ely performs an opening 'ith a larger s"uare structuring element.

Consider 'hich is a binary image containing a mixture of circles and lines. (uppose that 'e 'ant to separate out the circles from the lines, so that they can be counted. Apening 'ith a dis& shaped

structuring element // pixels in diameter gi es (ome of the circles are slightly distorted, but in general, the lines ha e been almost completely remo ed 'hile the circles remain almost completely unaffected.

!he image sho's another binary image. (uppose that this time 'e 'ish to separately extract the hori-ontal and ertical lines. !he result of an opening 'ith a <:C ertically oriented structuring element is

sho'n in !he image sho's 'hat happens if 'e use a C:< hori-ontally oriented structuring element instead. Fote that there are a fe' glitches in this last image 'here the diagonal lines cross ertical lines. !hese could easily be eliminated, ho'e er, using a slightly longer structuring element. 4nli&e erosion and dilation, the position of the origin of the structuring element does not really matter for opening and closing the result is independent of it. 9rayle el opening can similarly be used to select and preser e particular intensity patterns 'hile attenuating others. $s a simple example 'e start 'ith and then perform grayle el opening 'ith a

flat ;:; s"uare structuring element to produce !he important thing to notice here is the 'ay in 'hich bright features smaller than the structuring element ha e been greatly reduced in intensity, 'hile larger features ha e remained more or less unchanged in intensity. !hus the fine grained hair and 'his&ers in the image ha e been much reduced in intensity, 'hile the nose region is still at much the same intensity as before. Fote that the image does ha e a more matt appearance than before since the opening has eliminated small specularities and texture fluctuations.

(imilarly, opening can be used to remo e Esalt noise7 in images. !he image

sho's an image

containing salt noise, and

sho's the result of opening 'ith a <:< s"uare structuring element. !he

noise has been entirely remo ed 'ith relati ely little degradation of the underlying image. 2o'e er, if the

noise consists of dar& points ,i.e. Epepper noise7. as it can be seen in

grayle el opening yields

2ere, no noise has been remo ed. $t some places 'here t'o nearby noise pixels ha e merged into one larger point, the noise le el has e en been increased. In this case of Epepper noise7, grayle el closing is a more appropriate operator. $s 'e ha e seen, opening can be ery useful for separating out particularly shaped obBects from the bac&ground, but it is far from being a uni ersal 0#3 obBect recogni-er=segmenter. For instance if 'e try and use a long thin structuring element to locate, say, pencils in our image, any one such element 'ill only find pencils at a particular orientation. If it is necessary to find pencils at other orientations then differently oriented elements must be used to loo& for each desired orientation. It is also necessary to be ery careful that the structuring element chosen does not eliminate too many desirable obBects, or retain too many undesirable ones, and sometimes this can be a delicate or e en impossible balance.

Consider, for example,

'hich contains t'o &inds of cell* small, blac& ones and larger, gray

ones. !hresholding the image at a alue of 21 yields in 'hich both &inds of cell are separated from the bac&ground. %e 'ant to retain only the large cells in the image, 'hile remo ing the small ones. !his can be done 'ith straightfor'ard opening. 4sing a 11 pixel circular structuring element yields

Most of the desired cells are in the image, 'hereas none of the blac& cells remained. 2o'e er, 'e cannot find any structuring element 'hich allo's us to detect the small cells and remo e the large ones. E ery structuring element that is small enough to allo' the dar& cells remain in the image 'ould not remo e the

large cells, either. !his is illustrated in structuring element to the thresholded image. Common *ariants

'hich is the result of applying a ! pixel 'ide circular

It is common for opening to be used in conBunction 'ith closing to achie e more subtle effects, as described in the section on closing. !he closing of an image f by a structuring element s ,denoted by f " s. is a dilation follo'ed by an erosion* f " s 6 , f srot. srot

Binary image

Closing* a 0:0 s"uare structuring element

In this case, the dilation and erosion should be performed 'ith a rotated by /@8 structuring element. !ypically, the latter is symmetrical, so that the rotated and initial ersions of it do not differ.

Closing 'ith a <:< s"uare structuring element ,'''.cs.princeton.edu=>pshilane=class=mosaic=..

Closing is so called because it can fill holes in the regions 'hile &eeping the initial region si-es. ?i&e opening, closing is idempotent* ,f " s. G s 6 f " s, and it is dual operation of opening ,Bust as opening is the dual operation of closing.* f " s 6 ,f c s.cH f s 6 ,f c " s.c.

In other 'ords, closing ,opening. of a binary image can be performed by ta&ing the complement of that image, opening ,closing. 'ith the structuring element, and ta&ing the complement of the result. Common *ariants Apening and closing are themsel es often used in combination to achie e more subtle results. If 'e represent the closing of an image f by #(f), and its opening by $(f), then some common combinations include* +roper Apening Min,f, #,$,#,f.... +roper Closing Max,f, $,#,$,f.... $utomedian Filter Max,$,#,$,f..., Min,f, #,$,#,f..... !hese operators are commonly &no'n as morphological filters.

$it+and+Miss ,ransform
!he hit and miss transform ,see also 2I+(0 'eb page. allo's to deri e information on ho' obBects in a binary image are related to their surroundings. !he operation re"uires a matched pair of structuring elements, Is/, s0J, that probe the inside and outside, respecti ely, of obBects in the image* f Is/, s0J 6 ,f s/. K ,f c s0..

Binary image

2it and miss transform* an elongated 0:; structuring element

$ pixel belonging to an obBect is preser ed by the hit and miss transform if and only if s/ translated to that pixel fits inside the obBect $F3 s0 translated to that pixel fits outside the obBect. It is assumed that s/ and s0 do not intersect, other'ise it 'ould be impossible for both fits to occur simultaneously. It is easier to describe it by considering s/ and s0 as a single structuring element 'ith /s for pixels of s/ and 8s for pixels of s0H in this case the hit#and#miss transform assigns / to an output pixel only if the obBect ,'ith the alue of /. and bac&ground ,'ith the alue of 8. pixels in the structuring element exactly match obBect ,/. and bac&ground ,8. pixels in the input image. Ather'ise that pixel is set to the bac&ground alue ,8.. !he hit and miss transform can be used for detecting specific shapes ,spatial arrangements of obBect and bac&ground pixel alues. if the t'o structuring elements present the desired shape, as 'ell as for thinning or thic&ening of obBect linear elements. $it+and+Miss ,ransform

Common !ames" 2it#and#miss !ransform, 2it#or#miss !ransform Brief #escription !he hit#and#miss transform is a general binary morphological operation that can be used to loo& for particular patterns of foreground and bac&ground pixels in an image. It is actually the basic operation of binary morphology since almost all the other binary morphological operators can be deri ed from it. $s 'ith other binary morphological operators it ta&es as input a binary image and a structuring element, and produces another binary image as output. $o% It &or's !he structuring element used in the hit#and#miss is a slight extension to the type that has been introduced for erosion and dilation, in that it can contain both foreground and bac&ground pixels, rather than Bust foreground pixels, i.e. both ones and -eros. Fote that the simpler type of structuring element used 'ith

erosion and dilation is often depicted containing both ones and -eros as 'ell, but in that case the -eros really stand for Edon7t care7s7, and are Bust used to fill out the structuring element to a con enient shaped &ernel, usually a s"uare. In all our illustrations, these Edon7t care7s7 are sho'n as blan&s in the &ernel in order to a oid confusion. $n example of the extended &ind of structuring element is sho'n in Figure /. $s usual 'e denote foreground pixels using ones, and bac&ground pixels using -eros.

Figure 1 Example of the extended type of structuring element used in hit#and#miss operations. !his particular element can be used to find corner points, as explained belo'. !he hit#and#miss operation is performed in much the same 'ay as other morphological operators, by translating the origin of the structuring element to all points in the image, and then comparing the structuring element 'ith the underlying image pixels. If the foreground and bac&ground pixels in the structuring element exactly match foreground and bac&ground pixels in the image, then the pixel underneath the origin of the structuring element is set to the foreground color. If it doesn7t match, then that pixel is set to the bac&ground color. For instance, the structuring element sho'n in Figure / can be used to find right angle con ex corner points in images. Fotice that the pixels in the element form the shape of a bottom#left con ex corner. %e assume that the origin of the element is at the center of the <:< element. In order to find all the corners in a binary image 'e need to run the hit#and#miss transform four times 'ith four different elements representing the four &inds of right angle corners found in binary images. Figure 0 sho's the four different elements used in this operation.

Figure 2 Four structuring elements used for corner finding in binary images using the hit# and#miss transform. Fote that they are really all the same element, but rotated by different amounts. $fter obtaining the locations of corners in each orientation, %e can then simply AR all these images together to get the final result sho'ing the locations of all right angle con ex corners in any orientation. Figure < sho's the effect of this corner detection on a simple binary image.

Figure - Effect of the hit#and#miss based right angle con ex corner detector on a simple binary image. Fote that the Edetector7 is rather sensiti e.

Implementations ary as to ho' they handle the hit#and#miss transform at the edges of images 'here the structuring element o erlaps the edge of the image. $ simple solution is to simply assume that any structuring element that o erlaps the image does not match underlying pixels, and hence the corresponding pixel in the output should be set to -ero. !he hit#and#miss transform has many applications in more complex morphological operations. It is being used to construct the thinning and thic&ening operators, and hence for all applications explained in these 'or&sheets. (uidelines for )se !he hit#and#miss transform is used to loo& for occurrences of particular binary patterns in fixed orientations. It can be used to loo& for se eral patterns ,or alternati ely, for the same pattern in se eral orientations as abo e. simply by running successi e transforms using different structuring elements, and then ARing the results together. !he operations of erosion, dilation, opening, closing, thinning and thic&ening can all be deri ed from the hit#and#miss transform in conBunction 'ith simple set operations. Figure L illustrates some structuring elements that can be used for locating arious binary features.

Figure . (ome applications of the hit#and#miss transform. 1 is used to locate isolated points in a binary image. 2 is used to locate the end points on a binary s&eleton Fote that this structuring element must be used in all its rotations so four hit#and#miss passes are re"uired. -a and -/ are used to locate the triple points ,Bunctions. on a s&eleton. Both structuring elements must be run in all orientations so eight hit#and#miss passes are re"uired. %e illustrate t'o of these applications on an image s&eleton.

%e start 'ith 'hich is the s&eleton of !he image sho's the triple points ,i.e. points 'here three lines meet. of the s&eleton. Fote that the hit#and#miss transform itself merely outputs single foreground pixels at each triple point ,the rest of the output image being blac&.. !o produce our example here, this image 'as then dilated once using a cross#shaped structuring element in order to mar& these isolated points clearly, and this 'as then ARed 'ith the original s&eleton in order to produce

the o erlay. !he image sho's the end points of the s&eleton. !his image 'as produced in a similar 'ay to the triple point image abo e, except of course that a different structuring element 'as used for the hit#and#miss operation. In addition, the isolated points produced by the transform 'ere dilated 'ith a s"uare in order to mar& them, rather than 'ith a cross. !he successful use of the hit#and#miss transform relies on being able to thin& of a relati ely small set of binary patterns that capture all the possible ariations and orientations of a feature that is to be located. For features larger than a fe' pixels across this is often not feasible.

0eferences for $M,

0. (on1ale1 and 0. &oods %igital &mage 'rocessing, $ddison#%esley +ublishing Company, /CC0, p ;0@.

0. $aralic' and 2. 3hapiro #omputer and (obot )ision, Dol. /, $ddison#%esley +ublishing Company, /CC0, Chap ;, pp /M@ # /N<. 4. 5ain *undamentals of %igital &mage 'rocessing, +rentice#2all, /C@C, Chap. C. #. *ernon +achine )ision, +rentice#2all, /CC/, p N;.

Morphological filtering of a binary image is conducted by considering compound operations li&e opening and closing as filters. !hey may act as filters of shape. For example, opening 'ith a disc structuring element smooths corners from the inside, and closing 'ith a disc smooths corners from the outside. But also these operations can filter out from an image any details that are smaller in si-e than the structuring element, e.g. opening is filtering the binary image at a scale defined by the si-e of the structuring element. Anly those portions of the image that fit the structuring element are passed by the filterH smaller structures are bloc&ed and excluded from the output image. !he si-e of the structuring element is most important to eliminate noisy details but not to damage obBects of interest.

0eferences
!hese lecture notes follo' Chapter // )Morphological image processing) of the textboo&

!ic' Efford. Digital Image Processing: A Practical Introduction Using Java TM. Pearson Education6 2777.

'ith extra examples and teaching materials ta&en mostly, 'ith corresponding references, from the %eb.