DSP Project

Khalifa University of Science, Technology and Research
Communication Engineering Department

ELEC 402 Signal Processing Laboratory
DSP Project

IMAGE SMOOTHING - VIDEO

Project Performers
Noorhan Bayassi 100032145
Fatima Adly 100033682
Noha Abou Aly 100032275
Ghada Shubair 100020255

Lab Instructor: Mr. Husameldin H. Mukhtar

Fall 2012
Wireless Control of Self Sustained Solar Power Generation System

DSP Project design
2
Table of Contents
Table of Contents _________________________________________________________________________ 2
List of Figures _____________________________________________________________________________ 3
Abstract ____________________________________________________________________________________ 4
1. Introduction ____________________________________________________________________________ 5
1.1 Aims________________________________________________________________________________________________ 9
1.2 Objectives _________________________________________________________________________________________ 9
2. Codes and Results ____________________________________________________________________ 10
3. Conclusion _____________________________________________________________________________ 38


DSP Project design
3

List of Figures

Figure 1: Moving average filter ........................................................................................................................ 6
Figure 2: Gaussian filter...................................................................................................................................... 7
Figure 3: Video ....................................................................................................................................................... 8
Figure 4: Original colored image and the Black & White image ...................................................... 10
Figure 5: Frequency & Phase spectrum of the filter ............................................................................. 13
Figure 6: Impulse response & Z-plane of the filter ............................................................................... 13
Figure 7: 'Smoothing-Horizontal direction .............................................................................................. 14
Figure 8: Smoothing-Vertical direction ..................................................................................................... 15
Figure 9: Impulse response of 2D filter ..................................................................................................... 17
Figure 10: Filtering in Horizontal & Vertical directions by method 1 and method 2 .............. 17
Figure 11: Original image.tif .......................................................................................................................... 18
Figure 12: Smoothing the image in 1D ...................................................................................................... 19
Figure 13: Smoothing the image in 2D (using 2 methods) ................................................................ 19
Figure 14: Original image.png (colored and B&W) .............................................................................. 19
Figure 17: Original image.jpeg(colored and B&W) ............................................................................... 20
Figure 20: Original image.jpeg (colored and B&W) .............................................................................. 22
Figure 21: Frequency & Phase spectrum .................................................................................................. 22
Figure 22: Impulse and Z-plane of the filter ............................................................................................ 22
Figure 242: Impulse response of 2D filter for N=5 ............................................................................... 23
Figure 26: Frequency & Phase spectrum .................................................................................................. 24
Figure 27: Impulse and Z-plane of the filter ............................................................................................ 24
Figure 30: Impulse response of 2D filter .................................................................................................. 25
Figure 31: Moving average filter .................................................................................................................. 28
Figure 32: Binomial filter ................................................................................................................................ 28
Figure 33: Savitzky-Golay filter .................................................................................................................... 28
Figure 35: Noisy image .................................................................................................................................... 30
Figure 38: Cropped image before and after filtering ........................................................................... 34
Figure 39: Correlation ...................................................................................................................................... 34
Figure 40: Video processing .......................................................................................................................... 37


DSP Project design
4

Abstract

In this project we will introduce Image smoothing and then we will move from
image to video. Image smoothing is a way of reprocessing the image in order to eliminate
noise by manipulating the redundancy in image data. This project includes five parts. In the
first part, we will design appropriate 1D low pass filter for horizontal direction. Then, we
will design 1D low pass filter for vertical direction. After that these two filters will be
combined creating a 2D filter. Then, in the analysis part we will test these filters on all test
images, consider changing filters parameters and see their effects. In addition, a
comparison of our designed filter with other smoothing filters will be made. The final part
of this project considers video processing which can be implemented by extending the
image into a time domain.


DSP Project design
5
1. Introduction

In this report we present the outcomes of implementing one of the many types of
image processing techniques, which is image smoothing for videos. Digital image
processing techniques have many applications, but why do we need them in the first place?
Digital image processing is motivated by two major applications; enhancing the quality of
the image so that it improved for human perception, and can be used for efficient storage
and transmission. For example, if we want to store an image on a computer this image will
need certain amount of disk space, applying the image processing properties this needed
space will be reduces. We can also use image processing to extract useful information, or
reduce the noise effect.

In the following sections of this introduction we will home in our point of interest for this
project which is image smoothing, illustrating filters that can be used, image formats and
evaluating metrics, and how image smoothing is implemented for video clips.

Image smoothing:

Image smoothing process recently became an important technique in digital image
processing. It is an attempt to improve the quality of the image and take out information
without compromising the clarity of the image. To smooth an image is to use a technique
that creates an approximating function that picks the important and valuable information
in an image and get rid of any other random or unwanted phenomena. In the smoothing
processing, some spikes in the image that are exposed to noise (such as salt and pepper
noise or Gaussian noise) are adapted, and points that are found to be lower than their
adjacent are increased so that the image as a whole looks smoother. Smoothing utilize an
uncomplicated, fast filtering algorithm that suppresses the noise power so that it protect
the sharp edges in an image.

Filter types:

1) Moving Average filter

The moving average filter or what is called rolling average filter is an FIR (finite impulse
response) filter that has a great role in digital signal processing field. Actually, its name is
coming from its function, as this moving average filter creates series of averages of
different points in the processed data to analyze it.

DSP Project design
6
The average moving filter or the rolling average filter is usually used in smoothing data
processes as it is used to smooth the short term (rapid) variations and to show up the long
term variations. When mathematically analyzing the average moving filter in signal
processing, it can be seen as an example of low pass filter and it is calculated by:

The kind of convolution using the moving average filter can be also called box smoothing.
Its name comes from the way it is implemented as the input image (signal) is convolved
with a pulse that looks like a box shape.

Figure 1: Moving average filter

2) Binomial smoothing

In binomial smoothing, the image is passed through a Gaussian filter, to be convolved with
a Gaussian function. As well known, the Gaussian function has a Gaussian shape in time
domain as well as frequency domain. Therefore, applying a Gaussian filter to an image
causes reducing the high-frequency components of the image which causes elimination the
noise effect.

DSP Project design
7

Figure 2: Gaussian filter

3) Savitzky-Golay smoothing

This smoothing way is sometimes preferred because it has the ability to keep important
properties of distribution like the width and maxima which is not easy for any other kind of
smoothing. In this process, the two important parameters that are the points which are
used in computing smoothing output values and the order of the polynomial are used to
manage smoothing process.

Image formats

There are different image formats that can undergo the smoothing process such as TIF
format which is Tagged Image File, PNG format (Portable Network Graphics) and the JPEG
format that is an abbreviation for Joint Photographic Expert Group. Smoothing process can
be implemented on both colored and B&W images, but in this project we shall deal with 2D
images which means they have no third dimension that is represented by the colors.
Colored images can still be used after removing the colors, i.e. convert the colored 3D image
into a 2D B&W image, and then start smoothing process.


DSP Project design
8
Evaluating matrices

There are many different evaluating tools that can be used to evaluate the quality of the
smoothing process. The evaluating metrics will be used as tools to compare between the
original image before applying the smoothing technique and the output smoothed image
(after filtering). One of these tools is the peak signal to noise ratio (PSNR) where the
maximum signal amplitude (power) is compared to the maximum noise amplitude
(power). Another way for comparison is to use the mean square error that is supposed to
be reduced after smoothing. Moreover, SSIM which is the structural similarity evaluates the
similarity between the smoothed image and the image before smoothing to evaluate the
quality of smoothing.

Video smoothing:

The image smoothing techniques can be easily applied to video clips. The main reason
behind smoothing videos is to remove the unwanted bits that may cause noise for this
video; another reason is to hide transitions between frames as the video consists of
grouped images connected together. Creating a flow of information presented by this video,
and improving the quality of this video is another reason to use video smoothing. Alongside
with enhancing the video we can add extra elements like effects, graphics and music which
another great feature results from smoothing.
Video stream is composed by N video frames/images, if images move fast enough then our
brain cant comprehend them as separate images and the illusion of motion is created.
Video is a 3D, it is a recording of moving pictures varying with time, it has a horizontal
component, vertical component, and sound in addition to the time which is varying with
the horizontal and vertical components.

Figure 3: Video

DSP Project design
9

1.1 Aims
The aim of this project is to have the students solve practical problems on Digital Signal
Processing by using analytical thinking and MATLAB.

1.2 Objectives
Become familiar with a specific problem and application in digital signal processing.
Implement the project in MATLAB.


DSP Project design
10
2. Codes and Results

Task 1: Image acquisition

In this task, we downloaded images to work on. We had the choice of either
downloading black & white images which are 2D in nature (rows and columns only), or to
download a colored image which has a third dimension that is the colors.
The following MATLAB code shows the steps of downloading a colored image (3D) and
removing its third dimension (converting it into gray scale).

MATLAB code
% This code implements DSP Project # 6:
% Image Smoothing - Video processing
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Part 1: Image acquisition

clc; clear all; close all;
img = imread('Teletubbies','jpeg'); % read the test image
figure(1)
imshow(img)% display the test image
title('Origional colored image')
[row,col,K]=size(img);% obtain the dimensions of the image
gray=im2double(rgb2gray(img));% grayscale version of the img and converting
the image into type double
figure(2)
imshow(gray)% dispaly the test image
title('Origional gray image')

Figure 4: Original colored image and the Black & White image

Origional colored image
Origional gray image

DSP Project design
11
Comment:
The MATLAB has some built-in black and white images that are 2 dimensions
imread('x.tif') as required by this project. However, we preferred to work with an
image of our choice. The image we have chosen was a colored one, so in order to convert it
into a 2D image we used the following MATLAB command: rgb2gray(img), which
converts the RGB images into a grayscale intensity image. And to be able to perform the
filtering process on the image, it had to be of type double. To convert the date type of the
image into double the following command was used: im2double(rgb2gray(img)).

Task 2: Designing filter

In this task we were asked to design a 1D symmetric low-pass FIR filter of our choice,
and to apply it on the image in the horizontal direction, and then the vertical direction
separately at first. Then the designed filter shall be applied on the image for both directions
simultaneously. The goal was to investigate the effect of the filter on the image.
The filter should be represented by its impulse response, which is given by:

The filter that we have chosen was the symmetric moving average (SMA) filter, its
summation index is from N till N (hence called symmetric), and is a LPF but 2-sided. The
difference equation of it is shown as follow:

To get the impulse response, we have replaced the input signal in equation 2 by a delta. The
results of this is the same as in equation 1, where the coefficients of the filter k=
. The
Matlab code for designing the SMA, and displaying its frequency response, Z-plane, and
impulse response, is shown below:

MATLAB code
% Part 2: SMAFilter Design
N=2; % the order of the filter
b1=1./(2.*N+1);
1
2

DSP Project design
12

% The filter
m = -N:N;
w = -pi:(2*pi)/2^10:pi;
Hb = 0;

for i = 1:length(m)

Hb=Hb+b1.*exp(-m(i).*1i*w); % the exponential format instead of the
freqz, since the freqz command can no longer be used, as it cant
differentiate between symmetric and asymmetric system

end

figure(3)
subplot(3,1,1); plot(w./pi,abs(Hb),'linewidth',2);grid;hold on;
xlabel(' Normalized Frequence (Hz) '); ylabel('Magnitude in linear')
title('Frequency response of the SMA filter: Magnitude');
subplot(3,1,2); semilogy(w./pi,abs(Hb),'linewidth',2);grid;hold on;
xlabel(' Normalized Frequence (Hz) '); ylabel('Magnitude in dB')
title('Frequency response of the SMA filter: Magnitude');
subplot(3,1,3); plot(w./pi,angle(Hb),'linewidth',2);grid;
xlabel(' Normalized Frequence (Hz) ');ylabel('Phase');
title('Frequency response of the SMA filter: Phase');

% The coefficinets of the SMA filter are:
m1=-N:N;

for j=1:length(m1)

num(j)=1/(2*N+1);

end

b=num;
figure(4)
impz(num,1);grid
xlabel('Time')
ylabel('amplitude')
title(' Impulse response of the SMA filter')
ylim([-0.5 0.5])

% Plotting of Zplane of the SMA coeffecinets:
figure(5)
zplane(num,1);grid
title('Z plane of the SMA filter')


DSP Project design
13

Figure 5: Frequency & Phase spectrum of the filter

Figure 6: Impulse response & Z-plane of the filter

Comment:
Symmetric moving average filters are a good fit for our project purpose of image
smoothing, since in general moving average filters are considered to be a method for
smoothing time series. SMA filters moves over time, in that each data point of the series is
sequentially included in the averaging, while the oldest data point in the span of the
average is removed.
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
0
0.5
1
Normalized Frequence (Hz)
M
a
g
n
i
t
u
d
e

i
n

l
i
n
e
a
r
Frequency response of the SMA filter: Magnitude
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
10
-4
10
-2
10
0
M
a
g
n
i
t
u
d
e

i
n

d
B
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
-5
0
5
P
h
a
s
e
Frequency response of the SMA filter: Phase
0 0.5 1 1.5 2 2.5 3 3.5 4
-0.5
-0.4
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
0.4
0.5
Time
a
m
p
l
i
t
u
d
e
Impulse response of the SMA filter
-1 -0.5 0 0.5 1
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
4
Real Part
I
m
a
g
i
n
a
r
y

P
a
r
t
Z plane of the SMA filter

DSP Project design
14
The above figures show the frequency and phase spectrum of the SMA filter, its impulse
and Z-plane representation. It can be noted that the magnitude of the filter reflects a
behavior of a low pass filter. On the other hand, the phase response for this filter is zero,
since the impulse response for the filter is symmetrical around sample zero. And its Z-plane
consists of all zeros, which is expected; since it is an FIR filter. The impulse response is a
sequence of deltas which is requested by this task, and hence when applying it to the image
it will process the image pixel by pixel as will be seen in the following tasks.

Task 3: 1D Filter Horizontal Direction

In this task we shall implement the filter designed in the previous task on the input
images across the horizontal direction.

MATLAB code
% Part 3: 1D Filter Horizontal Direction

for i=1:1:row
horiz_test(i,:) =filter(b,1,gray(i,:));
end
figure(6)
imshow(horiz_test)
title('Smoothing : Horizontal direction')

Figure 7: 'Smoothing-Horizontal direction

Smoothing : Horizontal direction

DSP Project design
15
Comment:
In the MATLAB code we specified that the filtering process should be implemented in the
horizontal direction only by specifying that in the for-loop, which starts from 1 till the last
row of the input image.
Filtering techniques when implemented on images has several reasons, such as improving
or enhancing the image. But as shown in the above figure the filtering technique here
served the purpose of smoothing the input image. The above figure shows the input image
after filtering in the horizontal direction, and we observe that the image got smoother.

Task 4: 1D Filter Vertical Direction

In this task we shall implement the filter designed in the previous task on the input
images across the vertical direction.

MATLAB code
% Part 4: 1D Filter Vertical Direction

for j=1:1:col
ver_test(:,j) =filter(b,1,gray(:,j));
end
figure(7)
imshow(ver_test)
title('Smoothing : Vertical direction')

Figure 8: Smoothing-Vertical direction

Smoothing : Vertical direction

DSP Project design
16
Comment:
In the MATLAB code we specified that the filtering process should be implemented in the
vertical direction only by specifying that in the for-loop, which starts from 1 till the last
column of the input image.
The above figure shows the input image after filtering in the vertical direction, and we
observe that the image got smoother.

Task 5: Combined 1D filters (both horizontal and vertical direction)

In this task we shall use the filter designed in task 2, in a combination of the two 1D
filters in tasks 3 & 4. Meaning that we will apply the filter on the input image and the
resultant image will be smoothed in both horizontal and vertical directions. (Such filters
are called filters with separable kernels).
The corresponding impulse response of the filter should be as in equation 3:

There are different methods to implement the 2D filter. The following Matlab shows two
methods. The first is to use the Matlab command conv2, where we designed a new 2D filter
by convolving the previous designed filter with itself. In the second method, a nested loop
was used.

MATLAB code
% Part 5: Horizontal and Vertical Directions

% Method 1 (M1)
filter2d=conv2(b,b);
figure(8)
impz(conv2(b,b))
combine_test=conv2(filter2d,gray);
figure(9)
imshow(combine_test)
title('Smoothing : Horizontal and Vertical directions')

% Method 2
for i=1:1:row
combine_test2(i,:) =filter(b,1,grayN(i,:));
end
for j=1:1:col
combine_test2(:,j) =filter(b,1,grayN(:,j));
end
3

DSP Project design
17
figure(10)
imshow(combine_test2)

Figure 9: Impulse response of 2D filter

Figure 10: Filtering in Horizontal & Vertical directions by method 1 and method 2
Comment:
In order to filter the image in both directions, the MATLAB command conv2 was used,
which is a two dimensional convolution. In the convolution the output pixel is computed by
multiplying elements of two matrices and summing the results. The matrices are the input
image and the designed filter.
The impulse response of the filter was found to be a triangle, which is expected since the
1D filter was a rectangle and convolving two rectangles together gives a triangle (ramp).
The second method to implement convolution was to filter the image in the horizontal
direction (or vertical), then take the resultant filtered image and filter it in the other
direction. The filtered images using the two methods were fairly close in smoothing the
image to one another, hence both methods are valid.
0 1 2 3 4 5 6 7 8
0
0.05
0.1
0.15
0.2
0.25
n (samples)

A
m
p
l
i
t
u
d
e
Impulse Response
Smoothing : Horizontal and Vertical directions

DSP Project design
18

Task 6: Analysis and investigations

As for this task, we analyzed the work done so far by testing the filter on different input
images, changing the parameters of the filter such as the order of the filter, comparing the
filter we designed to the typical one used etc.

a) Test your smoothing filters on all test images.

We repeated the code for different images of different types (.tif, .png & .jpeg). The
MATLAB code remained the same except this line where the image is read in MATLAB:

P.S: Since the filter doesnt change only the figures of the image are shown.

1- Image of type .tif:

MATLAB code
% the part of the code that changes: Images of type .tif are already B&W
(2D) no need for conversion rgb2gray(img)

gray=im2double(imread('eight.tif'));
[row,col,K]=size(gray);
figure(1)
imshow(gray)% dispaly the test image

The obtained figures:

Figure 11: Original image.tif

DSP Project design
19

Figure 12: Smoothing the image in 1D

Figure 13: Smoothing the image in 2D (using 2 methods)

2- Image of type .png:

MATLAB code
% the part of the code that changes:
img = imread('peppers','png'); % read the test image


Figure 14: Original image.png (colored and B&W)
Smoothing : Horizontal direction Smoothing : Vertical direction
Origional colored image Origional gray image

DSP Project design
20


3- Image of type .jpeg:

MATLAB code
img = imread('RM','jpeg'); % read the test image


Figure 17: Original image.jpeg(colored and B&W)

DSP Project design
21


Comment:
We observed that the filter worked with all kinds of images. Whether they were already 2
dimensions or needed to be converted from 3D to 2D. The filters performance doesnt
change, it smoothed the images as it supposed to.

b) Consider all key parameters involved in the design of your filter; analyze the
impact of such parameters on the smoothing process, for example, the impact of
the length of the 1D filters.

1- Increasing the order of the filter

MATLAB code



DSP Project design
22

Figure 20: Original image.jpeg (colored and B&W)

Figure 21: Frequency & Phase spectrum

Figure 22: Impulse and Z-plane of the filter

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
0
1
2
M
a
g
n
i
t
u
d
e

i
n

l
i
n
e
a
r
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
10
-4
10
-2
10
0
M
a
g
n
i
t
u
d
e

i
n

d
B
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
-5
0
5
P
h
a
s
e
0 1 2 3 4 5 6 7 8 9 10
-0.5
-0.4
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
0.4
0.5
Time
a
m
p
l
i
t
u
d
e
-1 -0.5 0 0.5 1
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
10
Real Part
I
m
a
g
i n
a
r
y

P
a
r
t

DSP Project design
23



Figure 242: Impulse response of 2D filter for N=5

2- Decreasing the order of the filter:

MATLAB code


0 2 4 6 8 10 12 14 16 18 20
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0.1
n (samples)

A
m
p
l
i
t
u
d
e
Impulse Response

DSP Project design
24


Figure 26: Frequency & Phase spectrum

Figure 27: Impulse and Z-plane of the filter

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
0
0.5
1
M
a
g
n
i
t
u
d
e

i
n

l
i
n
e
a
r
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
10
-4
10
-2
10
0
M
a
g
n
i
t
u
d
e

i
n

d
B
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
0
2
4
P
h
a
s
e
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
-0.5
-0.4
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
0.4
0.5
Time
a
m
p
l i t
u
d
e
-1 -0.5 0 0.5 1
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
2
Real Part
I
m
a
g
i n
a
r
y

P
a
r
t

DSP Project design
25


Figure 30: Impulse response of 2D filter
Comment:
From the above figures we observed that by increasing the order of the filter the resolution
of the filtered image decreases significantly. But if we choose a filter with a very low filter
order then the filter will output an image very close to the original input image without
fulfilling its purpose of smoothing the image.

c) Find in the literature typical image smoothing filters and compare them to the
filters you have designed. Draw conclusions regarding frequency characteristics
of those typical filters.

Typical image smoothing filters:

Filter Description

Moving average
filter (Designed
filter)

Is considered to be the simplest form of smoothing, it simply
replaces each data value with the average of neighboring values.
To avoid shifting the data, it is best to average the same number of
values before and after where the average is being calculated. In
equation form, the moving average is calculated by:
0 0.5 1 1.5 2 2.5 3 3.5 4
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
n (samples)

A
m
p
l
i
t
u
d
e
Impulse Response

DSP Project design
26

It is implemented by by convolving the input data with a box-
shaped pulse of 2*M+1 values all equal to 1/(2*M+1)

Binomial
Smoothing

It is a Gaussian filter. It convolves the data with normalized
coefficients derived from Pascals triangle at a level equal to the
Smoothing parameter.

Savitzky-Golay
Smoothing

It uses a different set of pre-computed coefficients popular in the
field of chemistry. It is a type of Least Squares Polynomial
smoothing. The amount of smoothing is controlled by two
parameters: the polynomial order and the number of points used
to compute each smoothed output value.

Frequency characteristics of the filters:

MATLAB code
%% This code shows the frequency response of typical Smoothing Filters
clc; clear all
% Moving average Filter
filter2d=conv2(b,b);
figure(1)
freqz2(filter2d)
combine_test=conv2(filter2d,gray);
figure(2)
imshow(combine_test)
title('Smoothing 2D: Moving average filter')

% Binomial Filter
binomial=pascal(5);
[x,y,z]=size(binomial)
figure(3)
freqz2(binomial)
combine_test=conv2(binomial,gray);
figure(4)
imshow(combine_test,[])
title('Smoothing 2D: Binomial filter')


DSP Project design
27
% Savitzky-Golay Filter
xn=2;x=-xn:xn;n=2;dn=0;Savitzky_Golay=[];
for x0=-xn:xn
h=Golay(x,n,dn,x0,1);
Savitzky_Golay=[Savitzky_Golay h'];
end
[x,y,z]=size(Savitzky_Golay)
figure(5)
freqz2(double(Savitzky_Golay))
combine_test=conv2(double(Savitzky_Golay),gray);
figure(6)
imshow(combine_test,[])
title('Smoothing 2D: Savitzky-Golay filter')
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% for Savitzky-Golay Filter the following function is called
function fc=Golay(x,n,dn,x0,flag)
if nargin<5
flag=0;
end
if nargin<4
x0=0;
end
if nargin<3
dn=0;
end
if nargin<2
n=1;
end
if(n>length(x)-1)
error('Polynomial Order too Large');
end
if(dn>n)
error('Differentiation Order too Large!');
end
if flag
x=sym(x);
x0=sym(x0);
end
x=x(:);
A=ones(length(x),1);
for k=1:n
A=[A x.^k];
end
h=inv(A'*A)*A';
if flag
hx=sym([(zeros(1,dn)) prod(1:dn)]); %order=0:dn-1,& dn,respectively
for k=1:n-dn %order=dn+1:n=dn+k
hx=[hx sym(x0^k*prod(dn+k:-1:k+1))];
end
else
hx=[(zeros(1,dn)) prod(1:dn)]; %order=0:dn-1,& dn,respectively
for k=1:n-dn %order=dn+1:n=dn+k
hx=[hx x0^k*prod(dn+k:-1:k+1)];
end
end

DSP Project design
28
hx=repmat(hx',[1,size(h,2)]);
h=h.*hx;
fc=sum(h);

Figure 31: Moving average filter

Figure 32: Binomial filter

Figure 33: Savitzky-Golay filter
-1
-0.5
0
0.5
1
-1
-0.5
0
0.5
1
0
0.2
0.4
0.6
0.8
1
F
x
F
y
M
a
g
n
i
t
u
d
e
Smoothing 2D: Moving average filter
-1
-0.5
0
0.5
1
-1
-0.5
0
0.5
1
0
1
2
3
4
5
F
x
F
y
M
a
g
n
i
t
u
d
e
Smoothing 2D: Binomial filter
-1
-0.5
0
0.5
1
-1
-0.5
0
0.5
1
0
1
2
3
4
5
F
x
F
y
M
a
g
n
i
t
u
d
e
Smoothing 2D: Savitzky-Golay filter

DSP Project design
29

d) In order to study the effectiveness of the smoothing process, some metrics of
evaluation is often needed. What metrics exists for such an evaluation?
Implement at least one metric and evaluate your results.

There are different evaluating metrics that can be used like PSNR (Peak Signal to Noise
Ratio), MSE (Mean Square Error), Normalized Cross-Correlation and SSIM (Structural
SIMilarity). We shall see the effect of all these evaluating metrics.

PSNR (Peak Signal to Noise Ratio), MSE (Mean Square Error)

We have found the PSNR and MSE values for different scenarios of the original input image.
These include:

Noisy input image
B&W noiseless image with different filter orders

1- PSNR and MSE for noisy input image

In the following MATLAB code, we added Gaussian noise to the input image, and then
passed it through the filter and observed the output images. Then we calculated the PSNR
and the MSE for the filtered image in comparison:

MATLAB code
img = imread('Teletubbies','jpeg'); % read the test image

figure(1)
imshow(img)% display the test image
title('Origional colored image')
[row,col,K]=size(img);% obtain the dimensions of the image
gray=im2double(rgb2gray(img));% grayscale version of the img and converting
the image into type double

figure(2)
imshow(gray)% display the test image
grayN=imnoise(gray , 'gaussian', 0,0.02 );% adding noise

figure(11)
imshow(grayN)% display the test image
title('Noisy gray image')

%the rest of the MATLAB code remains the same


DSP Project design
30


Figure 35: Noisy image


Noisy gray image
Smoothing : Horizontal direction
Smoothing : Vertical direction

DSP Project design
31


Then, to find the PSNR and MSE between the noisy image and the smoothed image, the
following MATLAB code was used:

MATLAB code
% grayN - Original noisy image
% combine_test - filtered image
% MSE Mean Square Error
% PSNR Peak Signal to Noise Ratio

N = size(grayN);
acc = 0;

for k1=1:N(1)
for k2=1:N(2)

acc = acc+ ( grayN(k1,k2) - combine_test(k1,k2) )^2;
end
end

MSE = acc/(N(1)*N(2))
PSNR = 10*log10((255^2)/MSE)

MATLAB command window
MSE =
0.0503
PSNR =
61.1119

DSP Project design
32

2- PSNR and MSE for black and white image with different filter orders

We have removed the noise from the original signal and then smoothed the image. The
PSNR and MSE were found between them. However, we have tried this for different filter
orders; 1
st
, 2
nd
, and 3
rd
orders. The same MATLAB code is used as above except for the
input image.

(1
st
order filter)

MSE =
0.0120
PSNR =
67.3412

(2
nd
order filter)

MSE =
0.0195
PSNR =
65.2196

(3
rd
order filter)

MSE =
0.0273
PSNR =
63.7682


DSP Project design
33
Comment:
The metrics investigated above are the PSNR and MSE. The PSNR is often used as a quality
measurement between the original and a compressed image. The higher the PSNR, the
better is the quality of the compressed or reconstructed image. And the MSE used to
compare image compression quality. The MSE represents the cumulative squared error
between the compressed and the original image, whereas PSNR represents a measure of
the peak error. The lower the value of MSE, the lower is the error. They can be represented
in the following equations:

Where, M and N are the number of rows and columns in the input images, respectively.

R is the maximum fluctuation in the input image data type. For example, if the input image
has a double-precision floating-point data type, then R is 1. If it has an 8-bit unsigned
integer data type, R is 255, etc. R = 255 was used for calculating PSNR.

From the value of MSE and PSNR we observed that for noiseless images the PSNR was
higher which is better as expected, and the MSE was less. And as we increase the order of
the filter the PSNR drops and the MSE increases! This was also expected since as the order
of the filter increases the resolution of the filtered image decreases due to the smoothing
process, because of that the PSNR gets reduced and the MSE increases.

Normalized cross correlation

In the following MATLAB code, the normalized cross correlation between the original
image and the smoothed image is tested.

4
5

DSP Project design
34
MATLAB code
% First: Normalized Cross-Correlation
% We want to check the cross corelation between the two images
% To do so, we croped the images filter as folows:
% Cropping Process:
croped_gray = [111 60 65 88]; % The size of the croped image
croped_combine_test= [111 60 65 88];
sub_gray = imcrop(gray,croped_gray);
sub_combine_test = imcrop(combine_test,croped_combine_test);

% display sub images
figure(12), imshow(sub_gray)
figure(13), imshow(sub_combine_test)
% To find the correlation:
c = normxcorr2(sub_gray(:,:,1),sub_combine_test(:,:,1));
figure(14), surf(c), shading flat

Figure 38: Cropped image before and after filtering

Figure 39: Correlation

0
50
100
150
0
50
100
150
200
-1
-0.5
0
0.5
1

DSP Project design
35
Comment:
From the above figure we find that the correlation is high between the two images, and the
differences come from the smoothing process.

SSIM (Structural SIMilarity)

In the following MATLAB code, the structural similarity between the original image and the
smoothed image is tested.

MATLAB code
%structural similarity
window = fspecial('gaussian', 11, 1.5); %
K(1) = 0.01; % default settings
K(2) = 0.03; %
L = 255;
[M N] = size(gray);

gray = double(gray);
combine_test2 = double(combine_test2);

% automatic downsampling
f = max(1,round(min(M,N)/256));
%downsampling by f
%use a simple low-pass filter
if(f>1)
lpf = ones(f,f);
lpf = lpf/sum(lpf(:));
gray = imfilter(gray,lpf,'symmetric','same');
combine_test2 = imfilter(combine_test2,lpf,'symmetric','same');

gray = gray(1:f:end,1:f:end);
combine_test2 = combine_test2(1:f:end,1:f:end);
end

C1 = (K(1)*L)^2;
C2 = (K(2)*L)^2;
window = window/sum(sum(window));

mu1 = filter2(window, gray, 'valid');
mu2 = filter2(window, combine_test2, 'valid');
mu1_sq = mu1.*mu1;
mu2_sq = mu2.*mu2;
mu1_mu2 = mu1.*mu2;
sigma1_sq = filter2(window, gray.*gray, 'valid') - mu1_sq;
sigma2_sq = filter2(window, combine_test2.*combine_test2, 'valid') - mu2_sq;
sigma12 = filter2(window, gray.*combine_test2, 'valid') - mu1_mu2;

if (C1 > 0 && C2 > 0)
ssim_map = ((2*mu1_mu2 + C1).*(2*sigma12 + C2))./((mu1_sq + mu2_sq +

DSP Project design
36
C1).*(sigma1_sq + sigma2_sq + C2));
else
numerator = (2*mu1_mu2 + C1).*(2*sigma12 + C2);
denominator = (mu1_sq + mu2_sq + C1).*(sigma1_sq + sigma2_sq + C2);
ssim_map = ones(size(mu1));
index = (denominator > 0);
ssim_map(index) = (numerator(index))./(denominator(index));
index = (denominator ~= 0);
ssim_map(index) = numerator(index)./denominator(index);
end

mssim = mean2(ssim_map) %Gives the mssim value


mssim =

0.9994

Comment:

The structural similarity was implemented used the following equation:

Where C1 and C2 are constants, is the mean intensity, is the standard deviation.

In the above MATLAB code we measured the mean SSIM index value between 2 images, the
original image before smoothing and the output image after smoothing. If the two
compared images are exactly the same the mssim = 1, but if there are some differences then
the value of mssim will be less than 1. We found that the mssim = 0.9994 < 1, which is
expected since the smoothed output image is not identical to the output image its a little
bit blurry.

Task 7: Video processing

In this final task we are asked to extend the image processing to the time domain, which
means we shall apply it to video clips. We are required to design an additional filter for the
time domain. The filters impulse response is illustrated below:

6

DSP Project design
37

MATLAB code
% Part 6: Video processing

vid=VideoReader('Teletubbies.avi');
vidFrames = im2double(read(vid)); % Removing the audio part of the video. 3D
signal

[xx yy zz kk]=size(vidFrames); % where xx= length, yy=width, zz= RBG and kk=
frame number

for i=1:kk

Frame(:,:,:)=vidFrames(:,:,:,i); % Take each frame seperate
X=rgb2gray(Frame); % Convert RGB to black and white to be able to filter
it with our filter
Y=conv2(X,filter2d); % Convolution ( Smoothing process)

figure(15)
subplot(1,2,1); imshow(Frame); % Display the origional video
title('Origional video')
subplot(1,2,2);
imshow(Y); % Display the smoothed video
title('Smoothing of the video')

end

Figure 40: Video processing

Comment:
MATLAB can solve many common video processing challenges, from importing video data
from files to analyze and process data with standard algorithms. To read video in MATLAB
we used the VideoReader command, this function supports formats such as .mpg & .avi
which we used. Using the VideoReader we created an object that access the file and then
used the Read method to bring the data into MATLAB. To remove the audio from the video,
we used software called "format factory". But the video was still colored, so we accessed
each frame individually, and converted it to a gray-scale image. This was done as in the
Origional video
Smoothing of the video

DSP Project design
38
previous tasks, using the command rgb2gray. After that, we filtered each frame using the 2-
D filter designed in the previous task. The filter coefficients were of the type double, while
the image data are of the type uni8. So to convert from uni8 to double, we used the
MATLAB command im2double. Finally, we showed each frame inside the loop in the same
sequence they appeared in the original video, this makes it look like what we expect.


DSP Project design
39
3. Conclusion
At the end of this project, we have learnt an important application of digital signal
processing which is image smoothing. In this project we firstly downloaded the input
images. Then we designed a 1D low pass filter for horizontal direction. After that we did the
same but now for the vertical direction. Moreover, we combined the two filters which
resulted in a 2D filter. We applied these filters on the test images and noticed the effect of
changing the design parameters of these filters. Furthermore, we compared between the
moving average filter, binomial and golay filter and noticed their performances in
smoothing images. Finally, by adding a time component to the images, we processed a
video and noted that the time domain filter should be causal.
All in all, we have learnt how to solve practical problems on digital signal processing by
using analytical thinking and MATLAB. However, we are now more familiar with
applications in digital signal processing. And we learnt how to implement image smoothing
project in MATLAB.
By this we achieve all the objectives of this project and we are now prepared for other
applications of digital signal processing.

DSP Project

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

DSP Project

Uploaded by

Copyright:

Available Formats

Khalifa University of Science, Technology and Research

Communication Engineering Department

You might also like