It 2012

Practical Workbook INFORMATION THEORY
Name Year Batch Roll No
: ________________________ : ________________________ : ________________________ : ________________________
Department: ________________________
4th edition: 2012
Department of Computer & Information Systems Engineering NED University of Engineering & Technology, Karachi 75270, Pakistan
INTRODUCTION
The laboratory sessions in the practical workbook of Information Theory cover the concepts, tools, techniques and activities related to the error detection and correction and fundamental communication process of information transfer. The workbook starts with an overview of MATLAB followed by its application to coding theory.
The next couple of laboratory sessions cover the different types of error detecting and correcting schemes, their encoding and decoding process and the bit error rate. The concept of source encoding is then introduced with emphasis on Huffman Encoding. It is then followed by the various concept of cryptography.
Next five lab sessions once again deals with encoding and decoding process but on a different platform, their implementation is based on models or blocks.
All the concepts described throughout the workbook will be verified through implementation in MATLAB and Simulink.
CONTENTS
Lab Session No.
Object
Page No.
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Application of MATLAB to coding theory Learning basic commands of MATLAB
1 8
Encoding messages for a forward error correction system with a given Linear 15 block code and verifying through simulation Decoding encoded words for a forward error correction system with a given Linear 20 block code and verifying through simulation Encoding the messages for a system with a given cyclic polynomial code and 30 verifying through simulation Decoding the messages for a system with a given cyclic polynomial code and 41 verifying through simulation Understanding the concept of loss less data compression technique using Huffman 49 coding Implementation of Hill cipher technique in cryptography 53 Implementation of Vigenere cipher technique in cryptography Learning basic model designs with Simulink Encoding the data bits using a Binary Cyclic block encoder in Simulink Decoding the code words using a Binary Cyclic block decoder in Simulink Encoding the data bits using a Binary Linear block encoder in Simulink Decoding the code words using Binary Linear block decoder in Simulink Appendix A: Introduction to Simulink 60 63 67 72 75 79 82
Information Theory
Lab Session 01
NED University of Engineering & Technology Department of Computer & Information Systems Engineering
Lab Session 01
OBJECT
Application of MATLAB to coding theory
THEORY
INTRODUCTION: MATLAB is a high-performance language for technical computing. It integrates computation, visualization, and programming in an easy-to-use environment where problems and solutions are expressed in familiar mathematical notation. Starting MATLAB On Windows platforms, start MATLAB by double-clicking the MATLAB shortcut icon on your Windows desktop. Quitting MATLAB To end your MATLAB session, select File -> Exit MATLAB in the desktop, or type quit in the Command Window. THE MATLAB ENVIRONMENT The MATLAB environment consists of five main parts: 1) Development Environment This is the set of tools and facilities that help you use MATLAB functions and files. Many of these tools are graphical user interfaces. It includes the MATLAB desktop and Command Window, a command history, an editor and debugger, and browsers for viewing help, the workspace, files, and the search path. SUMMARY OF DESKTOP TOOLS Array Editor: View array contents in a table format and edit the values. Command Window: Run MATLAB functions. Command History: View a log of the functions you entered in the Command Window, copy them, execute them, and more. Current Directory Browser: View files, perform file operations such as open, find files and file content, and manage and tune your files. Help Browser: View and search the documentation for all your Math Works products. Start Button: Run tools and access documentation for all of your Math Works products, and create and use MATLAB shortcuts. Workspace Browser: View and make changes to the contents of the workspace.
Information Theory
Lab Session 01
GETTING STARTED
This session provides a brief overview of essential MATLAB commands. You will learn this material more quickly if you use MATLAB interactively as you are reviewing this manual . The MATLAB commands will be shown in the following font style: Monaco font the prompt for a user input is shown by the double arrow Type help pi at the prompt help pi PI PI = 4*atan(1) = 3.1415926535897.... so we see that MATLAB has the number "built-in". As another example help exp EXP EXP(X) is the exponential of the elements of X, e to the X. Sometimes you do not know the exact command to perform a particular operation. In this case, one can simply type help and MATLAB will provide a list of commands (and m-files, to be discussed later) that are available. If you do not know the exact command for the function that you are after, another useful command is lookfor. This command works somewhat like an index. If you did not know the command for the exponential function was exp, you could type lookfor exponential EXP Exponential. EXPM Matrix exponential. EXPM1 Matrix exponential via Pade' approximation. EXPM2 Matrix exponential via Taylor series approximation. EXPM3 Matrix exponential via eigenvalues and eigenvectors. EXPME Used by LINSIM to calculate matrix exponentials. a) Entering Matrices The best way for you to get started with MATLAB is to learn how to handle matrices. Start MATLAB and follow along with each example. You can enter matrices into MATLAB by entering an explicit list of elements or generating matrices using built-in functions. You only have to follow a few basic conventions: Separate the elements of a row with blanks or commas. Use a semicolon ; to indicate the end of each row. Surround the entire list of elements with square brackets [ ]. Consider the following vector, x (recall that a vector is simply a matrix with only one row or column) x = [1,3,5,7,9,11] x = 1 3 5 7 9 11 Notice that a row vector is the default. We could have used spaces as the delimiter between columns x = [1 3 5 7 9 11]
Information Theory
Lab Session 01
x = 1 3 5 7 9 11 There is a faster way to enter matrices or vectors that have a linear pattern. For example, the following command creates the previous vector x = 1:2:11 (here what does 2 indicate? Will be discussed in proceeding lab sessions) x = 1 3 5 7 9 11 Transposing a row vector yields a column vector ( 'is the transpose command in MATLAB) y = x' y =1 3 5 7 9 11 Say that we want to create a vector z, which has elements from 5 to 30, by 5's z = 5:5:30 z = 5 10 15 20 25 30 If we wish to suppress printing, we can add a semicolon (;) after any MATLAB command z = 5:5:30; The z vector is generated, but not printed in the command window. We can find the value of the third element in the z vector, z(3), by typing z(3) ans = 15 (Notice that a new variable, ans, was defined automatically.) b) The MATLAB Workspace We can view the variables currently in the workspace by typing who Your variables are: ans x y z leaving 621420 bytes of memory free. More detail about the size of the matrices can be obtained by typing whos Name ans x y z Size 1 by 1 by 6 by 1 by 1 6 1 6 Total 1 6 6 6 Complex No No No No
Grand total is (19 * 8) = 152 bytes, leaving 622256 bytes of memory free. We can also find the size of a matrix or vector by typing [m,n]=size(x) m =1 n =6
Information Theory
Lab Session 01
Where m represents the number of rows and n represents the number of columns. If we do not put place arguments for the rows and columns, we find size(x) ans = 1 6 Since x is a vector, we can also use the length command length(x) ans = 6 It should be noted that MATLAB is case sensitive with respect to variable names. An X matrix can coexist with an x matrix. MATLAB is not case sensitive with respect to "built-in" MATLAB functions. For example, the length command can be upper or lower case LENGTH(x) ans = 6 Notice that we have not named an upper case X variable. See what happens when we try to find the length of X LENGTH(X) ??? Undefined function or variable. Symbol in question ==> X Sometimes it is desirable to clear all of the variables in a workspace. This is done by simply typing clear more frequently you may wish to clear a particular variable, such as x clear x You may wish to quit MATLAB but save your variables so you don't have to retype or recalculate them during your next MATLAB session. To save all of your variables, use save file_name (Saving your variables does not remove them from your workspace; only clear can do that) You can also save just a few of your variables save file_name x y z To load a set of previously saved variables load file_name c) Complex variables Both i and j represent the imaginary number, -1, by default i ans = 0 + 1.0000i j ans = 0 + 1.0000i sqrt(-3) ans = 0 + 1.7321i
Information Theory
Lab Session 01
Note that these variables (i and j) can be redefined (as the index in a for loop, for example), not included in your course. Matrices can be created where some of the elements are complex and the others are real a = [sqrt(4), 1;sqrt(-4), -5] a = 2.0000 1.0000 0 + 2.0000i -5.0000 Recall that the semicolon designates the end of a row.
d) The Colon Operator The colon : is one of the most important MATLAB operators. It occurs in several different forms. The expression 1:10 is a row vector containing the integers from 1 to 10, 1 2 3 4 5 6 To obtain non unit spacing, specify an increment. For example, 100:-7:50 is 100 93 and 0:pi/4:pi is 0 0.7854 7 8 9 10
86
79
72
65
58
51
1.5708
2.3562
3.1416
Subscript expressions involving colons refer to portions of a matrix. A(1:k,j) is the first k elements of the jth column of A. So sum(A(1:4,4)) computes the sum of the fourth column. But there is a better way. The colon by itself refers to all the elements in a row or column of a matrix and the keyword end refers to the last row or column. So sum(A(:,end)) computes the sum of the elements in the last column of A. ans = e) Operators Expressions use familiar arithmetic operators and precedence rules. + * / ^ ( ) Addition Subtraction Multiplication Division Power Complex conjugate transpose Specify evaluation order 34
Information Theory
Lab Session 01
Summary of Commonly Used Commands Command clear clc diary exp format function gtext help hold length lookfor plot size subplot who whos * ' ; .* ./ : purpose removes all variables from workspace clears command window save the text of a MATLAB session exponential function output display format user generated function place text on a plot help holds current plot and allows new plot to be placed on current plot length of a vector keyword search on help variables plots vectors size of the array multiple plots in a figure window view variables in workspace View variables in workspace, with more detail (size, etc.) matrix multiplication Transpose suppress printing (also - end of row, when used in matrices) element by element multiplication element by element division denotes a column in a matrix or creates a vector
Exercises:
Write the shortest MATLAB code for each problem, avoiding unnecessary details if at all possible, along with the final displayed output.
1.1)
Create a session diary myfirstdiary that records the executions of all script files.
Declare predefined variables (, i, j) along with variable X . Where X= [1 2 3; 4 5 6]. Now answer the following questions, also show the output wherever necessary:
1.1.1) What is the difference between clear and clear X command?
_____________________________________________________________________ _____________________________________________________________________
1.1.2) What is the difference between CLC and clear command?
_____________________________________________________________________ _____________________________________________________________________ 6
Information Theory
Lab Session 01
1.1.3) What is the difference between displaying the value of before and after
using the command format long? _____________________________________________________________________ _____________________________________________________________________

1.1.4) What is the purpose of diary command?
_____________________________________________________________________ _____________________________________________________________________
1.2)
Let x=20; y=25 Calculate : 1.2.1) cos(x) _____________________________________________________________________ 1.2.2) sqrt(y) _____________________________________________________________________ 1.2.3) tan(x+y) _____________________________________________________________________ 1.2.4) sum the outputs of part a, b and c _____________________________________________________________________
1.3)
A fruit merchant has 10 apples, 15 bananas and 5 peaches. Each entity weighs 50gms. 1.3.1) Calculate the total quantity of fruits available _____________________________________________________________________ _____________________________________________________________________ _____________________________________________________________________ 1.3.2) Calculate the weight of fruit (based on individual category) the merchant has _____________________________________________________________________ _____________________________________________________________________ _____________________________________________________________________ 1.3.3) Calculate the total weight of fruits the merchant has _____________________________________________________________________ _____________________________________________________________________ _____________________________________________________________________
Information Theory
Lab Session 02
Lab Session 02
OBJECT
Learning basic commands of MATLAB
THEORY
INTRODUCTION:
MATLAB is a high-performance language for technical computing. It integrates computation, visualization, and programming in an easy-to-use environment where problems and solutions are expressed in familiar mathematical notation. The purpose of this lab session is to review MATLAB for those that have used it before, and to provide a brief introduction to MATLAB for those that have not used it before. This is a "hands-on" session of introduction. After going through this session, you should be able to: Enter matrices Perform matrix operations Use MATLAB functions Write simple m-files
a) Entering Matrices The best way for you to get started with MATLAB is to learn how to handle matrices. Start MATLAB and follow along with each example. You can enter matrices into MATLAB by entering an explicit list of elements or generating matrices using built-in functions. You only have to follow a few basic conventions: Separate the elements of a row with blanks or commas. Use a semicolon ; to indicate the end of each row. Surround the entire list of elements with square brackets [ ]. To enter a matrix A, simply type in the Command Window A = [16 3 2 13; 5 10 11 8; 9 6 7 12; 4 15 14 1] MATLAB displays the matrix you just entered. A = 16 3 2 13 5 10 11 8 9 6 7 12 4 15 14 1 Once you have entered the matrix, it is automatically remembered in the MATLAB workspace. You can refer to it simply as A. b) Sum, Transpose, and diagonal The first statement to try is sum(A) MATLAB replies with ans = 34 34 34 34 8
Information Theory
Lab Session 02
When you do not specify an output variable, MATLAB uses the variable ans, short for answer, to store the results of a calculation. You have computed a row vector containing the sums of the columns of A. How about the row sums? MATLAB has a preference for working with the columns of a matrix, so the easiest way to get the row sums is to transpose the matrix, compute the column sums of the transpose, and then transpose the result. The transpose operation is denoted by an apostrophe or single quote . It flips a matrix about its main diagonal and it turns a row vector into a column vector. So A' produces ans = 16 5 9 4 3 10 6 15 2 11 7 14 13 8 12 1 And sum(A')' produces a column vector containing the row sums ans = 34 34 34 34 The sum of the elements on the main diagonal is obtained with the sum and the diag functions. diag(A) produces ans = 16 10 7 1 and sum(diag(A)) produces ans = 34 c) Subscripts The element in row i and column j of A is denoted by A(i,j). For example, A(4,2) is the number in the fourth row and second column. For our square matrix A, A(4,2) is 15. So to compute the sum of the elements in the fourth column of A, type A(1,4) + A(2,4) + A(3,4) + A(4,4) This produces ans = 34
Information Theory
Lab Session 02
d) Generating Matrices: MATLAB provides functions that generate basic matrices three of which are given below. Zeros Ones Rand Here are some examples. Z = zeros(2,4) Z = 0 0 0 0 F = 5*ones(3,3) F = 5 5 5 5 5 5 All zeros All ones Uniformly distributed random elements
0 0 5 5 5
0 0
N = 10*rand(1,10)) N = 9 2 6 4 8 7 4 0 8 4
e) The Identity Matrix Generally accepted mathematical notation uses the capital letter to denote identity matrices, matrices of various sizes with ones on the main diagonal and zeros elsewhere. These matrices have the property that and whenever the dimensions are compatible. The original version of MATLAB could not use for this purpose because it did not distinguish between upper and lowercase letters and already served double duty as a subscript and as the complex unit. So an English language pun was introduced. The function eye(m,n) returns an m-by-n rectangular identity matrix and eye(n) returns an n-by-n square identity matrix. f) M-Files You can create your own matrices using M-files, which are text files containing MATLAB code. Use the MATLAB Editor or another text editor to create a file containing the same statements you would type at the MATLAB command line. Save the file under a name that ends in .m. For example, create a file containing these five lines. A = [16.0 5.0 9.0 4.0 3.0 10.0 6.0 15.0 2.0 11.0 7.0 14.0 13.0 8.0 12.0 1.0
];
Store the file under the name first.m. Then the statement first reads the file and creates a variable, A, containing our example matrix. 10
Information Theory
Lab Session 02
g) Concatenation Concatenation is the process of joining small matrices to make bigger ones. In fact, you made your first matrix by concatenating its individual elements. The pair of square brackets, [], is the concatenation operator. For an example, start with the 4-by-4 matrix A, and form B = [A A+32; A+48 A+16]
The result is an 8-by-8 matrix, obtained by joining the four submatrices. B = 16 3 2 13 48 35 34 45 5 10 11 8 37 42 43 40 9 6 7 12 41 38 39 44 4 15 14 1 36 47 46 33 64 51 50 61 32 19 18 29 53 58 59 56 21 26 27 24 57 54 55 60 25 22 23 28 52 63 62 49 20 31 30 17 h) Deleting Rows and Columns You can delete rows and columns from a matrix using just a pair of square brackets. Start with X = A; Then, to delete the second column of X, use X(:,2) = [] This changes X to X = 16 5 9 4 2 11 7 14 13 8 12 1
If you delete a single element from a matrix, the result is not a matrix anymore. So, expressions like X(1,2) = [] will result in an error. However, using a single subscript deletes a single element, or sequence of elements, and reshapes the remaining elements into a row vector. So X(2:2:10) = [] results in X = 16 9 2 7 13 12 1
i) Scalar Expansion Matrices and scalars can be combined in several different ways. For example, a scalar is subtracted from a matrix by subtracting it from each element. The average value of the elements in matrix A is 8.5, so B = A - 8.5 forms a matrix whose column sums are zero. 11
Information Theory
Lab Session 02
B =
7.5 -3.5 0.5 -4.5
-5.5 1.5 -2.5 6.5 0 0
-6.5 2.5 -1.5 5.5 0
4.5 -0.5 3.5 -7.5
sum(B) ans = 0
With scalar expansion, MATLAB assigns a specified scalar to all indices in a range. For example, B(1:2,2:3) = 0 zeroes out a portion of B. B = 7.5 0 -3.5 0 0.5 -2.5 -4.5 6.5 0 0 -1.5 5.5 4.5 -0.5 3.5 -7.5
j) Adding and Subtracting Matrices Addition and subtraction of matrices is defined just as it is for arrays, element-by-element. Adding A to B and then subtracting A from the result recovers B. Taking the same matrix A and defining another matrix B B = [1 3 2 11; 5 9 10 6; 10 6 19 12; 14 5 1 10] X = A + B X = 17 6 10 19 19 12 18 20 Y = X - A Y = 1 5 10 14 3 9 6 5 4 21 26 15 2 10 19 1 24 14 24 11 11 6 12 10
Addition and subtraction require both matrices to have the same dimension, or one of them should be a scalar. If the dimensions are incompatible, an error will result. C = [2 0 -1]; X = A + C will generate the following error. Error using ==> + Matrix dimensions must agree. w = v + s w = 9 7 6
12
Information Theory
Lab Session 02
k) Vector Products A row vector and a column vector of the same length can be multiplied in either order. The result is either a scalar, the inner product, or a matrix, the outer product. u = [3; 1; 4]; v = [2 0 -1]; x = v*u x = 2
X = u*v X = 6 2 8 0 0 0 -3 -1 -4
l)
Building Tables
Array operations are useful for building tables. Suppose n is the column vector n = (0:9)'; Then pows = [n n.^2 2.^n] builds a table of squares and powers of 2. pows = 0 1 2 3 4 5 6 7 8 9 0 1 4 9 16 25 36 49 64 81 1 2 4 8 16 32 64 128 256 512
The elementary math functions operate on arrays element by element. x = (1:0.1:2)'; logs = [x log10(x)] builds a table of logarithms. logs =1.0 0 1.1 0.04139 1.2 0.07918 1.3 0.11394 1.4 0.14613 1.5 0.17609 1.6 0.20412 1.7 0.23045 1.8 0.25527 1.9 0.27875 2.0 0.30103 13
Information Theory
Lab Session 02
Exercise:
2.1) Calculate the rank, determinant and matrix inverse of the following matrices (use help rank, help det and help inv)
A=
C=
___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 2.1.1) Find CC ; use same value of C as given in Q1 ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 2.2) Calculate 2.2.1) xTx, and 2.2.2) xxT, where x= ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 14
-1
Information Theory
Lab Session 03
Lab Session 03
OBJECT
Encoding messages for a forward error correction system with a given Linear block code and verifying through simulation
THEORY
In mathematics and information theory, linear code is an important type of block code used in error correction and detection schemes. Linear codes allow for more efficient encoding and decoding algorithms than other codes. Linear codes are applied in methods of transmitting symbols (e.g., bits) on communications channel so that, if errors occur in the communication, some errors can be detected by the recipient of the message block. A linear code of length n transmits blocks containing n symbols. For example, the (7, 4) Hamming code is a binary linear code which represents 4-bit values with 7 bits. In this way, the recipient can detect errors as severe as 2 bits per block. In linear block codes, a block of k information bits is followed by a group of r check bits derived from the information bits and at the receiver the check bits are used to verify the information bits which are preceding the check bits. No of code words = 2k Block length = n Code rate = k/n Each block of k bits is encoded into block of n bits (n>k) by adding n-k = r check bits. Where 2r k + r + 1. -------------------------(A)
The check bits are determined by some predetermined rule. C=DG Where: C = code vector D = Data (message) vector G = Generator matrix which is defined as [ Ik | P]; Ik is identity matrix of order k and P is the predefined encoding rule. MATLAB Syntax: The encode function is used for encoding. The syntax is as follows. code = encode(msg,n,k,'linear/fmt',genmat) where, the codeword length is n and the message length is k. msg represents data or message. It can be in decimal or binary format. The default value for this parameter is binary. We will use binary format in this and foregoing lab sessions.
15
Information Theory
Lab Session 03
For example: Format of msg can be a binary column vector as given below. msg = [0 1 1 0 0 1 0 1 1 0 0 1]'. The ' symbol indicates matrix transpose. Format of msg can also be binary matrix with k columns. In this case format of code will be binary matrix with n columns. msg = [0 1 1 0; 0 1 0 1; 1 0 0 1]. Here k = 4. For Linear Block codes, encode function encodes msg using genmat as the generator matrix. genmat, a k-by-n matrix, is required as input. Example: The example below illustrates two different information formats (binary vector and binary matrix) for linear block code. The two messages have identical content in different formats. As a result, the two codes created by encode function have identical content in correspondingly different formats. Here k = 11. Putting r = 4 to satisfy (A) r = 4; % r is the number of check bits. k = 11; % Message length n = k + r % Codeword length = 15 using formula n-k=r % Create 100 messages, k bits each. msg1 = randint(100*k,1,[0,1]); % As a column vector msg2 = vec2mat(msg1,k); % As a k-column matrix % Create 100 codewords, n bits each. P =[1 1 1 1; 0 1 1 1; 1 1 1 0; 1 1 0 1; 0 0 1 1; 0 1 0 1; 0 1 1 0; 1 0 0 1; 1 0 1 0; 1 0 1 1; 1 1 0 0] genmat=[eye(11) P]; % concatenate P submatrix or predefined rule with Identity matrix. code1 = encode(msg1,n,k,'linear/binary',genmat); code2 = encode(msg2,n,k,'linear/binary',genmat); if ( vec2mat(code1,n)==code2 ) disp('All two formats produced the same content.') end Instead of randomly generating data words, you can create a data matrix of your own containing the data words that you want to encode.
16
Information Theory
Lab Session 03
Exercise: 3.1) Given (6,3) linear block code generated by the predefined matrix [0 1 1; 1 0 1; 1 1 0].
3.1.1) Encode the messages [1 1 1] and [1 0 1] manually and verify through MATLAB. 111 = __________. 101 = _________
___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 3.1.2) Encode all the possible messages in MATLAB and write the encoded words.
Message Code Vector 000 001 010 011 100 101 110 111
17
Information Theory
Lab Session 03
3.1.3) The minimum hamming distance = _____________. 3.1.4) The no. of errors this code can detect is ____________ 3.1.5) The no. of errors this code can correct is ____________ 3.1.6) The code rate = ___________________. 3.1.7) Is this code a systematic code? Justify your answer. (HINT: see the tabulated code words obtained in Q1: b) ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ 3.2) Consider a systematic (6, 3) block code generated by the submatrix [1 1 0; 0 1 1; 1 0 1].
3.2.1) Write the value of n and k. ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ 3.2.2) Tabulate the code words through simulation and write their corresponding weights. Message Code Vector Weight dmin 000 001 010 011 100 101 110 111 3.2.3) What is the value of dmin ? ________________________________________________________________________
18
Information Theory
Lab Session 03
3.3)
Write MATLAB code to create 10 messages randomly having size of 4 bits each. Design appropriate encoding rule and use encode function to encode the generated messages.
___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________
19
Information Theory
Lab Session 04
Lab Session 04
OBJECT
Decoding encoded words for a forward error correction system with a given Linear Block code and verifying through simulation
THEORY
Let Y stands for the received vector when a particular code vector X has been transmitted. Any transmission errors will result in Y X. The decoder detects or corrects errors in Y using stored information about the code. A direct way of performing error detection would be to compare Y with every vector in the code. This method requires storing all 2k code vectors at the receiver and performing up to 2k comparisons. But efficient codes generally have large values of k, which implies rather extensive and expensive decoding hardware. More practical decoding methods for codes with large k involve parity check information derived from the codes P submatrix. Associated with any systematic linear (n,k) block code is a (n-k) n matrix called the parity check matrix H. This matrix is defined by H = [P | Ir] Where Ir is the r r identity matrix and n - k = r. The parity check matrix has a crucial property for error detection which is XHT = (0 0 0 0 ..0) provided that X belongs to the set of code vectors. However, when Y is not a code vector, the product YHT contains at least one nonzero element. Therefore, given HT and a received vector Y, error detection can be based on S = Y HT an r-bit vector called the syndrome. If all elements of S equal zero, then either Y equals the transmitted vector X and there are no transmission errors, or Y equals some other code vector and the transmission errors are undetectable. Otherwise, errors are indicated by the presence of nonzero elements in S. Thus a decoder for error detection simply takes the form of a syndrome calculator. We develop the decoding method by introducing an n-bit error vector E whose non zero elements mark the positions of transmission errors in Y. For instance, if X = (1 0 1 1 0) and Y = (1 0 0 1 1) then E = (0 0 1 0 1). In general, Y=X E And conversely, X=Y E Substituting Y = X + E into S = YHT, we obtain S = (X E)HT S =XHT EHT S = EHT which reveals that the syndrome depends entirely on the error pattern, not the specific transmitted vector.
20
Information Theory
Lab Session 04
Syndrome decoding: example An (8, 4) binary linear block code C is defined by systematic matrices: 1000|0111 0100|1011 0010|1101 0001|1110 0111|1000 1011|0100 1101|0010 1110|0001
G=
H=
Consider two possible messages: m1 = [0 1 1 0] C = DG or MG C = [

C = [[( ) ][ ]
m2 = [1 0 1 1]
)]
[(
)]]
First and last elements of the matrix are shown above

C = [(
) ( ) (
) ( ) (
) ( )]
) (
)]
C = [0 1 1 0 0 1 1 0] Hence, c1 = [0 1 1 0 0 1 1 0]
c2 = [1 0 1 1 0 1 0 0]
Suppose error pattern e = [0 0 0 0 0 1 0 0] is added to both code words. r1 = [0 1 1 0 0 0 1 0] r2 = [0 1 0 0 1 1 1 1]
Calculating the syndrome using S = YHT. Here Y is the received erroneous code word. s1 = [0 1 0 0] s2 = [0 1 0 0 ]
The syndromes are the same and equal column 6 of H, so decoder corrects bit 6. Bit Error Rate: Bit error rate = average no. of erroneous bits per block / total no. of bits per block MATLAB FUNCTIONS: In order to develop MATLAB code for decoding and calculating the bit error rate, the following functions may be used. 1) DECODE: Function: Block decoder Syntax msg = decode(code,n,k,'linear/fmt',genmat,trt) [msg,err] = decode(...)
21
Information Theory
Lab Session 04
trt Uses syndtable to create the syndrome decoding table associated with the method's parity-check matrix. Syndtable function is described later in this lab session. Description The decode function aims to recover messages that were encoded using an error-correction coding technique. The technique and the defining parameters must match those that were used to encode the original signal. Decode function decodes code, which is a linear block code determined by the k-by-n generator matrix genmat. genmat is required as input. decode tries to correct errors using the decoding table trt, where trt is a 2^(n-k)-by-n matrix. 2) SYNDTABLE: Function: Produces syndrome decoding table Syntax: t = syndtable(h) Description t = syndtable(h) returns a decoding table for an error-correcting binary code having codeword length n and message length k. h is an (n-k)-by-n parity-check matrix for the code. t is a 2n-k-by-n binary matrix. The rth row of t is an error pattern for a received binary codeword whose syndrome has decimal integer value r-1. (The syndrome of a received codeword is its product with the transpose of the parity-check matrix.). 3) BITERR: Function: Computes number of bit errors and bit error rate. Syntax [number,ratio] = biterr(x,y) Description The biterr function compares unsigned binary representations of elements in vector x with those in vector y. Each element of x and y must be a nonnegative decimal integer; biterr converts each element into its natural unsigned binary representation. number is a scalar or vector that indicates the number of bits that differ. ratio is number divided by the total number of bits. The total number of bits, the size of number, and the elements that biterr compares are determined by the dimensions of x and y. 4) GEN2PAR Function: Convert between parity-check and generator matrices Syntax parmat = gen2par(genmat) genmat = gen2par(parmat) Description parmat = gen2par(genmat) converts the standard-form binary generator matrix genmat into the corresponding parity-check matrix parmat. genmat = gen2par(parmat) converts the standard-form binary parity-check matrix parmat into the corresponding generator matrix genmat.
22
Information Theory
Lab Session 04
Example 1 The command below compares the column vector [0; 0; 0] to each column of a random binary matrix. The output is the number and proportion of 1s in the matrix. [number,ratio] = biterr([0;0;0],randint(3,5)) The output is number = 2 ratio = 0.6667 1 0.3333 1 0.3333 0 0 1 0.3333
Example 2 The script below adds errors to 10% of the elements in a matrix. Each entry in the matrix is a two-bit number in decimal form. The script computes the bit error rate using biterr. x = randint(100,100,4); % Original signal % Create errors to add to ten percent of the elements of x. % Errors can be either 1, 2, or 3 (not zero). errorplace = (rand(100,100) > .9); % Where to put errors errorvalue = randint(100,100,[1,3]); % Value of the errors errors = errorplace.*errorvalue; y = rem(x+errors,4); % Signal with errors added, mod 4 [num_bit,ratio_bit] = biterr(x,y,2) The output is num_bit = ratio_bit = 1304 0.0652
Example 3 The commands below convert the parity-check matrix for a Hamming code into the corresponding generator matrix and back again. parmat = hammgen(3) genmat = gen2par(parmat) parmat2 = gen2par(genmat) % Ans should be the same as parmat above The output parmat = 1 0 0 genmat = 1 0 1 1 parmat2 = 1 0 0 is 0 1 0 1 1 1 0 0 1 0 0 0 1 0 1 1 1 0 0 1 1 1 0 1 0 0 0 1 1 0 0 1 1 0 1 0 0 0 1 1 1 1 1 0 0 1 0 1 1 1 1 0 1 0 0 0 1 1 0 1
23
Information Theory
Lab Session 04
The above example uses hammgen function whose explanation is as follows. h = hammgen(m) The codeword length is n. n has the form 2m-1 for some positive integer m greater than or equal to 3. The message length, k, has the form n-m. hammgen function produces an m-byn parity-check matrix for a Hamming code having codeword length n = 2m-1. The input m is a positive integer greater than or equal to 3. The message length of the code is n-m. You can also provide your own generator matrix as input. Example 4 The next example creates a linear code, adds noise, and then decodes the noisy code. It uses the decode function. n = 3; k = 2; % A (3,2) linear code msg = randint(100,k,[0,1]); % 100 messages, k bits each code = encode(msg,n,k,'linear/binary');% Add noise. noisycode = rem(code + randerr(100,n,[0 1;.7 .3]), 2); newmsg = decode(noisycode,n,k,'linear'); % Try to decode. % Compute error rate for decoding the noisy code. [number,ratio] = biterr(newmsg,msg); disp(['The bit error rate is ',num2str(ratio)]) The output is below. Your error rate results might vary because the noise is random. The bit error rate is 0.08 Exercises: 4.1) Apply table lookup decoding to (7,4) hamming code. The P submatrix is given as 1 1 1 0 0 1 1 1 1 1 0 1
4.1.1) Write the value of n and k. ________________________________________________________________________ 4.1.2) Create G and H matrices. ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________
24
Information Theory
Lab Session 04
4.1.3) Tabulate the code words through simulation and write their corresponding weights.
Message Code Vector Weights 0000 0001 0010 0011 0100 0101 0110 0111 1000 1001 1010 1011 1100 1101 1110 1111
4.1.4) The minimum hamming distance = _____________. 4.1.5) The no. of errors this code can detect is ________________________________________________________________________ ________________________________________________________________________ 4.1.6) The no. of errors this code can correct is ________________________________________________________________________ ________________________________________________________________________ 4.1.7) Find syndromes for all possible single error patterns.
S 000 101
E 0000000 1000000
25
Information Theory
Lab Session 04
4.1.8) Suppose a received word happens to have two errors, such that E = (1 0 0 0 0 1 0). Calculate the syndrome. ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ 4.1.9) What will be the error pattern (write form table). ________________________________________________________________________ 4.1.10) Write the corrected word. ________________________________________________________________________ 4.1.11) What observations do you make? How many errors the decoded output word will contain. ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ 4.2) Write MATLAB code to construct lookup table for the (6,3) block code generated by the submatrix 1 1 0 0 1 1 1 0 1 ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________
26
Information Theory
Lab Session 04
4.3)
Consider a (5,3) block code obtained by deleting the last column of the P submatrix in above question. Construct the lookup table and show that this code could be used for error detection and not correction. Also write MATLAB code to implement the given functionality.
___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 4.4) A (7,4) LBC has generator matrix G given by 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 1 1 0 1 1 0 1 1 0 1 1 1
27
Information Theory
Lab Session 04
4.4.1) Determine the PCM for above matrix. ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ 4.4.2) Write MATLAB code to encode all possible data words and then decode all the encoded words. ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ 4.4.3) Insert double error in any code block. Decode manually and through simulation. ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________
28
Information Theory
Lab Session 04
4.4.4) The bit error rate for single error in first two code blocks is ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ 4.4.5) Insert single error in every code block. Write MATLAB code to find bit error rate. ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________
29
Information Theory
Lab Session 05
Lab Session 05
OBJECT
Encoding the messages for a system with a given cyclic polynomial code and verifying through simulation
THEORY
Cyclic codes are a subclass of linear block codes with a cyclic structure that leads to more practical implementation. Block codes used in forward error correction systems are almost always cyclic codes. In cyclic coding, data is sent with a checksum. When arrives, checksum is recalculated. It should match the one that was sent. To describe a cyclic code, we express an arbitrary n-bit vector in the form X = (xn-1 xn-2 x1 x0) Now suppose that X has been loaded into a shift register with feedback connection from the first to last stage. Shifting all bits one position to the left yields the cyclic shift of X, written as X' = (xn-2 xn-3 x1 x0 xn-1) A second shift produces X = (xn-3 x1 x0 xn-1 xn-2) and so forth. A linear code is cyclic if every cyclic shift of a code vector X is another vector in the code. Since cyclic codes are linear block codes, all the properties of linear block codes apply to cyclic codes. Cyclic codes are used in applications where burst errors can occur. Burst error is an error in which a group of adjacent bits is affected. Encoding of Cyclic Codes Direct Approach: Let M(x) is the message polynomial and G(x) is the generator polynomial. Then the code vector V(x) can be encoded as V(x) = M(x). G(x) A linear code is cyclic if every cyclic shift of a code vector X is another vector in the code.
30
Information Theory
Lab Session 05
Example 1 The (6, 2) repetition code having valid code C = {[000000], [010101], [101010], [111111]} is cyclic, since a cyclic shift of any of its code vectors results in a vector that is an element of C. Example 2 The (5, 2) LBC is defined by the generator matrix G = [1 0 1 1 1; 0 1 1 0 1] is a single error correcting code. We need to determine if it is cyclic. The valid code words are C = {[00000], [01101], [10111], [11010]} The given code is NOT a cyclic code because cyclic shift of (11010) results in (10101) which is not a valid code word. Also cyclic shift of (10111) results in (01111) which is not a member of C. It is convenient to think of words as polynomials rather than vectors. Since the code is binary, the coefficients will be 0 or 1. The degree of polynomial equals to its highest exponent. For example the degree of the polynomial obtained from (1 0 1 1) is 3 Example 3 Given (6, 3) code with G(x) = (1 1 0 1). Encode M(x) = (101) V(x) = M(x). G(x) V(x) = (x3 + x2 + 1). (x2 + 1) V(x) = x5 + x3 + x4 + x2 + x2 + 1 V(x) = 1.x5 + 1.x4 + 1.x3 + (1 1) x2 + 0.x + 1 V(x) = 1.x5 + 1.x4 + 1.x3 + 0.x2 + 0.x + 1 V(x) = (1 1 1 0 0 1) The degree of polynomial is defined to be the highest power of x in the polynomial for which the coefficient of x is not zero. The degree of polynomial M(x) is denoted by deg [M(x)]. If M is our set of k bit source vectors, the highest degree polynomial in the set of polynomials M(x) that describes our source vectors is k-1. If deg [G(x)] = r, then the highest degree of our set of code polynomials V(x) is n-1 or k-1+r. Therefore deg [G(x)] tells us the number of check bits in the code. Property of generator Polynomial G(x) is the generator polynomial for a linear cyclic code of length n if and only if G(x) divides 1+xn without a remainder. Example 4 Given (7, 4) cyclic code. Find a generator polynomial for a code of length n = 7 for encoding of data of length k = 4.
31
Information Theory
Lab Session 05
Solution: G(x) should be a degree 7 4 = 3 and should divide 1 + xn without a remainder. 1 + x7 can be factorized as 1 + x7 = (1+x+x3) (1+x2+x3) (1+x) So we can choose for G(x) either (1+x+x3) or (1+x2+x3). Systematic form: Let U(x) = uk-1 xk-1 + uk-2 xk-2 + ... + u1 x+ u0 be an information polynomial and G (x) = g n - k x n - k + g n - k - 1xn -k-1 +... + g1 x + g0 be a generator polynomial of an (n, k) cyclic code. The code polynomial is given by V(x) = xn-k U(x) / G(x) = Vs(x) / G(x) Where Vs(x) = x n -k U(x) Example 5 Consider a message 110010 represented by the polynomial M(x) = x5 + x4 + x Consider a generating polynomial G(x) = x3 + x2 + 1 (1101) This is used to generate a 3 bit CRC = C(x) to be appended to M(x). STEPS: 1. Multiply M(x) by x3 (highest power in G(x)). I.e. Add 3 zeros. 110010000 2. Divide the result by G(x). The remainder = C(x). 1101 long division into 110010000 (with subtraction mod 2) = 100100 remainder 100 Special case: This won't work if bit string = all zeros. We don't allow such an M(x). Transmit 110010000 + 100. = 110010100 To be precise, transmit: T(x) = x3M(x) + C(x)
3. Receiver end: Receive T(x). Divide by G(x), should have remainder 0.
00
32
Information Theory
Lab Session 05
MATLAB FUNCTIONS 1) Encode Function: Block encoder Syntax Code = encode (msg,n,k,'cyclic/fmt',genpoly) Description Encode function encodes msg and creates a systematic cyclic code. genpoly is a row vector that gives the coefficients, in order of ascending powers, of the binary generator polynomial. The default value of genpoly is cyclpoly (n,k). This function is explained below. By definition, the generator polynomial for an [n, k] cyclic code must have degree n-k and must divide xn-1. You are already familiar with the input arguments msg, n and k. 2) Cyclpoly Function: Produce generator polynomials for a cyclic code. Syntax pol = cyclpoly (n, k) Description In MATLAB a polynomial is represented as a row containing the coefficients in order of ascending powers. The results you would get will be in ascending order of coefficients while in manual calculation the results will be in descending order. It does not matter what convention you follow but it should be consistent. Cyclpoly function returns the row vector representing one nontrivial generator polynomial for a cyclic code having codeword length n and message length k. Example 6 The following example creates a cyclic code, adds noise, and then decodes the noisy code. It uses the decode function. n = 3; k = 2; % A (3, 2) cyclic code msg = randint(100,k,[0,1]); % 100 messages, k bits each Code = encode (msg, n, k, 'cyclic/binary'); % Add noise. noisycode = rem(code + randerr (100,n,[0 1;.7 .3]), 2); newmsg = decode(noisycode, n, k, 'cyclic'); % Try to decode. % Compute error rate for decoding the noisy code. [Number, ratio] = biterr (newmsg, msg); disp (['The bit error rate is ',num2str(ratio)]) Example 7 The example below illustrates the use of err and cerr. The script encodes a five-bit message using a cyclic code. The codeword has fifteen bits. No Error is added to the codeword. Then decode is used to recover the original message. m = 4 n = 2^m-1 % Codeword length is 15. k = 5 % Message length msg = ones(1,5) % One message of five bits.
33
Information Theory
Lab Session 05
Code = encode (msg,n,k,'cyclic') noisycode = code % No error
% Encode the message.
% decodes and tries to correct the errors. [newmsg,err,ccode,cerr] = decode (noisycode,n,k,'cyclic') disp ('Transpose of err is') disp (err') disp ('Transpose of cerr is') disp (cerr') The Galois Field Computation in MATLAB A Galois field is an algebraic field that has a finite number of members. Galois fields having 2^m members are used in error-control coding and are denoted GF (2^m) where m is an integer between 1 and 16. This portion describes how to work with fields that have 2^m members with m = 0. 3) gf Function: Creates a Galois field array. Syntax x_gf = gf(x,m) x_gf = gf(x) Description x_gf = gf(x,m) creates a Galois field array from the matrix x. The Galois field has 2^m elements, where m is an integer between 1 and 16. The elements of x must be integers between 0 and 2^m-1. The output x_gf is a variable that MATLAB recognizes as a Galois field array, rather than an array of integers. As a result, when you manipulate x_gf using operators or functions such as + or det, MATLAB works within the Galois field you have specified. X_gf = gf(x) creates a GF (2) array from the matrix x. Each element of x must be 0 or 1. Example 8 Let g = [0 1 1; 1 0 1; 1 1 0] d = [0 1 0] Run the following statement c = d*g Output: c = 1 0 1 Using d = [1 0 1] rerun the above statement. Output:
34
Information Theory
Lab Session 05
c =
When you perform modulo 2 multiplications on two binary vectors, you get a binary vector as result. This is not the case here because MATLAB considers these vector elements as integers. For this purpose we convert the vectors into Galois field of order 2. Now convert all the given vectors and g matrix to Galois field and do the computation again. G=gf (g) D=gf (d) c=D*G Array elements = 1 0 1
4) conv Function: Convolution and polynomial multiplication Syntax: w = conv (u,v) Description: conv function convolves vectors u and v. Algebraically, convolution is the same operation as multiplying the polynomials whose coefficients are the elements of u and v. Example 9 m = [1 1 0 0 1 0] g = [1 1 0 1] Compute c = m*g manually. Answer: 101101010. Run the following statement. c = m*g Note the error you get. Convert m and g in to Galois field arrays. M = gf(m) G = gf(g) Rerun the statement C=M*G Now note the error that you get. Despite of the fact that we have converted m and g into Galois field, still multiplication is not possible because sizes of m and g dont match. Now use conv function C = conv (M, G) Note that now you get the correct result. 5) deconv Function: Deconvolution and polynomial division. Syntax: [q,r] = deconv (v,u) Description:
35
Information Theory
Lab Session 05
Deconv function deconvolves vector u out of vector v, using long division. The quotient is returned in vector q and the remainder in vector r such that v = conv (u,q)+r. If u and v are vectors of polynomial coefficients, convolving them is equivalent to multiplying the two polynomials, and deconvolution is polynomial division. The result of dividing v by u is quotient q and remainder r. Example 10 m=[1 1 0 0 1 0] g=[1 1 0 1] Using long division we compute (x3 * m) / g manually. Answer: Quotient: 100100 Remainder:100 Now perform the same functionality using MATLAB. M=gf(m) G=gf(g) h=[1 0 0 0] H=gf(h) C=conv(H,M) [q,r]=deconv(C,G)
% or x3 that is highest power of g(x) % multiplying x3 * m(x) % dividing [x3*m(x)] by g(x)
See that the results you get in both the cases match. Exercises: 5.1) The generator polynomial of a (7, 4) cyclic code is given by G(x) = 1 + x2 + x3
Find V by direct method as well as the systematic approach if M(x) = (1 0 1 0). Write code to implement the above functionality in MATLAB using encode function and then conv function. ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________
36
Information Theory
Lab Session 05
___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 5.2) Consider the cyclic (7, 4) cyclic code generated by G(x) = x3 + x2 + 1. 5.2.1) Use long division to calculate the check bit polynomial C(x) when M = (1 1 0 0). Also write the complete code polynomial. ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________
37
Information Theory
Lab Session 05
5.2.2) Write MATLAB code to perform the above functionality. Use encode function and then conv/deconv functions. ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ The generator polynomial of (7, 4) cyclic codes is given by G(x) = 1 + x2 + x3. Find V by the systematic approach if M(x) = (1 0 0 1). ___________________________________________________________________________ 5.3) ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________
38
Information Theory
Lab Session 05
5.4) Consider the cyclic (7, 4) hamming code generated by G(x) = x3 + x + 1. 5.4.1) Use systematic approach to calculate the check bit polynomial C(x) when M = (1 1 0 1). Also write the complete code polynomial. ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ 5.4.2) Write MATLAB code to implement the functionality explained in part (a). ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________
39
Information Theory
Lab Session 05
5.5) Consider (6, 3) cyclic code. Create MATLAB code to perform the following tasks. 5.5.1) Create generator polynomial. ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ 5.5.2) Write all possible code words and encode using encode function. ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________
40
Information Theory
Lab Session 06
Lab Session 06
OBJECT
Decoding the messages for a system with a given cyclic polynomial code and verifying through simulation
THEORY
Cyclic codes are a subclass of linear block codes with a cyclic structure that leads to more practical implementation. Block codes used in forward error correction systems are almost always cyclic codes. In cyclic coding, data is sent with a checksum. When arrives, checksum is recalculated. It should match the one that was sent. To describe a cyclic code, we express an arbitrary n-bit vector in the form X = (xn-1 xn-2 x1 x0) Now suppose that X has been loaded into a shift register with feedback connection from the first to last stage. Shifting all bits one position to the left yields the cyclic shift of X, written as X' = (xn-2 xn-3 x1 x0 xn-1) A second shift produces X = (xn-3 x1 x0 xn-1 xn-2) and so forth. A linear code is cyclic if every cyclic shift of a code vector X is another vector in the code. Since cyclic codes are linear block codes, all the properties of linear block codes apply to cyclic codes. Cyclic codes are used in applications where burst errors can occur. Burst error is an error in which a group of adjacent bits is affected. Decoding of Cyclic codes: Let T(x) and E(x) be the code polynomial of a systematic cyclic code and an error polynomial of degree n - 1 or less, respectively. An error is the same as adding some E(x) to T(x) for e.g. adding 1010011000001110000 will flip the bits at the locations where "1" is in the error bit string. Instead of T(x) arriving, T(x) + E(x) arrives. So R(x) = T(x) E(x) In general, each 1 bit in E(x) corresponds to a bit that has been flipped in the message. If there are k 1 bits in E(x), k single-bit errors have occurred. A burst error looks like 1....1 Far end receives T(x) + E(x) = R(x). T(x) is multiple of G(x) (remainder zero). Hence remainder when you divide (T(x) + E(x)) by G(x) = remainder when you divide E(x) by G(x). For e.g. remainder when divide (1000+n) by 10 = remainder when you divide n by 10 Here the remainder of E(x) / G(x) is called syndrome vector denoted by S(x). If remainder when you divide E(x) by G(x) is zero, the error will not be detected. In general, if E(x) is a multiple of G(x), the error will not be detected. Otherwise, it will. All other error patterns will be caught. 41
Information Theory
Lab Session 06
MATLAB FUNCTIONS 1) Decode Function: Block decoder Syntax msg = decode (code, n ,k, 'cyclic/fmt', genpoly, trt) [msg, err] = decode (...) [msg, err, ccode] = decode (...) Description decode function decodes the cyclic code code and tries to correct errors using the decoding table trt, where trt is a 2^(n-k)-by-n matrix. genpoly is a row vector that gives the coefficients, in order of ascending powers, of the binary generator polynomial of the code. The default value of genpoly is cyclpoly (n, k). By definition, the generator polynomial for an [n, k] cyclic code must have degree n-k and must divide xn-1. 2) Cyclpoly Function: Produce generator polynomials for a cyclic code. Syntax pol = cyclpoly (n, k) Description In MATLAB a polynomial is represented as a row containing the coefficients in order of ascending powers. The results you would get will be in ascending order of coefficients while in manual calculation the results will be in descending order. It does not matter what convention you follow but it should be consistent. Cyclpoly function returns the row vector representing one nontrivial generator polynomial for a cyclic code having codeword length n and message length k. Example 1 The following example creates a cyclic code, adds noise, and then decodes the noisy code. It uses the decode function. n = 3; k = 2; % A (3, 2) cyclic code msg = randint(100,k,[0,1]); % 100 messages, k bits each Code = encode (msg, n, k, 'cyclic/binary'); % Add noise. noisycode = rem(code + randerr (100,n,[0 1;.7 .3]), 2); newmsg = decode(noisycode, n, k, 'cyclic'); % Try to decode. % Compute error rate for decoding the noisy code. [Number, ratio] = biterr (newmsg, msg); disp (['The bit error rate is ',num2str(ratio)]) Example 2 The example below illustrates the use of err and cerr. The script encodes a five-bit message using a cyclic code. The codeword has fifteen bits. No Error is added to the codeword. Then decode is used to recover the original message. m = 4 n = 2^m-1 % Codeword length is 15. k = 5 % Message length msg = ones(1,5) % One message of five bits. Code = encode (msg,n,k,'cyclic') % Encode the message.
42
Information Theory
Lab Session 06
noisycode = code % No error % decodes and tries to correct the errors. [newmsg,err,ccode,cerr] = decode (noisycode,n,k,'cyclic') disp ('Transpose of err is') disp (err') disp ('Transpose of cerr is') disp (cerr') The Galois Field Computation in MATLAB A Galois field is an algebraic field that has a finite number of members. Galois fields having 2^m members are used in error-control coding and are denoted GF (2^m) where m is an integer between 1 and 16. This portion describes how to work with fields that have 2^m members with m = 0. 3) gf Function: Creates a Galois field array. Syntax x_gf = gf(x,m) x_gf = gf(x) Description x_gf = gf(x,m) creates a Galois field array from the matrix x. The Galois field has 2^m elements, where m is an integer between 1 and 16. The elements of x must be integers between 0 and 2^m-1. The output x_gf is a variable that MATLAB recognizes as a Galois field array, rather than an array of integers. As a result, when you manipulate x_gf using operators or functions such as + or det, MATLAB works within the Galois field you have specified. X_gf = gf(x) creates a GF(2) array from the matrix x. Each element of x must be 0 or 1. Example 3: Let g = [0 1 1; 1 0 1; 1 1 0] d = [0 1 0] Run the following statement c = d*g Output: c = 1 0 1 Using d = [1 0 1] rerun the above statement. Output: c = 1 2 1
When you perform modulo 2 multiplications on two binary vectors, you get a binary vector as result. This is not the case here because MATLAB considers these vector elements as integers. For this purpose we convert the vectors into Galois field of order 2. Now convert all the given vectors and g matrix to Galois field and do the computation again. G=gf (g) D=gf (d) c=D*G Array elements = 1 0 1
43
Information Theory
Lab Session 06
4) conv Function: Convolution and polynomial multiplication Syntax: w = conv (u,v) Description: conv function convolves vectors u and v. Algebraically, convolution is the same operation as multiplying the polynomials whose coefficients are the elements of u and v. Example 4: m = [1 1 0 0 1 0] g = [1 1 0 1] Compute c = m*g manually. Answer: 101101010. Run the following statement. c = m*g Note the error you get. Convert m and g in to Galois field arrays. M = gf(m) G = gf(g) Rerun the statement C=M*G Now note the error that you get. Despite of the fact that we have converted m and g into Galois field, still multiplication is not possible because sizes of m and g dont match. Now use conv function C = conv (M, G) Note that now you get the correct result. 5) deconv Function: Deconvolution and polynomial division. Syntax: [q,r] = deconv (v,u) Description: Deconv function deconvolves vector u out of vector v, using long division. The quotient is returned in vector q and the remainder in vector r such that v = conv (u,q)+r. If u and v are vectors of polynomial coefficients, convolving them is equivalent to multiplying the two polynomials, and deconvolution is polynomial division. The result of dividing v by u is quotient q and remainder r. Example 5: m=[1 1 0 0 1 0] g=[1 1 0 1] Using long division we compute (x3 * m) / g manually. Answer: Quotient: 100100 Remainder:100 Now perform the same functionality using MATLAB.
44
Information Theory
Lab Session 06
M=gf(m) G=gf(g) h=[1 0 0 0] H=gf(h) C=conv(H,M) [q,r]=deconv(C,G)
% or x3 that is highest power of g(x) % multiplying x3 * m(x) % dividing [x3*m(x)] by g(x)
See that the results you get in both the cases match. Exercises: 6.1) Consider the cyclic (7, 4) cyclic code generated by G(x) = x3 + x + 1. 6.1.1) Use long division to calculate the check bit polynomial C(x) when M = (1 1 0 0). Also write the complete code polynomial. ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ 6.1.2) Write MATLAB code to perform the above functionality. Use decode functions and then conv/deconv functions. ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________
45
Information Theory
Lab Session 06
________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ The generator polynomial of (7, 4) cyclic codes is given by G(x) = 1 + x2 + x3. Find V by the systematic approach if M(x) = (1 0 0 1).Let the received word is [1 1 1 0 0 1 1]. Find the syndrome vector. ___________________________________________________________________________ 6.2) ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 6.3) Consider the cyclic (7, 4) hamming code generated by G(x) = x3 + x + 1. 6.3.1) Use systematic approach to calculate the check bit polynomial C(x) when M = (1 1 0 1). Also write the complete code polynomial. ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________
46
Information Theory
Lab Session 06
___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 6.3.2) Write MATLAB code to implement the functionality explained in part (a).
___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ Find the syndrome vectors for the following erroneous received words. i. 1010101 ii. 1000001 ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 6.3.3)
47
Information Theory
Lab Session 06
6.4) Consider (6, 3) cyclic code. Create MATLAB code to perform the following tasks. 6.4.1) Create generator polynomial. ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 6.4.2) Write all possible code words and encode using encode function. ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 6.4.3) Use decode function to decode all the encoded words. ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 6.4.4) Induce single error in all the encoded words and then decode. Record the output that you get.
___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________
48
Information Theory
Lab Session 07
Lab Session 07
OBJECT
Understanding the concept of loss-less data compression technique using Huffman coding
THEORY
Huffman coding
Huffman coding is a compression method that converts characters into variable length bit strings. Most-frequently occurring characters are converted to shortest bit strings; least frequent, the longest. Huffman coding uses a specific method for choosing the representation for each symbol, resulting in a prefix-free code (sometimes called "prefix codes") (that is, the bit string representing some particular symbol is never a prefix of the bit string representing any other symbol). The Prefix Property Data encoded using Huffman coding is uniquely decodable. This is because Huffman codes satisfy an important property called the prefix property. This property guarantees that no codeword is a prefix of another Huffman codeword. For example, 10 and 101 cannot simultaneously be valid Huffman code words because the first is a prefix of the second. Thus, any bit stream is uniquely decodable with a given Huffman code. We can see by inspection that the code words generated below are valid Huffman code words. To see why the prefix property is essential, consider the code words given below in which e is encoded with 110 which is a prefix of f. character codeword a 0 b 101 c 100 d 111 e 110 f 1100
The decoding of 11000100110 is ambiguous: 11000100110 => face 11000100110 => eaace
Huffman Coding in MATLAB

In MATLAB Huffman code dictionary associates each data symbol with a codeword. It has the property that no codeword in the dictionary is a prefix of any other codeword in the dictionary. The huffmandict, huffmanenco, and huffmandeco functions support Huffman coding and decoding. Creating a Huffman Code Dictionary dict = huffmandict(symbols, p) Huffman coding requires statistical information about the source of the data being encoded. In particular, the p input argument in the huffmandict function lists the probability with which the source produces each symbol in its alphabet. 49
Information Theory
Lab Session 07
For example, consider a data source that produces 1s with probability 0.1, 2s with probability 0.1, and 3s with probability 0.8. The main computational step in encoding data from this source using a Huffman code is to create a dictionary that associates each data symbol with a codeword. The commands below create such a dictionary and then show the codeword vector associated with a particular value from the data source. symbols = [1 2 3]; % Data symbols p = [0.1 0.1 0.8]; % Probability of each data symbol dict = huffmandict(symbols, p) % Create the dictionary. dict {1,:} % Show first row of the dictionary. dict {2,:} % Show second row of the dictionary. dict {3,:} % Show third row of the dictionary. The output below shows that the most probable data symbol, 3, is associated with a one-digit codeword, while less probable data symbols are associated with two-digit code words. The output also shows, for example, that a Huffman encoder receiving the data symbol 1 should substitute the sequence 11. dict = ans ans ans ans ans ans = = = = = = [1] [2] [3] 1 1 2 1 3 0 [1x2 double] [1x2 double] [ 0] 1 0
Creating and Decoding a Huffman Code The example below performs Huffman encoding and decoding using huffmanenco and huffmandeco functions. A source is used whose alphabet has three symbols. Notice that the huffmanenco and huffmandeco functions use the dictionary that huffmandict created. sig = repmat([3 3 1 3 3 3 3 3 2 3],1,50); % Data to encode symbols = [1 2 3]; % Distinct data symbols appearing in sig p = [0.1 0.1 0.8]; % Probability of each data symbol dict = huffmandict(symbols,p); % Create the dictionary. hcode = huffmanenco(sig,dict); % Encode the data. dhsig = huffmandeco(hcode,dict); % Decode the code.
50
Information Theory
Lab Session 07
Exercises: 7.1) Five source symbols of the alphabet of a source and their probabilities are shown Symbol S0 S1 S2 S3 S4 Probability 0.4 0.2 0.2 0.1 0.1
7.1.1) Build the Huffman code tree for the message and find the codeword for each character. ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 7.1.2) Write MATLAB code to obtain Huffman coding for the given characters. ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________
51
Information Theory
Lab Session 07
7.2)
Construct a Huffman code for the source X emitting symbols with probabilities as shown below. Symbol A C E Probability 0.4 0.1 0.06 Symbol B D F Probability 0.3 0.1 0.04
Determine: 7.2.1) The code word for each symbol. ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 7.2.2) The average number of transmitted binary digits per code word. ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________
7.2.3) Write MATLAB code to obtain Huffman code for the given source. ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________
52
Information Theory
Lab Session 08
Lab Session 08
OBJECT
Implementation of Hill cipher technique in cryptography
THEORY
Cryptography:
Cryptography is the science of keeping secrets secret. Assume a sender referred to here as Jake wants to send a message m to a receiver referred to as Jill. He uses an insecure communication channel. For example, the channel could be a computer network or a telephone line. There is a problem if the message contains confidential information. The message could be intercepted and read by an eavesdropper. Or, even worse, the adversary might be able to modify the message during transmission in such a way that the legitimate recipient Jill does not detect the manipulation. One objective of cryptography is to provide methods for preventing such attacks. The fundamental and classical task of cryptography is to provide confidentiality by encryption methods. The message to be transmitted can be some text, numerical data. An executable program or any other kind of information called the plaintext. Providing confidentiality is not the only objective of cryptography. Cryptography is also used to provide solutions for other problems: 1. Data integrity. The receiver of a message should be able to check whether the message was modified during transmission, either accidently or deliberately. No one should be able to substitute a false message for the original message, or for parts of it. 2. Authentication. The receiver of a message should be able to verify its origin. No one should be able to send a message to Jake and pretend to be Jill (data origin authentication). When initiating a communication, Jake and Jill should be able to identify each other (entity authentication). 3. Non-repudiation. The sender should not be able to later deny that she sent a message. Few terminologies: Cryptanalysis is the (art and) science of breaking a code Cryptology is the branch of math needed for cryptography and cryptanalysis The original message is plaintext (sometimes also called clear text) Disguising the content of a message is called encryption. This results in cipher text, the encrypted message. Turning cipher text back into plaintext is called decryption Flow of Cryptographic Algorithm: A cryptographic algorithm (also called a cipher) is the mathematical function used for encryption and decryption. Usually there are two functions, one for encryption and one for decryption. If security is based on keeping the algorithm secret, then its a restricted algorithm. Restricted algorithms are not a good idea because every time a user leaves a group; the algorithm has to be changed. A group must develop their own algorithm; if they dont have the expertise, then it will be subpar. 53
Information Theory
Lab Session 08
Modern algorithms use a key, denoted by K. A key is taken from a large range of possible values, the keyspace. A key is used as additional input for the en-/decryption function to do the en-/decrypting.
Cryptography in MATLAB
The cryptographic technique, which is known as a Hill cipher, involves arithmetic in a finite field. Almost all modern computers use the ASCII character set to store basic text. ASCII stands for American Standard Code for Information Interchange. The character set uses 7 of the 8 bits in a byte to encode 128 characters. The first 32 characters are nonprinting control characters, such as tab, backspace, and end-of-line. The 128th character is another nonprinting character that corresponds to the Delete key on your keyboard. In between these control characters are 95 printable characters, including a space, 10 digits, 26 lowercase letters, 26 uppercase letters, and 32 punctuation marks. MATLAB can easily display all the printable characters in the order determined by their ASCII encoding. Start with x = reshape (32:127, 32, 3)' This produces a 3-by-32 matrix. x =[ ]
The char function converts numbers to characters. The statement c = char(x) produces c= !"#$%&'()*+,-./0123456789:;<=>? @ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_ `abcdefghijklmnopqrstuvwxyz{|}~
Line1 Line2 Line3
Count all the characters that are displayed. Line1 has 31 characters, line2 has 32 characters and line3 has 31 characters. According to the size of matrix x there should be 96 characters in total but c has produced just 94 characters. Actually c has produced 96 characters but 2 characters are nonprinting characters. char(32) is (blank) char(127) is delete 54
Information Theory
Lab Session 08
The last printable character in c is the tilde, indicating that char(126) is the same as '~' The characters representing digits are in the first line of c. In fact, d = char(48:57) displays a 10-character string d = 0123456789 This string can be converted to the corresponding numerical values with double or real. The statement double(d) - '0' produces 0 1 2 3 4 5 6 7 8 9
Comparing the second and third lines of c, we see that the ASCII encoding of the lowercase letters is obtained by adding 32 to the ASCII encoding of the uppercase letters. Understanding this encoding allows us to use vector and matrix operations in MATLAB to manipulate text. The ASCII standard is often extended to make use of all eight bits in a byte, but the characters that are displayed depend on the computer and operating system you are using, the font you have chosen, and even the country you live in. Try char(reshape(160:255,32,3)') Our encryption technique involves modular arithmetic. All the quantities involved are integers and the result of any arithmetic operation is reduced by taking the remainder or modulus with respect to a prime number p. The functions rem(x,y) and mod(x,y) both compute the remainder if x is divided by y. They produce the same result if x and y have the same sign; the result also has that sign. But if x and y have opposite signs, then rem(x,y) has the same sign as x, while mod(x,y) has the same sign as y. Here is a table: x = [37 -37 37 -37]'; y = [10 10 -10 -10]'; r = [ x y rem(x,y) mod(x,y)]
produces
We have chosen to encrypt text that uses the entire ASCII character set, not just the letters. There are 95 such characters (including space). The next larger prime number is p = 97, so we represent the p characters by the integers 0: p-1 and do arithmetic mod p. The characters are encoded two at a time. Each pair of characters is represented by a 2-vector, x. For example, suppose the text contains the pair of letters 'TV'. The ASCII values for this pair of letters are 84 and 86. Subtracting 32 to make the representation start at 0 produces the column vector 55
Information Theory
Lab Session 08
x=(
The encryption is done with a 2-by-2 matrix-vector multiplication over the integers mod p. The symbol is used to indicate that two integers have the same remainder, modulo the specified prime: y Ax, mod p, where A is the matrix ( For our example, the product Ax is ( If this is reduced mod p, the result is ( ) ) )
Converting this back to characters by adding 32 produces 1U. Now comes the interesting part. Over the integers modulo p, the matrix A is its own inverse. If y then x Ay, mod p. In other words, in arithmetic mod p, A2 is the identity matrix. You can check this with MATLAB.
p = 97; A = [71 2; 2 26] I = mod(A^2,p) produces
Ax, mod p,
This means that the encryption process is its own inverse. The same function can be used to both encrypt and decrypt a message. The M-file crypto.m begins with a preamble. function y = crypto(x) % CRYPTO Cryptography example. % y = crypto(x) converts an ASCII text string into another % coded string. The function is its own inverse, so % crypto(crypto(x)) gives x back. % See also: ENCRYPT. A comment precedes the statement that assigns the prime p. % Use a two-character Hill cipher with arithmetic % modulo 97, a prime. p = 97; 56
Information Theory
Lab Session 08
Choose two characters above ASCII 128 to expand the size of the character set from 95 to 97. c1 = char(169); c2 = char(174); x(x==c1) = 127; x(x==c2) = 128; The conversion from characters to numerical values is done by x = mod(real(x-32),p); Prepare for the matrix-vector product by forming a matrix with two rows and lots of columns. n = 2*floor(length(x)/2); X = reshape(x(1:n),2,n/2); All this preparation has been so that we can do the actual finite field arithmetic quickly and easily. % Encode with matrix multiplication modulo p. A = [71 2; 2 26]; Y = mod(A*X,p); Reshape into a single row. y = reshape(Y,1,n); If length(x) is odd, encode the last character if length(x) > n y(n+1) = mod((p-1)*x(n+1),p); end Finally, convert the numbers back to characters. y = char(y+32); y(y==127) = c1; y(y==128) = c2; Let's follow the computation of y = crypto('Hello world'). We begin with a character string. x = 'Hello world' This is converted to an integer vector. x = 40 69 76 76 79 0 87 79 82 76 68 length(x) is odd, so the reshaping temporarily ignores the last element X= A conventional matrix-vector multiplication A*X produces an intermediate matrix. 2978 5548 5609 6335 5974 1874 2128 158 2228 2140 Then the mod(.,p) operation produces Y= This is rearranged to a row vector. y = 68 31 19 91 80 61 30 94 57 6 Now the last element of x is encoded by itself and attached to the end of y. 57
Information Theory
Lab Session 08
y = 68 31 19 91 80 61 30 94 57 6 29 Finally, y is converted back to a character string to produce the encrypted result. y = 'd?3{p]>~Y&=' If we now compute crypto(y), we get back our original 'Hello world'. Exercises: 8.1) If x is the character string consisting of just two blanks, x = ' ', then crypto(x) is actually equal to x. Why does this happen? Are there any other two-character strings that crypto do not change? ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 8.2) Find another 2-by-2 integer matrix A for which mod(A*A,97) is the identity matrix. Replace the matrix in crypto.m with your matrix and verify that the function still works correctly. ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 58
Information Theory
Lab Session 08
8.3) The function crypto works with 97 characters instead of 95. It can produce output, and correctly handle input, that contains two characters with ASCII values greater than 127. What are these characters? Why are they necessary? What happens to other characters with ASCII values greater than 127? ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 8.4) Create a new crypto function that works with just 29 characters: the 26 lowercase letters, plus blank, period, and comma. You will need to find a 2-by-2 integer matrix A for which mod(A*A,29) is the identity matrix.
___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 59
Information Theory
Lab Session 09
Lab Session 09
OBJECT
Implementation of Vigenere cipher technique in cryptography
THEORY
This cipher produces encrypted cipher text from an input plaintext message using a key and a matrix of substitution alphabets. Recovery of the plaintext from the cipher text requires the key. By modern standards, this isn't a terribly secure cryptosystem, but it is easily implemented, and difficult enough that encrypted messages can't be casually deciphered. The Vigenere cipher encrypts its inputs by using the key and the plaintext as indices into a fixed lookup table: the Vigenere square. For ease of computation, the algorithm first maps letters to numbers: A=1, B=2... SPACE=27. As shown in the matrix below, each row in the square is derived from the row above by circularly shifting it one place to the left: 1 2 2 3 3 4 ... 27 1 3 4 5 ... 26 27 4 5 6 ... 27 1 5 6 7 ... 1 2 2 3 4 ... 25 26
To encrypt, replicate the letters in the key so the key and plaintext are the same length. Then, derive each cipher text letter by lookup in the Vigenere square: use the key letter as the row index and the plaintext letter as the column index. If the key K and the plaintext P are n letters long, form the cipher text result C by indexing into the Vigenere square V, as follows: C(n) = V(K(n), P(n)) Decryption simply reverses the processs, using the key letter to determine the row index and the ciphertext letter to determine the colum index, which is the plaintext letter. For example: P(n) = FIND(V(K(n,:)) == C(n))
Implementing the Vigenere Cipher in MATLAB

To implement the Vigenere cipher in MATLAB, I wrote three functions:

vigenere - Generates the Vigenere square. encrypt - Transforms a plaintext message into ciphertext.
The vigenere function uses three additional MATLAB functions: circshift - Creates each row of the Vigenere square. arrayfun - Iterates over the row indices, calling circshift on each; returns a cell array of the rows. 60
Information Theory
Lab Session 09
reshape - Builds a matrix from the cell array. function v = vigenere % VIGENERE Return a Vigenere square in a 27x27 matrix. % Square 27 on a side count = 27; % Map letters to numbers, A=1, B=2, ... SPACE=27 alpha = 1:count; % Create a matrix with 27 shifted substitution alphabets % 1 2 3 4 5 ... 26 27 % 2 3 4 5 6 ... 27 1 % 3 4 5 6 7 ... 1 2 % etc. v = arrayfun(@(n) circshift(alpha, [0, -n]), 0:count-1,... 'UniformOutput', false); v = reshape([v{:}], count, count); encrypt creates a Vigenere square and transforms the key and plaintext from strings into vectors of numbers. It encrypts the plaintext and then translates the encrypted message from a vector of numbers into a character string. encrypt calls arrayfun to perform the encryption lookup instead of using a for-loop. arrayfun is faster and it won't ever get the length of the plaintext wrong, as a hand-written for-loop might. function ciphertext = encrypt(plaintext, key) v = vigenere; % Squeeze out everything except letters and the space character exclude = regexp(plaintext, '[^a-zA-Z ]'); plaintext(exclude) = []; % Make the key and the plaintext lower case, and convert to % numeric values. key = lower(key) - double('a') + 1; key(key < 0) = 27; plaintext = lower(plaintext) - double('a') + 1; plaintext(plaintext < 0) = 27; % Replicate the key so that it is as long as the plaintext. keyIndex = mod(0:(numel(plaintext)-1), numel(key))+1; k = key(keyIndex); % Encrypt: C(n) = V(k(n), plaintext(n)) ciphertext = arrayfun(@(m,n) v(m,n), k, plaintext) - 1; ciphertext(ciphertext == 26) = double(' ') - double('a'); ciphertext = upper(char(ciphertext + double('a'))); decrypt reverses the encryption process by searching for the ciphertext letter in the key letter row. The column index of the ciphertext letter maps back (1=A, 2=B, ..., 27=SPACE) to the 61
Information Theory
Lab Session 09
plaintext letter. Like encrypt, decrypt loops over the cipertext using arrayfun instead of a forloop. function plaintext = decrypt(ciphertext,key) v = vigenere; % Convert the key and the ciphertext to one-based numeric values; % A=1, B=2, etc. SPACE=27 key = lower(key) - double('a') + 1; key(key < 0) = 27; ciphertext = lower(ciphertext) - double('a') + 1; ciphertext(ciphertext < 0) = 27; % Replicate the key so that it is as long as the ciphertext. keyIndex = mod(0:(numel(ciphertext)-1), numel(key))+1; k = key(keyIndex); % Decrypt. Each letter of the key determines a row in the Vigenere % square. In that row, find the column index of the corresponding % ciphertext letter. Convert the index back to a letter to determine % the decrypted character (1=A, 2=B, etc.). plaintext = arrayfun(@(m,n) find(v(m,:) == n), k, ciphertext) 1; plaintext(plaintext == 26) = double(' ') - double('a'); plaintext = upper(char(plaintext + double('a'))); Most of the complexity in both encrypt and decrypt stems from extending the algorithm to allow spaces in the plaintext and the key. Without spaces, reading decrypted messages is difficult and error-prone. Look how easy it is to implement a Caesar cipher with shift 5 in MATLAB: >> myword='supercalifragilisticexpialidocious' myword = supercalifragilisticexpialidocious >> secretword=setstr(rem(myword+5-97,26)+97) secretword = xzujwhfqnkwflnqnxynhjcunfqnithntzx >> undecodedword=setstr(rem(secretword-5-97+26,26)+97) undecodedword = supercalifragilisticexpialidocious Explanation: MATLAB automatically converts the letters in a string to their ASCII decimal equivalents if the string is used in a numeric context. Since the letters "a" to "z" are in ASCII positions 97 through 122, subtracting 97 translates to the range 0 to 25. Adding 5 applies the shift, and taking the remainder upon division by 26 via the function rem reduces modulo26. Adding 97 translate back to the range 97 to 122, and applying the setstr function converts back to an alphabetic string. Reversing the operations decodes the message.
62
Information Theory
Lab Session 10
Lab Session 10
OBJECT
Learning basic model designs with Simulink
To help students become familiar with the Simulink environment.
THEORY
Simulink is a program for simulating signals and dynamic systems. As an extension of MATLAB, Simulink adds many features specific to the simulation of dynamic systems while retaining all of MATLABs general purpose functionality. Simulink has two phases of use: model definition and model analysis. A typical session starts by either defining a new model or by recalling a previously defined model, and then proceeds to analyze that model. In order to facilitate the model definition, Simulink has a large class of windows called block diagram windows. In these windows, models are created and edited principally using mouse-driven commands. An important part of mastering Simulink is to become familiar with manipulations of various model components in these windows. After you define a model, you can analyze it either by choosing options from the Simulink menus or by entering commands in the MATLAB command window. The progress of an ongoing simulation can be viewed while it is running, and the final results can be made available in the MATLAB workspace when the simulation is complete.
Prelab
Students are required to thoroughly go through Appendix B before attending this lab session Examples: 1. Product: Multiply and divide scalars and non-scalars or multiply and invert matrices Library Math Operations Description
Default Product Block Use By default, the Product block outputs the result of multiplying two inputs: two scalars, a scalar and a non-scalar, or two non-scalars that have the same dimensions. The default parameter values that specify this behavior are:
Multiplication: Element-wise (.*) Number of inputs: 2
Setting non-default values for either of these parameters can change a Product block to function as a Divide block or a Product of Elements block.
63
Information Theory
Lab Session 10
The following table shows the output of the Product block for example inputs using default block parameter values. Inputs and Behavior Scalar X Scalar Output the product of the two inputs. Scalar X Non-scalar Output a non-scalar having the same dimensions as the input non-scalar. Each element of the output nonscalar is the product of the input scalar and the corresponding element of the input non-scalar. Non-scalar X Non-scalar Output a non-scalar having the same dimensions as the inputs. Each element of the output is the product of corresponding elements of the inputs. 2. Logical operations: Perform specified logical operation on input Library Logic and Bit Operations Description Example
The Logical Operator block performs the specified logical operation on its inputs. An input value is TRUE (1) if it is nonzero and FALSE (0) if it is zero. You select the Boolean operation connecting the inputs with the Operator parameter list. If you select rectangular as the Icon shape property, the block updates to display the name of the selected operator. The supported operations are given below. Operation AND OR NAND NOR XOR NXOR NOT Description TRUE if all inputs are TRUE TRUE if at least one input is TRUE TRUE if at least one input is FALSE TRUE when no inputs are TRUE TRUE if an odd number of inputs are TRUE TRUE if an even number of inputs are TRUE TRUE if the input is FALSE
If you select distinctive as the Icon shape, the block's appearance indicates its function. Simulink software displays a distinctive shape for the selected operator, conforming to the IEEE Standard Graphic Symbols for Logic Functions: 64
Information Theory
Lab Session 10
The number of input ports is specified with the Number of input ports parameter. The output type is specified with the Output data type parameter. An output value is 1 if TRUE and 0 if FALSE. Note: The output data type should represent zero exactly. Data types that satisfy this condition include signed and unsigned integers, and any floating-point data type. The size of the output depends on input vector size and the selected operator: If the block has more than one input, any non-scalar inputs must have the same dimensions. For example, if any input is a 2-by-2 array, all other non-scalar inputs must also be 2-by-2 arrays. Scalar inputs are expanded to have the same dimensions as the non-scalar inputs. If the block has more than one input, the output has the same dimensions as the inputs (after scalar expansion) and each output element is the result of applying the specified logical operation to the corresponding input elements. For example, if the specified operation is AND and the inputs are 2-by-2 arrays, the output is a 2-by-2 array whose top left element is the result of applying AND to the top left elements of the inputs, etc.

For a single vector input, the block applies the operation (except the NOT operator) to all elements of the vector. The output is always a scalar. The NOT operator accepts only one input, which can be a scalar or a vector. If the input is a vector, the output is a vector of the same size containing the logical complements of the input vector elements.
When configured as a multi-input XOR gate, this block performs an addition- modulo-two operation as mandated by the IEEE Standard for Logic Elements. Exercises: 10.1) Design a model that compares (>, <, >=, <=, = = ) the two positive real numbers and produces the output either TRUE (1) or FALSE (0) 10.2) Design a model that: 10.2.1) Identifies the minimum absolute value from the list of numbers available. 10.2.2) Identifies the maximum absolute value from the list of numbers available. 10.3) Design a model that performs the summation of any four elements (say 20,10,30,45) 10.4) Design a model that performs the following operation: -20+10-30+45. 10.5) Design a model that sorts (ascending and descending both) at least 10 random numbers. 10.6) Design a model that concatenates three column vectors with three elements in each column.
65
Information Theory
Lab Session 10
Attach printout of simulated models here.
66
Information Theory
Lab Session 11
Lab Session 11
OBJECT
Encoding the data bits using a Binary Cyclic block encoder in Simulink
THEORY
Cyclic codes are a subclass of linear block codes with a cyclic structure that leads to more practical implementation. Block codes used in forward error correction systems are almost always cyclic codes. In cyclic coding, data is sent with a checksum. When arrives, checksum is recalculated. It should match the one that was sent. Refer lab session #5 for further description. SIMULINK FUNCTIONS 1. Constant Generate constant value Library Sources Description
The Constant block generates a real or complex constant value. The block generates scalar (one-element array), vector (1-D array), or matrix (2-D array) output, depending on the dimensionality of the Constant value parameter and the setting of the Interpret vector parameters as 1-D parameter. Also, the block can generate either a sample-based or framebased signal, depending on the setting of the Sampling mode parameter. The output of the block has the same dimensions and elements as the Constant value parameter. If you specify a vector for this parameter, and you want the block to interpret it as a vector (i.e., a 1-D array), select the Interpret vector parameters as 1-D parameter; otherwise, the block treats the Constant value parameter as a matrix (i.e., a 2-D array). Data Type Support By default, the Constant block outputs a signal whose data type and complexity are the same as that of the block's Constant value parameter. However, you can specify the output to be any data type supported by Simulink software, including fixed-point and enumerated data types. 2. Binary Cyclic Encoder Create systematic cyclic code from binary vector data Library Block sublibrary of Channel Coding
67
Information Theory
Lab Session 11
Description
The Binary Cyclic Encoder block creates a systematic cyclic code with message length K and codeword length N. The number N must have the form 2M-1, where M is an integer greater than or equal to 3. The input must contain exactly K elements. If it is frame-based, then it must be a column vector. The output is a vector of length N. You can determine the systematic cyclic coding scheme in one of two ways: To create an [N, K] code, enter N and K as the first and second dialog parameters, respectively. The block computes an appropriate generator polynomial, namely, cyclpoly(N,K,'min'). To create a code with codeword length N and a particular degree-(N-K) binary generator polynomial, enter N as the first parameter and a binary vector as the second parameter. The vector represents the generator polynomial by listing its coefficients in order of ascending exponents. You can create cyclic generator polynomials using the Communications Toolbox cyclpoly function. This block supports double and boolean data types. Dialog Box
Codeword length N The codeword length, which is also the output vector length. Message length K, or generator polynomial Either the message length, which is also the input vector length; or a binary vector that represents the generator polynomial for the code. 3. Display Show value of input Library Sinks
68
Information Theory
Lab Session 11
Description
The Display block shows the value of its input on its icon. You control the display format using the Format parameter:

short displays a 5-digit scaled value with fixed decimal point long displays a 15-digit scaled value with fixed decimal point short_e displays a 5-digit value with a floating decimal point long_e displays a 16-digit value with a floating decimal point bank displays a value in fixed dollars and cents format (but with no $ or commas) hex (Stored Integer) displays the stored integer value of a fixed-point input in hexadecimal format binary (Stored Integer) displays the stored integer value of a fixed-point input in binary format decimal (Stored Integer) displays the stored integer value of a fixed-point input in decimal format octal (Stored Integer) displays the stored integer value of a fixed-point input in octal format
If the block input is an array, you can resize the block to show more than just the first element. You can resize the block vertically or horizontally; the block adds display fields in the appropriate direction. A black triangle indicates that the block is not displaying all input array elements. For example, the following figure shows a model that passes a vector (1-D array) to a Display block. The black triangle on the Display block indicates more data to be displayed.
The following figure shows the resized block displaying both input elements.
Data Type Support The Display block accepts and outputs real or complex signals of any data type supported by Simulink software, including fixed-point and enumerated data types.
69
Information Theory
Lab Session 11
Parameters and Dialog Box
Format Specify the format of the data displayed, as discussed in description above. The default is short. Decimation Specify how often to display data. The default value, 1, displays every input point. Floating display If selected, the block's input port disappears, which enables the block to be used as a floating Display block.
Exercises: 11.1 Encode all possible data words for (7,4) cyclic code, given g(x) = (1 + x + x3) 11.2 Encode all possible data words for (7,4) cyclic code, given g(x) = (1 + x2 + x3)
70
Information Theory
Lab Session 11
71
Information Theory
Lab Session 12
Lab Session 12
OBJECT
Decoding the code words using a Binary Cyclic block decoder in Simulink
THEORY
Decoding of Cyclic codes: Refer lab session#6 for description. SIMULINK FUNCTIONS 1. Binary Cyclic Decoder Decode systematic cyclic code to recover binary vector data Library Block sublibrary of Channel Coding Description
The Binary Cyclic Decoder block recovers a message vector from a codeword vector of a binary systematic cyclic code. For proper decoding, the parameter values in this block should match those in the corresponding Binary Cyclic Encoder block. If the cyclic code has message length K and codeword length N, then N must have the form 2M-1 for some integer M greater than or equal to 3. The input must contain exactly N elements. If it is frame-based, then it must be a column vector. The output is a vector of length K. You can determine the systematic cyclic coding scheme in one of two ways: To create an [N,K] code, enter N and K as the first and second dialog parameters, respectively. The block computes an appropriate generator polynomial, namely, cyclpoly(N,K,'min'). To create a code with codeword length N and a particular degree-(N-K) binary generator polynomial, enter N as the first parameter and a binary vector as the second parameter. The vector represents the generator polynomial by listing its coefficients in order of ascending exponents. You can create cyclic generator polynomials using the Communications Toolbox cyclpoly function. This block supports double and boolean data types.
72
Information Theory
Lab Session 12
Dialog Box
Codeword length N The codeword length N, which is also the input vector length. Message length K, or generator polynomial Either the message length, which is also the output vector length; or a binary vector that represents the generator polynomial for the code. 2. Display Refer lab session #11 Exercise: 12.1 Design a model that decodes all the encoded codewords of Q 11.1 from lab session #11
73
Information Theory
Lab Session 12
74
Information Theory
Lab Session 13
Lab Session 13
OBJECT
Encoding the data bits using a Binary Linear block encoder in Simulink
THEORY
In mathematics and information theory, linear code is an important type of block code used in error correction and detection schemes. Linear codes allow for more efficient encoding and decoding algorithms than other codes. Linear codes are applied in methods of transmitting symbols (e.g., bits) on communications channel so that, if errors occur in the communication, some errors can be detected by the recipient of the message block. A linear code of length n transmits blocks containing n symbols. For example, the (7, 4) Hamming code is a binary linear code which represents 4-bit values with 7 bits. In this way, the recipient can detect errors as severe as 2 bits per block. In linear block codes, a block of k information bits is followed by a group of r check bits derived from the information bits and at the receiver the check bits are used to verify the information bits which are preceding the check bits. For further information refer lab session# 3 SIMULINK FUNCTIONS: 1. Binary Linear Encoder Create linear block code from binary vector data Library Block sublibrary of Channel Coding Description
The Binary Linear Encoder block creates a binary linear block code using a generator matrix that you specify. If K is the message length of the code, then the Generator matrix parameter must have K rows. If N is the codeword length of the code, then Generator matrix must have N columns. The input must contain exactly K elements. If it is frame-based, then it must be a column vector. The output is a vector of length N. This block supports double and boolean data types.
75
Information Theory
Lab Session 13
Dialog Box
Generator matrix A K-by-N matrix, where K is the message length and N is the codeword length. 2. Display Refer lab session #11 3. Constant Refer lab session #11 Exercises: 13.1) Given (6,3) linear block code generated by the predefined matrix [0 1 1; 1 0 1; 1 1 0]. 13.1.1) Encode the messages [1 0 0] and [1 1 0] manually and verify through Simulink. 100 = _________ 110 = _________
___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________
76
Information Theory
Lab Session 13
13.1.2) Encode all the possible messages in Simulink and write the encoded words. Is this code a systematic code? Justify your answer. ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 13.2) Consider a systematic (6, 3) block code generated by the submatrix [1 1 0; 0 1 1; 1 0 1]. Design a model that encodes all possible data words.
77
Information Theory
Lab Session 13
78
Information Theory
Lab Session 14
Lab Session 14
OBJECT
Decoding the code words using a Binary Linear block decoder in Simulink
THEORY
Refer lab session# 4 for description. SIMULINK FUNCTIONS: In order to develop models in Simulink for decoding following functions are to be used. 1. Binary Linear Decoder Decode linear block code to recover binary vector data Library Block sublibrary of Channel Coding Description
The Binary Linear Decoder block recovers a binary message vector from a binary code word vector of a linear block code. The Generator matrix parameter is the generator matrix for the block code. For proper decoding, this should match the Generator matrix parameter in the corresponding Binary Linear Encoder block. If N is the code word length of the code, then Generator matrix must have N columns. If K is the message length of the code, then the Generator matrix parameter must have K rows. The input must contain exactly N elements. If it is frame-based, then it must be a column vector. The output is a vector of length K. The decoder tries to correct errors, using the Decoding table parameter. If Decoding table is the scalar 0, then the block defaults to the table produced by the Communications Toolbox function syndtable. Otherwise, Decoding table must be a 2N-K-by-N binary matrix. The rth row of this matrix is the correction vector for a received binary code word whose syndrome has decimal integer value r-1. The syndrome of a received code word is its product with the transpose of the parity-check matrix. This block supports double and boolean data types.
79
Information Theory
Lab Session 14
Dialog Box
Generator matrix Generator matrix for the code; same as in Binary Linear Encoder block. Decoding table Either a 2N-K-by-N matrix that lists correction vectors for each code word's syndrome; or the scalar 0, in which case the block defaults to the table corresponding to the Generator matrix parameter. 2. Display Refer lab session #11 3. Constant Refer lab session #11 Exercise: 14.1) Design a model that decodes all the code words that were encoded in Q 13.1.2 and tabulate them.
80
Information Theory
Lab Session 14
Attach printout of simulated model here.
81
Information Theory
Appendix A
Appendix A Introduction to Simulink

Simulink is a graphical extension to MATLAB for the modeling and simulation of systems. In Simulink, systems are drawn on screen as block diagrams. Many elements of block diagrams are available (such as transfer functions, summing junctions, etc.), as well as virtual input devices (such as function generators) and output devices (such as oscilloscopes). Simulink is integrated with MATLAB and data can be easily transferred between the programs. In this tutorial, we will introduce the basics of using Simulink to model and simulate a system. Simulink is supported on UNIX, Macintosh, and Windows environments, and it is included in the student version of MATLAB for personal computers. For more information on Simulink, contact the MathWorks. The idea behind these tutorials is that you can view them in one window while running Simulink in another window. Do not confuse the windows, icons, and menus in the tutorials for your actual Simulink windows. Most images in these tutorials are not live - they simply display what you should see in your own Simulink windows. All Simulink operations should be done in your Simulink windows. Starting Simulink Simulink is started from the MATLAB command prompt by entering the following command: >>simulink Alternatively, you can click on the "Simulink Library Browser" button at the top of the MATLAB command window. The Simulink Library Browser window should now appear on the screen. Most of the blocks needed for modeling basic systems can be found in the subfolders of the main "Simulink" folder (opened by clicking on the "+" in front of "Simulink"). 1. Basic Elements There are two major classes of elements in Simulink: blocks and lines. Blocks are used to generate, modify, combine, output, and display signals. Lines are used to transfer signals from one block to another. a. Blocks The subfolders underneath the "Simulink" folder indicate the general classes of blocks available for us to use:

Continuous: Linear, continuous-time system elements (integrators, transfer functions, statespace models, etc.) Discrete: Linear, discrete-time system elements (integrators, transfer functions, state-space models, etc.) Functions & Tables: User-defined functions and tables for interpolating function values Math: Mathematical operators (sum, gain, dot product, etc.) Nonlinear: Nonlinear operators (coulomb/viscous friction, switches, relays, etc.)
82
Information Theory

Appendix A
Signals & Systems: Blocks for controlling/monitoring signal(s) and for creating subsystems Sinks: Used to output or display signals (displays, scopes, graphs, etc.) Sources: Used to generate various signals (step, ramp, sinusoidal, etc.)
Blocks have zero to several input terminals and zero to several output terminals. Unused input terminals are indicated by a small open triangle. Unused output terminals are indicated by a small triangular point. The block shown below has an unused input terminal on the left and an unused output terminal on the right.
b. Lines Lines transmit signals in the direction indicated by the arrow. Lines must always transmit signals from the output terminal of one block to the input terminal of another block. One exception to this is that a line can tap off of another line. This sends the original signal to each of two (or more) destination blocks, as shown below:
Lines can never inject a signal into another line; lines must be combined through the use of a block such as a summing junction. A signal can be either a scalar signal or a vector signal. For Single-Input, Single-Output systems, scalar signals are generally used. For Multi-Input, Multi-Output systems, vector signals are often used, consisting of two or more scalar signals. The lines used to transmit scalar and vector signals are identical. The type of signal carried by a line is determined by the blocks on either end of the line. I. Building a System
To demonstrate how a system is represented using Simulink, we will build the block diagram for a simple model consisting of a sinusoidal input multiplied by a constant gain, which is shown below:
83
Information Theory
Appendix A
This model will consist of three blocks: Sine Wave, Gain, and Scope. The Sine Wave is a Source Block from which a sinusoidal input signal originates. This signal is transferred through a line in the direction indicated by the arrow to the Gain Math Block. The Gain block modifies its input signal (multiplies it by a constant value) and outputs a new signal through a line to the Scope block. The Scope is a Sink Block used to display a signal (much like an oscilloscope). We begin building our system by bringing up a new model window in which to create the block diagram. This is done by clicking on the "New Model" button in the toolbar of the Simulink Library Browser (looks like a blank page). Building the system model is then accomplished through a series of steps: 1. The necessary blocks are gathered from the Library Browser and placed in the model window. 2. The parameters of the blocks are then modified to correspond with the system we are modeling. 3. Finally, the blocks are connected with lines to complete the model. Each of these steps will be explained in detail using our example system. Once a system is built, simulations are run to analyze its behavior. II. Gathering Blocks
Each of the blocks we will use in our example model will be taken from the Simulink Library Browser. To place the Sine Wave block into the model window, follow these steps: 1. Click on the "+" in front of "Sources" (this is a subfolder beneath the "Simulink" folder) to display the various source blocks available for us to use. 2. Scroll down until you see the "Sine Wave" block. Clicking on this will display a short explanation of what that block does in the space below the folder list:
3. To insert a Sine Wave block into your model window, click on it in the Library Browser and drag the block into your workspace.
84
Information Theory
Appendix A
The same method can be used to place the Gain and Scope blocks in the model window. The "Gain" block can be found in the "Math" subfolder and the "Scope" block is located in the "Sink" subfolder. Arrange the three blocks in the workspace (done by selecting and dragging an individual block to a new location) so that they look similar to the following:
III.
Modifying the Blocks
Simulink allows us to modify the blocks in our model so that they accurately reflect the characteristics of the system we are analyzing. For example, we can modify the Sine Wave block by double-clicking on it. Doing so will cause the following window to appear:
This window allows us to adjust the amplitude, frequency, and phase shift of the sinusoidal input. The "Sample time" value indicates the time interval between successive readings of the signal. Setting this value to 0 indicates the signal is sampled continuously. Let us assume that our system's sinusoidal input has:

Amplitude = 2 Frequency = pi Phase = pi/2
85
Information Theory
Appendix A
Enter these values into the appropriate fields (leave the "Sample time" set to 0) and click "OK" to accept them and exit the window. Note that the frequency and phase for our system contain 'pi' (3.1415...). These values can be entered into Simulink just as they have been shown. Next, we modify the Gain block by double-clicking on it in the model window. The following window will then appear:
Note that Simulink gives a brief explanation of the block's function in the top portion of this window. In the case of the Gain block, the signal input to the block (u) is multiplied by a constant (k) to create the block's output signal (y). Changing the "Gain" parameter in this window changes the value of k. For our system, we will let k = 5. Enter this value in the "Gain" field, and click "OK" to close the window. The Scope block simply plots its input signal as a function of time, and thus there are no system parameters that we can change for it. We will look at the Scope block in more detail after we have run our simulation. IV. Connecting the Blocks For a block diagram to accurately reflect the system we are modeling, the Simulink blocks must be properly connected. In our example system, the signal output by the Sine Wave block is transmitted to the Gain block. The Gain block amplifies this signal and outputs its new value to the Scope block, which graphs the signal as a function of time. Thus, we need to draw lines from the output of the Sine Wave block to the input of the Gain block, and from the output of the Gain block to the input of the Scope block. Lines are drawn by dragging the mouse from where a signal starts (output terminal of a block) to where it ends (input terminal of another block). When drawing lines, it is important to make sure that the signal reaches each of its intended terminals. Simulink will turn the mouse pointer into a crosshair when it is close enough to an output terminal to begin drawing a line, and the pointer will change into a double crosshair when it is close enough to snap to an input terminal. A signal is properly connected if its arrowhead is filled in. If the arrowhead is open, it means the signal is not connected to both blocks. To fix an open signal, you can treat the open arrowhead as an output terminal and continue drawing the line to an input terminal in the same manner as explained before.
Properly Connected Signal
Open Signal
86
Information Theory
Appendix A
When drawing lines, you do not need to worry about the path you follow. The lines will route themselves automatically. Once blocks are connected, they can be repositioned for a neater appearance. This is done by clicking on and dragging each block to its desired location (signals will stay properly connected and will re-route themselves). After drawing in the lines and repositioning the blocks, the example system model should look like:
In some models, it will be necessary to branch a signal so that it is transmitted to two or more different input terminals. This is done by first placing the mouse cursor at the location where the signal is to branch. Then, using either the CTRL key in conjunction with the left mouse button or just the right mouse button, drag the new line to its intended destination. This method was used to construct the branch in the Sine Wave output signal shown below:
The routing of lines and the location of branches can be changed by dragging them to their desired new position. To delete an incorrectly drawn line, simply click on it to select it, and hit the DELETE key. V. Running Simulations
Now that our model has been constructed, we are ready to simulate the system. To do this, go to the Simulation menu and click on Start, or just click on the "Start/Pause Simulation" button in the model window toolbar (looks like the "Play" button on a VCR). Because our example is a relatively simple model, its simulation runs almost instantaneously. With more complicated systems, however, you will
87
Information Theory
Appendix A
be able to see the progress of the simulation by observing its running time in the lower box of the model window. Double-click the Scope block to view the output of the Gain block for the simulation as a function of time. Once the Scope window appears, click the "Autoscale" button in its toolbar (looks like a pair of binoculars) to scale the graph to better fit the window. Having done this, you should see the following:
Note that the output of our system appears as a cosine curve with a period of 2 seconds and amplitude equal to 10. Does this result agree with the system parameters we set? Its amplitude makes sense when we consider that the amplitude of the input signal was 2 and the constant gain of the system was 5 (2 x 5 = 10). The output's period should be the same as that of the input signal, and this value is a function of the frequency we entered for the Sine Wave block (which was set equal to pi). Finally, the output's shape as a cosine curve is due to the phase value of pi/2 we set for the input (sine and cosine graphs differ by a phase shift of pi/2). What if we were to modify the gain of the system to be 0.5? How would this affect the output of the Gain block as observed by the Scope? Make this change by double-clicking on the Gain block and changing the gain value to 0.5. Then, re-run the simulation and view the Scope (the Scope graph will not change unless the simulation is re-run, even though the gain value has been modified). The Scope graph should now look like the following:
88
Information Theory
Appendix A
Note that the only difference between this output and the one from our original system is the amplitude of the cosine curve. In the second case, the amplitude is equal to 1, or 1/10th of 10, which is a result of the gain value being 1/10th as large as it originally was.
TABLE OF SIMULINK BLOCKS
Clock Constant From File From Workspace Signal Generator Sine Wave Step Fcn White Noise
SOURCES Provide and display Inject a constant value Read data from a file Read data from a matrix Generate various waveforms Generate a sine wave Generate a step function Generate random noise
Scope To File To Workspace
SINKS Display signals during a simulation Write data to a file Write data to a matrix
Discrete State-Space Discrete Transfer Fcn Discrete Zero-Pole Filter Unit Delay
DISCRETE Discrete state-space system Discrete transfer function Discrete system as zeros, poles and gain Implement IIR and FIR filters Delay signal for one sample period
Derivative Gain Integrator State-Space Sum Transfer Fcn Zero-Pole
LINEAR Output the time derivative of the input Multiply input by a constant Integrate a signal Implement a linear state-space system Sum inputs together Implement a linear transfer function Specify a linear system as zeros, poles and gain
Demux Inport Mux Outport
CONNECTIONS Separate vector signal into scalar signals Provide a link to an external input Group scalar lines into a vector line Provide a link to an external output
89
Information Theory
Appendix A
Abs Backslash Dead Zone Fcn Look Up Table MATLAB Fcn Product Rate Limiter Relay Saturation S-function Switch Transport Delay
NONLINEAR Output the absolute value of the input Model hysteresis Provide a region of zero output General purpose function of the input Perform piecewise linear mapping Apply MATLAB function to the input Multiply inputs together Limit the rate of change of a signal Switch output between two values Limit execution of the signal Make an S-function into a block Switch between two inputs Delay the input by a given amount of time
Sources Display Devices Filters Controllers Linear Nonlinear Discrete Logic System ID Robust Control Demos MuTools Demos Block Library Demos
EXTRAS Additional souces Additional display deices Additional filters PID, anti-windup and optimal controllers Additional linear blocks Additional nonlinear blocks Additional discrete blocks Logic blocks (AND, OR, NAND, etc.) System identification blocks Robust control demos Mu analysis of control Systems Most commonly used blocks Simulink demos
90

It 2012

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

It 2012

Uploaded by

Copyright:

Available Formats

Practical Workbook INFORMATION THEORY

Name Year Batch Roll No

: ________________________ : ________________________ : ________________________ : ________________________

4th edition: 2012

Lab Session No.

Application of MATLAB to coding theory Learning basic commands of MATLAB

using the command format long? _____________________________________________________________________ _____________________________________________________________________

7.5 -3.5 0.5 -4.5

-5.5 1.5 -2.5 6.5 0 0

-6.5 2.5 -1.5 5.5 0

4.5 -0.5 3.5 -7.5

Consider two possible messages: m1 = [0 1 1 0] C = DG or MG C = [

First and last elements of the matrix are shown above

Suppose error pattern e = [0 0 0 0 0 1 0 0] is added to both code words. r1 = [0 1 1 0 0 0 1 0] r2 = [0 1 0 0 1 1 1 1]

3. Receiver end: Receive T(x). Divide by G(x), should have remainder 0.

Code = encode (msg,n,k,'cyclic') noisycode = code % No error

% Encode the message.

% or x3 that is highest power of g(x) % multiplying x3 * m(x) % dividing [x3*m(x)] by g(x)

M=gf(m) G=gf(g) h=[1 0 0 0] H=gf(h) C=conv(H,M) [q,r]=deconv(C,G)

% or x3 that is highest power of g(x) % multiplying x3 * m(x) % dividing [x3*m(x)] by g(x)

Huffman Coding in MATLAB

Line1 Line2 Line3

Implementing the Vigenere Cipher in MATLAB

Attach printout of simulated models here.

Parameters and Dialog Box

Attach printout of simulated models here.

Attach printout of simulated models here.

Attach printout of simulated models here.

Attach printout of simulated model here.

Appendix A Introduction to Simulink

Modifying the Blocks

Amplitude = 2 Frequency = pi Phase = pi/2

Properly Connected Signal

TABLE OF SIMULINK BLOCKS

Scope To File To Workspace

Derivative Gain Integrator State-Space Sum Transfer Fcn Zero-Pole

Demux Inport Mux Outport

You might also like

: : : :

using the command format long? _ _