Lecture notes Telecommunications Engineering I by Jorma Kekalainen

TELECOMMUNICATIONS
ENGINEERING I
Jorma Kekalainen

595

TELECOMMUNICATIONS
ENGINEERING I
Basic probability concepts

596

Probability concepts and stochastic processes 1

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Need for probabilistic tools

597

History of probability theory

598

Probability concepts and stochastic processes 2

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Basic definitions

599

Examples

600

Probability concepts and stochastic processes 3

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Frequency definition of probability

601

Some set theory concepts

602

Probability concepts and stochastic processes 4

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Axiomatic definition of
probability

603

Axiomatic definition of
probability

604

Probability concepts and stochastic processes 5

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Example: Venn diagrams

605

Properties of probability

606

Probability concepts and stochastic processes 6

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Properties of probability

607

Joint and marginal probability

608

Probability concepts and stochastic processes 7

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Conditional probability

609

Bayes rule

610

Probability concepts and stochastic processes 8

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Independence

611

Law of total probability

612

Probability concepts and stochastic processes 9

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Example

613

Example: Binary symmetric

channel

614

Probability concepts and stochastic processes 10

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

BSC example: Answers

615

Example: Probabilities

616

Probability concepts and stochastic processes 11

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Numerical BSC-example

Note: There is no need to calculate P(1r) for decoding purpose, since it appears in617
both
P(1s|1r) and P(0s|1r), and does not change their relationship

Telecommunications
Engineering I
Random variables

618

Probability concepts and stochastic processes 12

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Random variable (r.v.)

619
Note: Random variable is actually a function, not a variable.

Random variables

620

Probability concepts and stochastic processes 13

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Sample space vs. random variable

vs. probability

621

Cumulative distribution function

(cdf, CDF)

622
Note: CDF is also called the probability distribution function.

Probability concepts and stochastic processes 14

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Properties of cdf

623

Example

624

Probability concepts and stochastic processes 15

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Example

625

Probability density function (pdf)

626

Probability concepts and stochastic processes 16

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Example

627

Probability mass function (pmf)

628

Probability concepts and stochastic processes 17

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Example

629

Several random variables

630

Probability concepts and stochastic processes 18

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Example

631

Example

632

Probability concepts and stochastic processes 19

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Marginal cdf and pdf

633

Conditional pdf

634

Probability concepts and stochastic processes 20

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Independent random variables

635

Independent random variables

636

Probability concepts and stochastic processes 21

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Statistical averages

637

Expectation

638

Probability concepts and stochastic processes 22

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Standard deviation

Standard deviation is a widely used measurement of

variability or diversity in statistics and probability
theory.
It shows how much variation or "dispersion" there is
from the average (mean, or expected value).
A low standard deviation indicates that the data
points tend to be very close to the mean, whereas high
standard deviation indicates that the data are spread
out over a large range of values.

639

Standard deviation

A plot of a normal distribution.

Each colored band has a width of one standard deviation.

640

Probability concepts and stochastic processes 23

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Variance

641

Variance

642

Probability concepts and stochastic processes 24

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Additional property of variance

643

Covariance

644
Note: Variance is a special case of the covariance when the two variables are identical.

Probability concepts and stochastic processes 25

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Covariance

645

Correlation coefficient

646

Probability concepts and stochastic processes 26

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Correlation coefficient

647

Correlation coefficient

648

Probability concepts and stochastic processes 27

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Telecommunications
Engineering I
Stochastic processes

649

Deterministic vs. random process

650

Probability concepts and stochastic processes 28

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Definition and notation

651

Random variable vs. random

process

652

Probability concepts and stochastic processes 29

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Random variable vs. random

process

653

Random process

654

Probability concepts and stochastic processes 30

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Random process examples

655

Probability distribution of a
random process

656

Probability concepts and stochastic processes 31

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Stationarity

657

Stationarity

658

Probability concepts and stochastic processes 32

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

2nd-order stationarity

659

Strict stationarity

660

Probability concepts and stochastic processes 33

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Strict stationarity

661

Recall: Covariance

662

Probability concepts and stochastic processes 34

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Autocorrelation

663

Correlation of random processes

Note: Steps to get E{X(t1)X*(t2)}

1: For each sample function X(t, sj), calculate X(t1, sj) X*(t2, sj). 664
2: Take average over all possible sample functions sj (see examples later).

Probability concepts and stochastic processes 35

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Weak or wide-sense stationarity

(WSS)

665

Properties of autocorrelation

666

Probability concepts and stochastic processes 36

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Properties of autocorrelation:
Proofs

667

Example

668

Probability concepts and stochastic processes 37

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Example

Note: We cannot write

because we should calculate X(t1, sj) X*(t2, sj) for each sample function before
averaging. For each sample function, the value of is the same at t1 and t2. 669
Otherwise there will be two random variables and 1 and 2.

Note: 670

Probability concepts and stochastic processes 38

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Example

671

Example

672

Probability concepts and stochastic processes 39

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Telecommunications
Engineering I
Power spectral density

673

Power spectral density

for deterministic signals

674

Probability concepts and stochastic processes 40

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Power spectral density

for deterministic signals

675

Power spectral density for

random processes

676

Probability concepts and stochastic processes 41

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

PSD for random processes

677

Power spectral density for

random processes

678
Note: PSD definition is not easy to use. In practice, we use autocorrelation to find psd.

Probability concepts and stochastic processes 42

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Example

(( ))

679
Note: This example shows how difficult it is to use the psd definition

Wiener-Khinchine Theorem

Note: In practice, we nearly always use autocorrelation to find psd, instead of the 680
psd
definition.

Probability concepts and stochastic processes 43

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Application of W-K Theorem

681
Note how much easier is to use W-K Theorem than the psd definition in calculation of
this kind example.

Application of W-K Theorem

682

Probability concepts and stochastic processes 44

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Properties of PSD

683

Proofs

684

Probability concepts and stochastic processes 45

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Symmetry of PSD of real WSS random processes

685

Telecommuincations
Engineering I
Linear systems and
random processes

686

Probability concepts and stochastic processes 46

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Recap: Independency and correlation

687
Note:

Cross-correlation

688

Probability concepts and stochastic processes 47

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Filtered random process

689

Jointly stationary I/O-processes

690
Note: Also

Probability concepts and stochastic processes 48

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Proofs

691

Proofs

692

Probability concepts and stochastic processes 49

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Proofs

693
Note: RXY(t1,t2)=RXY(t1-t2)=RXY()RZ() and RZ(0)=Zf(t) Z(t) is WSS.

Proofs

694

Probability concepts and stochastic processes 50

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

RYX()=RX() h()

695

Time domain summary

696

Probability concepts and stochastic processes 51

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Spectral functions

697

Spectral functions

698

Probability concepts and stochastic processes 52

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Spectral functions

699

Spectral functions

700

Probability concepts and stochastic processes 53

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Time domain Frequency domain

701

Example

702

Probability concepts and stochastic processes 54

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Example

703

Example

704

Probability concepts and stochastic processes 55

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Example

705

Telecommunications
Engineering I
Gaussian processes

706

Probability concepts and stochastic processes 56

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Gaussian PDF

707

Mean vector and covariance matrix

708

Probability concepts and stochastic processes 57

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Covariance matrix

709

Joint Gaussian PDF

710

Probability concepts and stochastic processes 58

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Special case

711
Note: Independent uncorrelated, but converse is not generally true.

Gaussian random processes

712

Probability concepts and stochastic processes 59

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Telecommunications
Engineering I
Noises

713

White noise
Noise: A random and unwanted" signal.

714

Probability concepts and stochastic processes 60

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

White noise

715

Gaussian noise

716

Probability concepts and stochastic processes 61

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Gaussian white noise

717

Example

718

Probability concepts and stochastic processes 62

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Solution

719

Example

720

Probability concepts and stochastic processes 63

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Solution

721

Telecommunications
Engineering I
Narrowband noise

722

Probability concepts and stochastic processes 64

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Recap: Complex envelope

723

Recap: Complex envelope

724

Probability concepts and stochastic processes 65

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Bandpass (narrowband)
random process

725

Narrowband noise

726

Probability concepts and stochastic processes 66

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Narrowband noise

727

Some properties of narrowband noise

728

Probability concepts and stochastic processes 67

Lecture notes Telecommunications Engineering I by Jorma Kekalainen

Some properties of narrowband noise

729

Example

730

Probability concepts and stochastic processes 68