Image Processing and Computer Vision

with MATLAB and SIMULINK

By Dr Jasmina Lazić
Application Engineering, MathWorks

© 2015 The MathWorks, Inc.

1
The Computer Vision Ecosystem
Simulink™
Control System
Embedded
Neural Network Toolbox™ Robotic Vision Toolbox™
Coder™ DSP System
Multi-variable SP Toolbox™
Computer Control Non-linear SP
Intelligence Robotics Signal Processing
Computer Vision Artificial
Signal Processing Toolbox™
System Toolbox™ Intelligence
Cognitive
Vision Computer Machine
Machine
Learning
Vision Vision Physics Optics SimScape™
SimMechanics™
Image
Statistics Toolbox™ Statistics Processing
Mathematics Imaging
Geometry
MATLAB™ Optimization Smart
Cameras
Image Acquisition
Neurobiology Toolbox™
Optimization
Toolbox™
Biological Vision
Image Processing
SimBiology™ Toolbox™
2
Applications: Image and Video Processing

 Medical imaging
 Surveillance
 Robotics
 Automotive safety
 Consumer electronics
 Geospatial computing
 Machine vision
 and more…

3
Agenda

• Introduction • Feature Detect, Extract & Match

• Image Acquisition • Object Detection and Tracking

• Explore-Enhance-Segment-Analyze • 3D Vision

• Image Registration • Embedded Applications

4
http://www.mathworks.co.uk/products/image-video-processing

5
Agenda

• Introduction
• Introduction • Feature Detect, Extract & Match

• Image Acquisition
• Image
 MATLAB live video with Microsoft Kinect • Object Detection and Tracking
Acquisition
Demo:
 Demo: Image Acquisition App

• Explore-Enhance-Segment-Analyze • 3D Vision

• Image Registration • Embedded Applications

6
Image Acquisition

 Acquire images and video directly into MATLAB and Simulink

 Configure device properties

7
Agenda

• Introduction
• Feature Detect, Extract & Match

• Image
Image Acquisition
Acquisition


Demo: Live video in MATLAB
Demo: Image Acquisition App
• Object Detection and Tracking
 Demo: Live video in SIMULINK
• Explore-Enhance-Segment-Analyze
 Demo: Tissue segmentation
• Explore-Enhance-Segment-Analyze
 Demo: Image analysis Apps • 3D Vision
 Demo: Digital display reader

• Image Registration • Embedded Applications

8
Standard Segmentation Workflow

9
Standard Segmentation Workflow

 Explore image

10
Standard Segmentation Workflow

 Explore image
 Enhance image

11
Standard Segmentation Workflow

 Explore image
 Enhance image
 Segment image

12
Standard Segmentation Workflow

 Explore image
 Enhance image
 Segment image
 Analyze image

13
Demos: Explore-Enhance-Segment-Analyze

14
Image Processing Apps

NEW!
NEW!

Image Viewer

Color Thresholder
Region Analyzer
15
Optical Character Recognition

 Support for English, Japanese + others

 Simple interface:

>> results = ocr(I)

16
Segmentation – other techniques
Watershed and Active Contours (NEW)

17
Agenda

• Introduction
• Feature Detect, Extract & Match

• Image Acquisition
• Object Detection and Tracking

• Explore-Enhance-Segment-Analyze
Explore-Enhance-Segment-Analyze
 Demo: Tissue segmentation
 Demo: Image analysis Apps • 3D Vision
 Demo: Digital display reader

• Image Registration
• Image
 Registration
Demo: Video Mosaicking (Simulink) • Embedded Applications

18
Image Registration

 2D and 3D
 Align multi-model images

19
Image Registration

 NEW: Non-rigid image

registration and warping

20
Demo: Video Mosaicking

21
Agenda

• Introduction • Feature Detect, Extract & Match

• Feature
 Detect,
Geometry Extract & Match
Estimation

• Image Acquisition
• Object Detection and Tracking

• Explore-Enhance-Segment-Analyze
• 3D Vision

• Image
Image Registration
Registration
 Demo: Video Mosaicking (Simulink) • Embedded Applications

22
Feature Detection, Extraction and Matching

Edge Corner Template SURF MSER

23
Point Tracker

 Simple interface:

points = detectXFeatures(I);
tracker = vision.PointTracker;
initialize(tracker, points.Location, frame);
while ~finished
frame = step(videoFileReader);
[points, validity] = step(tracker, frame);
end

24
Geometry Estimation

 RANSAC – Random Sampling and Consensus

 Gets geometry from a minimal sample set and checks it against all matches

25
Agenda

• Introduction • Feature
Feature Detect,
Detect,Extract
Extract&&Match
Match
 Geometry Estimation

• Image Acquisition • Object Detection and Tracking

• Object
 Detection
Demo: Pedestrianand Tracking
Tracking

• Explore-Enhance-Segment-Analyze
• 3D Vision

• Image Registration
• Embedded Applications

26
Object Detection and Tracking

27
Face Detection, People Detection and Foreground Detection

28
Training Image Labeler App

29
Demo: Tracking multiple objects with a Kalman Filter

30
Demo: Tracking multiple objects with a Kalman Filter

 Kalman Filter interface:

– Configure:
kalmanFilter = configureKalmanFilter( ...
'ConstantVelocity', centroid, ...
[200, 50], [100, 25], 100);

– Predict:
predictedCentroid = predict(kalmanFilter);

– Update:
c2 = correct(kalmanFilter, centroid);

31
Bag of Words for Category Classification

1. Create Bag of Features 3. Repeat for all classes

4. Use machine learning algorithm

to discriminate between classes
2. Encode Input Images

32
Agenda

• Introduction • Feature Detect, Extract & Match

• Image Acquisition • Object

Object Detection
Detectionand
andTracking
Tracking
 Demo: Multi-object Tracking

• Explore-Enhance-Segment-Analyze • 3D Vision
 Demo: Camera Calibrator App
• 3D

Vision
Demo: 3D Reconstruction
 Demo: Depth Mapping

• Image Registration
• Embedded Applications

33
3D Vision
Pyramid with dynamic programming and sub-pixel accuracy
15

2.5

1.5

1
10
0.5

y (meters)
0

-0.5

-1
5 -1.5

-2

-2.5

-8
0
-6
-3 -4
-1 -2
-4 1 0
3 2

z (meters) x (meters)

34
NEW Apps: Camera Calibrator and Stereo Camera Calibrator

 Simplified workflow estimating

camera intrinsic and extrinsic
parameters

 Removes the effects of lens

distortion from an image

 Automatically detects
checkerboard patterns

35
Point Cloud Processing in Matlab
As of R2015a
 Computer Vision System Toolbox (R2014b/15a)
pointCloud Object for storing a 3-D point cloud
pcdenoise Remove noise from a 3-D point cloud

pcdownsample Downsample a 3-D point cloud

pcmerge Merge two 3-D point clouds
pcregrigd Register two point clouds with ICP algorithm
pctransform Rigid transform a 3-D point cloud
showPointCloud Plot 3-D point cloud
Pcread Read a 3-D point cloud from PLY file
Pcwrite Write a 3-D point cloud to PLY file
alignColorToDepth Align Kinect color image to depth image
depthToPointCloud Convert Kinect depth image to a 3-D point cloud

 Computational Geometry in base Matlab

 Shipping Example: 3-D Point Cloud Registration and Stitching

37
Agenda
• Introduction
• Image Acquisition
• Explore-Enhance-Segment-Analyze
• Image Registration
• Feature Detect, Extract & Match
• Object Detection and Tracking
• 3D Vision
Vision
 Demo: Camera Calibrator App
 Demo: 3D Reconstruction
 Demo: Depth Mapping
• Embedded Applications
 Demo: Ball Tracker
• Embedded

Applications
Demo: Quad-copter Model
39
Embedded Applications

bwdist imbothat imtophat

 More than 60 Image Processing bwlookup imclearborder imwarp
functions support code generation bwmorph imclose iptcheckmap
bwpack imcomplement iptcheckconn
bwselect imdilate label2rgb
bwtraceboundary imerode mean2
 Most Computer Vision functionality bwunpack imextendedmax medfilt2
conndef imextendedmin multithresh
supports code generation edge imfill ordfilt2
fitgeotrans imfilter padarray
fspecial imhist rgb2ycbcr
 Many features generate platform- getrangefromclass imhmax strel
histeq imhmin stretchlim
independent code im2double imlincomb ycbcr2rgb
im2int16 imopen imref2d
im2single imquantize imref3d
im2uint16 imreconstruct affine2d
im2uint8 imregionalmax projective2d
imadjust imregionalmin

40
Demo: Ball Tracker

41
Agenda

• Introduction • Feature Detect, Extract & Match

• Image Acquisition • Object Detection and Tracking

• Explore-Enhance-Segment-Analyze • 3D Vision

• Image Registration • Embedded Applications

42
Explore-Enhance-Segment-Analyze

Feature Detection
3D Vision

Image Registration
Object Detection and Tracking

Apps

Image Image Color Region Camera

Acquisition Viewer Thresholder Analyzer Calibrator

http://www.mathworks.co.uk/products/image-video-processing 43
© 2015 The MathWorks, Inc.
46