Solution 2
HVCA course
Hanan Shteingart 036182954
1. ImageDerivatives.m
function [Ix, Iy, It] = ImageDerivatives(I1, I2)
Kx = 1/4*[ 1 -1 ; 1 -1];
Ky = 1/4*[-1 -1 ; 1 1];
Kt = 1/4*[ 1 1 ; 1 1];
Ix = (conv2(I1, Kx,'same') + conv2(I2, Kx,'same'));
Iy = (conv2(I1, Ky,'same') + conv2(I2, Ky,'same'));
It = conv2(I2, Kt,'same') - conv2(I1, Kt,'same');
2. LK_alg.m
function [v_prev v_prev_trace]= LK_alg(I1, I2, Lambda, mask,
v_initial, num_iterations);
% v_prev is the final estimation
% v_prev_trace is the curve of estimated velocities
%% init
v_prev = v_initial;
I2w = I2;
currentmask = mask;
%% iterations
v_prev_trace = zeros(num_iterations + 1,2);
v_prev_trace(1,:) = v_prev;
for iteration = 1:num_iterations
% translate I2 according to previously estimated velocity
[I2w warpMask] = warp(I2, v_prev);
% update mask
currentmask = currentmask.*warpMask;
% caculate velocities:
[Ix, Iy, It] = ImageDerivatives(I1.*currentmask,
I2w.*currentmask); % should we mask here? two images or only one?
% build A and B matrices
IxySum = sum(sum(Ix.*Iy));
IxxSum = sum(Ix(:).^2);
IyySum = sum(Iy(:).^2);
A = [IxxSum+Lambda, IxySum; IxySum, IyySum+Lambda];
IxtSum = sum(sum(Ix.*It));
IytSum = sum(sum(Iy.*It));
B = - [IxtSum ; IytSum];
% estimate new velocity
v_estimated = A^-1*B;
% update velocity
v_prev = v_prev + v_estimated;
v_prev_trace(iteration + 1, :) = v_prev;
end
3. blur_downsample.m
function Iblur = blur_downsample(I)
load GaussKernel;
Iblur = conv2(I, GaussKernel, 'same');
Iblur = Iblur(1:2:end, 1:2:end);
4. Full_LK.m
function [v v_trace]= Full_LK_alg(I1, I2, Lambda, mask,
num_iterations)
%% init estimation
I1_blur = blur_downsample(I1);
I2_blur = blur_downsample(I2);
mask_blur = blur_downsample(mask);
v0 = [0;0];
num_iterations0 = 1;
v_blurred = LK_alg(I1_blur, I2_blur, Lambda, mask_blur, v0,
num_iterations0);
v1 = 2*v_blurred;
%% iterative estimation
[v v_trace] = LK_alg(I1, I2, Lambda, mask, v1, num_iterations);
5. Tree & Flowers
Tree:
v = (-2.01, -0.32) ,|v| = 2.04
Flower: v = (-1.13, -0.02) ,|v| = 1.13
The tree is moving faster (It is closer to the aperture).
%question5
%%
clc; close all;
%%
im1 = double(imread('flower-i1.tif'));
im2 = double(imread('flower-i2.tif'));
%mymovie(im1,im2);
% the tree is moving faster (because it's closer)
%% make masks
ixtree = 90:130;
iytree = 15:120;
masktree = zeros(size(im1));
masktreey = masktree;
masktreex = masktree;
masktreex(:, 90:130) = 1;
masktreey(15:120, :) = 1;
masktree = masktreex.*masktreey;
iyflower = 80:120;
ixflower = [1:89, 131:180];
maskflower = zeros(size(im1));
%maskflower(80:120,:) = 1;
maskflower(100:120,1:100) = 1;
%maskflower= maskflower.*(1-masktree);
%% plot masks
figure; colormap gray;
subplot(2,3,1);
imagesc(im1); axis equal tight;
subplot(2,3,4);
imagesc(im2); axis equal tight;
subplot(2,3,2);
imagesc(im1.*masktree); axis equal tight;
subplot(2,3,5);
imagesc(im2.*masktree); axis equal tight;
subplot(2,3,3);
imagesc(im1.*maskflower); axis equal tight;
subplot(2,3,6);
imagesc(im2.*maskflower); axis equal tight;
%% calc velocities
num_iterations = 50;
Lambda = 0;
I1 = im1;
I2 = im2;
% %% test:
% I2 = zeros(size(I1));
% vx = 0;
% vy = 0;
% I2(vy+1:end,vx+1:end) = I1(1:end-vy,1:end-vx);
% nomask = ones(size(I1));
% [vall, vall_trace] = Full_LK_alg(I1, I2, Lambda, nomask,
num_iterations);
%[vtree vtree_trace] = Full_LK_alg(I1, I2, Lambda, masktree,
num_iterations);
[vtree vtree_trace] = Full_LK_alg(I1(iytree, ixtree), I2(iytree,
ixtree), Lambda, ones(length(iytree), length(ixtree)),
num_iterations);
%[vflower vflower_trace] = Full_LK_alg(I1, I2, Lambda, maskflower,
num_iterations);
[vflower vflower_trace] = Full_LK_alg(I1(iyflower, ixflower),
I2(iyflower, ixflower), Lambda, ones(length(iyflower),
length(ixflower)), num_iterations);
fprintf('tree:
v = (%3.2f, %3.2f) ,|v| = %3.2f\n', vtree(1),
vtree(2), norm(vtree));
fprintf('flower:
v = (%3.2f, %3.2f) ,|v| = %3.2f\n', vflower(1),
vflower(2), norm(vflower));
iter = 0:num_iterations;
%%
figure;
plot(iter, vtree_trace(:,1),'b', iter, vtree_trace(:,2),'b:', iter,
vflower_trace(:,1),'r', iter, vflower_trace(:,2),'r:');
legend('tree - x', 'tree - y', 'flower - x','flower y','location','bestoutside');
xlabel('iteration');
ylabel('speed [pixels]');
%% compare to exact correlations (failed to do so due to resolution
%% problems...)
c_tree = conv2(I1.*masktree, rot90(I2.*masktree,2),'same');
vtree_xcorr = max2(c_tree);
c_flower = conv2(I1.*maskflower, rot90(I2.*maskflower,2),'same');
vflower_xcorr = max2(c_flower);
6. Gaussian Spots
Discussion:
a. There is a scaling phenomena where the error curve of the
broad spots behaves similarly to that of the narrow one but with
a scale relative to the ratio between the spot size (10)
b. The error on the Y axis is identically zero
c. Asymptotically, the error in estimation is linear and proportional
of the true velocity. This happens due to the failure of the
approximation near I1 (Taylor).
d. The error initially increases but then decreases back to zero and
then start increasing again. This is due to the exact shape of the
Gauss spot which has changing derivatives (see plot below). I
guess the error can be analyzed analytically but we weren't
asked to do so.
% Question 6
% ============
clc; clear all; close all;
%% Params
% Question params
imageSize = 128;
mue0 = [0 0];
spot_sizes = [10 1];
velocities = [0:1:50];
% KL params
Lambda = 0;
num_iterations = 0;
mask = ones(imageSize, imageSize);
%% init
spot_n = 0;
estimated_velocities = zeros(length(spot_sizes), length(velocities),
2);
%% loop over different setups
%spot_sizes = 1;
%velocities = 10;
for spot_size = spot_sizes
spot_n = spot_n+1;
I1 = GausSpot(imageSize, spot_size, mue0);
velocity_n = 0;
%fprintf('spot size %d\n', spot_size);
for velocity = velocities
velocity_n = velocity_n+1;
I2 = GausSpot(imageSize, spot_size, mue0 + [velocity 0]);
%fprintf('\t\ttial %d\n', n);
est_v = Full_LK_alg(I1, I2, Lambda, mask, num_iterations);
estimated_bias(spot_n, velocity_n, :) = est_v - [velocity;
0];
estimated_L2(spot_n, velocity_n) = norm(est_v - [velocity;
0]);
end
end
%% plot
figure;
plot(velocities, estimated_L2(1,:), velocities, estimated_L2(2,:)
,'b.:');
legend('Broad Spot Bias', 'Narrow Spot Bias');
title(sprintf('Q6: Bias (L2) of Broad and Narrow Gaussian Spot.
\\lambda=%3.2f number of iteration=%d', Lambda, num_iterations));
xlabel('X velocity');
ylabel('estimation error norm');
figure;
plot(velocities, squeeze(estimated_bias(1,:,:)));
hold on;
plot(velocities, squeeze(estimated_bias(2,:,:)),'.:');
hold off;
legend('Broad X', 'Broad Y','Narrow X', 'Narrow Y');
xlabel('X velocity');
ylabel('Coordinates(x&y) Bias of Broad and Narrow Gaussian Spot');
title(sprintf('Q6: Bias (xy coordinates) of Broad and Narrow Gaussian
Spot. \\lambda=%3.2f number of iteration=%d', Lambda,
num_iterations));
%%
figure;
plot(velocities, estimated_L2(1,:)./velocities, velocities,
estimated_L2(2,:)./velocities ,'b.:');
legend('Broad Spot Bias', 'Narrow Spot Bias');
title(sprintf('Q6: Bias (L2) of Broad and Narrow Gaussian Spot
Relative to True Velocity. \\lambda=%3.2f number of iteration=%d',
Lambda, num_iterations));
%% Discussion
% The estimation errors change almost linearly as a function of
speed. The larger the speed
% the less accurate is the Taylor expansion used in the KL algorithm.
% A smaller spot produce largers errors then the broader one at
velocities
% larger then 4 (roughly 4 time fold the spot size). This phenomena
is due
% to the fact that the scale is smaller so that the Taylor expansion
is less
% accurate far from the spot.
7. Rhombus Velocity Estimation
From the rhombusMovie.m I've found the angles of the broad and narrow
rhombus to be (0,45) and (30,45) respectively.
The IOC and VA:
Broad rhombus:
VIOC  100
2
  45
2
Thin rhombus:
VVA 
v1  2  450
v2  
vVA  0.5985  51.2060
vIOC  100
1
  x60
2
(0.75,0.933)
Discussion:
a. Lower contrast produce larger errors but if the number of iteration
increases, the error decreases.
b. The broad rhombus (dashed in the plot above) has smaller error
relative to the thin one at the same number of iterations and same
contrast.
c. The more iteration the better is the estimation.
d. The solution approximates the IOC solution at high contrast.
e. At low contrast, the solution resembles the VA for the broad shape but
less for the thin.
f. My perception is that :
a. the broad shape
i. in low contrast is moving right and slightly down
ii. in high contrast is moving sideways
b. the thin shape:
i. in low contrast is moving on the diagonal (VA)
ii. in high contrast is moving sideways (IOC)
g. the reason for the above perception is the influence of edges.
% Question 7
% ==========
clear all; close all; clc;
% Question params
fat_vec = [0 1];
contrast_vec = [0:0.01:1];
% KL params
Lambda = 0.01;
Niter_vec = [0,5, 10];
imageSize = 128;
mask = ones(imageSize, imageSize);
fat_n = 0;
v0 = [1;0];
estimated_velocities = zeros(length(fat_vec), length(contrast_vec),
length(Niter_vec),2);
for fat = fat_vec
fat_n = fat_n+1;
contrast_n = 0;
for contrast = contrast_vec
contrast_n = contrast_n + 1;
[im1,im2]=rhombusMovie(fat,contrast);
iter_n = 0;
for num_iteration = Niter_vec;
iter_n = iter_n + 1;
estimated_velocities(fat_n, contrast_n, iter_n, :) =
Full_LK_alg(im1, im2, Lambda, mask, num_iteration);
estimates_error(fat_n, contrast_n, iter_n,:) =
squeeze(estimated_velocities(fat_n, contrast_n, iter_n, :)) - v0;
estimates_bias(fat_n, contrast_n, iter_n) =
norm(squeeze(estimates_error(fat_n, contrast_n, iter_n,:)));
end
end
end
%% plot
figure(1);
plot(contrast_vec, squeeze(estimates_bias(1, :, :)),'-');
title('thin rhombus');
xlabel('contrast');
ylabel('norm of estimation error');
hold on;
plot(contrast_vec, squeeze(estimates_bias(2, :, :)),':');
title('rhombus');
xlabel('contrast');
ylabel('norm of estimation error');
hold off;
set(gca,'yscale','log');
set(gca,'xscale','log');
%%
s = cellstr(num2str(Niter_vec'));
Niter = length(Niter_vec);
for i=1:Niter
s0{i} = ['narrow - num iteration=' s{i}];
end
for i=1:Niter
s1{i} = ['broad - num iteration=' s{i}];
end
s = {s0{:} s1{:}};
legend(s,'location','southwest');
set(gca,'yscale','log');
%%
figure;
clc; close all;
plot(contrast_vec, squeeze(estimated_velocities(1, :, :, 1)),'-');
hold on;
plot(contrast_vec, squeeze(estimated_velocities(1, :, :, 2)),':');
plot(contrast_vec, squeeze(estimated_velocities(2, :, :, 1)),'.-');
plot(contrast_vec, squeeze(estimates_error(2, :, :, 2)),':.'); hold
off;
legend(...
'x - broad itr 0',...
'x - broad itr 5',...
'x - broad itr 10',...
'y - broad itr 0',...
'y - broad itr 5',...
'y - broad itr 10',...
'x - thin itr 0',...
'x - thin itr 5',...
'x - thin itr 10',...
'y - thin itr 0',...
'y - thin itr 5',...
'y - thin itr 10',...
'location','bestoutside');
title('rhombus estimation (x&y)');
ylabel('velocity (x&y)');
xlabel('contrast');set(gca,'xscale','log');
%% VIOC and VVA
o1 = deg2rad(30);
o2 = deg2rad(45);
v1 = -sin(o1);
v2 = -sin(o2);
VIOCx = (v1*cos(o2)-v2*cos(o1))/sin(o2-o1)
VIOCy = (v1*sin(o2)-v2*sin(o1))/sin(o2-o1)
8. Plaids moving
The algorithm doesn't agree to the velocity of (Vx,0) for the second kind of
plaid but does agree to it for the first set.
ima1 = double(imread('movie_a_frame1.tif'));
ima2 = double(imread('movie_a_frame2.tif'));
imb1 = double(imread('movie_b_frame1.tif'));
imb2 = double(imread('movie_b_frame2.tif'));
Lambda = 0;
mask = ones(size(ima1));
num_iteration = 10;
[va, va_trace] = Full_LK_alg(ima1, ima2, Lambda, mask,
num_iteration);
[vb, vb_trace] = Full_LK_alg(imb1, imb2, Lambda, mask,
num_iteration);
plot(0:num_iteration, va_trace, 0:num_iteration, vb_trace);
legend('Vax', 'Vay', 'Vbx','Vby');
xlabel('iteration');
ylabel('speed');