Contents

close all
clear
clc

% each frames is stored in a separate color image, store it to a 3D array by
% 1) converting each image in grayscale
% 2) stacking it in a separate layer of a 3D dimensional layer
V = zeros(144, 180, 42);
for ii = [1 : 42]
    % load the image and convert it to grayscale
    im = double(imread(sprintf('%08g.png', ii)));
    % standard conversions are performed by weighted linear combination, GRAY = 0.2989 * R + 0.5870 * G + 0.1140 * B
    imGray = (im(:, :, 1) + im(:, :, 2) + im(:, :, 3)) /3;
    V(:, :, ii) = imGray;
    % figure(1), imshow(V(:,:, ii), [])
end
disp('frames loaded');

frames loaded

Try to "segment" the foreground (i.e. the lady running) from the background (i.e., the wall and the road)

you can plot the temporal evolution of each pixel pixel without lady running (manually selected)

locationNoLady = [27, 52];
pixelNoLady = squeeze(V(locationNoLady(1), locationNoLady(2), :)); % squeeze gets rid of empty dimensions (first two)
% pixel crossed by the lady (manually selected)
locationLady = [102, 158];
pixelLady = squeeze(V(locationLady(1), locationLady(2), :)); % squeeze gets rid of empty dimensions (first two)

% have a look at the pixel intensities over time
figure(3),
plot(pixelNoLady, 'b--','LineWidth', 4),
hold on
plot(pixelLady, 'r', 'LineWidth', 4),
title(['Intensity (b, no lady) at pixel: ', num2str(locationNoLady), ' (r, no lady) at pixel: ', num2str(locationLady)]);
legend('no lady', 'lady')
xlabel('time')
ylabel('image intensity');
grid on

Background modeling

the mean of each pixel along the temporal dimension can be used as estimate of the bacground

bkg = mean(V, 3);

% The mean is affected by pixel values covered by the lady. The median is
% a much more robust estimate of the bkgr values and provide less false positives
% bkg = median(V, 3);

Background subtraction

% A very simple rule: "any pixel departing from its reference background value more than a fixed threshold is considered
% a foreground pixel"

% we need to set a suitable threshold. Let's try different values
THRESHOLD = 15;

for ii = 1 : 5: 42
    % load the image and convert it to grayscale
    im = double(imread(sprintf('%08g.png', ii)));
    imGray = (im(:, :,1) + im(:,:,2) + im(:,:,3)) /3;

    frg = abs(imGray - bkg);
    % try playing with the threshold, setting it to 0 will result in many
    % "false positives" since, because of noise, bacgkround values are not
    % contant even when there is no lady running
    frg = frg > THRESHOLD;

    figure(1), imshow(frg .* imGray, []), title('currentFrame')
    figure(2), imshow(frg, []), title('foreground');
    pause(0.1);
end

Issues:

1) the mean is not robust to the presence of the foreground (the mean in a pixel where the lady is passing through) 2) Setting the thrshold is difficult: an option would be to build confidence intervals on the mean. However, the standard deviation is also mainly affected by the presence of the lady 3) the threshold shuold also vary "pixel-wise" since there are regions that normally changes (e.g. leaves moved by the wind) 4) of course the background should be also adopted to avoid many false positive (e.g. illumination changes)

% in the following example you see that the "3-sigma" rule would not allow to detect the foreground since the standard
% deviation is too large
meanLady = mean(pixelLady);
stdLady = std(pixelLady);
medianLady = median(pixelLady);
figure(3),
plot(pixelLady, 'r', 'LineWidth', 4), title(['Intensity at pixel: ', num2str(locationLady)]);
hold on
plot([1, 42], meanLady * [1,1] , 'b-.', 'LineWidth', 4)
plot([1, 42], medianLady * [1,1] , 'k:', 'LineWidth', 4)
plot([1, 42], meanLady + 3 * stdLady * [1,1] , 'b--', 'LineWidth', 2)
plot([1, 42], meanLady - 3 * stdLady * [1,1] , 'b--', 'LineWidth', 2)
hold off

legend('intensity values', 'mean', 'median', 'Confidence Intervals on the mean')

Published with MATLAB® R2015b