Contents
close all clear clc % each frames is stored in a separate color image, store it to a 3D array by % 1) converting each image in grayscale % 2) stacking it in a separate layer of a 3D dimensional layer V = zeros(144, 180, 42); for ii = [1 : 42] % load the image and convert it to grayscale im = double(imread(sprintf('%08g.png', ii))); % standard conversions are performed by weighted linear combination, GRAY = 0.2989 * R + 0.5870 * G + 0.1140 * B imGray = (im(:, :, 1) + im(:, :, 2) + im(:, :, 3)) /3; V(:, :, ii) = imGray; % figure(1), imshow(V(:,:, ii), []) end disp('frames loaded');
frames loaded
Try to "segment" the foreground (i.e. the lady running) from the background (i.e., the wall and the road)
you can plot the temporal evolution of each pixel pixel without lady running (manually selected)
locationNoLady = [27, 52]; pixelNoLady = squeeze(V(locationNoLady(1), locationNoLady(2), :)); % squeeze gets rid of empty dimensions (first two) % pixel crossed by the lady (manually selected) locationLady = [102, 158]; pixelLady = squeeze(V(locationLady(1), locationLady(2), :)); % squeeze gets rid of empty dimensions (first two) % have a look at the pixel intensities over time figure(3), plot(pixelNoLady, 'b--','LineWidth', 4), hold on plot(pixelLady, 'r', 'LineWidth', 4), title(['Intensity (b, no lady) at pixel: ', num2str(locationNoLady), ' (r, no lady) at pixel: ', num2str(locationLady)]); legend('no lady', 'lady') xlabel('time') ylabel('image intensity'); grid on
Background modeling
the mean of each pixel along the temporal dimension can be used as estimate of the bacground
bkg = mean(V, 3); % The mean is affected by pixel values covered by the lady. The median is % a much more robust estimate of the bkgr values and provide less false positives % bkg = median(V, 3);
Background subtraction
% A very simple rule: "any pixel departing from its reference background value more than a fixed threshold is considered % a foreground pixel" % we need to set a suitable threshold. Let's try different values THRESHOLD = 15; for ii = 1 : 5: 42 % load the image and convert it to grayscale im = double(imread(sprintf('%08g.png', ii))); imGray = (im(:, :,1) + im(:,:,2) + im(:,:,3)) /3; frg = abs(imGray - bkg); % try playing with the threshold, setting it to 0 will result in many % "false positives" since, because of noise, bacgkround values are not % contant even when there is no lady running frg = frg > THRESHOLD; figure(1), imshow(frg .* imGray, []), title('currentFrame') figure(2), imshow(frg, []), title('foreground'); pause(0.1); end
Issues:
1) the mean is not robust to the presence of the foreground (the mean in a pixel where the lady is passing through) 2) Setting the thrshold is difficult: an option would be to build confidence intervals on the mean. However, the standard deviation is also mainly affected by the presence of the lady 3) the threshold shuold also vary "pixel-wise" since there are regions that normally changes (e.g. leaves moved by the wind) 4) of course the background should be also adopted to avoid many false positive (e.g. illumination changes)
% in the following example you see that the "3-sigma" rule would not allow to detect the foreground since the standard % deviation is too large meanLady = mean(pixelLady); stdLady = std(pixelLady); medianLady = median(pixelLady); figure(3), plot(pixelLady, 'r', 'LineWidth', 4), title(['Intensity at pixel: ', num2str(locationLady)]); hold on plot([1, 42], meanLady * [1,1] , 'b-.', 'LineWidth', 4) plot([1, 42], medianLady * [1,1] , 'k:', 'LineWidth', 4) plot([1, 42], meanLady + 3 * stdLady * [1,1] , 'b--', 'LineWidth', 2) plot([1, 42], meanLady - 3 * stdLady * [1,1] , 'b--', 'LineWidth', 2) hold off legend('intensity values', 'mean', 'median', 'Confidence Intervals on the mean')