How to automatically identify text lines from projection plot?

I have been reading about automatic text line recognition in Matlab and although there are many advanced methods to do this every paper mentions that the simplest way of detecting text lines is via horizontal projections. So I decided to try this method for myself.

I am very new to this and have hit a brick wall, I have reached a level beyond which I do not know how to proceed. This is what I have achieved so far:

I'm trying for a system that is language independent and only interested in text lines, so I chose Arabic text:

I used the function ``radon`` to get the projections.

    img = rgb2gray(imread('arabic.jpg'));
    [R, xp] = radon(bw_closed, [0 90]);
    figure; plot(xp,R(:,2)); title('at angle 90');

This is the plot(projection)

So clearly the 5 peaks represent the 5 lines detected but how do I go from here to segmenting the original document?

Can anyone help me beyond this point? All the papers I read make no mention of how to proceed from this step, they just say that from the projections we have our detected lines.

What I'm asking is how, from the plot data can I tell matlab what is the line of text and what is the gab between lines?

[VIBHATH V B]

Hi.......... Can u send me the whole code for the above? What is 'bw-closed' here? My E-mail Id is : [e-mail]

Answer 1

[Image Analyst]

I would just find where the black is in the profile

darkPixels = R < 20; % Threshold
% label
labeledRegions = bwlabel(darkPixels);
% Find centroids
measurements = regionprops(labeledRegions, 'Centroid');
% Get them into an array
allCentroids = [measurements.Centroid];

Now you can just crop out some line of text you're interested in, into a separate image:

thisLine = yourImage(allCentroids(k):allCentroids(k+1), :);

[Faraz]

Thank you for your answer but it does not work as expected. Can you please tell me why you chose 20 as the threshold value? Was it from the plotted graph?

Also the bwlabel finds 63 elements as compared to the 5 lines. Finally can you please comment on what is happening in the last line of your code? I do not fully understand how I can crop the detected objects to a separate image? Will the newline be an image I can view via imshow? Thank you

[Image Analyst]

What is plotted? Isn't it R? If so, there are up to 6 regions where R is less than 20, or 10 or 1 or whatever lowest value you want to pick. I don't see how it can find 63. What does this say:

[labeledRegions, numberOfRegions] = bwlabel(darkPixels);
fprintf('Number of regions = %d\n', numberOfRegions);

To extract an image between row1 and row2, you do

croppedImage = grayImage(row1:row2, :);

That extracts all rows between row1 and row2 (inclusive). The column means take all columns so the extracted region goes all the way across the image.

[Faraz]

Oh my. I am really sorry, in my haste I thresholded the original image rather than R. That is why I got 63 elements. I now fully understand your code and thank you for explaining it.

I was really happy when it extracted the first line successfully, using the command:

thisLine = yourImage(allCentroids(k):allCentroids(k+1), :);

but

thisLine = yourImage(allCentroids(5):allCentroids(6), :);

produced this

rows(2):(3), (3):(4) and (5):(6) produce nothing. Was this because of the selected threshold value? I tried changing it but got the same results.

Thank you

[Image Analyst]

Alright, I'll do a complete demo starting with the binary image you showed above.

clc;    % Clear the command window.
close all;  % Close all figures (except those of imtool.)
clear;  % Erase all existing variables. Or clearvars if you want.
workspace;  % Make sure the workspace panel is showing.
format long g;
format compact;
fontSize = 20;
% Read in a demo image.
folder = 'C:\Users\Mark\Documents\Temporary';
baseFileName = 'arabic.jpg';
% Get the full filename, with path prepended.
fullFileName = fullfile(folder, baseFileName);
% Check if file exists.
if ~exist(fullFileName, 'file')
  % File doesn't exist -- didn't find it there.  Check the search path for it.
  fullFileNameOnSearchPath = baseFileName; % No path this time.
  if ~exist(fullFileNameOnSearchPath, 'file')
    % Still didn't find it.  Alert user.
    errorMessage = sprintf('Error: %s does not exist in the search path folders.', fullFileName);
    uiwait(warndlg(errorMessage));
    return;
  end
end
grayImage = imread(fullFileName);
% Get the dimensions of the image.  
% numberOfColorBands should be = 1.
[rows, columns, numberOfColorBands] = size(grayImage);
if numberOfColorBands > 1
  % It's not really gray scale like we expected - it's color.
  % Convert it to gray scale by taking only the green channel.
  grayImage = grayImage(:, :, 2); % Take green channel.
end
% Display the original gray scale image.
subplot(2, 2, 1);
% Text touches the top and bottom line, so let's pad the image
% to make a black space at top so we can find the dark centroids.
grayImage = padarray(grayImage, 10);
imshow(grayImage, []);
title('Original Grayscale Image', 'FontSize', fontSize);
% Enlarge figure to full screen.
set(gcf, 'Units', 'Normalized', 'OuterPosition', [0 0 1 1]);
% Give a name to the title bar.
set(gcf, 'Name', 'Demo by ImageAnalyst', 'NumberTitle', 'Off') 

% Let's compute and display the histogram.
[pixelCount, grayLevels] = imhist(grayImage);
subplot(2, 2, 2); 
bar(grayLevels, pixelCount);
grid on;
title('Histogram of original image', 'FontSize', fontSize);
xlim([0 grayLevels(end)]); % Scale x axis manually.

verticalProjection = sum(grayImage, 2);
subplot(2, 2, 3); 
plot(verticalProjection, 'b-');
grid on;

darkPixels = verticalProjection < 5000; % Threshold
% label
[labeledRegions, numberOfRegions] = bwlabel(darkPixels);
fprintf('Number of regions = %d\n', numberOfRegions);
% Find centroids
measurements = regionprops(labeledRegions, 'Centroid');
% Get them into an array
allCentroids = [measurements.Centroid];
xCentroids = int32(allCentroids(1:2:end));
yCentroids = int32(allCentroids(2:2:end));

% Now you can just crop out some line of text you're interested in, into a separate image:
plotLocation = 12;
for band = 1 : numberOfRegions-1
  row1 = yCentroids(band);
  row2 = yCentroids(band+1);
  thisLine = grayImage(row1 : row2, :);
  plotLocation
  subplot(10, 2, plotLocation);
  imshow(thisLine, []);
  plotLocation = plotLocation + 2
end

Does that work for you?

[Faraz]

That was wonderful. I learned much more than feature extraction by studying your code. Thank you

[Faraz]

@ImageAnalyst, I know I am coming back to an old question of mine but I wanted to use this method on another image of mine.

I have to ask, can you tell me why you chose 5000 as your threshold value in

darkPixels = verticalProjection < 5000; % Threshold

Now on this new image, it is only producing a vector of all 1's. Was 5000 an arbitrary value you selected?

[Salaheddin Hosseinzadeh]

Great job Image Analyst!

[Image Analyst]

I looked at the profile. Look how it goes from 10,000 to 70,000 in the above plot (Click See more comments to display it again). See that the low point between the letters is about 1000 or so (like 1/10th of the height of the first grid line)? And the "noise" in the spikes doesn't happen until the signal rises above about 7,000 or so. So that's why I chose 5000 - it seems like it will reliably find the black bands between the words.

[MONIKA SAINI]

Please Image Analyst Tell me line segmentation for my text. I tried your code but Its not working.Please help me.

[Image Analyst]

Threshold the red channel

lines = rgbImage(:,:,1) < 128; % or whatever value works.

[Rutika Titre]

Hello sir,

I have segmented line of my input image but I m not getting proper output. Can you please help me sir?.

How to get horizontal projection as well. I have tried a code for horizontal projection. It is working but still, I want to confirm it once.

Thank you Sir.

[Rutika Titre]

Sir, Here is my code for horizontal and vertical projection profile for text line detection.

[Image Analyst]

Rutika, I don't have your image. Just try adjusting the thresholds to get what you want.

[Rutika Titre]

Good morning Sir, I have attached binary image and sir actually I was thinking to make my code generalized for any input image. I have attached vertical and horizontal projection profile code also small request can u please check it and tell me I am doing it right. Thank You for your kind consideration Sir.

[Rutika Titre]

[Rutika Titre]

Sir I got my proper output, input image threshold value is 1000. Sir, can u please suggest to make it this threshold generalized for any number of lines? Thank YOu

[Image Analyst]

My code is attached. I also ended up with a threshold of 1000. I suggest you use a threshold of the same as the number of columns in the image. If it's truly a binary image and want to take it is there is any pixels at all that is white, then you can use any() instead of thresholding.

[Rutika Titre]

Thank you Sir, about horizontal projection profile and vertical projection profile can u please tell whether my approach is right.I have attached code "projection.m" if any changes please do let me know. Thank u.

[ayushi]

sir @Image Analyst i tried this code:

h = sum(a,2);
  plot(h,1:size(a,1),'Parent',handles.axes2)
   title('HISTOGRAM OF Cropped IMAGE', 'FontSize', 15)
   % Find the dark lanes that define the zones between the lines with letters on them.
% Rather than find the letters themselves, we will find the dark spaces between the lines.
% Then we will go from the center of each one of those to the center of the next one down the page.
% Get the vertical projection by summing the image horizontally.
verticalProjection = sum(a, 2);
pause(3)
subplot(2, 2, 3);
plot(verticalProjection, 'b-');
grid on;
title('Vertical Projection', 'FontSize', 15);
% Find the dark lanes that define the zones between the lines with letters on them.
% Rather than find the letters themselves, we will find the dark spaces between the lines.
% Then we will go from the center of each one of those to the center of the next one down the page.
darkPixels = verticalProjection < 1000; % Threshold label
[labeledRegions, numberOfRegions] = bwlabel(darkPixels);
fprintf('Number of regions = %d\n', numberOfRegions);
% Find centroids
measurements = regionprops(labeledRegions, 'Centroid');
% Get them into an array
allCentroids = [measurements.Centroid];
xCentroids = int32(allCentroids(1:2:end));
yCentroids = int32(allCentroids(2:2:end));
fprintf('I found %d black spaces between the lines of text\n', length(yCentroids));

but i am not getting the proper segmented lines here I am uploading the final segmented lines image. please suggest me where i need to change the code in order to get single single line one by one please help me to correct my code and also uploading the original image along with this

[Image Analyst]

Try using radon() or hough() to find the lines and then rotate your image (straighten it).

[ayushi]

sir i tried this but it i giving me the same segmented output but in the vertically what i want too do is to get accurate segmentation of line

[Elhadj Noure]

Hello sir @Image Analyst, thank you for your advice and help. I work on extracting the line and words from the Arabic handwritten image and I have esseyé this code, but it does not work with me. Please help me.

[Image Analyst]

Try to find papers here that describe how to do it: http://www.visionbib.com/bibliography/contents.html

[Elhadj Noure]

Excuse me I did not understand. is not possible to apply your code to extract lines ??

[moahaimen talib]

<<

>>

sir @image analyst i treied the code but thats what i got

[HJ Akhtar]

Sir @Image Analyst! Can you please tell me the algorithm name you used in this code... Or tell reference to the paper(if any). Your code helped a lot. Thankyou https://www.mathworks.com/matlabcentral/answers/112857-how-to-automatically-identify-text-lines-from-projection-plot#comment_190591

[Image Analyst]

I don't think it has a name. It's just something I thought up. Something that basic usually doesn't have a name.

[Bachtiar Muhammad Lubis]

@ HJ Akhtar : i've read some papers and found that this is projection profile algorihm. just googling "image segmentation using projection profile".

Answer 2

[fawzi]

what if the lines are curved ? in this case the projection is not useful. Can you help me in this problem

Answer 3

[Rinku]

try this code;

img = rgb2gray(imread('arabic.jpg'));
[R, xp] = radon(bw_closed, [0 90]);
figure; plot(xp,R(:,2)); title('at angle 90');
r = R(:,2);
r=r(92:391); % your image region
blank = r < 3; % region without text
[labeled, n] = bwlabel(blank);
C = regionprops(labeled, 'Centroid');  % find the centers of blank regions
for i=1:length(C)-1
    subplot(length(C)-1,1,i)
    imshow(img(C(i).Centroid(2):C(i+1).Centroid(2),:));        
end

[VIBHATH V B]

Hi......... I have to segment each and every lines of text and have to recognize each line. How? Can u give me the code for that?

[moahaimen talib]

can anyone tell me what is bw_close i need this badly

[Image Analyst]

From the sounds of it, it's a binary image upon which they did a morphological closing with the imclose() function. Then they did a radon transform with just two angles: 0 and 90, so R(:,2) would be the horizontal profile of bw_close, so R(:,2) is the same as sum(bwClose, 1) I think.

[moahaimen talib]

great answering thank you i understand that now

[Urooba zaki]

respected sir this code shows an error... i think the function file is missing ... plz reply Undefined function or variable 'bw_closed'.