OCR on a screenshot

Hello to the community,

I have a database and want to extract the list of what's in the database. The database can only be accessed via a third party software and this software doesn't let me do what I want so I thought I could take a few screenshots and get all what I need using "ocr"

Given it's a screeshot, the image quality is as good as it can be, albeit the resolution is limited, but despite of that the result contains several inaccuracies - see below the original screenshot and the ocr output

I can correct some of the errors by post-processing the text (find/replace sort of thing eg °7o -> %) but others are less obvious

I appreciate this is not the typical application for the ocr function, but is there a way to increase the accuracy? can I, for instance, tell ocr that all the characters are from a given font or something like that? is there a better function to do this?

Here's how I'm pre-processing the image, not much as you can imagine:

screenshot_edit = imresize(screenshot,3);

screenshot_edit = uint8(255*imbinarize(rgb2gray(screenshot_edit)));

and then simply

ocrResults = ocr(screenshot_edit);

SCREENSHOT:

OCR OUTPUT:

thank you very much

Paolo

Answer 1

[Mike Croucher]

Hi Paolo

Just because I am MathWorks staff doesn't mean anything here because until right now I've never used MATLAB's ocr function. But I was curious about what could be done

image = importdata('image.png');
image = imresize(image.cdata,4);
image = uint8(255*imbinarize(rgb2gray(image)));
text = ocr(image,CharacterSet="ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz1234567890%():.[],-@");
text.Text

My first attempt was very similar to yours. The main difference is that I restricted the charachter set. I also found that resizing to 4 times original size did a little better . Seems to get all of the % correct now but still gets some of the digits wrong sometimes.

    '20200929215922 (1.1): CAL: cal signal [Segment: 135454] (Local 10: 279233, Remote ID: -)
     
     20200929220012 (2.1): STEADY: IDLE [Segment: 135444] (Local ID: 279273, Remote ID: -)
     
     20201002134705 (1 . 1): CAL: 0,5 Vpp 500 Hz [Segment: 135451] (Local 10: 279230, Remote ID: -)
     20201002164537 (2.1): START [Segment: 135443] (Local 10: 279411, Remote ID: -)
     
     20201002164642.1: pre-start [Segment: 135443] (Local 10: 233710, Remote ID: -)
     
     20201002164737 (3.1): STEADY: LOW IDLE [Segment: 135443] (Local ID: 279273, Remote ID: -)
     
     20201002170209 (4.1): ACCEL: T0 92.6% NH 3: DECEL T0 L] I [Segment: 135443] (Local ID: 279267, Remote ID: -)
     20201002171153 (5.1): STEADY: LOW IDLE [Segment: 135443] (Local ID: 279263, Remote ID: -)
     20201002172400.1: 5mins @low idle [Segment: 135443] (Local 10: 231226, Remote ID: -)
     
     20201002172353 (6.1): ACCEL: T0 92.6% NH [Segment 135443] (Local ID: 279412, Remote ID: -)

I wanted to try the custom font trainer Train Optical Character Recognition for Custom Fonts - MATLAB & Simulink (mathworks.com). I had never used this before so initially tried it using your original image. Never managed to get it to resolve things like . and : well. So I created a new image 2x bigger than the original

image = importdata('image.png');
image = imresize(image.cdata,2);
image = uint8(255*imbinarize(rgb2gray(image)));
imwrite(image,'modimage.png')

I used the OCR Trainer on modimage.png to eventualy create myLang.traineddata (attached here as a .zip file).

Use this like this:

image = importdata('image.png');
image = imresize(image.cdata,2);
image = uint8(255*imbinarize(rgb2gray(image)));
ocrResults  = ocr(image,Language='C:\Users\Mike Croucher\Desktop\myLang\tessdata\myLang.traineddata');
%figure; imshow(Iocr);
ocrResults.Text

which gave the output

    '20200929215922 (1.1): CAL: cal signal [Segment: 135454] (Local ID: 279283, Remote ID: -)
     
     2o2oo92922oo12 (2.1): STEADY: IDLE [Segment: 135444] (Local ID: 279273, Remote ID: -)
     
     20201002134705 (1.1): CAL: 0,5 Vpp 500 Hz [Segment: 135451] (Local ID: 279280, Remote ID: -)
     2o2o1oo2164537 (2.1): START [Segment: 135443] (Local ID: 279411, Remote ID: -)
     
     20201002164642.1: pre-start [Segment: 135443] (Local ID: 283710, Remote ID: -)
     
     2o2o1oo2164737 (3.1): STEADY: LOW IDLE [Segment: 135443] (Local ID: 279278, Remote ID: -)
     
     20201002170209 (4.1): ACCEL: TO 92.6%.: NH & DECEL TO L/I [Segment: 135443] (Local ID: 279267, Remote ID: -)
     2o2o1oo2171153 (5.1): STEADY: LOW IDLE [Segment: 135443] (Local ID: 279268, Remote ID: -)
     20201002172400.1: 5mins @low idle [Segment: 135443] (Local ID: 281226, Remote ID: -)
     
     20201002172858 (6.1): ACCEL: TO 92.6%.: NH [Segment: 135443] (Local ID: 279412, Remote ID: -)
     
     '

The only mistake I can see by eye is that it confuses 0 for o sometimes.

Curious how well this generalises on other screenshots. If you give it a screenshot that has any charachters not included in the original, it won't work well because it will have never seen them before.

[Paolo Mazzoleni]

Thank you very much Mike, it works quite well now and some small mistake was easy to correct

Answer 2

[Walter Roberson]

https://www.mathworks.com/help/vision/ug/train-optical-character-recognition-for-custom-fonts.html