Extracting information from a plot

I have a map of objects on a basic plot. there are areas on the plot that don’t have many objects at all. I want to identify and extract these ‘void’ areas on my plot. I was thinking of setting a threshold of less than 5 objects per ‘degree’ on the plot will count as a ‘void’ area. How would I go about doing this?

[jgg]

Could you give an example of what you're looking at? It's hard to analyze your problem without seeing it more concretely.

[jgillis16]

[jgillis16]

But, just focusing on the midsection 'u band' you can see on the plot.

Answer 1

[Image Analyst]

You can create an image where you place a dot (set a pixel to a value of 1) wherever there is a data point. If the pixel is already set, then add 1 to the pixel value. Then you can scan it with a moving window with conv2() to count the dots in the window and find out where the count is less than some number. For example (untested)

clc;    % Clear the command window.
close all;  % Close all figures (except those of imtool.)
clear;  % Erase all existing variables. Or clearvars if you want.
workspace;  % Make sure the workspace panel is showing.
format long g;
format compact;
fontSize = 20;

% Create random data.
numPoints = 100000;
x = rand(1,numPoints);
y = rand(1, numPoints);
scaleFactor = 1000; % Adjust depending on the range of x and y.
% Aim to have rows and columns be around a thousand or so.
rows = ceil(max(y) * scaleFactor);
columns = ceil(max(x) * scaleFactor);
% Allocate empty array - no counts at all.
dotImage = zeros(rows, columns);
% Place dots in proper locations on our dot image.
for k = 1 : length(x)
    row = ceil(scaleFactor * y(k));
    col = ceil(scaleFactor * x(k));
    dotImage(row, col) = dotImage(row, col) + 1;
end
subplot(2,2,1);
imshow(dotImage, []);
title('Dots Image', 'FontSize', fontSize, 'Interpreter', 'None');

% Now count the dots in a 11-by-11 moving window
filterWindow = ones(11);
countsImage = conv2(dotImage, filterWindow, 'same');
subplot(2,2,2);
imshow(countsImage, []);
title('Counts Image', 'FontSize', fontSize, 'Interpreter', 'None');

% Set up figure properties:
% Enlarge figure to full screen.
set(gcf, 'Units', 'Normalized', 'OuterPosition', [0 0 1 1]);
% Get rid of tool bar and pulldown menus that are along top of figure.
set(gcf, 'Toolbar', 'none', 'Menu', 'none');
% Give a name to the title bar.
set(gcf, 'Name', 'Demo by ImageAnalyst', 'NumberTitle', 'Off') 

% Let's compute and display the histogram.
[pixelCount, grayLevels] = hist(countsImage(:), 30);
subplot(2, 2, 3); 
bar(grayLevels, pixelCount); % Plot it as a bar chart.
grid on;
title('Histogram of counts image', 'FontSize', fontSize, 'Interpreter', 'None');
xlabel('Gray Level', 'FontSize', fontSize);
ylabel('Pixel Count', 'FontSize', fontSize);
xlim([0 grayLevels(end)]); % Scale x axis manually.

% Find out where the counts in the window is less than 50.
lowCountsImage = countsImage < 7;
subplot(2,2,4);
imshow(lowCountsImage, []);
title('Low Counts Map', 'FontSize', fontSize, 'Interpreter', 'None');

[jgillis16]

Thanks image! Now, my end goal (Walter this might help clarify the original question as well) is to impose a number (a correction in a sense) to the identified areas on the graph that fall below the galaxy threshold.

So what I mean is that, for example, I should know that in the sections of 100-150 on the x axis and -40 to -60, I should multiply the data set by that number.

Hope it's clear, and thanks again guys.

[Image Analyst]

I'm not sure what you want to do. Do you want to "add" more dots to regions where they are sparse? If so, how would you decide where to place them? Just figure out how many to place and randomly place them?

Or if you want to normalize by the number of dots in a moving window, then what would that image divide? I mean if you have (something)/countsImage, what is the (something)?

[jgillis16]

No, I want to identify the regions where the dots are sparse so that I may go back to the original text file where all the data is coming from, pull out those dots that come from that area and multiply their mass that I pull out from there by the number (the correction).

The first step will be if I can see what areas have sparsely populated dots.

Then, we need to identify which parts of the text file contain these identified areas. Would you like me to attach a sample of the text file? (Sample because it is quite a large file).

[Image Analyst]

lowCountsImage is essentially a map of the sparse regions.

To determine if a new point belongs to one of the sparse regions, you have to convert the (y, x) value to a pixel location (row, column) (just like you did when you created the dots image) and then see if lowCountsImage is true or false there at that location.

If it's true, then that new data point is in a sparse region, if it's false, then that data point is in a densely populated region.

[jgillis16]

Alright thanks! Sorry for all the questions (image analysis itself with matlab is very new to me). How would I apply this to a pre-loaded image/code that generates the figure attached to the original question instead of generating random points?

[Image Analyst]

There are a variety of ways to import your data into MATLAB: dlmread(), csvread(), importdata(), load(), xlsread(), textscan(), readtable(), fread(), etc. Try some of those.

[jgillis16]

Well, I had previously written code to import my data into matlab to produce the graph you see above.

    load GWGCCatalog222.txt %loads text into workspace
    readCatalog( GWGCCatalog222 )
    fid = fopen( 'GWGCCatalog222.txt');
    trashLine = fgets(fid); %Skips the first line
    data = textscan(fid, '%f%s%f%f%s%f%f%f%f%f%f%f%f%f%f%f%f%f', 'Delimiter', '|', 'TreatAsEmpty','~');
    fclose(fid);
    GalList.ra = data{3};
    GalList.dec = data{4};
    GalList.ra1 = GalList.ra * pi/12;
    GalList.dec1 = GalList.dec * pi/180 +90;
    GalaxyList = GalList;

And, I (naively) incorporated this code into your demo code as the following.

clc;    % Clear the command window.
close all;  % Close all figures (except those of imtool.)
clear;  % Erase all existing variables. Or clearvars if you want.
workspace;  % Make sure the workspace panel is showing.
format long g;
format compact;
fontSize = 20;
load GWGCCatalog222.txt %loads text into workspace
readCatalog( GWGCCatalog222 )
fid = fopen( 'GWGCCatalog222.txt');
trashLine = fgets(fid); %Skips the first line
data = textscan(fid, '%f%s%f%f%s%f%f%f%f%f%f%f%f%f%f%f%f%f', 'Delimiter', '|', 'TreatAsEmpty','~');
fclose(fid);
GalList.ra = data{3};
GalList.dec = data{4};
GalList.ra1 = GalList.ra * pi/12;
GalList.dec1 = GalList.dec * pi/180 +90;
GalaxyList = GalList;

% % Create random data.
% numPoints = 100000;
% x = rand(1,numPoints);
% y = rand(1, numPoints);
scaleFactor = 1000; % Adjust depending on the range of x and y.
% Aim to have rows and columns be around a thousand or so.
rows = ceil(max(GalList.dec1) * scaleFactor);
columns = ceil(max(GalList.ra1) * scaleFactor);
% Allocate empty array - no counts at all.
dotImage = zeros(rows, columns);
% Place dots in proper locations on our dot image.
for k = 1 : length(GalList.ra1)
    row = ceil(scaleFactor * GalList.dec1(k));
    col = ceil(scaleFactor * GalList.ra1(k));
    dotImage(row, col) = dotImage(row, col) + 1;
end
subplot(2,2,1);
imshow(dotImage, []);
title('Dots Image', 'FontSize', fontSize, 'Interpreter', 'None');
% Now count the dots in a 11-by-11 moving window
filterWindow = ones(11);
countsImage = conv2(dotImage, filterWindow, 'same');
subplot(2,2,2);
imshow(countsImage, []);
title('Counts Image', 'FontSize', fontSize, 'Interpreter', 'None');
% Set up figure properties:
% Enlarge figure to full screen.
set(gcf, 'Units', 'Normalized', 'OuterPosition', [0 0 1 1]);
% Get rid of tool bar and pulldown menus that are along top of figure.
%set(gcf, 'Toolbar', 'none', 'Menu', 'none');
% Give a name to the title bar.
%set(gcf, 'Name', 'Demo', 'NumberTitle', 'Off') 
% Let's compute and display the histogram.
[pixelCount, grayLevels] = hist(countsImage(:), 30);
subplot(2, 2, 3); 
bar(grayLevels, pixelCount); % Plot it as a bar chart.
grid on;
title('Histogram of counts image', 'FontSize', fontSize, 'Interpreter', 'None');
xlabel('Gray Level', 'FontSize', fontSize);
ylabel('Pixel Count', 'FontSize', fontSize);
xlim([0 grayLevels(end)]); % Scale x axis manually.

% Find out where the counts in the window is less than 50.
lowCountsImage = countsImage < 7;
subplot(2,2,4);
imshow(lowCountsImage, []);
title('Low Counts Map', 'FontSize', fontSize, 'Interpreter', 'None');

And, then proceeded to have the following error returned.

Error using zeros
Requested 91560x6283 (4.3GB) array exceeds maximum array size preference. Creation of arrays greater than this limit may take a long
time and cause MATLAB to become unresponsive. See array size limit or preference panel for more information.
Error in GPcounttest (line 36)
dotImage = zeros(rows, columns);

Any suggestions on where I am going wrong?

[Image Analyst]

What are the min and max of your data, max(GalList.dec1)? It looks like the maxes are 91.560 and 62.83. You will have to reduce the scale factor to 10 or so, and might even have to add something if your min value is less than zero.

The key line of code, which is actually a comment, seems to be the one you overlooked:

% Aim to have rows and columns be around a thousand or so.

Adjust scale factor so that that is true.

[jgillis16]

I adjusted the scale factor, but it seems that I left out a constant to multiply GalList.dec1 originally by, but the min is -89.3346 and max is 89.3464. I understand the scale factor issue, but now we face the problem that the min value is less than zero...

Could we possibly just adjust the following piece of code:

GalList.dec1 = GalList.dec;

to the following?

GalList.dec1 = GalList.dec + 90;

Now, with the scale factor set to 5, the search was 'too' refined. But, I had another question on explicitly what the moving window does? For example, what does it mean when I increase the window from an 11x11 to a 30x30?

[jgillis16]

And, (sorry again for all the questions!) how would I convert back to the x,y coordinates from the pixel values? I would like to have essentially a text output of all the sparse regions associated with the low counts image in an 'x,y value' format.

Answer 2

[Walter Roberson]

See the technique in http://www.mathworks.com/matlabcentral/fileexchange/31726-data-density-plot

Or you could do a 2D histogram.

I do not understand what you mean by 5 objects per degree in reference to a 2D plot: that would only make sense to me if you had a fixed origin point and you were more or less doing a radon transform.

[jgillis16]

Now, my end goal (Walter this might help clarify the original question as well) is to impose a number (a correction in a sense) to the identified areas on the graph that fall below the galaxy threshold.

So what I mean is that, for example, I should know that in the sections of 100-150 on the x axis and -40 to -60, I should multiply the data set by that number.

Hope it's clear, and thanks again guys.