Manual Calculation of Entropy not matching matlab

Hello, I am having trouble calculating entropy "by hand." I was doing this to learn more about it but now side-tracked because what I am doing doesn't result in the same answer as matlab. Here is my code:

matrix = abs(randn(1,256));
nbins = 256;

min_mat = min(matrix);
max_mat = max(matrix);

nbins_step = (max_mat-min_mat)/256;

bins = min_mat:nbins_step:max_mat;

bins(1) = bins(1)-nbins_step; %Adjust so that we can always use "greater than" bottom bin edge

%Count the number of occurances per bin
for ii = 1:nbins
pdist_bins(ii) = sum(matrix > bins(ii) & matrix <= bins(ii+1));
end
sum(pdist_bins) %Make sure that we have the same number of elements as we started with
pdist_norm = pdist_bins./numel(matrix); %Normalize
sum(pdist_norm) %Make sure adds to 1

%Remove 0s
pdist_no_zero = pdist_norm;
pdist_no_zero(pdist_no_zero == 0) = [];
entropy_calc = -sum(pdist_no_zero.*log2(pdist_no_zero))

%Matlab version
t = entropy(matrix)

I used my code on a simple example for a matrix [1 1 1 1 1 1 2 2 2 3 3 3] and got the correct answer (whereas matlab outputs 0...)

I am missing something but not sure. I think it is related to my choice of bins which may differ to matlabs imhist but I didn't think the result would differ by a lot (which I think it does).

Any help would be appreciated, thanks!

Answer 1

[Walter Roberson]

entropy(uint8(matrix))

To understand this you need to refer to the Tips section, where it says that anything other than logical is converted to uint8. Internally this is done by im2uint8(). im2uint8() assumes that floating point values are in the range 0 to 1, so when you pass in floating point values greater than 1, it assumes they should all saturate to the maximum uint8 value.

[thatguy14]

oh! Okay that's simple and also explains why the entropy returned by matlab is less than a hand calculation. So I guess I am doing it correctly! Thanks

[Walter Roberson]

The context is important for this purpose: entropy() is an Image Processing Toolbox function, and floating point data for images is expected to be in the range 0 to 1.