Error when assigning categories to a tall array

I ran into a wiered problem for me as when I tried to assign categories to a tall array, it failed while it went through if I used the gathered version. See attached a small dataset as a test dataset (only 100 strings), and below a working example:

% test

load('testSmall.mat');

code    = "1";
meaning = "test";


res      = categorical(testSmall, code, meaning); % ok

testTall = tall(testSmall);
res2     = categorical(testTall, code, meaning);  % error occurred

The error message:

Error using categorical (line 528)

VALUESET must be numeric when DATA is numeric.

Error in tall/categorical>iGetOutputAdaptor (line 89)

localData = categorical([], extraArgs{:}, varargin{idx:end});

Error in tall/categorical (line 44)

outAdaptor = iGetOutputAdaptor(tdata,varargin{:});

It happened that when I nevigate into the categorical function, the first parameter (inputData) was always empty when I use the tall array testTall as the input. Why this happens?

Answer 1

[Sean de Wolski]

That also appears to be a bug in how tall handles scalar strings for cats and values. If you make code and meaning into vectors it works fine. Here I give them a dummy value:

code    = ["1" "2"];
meaning = ["test" "answer"];
res      = categorical(testSmall, code, meaning); % ok
testTall = tall(testSmall);
res2     = categorical(testTall, code, meaning);  

I'll report this one to development as it's completely reproducible.

[Peng Li]

yeah Thanks Sean. as long as there are more than two elements, it works perfectly. I can do this but it is just annoying for me given this code bug.

[Peng Li]

So all my three questions are related. I've kind of solved the second and the third. My first is still hanging there. Any suggestion please? Without solving this, I just couldn't get my hands on this dataset. Instead, I'm working on a subset these days to get the program for other processings moved on.

https://www.mathworks.com/matlabcentral/answers/512097-internal-problem-while-evaluating-tall-expression-requested-40-5-gb-array

[Sean de Wolski]

I (we) really need a few rows of representative data to help you any further with the original question.

[Peng Li]

understand. I'm wondering if it's fine if I sent you a subset privately?

[Sean de Wolski]

Sure. Message me >>char([115 100 101 119 111 108 115 107 64 109 97 116 104 119 111 114 107 115 46 99 111 109]) and I can send you a secure file transfer link.

[Peng Li]

yeah great! messaged you with a dropbox link (hope this is secure as well lol).

[Sean de Wolski]

I haven't recieved it; I think my spam filter blocks dropbox links!

[Peng Li]

Sorry Sean. Just saw your reply. I'll drop you an email again.

[Peng Li]

Sean, any chance that you received my new email?

[Sean de Wolski]

I have not. I just checked my spam filter as well and verified that the email from the MATLAB command above works.

[Sean de Wolski]

sent