Cant convert more than 80 columns of data to categorical

I am trying to convert all coloumns of a 1000x170 table to categorical data. "bt_000" would be the 81st coloumn.

It would seem that the function does not work after the 80th coloumn, any remedies?

Thank you.

[dpb]

It's not the number or the postion but the content of the column that's the problem, methinks.

Bet it will error the same way if you try that column alone.

Answer 1

[Walter Roberson]

When you categorical() numeric values, then internally it uses %-0.5g format to convert the values to character vectors. It then errors if the number of unique values that result is different than the number of unique numbers.

Which is to say that when you categorical() numeric values, it is a problem if two of the unique values are the same up to the 5th decimal digit.

[dpb]

Huh. Who wooda' thunk...

Takes a lot of reading to find but under "Tips" section one finds:

• If the input array has numeric, datetime, or duration values that are too close together, then the categorical function truncates them to duplicate values. For example, categorical([1 1.00001]) truncates the second element of the input array. To create categories from numeric data, use the discretize function.

It seems should be much more prominent and explicit than that in the documentation and the error message could be a whole lot more informative. As is, I'll agree w/ the OP here that there's no readily decipherable clue as to what the problem is.

[dpb]

That doesn't say it errors because of it, though...that's probably worth a documentation addendum note (not that anybody ever actually reads the doc first).

[SkarletSam]

Seems like fuzziness with the categorical was the root of the problem. There were some columns that had large numbers that were really close to each other in numeric value. Took those out and the varfun worked. I discretized the columns that didnt work inidividually and added onto the newTable. Thanks for the input!