How to do one hot encoding of unusual letters in matlab?

4 ビュー (過去 30 日間)
Sanjana Sankar
Sanjana Sankar 2019 年 7 月 19 日
コメント済み: Sanjana Sankar 2019 年 7 月 31 日
I have a table of german characters. I would like to one hot encode them so that i can input it to a neural network. How should i go about?
The problem is some of the characters are not accepted by matlab as characters. For example, 'ä' 'ö' 'ü' 'ß'
Regardless, I would like to know how to one hot encode any character from a TABLE in matlab.
Thanks in advance!
  3 件のコメント
Mario Chiappelli
Mario Chiappelli 2019 年 7 月 19 日
Was ist heiss Codierung?
Sanjana Sankar
Sanjana Sankar 2019 年 7 月 22 日
Please check the link provided by Walter about one-hot encoding

サインインしてコメントする。

採用された回答

Walter Roberson
Walter Roberson 2019 年 7 月 19 日
編集済み: Walter Roberson 2019 年 7 月 19 日
https://machinelearningmastery.com/why-one-hot-encode-data-in-machine-learning/ describes One-Hot Encoding (a term I was not aware of)
You might want to first construct a list of permitted characters, and map the input into an offset in that list. That will potentially save you from wasting bits on characters such as Œœ that you are not using.ŒŒ
  5 件のコメント
Walter Roberson
Walter Roberson 2019 年 7 月 30 日
permitted = ['A' : 'Z', 'a' : 'z', 'ä', 'ö', 'ü', 'ß', ' ', '.' ] ;
[found, idx] = ismember(YourText, permitted);
assert(all(found), 'unpermitted character detected')
OneHot = ind2vec(idx) ;
Sanjana Sankar
Sanjana Sankar 2019 年 7 月 31 日
OneHot = full(ind2vec(idx,max(idx)));
This is exactly what I was looking for and I got it from your code. Thank You! This works :)

サインインしてコメントする。

その他の回答 (1 件)

Guillaume
Guillaume 2019 年 7 月 30 日
file containing the characters that I want to one hot encode. Do you know how I can go about now
%phoneme_set: A cell array of phonemes to one-hot encode
assert(numel(phoneme_set) > 64, 'Cannot one-hot encode more than 64 phonemes with a 64-bit integer')
phoneme_set(:, 2) = num2cell(2 .^ uint64(0:size(phoneme_set, 1)-1))
  3 件のコメント
Guillaume
Guillaume 2019 年 7 月 30 日
number binary pattern (64 bits)
2^0 0000000000000000000000000000000000000000000000000000000000000001
2^1 0000000000000000000000000000000000000000000000000000000000000010
2^2 0000000000000000000000000000000000000000000000000000000000000100
...
2^63 1000000000000000000000000000000000000000000000000000000000000000
This is what one-hot encoding is. As I commented, this is useful for FPGAs and similar which operate at the bit level. On the generic processor of a computer, it's a complete waste of space but my answer does what you asked.
You could encode the one-hot encoded numbers that I generate as a vector of 0 and 1 (double) for even more waste of space:
phoneme_set(:, 3) = num2cell(fliplr(eye(size(phoneme_set, 1), 64)), 2)
Note that the binary pattern of the 0s in that encoding is 0000000000000000000000000000000000000000000000000000000000000000b and of the 1s is 0011111111110000000000000000000000000000000000000000000000000000b.
However, I suspect that what we have here is a XY problem. You have some unspecified problem doing something and you think you can solve it by using one-hot encoding (without really understanding what it means) so ask about one-hot encoding instead of your actual problem.
Sanjana Sankar
Sanjana Sankar 2019 年 7 月 30 日
I have to feed characters to a neural network. And the most common method is one hot encoding. But I haven't found any examples on how it is done in MATLAB.

サインインしてコメントする。

カテゴリ

Help Center および File ExchangeImage Data Workflows についてさらに検索

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by