Pattern recognition by neural networks, using binary data sparse matrices

My neural network has to learn to output yes or no, based on the occurrence of particular groups in the input. There are 21 groups, so a given group is represented by twenty '0's and one '1'. The position of the '1' indicates the group. Three groups are presented to the network at any time, so a single input is a 63x1 vector with sixty '0's and three '1's.

Currently my network adjusts a 63x1 weight matrix for each neuron. I think I am forcing it to learn too much (slightly pointless) information though. What I would like is for the network to only learn information about the position of the '1's, about the presence not the absence of a group. That is, I only want it to adjust three weights at a time, and if it sees a zero I want it to read "do nothing to that position's associated weights".

Does anyone know a way I could implement a design like this? I tried using a sparse matrix as the input but it doesn't seem to make a difference. Is it possible to put an if condition on the adjustment of weights?

Thank you for any thoughts.

Answer 1

[Greg Heath]

The design is straightforward. Just train using a 63xN dim input matrix and a 21xN dim target matrix.

However, N needs to be large. The complete input space contains 21^3 vectors. I don't know how many you need in the training set to get good performance on nontraining data.

Hope this helps.

Thank you for formally accepting my answer

Greg

Answer 2

[Laura Lee]

Hi Greg,

Thank you for your response. I am slightly confused by your chosen dimensions though. Currently my design has a 63xN dim input matrix and a 2xN dim target matrix. The 2 classes of the target matrix are yes (1) or no (0). Why did you decide the target matrix should be 21xN? Also how did you come up with the complete input space containing 21^3 vectors? It has 21*3*N scalars, or 3*N vectors, each 21x1.

Also, using this design the performance is poor - and although the training data is a limitation, somebody has achieved better results using the same data.

My latest idea is to use find(X) to give the indices of the 1's as this is the only important information - the zeroes do not mean anything. Could providing a 3xN matrix, with 3 indices each between 1-21 be an improvement in design?

Thanks for your help, Laura

Answer 3

[Greg Heath]

There are 21 ways to choose each of the 3 groups. Therefore, there are 21^3 possible inputs. If there can be no duplicates this reduces to 21*20*19. Now you can choose your N inputs from one of these two pools.

The 21 X N output matrix contains a count {0,1,2,3} of how many input vectors are from each group.

You could use a more condensed 5-bit binary coding for each group and have a 15XN input matrix.

Hope this helps.

Thank you for formally accepting my answer

Greg