How do we decide the number of hiddenlayers in a PatternNet?

Hi,

Well, since I'm working on PatternNets (Classification), my doubt is, how do we decide then number of hiddenlayers the network requires.

Before that, I have 8 predictors and 1 output response containing 4 classes.

I know that in order to define a hidden layer, we use :

MyNet = patternnet(10);

But how do I decide I need 10 and not 8 or any other number?

Any suggestion would be helpful. Thank you.

Answer 1

[Greg Heath]

1. patternnet(10) indicates ONE HIDDEN LAYER WITH TEN NODES
2. It is important to be mindful of the number of layers and nodes.
3. The set of input nodes IS NOT CONSIDERED A LAYER ( Probably because no mathematical operations are performed there)
4. Three sets of nodes (INPUT/HIDDEN/OUTPUT) is sufficient. Let

size(inputmatrix) = [ I N ]

size(targetmatrix) = [ O N ]

Then with a single middle "HIDDEN" layer of size H,

The size of the typical two layer net is

size(net) = I - H - O

a. I input nodes

b. H hidden layer nodes

c. O output layer nodes

4. The total number of weights is

Nw = ( I + 1 ) * H + ( H + 1 ) * O

= O + ( I + O + 1 ) * H

where the "1s" represent constant bias connections

5. The data is typically divided into a training subset of size Ntrn ~ 0.7*N, a validation subset of size Nval ~ 0.15*N and a test subset of size Ntst ~ 0.15*N.

6. For good approximate solutions, the number of training equations Ntrneq = Ntrn*O should be sufficiently larger than the number of unknown weights Nw

7. The inequality Nw << Ntrneq results in

H << ( 0.7 * N * O - O )/(I+O+1)

8. A factor of 10 is usually sufficient

Hope this helps.

Greg

[abhisrisai]

Hello Greg,

Thank you very much for the answer, it was helpful. Just another doubt, what happens to the prediction performance when we have abundant H nodes. In my case, I have 8 predictors and I feel 10 H nodes are sufficient. What if I up the number to 40 H nodes? Does it affect my prediction and how?

Thank you.