This window deals with the parameters used by the algorithm that adds hidden layer units to a network while training is in progress. The idea is that adding an extra unit will get the network out of a local minimum.
There is no known way to predict how many hidden units are needed to give the best generalization. One way to cope with this problem is to start out with a single hidden unit and if the network is not converging or not converging quickly enough add another hidden unit and another and another until it finally does converge. The first method proposed to do this was dynamic node creation by Timur Ash.
The Ash algorithm keeps track of the recent history of errors (call this array neterror). The number of recent errors stored will be the variable, window. The Ash algorithm keeps track of when the last hidden unit was added (make this variable, lastadd) and the error at that point (make this variable lastadderror). Now, if at time t, t >= window and the error is not going down fast enough then a hidden unit is added. "Not fast enough" is defined by:
(neterror[t-window] - neterror[t]) / lastadderror < triggerwhere trigger is some value between 0 and 1. With this arrangement at least window iterations must be done after adding a hidden unit before another unit can be added.
The typed version of the command has four sub-commands within it to set the network building parameters. In the command:
n w 25 t 0.1 m 10 f+the w sets the window to 25 iterations, t sets the trigger to 0.1 and the m sets the maximum number of hidden units that can be added to 10. The f parameter is + or - to freeze or not freeze the set of weights in the network, this is done before the new node and weights are added, the new weights will of course be changed by the training algorithm. There are entry boxes and buttons in the network building window to set these parameters.
Setting the network building algorithm is done in `a' command, or by clicking the Network Building Algorithm button which brings up a menu. The possible options are no network building, use the Ash algorithm or add nodes configured for the pseudo-cascade correlation style network. The typed version of the commands are:
a n - * no network building a n a * Ash algorithm a n p * pseudo-cascade correlation
If you use the network building capability within a recurrent network, adding additional hidden units will cause additional input units to be added.