Copyright 1996-2000 by Donald R. Tveter,
commercial use is prohibited. Short quotations are permitted
provided proper attribution is given.
Up to Backpropagator's Review
Last Change to This File: January 20, 2000
For people who need a really basic treatment of backprop
including a simple numerical example and deriving the formulas
I've put together the following:
- a 12 page postscript file
derived from the not yet finished user manual for my Professional Backprop Software.
Note that there is free Unix,
Mac and PC software that will let you read and print
postscript files on these systems. If you're serious about
neural networking you will need postscript on your system
some day because all the online papers use it.
- An HTML Version. This is new
as of January 20, 2000 and composed of mostly HTML with some
gifs for the figures, a total of about 100k. Mathematical
symbols will come out screwy under X unless you take some
extra steps, see:
http://hutchinson.belmont.ma.us/tth/Xfonts.html. The quick fix described
at the beginning of the document gives small size math symbols
but that's nice enough that I did not try any of the other
There is a short
online article by G. Thimm, P. Moerland and
E. Fiesler that relates the gain to the learning rate and the size
of the initial weights.
Backprop training can be quite complicated and to do it properly see the
online article by Lutz Prechelt.
Lutz Prechelt has established a page on benchmarking algorithms at the
University of Karlsruhe, Germany.
More background can be found in
two online articles by Warren Sarle.
For an overview of some of the more well known variations on backprop
see the article by
If you have any questions or comments, write me.
To Don's Home Page