Backpropogation

Proceedings of the 2005 Informing Science and IT Education Joint Conference

Speeding Up Back-Propagation Neural Networks
Mohammed A. Otair Jordan University of Science and Technology, Irbed, Jordan
otair@just.edu.jo

Walid A. Salameh Princess Summaya University for Science and Technology, Amman, Jordan
walid@psut.edu.jo

Abstract
There are many successful applications of Backpropagation (BP) for training multilayer neural networks. However, it has many shortcomings. Learning often takes long time to converge, and it may fall into local minima. One of the possible remedies to escape from local minima is by using a very small learning rate, which slows down the learning process. The proposed algorithm presented in this study used for training depends on a multilayer neural network with a very small learning rate, especially when using a large training set size. It can be applied in a generic manner for any network size that uses a backpropgation algorithm through an optical time (seen time). The paper describes the proposed algorithm, and how it can improve the performance of backpropagation (BP). The feasibility of proposed algorithm is shown through out number of experiments on different network architectures. Keywords: Neural Networks, Backpropagation, Modified backprpoagation, Non-Linear function, Optical Algorithm.

Introduction
The Backpropagation (BP) algorithm (Rumelhart, Hinton, & Williams, 1986; Rumelhart, Durbin, Golden, & Chauvin, 1992) is perhaps the most widely used supervised training algorithm for multi-layered feedforward neural networks. However, in some cases, the standard Backpropagation takes unendurable time to adapt the weights between the units in the network to minimize the mean squared errors between the desired outputs and the actual network outputs (Callan, 1999; Carling, 1992; Freeman, &, Skapura, 1992; Hakin, 1999; Maureen, 1993). There has been much research proposed to improve this algorithm; some of this research was...