| ### Chapter 9 - Faster Variations of Back-Propagation
Neural Smithing: Supervised Learning in Feedforward Artificial Neural Networks
Russell D. Reed and Robert J. Marks II
Copyright © 1999 Massachusetts Institute of Technology

## 9.9 Fuzzy Control of Back-Propagation

Network training is a dynamic process and the algorithms described in this chapter can be viewed as control systems whose purpose is to accelerate learning while avoiding instability. Fuzzy logic is a convenient way to convert a set of heuristics into a working algorithm and has been used with success in many simple control applications.

The main difference between fuzzy logic and conventional Boolean logic is that fuzzy logic deals with propositions that can have varying degrees of membership between true and false. This is useful for control applications because it allows the behavior to be described by easily understood IF-THEN rules that are interpolated to give smooth transitions between regions where different rules are active. Mechanisms of fuzzy inferencing are described in many references, so the details will be omitted here.

Many papers have been written on applications of fuzzy logic to neural network training. Control of back-propagation using fuzzy logic has been proposed by Arabshahi et al. [10], Choi et al. [81], and others. Arabshahi et al. [10] controls a single global learning rate while Choi [81] extends control to include the momentum term. Comparisons are made with standard back-propagation and Jacobs' delta-bar-delta rule on the 3-bit parity problem.

The central idea is to establish a set of IF-THEN rules for parameter control that are implemented using fuzzy logic. In Arabshahi et al. [10], the rule antecedants (the IF parts) are expressions involving the error E and the change in error CE = En - En-1 from one iteration to the next while the THEN parts (the rule consequents) specify Δη, how the learning rate should change given the conditions described in the antecedent. One rule might say that when E is low, and CE is low then the learning rate should increase by a small amount, for example. If both E and CE were actually low then this rule would be satisfied and the consequent would be asserted strongly. For slightly different values of E or CE, however, for example, when E is "lowish, but not really low," the rule would be satisfied less well and its consequent would be asserted less strongly.

At any particular time, many different rules will be satisfied to varying degrees and conflicts between active rules suggesting different actions are resolved by methods of fuzzy inferencing to obtain a single overall output.

In simple applications like this, fuzzy logic is used for interpolation. That is, the designer specifies the desired response at selected points on a grid in the input space and relies on fuzzy inferencing to interpolate between the points in a reasonable way. The designer thereby avoids the sometimes difficult problem of finding a clever algorithm or function that generates the desired response from the given inputs. An advantage of fuzzy systems, and local interpolation methods in general, is that the effects of each rule are relatively localized; rules don't interact globally so it is relatively easy to tune individual rules to improve local performance without worry that this will cause problems in other areas.

A fuzzy implementation of a set of heuristics, for example, the delta-bar-delta heuristics, will usually have the same basic behavior as the heuristics implemented by other means, although there may be small-scale differences. That is, the final performance of the system is determined much more by the quality of the heuristics than by the mechanisms of how they are implemented. Factors such as which input variables are considered, how they are represented, and how they are presumed to interact in their effects on the response will have much stronger effects on performance than whether they are implemented by fuzzy logic or by some other means. Fuzzy logic does not substitute for understanding a problem, but it is a convenient way to convert understanding into a working algorithm.

Books24x7.com, Inc. © 1999-2001 – Feedback