Neuro-Adaptive Learning and ANFIS

Suppose that you want to apply fuzzy inference to a system for which you already have a collection of input/output data that you would like to use for modeling, model-following, or some similar scenario. Also, assume that you do not necessarily have a predetermined model structure based on the characteristics of variables in your system. In some modeling situations, discerning membership functions parameters by looking at data can be difficult or impossible. In these cases, rather than choosing the parameters associated with a given membership function arbitrarily, you can tailor the membership function parameters to the input/output data. Using Fuzzy Logic Toolbox™ software, you can tune Sugeno fuzzy inference systems using neuro-adaptive learning techniques similar to those used for training neural networks.

Fuzzy Logic Toolbox software provides a command-line function (anfis) and an interactive app (Neuro-Fuzzy Designer) for training an adaptive neuro-fuzzy inference system (ANFIS).

FIS Structure

Using ANFIS training methods, you can train Sugeno systems with the following properties:

  • Single output

  • Weighted average defuzzification

  • First or zeroth order system; that is, all output membership functions must be the same type, either 'linear' or 'constant'.

  • No rule sharing. Different rules cannot use the same output membership function; that is, the number of output membership functions must equal the number of rules.

  • Unity weight for each rule.

  • No custom membership functions or defuzzification methods.

To create such a fuzzy system in the MATLAB® workspace, you can:

  • Use the genfis function. When using this method, you can create your system using either grid partitioning or subtractive clustering. Grid partitioning can produce a large number of rules when the number of inputs reaches four or five. To reduce the number of rules, consider using the subtractive clustering method.

  • Use the Fuzzy Logic Designer app, and export the FIS to the MATLAB workspace.

  • Use the sugfis function.

  • Load a system from a file using the readfis function.

When training your system using the anfis function, specify the initial structure by creating an anfisOptions option set and setting the InitialFIS property. If you do not specify this property, the anfis function derives the FIS structure using grid partitioning.

When using the Neuro-Fuzzy Designer, in the Generate FIS section, you can create your FIS by:

  • Loading from a file (select Load from file)

  • Loading from the MATLAB workspace (select Load from worksp)

  • Using grid partitioning (select Grid partition)

  • Using subtractive clustering (select Sub. clustering)

Training Data

To train a fuzzy system using neuro-adaptive methods, you must collect input/output training data using experiments or simulations of the system you want to model. In general, ANFIS training works well if the training data is fully representative of the features of the data that the trained FIS is intended to model.

To specify your training data, you can:

  • Create an array in the MATLAB workspace. Each row contains a data point, with the final column containing the output value and the remaining columns containing input values. You can then pass this data to the trainingData input argument of the anfis function or load it into the Neuro-Fuzzy Designer app.

  • Load the data from a .dat file. Each line of the file contains a data point with values separated by white space. The final value on each line is the output, and the remaining values are the inputs.

When using the anfis function, create or load the input data and pass it to the trainingData input argument.

When using Neuro-Fuzzy Designer, in the Load data section, select Training, and then:

  • To load data from a file, select file.

  • To load data from the MATLAB workspace, select worksp.

Training Options

Both anfis and the Neuro-Fuzzy Designer allow you to adjust the optimization method, number of training epochs, and training error goal. However, anfis provides additional training options to control the training step size.

OptionanfisOptions PropertyNeuro-Fuzzy Designer Setting
Optimization MethodOptimizationMethodIn the Train FIS section, specify Optim. Method.
Number of training epochsEpochNumberIn the Train FIS section, specify Epochs.
Training error goalErrorGoalIn the Train FIS section, specify Error Tolerance.
Initial step sizeInitialStepSizeNot available
Step-size decrease rateStepSizeDecreaseRate
Step-size increase rateStepSizeIncreaseRate

Optimization Method

To train a fuzzy system using ANFIS, the Fuzzy Logic Toolbox software uses a back-propagation algorithm either alone or in combination with a least-squares algorithm. This training process tunes the membership function parameters of a FIS such that the system models your input/output data.

The following table shows the two methods that both anfis and the Neuro-Fuzzy Designer use for updating membership function parameters.

Optimization MethodanfisOptions SettingNeuro-Fuzzy Designer Setting
Backpropagation for all parameters (a steepest-descent method)OprimizationMethod = 'backpropagation'In the Train FIS section, under Optim. Method, select backpropa.
Hybrid method consisting of backpropagation for the parameters associated with the input membership functions, and least squares estimation for the parameters associated with the output membership functionsOprimizationMethod = 'hybrid'In the Train FIS section, under Optim. Method, select hybrid.

Step Size

When training using the anfis function, you can adjust the training step size options. During training, the software updates the step-size according to the following rules:

  • If the error undergoes four consecutive reductions, increase the step size by multiplying it by a constant (StepSizeIncreaseRate) greater than one.

  • If the error undergoes two consecutive combinations of one increase and one reduction, decrease the step size by multiplying it by a constant (StepSizeDecreaseRate) less than one.

Ideally, the step size increases at the start of training, reaches a maximum, and then decreases for the remainder of the training. To achieve this step size profile, adjust the initial step size (InitialStepSize), step-size increase rate, and step-size decrease rate.

Display Options

When training using the anfis function, you can specify what training progress information to display in the MATLABCommand Window. Using an anfisOptions option set, you can set the following display options.

  • DisplayANFISInformation — Display ANFIS information at the start of training

  • DisplayErrorValues — Display the training error at each epoch

  • DisplayStepSize — Display the step-size each time it changes.

  • DisplayFinalResults — Display the final training error and validation error

The Neuro-Fuzzy Designer does not provide user-specified display options. Instead, it displays the training progress as a plot.

Training Validation

Validation data lets you check the generalization capability of your trained fuzzy inference system. The validation data should fully represent the features of the data the FIS is intended to model, while also being sufficiently different from the training data to test training generalization. The software uses this data set to cross-validate the fuzzy inference model by applying the validation data to the model and seeing how well the model responds to this data.

Model validation is useful in the following situations:

  • Noisy data — In some cases, data is collected using noisy measurements, and the training data is unable to represent all the features of the data the FIS is intended to model.

  • Overfitting — Since the model structure used for ANFIS is fixed with a large number of parameters, there is a tendency for the model to overfit the data on which it is trained, especially when using a large number of training epochs. If overfitting does occur, the trained FIS may not generalize well to other independent data sets.

The idea behind using a checking data set for model validation is that, after a certain point in the training process, the model begins overfitting the training data set. In principle, the model error for the checking data set decreases up to the point that overfitting begins. After this point, the model error for the checking data increases. Overfitting is accounted for by testing the trained FIS against the checking data, and choosing the membership function parameters to be those associated with the minimum checking error if these errors indicate model overfitting.

Usually, the training and checking data sets are collected based on observations of the target system and are then stored in separate files. To specify validation data when using the:

  • anfis function, create an anfisOptions object, and set the ValidationData option.

  • Neuro-Fuzzy Designer, in the Load data section, select Checking.

The array and file formats for the checking data are the same as those for the training data.

Training Results

When you train your fuzzy system using the anfis function, you can obtain the following trained fuzzy systems:

  • The fis output argument is the fuzzy system for which the training error is minimum. This system is always returned by the anfis function, and corresponds to the FIS returned by Neuro-Fuzzy Designer when you do not specify checking data.

  • The chkFIS output argument is the fuzzy system for which the validation error is minimum. This system is returned only when you specify validation data using anfisOptions, and corresponds to the FIS returned by Neuro-Fuzzy Designer when you specify checking data. This FIS object is the one that you should use for further calculation if checking data is used for cross-validation.

You can obtain the error associated with each of the trained fuzzy systems. In each case, the returned error is the root mean squared error (RMSE), and is returned as a vector. Each element of the vector is the RMSE error value at each training epoch.

  • Training error — Difference between the training data output value and the output of the fuzzy inference system for the corresponding training data input values.

  • Validation error — Difference between the checking data output value and the output of the fuzzy inference system for the corresponding checking data input values. This error is returned only when you specify validation data using anfisOptions.

During training, the Neuro-Fuzzy Designer app plots the training and checking error for each training epoch. Exporting training and checking error from the app is not supported. To obtain the training error, you must retrain the system from the command-line. For an example, see Save Training Error Data to MATLAB Workspace.

To further test your trained fuzzy system, you can use an additional set of testing data that you did not use for training or validation. To do so:

  • When training a system at the command-line, use the evalfis function.

  • When using Neuro-Fuzzy Designer, in the Load data section, select Testing, and click Load Data. To evaluate the trained system for any loaded data set, in the Test FIS section, select a data set, and click Test Now.

Training Algorithm Differences

The Neuro-Fuzzy Designer app manages training epochs in a manner different from the anfis function. This difference produces variations in the training results.

To train a system for N epochs at the command line, you call the anfis function one time, specifying the number of epochs as N. However, the Neuro-Fuzzy Designer app calls the anfis function N times, specifying the number of epochs as 2 each time.

For a command-line example that demonstrates the Neuro-Fuzzy Designer training algorithm, see Save Training Error Data to MATLAB Workspace.

References

[1] Jang, J.-S. R., "Fuzzy Modeling Using Generalized Neural Networks and Kalman Filter Algorithm," Proc. of the Ninth National Conf. on Artificial Intelligence (AAAI-91), pp. 762-767, July 1991.

[2] Jang, J.-S. R., "ANFIS: Adaptive-Network-based Fuzzy Inference Systems," IEEE Transactions on Systems, Man, and Cybernetics, Vol. 23, No. 3, pp. 665-685, May 1993.

[3] Jang, J.-S. R. and N. Gulley, "Gain scheduling based fuzzy controller design," Proc. of the International Joint Conference of the North American Fuzzy Information Processing Society Biannual Conference, the Industrial Fuzzy Control and Intelligent Systems Conference, and the NASA Joint Technology Workshop on Neural Networks and Fuzzy Logic, San Antonio, Texas, Dec. 1994.

[4] Jang, J.-S. R. and C.-T. Sun, "Neuro-fuzzy modeling and control," Proceedings of the IEEE, March 1995.

[5] Jang, J.-S. R. and C.-T. Sun, Neuro-Fuzzy and Soft Computing: A Computational Approach to Learning and Machine Intelligence, Prentice Hall, 1997.

[6] Wang, L.-X., Adaptive fuzzy systems and control: design and stability analysis, Prentice Hall, 1994.

[7] Widrow, B. and D. Stearns, Adaptive Signal Processing, Prentice Hall, 1985.

See Also

Apps

Functions

Related Topics