Test Data Against Trained System

Checking Data Helps Model Validation

In this section, we look at an example that loads similar training and checking data sets. The checking data set is corrupted by noise.

Loading Data

To work both of the following examples, you load the training data sets (fuzex1trnData and fuzex2trnData) and the checking data sets (fuzex1chkData and fuzex2chkData), into the Neuro-Fuzzy Designer from the workspace. You may also substitute your own data sets.

To load the data sets from the workspace into the Neuro-Fuzzy Designer:

  1. Type the following commands at the MATLAB® command line to load the data sets into the MATLAB workspace:

    load fuzex1trnData.dat
    load fuzex2trnData.dat
    load fuzex1chkData.dat
    load fuzex2chkData.dat
    

  2. Open the Neuro-Fuzzy Designer by typing neuroFuzzyDesigner in the MATLAB command line.

  3. To load the training data set from the workspace:

    1. In the Load data portion of the designer, select the following options:

      • Type: Training

      • From: worksp

    2. Click Load Data to open the Load from workspace dialog box.

    3. Type fuzex1trnData as shown in the following figure, and click OK.

      The training data set is used to train a fuzzy system by adjusting the membership function parameters that best model this data, and appears in the plot in the center of the app as a set of circles.

      The horizontal axis is marked data set index. This index indicates the row from which that input data value was obtained (whether or not the input is a vector or a scalar).

  4. To load the checking data set from the workspace:

    1. In the Load data section, select Checking in the Type column.

    2. Click Load Data to open the Load from workspace dialog box.

    3. Type fuzex1chkData as the variable name and click OK.

      The checking data appears in the plot as pluses superimposed on the training data.

      The next step is to specify an initial fuzzy inference system for anfis to train.

Initializing and Generating Your FIS

You can either initialize the FIS parameters to your own preference, or if you do not have any preference for how you want the initial membership functions to be parameterized, you can let anfis initialize the parameters for you, as described in the following sections:

Automatic FIS Structure Generation

To initialize your FIS using anfis:

  1. Choose Grid partition, the default partitioning method. The two partition methods, grid partitioning and subtractive clustering, are described in genfis.

  2. Click Generate FIS . Clicking this button displays a menu from which you can choose the number of membership functions, MFs, and the type of input and output membership functions. There are only two choices for the output membership function: constant and linear. This limitation of output membership function choices is because anfis only operates on Sugeno-type systems.

  3. Fill in the entries as shown in the following figure, and click OK.

You can also implement this FIS generation from the command line using genfis.

Specifying Your Own Membership Functions for ANFIS

You can choose your own preferred membership functions with specific parameters to be used by anfis as an initial FIS for training.

To define your own FIS structure and parameters:

  1. Open the Membership functions menu item from the Edit menu.

  2. Add your desired membership functions (the custom membership option will be disabled for anfis). The output membership functions must either be all constant or all linear. For carrying out this and the following step, see The Fuzzy Logic Designer and The Membership Function Editor.

  3. Select the Rules menu item in the Edit menu, and use the Rule Editor to generate the rules (see The Rule Editor).

  4. Select the FIS Properties menu item from the Edit menu. Name your FIS, and save it to either the workspace or to file.

  5. Click the Close button to return to the Neuro-Fuzzy Designer to train the FIS.

  6. To load an existing FIS for ANFIS initialization, in the Generate FIS portion of the designer, click Load from worksp or Load from file. You load your FIS from a file if you have saved an FIS previously that you would like to use. Otherwise you load your FIS from the workspace.

Viewing Your FIS Structure

After you generate the FIS, you can view the model structure by clicking the Structure button in the middle of the right side of the editor. A new editor appears, as follows.

The branches in this graph are color coded. Color coding of branches characterize the rules and indicate whether or not and, not, or or are used in the rules. The input is represented by the left-most node and the output by the right-most node. The node represents a normalization factor for the rules. Clicking on the nodes indicates information about the structure.

You can view the membership functions or the rules by opening either the Membership Function Editor, or the Rule Editor from the Edit menu.

ANFIS Training

The two anfis parameter optimization method options available for FIS training are hybrid (the default, mixed least squares and backpropagation) and backpropa (backpropagation). Error Tolerance is used to create a training stopping criterion, which is related to the error size. The training will stop after the training data error remains within this tolerance. This is best left set to 0 if you are unsure how your training error may behave.

Note

If you want to save the training error data generated during ANFIS training to the MATLAB workspace, you must train the FIS at the command line. For an example, Save Training Error Data to MATLAB Workspace.

To start the training:

  1. Leave the optimization method at hybrid.

  2. Set the number of training Epochs to 40 (the default value is 3).

  3. Select Train Now.

    The following window appears on your screen.

    The plot shows the checking error as ♦ ♦ on the top . The training error appears as * * on the bottom. The checking error decreases up to a certain point in the training, and then it increases. This increase represents the point of model overfitting. anfis chooses the model parameters associated with the minimum checking error (just prior to this jump point). This example shows why the checking data option of anfis is useful.

Testing Your Data Against the Trained FIS

To test your FIS against the checking data, select Checking data in the Test FIS portion of the Neuro-Fuzzy Designer, and click Test Now. When you test the checking data against the FIS, it looks satisfactory.

Loading More Data with anfis

If you load data into anfis after clearing previously loaded data, you must make sure that the newly loaded data sets have the same number of inputs as the previously loaded ones did. Otherwise, you must start a new Neuro-Fuzzy Designer session from the command line.

Checking Data Option and Clearing Data

If you do not want to use the checking data option of anfis, then do not load any checking data before you train the FIS. If you decide to retrain your FIS with no checking data, you can unload the checking data in one of two ways:

  • Select the Checking option button in the Load data portion of the Neuro-Fuzzy Designer, and then click Clear Data to unload the checking data.

  • Close the Neuro-Fuzzy Designer, and go to the MATLAB command line, and retype neuroFuzzyDesigner. In this case you must reload the training data.

After clearing the data, you must regenerate your FIS. After the FIS is generated, you can use your first training experience to decide on the number of training epochs you want for the second round of training.

Checking Data Does Not Validate Model

This example examines what happens when the training and checking data sets are sufficiently different. To see how the Neuro-Fuzzy Designer can be used to learn something about data sets and how they differ:

  1. Clear the Neuro-Fuzzy Designer:

    • Clear both the training and checking data.

    • (optional) Click the Clear Plot button on the right.

  2. Load fuzex2trnData and fuzex2chkData (respectively, the training data and checking data) from the MATLAB workspace just as you did in the previous example.

You should see a plot similar to the one in the following figure. The training data appears as circles superimposed with the checking data, appearing as pluses.

Train the FIS for this system exactly as you did in the previous example, except now choose 60 Epochs before training. You should get the following plot, showing the checking error as ♦ ♦ on top and the training error as * * on the bottom.

In this case, the checking error is quite large. It appears that the minimum checking error occurs within the first epoch. Using the checking data option with anfis automatically sets the FIS parameters to be those associated with the minimum checking error. Clearly this set of membership functions is not the best choice for modeling the training data.

This example illustrates the problem discussed earlier wherein the checking data set presented to anfis for training was sufficiently different from the training data set. As a result, the trained FIS did not capture the features of this data set very well. It is important to know the features of your data set well when you select your training and checking data. When you do not know the features of your data, you can analyze the checking error plots to see whether or not the checking data performed sufficiently well with the trained model.

In this example, the checking error is sufficiently large to indicate that either you need to select more data for training or modify your membership function choices (both the number of membership functions and the type). Otherwise, the system can be retrained without the checking data, if you think the training data sufficiently captures the features you are trying to represent.

To complete this example, test the trained FIS model against the checking data. To do so, select Checking data in the Test FIS portion of the app, and click Test Now. The following plot in the app indicates that there is quite a discrepancy between the checking data output and the FIS output.

See Also

Related Topics