what is the code of plotroc only in testing sample

I use the nprtool for the study in matlab, the code as following:

% Solve a Pattern Recognition Problem with a Neural Network

% Script generated by Neural Pattern Recognition app

% Created 01-Apr-2019 17:25:49

%

% This script assumes these variables are defined:

%

% xdata - input data.

% ydata - target data.

x = xdata';

t = ydata';

% Choose a Training Function

% For a list of all training functions type: help nntrain

% 'trainlm' is usually fastest.

% 'trainbr' takes longer but may be better for challenging problems.

% 'trainscg' uses less memory. Suitable in low memory situations.

trainFcn = 'trainscg'; % Scaled conjugate gradient backpropagation.

% Create a Pattern Recognition Network

hiddenLayerSize = 20;

net = patternnet(hiddenLayerSize, trainFcn);

% Choose Input and Output Pre/Post-Processing Functions

% For a list of all processing functions type: help nnprocess

net.input.processFcns = {'removeconstantrows','mapminmax'};

% Setup Division of Data for Training, Validation, Testing

% For a list of all data division functions type: help nndivision

net.divideFcn = 'dividerand'; % Divide data randomly

net.divideMode = 'sample'; % Divide up every sample

net.divideParam.trainRatio = 70/100;

net.divideParam.valRatio = 15/100;

net.divideParam.testRatio = 15/100;

% Choose a Performance Function

% For a list of all performance functions type: help nnperformance

net.performFcn = 'crossentropy'; % Cross-Entropy

% Choose Plot Functions

% For a list of all plot functions type: help nnplot

net.plotFcns = {'plotperform','plottrainstate','ploterrhist', ...

'plotconfusion', 'plotroc'};

% Train the Network

[net,tr] = train(net,x,t);

% Test the Network

y = net(x);

e = gsubtract(t,y);

performance = perform(net,t,y)

tind = vec2ind(t);

yind = vec2ind(y);

percentErrors = sum(tind ~= yind)/numel(tind);

% Recalculate Training, Validation and Test Performance

trainTargets = t .* tr.trainMask{1};

valTargets = t .* tr.valMask{1};

testTargets = t .* tr.testMask{1};

trainPerformance = perform(net,trainTargets,y)

valPerformance = perform(net,valTargets,y)

testPerformance = perform(net,testTargets,y)

% View the Network

view(net)

Question: plotroc(t,y) can acquired ALL ROC curve, but what is the code of plotroc only in testing sample? Thank you for answering.

Answer 1

[Aditya]

Hi Zhou,

By default, the plotroc function in MATLAB's Neural Network Toolbox will plot the ROC curve using all data (training, validation, and testing samples together) when you provide it with the entire target (t) and output (y) matrices. However, if you are interested in evaluating your neural network's performance specifically on the test set, you need to extract only the test samples from your data before plotting the ROC curve.

After training your network, the train function returns a structure tr that contains logical masks indicating which samples were used for training, validation, and testing. For sample-wise division (which is the default in your code), you can use tr.testMask{1} to identify the test samples. By applying this mask to your target and output matrices, you can isolate the test data.

Here is how you can do this in MATLAB:

% Extract the logical mask for test samples
testMask = tr.testMask{1};

% Use the mask to get only test targets and outputs
t_test = t(:, testMask);
y_test = y(:, testMask);

% Plot the ROC curve for the test set only
plotroc(t_test, y_test)

In this code, t_test and y_test contain only the columns corresponding to the test samples. When you call plotroc(t_test, y_test), MATLAB will display the ROC curve strictly for the test data, allowing you to assess the generalization performance of your neural network more accurately.

This approach can easily be adapted for the training or validation sets by using tr.trainMask{1} or tr.valMask{1} instead, helping you to visualize and compare ROC curves for different subsets of your data.