ROC curve for the validation set

12 ビュー (過去 30 日間)
Uerm 2020 年 1 月 13 日
コメント済み: Silvia 2020 年 2 月 19 日 13:38
Hi,
I am working on a classification problem, where I use 10-fold cross-validation. I have made the code for the ROC curve for the training part of the data. I want to do the same for the validation set. How can I do that? My code is below:
indices = crossvalind('Kfold',trainingData(:,end),10);
for i = 1:10
test = (indices == i);
train = ~test;
% Linear SVM
classificationLinearSVM128 = fitcsvm(...
trainingData(train,1:end-1),...
trainingData(train,end), ...
'KernelFunction', 'linear', ...
'PolynomialOrder', [], ...
'KernelScale', 'auto', ...
'BoxConstraint', 1, ...
'Standardize', true, ...
'ClassNames', [0; 1]);
% Training
[predsLinSVM128train,~] = predict(classificationLinearSVM128,trainingData(train,1:end-1));
targetsLinSVM128train = trainingData(train,end);
[~,scoresLinSVM128] = resubPredict(fitPosterior(classificationLinearSVM128));
[xLinSVM128,yLinSVM128,~,aucLinSVM128] = perfcurve(trainingData(train,end),scoresLinSVM128(:,2),1);
% Validation
[predsLinSVM128test,~] = predict(classificationLinearSVM128,trainingData(test,1:end-1));
targetsLinSVM128test = trainingData(test,end);
[~,scoresLinSVM128test] = resubPredict(fitPosterior(classificationLinearSVM128));
%[xLinSVM128test,yLinSVM128test,~,aucLinSVM128test] = perfcurve(trainingData(test,end),scoresLinSVM128test(:,2),1);
end
figure()
subplot(121)
confusionchart(targetsLinSVM128train,predsLinSVM128train)
title('Linear SVM, training')
subplot(122)
confusionchart(targetsLinSVM128test,predsLinSVM128test)
title('Linear SVM, validation')
figure()
plot(xLinSVM128,yLinSVM128,'LineWidth',2)
xlabel('False Positive Rate')
ylabel('True Positive Rate')
title('ROC, Linear SVM')
I have tried the line that I have commented, but it does not work. Can anyone help with this? The positive class is 1.

0 件のコメント

サインイン to comment.

回答 (1 件)

Shashank Gupta 2020 年 1 月 21 日
Hi Uerm,
I don’t see any point, why “perfcurve" function should not work. You even used the same function in your training part and it seems to be working there. I am not really sure why it is not working. Also you can verify it by calculating FPR(False positive rate) and TPR(True positive rate) manually or through ROC function, plot the FPR vs TPR and see if it matches with the perfcurve function output.
Also let me know if you find something interesting. It is supposed to work perfectly. If you are getting any error while using the function post that as well.

2 件のコメント

Uerm 2020 年 1 月 24 日 13:51
Hi Shashank,
Thanks for your reply. Fortunately, I did manage to make it work for both the training part and the validation part.
Silvia 2020 年 2 月 19 日 13:38
Hi Uerm,
How did you solve the problem?
Thank you

サインイン to comment.

サインイン してこの質問に回答します。

R2019b

Translated by