Polyspace BugFinder - detect new errors in CI environment and fail build
4 ビュー (過去 30 日間)
古いコメントを表示
Within the Polyspace metrics server, you are able to compare two versions and use the checkbox 'New Findings Only' to filter the new errors/defects created between the two versions.
I would like to replicate this within a CI environment using Jenkins, where a build becomes "Unstable" or "Failed" on new errors.
We are using Polyspace 2018b to run BugFinder on our project, and then uploading the results to the metrics server with the verification version set as our build number. We are then using the MATLAB API to extract the results from a matlab table into a CSV format, using the following code:
resObj = polyspace.BugFinderResults(cd).getResults() %cd is bugfinder results directory
writetable(resObj, 'bugfinder-results.csv')
We are then parsing the results to show summary, defect overview, MISRA violations etc as plots (using plots plugin). We are also checking for any values 'yes' in the "New" column, and creating a Pass/Fail JUnit test case, for Jenkins to read and mark the build unstable if there are any new errors found in polyspace.
The issue we are having is that when pulling the latest results and extrating the rows from the "New" column. all rows show 'no' when on the metrics server we can see new errors introduced. I was expecting it to store the new errors against the latest result itself. Is this not the case? We have also tried first downloading the results of the previous build, running BugFinder on the latest code (using the same directory of the previous results) however this does not make any difference. Due to our workspace and build agents setup, the same branch of code does not always run on the same build agent, and the workspace is cleaned on every build.
Is it possible to detect new errors outside the metrics server, and how can we go about adding this to a CI environment?
Please let me know if you need further information, and thank you for your assistance in advance.
0 件のコメント
回答 (1 件)
Hitesh
2025 年 1 月 3 日
編集済み: Hitesh
2025 年 1 月 3 日
Hi Balaal,
Kindly refer to the following ML Answer on configuring Polyspace with Jenkins using MATLAB R2018b and earlier versions:
You need to ensure that Jenkins use a post-build action to archive the CSV result file to a persistent storage location for avoiding any conflicts between metrics server and the csv files.
I am assuming that you have CSV files from two different builds and that each error can be uniquely identified using a combination of relevant fields such as "error ID", "file name", and "line number".Kindly refer to the following code as example to identify the new Error and generateJunitReport for it:
function detectNewErrors(previousCsv, currentCsv, junitOutput)
% Read the previous and current results from CSV files
previousResults = readtable(previousCsv);
currentResults = readtable(currentCsv);
% Extract unique identifiers for each error
% Assuming each error can be identified by a combination of 'ErrorID', 'File', and 'Line'
previousErrors = strcat(previousResults.ErrorID, '_', previousResults.File, '_', string(previousResults.Line));
currentErrors = strcat(currentResults.ErrorID, '_', currentResults.File, '_', string(currentResults.Line));
% Find new errors in the current results
newErrors = setdiff(currentErrors, previousErrors);
% Display new errors
if ~isempty(newErrors)
disp('New errors detected:');
for i = 1:length(newErrors)
disp(newErrors(i));
end
% Generate a JUnit report indicating failure
generateJUnitReport(newErrors, junitOutput, false);
else
disp('No new errors detected.');
% Generate a JUnit report indicating success
generateJUnitReport(newErrors, junitOutput, true);
end
end
function generateJUnitReport(newErrors, junitOutput, isSuccess)
% Open file for writing
fid = fopen(junitOutput, 'w');
% Write XML header
fprintf(fid, '<?xml version="1.0" encoding="UTF-8"?>\n');
fprintf(fid, '<testsuites>\n');
fprintf(fid, ' <testsuite name="Polyspace New Error Check" tests="%d" failures="%d">\n', length(newErrors), ~isSuccess);
if isSuccess
% Write a passing test case if no new errors
fprintf(fid, ' <testcase classname="Polyspace" name="NoNewErrors"/>\n');
else
% Write a failing test case for each new error
for i = 1:length(newErrors)
fprintf(fid, ' <testcase classname="Polyspace" name="%s">\n', newErrors{i});
fprintf(fid, ' <failure type="NewError">New error detected: %s</failure>\n', newErrors{i});
fprintf(fid, ' </testcase>\n');
end
end
% Close XML tags
fprintf(fid, ' </testsuite>\n');
fprintf(fid, '</testsuites>\n');
% Close file
fclose(fid);
end
For more information regarding "Using Polyspace Products in Continuous Integration and DevOps Workflows", refer to the following MATLAB documentation:
0 件のコメント
参考
カテゴリ
Help Center および File Exchange で Bug Finder Analysis についてさらに検索
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!