Class-based tests with CurrentFolderFixture excluded from testsuite unless run manually

I'm getting started with using class-based unit tests on my code, and I want to create a testsuite script to run all of my test methods. Right now I have each test in its own subfolder of a folder (UnitTest/Module/Submodule1,UnitTest/Module/Submodule2, etc), and I'm calling:

suite = testsuite('UnitTest/Module', 'IncludingSubfolders', true);

The error I'm having is that the tests are being excluded with the following warning messages:

Warning: "C:\MyProject\UnitTest\Module\Submodule1\Submodule1UnitTest.m" was excluded.
Caused by:
Error: File: Submodule1UnitTest.m Line: 6 Column: 38
Folder "UnitTest/Module/Submodule1/" cannot be found. 
> In matlab.unittest.internal.InvalidTestFactory/createSuiteImplicitly (line 37)
In matlab.unittest.TestSuite.fromFolder (line 583)
In matlab.unittest.TestSuite>@(fld)TestSuite.fromFolder(fld,selector,'IncludingSubfolders',includeSubfolders) (line 599)
In matlab.unittest.TestSuite.fromFolder (line 599)
In testsuite>createSuite (line 221)
In testsuite (line 128)
In ModuleRunAllTests (line 1)

The line it's pointing to as the error is where I define the CurrentFolderFixture test fixture.

%% Submodule1 Unit Test
% Author: Pi
%
% Created: 10/27/2022
classdef (SharedTestFixtures={matlab.unittest.fixtures.CurrentFolderFixture( ...
        'UnitTest/Module/Submodule1/')}) ...
        Submodule1UnitTest < BaselineTestCase

BaselineTestCase is inheriting from matlab.unittest.TestCase.

What's weird is that the test is excluded unless I manually run the test in the matlab session first. Once I've manually run the test outside of the testsuite, then the testsuite will include the test and run it as expected. The script for generating the testsuite currently lives in 'MyProject/UnitTest/', and I am running the script with my working directory as 'MyProject/', but I have also had the same problem when running with directly from 'MyProject/UnitTest/'.

Any ideas what could be causing the tests to be excluded, or why running the tests manually would allow testsuite to find them?

Answer 1

[Andrew Pyzdek]

%% Submodule1 Unit Test

% Author: Pi

%

% Created: 09/27/2022

classdef (SharedTestFixtures={... % Run in this folder with ASM Project

matlab.unittest.fixtures.CurrentFolderFixture(fileparts(mfilename('fullpath'))), ...

matlab.unittest.fixtures.ProjectFixture(fullfile(fileparts(mfilename('fullpath')),'../../..'))}) ...

Submodule1UnitTest < BaselineTestCase

I found a workaround for this issue. My problem stemmed from using a relative path for CurrentFolderFixture, which behaved differently when the class was already loaded into memory as opposed to being invoked initially as part of testsuite. Using mfilename('fullpath') to explicitly create the absolute filepath directly from the m-file containing the class fixes the issue, albeit in a clumsy way. Ultimately, it would help if testsuite() didn't change the working directory to each subfolder during operation, but without that option this is the best workaround I could come up with.

Answer 2

[Marek Plotka]

In matlab version since R2022b you can use 'InvalidFileFoundAction' parameter set to 'error':

https://www.mathworks.com/help/matlab/ref/testsuite.html?s_tid=doc_ta