How do I hide the Cody Coursework Test Suite from Students?

With the introduction of Cody Coursework, I'm often asked how to hide the Test Suite in Cody Coursework.

Answer 1

[MathWorks Community Team]

First, we need to understand why MathWorks has made the Test Suite visible to students. Students are encouraged to use MATLAB on their computer to solve problems. To make this possible, students must be able to copy and paste the Test Suite code into a MATLAB script which, in turn, calls their solution function. Once satisfied, they then copy their solution function into Cody Coursework for assessment. For this reason, if your real concern regards student attempts to cheat the system, please first try the information offered here before using the approach discussed below.

If you're still determined to hide the test suite, I'd recommend hiding some and leaving some public.

First, compile a MATLAB function containing test suite code you'd like hidden as Protected MATLAB code.

Second, post this .p file somewhere online accessible by Cody Coursework. This must not be behind the school's firewall, it must be publicly accessible. One option is to use the public folder of a free dropbox account. If using DropBox, it will help to "copy the public link" using their web interface.

Third, implement a test case of your test suite to call your protected test suite function, for example:

   %%Execute Hidden Tests
   urlwrite('http://url/ProfsFunction.p','./ProfsFunction.p');
   rehash
   assert(ProfsFunction());

Fourth, add a test case to prevent student attempts to call on the .p file themselves, for example by parsing the students' solution file directly,

   %%Test for attempts to cheat
   found = 0;
   fid = fopen('StudentsFunction.m');
   while(~feof(fid))
      s = fgetl(fid);
      k = regexp(s,'urlwrite\(|ftp\(|mget\(|web\(|fprintf\(|fwrite\(|sprintf\(');
      if ~isempty(k)
          found = 1;
      end
   end
   fclose(fid);
   assert(isequal(found,0))

Fifth, use the solution map as a "line of last defense." Typically, honest solutions which are correct follow similar patterns. Horizontal bars of these patterns form on the solution map. The few stray solutions below these bars which Cody Coursework has marked correct merit manual code inspection.

There are many possibilities the power of the MATLAB language offers for implementing test suites. We will add more information and examples in future, and I encourage you to share your own insights as answers or comments below.

[C Lira]

When I try to implement your code under 'Third', I get

Error using ProfsFunction (line 2)
Not enough input arguments.
Error in testurl (line 3)
assert(ProfsFunction());

I was expecting that this was to test if the internet was working, but I don't understand what this is supposed to do. Doesn't that need to be a full assertion test of the output with some input values and an evaluation of the results?

[C Lira]

If you try this, recognize that students can run urlwrite on their own installation of Matlab. I verified it. They end up with ProfsFunction.p in their local directory. Then they can run all kinds of input to see what answers they should get. They can't read the file in plain text, but they can run without using Cody.

Also, be sure to suppress all output that you don't want students to see using semicolons, or the students will see it when they run the code on their machine. It does allow you to test their code against a random input, if you run your case and their case with the same input values, however, it does not effectively hide the output values from students for a given input.

[Jan]

See also Bob's related thread http://www.mathworks.com/matlabcentral/answers/119254-how-do-i-write-an-effective-test-suite-in-cody-coursework .

As I've added as comment there also: Please note, that the debugger can reveal very detailed, what's going on in a P-coded function. Therefore, and for some other reasons, I think that automatic grading of homework is a bad strategy for teaching. There cannot be a reliable replacement of controlling the student's code personally and it is a good way of "teaching" to use the chance to suggest improvements of the code.