Deep Learning Experiment Manager results are not reproducable

So I use the Experiment Manager (EM) for a regression project.

While, when I repeat the same experiment with the EM the resulting table shows very similar results in regards of the Validation RMSE. The label to predict represents 24 (hourly) energy prices of the next day.

Nevertheless whenever I use the same hyperparameter settings within my ordinary live script (pretty much the exact same code as the EM setup), I will get much worse Val RMSE - like by a factor of 2.

Is it possible, that the EM calculates the RMSE another way, than the ordinary Live Script / Visualization does?

[Khoo Yit Phang]

Experiment Manager should give you the same results as it calls trainNetwork under the hood. E.g., you should get the same results by calling your setup function and trainNetwork directly:

params.myLearningRate = 0.05;
[datastore, layergraph, options] = mySetupFunction(params);
[net, info] = trainNetwork(datastore, layergraph, options);
disp(info.FinalValidationRMSE);

Can you try the above and see if it gives you similar results?