Fitting problems like the one Hussein describes tend to be difficult to troubleshoot online, especially when the model/data cannot be shared. In order to get around this, Hussein and I set up a call to determine what was happening and found the problem: the parameter that was being estimated was also the subject of an initial assignment. This prevents the solver from being able to optimize the parameter value because with every iteration of the optimization the parameter value is being overwritten by the initial assignment. This does not only happen when estimating parameters that are subject to an initial assignment but generally when estimating parameters that are subject to any type of rule (initial/repeated assignment, algebraic or rate rule).
A symptom of this problem is - if the Progress Plot is enabled - you will see that the parameter value is not changing with every iteration, and/or that the optimization converges at iteration #1.
Other potential causes we investigated:
- Is the dosing correctly set up? If the dose is not set up correctly, the model is not being perturbed. Think of a PK model where there is no dose and all initial conditions are at 0: all states will remain 0 throughout the simulation. Even though the optimization algorithm will alter the parameter value during the estimation, because the model states remain 0 throughout the optimization, the model behavior will not change. There can be several reasons why a dose might not be set up correctly. Primarily, the TargetName for the dose might be incorrect. Alternatively, a parameter that would govern e.g. absorption of the dose to the central compartment might be set to zero, meaning that the dose does not reach the response variable.
- There could have been a problem with the ODE solver tolerances. This relates to the order of magnitude of the data you are trying to fit with respect to the order of magnitude of your tolerances. If the solver tolerances are on the order of magnitude (or larger) of your data, the error (associated with the accuracy of the solver) of your model of your simulation will be on the same order of magnitude as your data. As a result, the objective function is for not sensitive to your model response but rather picks up the error of the solver.
- There could have been a problem with the termination tolerances of the optimization algorithm. The optimization algorithms in SimBiology and MATLAB have optimization settings, which you can set with the ' options ' argument in sbiofit, with default values for each algorithm found here . E.g. for nlinfit, this options algorithm has to be a statset options structure. For nlinfit, the options has two name-value pairs associated that control the fitting tolerances: TolX tolerance on your estimated parameter) and TolFun (tolerance on the objective function). If you know the approximate order of magnitude of your the parameter you are estimating, you want to make sure your these tolerances are scaled accordingly. More on termination criteria can be found in this article.
General recommendations for troubleshooting fitting tasks
- Start small: bring the estimation task down to the smallest meaningful objective. If you want to estimate 10 parameter, try to start with one or two instead, make that work and then move up.
- Perform a sensitivity analysis to see if your model is sensitive to the parameters you are trying to estimate.
- Create a simulation task in the SimBiology desktop where you plot your data ( plot external data), together with sliders for the parameters you are trying to estimate. You can then see whether, by varying these parameter values, you can bring the model response in line with your data. This will also give you a good initial estimate for those parameters. If your model response does not change significantly in response to changes in your parameter(s) this could indicate that your model is not sensitive to those parameters (at least not locally). If all of this work however, you know that your dose, your model and your parameters work.
- Check whether you don't overrule the value of the parameter you are trying to estimate using a rule (initial/repeated assignment, algebraic rule, rate rule), see above.
- Check whether your data columns for doses and responses are correctly mapped to your model states (species).
- Check whether your units are consistent between your data and your model. You can use unit conversion to automatically achieve this.
- Check whether - for the fit task - the parameter values and initial conditions that are used for the model, make sense. You can do this by passing the relevant dose(s) and variant(s) to the getequations function. In the SimBiology App, you can look at your equation view (When you have your model open, in the Model Tab, click Open -> Equations). Subsequently, - in the Model tab - click "Show Tasks" and select your fit task and inspect the initial conditions for your parameters and species.
