Unit testing on Simbiology created model

I am working in a university department where various PBPK models using Simbiology are created based upon a standard PBPK model template. I have been asked to look into the feasibility of performing unit testing on the standard adult human PBPK model template that is used as the bases of all models created and to apply these unit tests to the other PBPK models applications to check that modifications performed are only application specific. These units for example may be characteristics such as anatomy rules (weight, height, organ volumes ) which are less likey to be changed between models.

I understand that the are no unit testing features in the Simbiology desktop app but models can be exported in to theMATLAB environment where unit testing frameworks exist. However when exporting the Simbiology models into MATLAB, the models are exported as objects rather than as source code and the object data is returned as a character vector using commands such as getequations(). The returned data does not appear to be in a directly useable when trying to disect the overall model into small chunks (units) and check for expected and unexpected outputs when defined inputs are applied to that unit. The object commands available in MATLAB appear to be focussed on creating/modifying models and running full model simulations.

Overall, I am trying to assertain if there are any methods that enable Simbiology models to be broken down into smaller components and allow tesing to be done on those components only. We have a manual comparison tool for checking differences between the component files (parameters,species, compartments, rules, reactions etc) of two models but looking for a more robust and automated approach to checking models. I would appreciate any suggestions on how unit testing could be done on Simbiology models or any other approaches that I could utilise.

Answer 1

[Florian Augustin]

Hi Andrew,

That's an interesting project you are investigating! Below are a couple thoughts on how you could go about testing building of your model's building blocks (I will be referring to "units" as ”building blocks”, just to avoid confusion with SimBiology’s unit system or MATLAB’s unit testing framework).

Here are two ideas for how you could test the building blocks:

1. Simulation based: For this you would have to do a manual step and split your template model into the building blocks. Just to name an example, if the liver is a building block then delete all reactions, parameters, species that are not part of, feed into, or are an output of the liver block. Note that this requires to remember to make any changes to your template model to the template building blocks as well (you can use your testing framework to ensure that's the case, more on this later). To test the derived model against the template, you could just simulate it and record the input/outputs of building blocks that are expected to be unchanged from the template (e.g. inflow and outflow of liver). Then use the inflow as initial conditions for the template building-block and verify that the outputs match. To ensure that any updates to the template model are also made to the template building-block, you could use the same testing framework: simulate it the template model, then use the inflows/outflows to verify that your template building blocks behave as expected.
2. Model based: The simulation results of your template model are determined by the reaction rates, rules, and initial values of the components in the model object (if doses, events, or observables need to be tested, they can be included as well). Just as in the simulation based approach, you’d also need to define the individual building blocks of your template model (e.g. by taging them using the Tag property, e.g. “liver”, “heart”, “lung”, …) or by simply knowing which species, parameters, reactions make up a building block. Then check if key property values like units, reaction rates, kinetic laws, rule strings, etc. match between the respective components of the template model and the derived model. This verification would not require any simulation. I guess this approach is similar to what you are already doing with your model comparisson, but it may be a little more automated.

In either case you could use the Matlab Unit Testing framework for your verification. There are many ways to do this, ranging from writing your own helper methods for the verification to implementing your own constraints. In any case, a general idea would be to write functions for each building block of the template model that take the derived model; this function would do then the comparison between the model parts, either by simulation or by comparting property values. All this can even be automatically run on every submission to Git.

Would either of this approach work for your project?

-Florian

[Andrew Lloyd]

Thanks Florian. Appreciate your suggestions