Running experiments

As of Simile v7.0, a model can be run simultaneously with its default parameters and with various other combinations of parameters, using multiple processors if they are available. A combination of such runs is known as an experiment, as it allows comparisons between output data from the default case with data from the experimental variations. The feature is similar in some ways to the experiment feature in Apsim NG, but Simile’s version does not allow parts of the actual model to be changed between cases, although this functionality may be added in the future.

Any model containing file parameters may be used as the basis for an experiment. When you run such a model, the explorer pane (beneath the run control pane) contains two tabs, titled ‘Outputs’ and ‘Parameters’. ‘Outputs’ shows a tree diagram containing all the values in the model arranged by submodel hierarchy, while ‘Parameters’ shows a similar arrangement with just the file parameters, including an entry field and setup button for each to allow them to be altered. The entry field will include the current value of the parameter if one has been entered, or the component’s equation if using a default value included in the model (indicated by the caption appearing in blue). This tree diagram appears under the ‘Default case’ heading, indicating that these values correspond to the ‘real world’. There is also an  ‘Experimental conditions’ heading, with an empty tree. This tree can be built up by adding experimental cases.

Adding experimental cases

To add an experimental case or set of cases, right-click (or ctrl-click if using a Mac) on the experimental conditions header. This will bring up a menu of different ways to add conditions.

 Parameter value

This is the simplest way to add an experimental condition. It creates a condition in which just one parameter is varied from its default value, although that can be an array parameter. When you select this entry from the insert menu, the first thing to appear will be a dialogue box inviting you to enter a name for the new experimental case. This can be some description of the experimental procedure it is modelling, e.g., ‘swab all’.

After this dialogue is closed, a message will appear in the Parameters tree inviting you to select a model component to which to apply the new value. You can select this from the Parameters tree, or from the Outputs tree or the model diagram, but it must be a file parameter. Although the icon shows a fixed parameter, it can also be a  variable parameter, in which case the data for it will be an alternating list of time points and values to apply at those times. Once you have selected a parameter, an entry field for it will appear under the Experimental conditions header, as part of a new tree.

You can now provide a value for this version of the parameter, using any of the methods available to load values for file parameters. If it is a variable parameter, you will need to provide a time series. Because the caption of the entry is the component’s caption followed by the name of the experimental case, there may not be enough room in the pane for the entry widgets to show, in which case you can enlarge it by dragging the handle on the vertical sash between the explorer and visualization tool panes.

Once this is done, after you next reset the model, there will be two cases – the default case with the original value of that parameter, and the newly named case with the newly entered value. Visualization tools, component popups etc will show values from both cases, labelled as if they were array members.

You can add several values in this way, either for different parameters or for the same parameter in different cases. Each one will create an additional experimental case in which that particular value is applied to that parameter, but everything else is the same as the default case.

 List for parameter

If you want to add a number of experimental cases in which one particular parameter as one of a range of different values, select the entry “list for parameter” from the insert menu. After selection, you go immediately to choosing a file parameter to which to apply the different values. After choosing it, a new entry field is added to the experiment tree with the label of the parameter followed by ‘s’, meaning you can add multiple values.

At this point you can add a series of values using any of the means you would normally use to set up an array parameter. The indices can be text or numerical, and will become the names of the experimental cases for the corresponding values. If the parameter is itself an array or a time series, you need to create a nested array for the value, with the outer indices becoming the experimental case names.

If the parameter is a fixed scalar value, you can enter a series of cases for which it has a sequence of values. Instead of the normal index/value pairs, enter an expression of the form “a to b step c” for the range and frequency of values to use in the experiment. For instance if you set a list for a parameter called “level” with the entry “100 to 500 step 80” you will get cases with level set to 100, 180, 260, 340, 420 and 500. These cases will be named “level=100”, “level=180”, etc. If the step is 1 it can be left out of the entry, e.g., “2 to 6” gives cases for values of 2,3,4,5 and 6.

 Multi-factor case

If you want to create an experimental condition in which several parameter values are varied from their defaults, this is what you need to add to the experiment tree. When you select it from the insert menu, you are prompted to supply a name for the new experimental case, but that is all. The case will then appear under the experimental conditions tree. At this point, the case has the same values as the default case for all the parameters. You can add parameter values to this case by right clicking on the header. Note that only single parameter values can be added inside a multi-factor case, and only one entry field should be added for any parameter.

When you add a parameter value to a multi-factor case, you do not get to select a name for the case, because you are adding to the setup of the existing case. Instead it goes straight to picking the parameter which is to be adjusted. Continue to insert parameter value fields and enter values for them until your experimental case is complete. When you run the model now, you will see the default case and the named multi-factor case, with all the values you entered applied to their respective parameters in the latter.

 Set of permutations

In some experimental designs, a few parameters are varied and the results for each combination of possible variations are analyzed alongside the default condition. This kind of design is useful for finding particular combinations that produce exceptional results, though when working with Simile the PEST interface should also be considered for this purpose.

Simile’s experiment generator provides a tool for setting up such an experiment. The ‘set of permutations’ level can be added to the tree, and like the multi-factor case, other conditions can be added beneath it. However, rather than combining these conditions into a single case, it generates a case for each possible subset of them.

So, when a set of permutations is added, it does not itself generate a new case, or require a case name to be provided. Any of the other case specifications can be added under a permutation, and they will behave exactly as if they were added under the top level as described above. But if more than one is added, then as well as the cases they specify themselves, cases will be generated for all possible combinations of more than one of them.

For instance, if your model has two fixed parameters called ‘speed’ and ‘weight’, you might add a permutations level, and under it ad a parameter value for ‘speed’ as a case called ‘slow’, and a list for parameter ‘weight’, specified as ‘10 to 30 step 10’. The model execution will now contain the cases that these specify themselves, i.e., ‘slow’, ‘weight=10’, ‘weight=20’ and ‘weight=30’, but it will also contain combinations, named by combining the names of the cases that compose them, i.e., ‘slow+weight=10’, ‘slow+weight=20’ and ‘slow+weight=30’.

Note that adding a lot of separate cases under a permutation will very quickly give rise to a large number of conditions, e.g., five individual parameter values will give rise to 31 experimental cases in addition to the default.

In: Contents >> Working with external data