Running models: diagnostic dialogues

The point at which you attempt to run a newly built model is where you are most likely to encounter failures due to incompleteness or inconsistencies in the model's definition. You should not give up at this point; a bit of attention to the messages will enable the problems to be fixed.

The first dialogue that appears will have an outline description of the problem, and only two buttons. Depending on the nature of the problem, these can be:

• OK / More info...

The problem has stopped whatever task was in progress from being completed. You can hit "OK" to go back and try again, or "More info..." to see a more detailed message, and get a reference to these help pages where appropriate.

• Give up / See all...

The task in progress can continue, although it will not be successfully concluded. You can hit "Give up", fix the indicated problem and try again, or "See all..." to keep running the task and display a list of all the problems found at the end.

• Default option / More options...

There are several possible actions that can be taken at this point. The most frequently chosen is named on the left button, while the right button opens another dialogue displaying all the options.

Errors during model building

These are problems that come up during the process of converting the Simile model to code that can run on the computer. There are many possible causes, the simplest being that not all components have their values specified (i.e., they are still red). Other building errors can arise if a component has been specified correctly, but its specification does not make sense in its context in the model, often because the context was changed after the component was edited. Several of these messages include useful information to help fix the problem, for instance:

Problem building code, e.g., Simile failed to convert var1 (in submodel submodel1) into a program instruction.

Something like this means that the component has an equation, but it no longer fits the component's place in the model. If you hit 'More info', then 'See full error text', you will see the error you would get if you tried to enter the same equation for the component now (or else you can just do so!). For instance, the parameter names that can be used in the equation might have changed, or a submodel index might no longer be valid.

Problem with model design, e.g., This model cannot be executed because it contains the following circular set(s) of function evaluations: [var1,increment(var2_sum),var2_sum,var2,made_for(var1,var2,1)]

Here, no one component is wrong, but the model cannot be built because together they do not make sense. In this case, a set of influence arrows form a circle, so there is no sensible order in which to evaluate the components they join together (see rules for using influences). The error message includes a list of all the components that form the circular chain, plus other entries representing intermediate steps performed by Simile in evaluating components. 

Problem during compilation of model code

Simile has managed to convert the model into executable code, but the compiler (a separate program that turns c++ code into binary instructions) is having none of it. This usually means that there is a bug in Simile which is causing it to generate incorrect c++ code. The only thing you can do is file a bug report including your model code to Simulistics. However a compilation error is also triggered by an explicit division by zero, since the compiler will not build binary code that inevitably gives bad results. In that case you can rebuild your model using the 'debug' menu entry to find the location of the division-by-zero expressed in terms of model components (i.e., which component, in which submodel).

Problems linking model with parameters or visualization tools

These can occur either when a new helper setup or parameter file is loaded while a model is running, or if a model is changed and rerun while keeping the same helper setup or parameters. It means the component names listed in the file no longer exactly match the components in the model.

For example, the above is the "Some parameter values unused" dialogue.  If a parameter metafile contains parameterization information relating to components that do not exist in the model you are trying to run, you have four options: Discard values (the default: ignore values for the missing component and keep loading the rest of the data), Give up, Use elsewhere (find a model component with a different name that can take the values for the missing one) or see help. The initial dialogue offers only the "Discard values" option and "More options...". Choosing the latter brings up the dialogue displayed above.

The Use elsewhere button is useful in the case where you have changed the name of a model component or a submodel containing it, or added or deleted a submodel boundary around it, since saving the parameter metafile. In the case above, the dialogue is pointing out that some parameters are for a component in a submodel that no longer exists in the model. In this case, hitting Use Elsewhere will bring up a tree diagram showing all the submodel captions in your model, and their nesting relation. When you select a submodel from this list, Simile will try to find the components from the lost submodel in that submodel instead, and supply the parameter data to them. If this works, you should re-save the parameter metafile later to avoid having to do the same thing again next time you open the model.

Problems during model execution

e.g., "Simile ran into a problem trying to run this model.
While calculating the value of {external procedure} during execution of the model at time 1275.0, there was an OS signal: 11 (SIGSEGV)."

These messages relate to problems in a model's logic that only become apparent when the model is running. These occur because some of Simile's advanced features require that the modeller stick to certain rules when using them, or else something undefined will happen in the model. The most commonly used such features are:

• The element() function. The second argument can be an expression, but it must always evaluate to a number between 1 and the number of elements in the array that is the first argument, otherwise it will attempt to get a value that does not exist.
• Dashed influences. These allow a model containing a circle of influences to be executed, specifying that the component at the head of the dashed influence can be evaluated without waiting for the one at the tail. But its equation should contain something like an 'if' clause to insert a boundary condition in any case where the tail does not already have a value anyway.
• Base instance lookup. While Simile will not allow a base instance lookup if the index is out of range, it is possible to cause other problems with this feature, for instance by looking up multiple base instances in the wrong order, e.g., any(index(1) is [10, 5, 1]).

When an error like this occurs, the model cannot continue to run. If it happens during reset or initialization, the execution LED goes grey and the model must be rebuilt before it will run again. If it happens during execution, the LED goes red and the model can be reset and run again. However, in either case, if the same problem occurs again, Simile will crash.

To get more information about the cause of these bugs, run the model using the 'debug' menu entry. This causes the model to be built and run in a scripting language, where attempts to access undefined values will produce a list of procedure calls from which Simile can deduce what actually was missing in the model. Here's an example:

Here we can see the name of the variable in whose equation the error occurred (event1), the name and indices of the submodel instance containing it (Collide) and so on up to the desktop, and the identity of the missing quantity (the variable "worn") and why it failed (it was looking for the value from the instance of its parent submodel "seatbelt" with index 0 -- the index of the first element is 1). The message about using the debug option is irrelevant -- to get this level of detail, the modeller must already be using it. There are similar messages if the variable itself is missing, for instance due to unprotected use of the dashed influence, or a bad index for an array variable.

 

In: Help >> Running models

 

Running in browser (Simile v6.12 on)

If this option is selected, then instead of a run control window appearing as part of the Simile user interface, a new browser window or tab will be created to show the model run status. This looks similar to the SimiLive web interface, and actually downloads some code from the SimiLive server in order to operate, but the model data is coming from the local Simile which will be acting as a web server.

If the option is selected when the model is already running using the Simile single-window run interface, then then that Simile window will be hidden and the browser page will be layed out according to the layout of visualization tools previously in use, i.e., with the same set of notebook tabs and the same panes and display tools visible in each tab. There will be an extra notebook tab showing the model diagram, complete with equation and value tooltips over the model components. Switching to and from execution in the browser causes the model to reset. Further visualization tools can be added using the tool buttons in the browser page. However these changes are lost when going back to running the model in Simile's own run environment.

This mode of model execution is useful for previewing how model execution will look if it is to be published for use via SimiLive. Also, the 3-D shape viewer in browser execution is implemented in WebGL and produces fully rendered displays, which look a lot better than the displays in Simile's own shape viewer, and also has better performance rendering complex scenes.

Run control panel

The run control panel is a notebook containing two tabs. The first of these, headed 'Run control', is the one usually displayed, and provides the most commonly used actions for controlling the execution, as well as indications of progress. The other tab, 'Run settings', gives access to selections which fine-tune the model execution, and will not usually be displayed.

Run control tab

The run control is used to start and stop the simulation of the model.

• the play button will begin execution at the current time for a number of time steps set using "Execute for".

• the stop button will stop the simulation (if one is running) and set the current time to zero.

• the pause button will pause the simulation at the current time.

Note that the pause button is only displayed if a simulation is running, and the play button is only displayed if one is not.

The progress bar indicates the elapsed time as a proportion of the total. To the right of the progress bar is an LED which changes colour to indicate the current status of the simulation. This table explains the meanings of the most commonly seen LED colours.

LED colour	Corresponding model status
Black	Model is stopped and ready to run
Purple	Model is stopped and ready to run, but has been edited since the execution was started so the results might not correspond to the current model diagram
Yellow	Model is initializing, resetting or loading data from file parameters
Green	Model is executing
Blue	Display tools are busy updating their images
Red	The model has been stopped, either by executing the stop(...) function, or by a mathematical error, and needs to be reset
Grey	The model code is loaded but some fixed parameter values must be supplied for it to run
White	The model has been stopped by a program error, and the executable code needs to be rebuilt

During a long run, the LED may appear some shade between green and blue; this can be interpreted as indicating what proportion of time Simile is spending actually executing the model as opposed to redrawing the displays of the visualization tools. From version 5.4 onwards, Simile is capable of running the model and updating its displays simultaneously when running on a multi-CPU computer; the LED is blue while doing both at once.

The other fields on this tab are:

• Execute for

  This setting controls the number of time units for which the model is run each time you push the "play" button. You can enter a negative number here to use your model for backcasting, i.e., extrapolating the past from the present.

• Current time

  This setting displays the current number of elapsed time units. It is possible to enter a value into this field, if you wish to begin a simulation at some time other than zero. Note, however, that unless your model contains events that occur at some pre-defined time, this is unlikely to make any difference.

• Display interval

  During the course of a simulation run, each helper extracts values from the model for whatever variables it is displaying. This happens at an interval determined by the value in this field. The display interval is specified as a fraction (or multiple) of the time unit. The display interval cannot be shorter than the shortest time step. Setting a short display interval will cause fast-changing values to be displayed with more detail, but will slow down execution of the model.

Starting with version 6.0, there is a checkbox to the left of the display interval field, labelled 'event;' if this is checked, the state of the display tools will also be updated whenever a discrete event occurs in the model. This is particularly useful in two cases: where values of events are being recorded, since an event only has a value at the exact point in time at which it occurs; and when an event causes a change in direction of a continuously varying component, in which case this option allows a graph point to be added at the exact position of the direction change.

• Time step

  The time step for a simulation model is the fraction of the unit of time at which all variables are updated within the model. For example, imagine a model consisting of a single compartment which is initially empty, and a flow into the compartment at a rate of 2 litres per day. In this case, if the unit of time is set to be the day, and a time step of 0.1 days is chosen, then Simile will calculate the change to the compartment every 1/10th of a day. Thus, after the first time step, the container would contain 0.2 litres, i.e. 1/10 of 2 litres, 0.4 litres after the next time step, and so on. After one day (ten time steps) the compartment will contain 2 litres.

Run settings tab

The following actions are available on the second tab of the run control panel:

• Select time units

  This setting is for use with physical units, to specify the particular time unit, rather than using abstract time units. If you have not specified consistent physical units for all flows, you should not change this setting from the default "unit". If you have specified consistent physical units, you should choose the unit you wish to use to specify how long to execute the model for.

• Integration method

  The integration method determines how the update of state variables is calculated on each time step. If your model uses Systems Dynamics elements only (i.e. compartments, flows and variables) then you will find Runge-Kutta to be more accurate with less computational effort. For all other models, Euler integration will be more reliable. Please see the page on Runge-Kutta integration for more details.

• Adaptive step size variation

  Some problem domains have behaviour that alters between slow, steady changes and rapid fluctuations. These are known as 'stiff systems' and one way to model them accurately is to use adaptive step size variation. What happens is that the model execution monitor makes an estimate of the integration error after each time step, and if it is greater than a certain limit, it reverts the model to the state it was in at the start of that time step and re-does it as two time steps of half the duration. The error estimation and reduction process is repeated for each of those time steps, but if a very low error value is estimated after the second of two equal steps, the time step returns to its previous longer value before execution continues. The value entered in the Time step field is now the maximum length of a time step.

  To enable adaptive step size variation, check the box to the left of the word "Adaptive;" and enter a suitable error limit in the entry box to the right. Note that the error limit is used as an absolute value, so models with large numerical values will tend to be run with shorter time steps than those with small numerical values. This changes with Simile v5.7: the error sensitivity now depends on the difference between the user-supplied minimum and maximum values for the compartment (see Equation Dialogue). The larger this difference, the less sensitive the step size will be to integration errors for a given compartment. If either the minimum or maximum value is not specified, it will treat the difference as 100, which results in the same behaviour as for earlier Simile versions.

• Limiting execution speed

  Simple models run very, very fast in Simile. This may not always be desirable, for instance if one is trying to control an ongoing simulation in real time by using a slider, or if trying to stop it in response to some event. So it is possible to limit the frequency at which model time steps are executed. To do this, check the box to the left of the label "Limit updates/sec to:" and enter your maximum update rate to the right.

• Time at reset

  Normally, the model time after reset is zero. However, it is sometimes useful to have the model time set differently at reset, for instance if the model's initial state relates to a historical date and it is parameterized with a time series that is indexed by subsequent dates. In this case, the time units may (or may not) be set to Year, and 'time at reset' set to a given year, so the model time corresponds to the calendar time in the domain being simulated. This will also result in the X axis of graphs, etc, being annotated with values that can be read as the calendar time.

• "Pause on:" occurrences

  If an occurrence type is selected here, the model will pause whenever that thing happens. The preferences dialogue includes an option that decides whether a dialogue will be displayed when this happens. In any case an entry will be added to the Log tab, describing what happened, and the model can be restarted by hitting the 'Play' button again. The occurrence types are events (which does not include time series events) and compartment under/overruns. If this is selected, the model will pause every time a compartment goes below its minimum or above its maximum value, as set in the equation dialogue. This can be useful for debugging models.

Log tab

This tab contains a list of descriptions of things that happen while running the model, including events and compartment under/overruns if they are selected (see above), and user-defined pauses. The contents of the log tab are formatted as a table with fields enclosed in curly brackets, and can be cut and pasetd into other applications, e.g., spreadsheets.

Further notes

What display interval should I use?

When choosing a display interval, bear in mind the following factors:

• If it is very small, then you will slow down the simulation. Updating the display is relatively time-consuming, and the smaller the display interval, the more frequently this happens.

• If it is too large, you may miss important dynamics, especially in a simulation that shows sudden swings in behaviour. An extreme case of this is when you are simulating switching behaviour: for example, in simulating overflow of a container with fixed capacity by allowing an overflow to operate when the contents of the container exceeds a maximum. In this case, you may get a very misleading graph for the overflow (perhaps showing extended periods of no overflow followed by extended periods of an overflow). If you do obtain apparently aberrant behaviour, then set the display interval to be the same as the time step, so that you are quite sure that you are seeing exactly what is happening to the variable.

Observing the proportion of time that the LED status indicator is blue will give you an idea of how important it is to choose the optimum display interval. If the status indicator is blue most of the time you will be able to improve the performance significantly by increasing the display interval. If it is green most of the time, increasing the display interval will have little effect.

Time steps

If the processes that you are simulating in your model are truly continuous, then you are actually specifying a model based on differential equations, with compartments corresponding to state variables. In general, digital computers cannot solve such models exactly, so they resort to various numerical methods for calculating the value of the state variable forward through time. There is a large numbers of such methods, with some of them producing far greater accuracy for a given amount of computational effort than others. Simile uses the simplest such method, the Euler method. The reason for this is that, in general, a Simile model can contain features that make it unsuitable for solving through other, more advanced, numerical methods.

What time step should I use?

There is no simple answer to this question. If the processes you are modelling happen on a discrete-time basis - for example, animals reproducing once per year - then you can use a time step of 1. Otherwise, explore the effect of reducing the time step. If you get no significant difference in model behaviour, then use the bigger time step. If you do get a significant difference, then keep on reducing the time step until you cease to get a change in behaviour.

A useful rule-of-thumb is that no state variable should change by more than two parts in one hundred over one time step.

Why is the time step labelled as being "Time step #1"?

Different submodels in a model can be specified as operating on different time steps (in the submodel properties dialogue). For example, you might have a model with a human demographic submodel operating on an annual time step, and a lake pollution submodel operating on a 1/100th of a year basis. In that case:

Time step #1 = 1

Time step #2 = 0.01

How does adaptive step size variation work?

Simile's adaptive step size variation mechanism differs from other implementations. How it works is, at the end of a time step, a compartment's rate of change is calculated and compared with what it was expected to be when its value at that time was calculated. In the simple case of Euler integration, a compartment's value is calculated with the assumption that its rate of change stays the same throughout the time step, so the estimated error is the difference between the new rate of change and the previous one, times the current step size. Error estimation for Runge-Kutta integration works in a way analogous to this, but the estimated rate of change is a function of the intermediate rates of change generated during the application of that method.

Back-casting

Entering a negative number in the "Execute for..." field in the Run Control enables certain back-casting calculations to be performed. Time will obey the usual rule of stepping from the "Current time" (which you can enter) in steps of magnitude given by "Time step" for the total number of units specified by "Execute for". For example, you could request execution for 100 time units, enter a current time of -100, and a time step of 0.1. Simile will then perform as it usually does by default with 1000 calculations, but in reverse.

In the model equations, you have entered "Initial values" for the compartments (state variables). When the model is first built, or subsequently reset, the "Current time" is set to zero and the compartments are allocated these initial values. Even if the "Current time" is manually changed after this, the compartment values are not changed. It is therefore not possible to enter time-dependent initial values in the model. On the other hand, if you want to enter "Initial values" that represent the state of the system at some non-zero time, this can be achieved by altering the "Current time" setting after the model is built or reset.

If, however, the model is not reset, then the state variables will retain their values from the end of the previous run. This allows a model's calculations to be repeated in reverse, simply by reversing the sign of the time step. (This is a good check on the numerical accuracy of the solution: if the initial values are not re-calculated exactly at the end of the reverse calculation, then the difference is equal to double the error.)

Note: Only models consisting of System Dynamics elements only can reasonably be treated in this way. If population submodels are used, it is not possible to reverse the action of the population control elements through time.

This works with both Euler and Runge-Kutta integration, and given the restriction above, it can usually be assumed that Runge-Kutta integration is preferable.

In: Contents>> Running models

 

Inspect model variable

The snapshot tool is only available once a model has been built and is ready to run. It is provided to inspect the values of complex data structures held by some model elements. By hovering over a model variable it is possible to view pop-up windows that contain an element's equation, description and value. But in some cases, particularly where the data have more than one dimension, this pop-up window is rather hard to read. In any case, if the data is a very large array, the central portion will be elided to prevent the popup window being too large. As an alternative, use the snapshot tool. This will pop-up a larger, permanent window with the values structured by dimension.

As its name implies, the snapshot tool provides a view of the data structure at a moment in time. You can leave the window displaying the snapshot open, run the model for some more time, and then click again on the same element to open a second window that displays the new data structure. Having the two windows open at the same time, each labelled with the time at which the snapshot was taken, is the easiest way to trace changes to an element or sequence of elements. If you would rather just see the current data, the snapshot tool includes an update button () which will refresh the displayed data with the current values from the model.

Saving data from snapshot tool

Normally you would use the data table helper to create a file containing model data. But because this tool is more oriented to displaying data interactively, it can operate very slowly with large tables. The snapshot tool is much quicker, although less flexible. Another alternative is to use the data logger helper, which is oriented towards storing data from many variables at once.

To save data from the snapshot tool at a given time point, hit the save button. It will prompt you for a file name and the data will be saved to this file as a comma-separated (.csv) file that can be loaded into a spreadsheet. The indices will be listed in columns to the left, outermost first, and the actual data items will appear in the rightmost column.

You can also use the snapshot tool to save data as a time series as the model runs. To do this, hit the log button () and select a file name. Then start the model executing. The values displayed in the snapshot window will not change, but after each display interval the current values will be appended to the file. In this case the indices will be listed in rows at the top of the file, outermost first, with the actual data items appearing below them, one row for each time point. The times themselves will appear in the leftmost column. To stop logging, hit the log button again.

Remember that the snapshot tool is disabled until the model has been built. Only after building the model do the elements have values. The tool is available at any point during the model run.

In: Contents >> Working with model diagrams

Parameter estimation using PEST

PEST is an advanced software tool for model calibration, parameter estimation and predictive uncertainty analysis. It is open-source and freely available, and currently distributed by S. S. Papadopoulos & Associates, Inc.

You will need to install PEST separately on your computer, and ensure that it can be executed from the command line. How to install PEST

PEST communicates with Simile models by creating files containing parameter values, running the model then reading files containing model output values. The model will be run many times. How to set up PEST interaction

When using PEST, model inputs and outputs can be individual values or sequences over time. If an output is a time series, the series itself can be plotted on a display tool alongside the model output. Running PEST

Predictive analysis can be used to get an estimate of a model output at a time later than that covered by available data, or it can be used to provide an uncertainty band around a model output trace. Working with predictive analysis

In: Contents >> Running models