Working with external data : Worked Examples

Using the scenario file mechanism

Choose the use-case that is closest to your own requirement. Note, however, that in a typical application you will probably have more than one instance of some of these approaches, and you will be mixing several approaches. For example, one model may have several single-valued variables, some spatial data, and some time-series data. Together, these constitute the 'scenario'.

One a scenario has been set up according to the various use-cases detailed below, and saved in a scenario file, the scenario file can be loaded in subsequent Simile sessions simply by loading it in the scenario (parameter) file dialogue window.

Case 1: providing a single value for a scalar (single-valued) variable

The variable could represent, for example, a model parameter, a site condition, or be used to set the initial value of a compartment.

Case 2: providing a set of values for a variable inside a multiple-instance submodel: values stored in the scenario file

You can use this method if the variable is inside a submodel with a small number of instances.

Case 3: providing a set of values for an array variable: values stored in the scenario file

You can use this method if the variable is an array with a small number of elements.

Case 4: providing a set of values for a variable in a fixed-membership multiple-instance submodel: values stored in a separate file

You can use this method if the variable is inside a submodel with a large number of instances.

Case 5: providing a set of values for an array variable: values stored in a separate file

You can use this method if the variable is an array with a large number of elements

Case 6: providing values for 2-dimensional array or submodel variables

Arrays with 2 or more dimensions can be specified in a variety of ways

Case 7: Providing data for points in a time series

Each point may be a single value or an array of values; special points also discussed.

Scenario files can contain information for all the file parameters in a model, or just those relating to a single submodel, or a certain part of the submodel hierarchy. If you have a model which has file parameters spread across a lot of submodels, then the file parameter dialogue will include a lot of nested sunken frames, coresponding to the nested submodels containing the parameters. Each of these frames has a title, which is 'TOP LEVEL' for the whole model, or the caption for a submodel. To the right of this is a set of buttons, for clearing, loading and saving the parameters in that submodel and those contained within it. Saving parameters for submodels allows you to keep separate sets of parameters for different parts of your model, and to build scenarios by loading separate scenario files for different parts of the model, rather than having a single file for each possible combination of sub-scenarios.

Important: a scenario file must be loaded into the same submodel context from which it was saved, since the locations of the parameter components are specified relative to that submodel context.

In: Contents >> Working with external data

Working with external data : Scenario file : Case 1

Case 1: providing a single value for a scalar (single-valued) variable

This case applies when the model requires just one value. In other words, the variable in question is not inside a multiple-instance submodel, and has not been declared to be an array.

Typical circumstances are:

  • for providing a parameter value;
  • for setting some context-specific value: e.g. soil type, elevation
  • for initialising the value of a compartment.

On the model diagram

  1. Open up the equation dialogue for the variable, by double-clicking on it.
  2. Click the radio button labelled "Fixed Parameter".
  3. Close the equation dialogue window.

Note that the variable now appears with a tail, showing the change in its status. Note also that it is black, even though you have not supplied a value for it.

When you come to run the model

Simile will produce a File Parameter dialogue window, and this will contain an entry for this variable.

  1. Click in the associated edit field, and type in a value.
  2. Click Save if you wish to save this setting (along with any others), and specify a file to save it in.
  3. Click OK when you wish to proceed with the simulation.

In: Contents >> Working with external data >> Using scenario files

 

Working with external data : Scenario file : Case 2

Case 2: Providing a set of values for a variable in a fixed-membership multiple-instance submodel: values stored in the scenario file

This case applies when you have a variable in a fixed-membership submodel which has a small number of instances. A typical example might be a submodel whose job is to hold the values for each of several crop types. The number of crop types is probably small.

On the model diagram

  1. Open up the equation dialogue for the variable, by double-clicking on it.
  2. Click the radio button labelled "File Parameter".
  3. Close the equation dialogue window.

Note that the variable now appears with a tail, showing the change in its status. Note also that it is black, even though you have not supplied a value for it.

When you come to run the model

Simile will produce a File Parameter dialogue window, and this will contain an entry for this variable.

  1. Click in the associated edit field, and type in a n pairs of values, where n is the number of submodel instances. The first value of each pair should be the number 1, 2, 3, etc. The second should be the value you wish to assign to that instance. Thus, if the submodel has 4 instances, and you want to assign the values 4, 5, 6 and 9 to the 4 instances, then you should type in:

    1 4 2 5 3 6 4 9

    just like that (i.e. with no brackets, commas or whatever).

  2. Click Save if you wish to save this setting (along with any others), and specify a file to save it in.
  3. Click OK when you wish to proceed with the simulation.

In: Contents >> Working with external data >> Using scenario files

 

Working with external data : Scenario file : Case 3

Case 3: Providing a set of values for an array variable: values stored in the scenario file

This case applies when you have an array variable with a smallish number of elements. A typical example might be an array containing a parameter value for each of several crop types. The number of crop types is probably small (say, fewer than 20).

On the model diagram

  1. Open up the equation dialogue for the array variable that is to contain the time-series data, by double-clicking on it.
  2. Click the radio button labelled "File Parameter".
  3. Enter the expression makearray(1.0,n), changing n to the number of values in the time series, in the "Equation" edit field.
  4. Close the equation dialogue.

Note that the variable now appears with a tail, showing the change in its status. Note also that it is black, even though you have not supplied a value for it.

When you come to run the model

Simile will produce a File Parameter dialogue window, and this will contain an entry for this array variable.

  1. Click in the associated edit field, and type in a n pairs of values, where n is the number of submodel instances. The first value of each pair should be the number 1, 2, 3, etc. The second should be the value you wish to assign to that instance. Thus, if the submodel has four instances, and you want to assign the values 4, 5, 6 and 9 to the 4 instances, then you should type in:

    1 4 2 5 3 6 4 9

    just like that (i.e. with no brackets, commas or whatever).

  2. Click Save if you wish to save this setting (along with any others), and specify a file to save it in.
  3. Click OK when you wish to proceed with the simulation.

In: Contents >> Working with external data >> Using scenario files

Working with external data : Scenario file : Case 4

Case 4: Providing a set of values for a variable in a fixed-membership multiple-instance submodel: values stored in a separate file

This case applies when you have a variable in a fixed-membership submodel which has a largish number of instances. A typical example might be a submodel whose job is to hold the values for some attributes (e.g. elevation, soil type) of each of a large number of spatial patches.

On the model diagram

  1. Open up the equation dialogue for the variable, by double-clicking on it.
  2. Click the radio button labelled "File Parameter".
  3. Close the equation dialogue.

Note that the variable now appears with a tail, showing the change in its status. Note also that it is black, even though you have not supplied a value for it.

Check or prepare the data file

The data file should be in standard csv (comma-separated value) format. It should have one header line, consisting of one or more labels, separated by commas, followed by n lines. Each line should have the same number of values as there are labels in the header line, and again the values should be separated by commas. The number (n) of data lines should be at least as great as the number of instances for the submodel. One column should contain the values required for the variable in the model.

When you come to run the model

Simile will produce a File Parameter dialogue window, and this will contain an entry for this variable.

  1. Click on the Read table button for this variable.
  2. Select the "data in columns" tab of the table data dialogue.
  3. Browse through your file system to find the .csv file containing the data values.
    You will now see a window listing the columns in the table, by the label associated with each column.
  4. Double-click on the label corresponding to the column of values you wish to assign to the variable.
  5. Click on the OK button.
  6. Click Save if you wish to save this setting (along with any others), and specify a file to save it in.
  7. Click OK when you wish to proceed with the simulation.

In: Contents >> Working with external data >> Using scenario files

Working with external data : Scenario file : Case 5

Case 5: providing a set of values for an array variable: values stored in a separate file

This case applies when you have an variable with a largish number of elements. A typical example might be a variable outside a large multiple-instance submodel, containing a set of values that are to be used for each instance of the submodel..

On the model diagram

  1. Open up the equation dialogue for the variable, by double-clicking on it.
  2. Click the radio button labelled Fixed Parameter.
  3. Enter the expression makearray(1.0,n), replacing n by the number of elements in the array, into the "Equation" edit field.
  4. Close the equation dialogue.

Note that the variable now appears with a tail, showing the change in its status. It also has a 'stack' outline to indicate that it is an array. Note also that it is black, even though you have not supplied a value for it.

Check or prepare the data file

The data file should be in standard csv (comma-separated value) format. It should have one header line, consisting of one or more labels, separated by commas, followed by n lines. Each line should have the same number of values as there are labels in the header line, and again the values should be separated by commas. The number (n) of data lines should be at least as great as the number of instances for the submodel. One column should contain the values required for the variable in the model.

When you come to run the model

Simile will produce a File Parameter dialogue window, and this will contain an entry for this variable.

  1. Click on the Read table button for this variable
  2. Select the "data in columns" tab of the table data dialogue.
  3. Browse through your file system to find the .csv file containing the data values.

    You will now see a window listing the columns in the table, by the label associated with each column.

  4. Double-click on the label corresponding to the column of values you wish to assign to the variable.
  5. Click on the OK button.
  6. Click Save if you wish to save this setting (along with any others), and specify a file to save it in.
  7. Click OK when you wish to proceed with the simulation.

In: Contents >> Working with external data >> Using scenario files

 

Working with external data : 2-dimensional arrays : Case 6

Case 6: providing values for 2-dimensional variables

A variable will need a 2-dimensional array of parameter values if it is itself a 2-dimensional array variable, or if it is inside a 2-dimensional multiple-instance submodel, or if it is a 1-dimensional array variable inside an ordinary multiple-instance submodel, or if it is inside 2 nested multiple-instance submodels.

There are two ways of providing two-dimensional data for such a variable. The first two are generalizations of methods already described, and can also be used for higher-dimensional variables. The third involves reading data directly from a file in a 2-dimensional format, and was introduced in Simile v4.8.

Values direct from scenario file

This method is limited to small arrays. You enter the data in the edit field in the file parameter value as for one-dimensional arrays, but in place of each value there is another list of index and value pairs, in curly brackets. For instance, the data for a 4x3 array of integers could be supplied as:

1 {1 4 2 9 3 1} 2 {1 8 2 5 3 6} 3 {1 9 2 11 3 6} 4 {1 0 2 7 3 9}

Note that after these values are accepted by the dialogue, it will show the indices with a hash and colon, e.g., #2: to distinguish them from the data items. They can in fact be entered either with or without these decorations, making it easier to enter them by cutting and pasting.

It is fairly easy to see how this mechanism can be extended to cope with higher-dimensional arrays.

Values from a column in a .csv (comma-separated value) file

The data file should be in standard csv (comma-separated value) format. It should have one header line, consisting of one or more labels, separated by commas, followed by n lines. Each line should have the same number of values as there are labels in the header line, and again the values should be separated by commas. The number (n) of data lines should be at least as great as the number of instances for the submodel. One column should contain the values required for the variable in the model. There should also be one column containing the outer index values, and one column containing the inner index values.

For instance, a .csv file to generate the same array as in the last subsection, might look like this:

IndH,IndV,Val,

1,1,4,

1,2,8,

1,3,9,

1,4,0,

2,1,9,

2,2,5,

2,3,11,

2,4,7,

3,1,1,

3,2,6,

3,3,6,

3,4,9,

Note that the row index (which would normally correspond to the outer index in a 2-D array within Simile) actually appears after the column index in the file. This does not matter; the columns can be in any order in the file. So can the rows, since their ordering in the model data is going to be specified by the values in the index columns, rather than their line numbers.

When you come to run the model

Simile will produce a File Parameter dialogue window, and this will contain an entry for this variable.

  1. Click on the Read table button for this variable

  2. Select the 'Data from column' tab in the Table Data dialogue, if it is not already selected

  3. Browse through your file system to find the .csv file containing the data values.

  4. Click on the OK button.

    You will now see a window listing the columns in the table, by the label associated with each column.

  5. Drag the header of the column containing the values for the outermost index across into the 'indices' listbox. In the example above, this column header is IndV.

  6. Repeat this action for the inner index. In the example above, this column header is IndH.

  7. Double-click on the label corresponding to the column of values you wish to assign to the variable. In the example, it is Val.

  8. Click on the OK button.

  9. Click Save if you wish to save this setting (along with any others), and specify a file to save it in.

  10. Click OK when you wish to proceed with the simulation.

Values from a grid in a .csv file

In many cases, the data for a 2-d array is available as a grid of values in a .csv file, typically one saved by a spreadsheet. In such cases it is difficult to convert it to a single column with separate columns for indices, so Simile allows this type of data to be read directly. The rows in the grid will correspond to the outermost index of the Simile array, and the columns to the innermost index. If the grid data is arranged the other way round from this, the spreadsheet's 'transpose' function can be used to swap the rows and columns.

After you click the 'get values from file' button in the File Parameters dialogue, the Table Data dialogue contains four tabs. Initially, the 'Data in Column' tab is raised, but for this case we use the 'Data in Grid' tab. So first click on that tab to raise it.

Now we see a file selector, the same as in the previous examples. Use it to open your .csv file containing the grid of values.

On opening the file, the four entry fields for start/finish at row/column will be filled with the numbers of the first and last rows and columns respectively, providing indication of the total size of the grid of data in the file. Note that any header rows and/or columns will be included in these dimensions, so the 'start at' values will always be 1. At this point you can adjust the values to correspond to the part of the grid you actually want to load. At any time you can hit the View/Edit button on the right of the dialogue, to display a grid of the values in the part of the file currently selected for loading. Ths is useful if you are unsure which part of the file contains headers and which the numerical values. However, note that the table display tool only works up to a maximum of 10,000 rows or columns.

The size of the area you eventually select must match the dimensions of the array you need to load into the model. These dimensions are displayed in the header of the Table Data dialogue for convenience. Once you have chosen the appropriate start and end points, you can use the View/Edit button again to actually change some of the values (e.g., if some were badly formatted in the file) but be aware that if you do this and then save the scenario, all the values will be saved in the parameter file, rather than just a reference to the .csv file.

Values from an image or other binary-format file

Loading a 2-D array of data from an image is very much like loading it from a grid of numerical values. Follow the same procedure except, when you open the Table Data dialogue, select the 'Data from Image' tab. You then choose the area of the image from which you want to load values, exactly as was done for the case of loading data from a .csv grid, except in this case there are no header rows or columns to worry about.

There is an important extra step when using an image file - each pixel in an image has a colour, and you must specify how these colours are to be converted into numerical values to use in the model. This has to be done before you can use the View/Edit button, since the table will display numerical values. Each pixel will correspond to a number between a lower and an upper bound. These bounds are entered as 'Value for black' and 'Value for white', and these are the values that will be generated for black, and white, pixels respectively. Value for black can be greater than or less than value for white. Pixels of other colours will generate values between these two values, as specified in the 'For other colours' pulldown menu. For instance, if you select 'Use green level', a green pixel will give the same value as a white one, since the green level in white is 100%, while a red pixel will give the same value as a black one, since the red level in green is 0%.

If the image file has 8-bit colourmap data, as is the case for most GIF files, then these values can be used directly, and by setting the values for black and for white to 0 and 255 respectively, the values themselves will be preserved. Finally, some images contain transparent areas; when using such an image you must set a separate value for transparent pixels in the 'Value for clear' entry field. This need not be in the range between the values for black and for white.

Other grid data formats

Several other data file formats, principally GeoTIFFs, can be loaded into Simile model parameters if you have the Geospatial Data Abstraction Library (GDAL) installed on your computer. This library is not included in Simile's installation, but is distributed free of charge, and is available for all operating systems for which Simile is supported. The procedure for loading these data files is the same as for loading image data, except that since the values in the files are already interpreted as numerical rather than colour, there is no need to specify how they are to be converted to numeric values. Use the last tab in the table values dialogue (Data from GeoTIFF, etc).

In: Contents >> Working with external data >> Using scenario files

Working with external data : Time series : Case 7

Working with time series

There are two principle means of loading time series data into a model, depending on whether it is desired to store the data in the model itself, in which case the table(…) function is used, or whether the data are to be read from an external file, in which case a scenario file is appropriate. It is also possible to enter the data directly into an array, which may be convenient if there are only a few rows of data.

Using an intermediate array

Whichever way the data are provided, they are stored in an array. The following procedure is suitable for transforming the array into a time series.

  1. Make a scalar variable (i.e. not an array) whose value is to change each time-step.
  2. Draw an influence from the time-series array to the scalar variable.
  3. Set the equation for the scalar variable to:

    element([time_series], time()+1)

    where [time_series] is any array that is being used to store values for the time-series.

This procedure selects one value from the array, in the row corresponding to the current simulation time. Note that in the sub-expression time()+1, the +1 is to account for array indices starting at 1, whilst the simulation begins at time()=0.

Using a file directly

The following procedure is used:

  1. Create the file in comma-separated value (CSV) format.
  2. In your model, set the variable to be a "Variable parameter". You must enter maximum and minimum values here, to allow the parameter to be set using a slider.
  3. Build the model.
  4. From the File menu of the Run Control, select the "Parameters..." command. If you are using the single-window execution environment, there is also a toolbar button to display the parameter dialogue.
  5. From the resulting parameter dialogue box, click the "Get values from file" button (with the pencil icon) next to the variable name.
  6. Create a table in the usual way, with or without an index, depending on how the values in the file are distributed with respect to time. You will need an index if the values do not correspond to times 1, 2, 3 etc.
  7. Click OK.
  8. Click the reset (stop) button.

This last step is necessary to re-set the model to use the new parameters at zero time. If you do not do this, a warning will be issued. Note that the index can start from zero.

Time series values do not have to be at regular intervals - for example, if you give a variable parameter the time series:

#0: 1 #20: 2 #80: 3

then it will change on reset and at times 20 and 80 but keep its previous value (or respond to slider input) at other times.

Getting a time series to repeat

If you want your time series values to be used again after a certain period of time, there are two ways to do this. If editing the list directly in the file parameter dialogue, you can end it with, e.g.,

#100: restart

to cause the values to restart from 0 at time 100. This must be at the end of the line.

If you are loading the values from a .csv file, then the load file dialogue will include an entry box labelled "Wraparound at:" into which you can enter a time at which to restart the time series. Doing either of these will cause the time series to repeat indefinitely over the specified period as the model runs.

In: Contents >> Working with external data