Working with external data : Raw data in scenario file

Normally, the scenario file contains only references to data files, with information as to how the data is to be extracted. However, if you type the data straight into the entry field in the file parameter dialogue, or edit it with the table editor from the table data dialogue, the resulting data no longer corresponds to anything that can be got from a data file, so instead the data is stored directly in the scenario file. This is usually a bad thing, because the XML format used is unwieldy and verbose, and it may take a long time to load or save. But keeping references in the scenario file also has disadvantages, chiefly:

1. When publishing a model with a scenario file like this, the file itself must be included along with all the other files referenced by it, and these must be installed in the same place in the directory tree relative to the scenario file as they were when the scenario was created

2. The process of loading values from a .csv file is quite time consuming, as the file has to be parsed and checked for syntax errors before the ASCII numerals can be converted into their actual values, and this has to be done every time the data is loaded.

So we came up with a way to maximize convenience, and added a checkbox to the table data dialogue captioned "Include values in scenario files". If you check this, then when you save a scenario file including the parameter in question, it will include all the data for that parameter even if it is also available in the datafile. Furthermore, instead of it all being in longhand XML, it will be in raw binary format, base64-encoded into Ascii and stuffed into a CDATA field.

This means it is small. And fast. If the model uses a lot of data that isn't going to change from one scenario to the next, this is the best way of handling it. There is a slight downside; the values are unpacked and loaded directly into the model's data structures, and are never converted into the TclTk format that allows them to be displayed as text (except for a few at the start and end, to display in the file parameter dialogue, and these give you just the raw numbers if using booleans or enumerated types). So you cannot look at them or change them with the table editor -- but that gets a bit difficult with very large arrays anyway. The only way to change them is to load them once more from a datafile.

In: Contents >> Working with external data >> The table data dialogue