Main menu
You are here
Data table helper and data exportation
Hi,
The exportation of the results of my model is really an issue for me. I developed a landuse model including a multi-instance submodel with about 25000 instances (cells). I want to export a table of the evolution of a variable for these cells and for 30 "display steps" and it is terribly long (about 4h30 for my 1.8 GHz, 1Go RAM PC) to create the table and export it (of course, I have not checked the box “update at display intervals”). I notice that it is faster (but fastidious) to export my instances "part by part" (for example 10 exportations * 2500 instances) but it is still not satisfying and it force me to create some elements in my model only for the purpose of data export (in my example, I created 10 conditional submodels to select each 2500 instances).
Although it is one of most useful helpers, the data table helper is one of slowest. I was astonished in particular by the time it takes to convert the table created in Simile towards a csv file. Thus, I wonder if one can hope some improvements in the data table helper and the way to export data from Simile of if someone have developed a faster way to export data (I do not any idea on how to do that myself). I think that the speed of data exportation is now really a limiting factor to the implementation of “many-instance” models, such as, in my case, a landuse model.
Thank you
François
Data table helper and data exportation
The problem is that the table helper is based on the TclTk 'table widget' which is not very efficient for large tables. What's needed is a tool specifically for copying data to a file without having to display it in a nice layout. Or perhaps an enhancement to the table helper to allow it to copy values to file without displaying them all? I'll have a look at what can be done.
--Jasper
Data table helper and data exportation
Another thought on this subject : you can use the snapshot tool (the camera icon in the component bar) to create a list of a component's values which you can then save to a .csv file. This is much quicker for large data sets than the table helper but less flexible. You would have to run your model a step at a time for 30 steps, updating and saving the snapshot window contents manually each time, and then combine the data files using a spreadsheet.
I suppose it would be possible to add a 'log to file' function for this tool in time for v4.3; it wouldn't be as flexible for layout as the table helper but at least it would get your data out without slowing the model too much. Would this be useful?
--Jasper
Data table helper and data exportation
Thank you for your replies,
Yes, a "log to file" function would be usefull as I don't care much about the layout appearance.
A detail question about the 'snapshot' solution : I have a rand_const function in one of my models. If I run the lodel timestep by timestep, will this variable initialise at each time step ?
Thanks
François
Data table helper and data exportation
OK, 'log to file' will be included in the snapshot tool in Simile v4.3
the rand_const(...) function supplies a new value only when the model is reset. As long as you run and pause the model, or run it over short time intervals, the value of this function will stay the same.
--Jasper