ForGEM - Forest Genetics, Ecology and Management
The ForGEM model is a spatially explicit, individual tree model on genetics, ecology and management of forests. ForGEM is developed to assess impacts of environmental change and forest management on forest growth and dynamics, the rate of adaptive response of functional traits and the adaptive potential of tree populations (Kramer et al 2008, 2010).
The genetics module of ForGEM describes the quantitative genetics of functional traits of trees, by potentially assigning a genetic system to each of the model's parameters. Gene flow is described through the production and dispersal of pollen and seed and thus the spatial exchange of genetic information. This genetic system makes that seedlings differ in their genetic makeup from their parents and thereby in the parameter values that determine the ecophysiological processes.
The ForGEM model allows to calculate a large number of genetic statistics that characterize the genetic make-up of an individual tree, the genetic diversity of a population and differentiation between populations. Individual tree statistics include the genotype of the individual, and methods for its spatial distribution. The population statistics, within and among populations, include diversity measures, differentiation measures between populations, heterozygozity and F-statistics.
The ForGEM model is based on the ecophysiological functioning of whole trees with a mechanistic coupling between genetic and eco-physiological processes. The biophysical- and biochemical environment that drive the eco-physiological processes are described in detail by the model.
The user of the ForGEM model can define forest management, i.e. silvicultural operations, to realistically simulate species composition and demographic dynamics driven by forest management.
The ForGEM model can simulate an understory that consists of grass and herb species distributed over, usually 20x20m, grids though grid size is variable.
- 1 Model description
- 2 Initialization
- 3 Validation
- 4 Source code
- 5 Running ForGEM
- 5.1 Preparing forgem to run
- 5.2 Check if the thredds server is up and running
- 5.3 Check if the postgresql database is running
- 5.4 Running simulations in interactive mode
- 5.5 Running simulations in batch mode
- 5.6 Output
- 5.7 restarting a Forgem Job
- 5.8 Running ForGEM by Remote Desktop
- 5.9 postprocessing
- 6 external links
- 7 references
- genetics - quantitative genetics of functional traits of trees
- gene flow - production and dispersal of seeds and pollen
- ForGEM environmental heterogeneity: overstorey light interception - interception of light by seedling cohorts and crowns of individual saplings and adult trees
- photosynthesis and transpiration - of seedlings cohorts and individual trees
- phenology - annual cycle of bud burst, leaf fall, flowering, frost hardiness, photosynthetic capacity
- allocation - allocation of photosyntates to plant components
- growth - biomass - growth in biomass
- growth - structural - increment of height, stem diameter and stem volume, crownradius and -length of individual trees.
- tree dynamics - establishment and mortality of seedling cohorts and individual trees
- forest management - user-defined implementation of forest interventions
- forest management approaches - FMA's
- soil carbon - dynamics of soil organic matter
( OLD STRUCTURE:
- seedling cohort establishment, growth and competition - the establishment, growth and competition for light of seedling cohorts individual seedlings
- ForGEM environmental heterogeneity: overstorey light interception- light interception by individual crowns of adult trees
END OF OLD STYRUCTURE)
Initialization of the the ForGEM model means assigning initial values to the variables of the model that are simulated over time. The following aspects of the model needs to be initialized in order to run the model: climate, soil, stand, management and the genetic system
At the moment, ForGEM can only be run with the 32 bit NSM Windows versions of NSM.exe and NSMBatch.exe.
Depending on the #DEFINE settings within ForGEM, or from the command line for NSMBatch, a single simulation or multiple simulations can be run. The characteristics of the simulations are defined in the PostgreSQL table forgem.tbl_jobinfo in the database hpc at the server 'vle-models.wur.nl'. Each simulation has a unique RunID (integer value) and several RunID's can have the same JobID (text description). The different RunID's are replicates or have slightly different settings within one experiment (Job).
Preparing forgem to run
1. Create a disk called Y:\
Open explorer and go to the directory containing the trunk directory (e.g. D:\ForGEM ).
Create a share to yourself (mouse left click on the directory and choose Properties, choose the Sharing tab)
Add at the advanced sharing tab your own username.
Remember the name at the Network Path as written in the Sharing Tab (in the example \\D0135942\ForGEM)
Go back to the explorer and press the alt-key (this will show the menu items)
choose Tools - Map network drive...
choose Drive: Y:
Type the above Network Path in the Folder: field
check the Reconnect at logon and press finish
2. Create within the Drive Y a folder NSM and copy the NSMUtils directory to this folder
trunk Projects Utils Forgem Parameters Initialisation .... NSM NSMUtils
In interactive mode:
from within nsm execute the statement getenvironmentvar("COMPUTERNAME"), this gives the name of your computer
2. Around line 200 insert replace
OR _COMPUTER_NAME == LapTopMartJan)
with OR _COMPUTER_NAME == LapTopMartJan OR _COMPUTER_NAME = "name of your computer")
3. Around line 233 replace the text werf010 with your own username, to get the username, execute getenvironmentvar("USERNAME")
Check if the thredds server is up and running
The results should look like:
if not then contact the server manager of vle-models.wur.nl. He should make sure apache is running and has access to the E disk of the D0135942 computer.
Check if the postgresql database is running
This can be done by opening an access database and use an external connection to open for instance the forgem.tbl_jobinfo (table with information about status of jobs).
To create an odbc connection:
for first time use:
type in start menu odbc and click Data Sources (ODBC) (32 bit)
check in Drivers tab if PostgreSQL Unicode is available, when not install postgresql odbc drivers by means of psqlodbc.msi.
When installed, go to File DNS tab and press Add...
choose PostgreSQL Unicode and give a name e.g. hpc and type finish
fill in as shown, but use your own username and password
and press save
in ms-access choose external data, ODBC Database, link to the data source by creating a linked table and press ok.
in file Data Source shoose the hpc.dsn file and fill in the right username and password
choose table forgem.tbl_jobinfo and press ok
you should now be able to see the table and make changes. The connection information is now stored in ms-access
If not, then postgresql is not running or your ip adress is not known to postgresql ask the administrator of the server for help.
Running simulations in interactive mode
Very usefull for testing purposes, debugging is easy, run one scenarioname and one RunID
1. start NSM.exe and select the file Forgem_SingleRun.def 2. Around line 335 within the #elseif (1) section replace the values for ScenarioName and RunID. 3. Select the Exclamation mark (!) which executes the file When an error occurs, debugging can be done.
Running simulations in batch mode
The basic statements for running nsm in batch mode is described in running NSM without user-interface the syntax is
NSMBatch [File=nsm_filename] [Method=(RUN,compile)] [ErrorFile=filename] [OutputFile=filename] [commands]
Running one simulation: one RunID
Run the following command within a DOS box from the directory where the Forgem_SingleRun.def is located. JobID and RunID are the values from a particular scenario in the table forgem.tbl_jobinfo. The ErrorFile and OutputFile are set within the Forgem_SingleRun.def file.
runs the first RunID where HasStarted = FALSE (HasStarted = NULL and HasStarted=TRUE are skipped).
"NsmBatch.exe" File=Forgem_SingleRun.def "ScenarioName=\"JobID\"" OutputInterval=1
Make sure that the NsmBatch.exe can be found (e.g. Use "C:\Program Files (x86)\NSM\NsmBatch.exe" instead).
example running the first free RunID for ScenarioName = Fagus_gen_prod2
"C:\program files (x86)\NSM\NSMBatch.exe" file=Forgem_SingelRun.def "ScenarioName=\"Fagus_gen_prod2\"" OutputInterval=1; "#DEFINE PICEA_GEN"
Running multiple simulations: one JobID
RunForgem.bat is a wrapper around the statement above for running one or more simulations
The syntax is
RunForgem n JobID RunID [resetcode] [commands] Where n: the maximum number of RunIDs to run JobID: the scenarioname to run from the forgem.tbl_jobinfo table RunID: Value of RunID to run or 0: choose where HasStarted=FALSE resetcode: optional "#DEFINE _RESET_RUNS_FIRST". If this value is given then before the first run all values of the fields HasStarted and IsFinished are reset to FALSE where HasStarted=TRUE. Where the value HasStarted=NULL no change is made. commands: extra commands given e.g. ""#DEFINE _LOCAL_OUTPUT_DISK"" To choose options in the Forgem_SingleRun.def file The n RunID's to run are chosen on basis of the Priority field Where HasStarted=FALSE
RunForgem1.bat is a wrapper around the RunForgem.bat statement
The syntax is
RunForgem1 n JobID resetcode [commands] Where n: the maximum number of RunIDs to run JobID: the scenarioname to run from the forgem.tbl_jobinfo table resetcode: integer, only when unequal to 0 then before the first run all values of the fields HasStarted and IsFinished are reset to FALSE where HasStarted=TRUE. Where the value HasStarted=NULL no change is made. commands: extra commands given e.g. ""#DEFINE _LOCAL_OUTPUT_DISK"" To choose options in the Forgem_SingleRun.def file The n RunID's to run are chosen on basis of the Priority field Where HasStarted=FALSE
Example: run at least 10 free runs in sequence of the simulation called "Picea_runs", where free means runs where the value of HasStarted=FALSE
Running multiple JobID's
When the different runs should come from multiple JobID's the JobID parameter in runforgem1.bat should be set to "". The JobID's which should be used are selected in the forgem.tbl_runmultiplejobs table. Set the Selected field value to TRUE. And make sure that the field HasStarted for those JobID's are set to FALSE.
ToDo: test if "#DEFINE _RESET_RUNS_FIRST" works across multiple jobs.
Example: run at least 100 free runs across JobID's in sequence of the priority field
The output as of 31-October-2016 has been set to C:\TRUNK\Output\...
All output is written to local disk
restarting a Forgem Job
During the simulation, at each output interval a binary copy of te whole NSM workspace is stored in the file Workspace.bin. This is the latest output of the simulation.
REMARK! Restarting a particular forgem job can only be done properly with the define's set equal to the original run.
1. Locate the Workspace.bin file for the particular JobID and RunID e.g. "Y:\trunk\OutputJob-Picea_gen_2\Binaries\Scenario-30\Replicate-1\Workspace.bin" 2. use NSMBatch.exe and use as extra commands: a. "#DEFINE _RESTORE_FROM_BINARYFILE (\"Y:\trunk\OutputJob-Picea_gen_2\Binaries\Scenario-30\Replicate-1\Workspace.bin\")" b. "ScenarioName=\"\"" the ScenarioName JobID and RunID are recovered from Workspace.bin
NSMBatch.exe "file=Forgem_SingleRun.def" "#DEFINE _RESTORE_FROM_BINARYFILE (\"Y:\trunk\OutputJob-Picea_gen_2\Binaries\Scenario-30\Replicate-1\Workspace.bin\")" "#DEFINE _LOCAL_OUTPUT_DISK" "#DEFINE PICEA_GEN" "#DEFINE NOFLOWER" "#DEFINE _CORRECT_GENOME" "#DEFINE _SMALL_PLOT (0.8)" "ScenarioName=\"\""
Running ForGEM by Remote Desktop
If there are a lot of computers available e.g. such as SARA or a convenient computer of a colleque, then it is advisable to run ForGEM on these external computers by means of the Remote Destop application. As the jobs to run and the parameters are read from the vle-models.wur.nl PostgreSQL server, each computer should have access to this server. The IP adress of those computers should be given in the pg_hba.conf file on vle-models.wur.nl and the ports used should be accessible.
It is of course possible to attach the disk on which the nsm directory and forgem directory resides by means of remote desktop and run the program on the remote computer from this, but the risk of broken connections and crashes are a problem. Besides that, the main computer should run always (beware of computer hibernation!).
The executable for remote desktop is mstsc.exe and can be run from the command prompt or from the start menu. When connecting to the external computer it is very handy to set couple the drives of the main computer. Choose the Show Options button, Local Resources tab, More... button, and checkmark all the drives, press OK. You can save the connection with the external computer name for easy re-connection.
1. Install first and one time only the NSM program on the external computer see Installing 32 bit NSM
2. get the latest version of ForGEM from the server ... or copy the whole directory '...\trunk\....' to the external computer. This should be done whenever the program ForGEM has been changed and tested
3. Make sure that the path to the NSMBatch.exe and the *.bat files within forgem.bat and forgem1.bat are correct for the external computer
4. Run an interactive test to check if the database is acccessible and the netcdf weatherfiles area available on the thredds server. The working of the thredds server can also checked by pressing here. When this fails, problem should first be checked at server providing the databases, then check the open ports.
Running Jobs on Remote Desktop
Starting, restarting are done in similar way as described above. Collecting all data to central computer is done with the ROBOCOPY command with the /MOVE option. In the ...\trunk\sara\ directory there are several helper files (.bat) for making things easier.
A NSM program called CheckCopyAndRun.def which can be run on remote desktop with CheckCopyAndRun.bat makes this whole process easier. The .bat file can be copied to the remote desktop and doubleclicked.
CheckCopyAndRun.def makes use of the other .bat files from the sara directory. This program also checks how many free processes there are available for runs.
When all data are collected to one directory e.g. ...\trunk\Output\.... then
1. all data can be collected in combined .txt files by running the ...\trunk\projects\collect.def with NSM. Check the options at the beginning of the .def file
2. if genetics has been defined, postprocessing of the genetic data can be done by running: ...\trunk\Projects, this program uses AlleleInfo.txt as input.
- Kramer K, Buiteveld J, Forstreuter M, et al., 2008. Bridging the gap between ecophysiological and genetic knowledge to assess the adaptive potential of European beech. Ecological Modelling, 216, 333-353..
concepts and model description
Kramer, K., Werf, D.C. van der, 2010. Equilibrium and non-equilibrium concepts in forest genetic modelling: population- and individually-based approaches.. Forest Systems 19, 100-112..
Kramer K., Vreugdenhil S.J., van der Werf D.C., 2008. Effects of flooding on recruitment and survival of riparian tree species: a field and modelling study on the floodplains of the river Rhine. Forest Ecology and Management, 255, 3893–3903..
Schelhaas MJ, Kramer K, Peltola H, Werf D.C. van der, Wijdeven SMJ, 2007. Introducing tree interactions in wind damage simulation. Ecological Modelling, 197-209..
Schelhaas, M.J., 2008. The wind stability of different silvicultural systems for Douglas-fir in The Netherlands: a model-based approach. Forestry, 81(3): 399-414..
Kramer K, Buiteveld J, Forstreuter M, et al., 2008. Bridging the gap between ecophysiological and genetic knowledge to assess the adaptive potential of European beech. Ecological Modelling, 216, 333-353..