ForGEM - Forest Genetics, Ecology and Management

From VLECK
(Redirected from ForGEM)
Jump to: navigation, search
FORGEM


The ForGEM model is a spatially explicit, individual tree model on genetics, ecology and management of forests. ForGEM is developed to assess impacts of environmental change and forest management on forest growth and dynamics, the rate of adaptive response of functional traits and the adaptive potential of tree populations (Kramer et al 2008, 2010).[1]

The genetics module of ForGEM describes the quantitative genetics of functional traits of trees, by potentially assigning a genetic system to each of the model's parameters. Gene flow is described through the production and dispersal of pollen and seed and thus the spatial exchange of genetic information. This genetic system makes that seedlings differ in their genetic makeup from their parents and thereby in the parameter values that determine the ecophysiological processes.

The ForGEM model allows to calculate a large number of genetic statistics that characterize the genetic make-up of an individual tree, the genetic diversity of a population and differentiation between populations. Individual tree statistics include the genotype of the individual, and methods for its spatial distribution. The population statistics, within and among populations, include diversity measures, differentiation measures between populations, heterozygozity and F-statistics.

The ForGEM model is based on the ecophysiological functioning of whole trees with a mechanistic coupling between genetic and eco-physiological processes. The biophysical- and biochemical environment that drive the eco-physiological processes are described in detail by the model.

The user of the ForGEM model can define forest management, i.e. silvicultural operations, to realistically simulate species composition and demographic dynamics driven by forest management.

The ForGEM model can simulate an understory that consists of grass and herb species distributed over, usually 20x20m, grids though grid size is variable.

Schematic representation of the ForGEM model

Model description

Genetics:

  • genetics - quantitative genetics of functional traits of trees
  • gene flow - production and dispersal of seeds and pollen

Eco-physiology

Management


Water balance

Soil

( OLD STRUCTURE:

END OF OLD STYRUCTURE)

Initialization

Initialization of the the ForGEM model means assigning initial values to the variables of the model that are simulated over time. The following aspects of the model needs to be initialized in order to run the model: climate, soil, stand, management and the genetic system

Validation

Source code

ForGEM is programmed in NSM. The source code is curated in a sub-version repository (location). Please contact Bert van der Werf or Wim de Winter if you have trouble accessing the repository. [1]

Running ForGEM

At the moment, ForGEM can only be run with the 32 bit NSM Windows versions of NSM.exe and NSMBatch.exe.

Depending on the #DEFINE settings within ForGEM, or from the command line for NSMBatch, a single simulation or multiple simulations can be run. The characteristics of the simulations are defined in the PostgreSQL table forgem.tbl_jobinfo in the database hpc at the server 'vle-models.wur.nl'. Each simulation has a unique RunID (integer value) and several RunID's can have the same JobID (text description). The different RunID's are replicates or have slightly different settings within one experiment (Job).

Preparing forgem to run

1. Create a disk called Y:\

Open explorer and go to the directory containing the trunk directory (e.g. D:\ForGEM ).

Create a share to yourself (mouse left click on the directory and choose Properties, choose the Sharing tab)

Add at the advanced sharing tab your own username.

Share y.jpg


Remember the name at the Network Path as written in the Sharing Tab (in the example \\D0135942\ForGEM)

Go back to the explorer and press the alt-key (this will show the menu items)

choose Tools - Map network drive...

Share y2.jpg

choose Drive: Y:

Type the above Network Path in the Folder: field

check the Reconnect at logon and press finish

2. Create within the Drive Y a folder NSM and copy the NSMUtils directory to this folder

Resume:

Y:

 trunk
   Projects
   Utils
   Forgem
   Parameters
   Initialisation
   ....
 NSM
   NSMUtils



In interactive mode:

from within nsm execute the statement getenvironmentvar("COMPUTERNAME"), this gives the name of your computer

2. Around line 200 insert replace

OR _COMPUTER_NAME == LapTopMartJan)

with OR _COMPUTER_NAME == LapTopMartJan OR _COMPUTER_NAME = "name of your computer")

3. Around line 233 replace the text werf010 with your own username, to get the username, execute getenvironmentvar("USERNAME")

Check if the thredds server is up and running

link http://vle-models.wur.nl:8080/thredds/catalog.html

The results should look like:

Thredds.jpg

if not then contact the server manager of vle-models.wur.nl. He should make sure apache is running and has access to the E disk of the D0135942 computer.

Check if the postgresql database is running

This can be done by opening an access database and use an external connection to open for instance the forgem.tbl_jobinfo (table with information about status of jobs).

To create an odbc connection:

for first time use:

type in start menu odbc and click Data Sources (ODBC) (32 bit)

check in Drivers tab if PostgreSQL Unicode is available, when not install postgresql odbc drivers by means of psqlodbc.msi.

When installed, go to File DNS tab and press Add...

choose PostgreSQL Unicode and give a name e.g. hpc and type finish

fill in as shown, but use your own username and password

Hpc.jpg

and press save

in ms-access choose external data, ODBC Database, link to the data source by creating a linked table and press ok.

in file Data Source shoose the hpc.dsn file and fill in the right username and password

choose table forgem.tbl_jobinfo and press ok

you should now be able to see the table and make changes. The connection information is now stored in ms-access

If not, then postgresql is not running or your ip adress is not known to postgresql ask the administrator of the server for help.

Running simulations in interactive mode

Very usefull for testing purposes, debugging is easy, run one scenarioname and one RunID

procedure:

1. start NSM.exe and select the file Forgem_SingleRun.def

2. Around line 335 within the #elseif (1) section replace the values for ScenarioName and RunID.

3. Select the Exclamation mark (!) which executes the file

When an error occurs, debugging can be done.

Running simulations in batch mode

The basic statements for running nsm in batch mode is described in running NSM without user-interface the syntax is

NSMBatch [File=nsm_filename] [Method=(RUN,compile)] [ErrorFile=filename] [OutputFile=filename] [commands]

Running one simulation: one RunID

Run the following command within a DOS box from the directory where the Forgem_SingleRun.def is located. JobID and RunID are the values from a particular scenario in the table forgem.tbl_jobinfo. The ErrorFile and OutputFile are set within the Forgem_SingleRun.def file.

runs the first RunID where HasStarted = FALSE (HasStarted = NULL and HasStarted=TRUE are skipped).

"NsmBatch.exe" File=Forgem_SingleRun.def "ScenarioName=\"JobID\"" OutputInterval=1 

Make sure that the NsmBatch.exe can be found (e.g. Use "C:\Program Files (x86)\NSM\NsmBatch.exe" instead).

example running the first free RunID for ScenarioName = Fagus_gen_prod2

"C:\program files (x86)\NSM\NSMBatch.exe" file=Forgem_SingelRun.def "ScenarioName=\"Fagus_gen_prod2\"" OutputInterval=1; "#DEFINE PICEA_GEN"

Running multiple simulations: one JobID

RunForgem.bat is a wrapper around the statement above for running one or more simulations

The syntax is

RunForgem n JobID RunID [resetcode] [commands]

Where
    n:         the maximum number of RunIDs to run
    JobID:     the scenarioname to run from the forgem.tbl_jobinfo table
    RunID:     Value of RunID to run or 0: choose where HasStarted=FALSE
    resetcode: optional "#DEFINE _RESET_RUNS_FIRST". If this value is given
               then before the first run all values of the fields HasStarted 
               and IsFinished are reset to FALSE where HasStarted=TRUE. Where the
               value HasStarted=NULL no change is made.
    commands:  extra commands given e.g. ""#DEFINE _LOCAL_OUTPUT_DISK"" To choose options
               in the Forgem_SingleRun.def file

    The n RunID's to run are chosen on basis of the Priority field Where HasStarted=FALSE


RunForgem1.bat is a wrapper around the RunForgem.bat statement

The syntax is

RunForgem1 n JobID resetcode [commands]

Where
    n:         the maximum number of RunIDs to run
    JobID:     the scenarioname to run from the forgem.tbl_jobinfo table
    resetcode: integer, only when unequal to 0 then before the first run all values
               of the fields HasStarted and IsFinished are reset to FALSE where
               HasStarted=TRUE. Where the value HasStarted=NULL no change is made.
    commands:  extra commands given e.g. ""#DEFINE _LOCAL_OUTPUT_DISK"" To choose options
               in the Forgem_SingleRun.def file

    The n RunID's to run are chosen on basis of the Priority field Where HasStarted=FALSE

Example: run at least 10 free runs in sequence of the simulation called "Picea_runs", where free means runs where the value of HasStarted=FALSE

Runforgem1.jpg

Running multiple JobID's

When the different runs should come from multiple JobID's the JobID parameter in runforgem1.bat should be set to "". The JobID's which should be used are selected in the forgem.tbl_runmultiplejobs table. Set the Selected field value to TRUE. And make sure that the field HasStarted for those JobID's are set to FALSE.

ToDo: test if "#DEFINE _RESET_RUNS_FIRST" works across multiple jobs.

Example: run at least 100 free runs across JobID's in sequence of the priority field

Runforgem1-multiple.jpg

Output

The output as of 31-October-2016 has been set to C:\TRUNK\Output\...

All output is written to local disk

restarting a Forgem Job

During the simulation, at each output interval a binary copy of te whole NSM workspace is stored in the file Workspace.bin. This is the latest output of the simulation.

REMARK! Restarting a particular forgem job can only be done properly with the define's set equal to the original run.

Procedure:

1. Locate the Workspace.bin file for the particular JobID and RunID e.g. "Y:\trunk\OutputJob-Picea_gen_2\Binaries\Scenario-30\Replicate-1\Workspace.bin"

2. use NSMBatch.exe and use as extra commands:

   a. "#DEFINE _RESTORE_FROM_BINARYFILE (\"Y:\trunk\OutputJob-Picea_gen_2\Binaries\Scenario-30\Replicate-1\Workspace.bin\")"

   b. "ScenarioName=\"\""
      the ScenarioName JobID and RunID are recovered from Workspace.bin  

Example:

NSMBatch.exe "file=Forgem_SingleRun.def" "#DEFINE _RESTORE_FROM_BINARYFILE (\"Y:\trunk\OutputJob-Picea_gen_2\Binaries\Scenario-30\Replicate-1\Workspace.bin\")" "#DEFINE _LOCAL_OUTPUT_DISK" "#DEFINE PICEA_GEN" "#DEFINE NOFLOWER" "#DEFINE _CORRECT_GENOME" "#DEFINE _SMALL_PLOT (0.8)" "ScenarioName=\"\""

Running ForGEM by Remote Desktop

If there are a lot of computers available e.g. such as SARA or a convenient computer of a colleque, then it is advisable to run ForGEM on these external computers by means of the Remote Destop application. As the jobs to run and the parameters are read from the vle-models.wur.nl PostgreSQL server, each computer should have access to this server. The IP adress of those computers should be given in the pg_hba.conf file on vle-models.wur.nl and the ports used should be accessible.

It is of course possible to attach the disk on which the nsm directory and forgem directory resides by means of remote desktop and run the program on the remote computer from this, but the risk of broken connections and crashes are a problem. Besides that, the main computer should run always (beware of computer hibernation!).

remote desktop

The executable for remote desktop is mstsc.exe and can be run from the command prompt or from the start menu. When connecting to the external computer it is very handy to set couple the drives of the main computer. Choose the Show Options button, Local Resources tab, More... button, and checkmark all the drives, press OK. You can save the connection with the external computer name for easy re-connection.

Preliminaries

1. Install first and one time only the NSM program on the external computer see Installing 32 bit NSM

2. get the latest version of ForGEM from the server ... or copy the whole directory '...\trunk\....' to the external computer. This should be done whenever the program ForGEM has been changed and tested

3. Make sure that the path to the NSMBatch.exe and the *.bat files within forgem.bat and forgem1.bat are correct for the external computer

4. Run an interactive test to check if the database is acccessible and the netcdf weatherfiles area available on the thredds server. The working of the thredds server can also checked by pressing here. When this fails, problem should first be checked at server providing the databases, then check the open ports.

Running Jobs on Remote Desktop

Starting, restarting are done in similar way as described above. Collecting all data to central computer is done with the ROBOCOPY command with the /MOVE option. In the ...\trunk\sara\ directory there are several helper files (.bat) for making things easier.

A NSM program called CheckCopyAndRun.def which can be run on remote desktop with CheckCopyAndRun.bat makes this whole process easier. The .bat file can be copied to the remote desktop and doubleclicked.

CheckCopyAndRun.def makes use of the other .bat files from the sara directory. This program also checks how many free processes there are available for runs.

postprocessing

When all data are collected to one directory e.g. ...\trunk\Output\.... then

1. all data can be collected in combined .txt files by running the ...\trunk\projects\collect.def with NSM. Check the options at the beginning of the .def file

2. if genetics has been defined, postprocessing of the genetic data can be done by running: ...\trunk\Projects, this program uses AlleleInfo.txt as input.

external links

references

  1. 1.0 1.1 Kramer K, Buiteveld J, Forstreuter M, et al., 2008. Bridging the gap between ecophysiological and genetic knowledge to assess the adaptive potential of European beech. Ecological Modelling, 216, 333-353..

concepts and model description

Kramer, K., Werf, D.C. van der, 2010. Equilibrium and non-equilibrium concepts in forest genetic modelling: population- and individually-based approaches.. Forest Systems 19, 100-112..




applications

Kramer K., Vreugdenhil S.J., van der Werf D.C., 2008. Effects of flooding on recruitment and survival of riparian tree species: a field and modelling study on the floodplains of the river Rhine. Forest Ecology and Management, 255, 3893–3903..



Schelhaas MJ, Kramer K, Peltola H, Werf D.C. van der, Wijdeven SMJ, 2007. Introducing tree interactions in wind damage simulation. Ecological Modelling, 197-209..



Schelhaas, M.J., 2008. The wind stability of different silvicultural systems for Douglas-fir in The Netherlands: a model-based approach. Forestry, 81(3): 399-414..


Kramer K, Buiteveld J, Forstreuter M, et al., 2008. Bridging the gap between ecophysiological and genetic knowledge to assess the adaptive potential of European beech. Ecological Modelling, 216, 333-353..