Categories
WRF

WRF driven by GFS Ensemble data on ORR2

GFS Ensemble data are available to download from here:

  • https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/global-ensemble-forecast-system-gefs
  • Grid GENS 3
  • Start Date = End Date. Select files. I’m assuming that _00 is the “control” forecast, and _01 to _20 are the perturbed ensemble members.
  • When your data are ready to download, you will get an email notification. Click on the “Web Download” link. Each ensemble forecast member will have a link that looks something like this:
    gens_3_2017101200_00.g2.tar 03-Apr-2018 07:06 1.6G
    Right-click on the file and copy the link address. Now ssh into the ORR2 computer. Create a folder for this data, like GRIB/GENS/GENS00, and use wget with your copied link to download the data file.
  • Unpack your forecast data file using the tar command, e.g.:
    tar -xf gens_3_2017101200_12.g2.tar
  • You should now have lots of .grb2 files in that directory. You’re ready to run WRF.

The steps to run WRF driven by GENS are similar to those to run WRF with GFS, so follow this post: https://metclim.ucd.ie/2018/02/wrfv3-9-1-test-case-on-orr2/. A few differences to note:

  • link Vtable to ungrib/Variable_Tables/Vtable.GFSENS
  • ./link_grib.csh to both the gens-a and gens-b files.
  • In namelist.input, use: num_metgrid_levels = 27,

If all goes well, you should be able to run your WRF forecast. WRF doesn’t output MSLP, but you can use a code like this to calculate and plot it:

 

 

Categories
WRF

WRFV3.9.1 with ERA-Interim data on ORR2

Before trying to run WRF with ERA-Interim data, make sure you have completed the following:

    1. WPS: Use this as a template: namelist.wps, then run geogrid.exe to generate geo_em.d01.nc
    2. WPS: Copy the Vtable for ERA-Interim data into the WPS directory
      cp ungrib/Variable_Tables/Vtable.ERA-interim.ml Vtable

      Now link to your data files

      ./link_grib.csh ../../GRIB/ERAI/ERA

      and run ungrib.exe:

      [conor@orr2 WPS]$ ls -hl FILE*
      -rw-rw-r-- 1 conor conor 115M Mar 2 18:01 FILE:2017-12-01_12
      -rw-rw-r-- 1 conor conor 115M Mar 2 18:01 FILE:2017-12-01_18
      -rw-rw-r-- 1 conor conor 115M Mar 2 18:01 FILE:2017-12-02_00
      -rw-rw-r-- 1 conor conor 115M Mar 2 18:01 FILE:2017-12-02_06
      -rw-rw-r-- 1 conor conor 115M Mar 2 18:01 FILE:2017-12-02_12
      -rw-rw-r-- 1 conor conor 115M Mar 2 18:01 FILE:2017-12-02_18
      -rw-rw-r-- 1 conor conor 115M Mar 2 18:01 FILE:2017-12-03_00
      -rw-rw-r-- 1 conor conor 115M Mar 2 18:01 FILE:2017-12-03_06
      -rw-rw-r-- 1 conor conor 115M Mar 2 18:01 FILE:2017-12-03_12
    3. We need to convert the ERA-Interim data from model level to pressure levels. To do this, we need to create a file with information about the ERA-Interim model levels: (still in the WPS directory) create a file called ecmwf_coeffs file and put in the following data:
          0         0.000000    0.00000000
          1        20.000000    0.00000000
          2        38.425343    0.00000000
          3        63.647804    0.00000000
          4        95.636963    0.00000000
          5       134.483307    0.00000000
          6       180.584351    0.00000000
          7       234.779053    0.00000000
          8       298.495789    0.00000000
          9       373.971924    0.00000000
         10       464.618134    0.00000000
         11       575.651001    0.00000000
         12       713.218079    0.00000000
         13       883.660522    0.00000000
         14      1094.834717    0.00000000
         15      1356.474609    0.00000000
         16      1680.640259    0.00000000
         17      2082.273926    0.00000000
         18      2579.888672    0.00000000
         19      3196.421631    0.00000000
         20      3960.291504    0.00000000
         21      4906.708496    0.00000000
         22      6018.019531    0.00000000
         23      7306.631348    0.00000000
         24      8765.053711    0.00007582
         25     10376.126953    0.00046139
         26     12077.446289    0.00181516
         27     13775.325195    0.00508112
         28     15379.805664    0.01114291
         29     16819.474609    0.02067788
         30     18045.183594    0.03412116
         31     19027.695313    0.05169041
         32     19755.109375    0.07353383
         33     20222.205078    0.09967469
         34     20429.863281    0.13002251
         35     20384.480469    0.16438432
         36     20097.402344    0.20247594
         37     19584.330078    0.24393314
         38     18864.750000    0.28832296
         39     17961.357422    0.33515489
         40     16899.468750    0.38389215
         41     15706.447266    0.43396294
         42     14411.124023    0.48477158
         43     13043.218750    0.53570992
         44     11632.758789    0.58616841
         45     10209.500977    0.63554746
         46      8802.356445    0.68326861
         47      7438.803223    0.72878581
         48      6144.314941    0.77159661
         49      4941.778320    0.81125343
         50      3850.913330    0.84737492
         51      2887.696533    0.87965691
         52      2063.779785    0.90788388
         53      1385.912598    0.93194032
         54       855.361755    0.95182151
         55       467.333588    0.96764523
         56       210.393890    0.97966272
         57        65.889244    0.98827010
         58         7.367743    0.99401945
         59         0.000000    0.99763012
         60         0.000000    1.00000000
      

      Then copy the WRF conversion program into the WPS directory:

      cp ./util/calc_ecmwf_p.exe .

      and run it:

      ./calc_ecmwf_p.exe

      This should produce the files:

      [conor@orr2 WPS]$ ls -hl PRES*
      -rw-rw-r-- 1 conor conor 54M Mar 2 18:01 PRES:2017-12-01_12
      -rw-rw-r-- 1 conor conor 54M Mar 2 18:01 PRES:2017-12-01_18
      -rw-rw-r-- 1 conor conor 54M Mar 2 18:01 PRES:2017-12-02_00
      -rw-rw-r-- 1 conor conor 54M Mar 2 18:01 PRES:2017-12-02_06
      -rw-rw-r-- 1 conor conor 54M Mar 2 18:01 PRES:2017-12-02_12
      -rw-rw-r-- 1 conor conor 54M Mar 2 18:01 PRES:2017-12-02_18
      -rw-rw-r-- 1 conor conor 54M Mar 2 18:01 PRES:2017-12-03_00
      -rw-rw-r-- 1 conor conor 54M Mar 2 18:01 PRES:2017-12-03_06
      -rw-rw-r-- 1 conor conor 54M Mar 2 18:01 PRES:2017-12-03_12
      
    4. Run metgrid.exe, and you should see the following files:
      [conor@orr2 WPS]$ ls -hl met_em*
      -rw-rw-r-- 1 conor conor 13M Mar 2 18:01 met_em.d01.2017-12-01_12:00:00.nc
      -rw-rw-r-- 1 conor conor 13M Mar 2 18:01 met_em.d01.2017-12-01_18:00:00.nc
      -rw-rw-r-- 1 conor conor 13M Mar 2 18:01 met_em.d01.2017-12-02_00:00:00.nc
      -rw-rw-r-- 1 conor conor 13M Mar 2 18:01 met_em.d01.2017-12-02_06:00:00.nc
      -rw-rw-r-- 1 conor conor 13M Mar 2 18:01 met_em.d01.2017-12-02_12:00:00.nc
      -rw-rw-r-- 1 conor conor 13M Mar 2 18:01 met_em.d01.2017-12-02_18:00:00.nc
      -rw-rw-r-- 1 conor conor 13M Mar 2 18:01 met_em.d01.2017-12-03_00:00:00.nc
      -rw-rw-r-- 1 conor conor 13M Mar 2 18:01 met_em.d01.2017-12-03_06:00:00.nc
      -rw-rw-r-- 1 conor conor 13M Mar 2 18:01 met_em.d01.2017-12-03_12:00:00.nc
    5.  Change into your WRFV3/run directory and use this as a template: namelist.input. Copy or link the met_em files to your run directory, then run real.exe. You should get files like this:
      [conor@orr2 run]$ ls -hl wrfinput_d01 wrfbdy_d01 
      -rw-rw-r-- 1 conor conor  20M Mar  2 18:02 wrfbdy_d01
      -rw-rw-r-- 1 conor conor 8.6M Mar  2 18:02 wrfinput_d01
      
    6. Now run wrf: qsub RUN_WRF.qsub and you forecast should produce output files like this:
      [conor@orr2 run]$ ls -hl wrfout*
      -rw-rw-r-- 1 conor conor 395M Mar  2 18:05 wrfout_d01_2017-12-01_12:00:00
      

    Done!

Categories
WRF

Downloading ERA-Interim data for WRF

The ECMWF has a publicly accessible reanalysis dataset called ERA-Interim which can be used to run WRF. To download data for WRF, follow these steps:

  1. Go to the ECMWF website: https://www.ecmwf.int/ and click the “Log in” link in the top right-hand corner.
  2. If you already have an account, enter your username and password. If not, register by clicking “New users register here“. Fill in your name, UCD email address, Country: Ireland, Sector: Academic/Research. You will get an email with a link you need to click to confirm your application for an account. After a while you should receive another email with your ECMWF account details. Use these to log in, and change your password.
  3. We need three different types of data to run WRF:
    • Invariant data (things that don’t change)
    • Surface data
    • Model Level data
  4. We only need to download Invariant data once, which we can do here: http://apps.ecmwf.int/datasets/data/interim-full-invariant/. If you haven’t yet done so, click on “conditions of use”, fill in: Organisation: UCD, Organisation type: Academic/Research, Country: Ireland. If you agree with the conditions, click “Accept”. Select “Land-sea mask” and click “Retrieve GRIB”. Click “Retrieve now” on the next page, then you’ll see a page with the status of your request. When it changes from “queued” to “Download grib”, right-click on the download link and copy the link address. Open a terminal and ssh to the computer you will run WRF on (ORR2). If you need to, create a directory for this data. Mine is ~/GRIB/ERAI. Change into that directory, and use wget to download the data from the link you have copied. It will download with a strange name, like _mars-atls13-blahblah.grib, so move (mv) it to a sensible name, like ERA-Invariant.grib
  5. We need to download Surface data for each forecast date, which we can do here: http://apps.ecmwf.int/datasets/data/interim-full-daily/levtype=sfc/.
    • Select the month you want (I’m getting December 2017).
    • In the “Select Time” box, select all: 00:00:00 06:00:00 12:00:00 18:00:00
    • In the “Select Step” box, select zero: 0
    • In the “Select parameter” box, select the following:- 2 metre dewpoint temperature
      – 2 metre temperature
      – 10 metre U wind component
      – 10 metre V wind component
      – Mean sea level pressure
      – Sea surface temperature
      – Sea-ice cover
      – Skin temperature
      – Snow density
      – Snow depth
      – Soil temperature levels 1, 2, 3, 4
      – Surface pressure
      – Volumetric soil water layer 1, 2, 3, 4
    • Click on the “Retrieve GRIB” button, then click on “Retrieve now”. This will bring you to a status page. When your download is ready, copy the link, wget it to ORR2 ~/GRIB/ERAI/ and give it a sensible name, like: ERA-SFC-201712.grib
  6. Next we need the model level data, from here: http://apps.ecmwf.int/datasets/data/interim-full-daily/levtype=ml/. Select the same date, all times, and click on the following parameter names (which should tick all available levels):
    • Specific humidity
    • Temperature
    • U component of wind
    • V component of wind
      Then click on the “Retrieve GRIB” button, and “Retrieve now”. When the status of the request is complete, copy the “Download grib” link, wget to ORR2 and rename to something like: ERA-ML-201712.grib
      Finally, repeat the model level download procedure for one extra parameter, in a separate downloaded file:
    • Geopotential
      Download this file to ORR2 and rename to: ERA-MLZ-201712.grib

Your ERA-Interim files should now look something like this:

[conor@orr2 ERAI]$ ls -hl
total 346M
-rw-rw-r-- 1 conor conor 453K Mar 1 16:35 ERA-Invariant.grib
-rw-rw-r-- 1 conor conor 320M Mar 1 18:19 ERA-ML-201712.grib
-rw-rw-r-- 1 conor conor 1.4M Mar 2 13:12 ERA-MLZ-201712.grib
-rw-rw-r-- 1 conor conor 24M Mar 1 17:21 ERA-SFC-201712.grib

 

Categories
WRF

WRFV3.9.1 test case on ORR2

The ARF WRF tutorial is a great resource, check it out as well as the notes below.

Download GFS data

We will use GFS data to run a test forecast for storm “Ophelia”, from 14th to 17th October 2017. GFS forecast data can be downloaded here:

https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/global-forcast-system-gfs

Click on the ‘HAS’ data access link for ‘GFS Forecast’ at 0.5º resolution. Next, select:

  • UTC cycle: 12
  • Select Start Date: 2017/10/14
  • Select End Date: 2017/10/14 (same as start)
  • Submit Batch? Yes
  • Enter your email address
  • Click ‘Proceed with order’
  • Wait for email to tell you data are ready.

To save you all that hassle, I’ve already downloaded the data, and put it on ORR2: /share/apps/geog/GFSTEST/

WPS

Run geogrid.exe

Change into your WPS directory. You should see a file called ‘namelist.wps’. If you haven’t already done so, you should create a backup file, then copy one for us to use:

cd
cd WRFV3.9.1
cd WPS
mv namelist.wps namelist.wps.bak
cp namelist.wps.bak namelist.wps

Now edit namelist.wps:

emacs namelist.wps

and enter the values we need for geogrid.exe:

&share
 wrf_core = 'ARW',
 max_dom = 1,
 start_date = '2017-10-14_12:00:00',
 end_date = '2017-10-17_12:00:00',
 interval_seconds = 21600
 io_form_geogrid = 2,
/

&geogrid
 parent_id         =   1,
 parent_grid_ratio =   1,
 i_parent_start    =   1,
 j_parent_start    =   1,
 e_we              =  74,
 e_sn              =  61,
 geog_data_res = 'default',
 dx = 20000,
 dy = 20000,
 map_proj = 'lambert',
 ref_lat   =  53.5,
 ref_lon   = -8.,
 truelat1  =  52.5,
 truelat2  =  54.5,
 stand_lon = -8.,
 geog_data_path = '/share/apps/geog/'
/

&ungrib
 out_format = 'WPS',
 prefix = 'FILE',
/

&metgrid
 fg_name = 'FILE'
 io_form_metgrid = 2,
/

Then save (Ctrl-x-s) and exit (Ctrl-x-c). Note that the geog data that WPS needs to build our virtual world is large, and so I have put it in directory ‘/share/apps/geog/’ on ORR2 for us all to share.

You should now be ready to run:

./geogrid.exe

If all goes well, you will see messages on your terminal, ending with:

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!  Successful completion of geogrid.        !
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

You should now see a new file called ‘geo_em.d01.nc’. You could use the xarray library in python to plot the LANDMASK data from this file. You should get something that looks like this:

Run geogrid.exe

The purpose of UNGRIB is to unpack GRIB (GRIB1 and GRIB2) meteorological data and pack it into an intermediate file format.

We need to link the correct ‘Vtable’ file into the WPS directory so that WPS knows which type of data we are using as input. We are using GFS data, so we link that Vtable  into the WPS directory:

ln -sf ungrib/Variable_Tables/Vtable.GFS Vtable

Next we need to link in our GFS data. The test data are in a shared directory, so you can use the following command:

./link_grib.csh /share/apps/geog/GFSTEST/

Now run ungrib.exe:

./ungrib.exe

If all goes well, you should see:

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!  Successful completion of ungrib.   !
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

and a list on intermediate files:

[conor@orr2 WPS]$ ls -hl FILE*
-rw-rw-r-- 1 conor conor 186M Feb 16 14:19 FILE:2017-10-14_12
-rw-rw-r-- 1 conor conor 186M Feb 16 14:19 FILE:2017-10-14_18
-rw-rw-r-- 1 conor conor 186M Feb 16 14:19 FILE:2017-10-15_00
-rw-rw-r-- 1 conor conor 186M Feb 16 14:19 FILE:2017-10-15_06
-rw-rw-r-- 1 conor conor 186M Feb 16 14:19 FILE:2017-10-15_12
-rw-rw-r-- 1 conor conor 186M Feb 16 14:19 FILE:2017-10-15_18
-rw-rw-r-- 1 conor conor 186M Feb 16 14:19 FILE:2017-10-16_00
-rw-rw-r-- 1 conor conor 186M Feb 16 14:19 FILE:2017-10-16_06
-rw-rw-r-- 1 conor conor 186M Feb 16 14:19 FILE:2017-10-16_12
-rw-rw-r-- 1 conor conor 186M Feb 16 14:19 FILE:2017-10-16_18
-rw-rw-r-- 1 conor conor 186M Feb 16 14:19 FILE:2017-10-17_00
-rw-rw-r-- 1 conor conor 186M Feb 16 14:19 FILE:2017-10-17_06
-rw-rw-r-- 1 conor conor 186M Feb 16 14:19 FILE:2017-10-17_12

Run metgrid.exe

The purpose of METGRID is to horizontally interpolate the meteorological data onto your model domain.

./metgrid.exe

Again, you are hoping to see:

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!  Successful completion of metgrid.  !
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

and a list of new files:

[conor@orr2 WPS]$ ls -hl met_em*.nc
-rw-rw-r-- 1 conor conor 8.5M Feb 16 14:22 met_em.d01.2017-10-14_12:00:00.nc
-rw-rw-r-- 1 conor conor 8.5M Feb 16 14:22 met_em.d01.2017-10-14_18:00:00.nc
-rw-rw-r-- 1 conor conor 8.5M Feb 16 14:22 met_em.d01.2017-10-15_00:00:00.nc
-rw-rw-r-- 1 conor conor 8.5M Feb 16 14:22 met_em.d01.2017-10-15_06:00:00.nc
-rw-rw-r-- 1 conor conor 8.5M Feb 16 14:22 met_em.d01.2017-10-15_12:00:00.nc
-rw-rw-r-- 1 conor conor 8.5M Feb 16 14:22 met_em.d01.2017-10-15_18:00:00.nc
-rw-rw-r-- 1 conor conor 8.5M Feb 16 14:22 met_em.d01.2017-10-16_00:00:00.nc
-rw-rw-r-- 1 conor conor 8.5M Feb 16 14:22 met_em.d01.2017-10-16_06:00:00.nc
-rw-rw-r-- 1 conor conor 8.5M Feb 16 14:22 met_em.d01.2017-10-16_12:00:00.nc
-rw-rw-r-- 1 conor conor 8.5M Feb 16 14:22 met_em.d01.2017-10-16_18:00:00.nc
-rw-rw-r-- 1 conor conor 8.5M Feb 16 14:22 met_em.d01.2017-10-17_00:00:00.nc
-rw-rw-r-- 1 conor conor 8.5M Feb 16 14:22 met_em.d01.2017-10-17_06:00:00.nc
-rw-rw-r-- 1 conor conor 8.5M Feb 16 14:22 met_em.d01.2017-10-17_12:00:00.nc

Congratulations, you’ve finished the WPS part of preparing your forecast!

WRF

We now need to move from the WPS directory to the WRF/run directory:

cd ../WRFV3/run/

If you haven’t done so before, make a backup of ‘namelist.input’ and copy one for us to use:

mv namelist.input namelist.input.bak
cp namelist.input.bak namelist.input

Now open ‘namelist.input’ with emacs so we can set the following values in the first two sections of the namelist:

&time_control
run_days                            = 0,
run_hours                           = 6,
run_minutes                         = 0,
run_seconds                         = 0,
start_year                          = 2017,
start_month                         = 10,
start_day                           = 14,
start_hour                          = 12,
start_minute                        = 00,
start_second                        = 00,
end_year                            = 2017,
end_month                           = 10,
end_day                             = 15,
end_hour                            = 18,
end_minute                          = 00,
end_second                          = 00,
interval_seconds                    = 21600
input_from_file                     = .true.,
history_interval                    = 60,
frames_per_outfile                  = 1000,
restart                             = .false.,
restart_interval                    = 5000,
io_form_history                     = 2
io_form_restart                     = 2
io_form_input                       = 2
io_form_boundary                    = 2
debug_level                         = 0
/

&domains
time_step                           = 120,
time_step_fract_num                 = 0,
time_step_fract_den                 = 1,
max_dom                             = 1,
e_we                                = 74,
e_sn                                = 61,
e_vert                              = 30,
p_top_requested                     = 5000,
num_metgrid_levels                  = 32,
num_metgrid_soil_levels             = 4,
dx                                  = 20000,
dy                                  = 20000,
grid_id                             = 1,
parent_id                           = 0,
i_parent_start                      = 1,
j_parent_start                      = 1,
parent_grid_ratio                   = 1,
parent_time_step_ratio              = 1,
feedback                            = 1,
smooth_option                       = 0
/

Next, link the met_em files to this WRF/run directory:

ln -sf ../../WPS/met_em.d01.2017-10-1* .

Run real.exe

Before running the real.exe code, we neet to prepare the environment:

module purge
. /share/apps/intelPS/composer_xe_2015.3.187/bin/compilervars.sh intel64
export PATH=/share/apps/mvapich2intel/bin:$PATH

Then run real.exe:

./real.exe

This should create the following files:

[conor@orr2 run]$ ls -hl wrf*d01
-rw-rw-r-- 1 conor conor 20M Feb 16 14:37 wrfbdy_d01
-rw-rw-r-- 1 conor conor 12M Feb 16 14:37 wrfinput_d01

Submit wrf.exe

We can’t actually run the WRF forecast from the command line, as we want it to use more than one CPU core to run the code. Instead, we submit it to a queue. To do this, we need a qsub file. Use emacs to create your ‘RUN_WRF.qsub’ file with the following (change the value of RUNDIR):

#!/bin/bash -l
#$ -S /bin/bash
#$ -cwd
#$ -N oWRF
#$ -pe intel 8
#$ -q 68nht.q
#$ -V
#################################################
# Set Intel environment
module purge
. /share/apps/intelPS/composer_xe_2015.3.187/bin/compilervars.sh intel64
export PATH=/share/apps/mvapich2intel/bin:$PATH
#################################################
RUNDIR=/home/conor/WRFV3.9.1/WRFV3/run
#################################################
date
echo "Starting RUN_WRF"
cd $RUNDIR
rm -f namelist.output rsl.* wrfout* 
time mpirun -np 8 ./wrf.exe
echo "FINISHED WRF.EXE"
date

Then submit this job to the queue:

qsub RUN_WRF.qsub

You can check on the status of your job with the qstat command:

qstat

When it’s running, you should see something like this:

[conor@orr2 run]$ qstat
job-ID  prior   name       user         state submit/start at     queue                          slots ja-task-ID 
-----------------------------------------------------------------------------------------------------------------
   8333 0.50500 oWRF       conor        r     02/16/2018 14:54:30 68nht.q@compute-1-5.local          8

Once it’s finished, you should have some wrf output files in your run directory:

[conor@orr2 run]$ ls -hl wrfout*
-rw-rw-r-- 1 conor conor 96M Feb 16 14:57 wrfout_d01_2017-10-14_12:00:00

You could use xarray in python to plot some data from your forecast:

My jupyter notebook for the above plot is here: https://metclim.ucd.ie/public/PlotWRFout.ipynb