WRFV3.9.1 test case on ORR2

The ARF WRF tutorial is a great resource, check it out as well as the notes below.

Download GFS data

We will use GFS data to run a test forecast for storm “Ophelia”, from 14th to 17th October 2017. GFS forecast data can be downloaded here:

https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/global-forcast-system-gfs

Click on the ‘HAS’ data access link for ‘GFS Forecast’ at 0.5º resolution. Next, select:

• UTC cycle: 12
• Select Start Date: 2017/10/14
• Select End Date: 2017/10/14 (same as start)
• Submit Batch? Yes
• Enter your email address
• Click ‘Proceed with order’
• Wait for email to tell you data are ready.

To save you all that hassle, I’ve already downloaded the data, and put it on ORR2: /share/apps/geog/GFSTEST/

WPS

Run geogrid.exe

Change into your WPS directory. You should see a file called ‘namelist.wps’. If you haven’t already done so, you should create a backup file, then copy one for us to use:

cd
cd WRFV3.9.1
cd WPS
mv namelist.wps namelist.wps.bak
cp namelist.wps.bak namelist.wps

Now edit namelist.wps:

emacs namelist.wps

and enter the values we need for geogrid.exe:

&share
 wrf_core = 'ARW',
 max_dom = 1,
 start_date = '2017-10-14_12:00:00',
 end_date = '2017-10-17_12:00:00',
 interval_seconds = 21600
 io_form_geogrid = 2,
/

&geogrid
 parent_id         =   1,
 parent_grid_ratio =   1,
 i_parent_start    =   1,
 j_parent_start    =   1,
 e_we              =  74,
 e_sn              =  61,
 geog_data_res = 'default',
 dx = 20000,
 dy = 20000,
 map_proj = 'lambert',
 ref_lat   =  53.5,
 ref_lon   = -8.,
 truelat1  =  52.5,
 truelat2  =  54.5,
 stand_lon = -8.,
 geog_data_path = '/share/apps/geog/'
/

&ungrib
 out_format = 'WPS',
 prefix = 'FILE',
/

&metgrid
 fg_name = 'FILE'
 io_form_metgrid = 2,
/

Then save (Ctrl-x-s) and exit (Ctrl-x-c). Note that the geog data that WPS needs to build our virtual world is large, and so I have put it in directory ‘/share/apps/geog/’ on ORR2 for us all to share.

You should now be ready to run:

./geogrid.exe

If all goes well, you will see messages on your terminal, ending with:

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!  Successful completion of geogrid.        !
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

You should now see a new file called ‘geo_em.d01.nc’. You could use the xarray library in python to plot the LANDMASK data from this file. You should get something that looks like this:

Run geogrid.exe

The purpose of UNGRIB is to unpack GRIB (GRIB1 and GRIB2) meteorological data and pack it into an intermediate file format.

We need to link the correct ‘Vtable’ file into the WPS directory so that WPS knows which type of data we are using as input. We are using GFS data, so we link that Vtable  into the WPS directory:

ln -sf ungrib/Variable_Tables/Vtable.GFS Vtable

Next we need to link in our GFS data. The test data are in a shared directory, so you can use the following command:

./link_grib.csh /share/apps/geog/GFSTEST/

Now run ungrib.exe:

./ungrib.exe

If all goes well, you should see:

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!  Successful completion of ungrib.   !
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

and a list on intermediate files:

[conor@orr2 WPS]$ ls -hl FILE*
-rw-rw-r-- 1 conor conor 186M Feb 16 14:19 FILE:2017-10-14_12
-rw-rw-r-- 1 conor conor 186M Feb 16 14:19 FILE:2017-10-14_18
-rw-rw-r-- 1 conor conor 186M Feb 16 14:19 FILE:2017-10-15_00
-rw-rw-r-- 1 conor conor 186M Feb 16 14:19 FILE:2017-10-15_06
-rw-rw-r-- 1 conor conor 186M Feb 16 14:19 FILE:2017-10-15_12
-rw-rw-r-- 1 conor conor 186M Feb 16 14:19 FILE:2017-10-15_18
-rw-rw-r-- 1 conor conor 186M Feb 16 14:19 FILE:2017-10-16_00
-rw-rw-r-- 1 conor conor 186M Feb 16 14:19 FILE:2017-10-16_06
-rw-rw-r-- 1 conor conor 186M Feb 16 14:19 FILE:2017-10-16_12
-rw-rw-r-- 1 conor conor 186M Feb 16 14:19 FILE:2017-10-16_18
-rw-rw-r-- 1 conor conor 186M Feb 16 14:19 FILE:2017-10-17_00
-rw-rw-r-- 1 conor conor 186M Feb 16 14:19 FILE:2017-10-17_06
-rw-rw-r-- 1 conor conor 186M Feb 16 14:19 FILE:2017-10-17_12

Run metgrid.exe

The purpose of METGRID is to horizontally interpolate the meteorological data onto your model domain.

./metgrid.exe

Again, you are hoping to see:

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!  Successful completion of metgrid.  !
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

and a list of new files:

[conor@orr2 WPS]$ ls -hl met_em*.nc
-rw-rw-r-- 1 conor conor 8.5M Feb 16 14:22 met_em.d01.2017-10-14_12:00:00.nc
-rw-rw-r-- 1 conor conor 8.5M Feb 16 14:22 met_em.d01.2017-10-14_18:00:00.nc
-rw-rw-r-- 1 conor conor 8.5M Feb 16 14:22 met_em.d01.2017-10-15_00:00:00.nc
-rw-rw-r-- 1 conor conor 8.5M Feb 16 14:22 met_em.d01.2017-10-15_06:00:00.nc
-rw-rw-r-- 1 conor conor 8.5M Feb 16 14:22 met_em.d01.2017-10-15_12:00:00.nc
-rw-rw-r-- 1 conor conor 8.5M Feb 16 14:22 met_em.d01.2017-10-15_18:00:00.nc
-rw-rw-r-- 1 conor conor 8.5M Feb 16 14:22 met_em.d01.2017-10-16_00:00:00.nc
-rw-rw-r-- 1 conor conor 8.5M Feb 16 14:22 met_em.d01.2017-10-16_06:00:00.nc
-rw-rw-r-- 1 conor conor 8.5M Feb 16 14:22 met_em.d01.2017-10-16_12:00:00.nc
-rw-rw-r-- 1 conor conor 8.5M Feb 16 14:22 met_em.d01.2017-10-16_18:00:00.nc
-rw-rw-r-- 1 conor conor 8.5M Feb 16 14:22 met_em.d01.2017-10-17_00:00:00.nc
-rw-rw-r-- 1 conor conor 8.5M Feb 16 14:22 met_em.d01.2017-10-17_06:00:00.nc
-rw-rw-r-- 1 conor conor 8.5M Feb 16 14:22 met_em.d01.2017-10-17_12:00:00.nc

Congratulations, you’ve finished the WPS part of preparing your forecast!

WRF

We now need to move from the WPS directory to the WRF/run directory:

cd ../WRFV3/run/

If you haven’t done so before, make a backup of ‘namelist.input’ and copy one for us to use:

mv namelist.input namelist.input.bak
cp namelist.input.bak namelist.input

Now open ‘namelist.input’ with emacs so we can set the following values in the first two sections of the namelist:

&time_control
run_days                            = 0,
run_hours                           = 6,
run_minutes                         = 0,
run_seconds                         = 0,
start_year                          = 2017,
start_month                         = 10,
start_day                           = 14,
start_hour                          = 12,
start_minute                        = 00,
start_second                        = 00,
end_year                            = 2017,
end_month                           = 10,
end_day                             = 15,
end_hour                            = 18,
end_minute                          = 00,
end_second                          = 00,
interval_seconds                    = 21600
input_from_file                     = .true.,
history_interval                    = 60,
frames_per_outfile                  = 1000,
restart                             = .false.,
restart_interval                    = 5000,
io_form_history                     = 2
io_form_restart                     = 2
io_form_input                       = 2
io_form_boundary                    = 2
debug_level                         = 0
/

&domains
time_step                           = 120,
time_step_fract_num                 = 0,
time_step_fract_den                 = 1,
max_dom                             = 1,
e_we                                = 74,
e_sn                                = 61,
e_vert                              = 30,
p_top_requested                     = 5000,
num_metgrid_levels                  = 32,
num_metgrid_soil_levels             = 4,
dx                                  = 20000,
dy                                  = 20000,
grid_id                             = 1,
parent_id                           = 0,
i_parent_start                      = 1,
j_parent_start                      = 1,
parent_grid_ratio                   = 1,
parent_time_step_ratio              = 1,
feedback                            = 1,
smooth_option                       = 0
/

Next, link the met_em files to this WRF/run directory:

ln -sf ../../WPS/met_em.d01.2017-10-1* .

Run real.exe

Before running the real.exe code, we neet to prepare the environment:

module purge
. /share/apps/intelPS/composer_xe_2015.3.187/bin/compilervars.sh intel64
export PATH=/share/apps/mvapich2intel/bin:$PATH

Then run real.exe:

./real.exe

This should create the following files:

[conor@orr2 run]$ ls -hl wrf*d01
-rw-rw-r-- 1 conor conor 20M Feb 16 14:37 wrfbdy_d01
-rw-rw-r-- 1 conor conor 12M Feb 16 14:37 wrfinput_d01

Submit wrf.exe

We can’t actually run the WRF forecast from the command line, as we want it to use more than one CPU core to run the code. Instead, we submit it to a queue. To do this, we need a qsub file. Use emacs to create your ‘RUN_WRF.qsub’ file with the following (change the value of RUNDIR):

#!/bin/bash -l
#$ -S /bin/bash
#$ -cwd
#$ -N oWRF
#$ -pe intel 8
#$ -q 68nht.q
#$ -V
#################################################
# Set Intel environment
module purge
. /share/apps/intelPS/composer_xe_2015.3.187/bin/compilervars.sh intel64
export PATH=/share/apps/mvapich2intel/bin:$PATH
#################################################
RUNDIR=/home/conor/WRFV3.9.1/WRFV3/run
#################################################
date
echo "Starting RUN_WRF"
cd $RUNDIR
rm -f namelist.output rsl.* wrfout* 
time mpirun -np 8 ./wrf.exe
echo "FINISHED WRF.EXE"
date

Then submit this job to the queue:

qsub RUN_WRF.qsub

You can check on the status of your job with the qstat command:

qstat

When it’s running, you should see something like this:

[conor@orr2 run]$ qstat
job-ID  prior   name       user         state submit/start at     queue                          slots ja-task-ID 
-----------------------------------------------------------------------------------------------------------------
   8333 0.50500 oWRF       conor        r     02/16/2018 14:54:30 68nht.q@compute-1-5.local          8

Once it’s finished, you should have some wrf output files in your run directory:

[conor@orr2 run]$ ls -hl wrfout*
-rw-rw-r-- 1 conor conor 96M Feb 16 14:57 wrfout_d01_2017-10-14_12:00:00

You could use xarray in python to plot some data from your forecast:

My jupyter notebook for the above plot is here: https://metclim.ucd.ie/public/PlotWRFout.ipynb