Deterministic Skill Scores

To quantify the skill of forecast systems or reanalysis models, deterministic skill scores are used. This report describes some of the most common deterministic skill scores used in meteorology and energy/power forecasting and under which circumstances they are most suitable.

These include, among others: Mean error, root mean square error (RMSE) and mean absolute error.

Deterministic Skill Scores

Can CAPE be used as a post-processing tool for systematic errors in shortwave radiation forecasts?

This report is a result of research into the use of CAPE as a representation of convective instability in the atmosphere and convective clouds. These clouds would result in differing values of shortwave radiation (SW) and could produce a possible systematic error in reanalysis data. Through post-processing this systematic error could be reduced to create a better representation of SW in reanalysis over Ireland.

A climatology of CAPE for Great Britain found three main CAPE seasons:
‘land dominated CAPE’ between April and September, ‘Sea dominated CAPE’ between September and January and ‘low CAPE’ from January to April [Holley et al., 2014]. Similar results were found for Ireland.

Initial data analysis found no clear conclusions in correlations or analysis of large SW error events. CAPE is highly variable both spatially and temporally. As the ability of a model to accurately simulate CAPE
depends heavily on the vertical grid spacing in the critical layers, the plan is to return to this research later with a higher resolution model.

CAPE as a post-processing parameter

Journal Club: 04-07-2017

Article Title: Met Éireann high resolution reanalysis for Ireland

This article provided a description and initial data analysis of the high resolution reanalysis (MÉRA) produced by Met Éireann.  This reanalysis has been constructed with a configuration of the HARMONIE-AROME limited area NWP model at a 2.5km resolution, with ERA-Interim forcing at the boundaries.  MÉRA uses a combination of 3D-VAR and Optimal Interpolation to perform data assimilation within the model domain.

Model domain for MÉRA (left) with a plot of model orography. The right hand plot is the same but for ERA-Interim.

Skill scores were calculated for 2m temperature, 10m wind speed, mslp, radiosonde observations and rain-gauge observations for both MÉRA and ERA-Interim.  MÉRA offered improved performance compared to ERA-Interim both in terms of mean error and standard deviation of errors in the majority of cases.

Thoughts for research:

  • Should look to repeat our earlier analysis for MERRA2 and ERA-Interim with this dataset.  Also should be compared to Laura’s WRF-downscaled ERA-Interim data.
  • The dip in wind bias in the early nineties (fig.5(c) of paper) should be investigated to see if there are any issues with the wind observations we have been using.  Will contact Seamus Walsh at Met Éireann regarding this issue.
  • We have acquired MÉRA wind data but still need to get SW data.  Will contact Eoin Whelan regarding its availability.

EUMETNET

EUMETNET = European Meteorological Services Network

is a grouping of 31 European National Meteorological Services that provides a framework to organise co-operative programmes between its Members in the various fields of basic meteorological activities. All Members are National Meteorological (and Hydrological) Services of their respective countries. The current list of Members can be found here and in the map below.

The 2017 brochure can be found here. Activities include:

Continue reading “EUMETNET”