Evaluation of the performance of short-term curated daily airborne grass pollen forecasts in diverse biogeographical regions during the AusPollen Partnership project 2016–2020

When providing pollen forecasts to the community, there is a need to verify the accuracy of curated forecasts, but evaluation is not routinely reported. This study of the AusPollen Partnership compared multi-category grass pollen forecasts for up to six days ahead with daily airborne grass pollen concentrations measured in Brisbane, Canberra, Melbourne, and Sydney, Australia during four pollen seasons from 2016 to 2020. The accuracy of categorical grass pollen forecasts predicting grass pollen concentrations in the high and greater, or moderate and greater categories, were assessed as often applied in meteorology using Gerrity scores, equitable threat scores, false alarm ratios, success ratios, and probability of detection of correct category. The skill of grass pollen forecasts curated by aerobiologists were compared with two retrospectively calculated naïve reference forecast methods; climatology and persistence. For Brisbane and Melbourne, high or greater grass pollen levels occurred on average 32% and 22% of days, whereas for Canberra and Sydney, there were few high days, but moderate or greater pollen levels occurred on average 26% and 19% of days, respectively. Average annual Gerrity scores for curated forecasts of high or greater improved with experience from 0.20 to 0.66 in Brisbane, and from 0.39 to 0.55 in Melbourne between 2016 and 2019. Average Gerrity Scores for moderate or greater categories in Sydney were 0.45 and 0.43 in 2016 and 2018 respectively, and in Canberra were 0.34 and 0.41, in the same years. The skill of curated forecasts was usually better than persistence forecasts, but the accuracy of the curated forecasts decreased with longer lead times. Although persistence grass pollen forecasts consistently performed better than climatologies, persistence depends on previous day pollen concentrations being available. Short-term curated daily grass pollen forecasts of the AusPollen Partnership offer useful information for people with allergic rhinitis and asthma, to help facilitate behavioural change and reduce the health burden. There is a need in Australia to extend local pollen records through sustained pollen monitoring to track climate-related changes as well as improve reliability of daily pollen forecasts. Globally, continued evaluation will enable reporting of accurate pollen forecasts to community, clinicians and government stakeholders.