Only Happy When It Rains: A Brief History of Weather Forecasting
September 18 marked the 12th anniversary of the release of “I’m Only Happy When It Rains” Garbage’s sarcastic 1995 homage to the 1990s angst-filled Grunge Rock scene. However, the song’s title, when taken literally, marks the most accurate reflection of our company’s status when we were calibrating our acoustic disdrometer (i.e., rain measuring device). Basically, we needed rain data and had to wait patiently until the next storm hit the area. The song contrasts with a verse from The Beatles’ 1966 tune “Rain,” a song John Lennon said he “wrote about people who are always moaning about the weather all the time:”
Rain, I don’t mind
Shine, the weather’s fine.
Fortunately for us, some pioneers of weather forecasting thought about weather phenomena in Eurythmics-like terms:
Here comes the rain again
Falling on my head like a memory
Falling on my head like a new emotion.
And those emotions got translated into operational weather forecasts that benefit billions of people every day.
The Uncommercial Traveler, a tale of many cities.
So, what is weather forecasting and why do we care about it?
Weather forecasting as we know it today is a relatively young discipline established in the second half of the 20th Century, following the pioneering work done by British Vice Admiral, Robert FitzRoy in the second half of the 19th Century. The discipline owes its success to two major contributions, the implementation of the global weather observing system, and the advancements in computing that made it possible to numerically calculate future values of atmospheric variables from global observations.
The current global weather observation system relies on both conventional and satellite measurements and, as mentioned by Jean Coiffier,[1] provides an imperfect but effective description of the atmosphere at a given time. In particular, the World Meteorological Organization, WMO, collects data across the globe from more than 10,000 land observation stations; 700 ship stations; 1,000 weather balloons; 300 buoys; weather satellites; and from instruments aboard commercial aircrafts.[2] These datasets support the development of observation-based weather forecasting, and for instance, the US National Weather Service collects and analyzes more than 76 billion observations, and releases about 1.5 million forecasts and 50,000 warnings each year.[3]
Similarly, with the availability of large-scale computing and the advancement of our weather modeling capabilities, operational weather forecasting became a reality, and the development of other applications like air quality forecasting and modeling of hazardous materials in the atmosphere became possible.[4] In the agricultural context, minimum temperature forecasts allow the planning of freeze prevention methods; heat-wave forecasts allow the farmers to adjust labor-related activities and to schedule appropriate mitigation strategies. Better forecasts also improve the timing of planting and harvesting operations and the accuracy of crop disease models, as well as benefit irrigation scheduling, among many other applications.
Without forecasts we would have to rely on weather wisdom, the appearance of the clouds, and the behavior of animals. Hello, Groundhog Day!
The early days of weather forecasting
The fascination with weather and the idea of predicting it has been present for the last 26 centuries at least. Around 650 BCE, the Babylonians studied the appearance of clouds and tried to relate them to short-term weather changes. Three centuries later, the Chinese created an annual calendar that divided the year into 24 different weather types, and Aristotle wrote Meteorologica, a treatise that included theories about the formations of rain, clouds, winds, and even hurricanes.[5] Nevertheless, it took twenty centuries (and the creation of the thermometer and the barometer) to overthrow most of Aristotle’s theories.
In 1849, the Smithsonian Institute created weather maps using observations from weather instruments located near telegraphs, and in 1854, Vice Admiral Robert FitzRoy, remembered by some as Charles Darwin’s captain on the HMS Beagle, created what is known today as the UK’s Met Office and pioneered daily weather predictions, which he called “forecasts.”[6] In FitzRoy’s own words, “the term forecast is strictly applicable to such an opinion as is the result of scientific combination and calculation.”[ibid.]
How many human calculators does it take to “race the weather of the whole globe?
However, it was only after the wreck of the Royal Charter gold ship in a violent storm near the north coast of Anglesey in 1859 that FitzRoy was authorized to use telegraphs to issue “operational” storm warnings. The tragedy took the life of more than 450 passengers and crew members returning to Liverpool from Melbourne and the Australian gold fields. The aftermath of the disaster was later described by Charles Dickens in All the Year Round (Issue 11), and then included in his book The Uncommercial Traveler.[ibid.]
Soon after the creation of the UK’s Met Office, in 1870 the US Congress established the National Weather Service under the authority of the Army Signal Corps, and in 1894, US President William McKinley ordered the installation of a hurricane warning network in the West Indies. [Weather Forecasting Through the Ages]
A decade later in 1904, the Norwegian Vilhelm Bjerknes recognized that weather forecasting required both reliable observations and solving a system of equations that describe the observations’ evolution over time. Those equations were later studied by the German mathematicians Courant, Friedrichs and Lewy.[Coiffier]
Another important benchmark occurred between 1916 and 1922, when Lewis Fry Richardson used numerical methods to solve weather forecasting equations, making a six-hour forecast by hand. Alas, the result was not very useful as it took weeks to be produced.[ibid.] Nevertheless, those difficulties led Richardson to publish his book Weather Prediction by Numerical Process, where he conceptualized a system to forecast global weather involving 64,000 (human) calculators to, in his own words, “race the weather of the whole globe.”[ibid.]
Nowadays, deterministic predictions of weather cover way more than the six hours ahead forecasted by Richardson, but do not extend more than two weeks. This is because small uncertainties in the initial forecasts can become as large as the observed variations in the atmospheric flow patterns, i.e., a chaotic nonlinear system like the one found by Edward N. Lorentz at MIT in 1960.[Warner]
However, despite recent advancements in modeling, weather models operate by dividing the globe in grid cells separated horizontally by tens or hundreds of kilometers, depending on the model’s spatial resolution. Furthermore, the weather models are incomplete and not true representations of the workings of the atmosphere,[8] particularly at smaller scales. As a result, there are important differences between the real world and its representation in numerical weather models. These differences and limitations force statistical post-processing of the weather models’ output in order to link local scale observations to the coarser resolution model output.
Nevertheless, the implementation of this post-processing step is not possible everywhere in the world as weather stations are not ubiquitous. For this reason, most of the forecasts that are commercially or publicly available cannot offer real hyper-local predictions of environmental variables; thus, customers only get forecasts for the nearest weather station or from the closest grid point from the numerical weather models.
To end, for those still relying on The Old Farmer’s Almanac, I have bad news for you: its accuracy is no greater than that of a forecast constructed by getting random numbers from a slightly skewed distribution.[9] So, welcome to the future of agriculture!
Carlos F. Gaitán is Head of Machine Learning and Artificial Intelligence at Arable.
References:
-
Coiffier, Jean. Fundamentals of Numerical Weather Prediction (2011). Cambridge University Press.
-
Aguado, E. and Burt, James E. Understanding Weather and Climate, sixth edition (2013). Pearson.
-
NOAA, 2017, www.climate.gov. Accessed September, 2017.
-
Warner, Thomas T. Numerical Weather and Climate Prediction (2011). Cambridge University Press.
-
Weather Forecasting Through the Ages, Accessed September, 2017.
-
The birth of the weather forecast, Accessed September, 2017.
-
Sinking of the Royal Charter, Accessed September, 2017.
-
Wilks, D.S. Statistical Methods in the Atmospheric Sciences, third edition (2011). Academic Press.
-
Von Storch, H. & Zwiers., F.W. Statistical Analysis in Climate Research (2001). Cambridge University Press.