Blog by David Winter*

Ken Ring was on National Radio a couple of months ago, blathering on about his method of weather prediction. Ring thinks he can provide a forecast for any future date by looking at a weather map that’s about 18 years out of date.  

That’s how long it takes for the earth, the moon and the sun to cycle into the same relative positions in their orbits, and Ring thinks it’s the moon that drives weather down here on earth. The 18 year old weather map will tell him what the atmosphere was doing last time everything lined up this way, and that will be enough to predict the next event.

You probably agree that Ring’s methods sound like lunacy, but Ring continually claimed in his time on air that his method had an 85% accuracy rate. Reading Ring’s website, you can see he is pretty generous when he estimates his own accuracry, like the Texas sharpshooter who shoots the side of a barn then paints a target around the bullet hole to show his prowess with a shooting iron, Ring uses any vaguely similar weather event to prop up the accuracy of his predictions. My particular favourite from that page is his prediction for 100 mm or rain in New South Wales, which was accurate, it’s just that it arrived further West, two days later and was a only 20mm.

So, we shouldn’t take Ring’s self confidence too seriously and his method, which involves tricky maths and obscure terminology, is a prefect example of cargo cult science. But we don’t have to stop there, Ring makes specific predictions that we can test. Just as the worst thing you can say about homeopathy is not that it’s impossible for those dilutions to effect the human body, but that homeopathy has been shown to do nothing; the worst thing you could say about Ring’s weather forecasting methods is that they don’t work. I went digging for some of Ring’s old claims and was thrilled to see that my Sciblogs stablemate Gareth Renowden has already done all the hard work for me! Apparently Gareth was been dealing with crazy people and weather even before he launched Hot Topic, and back in 2006 he got his hands on on Ring’s predictions for rainfall and sunshine hours in each of the major centres. Gareth compared those predictions to the actual values and to the long term average kept by NIWA.

There’s a bunch of stuff that can be done with that data (which Garteth kindly made avaliable for anyone to download), but the most important thing (and this holds for almost any statistical analysis) is to take a look at it. Here’s what I came up with for the rainfall data using the rather wonderful ggplot2 and inkscape.

You can click on the graphic above to get a bigger version. In these graphs the long term average for each observation is the black line, Ring’s predictions if the blue circle and the actual rainfall for that month in that city is the green circle.

Now, let’s test the accuracy of Ring’s predictions, but against what? Statistical tests often compare a set of results to what you might expect to get “by chance” but that’s not very helpful in this case. For one, it’s not clear what the range of possible values should be, and second, comparing the forecasts to numbers picked at random ignores the seasonal effects everyone knows conrtibute to weather. You don’t have to know the position of the moon 18 years ago to know that Auckland is more rainy in July than January. Instead, let’s compare Ring’s forcasts to the easiest forcecast you could ever make – just saying rainfall for a given region in a given month would match the long-term average. If there was anything to Ring’s methods he should be able to do better than that. He didn’t, on average Ring’s prediction was 52.5 mm out from the actual rainfall where as “predicting” the average would have been 37.25 mm out.The long-term average was a better predictor, averaging 15.15 mm closer to the actual rainfall with a confidence interval spanning from -0.4 and 31 mm. We can’t (quite) say from that data that Ring’s forecasts have less predictive power than the long term average, but there is absolutely no evidence they’re any better.

Let’s lower the bar a little and forget about how close to the true value Ring’s predictions came. Was he at least able to get on the right side of the average? If he predicted a drier August than average was the real value likely to indeed be direr? In this case, if you just tossed a coin 48 times, calling heads drier and tails wetter, you’d expect to be right about half the time. And would have done better than Ring. His prediction was only on the right side of the average 17 times in 48 attemps, about 35% accuracy and signficantly worse that you would expect to get from tossing a coin (if you’re one of those p-value fetishists p, in this case, is equal to about 0.03) .

So Ken Ring can’t predict the weather. That probably doesn’t suprise anyone reading my blog or at sciblogs. I wish it suprised me that the uselessness of Ring’s forecasts is no barrier to him being taken serisouly, selling books and appearing on radio newspapers. The host of the show that started by little investigation evidently took some flack for having Ring on (it was a National Radio afterall) and his defence amounted to “well, we just let these people talk and you can decide what you think about it”. That seems impossibly weak to me, surely the least we should do is ask people who make outrageous claims to show what evidence they have to support those claims. I’m pretty satisfied there is nothing in astrological weather forecasting, but if advocates of the method really want to put it to the test it would be easy. Get someone to provide a year’s worth of old weather maps but let only half of them be of the right vintage for their method (about 18 years in Ring’s case) and see if the forecasts from the supposedly predictive weather maps are better than the ones from weather maps chosen at random. How likely do you think it is that test will be run?

– David Winter is a PhD student at the University of Otago and a blogger for SciBlogs.  This blog was not written for WeatherWatch.co.nz, instead it was published at SciBlogs but has been reposted here with permission from the author.

To comment directly to David’s blog at SciBlogs, please click here