I wrote a little script to grab the weather forecasts from the Weather Underground for the past 6 months. I want to know how good their predictions are from 1 to 4 days out. Unfortunately, it’s been 20 years since I last looked at a statistics book. For my first pass through I computed the root mean square error of predicting the daily high temperature. For predictions 1, 2, 3, and 4 days away, the values are 2.9, 3.4, 4.9 and 5.0 respectively. That doesn’t sound so good to me, but I need to figure out what other metrics to squeeze out of this. For example, I want to compare their forecasts against a simple prediction algorithm: the weather will be just like today. I also want to know the reliability of a forecast, which I think is the confidence interval (95% confidence the prediction is within 1 degree of the actual temperature). Finally, I have the data on conditions and precipitation to chew on, too. For now, the error residuals are looking pretty bleak. You should always carry an umbrella.