Millions Of Weather Records Pose Big Data Challenge For Scientists

...The International Environmental Data Rescue Organization (IEDRO) estimates that there are 100 million paper strip charts—records that list weather conditions—sitting in meteorological storage facilities throughout the world. That’s about 200 million observations unused by scientists, data that could greatly improve their models. Now, a few small groups of scientists are trying digitize these records, but they’re facing all kinds of obstacles.

Climate scientists often bemoan the lack of historic records. There are the famous data sets: the Vostok ice core drilled in the 1970s that looks back about 400,000 years, the Keeling curve started in 1958, data from satellites that watch sea ice retreat starting around 1979. But these are spot points in specific places that only span a short amount of time. To truly understand climate, researcher need a global records that reaches back hundreds of years.

Those are the kinds of records that data-rescue organizations like IEDRO are trying to recover. “There’s data tied up in paper records that goes all the way back to the lat 1800s,” says Theodore Allen, a graduate student at the University of Miami and IEDRO volunteer. “So rather than working on observations from 1960 to present, we can work on things from 1880 to present.” With that kind of information, climate scientists can make their models far more reliable...