I just ran through a quick data survey. How many participants in data.gov are actually complying with the Obama administration’s open data initiative directive to publish all the public data sets at <site name>/data.json. It turns out the answer is about 16%. It was a quick data survey because I could quickly setup a web viewer with that url and just iterate through a couple of hundred data.gov participants and mark the entries that came up with json. It was about a third of the time necessary for the first run through.
I had theorized that as data is filled in, subsequent runs grow faster. It’s nice to see that in practice and that the effect is pretty large. I can see things working even faster in future as I move from semi-automation to full automation.