On the Internet, there is the concept of the Deep Web – “large parts of the web that is not reachable by search engines.” By one estimate, 80% of the information on the web is “invisible.”
On the Open Data front, I submit that a lot of government data is also “invisible”, with a large majority of it locked up behind custom search engines with no option for bulk download, maps, heavily-formatted Excel files and PDFs.
And on this occasion of Martin Luther King Jr. Day, to borrow from Dr. King’s landmark “I Have a Dream” speech, I’d like to describe my dream for Open Data – “that one day, there will be a data commons, where governments, citizens and businesses alike can publish and consume data, in a standards-based format – linked, clean, replete with historical archives, queryable and writeable with a RESTful API, backed by an open reputation network, built using open source, using open standards.”
It doesn’t exactly roll off the tongue, nor is it nearly as great a consequence as the ongoing, centuries-long fight for civil liberties, nor is it remotely lyrical as Dr. King’s masterpiece, but I’m hopeful that we’ll get there.
So one day, we won’t have to hold any more PDF Liberation hackathons, learn and query hundreds of APIs, and build yet another disconnected data silo.
For without a dream, a vision to execute, how else can we get there?