This is the first in a series of articles about Why Open Data Matters. I’m completing my Master’s degree in Urban Planning from Hunter College (graduating May 30, 2013!), so I chose Urban Planners as for the first article in this series. PediaCities is a data informatics startup – we organize, wrangle, and link data about cities. We started with NYC partly because the best municipal open data law in the country was passed here last year.
Want to know how open data will affect your field, industry, or area of interest? Let me know in the comments.
Open Data in NYC
Open data will radically change the future of New York City by increasing transparency and efficiency in government, and fostering innovation. On March 7, 2012 Local Law 11 – Publishing Open Data was signed into law in New York City. The law mandates that all public government data (exceptions exist for data that is sensitive due to privacy or security concerns) be made available to the public on a single open data portal in machine-readable format. The first of the law’s provisions went into effect on March 7, 2013, although the law is not slated to be fully implemented until 2018. For updates and reporting on the implementation of the law I have compiled a list of links that can be accessed at http://bit.ly/NYClocal11. The purpose of the law can best be summarized by quoting Section 1 of the law:
“The council hereby finds and declares that it is in the best interest of New York city that its agencies and departments make their data available online using open standards. Making city data available online using open standards will make the operation of city government more transparent, effective and accountable to the public. It will streamline intra-governmental and inter-governmental communication and interoperability, permit the public to assist in identifying efficient solutions for government, promote innovative strategies for social progress, and create economic opportunities.”
-2012 Local Law 11 – Publishing Open Data http://www.nyc.gov/html/doitt/html/open/local_law_11_2012.shtml
To make the case for why open data should matter to urban planners I want to touch on three concepts mentioned in Section 1 of Local Law 11: transparency, efficiency, and innovation. The simplest reason for why open government data makes sense is that government data belongs to the people – the gathering, maintenance, and analysis of government data is paid for by tax dollars, so citizens should have access to this data. To be useful this data needs to be in a central location and cataloged so it can be findable. It also has to be published in “machine-readable format” so that the user of that data has the ability to perform their own analysis on the data. A PDF on a hidden page of an agency website doesn’t cut it. Transparency depends on access to information about what government does. Open data accomplishes this.
Open data also makes government more efficient. Before Local Law 11 many NYC agencies published datasets on their own websites. Having a central data portal can reduce the burden of database hosting on individual agencies. Having more open data can also reduce the frequency of Freedom of Information Act (FOIA) requests, which is currently costly. Open data protects agencies from being too dependent on a handful of personal relationships to obtain interdepartmental data. Open data that is published in a machine-readable format improves the efficiency of advocacy organizations and businesses that previously had to spend many hours on manual data entry. Open data provides access to more data for government agencies, allowing them to construct sophisticated data-driven policy. A good example of this comes from Mike Flower’s recent work with the Department of Buildings using many data sources to shortlist buildings with the highest liklihood of illegal conversions that would be fire hazards (see: http://slate.me/1081TcP). Before applying data analysis department of building inspectors issued vacate orders in only 13% of inspections, but by targeting the lit generated through data analysis that number rose to 70% (Mike Flowers, who is now the City’s Director of Analytics, descrribing his work with the Department of Buildings in his own words: (http://youtu.be/425QSx0U8lU). Data analysis like this can dramatically increase efficiency, but it depends on the availability of many data sources, which is bolstered by the availability of more open data.
Open data is an engine of innovation. When data is available and convenient software developers can build useful applications on top of credible data. The success of Apple’s iPhone has been largely contributed to the availability of apps that people find extremely useful. The vast majority of these apps were built by third-party developers. Apple developed a platform that was easy for software developers to innovate on top of. Government data can be the basis of a civic platform (for more on government as a platform, see O’Reilly: http://ofps.oreilly.com/titles/9780596804350/). New York City has worked to foster open data fueled innovation through App Contests and Hackathons (where software developers, designers, urban planners, and others gather to experiment with solutions that can be solved with technology). Technology incubators, helped by the city and universities, are nurturing new companies that depend on open data.*
But why does this really matter for urban planners?
As major producers and consumers of government data, urban planners must be aware of the new paradigm of open data. In New York City we will work for and with the agencies that must now comply with Local Law 11. In cities across the country and at the federal level, open data is becoming the new normal. The benefits to transparency, efficiency, and innovation are cause to embrace open data. However, the benefits of open data will not be apparent overnight. Even with legislation now on the books there is a long way to go to realize its full potential. Bureaucratic inertia is a major hurdle to fully implementing open data in many agencies. If we work with data in a government agency each of us can do our part to comply with open data requirements and move that agency forward ahead of the 2018 implementation deadline in NYC. If we need government data to do our work with an advocacy organization, private developer, or consulting agency we can use our influence and activism to push agencies to make their data open quickly. Regardless of where we work, if we have any control of a dataset that might be useful to the public we should go the extra mile to open that data up. If we need help implementing the publication of open data, or we need help understanding the reasons why it is important to make data open we should talk with the technology community behind the open data movement.
Ready to get more involved with open data?
For a great overview of how open data and technology are changing government and democracy read Citizenville: How to Take the Town Square Digital and Reinvent Government, by Gavin Newsom.
Take the Citizenville Challenge and encourage colleagues to do the same.
Meet the community behind the Open Data Movement in NY (http://nyctwg.org/); join and attend the Open New York Meetup (http://www.meetup.com/betanyc/).
Contribute to the New York Tech Meetup Technology Policy Idea Board.
Explore NYC’s official Open Data Portal at https://nycopendata.socrata.com/.
Suggest a new dataset be added to the portal at https://nycopendata.socrata.com/nominate.
Need help making sense of all the open data in NYC?
PediaCities can help!
More Open Data Resources
On the State and Federal Level:
Explore the Federal Open Data Portal at http://www.data.gov.
Explore the newly launched New York State Open Data Portal at https://data.ny.gov.
Stay up to date by following Open Knowledge Foundation, GovFresh, Tech President, O’Reilly Gov 2.0 Radar, Sunlight Foundation, Sunshine Week 2013, GovLoop, Code for America, Open Plans, DataKind, and the latest Knight Foundation News Challenge.
Twitter is a great place to learn about open data. Follow the hashtag #opendata, as well as #gov20, #opengov, and #civictech. I have compiled a list people actively tweeting about these topics at https://twitter.com/npstorey/civic-tech. You can follow me on Twitter at @npstorey.