OpenData.city is Live!

Today, we’re thrilled to announce  that OpenData.city is live!

As a company born out of Open Data, we’ve seen how fast the field has evolved.  When I first entered NYCBigApps back in 2010, NYC’s DataMine was primarily a list of 350 files being served out of a web server – no API, files in various formats, little metadata.

Since then, NYC has become the top Municipal Open Data City in the world – with its Landmark Open Data Law; NYCBigApps – the world’s largest, longest-running municipal Open Data Challenge; a vibrant civic tech community helmed by BetaNYC – the largest civic hacker community, more than 2100 strong; a tech-savvy administration and City Council; CUSP; a CIO, a CTO and a CAO; and now CivicHall.org – a community center dedicated to Civic Tech, managed by the same team that runs Personal Democracy Forum – the annual must-go confab for the Civic Tech community.

Even with all this – we always felt that something essential was still missing.  Even NYCFacets, which won us the NYCBigApps Grand Prize back in 2012, was our early attempt in closing what we still think is a gap in current open data portals – Context.

 

Context

With NYCFacets in 2012, we attempted to address the lack of context in the absence of searchable metadata, even creating a page-ranking algorithm of sorts to score datasets on freshness and quality using the metadata crawler we built.

When we released NYCpedia in 2013, we were trying to address this lack of context in the datasets themselves.  It was our first attempt in “humanizing” open data by location and time – from a list of hundreds of datasets primarily useful to techies and researchers, to something that my mother-in-law, local policy makers, businesses and citizens can use.

We did this by geotemporally correlating the datasets and cross-referencing it with other data sources (the Census for demographics, Flickr for pictures, Indeed for job listings, DNAinfo for hyperlocal news, etc.), so casual users can go to a neighborhood, and find out things about it.

We got a lot of great feedback and coverage, and a lot of people told us that they wanted something like NYCpedia in their jurisdictions. But at the end of the day, we built a website, not a product.

But today, that’s no longer the case.  Based on all the lessons we’ve learned from the past three years since we started Ontodia – from our experience in NYCBigApps – first as entrants, and since our win, as data partners helping third-party organizations publish data for the challenge; from building NYCpedia; from interviews with various policy-makers and researchers; and from deploying several CKAN data portals for various organizations, we built the PediaCities Open Data platform.

The PediaCities Open Data platform is composed of two major subsystems:

 

Open Data for All

Beyond having a managed Open Data portal with CKAN, we installed a number of plugins from its large open-source extensions ecosystem to make it more usable for both data consumers and data publishers.  And with our Pro option, we go even further – an analytics dashboard,  API management, enterprise connectors, and an app generator framework.

We also seeded the portal with great data and maps coming from trusted sources (US Census, Dept of Labor Statistics, etc.), pre-sliced for each jurisdiction, updating the data as those sources are changed.   And we’ll do this for all 3,000+ counties and 30,000+ towns and cities in the US.

We also offer a hosts of data-wrangling and geocoding services to help organizations publish great data – from screening datasets for security and privacy, hyperlocal geocoding that recognize local boundaries that constituents care about, and visualization services through us and our partner – CartoDB.

All this managed and overseen by our NOC partner – ExterNetworks, to ensure high availability.

And when we say Open Data for All, we mean Open Data for All… if your jurisdiction has less than 20 datasets, you can keep using the data portal for free!  And  since its powered by CKAN – there’s a No Lock-In Guarantee – that is, after one year,  you can take your CKAN with you if you wish to run it on your own, or with another CKAN provider.

 

Key Place Indicators

Alongside CKAN, we also developed an Indicators platform that allows you to track local issues you care about.  For this Beta launch, we seeded the platform with ISO 37120 indicators –  the result of a years-long effort by experts from around the world to create a standard way for Cities to track and “measure delivery of services and quality of life.”   Our value-add is that we precompute these indicators – from Data uploaded in our platform, and data that we harvest from various public sources.

And these are just our starter indicators. Soon, we will allow users to create their own indicators which are automatically computed as the underlying data changes.  Want to track vacant/abandoned lots?  You can build an indicator for that.  Do you want to track the density of green areas in a certain place? Yep!  Do you want to create a noise indicator at the zipcode level?  Pedestrian traffic in a given neighborhood? Average fuel prices for a jurisdiction?

It’s all possible – so long as the data and the polygon for the desired jurisdiction is there. 😉

We feel that by having a ready mechanism to measure desired outcomes on top of the aggregated, curated open data, we can incentivize data producers/mappers/sharers/gatherers/story-tellers to publish their data in a way that’s immediately usable.

And by always having the data available behind each indicator or visualization,  that we do our part in accelerating dynamic, live, data-driven decisions, as we go away from trapping data in PDFs that nobody reads and pretty maps and infographics with no data provenance.

 

End of Year Beta promotion

We’re just getting started.  We’ll be exposing more services over the next few months to accelerate Open Data-driven decisions and innovations.

And we’d like to inform our product choices with your real-world needs.  So if your jurisdiction signs up before the end of the year, we offer a 20% discount.

Rest assured, with our partner ExterNetworks, you’ll be publishing your Open Data using our extended version of CKAN with confidence, while we prioritize indicators and pre-seeding and harvesting data for jurisdictions that sign up.

So you’ll get all the benefits of Open Source – hundreds of CKAN contributors ranging from companies like Ontodia to State governments; no lock-in; the ability to tailor your data portal with our PaaS offering; access to hundreds of CKAN extensions – with the peace of mind of a managed cloud offering running on Amazon Web Services, backed by a Service Level Agreement.

Join us in our journey to democratize Open Data for All!

jnatividad

This entry has 1 replies

  1. Steven Adler says:

    Great work Joel. This is a real advance. Congrats.