Transit Night at the OpenGov Hack Night



At this week’s OpenGov Hack Night, we had presentations from Ed Zotti and Joe Iacobucci about transit and data in Chicago.

Joe Iacobucci is a Chicago transit enthusiast and gave a presentation on the link between public transit and economic development.

As an example, Iacobucci used a site called Mapnificent to show a locations access to transit by calculating how far a traveller could go by using public transit. Mapnificent uses transit data to calculate all the areas that can be reached by public transit in a given period of time.

For example, a rider that starts in the loop could go as far as Wilmette, Maywood, Homewood, or Calumet City in just 30 minutes. (Even without the Metra, a rider could still reach Evanston, Jefferson Park, or Englewood.) In comparison, a rider who starts in Cicero would only be able to reach as far as Greektown or Melrose Park in 30 minutes.

When you compare this information to real estate prices on sites like Zillow, you can see that the areas with greater transit options tend to have higher home values.

We also had a presentation from Ed Zotti, editor of The Straight Dope and assistant to the legendary Cecil Adams. Zotti ran through a recent history of Chicago Transit in terms of ridership capacity. Zotti has written extensively about transit in Chicago including a Chicago Reader feature on How to fix the El.

Resources for developers interested in transit issues.

Chicago has 43 sets transit data available to developers on the city’s data portal. This data includes everything from transit routes, bus ridership, to the annual boarding totals all the way from 1988. The CTA also has three APIs for developers to use in their own apps. These include bus tracker, train tracker, and a feed of all customer alerts.


The CTA even has instructions on how to build your own CTA Transit Info Display.

The next OpenGov Chicago Meetup:

The next OpenGov Chicago meetup will be this Thursday at the Chicago Community Trust. We’ve invited three speakers to give thoughtful critiques of the open government movement in Chicago. These speakers include:

  • Ramsin Canon, political editor at GapersBlock.com will provide perspective on the extent to which the movement benefits local communities.

  • Terry Pastika, Executive Director of the Citizen Advocacy Center will give a view of the current state of democracy in Illinois, with a focus on the Western and Far Western suburbs of Chicago.

  • Mike Stringer, Managing Partner at Datascope Analytics and organizer of the Data Science Chicago meetup group, will talk about whether we’re asking the questions most worth answering.

If you aren’t able to attend in person, the meeting will be live streamed on the Smart Chicago Collaborative blog.

Schoolcuts.org: Open Data and Civil Discourse


Last week, Chicago Public Schools announced that it was closing 61 schools due to budget constraints. Even before the list was announced, the plan to shut down schools was and still is generating lots of heated debate.

CPS has released data on each school, but it isn’t always organized in a way that makes it easy for parents to see what is going on at the school. To find out information on the school utilization, you would first visit a separate 19 page PDF file to see how CPS determines utilization. You then have to download an excel file and search through it to find the school you are interested in. This is a particularly thorny problem for parents and community members who care deeply about their schools as community anchors.

Schoolcuts.org Screenshot

What schoolcuts.org does is pull out all the available data on every school that is either being closed or receiving and put it in one place that’s easy for parents and community members to see.

https://soundcloud.com/morningshiftwbez/130322-morning-shift-seg-c

Listen to Schoolcuts.org’s Jeanee Olson talk about the site on WBEZ Morning Shift

Getting the data out there to the community in a format that’s easy to understand is extremely valuable. Not only is it important for parents to know what kind of schools that their children are being sent to, but having the data readily available makes for a better debate about school closings for all those involved.

One of the points of contention is that Chicago Public Schools has stated that children would only be moved to higher performing schools. CPS places schools into three tiers with regards to academic performance with he first tier being the best performing. However, there are several receiving schools that are Tier 3 – meaning they are the worst academically performing schools in the district. Because this data is open, and is being presented in plain language, community members can use this data to advocate for their schools.

Open data can and does aid in civil discourse.

Another point of contention is the role of charter schools and how they affect the neighborhood schools. One sides states that charter schools do a better job of teaching our children, while the other side states that opening additional charter schools robs resources from struggling neighborhood schools.

The Chicago Tribune wrote an editorial supporting charter schools stating that there were 19,000 students on waiting lists for charters schools in Chicago. This number was then disputed by WBEZ.

https://twitter.com/WBEZeducation/status/316361105410764800

WBEZ’s point was that the list of students on waiting lists for charters was generated by combining the waiting list of each school, some of which had students that had applied to multiple charter schools. 

Instead of just rhetoric, we’re now seeing debates in the public domain about the data. And that’s a good thing. This isn’t the only example of this being done. WBEZ’s Day by Datum blog recently provided a detailed explanation of the recent data spat between the Chicago Sun-Times and the CTA over crime data.

Sometimes the best civic apps are not the ones that give us the answers, but the ones that bring up the hard questions – David Eads

As we talk about open data and the ability of civic apps to solve problems and help us answer questions about civic lift, it’s important to realize the potential that open data has to improve civic discourse. Schoolcuts.org has helped to steer the course of the debate back to the data and that’s a powerful thing.

Data Potluck: 7 Million Rows of Data

There were a lot of people at this week’s data potluck

Data Potluck is a monthly event occurring the last Tuesday of every month at 6:00pm inside 1871.  Like the OpenGov Hack nights, these events focus on how open data and civic apps can help improve the citizen experience. However, these events have a more non-profit focus to them. Data Potluck was inspired by last year’s DataKind Data Drive which helped gather data for the Chicago area Red Cross. In order to keep the effort moving forward, Young-Jin Kim, Matt Gee and Nicholas Mader started the DataPotluck Meetup group.

DataPotluck’s other advantage? People bring food.

Rayid Ghani, Chief Scientist for Obama for America

At this month’s Data Potluck we had two presentations. The first was from  Rayid Ghani, former Chief Scientist for the Obama for America 2012 Campaign. Rayid explained how the Obama for America campaign used the power of predictive analysis and social media to help win the election. 

Rayid announced that the same model that made the Obama team so effective at their outreach efforts would be made available to non-profits.

Historical Traffic Congestion Data

The second presentation was by the City of Chicago’s Chief Analytics officer to announce the release of a seven million row dataset. Chicago has just released data on traffic congestion by segment.


To get an idea of just how big this data set is, a traffic segment is about a half mile. The city has 300 miles of road that the city keeps real-time traffic data for. The city refreshes the database that lives on the portal every ten minutes.

The city first released the real-time data in December, but civic developers wanted to take a look at historical data.

So, the city worked with Socrata to enable the city’s data portal to be able to handle such a massive volume of data. Now, civic developers can dig into all of the Chicago’s traffic data.

To help developers dig into the data, they’ve created a very well documented API.

This documentation includes code samples in multiple languages on how to access the data as well as definitions on all the different fields in the data set and the possible errors you could get.

Now that this data has been released, we’re excited to see what cool, useful, and interesting things that people will do with this data.

If you want to work with civic data:

For people who are interested in working with civic data, there are two opportunities that they should look into.

The first is the Chicago Data Science Fellowship. The University of Chicago and Argonne National Laboratory are recruiting people with statistics, programming, and data skills to work with real world data to make an impact on social issues.

The second is that the City of Chicago is hiring a data scientist to help ensure that Chicago becomes the very best civic data team in the world. The City of Chicago is looking to hire a new data scientist to join their team. If you are interested, you should apply on the city’s website.

Now Hosting: OpenStreetMap Metro Extracts

OpenStreetMap LogoThe Smart Chicago Collaborative is happy to announce that it is now providing the computing infrastructure behind Michal Migurski’s OpenStreetMap Metro Extracts project.

The Metro Extracts project provides small, easy to work with segments of the entire OpenStreetMap planet file. Michal has developed a system to automatically generate extracts for 168 major world cities and their surrounding areas. His program uses cost-effective spot EC2 instances and S3 file storage, all within the Smart Chicago hosting infrastructure.

A developer who wants to make a map of the Chicago metro area, for example, only needs to download a 86 megabyte file, instead of the 27 gigabyte planet file. That’s a 99.7% difference! The project also offers data for download in many other formats, including shapefiles that work with mapmaking tools like TileMill. This project will save developers time and effort as they focus on building beautiful, useful maps and applications.

The Metro Extracts project is continually expanding to cover more cities around the globe. Anyone may suggest additions to the list of cities; instructions for requesting a change are on the project’s Github repository.

Smart Chicago is very happy to sponsor OpenStreetMap development, including two upcoming events here in Chicago: a map-a-thon April 20 – 21, and an OpenStreetMap hack weekend, April 27 – 28. You can learn more about those events in our previous post.

Smart Chicago provides free hosting infrastructure for a number of civic developers and projects. For more information, including a signup form to get started, see our hosted web space project.

We are very excited about this project and its potential applications. We look forward to working with Michal to make it easier for Chicago-area developers to use OpenStreetMap data in their applications.

An open source admin tool for fusion tables

Smart Chicago’s Connect Chicago campaign is part of the federal Broadband Technology Opportunities Program. Smart Chicago has helped administer millions of dollars of these funds in programs for its partner, the City of Chicago.

The Connect Chicago location tool tool is a Ruby on Rails app that powers the map and detail page section of our site.




Origins

As part of this project, we hired Chicago developer Derek Eder, who has also worked on other Smart Chicago Collaborative since July 2012. The Connect Chicago Locator originated from Derek’s open source Searchable Map Template. It is powered by the free Google Fusion Tables service that lets you take spreadsheets of data and turn them into maps and other visualizations. His template, which can run on any web server, connects to your Fusion Table, reads in the data, and displays a searchable, filterable map. This template is super-useful and has been used by organizations like the City of Chicago, Chicago Public Schools, and the Chicago Tribune to get map-based sets up and running quickly.

The issue with the Searchable Map Template in the context of this project was that once the map is published, the only way to update it is to directly change the underlying Fusion Table through Google’s interface.

At Connect Chicago, we cover more that 250 places where you can use a computer for free, and it’s important that people in each of these locations (librarians, computer lab managers, social services agency leads, and so on) be able to update their detail page easily.

With an admin tool, managers at each location could update its own detail page with images, tables, contact info and transit directions. As far as we can tell know, no one has made a tool like this for Fusion Tables before.

We also knew that this general idea– the ability for non-technical people to update a robust web map with updated information– was broadly useful to Chicagoans. All of the code for this tool, including the underlying admin module, is available in this package.

One nice feature that I wanted in there, based on my long-term love for Flickr (30,000+ Creative Commons photos posted over nearly a decade): all photos on the detail pages are pulled from Flickr using system-generated unique tags. For example, here’s a bunch of photos I took of the Merlo Library on Belmont and tagged with “pcc-merlo-157 “).

Here’s some technical details from Derek about how we got this done:

The fusion_tables gem

The piece of technology that made these advanced features possible was the fusion_tables gem by Simon Tokumine. This gem, or code library, acts as an interface to read and write data to Fusion Tables from a Ruby application. It sounds simple, but in fact, adds a huge amount of power and convenience for building websites that interact with Fusion Tables. With this gem, we were able to fetch a row based on a URL and populate a detail page with the information. 

Here’s a snippet of how it works (viewing the Connect Chicago Fusion Table alongside will help):

#initialize Fusion Tables API
FT = GData::Client::FusionTables.new      
FT.clientlogin(APP_CONFIG['google_account'], APP_CONFIG['google_password'])
FT.set_api_key(APP_CONFIG['google_api_key'])

@location = FT.execute("SELECT * FROM 
1xy_wp4-NhtKPecuDlhsS8MLO0z-g5ayY1OfBhAg WHERE slug = 
‘roosevelt-library-1101-w-taylor-street';").first

From there, we have a handle on this @location object, and I can access different attributes like:

@location[:organization_name] # returns ‘Roosevelt Library’
@location[:address] # returns ‘1101 W. Taylor St’

Ruby on Rails Active Model framework

Now that we had an easy way to dynamically read data from our Fusion Table, we needed to give our site administrators some way to edit their data. On any web form, you want to make sure that the information people are entering is correct and what you expect. Form validation is the way to do this. Because we were using Ruby on Rails 3, we were able to leverage their powerful Active Model framework to handle it.

Active Model allows you to extend the non-database functionality of Active Record (an Object Relational Mapper) to any object. Remember our @location object we got from our fusion_tables gem? That is just a simple list of attributes and values (otherwise known as a Hash). By using Active Model, however, we can treat it like an Active Record object and get our form validation for free!

Inside the locations_controller:

@location = Location.new(location_edit)
if @location.valid?
   # save our data
else
   # oops! didn’t validate, take them back to the edit page
end

And here’s our location model:

class Location
  include ActiveModel::Validations
  include ActiveModel::Conversion
  extend ActiveModel::Naming

  # this line tells ActiveModel to make sure that orgainzation_name and others aren’t empty when validation is called
  validates_presence_of :organization_name, :organization_type, :org_phone, :address, :city, :state, :zip_code

  def create_method( name, &block )
    self.class.send( :define_method, name, &block )
  end

  def create_attr( name )
    create_method( "#{name}=".to_sym ) { |val| 
      instance_variable_set( "@" + name, val)
    }

    create_method( name.to_sym ) { 
      instance_variable_get( "@" + name ) 
    }
  end

  def initialize(attributes = {})
    #read in a hash of attributes from Fusion Tables and set them as attributes of the model
    #for more, see http://railscasts.com/episodes/219-active-model
    attributes.each do |name, value|
      name = "#{name}"
      create_attr name
      send("#{name}=", value)
    end
  end

  def persisted?
    false
  end
end

Advancing the utility of a useful tool

From here on out, if you need a Searchable Map Template with an easy-to-use admin tool for distributed detail page authorship, use the Connect Chicago Locator Tool. We look forward to seeing what comes of this!

 

Two OpenStreetMap Events

One of the main objectives of Smart Chicago is to help build the infrastructure of civic innovation in Chicago. That’s why we publish so much open source code, host so many apps, and support so many events.

Since so much of  civic innovation relates to place, there’s nothing more infrastructural than the maps upon which we publish information. That’s why we’re stoked to support Ian Dees and the work that he & others are doing around OpenStreetMap.

OpenStreetMap is the wiki-style map of the world. Just like Wikipedia, you can add or change any part of the map, making it better for the community around you. In Chicago, the map is particularly detailed thanks to help from many dedicated mappers and data released by the city. The more people contribute, the better the map will get.

OpenStreetMap

April is a busy OpenStreetMap month in Chicago. They are putting together two exciting weekends of events that we’re supporting along with our partner, Knight-Mozilla OpenNews: the Map-a-Thon and Hack Weekends.

OpenStreetMap Map-a-Thon

Beginning mappers are invited to be a part of a national OpenStreetMap Map-a-thon by learning how to use our tools to improve the map in your area. You can add your favorite restaurant or comic book store, a local school or hospital. During the map-a-thon we’ll walk you through the process of finding your area, creating an account, and making your first edit. With that foundation, you can go on to make an impact by adding tons of information relevant to you and your community!

Attend the Map-a-thon April 20th and 21st at 1871 on the 12th floor of the Merchandise Mart, 222 Merchandise Mart Plaza from noon to 6pm. Participants will enjoy food and drinks thanks to Smart Chicago Collaborative.

For more information about the map-a-thon and to RSVP, please visit the Meetup page for the event.

OpenStreetMap Hack Weekend

If you know your way around a compiler, feel comfortable with JSON and XML, or know the difference between an ellipsoid and a geoid, then the Hack Weekend is for you. We’re looking for those with technical know-how to help make a difference in OpenStreetMap’s core software by writing patches and new software to help make mapping faster and easier. Special thanks to Knight-Mozilla OpenNews for their support and sponsorship.

The hack weekend will be held April 26th and 28th at 1871 from 9am to dinner time each day.

For more information about the hack weekend, please visit the OSM wiki page for the event.