Month: March 2013

Now Hosting: OpenStreetMap Metro Extracts

Chris Gansen 1 Comment

OpenStreetMap LogoThe Smart Chicago Collaborative is happy to announce that it is now providing the computing infrastructure behind Michal Migurski’s OpenStreetMap Metro Extracts project.

The Metro Extracts project provides small, easy to work with segments of the entire OpenStreetMap planet file. Michal has developed a system to automatically generate extracts for 168 major world cities and their surrounding areas. His program uses cost-effective spot EC2 instances and S3 file storage, all within the Smart Chicago hosting infrastructure.

A developer who wants to make a map of the Chicago metro area, for example, only needs to download a 86 megabyte file, instead of the 27 gigabyte planet file. That’s a 99.7% difference! The project also offers data for download in many other formats, including shapefiles that work with mapmaking tools like TileMill. This project will save developers time and effort as they focus on building beautiful, useful maps and applications.

The Metro Extracts project is continually expanding to cover more cities around the globe. Anyone may suggest additions to the list of cities; instructions for requesting a change are on the project’s Github repository.

Smart Chicago is very happy to sponsor OpenStreetMap development, including two upcoming events here in Chicago: a map-a-thon April 20 – 21, and an OpenStreetMap hack weekend, April 27 – 28. You can learn more about those events in our previous post.

Smart Chicago provides free hosting infrastructure for a number of civic developers and projects. For more information, including a signup form to get started, see our hosted web space project.

We are very excited about this project and its potential applications. We look forward to working with Michal to make it easier for Chicago-area developers to use OpenStreetMap data in their applications.

An open source admin tool for fusion tables

Smart Chicago 1 Comment

Smart Chicago’s Connect Chicago campaign is part of the federal Broadband Technology Opportunities Program. Smart Chicago has helped administer millions of dollars of these funds in programs for its partner, the City of Chicago.

The Connect Chicago location tool tool is a Ruby on Rails app that powers the map and detail page section of our site.




Origins

As part of this project, we hired Chicago developer Derek Eder, who has also worked on other Smart Chicago Collaborative since July 2012. The Connect Chicago Locator originated from Derek’s open source Searchable Map Template. It is powered by the free Google Fusion Tables service that lets you take spreadsheets of data and turn them into maps and other visualizations. His template, which can run on any web server, connects to your Fusion Table, reads in the data, and displays a searchable, filterable map. This template is super-useful and has been used by organizations like the City of Chicago, Chicago Public Schools, and the Chicago Tribune to get map-based sets up and running quickly.

The issue with the Searchable Map Template in the context of this project was that once the map is published, the only way to update it is to directly change the underlying Fusion Table through Google’s interface.

At Connect Chicago, we cover more that 250 places where you can use a computer for free, and it’s important that people in each of these locations (librarians, computer lab managers, social services agency leads, and so on) be able to update their detail page easily.

With an admin tool, managers at each location could update its own detail page with images, tables, contact info and transit directions. As far as we can tell know, no one has made a tool like this for Fusion Tables before.

We also knew that this general idea– the ability for non-technical people to update a robust web map with updated information– was broadly useful to Chicagoans. All of the code for this tool, including the underlying admin module, is available in this package.

One nice feature that I wanted in there, based on my long-term love for Flickr (30,000+ Creative Commons photos posted over nearly a decade): all photos on the detail pages are pulled from Flickr using system-generated unique tags. For example, here’s a bunch of photos I took of the Merlo Library on Belmont and tagged with “pcc-merlo-157 “).

Here’s some technical details from Derek about how we got this done:

The fusion_tables gem

The piece of technology that made these advanced features possible was the fusion_tables gem by Simon Tokumine. This gem, or code library, acts as an interface to read and write data to Fusion Tables from a Ruby application. It sounds simple, but in fact, adds a huge amount of power and convenience for building websites that interact with Fusion Tables. With this gem, we were able to fetch a row based on a URL and populate a detail page with the information. 

Here’s a snippet of how it works (viewing the Connect Chicago Fusion Table alongside will help):

#initialize Fusion Tables API
FT = GData::Client::FusionTables.new      
FT.clientlogin(APP_CONFIG['google_account'], APP_CONFIG['google_password'])
FT.set_api_key(APP_CONFIG['google_api_key'])

@location = FT.execute("SELECT * FROM 
1xy_wp4-NhtKPecuDlhsS8MLO0z-g5ayY1OfBhAg WHERE slug = 
‘roosevelt-library-1101-w-taylor-street';").first

From there, we have a handle on this @location object, and I can access different attributes like:

@location[:organization_name] # returns ‘Roosevelt Library’
@location[:address] # returns ‘1101 W. Taylor St’

Ruby on Rails Active Model framework

Now that we had an easy way to dynamically read data from our Fusion Table, we needed to give our site administrators some way to edit their data. On any web form, you want to make sure that the information people are entering is correct and what you expect. Form validation is the way to do this. Because we were using Ruby on Rails 3, we were able to leverage their powerful Active Model framework to handle it.

Active Model allows you to extend the non-database functionality of Active Record (an Object Relational Mapper) to any object. Remember our @location object we got from our fusion_tables gem? That is just a simple list of attributes and values (otherwise known as a Hash). By using Active Model, however, we can treat it like an Active Record object and get our form validation for free!

Inside the locations_controller:

@location = Location.new(location_edit)
if @location.valid?
   # save our data
else
   # oops! didn’t validate, take them back to the edit page
end

And here’s our location model:

class Location
  include ActiveModel::Validations
  include ActiveModel::Conversion
  extend ActiveModel::Naming

  # this line tells ActiveModel to make sure that orgainzation_name and others aren’t empty when validation is called
  validates_presence_of :organization_name, :organization_type, :org_phone, :address, :city, :state, :zip_code

  def create_method( name, &block )
    self.class.send( :define_method, name, &block )
  end

  def create_attr( name )
    create_method( "#{name}=".to_sym ) { |val| 
      instance_variable_set( "@" + name, val)
    }

    create_method( name.to_sym ) { 
      instance_variable_get( "@" + name ) 
    }
  end

  def initialize(attributes = {})
    #read in a hash of attributes from Fusion Tables and set them as attributes of the model
    #for more, see http://railscasts.com/episodes/219-active-model
    attributes.each do |name, value|
      name = "#{name}"
      create_attr name
      send("#{name}=", value)
    end
  end

  def persisted?
    false
  end
end

Advancing the utility of a useful tool

From here on out, if you need a Searchable Map Template with an easy-to-use admin tool for distributed detail page authorship, use the Connect Chicago Locator Tool. We look forward to seeing what comes of this!

 

Two OpenStreetMap Events

Smart Chicago 1 Comment

One of the main objectives of Smart Chicago is to help build the infrastructure of civic innovation in Chicago. That’s why we publish so much open source code, host so many apps, and support so many events.

Since so much of  civic innovation relates to place, there’s nothing more infrastructural than the maps upon which we publish information. That’s why we’re stoked to support Ian Dees and the work that he & others are doing around OpenStreetMap.

OpenStreetMap is the wiki-style map of the world. Just like Wikipedia, you can add or change any part of the map, making it better for the community around you. In Chicago, the map is particularly detailed thanks to help from many dedicated mappers and data released by the city. The more people contribute, the better the map will get.

OpenStreetMap

April is a busy OpenStreetMap month in Chicago. They are putting together two exciting weekends of events that we’re supporting along with our partner, Knight-Mozilla OpenNews: the Map-a-Thon and Hack Weekends.

OpenStreetMap Map-a-Thon

Beginning mappers are invited to be a part of a national OpenStreetMap Map-a-thon by learning how to use our tools to improve the map in your area. You can add your favorite restaurant or comic book store, a local school or hospital. During the map-a-thon we’ll walk you through the process of finding your area, creating an account, and making your first edit. With that foundation, you can go on to make an impact by adding tons of information relevant to you and your community!

Attend the Map-a-thon April 20th and 21st at 1871 on the 12th floor of the Merchandise Mart, 222 Merchandise Mart Plaza from noon to 6pm. Participants will enjoy food and drinks thanks to Smart Chicago Collaborative.

For more information about the map-a-thon and to RSVP, please visit the Meetup page for the event.

OpenStreetMap Hack Weekend

If you know your way around a compiler, feel comfortable with JSON and XML, or know the difference between an ellipsoid and a geoid, then the Hack Weekend is for you. We’re looking for those with technical know-how to help make a difference in OpenStreetMap’s core software by writing patches and new software to help make mapping faster and easier. Special thanks to Knight-Mozilla OpenNews for their support and sponsorship.

The hack weekend will be held April 26th and 28th at 1871 from 9am to dinner time each day.

For more information about the hack weekend, please visit the OSM wiki page for the event.

OpenGov Hack Night: News Challenge Submissions and Mote.0.Bike

Christopher Whitaker

This is a weekly feature that will highlight what’s happening at the Chicago OpenGov Hack Night. The Chicago OpenGov Hack Nights are weekly events where technologists and community members come together to work with open data and build tools that improve the civic experience. The events are held at 6:00 pm each Tuesday at 1871.

This week’s first presentation: Knight News Challenge Submissions

newschallenge

The deadline for submissions for the Knight News Challenge was on Monday. Civic innovators are competing for their share of $5 million dollars that the Knight Foundation will be awarding to projects that help improve the way that citizens get information from their government. (Author’s note: Smart Chicago Collaborative Executive Director Dan O’Neil is on of the readers giving feedback for the Knight Foundation News Challenge. Projects were included in the post solely because they were presented at the hack night and for no other reason.)

Projects who presented at the OpenGov Hack Night included:

Schoolcuts.org / SchoolCircle.org: (4:34)

Their Knight News Entry: SchoolCircle.org engages parents, teachers, students, & community members with data visualizations about their schools made with a combination of publicly available and crowdsourced data. Users will be able to discuss, and advocate for, their school.

Chicago Crash Browser: (5:40)

Their Knight News Entry: Chicago Crash Browser is a new tool needed by planners and engineers to analyze where the Chicago should invest in infrastructure upgrades to eliminate traffic fatalities by 2022, and educate residents & elected officials about transportation safety.

Visible CTA (7:15)

Their Knight News Entry: To give a visual trip on all CTA routes and a bit of a walk in several directions from every stop. To connect a trip with where one could work through the Illinois Department of Employment Security and other government web sites.

Augmenting 311 Systems With Data Sourced From Social Media: (9:00)

Their Knight News Entry: Enhancing Open311 (and 311 systems in general) to accept information captured from social media outlets and filtered via machine learning and human interaction.

WeCountability (10:10)

Their Knight News Entry: WeCountability will flatten a city’s organization so that good ideas can make their ways from the people doing the day-to-day work to the people making the decisions.

Crowdsourcing Building Data: (10:44)

Their Knight News Entry: We will create a smartphone web app/website that community groups will use to share information about buildings that are abandoned or in foreclosure in order to make neighborhoods more stable, make policymaking more effective, and improve vacant building data quality through crowdsourcing.

Closed Loop (12:00)

Their Knight News Entry: A data mining tool that connects the dots between political campaign contributions, lobbying, and legislation – and detects unusual patterns for investigative journalists to look into.

Announcements:

Data Science for Social Good Fellowships (15:10)

This week’s second presentation: Mote.0.Bike (24:14)

School of the Art Institute of Chicago (SAIC) professors Doug Pancoast (Professor of Architecture) and Robb Drinkwater (Professor of Sound) dropped by the OpenGov Hack Night to discuss their project involving bike sensors.

The bike-mountable moto sensors

The bike-mountable moto sensors

“The interface reconsidered” was a collaborative course taught by Robb and Doug. They wanted to redefine interface, specifically in the context of the city.

They used an Arduino Micro-Controller combined with a GPS device to help collect data as people biked around the city. The Ardunio device is programmable and can be setup to collect a number of different data points including location, air quality, altitude, light, humidity and more.

Users can upload the data on the project website. The goal is to have multiple users all over the city collect data so that we can learn more about biking in Chicago. The project is being further by Colin Hutton and the site will see improvements over the next few weeks.

Dataset of the Week: Workforce Centers
Datasets don’t have to be big to be important. The City of Chicago has four different workforce centers. The city’s partnered with over 30 different community organizations to provide employment services.

Improving Adopt-a-sidewalk

Chris Gansen

TL;DR: Adopt-a-sidewalk is a flawed, under-utilized application with enormous potential. By refocusing the user experience on addressing actual needs of people in Chicago and showing meaningful activity, it could be a powerful tool for engaging citizens in supporting and  improving the civic infrastructure in their community.

Winter is officially in Chicago’s rearview mirror, although you would not notice from the chilly temperatures outside. This post is a reflection on one of Chicago’s winter-weather civic applications, Adopt-a-sidewalk, an application I helped bring online over a year ago, and how it can evolve to improve the lives of Chicago residents year-round.

Going Nowhere Fast @ Wal*Mart

Adopt-a-sidewalk is a Chicago-based version of the Adopt-a-hydrant web application built by Code for America in Boston back in 2011. Developed by Code for America fellow Erik Michaels-Ober, Adopt-a-hydrant lets residents of Boston volunteer to clear fire hydrants when there is a snow storm.

In the fall of 2011, City of Chicago officials, acutely aware of the severity and importance of swift snow removal, saw an opportunity to repurpose the code, and invited a group of civic developers to customize the application for use in Chicago. The key functional difference between the applications is that in Chicago, residents can request help clearing their sidewalk. Adopt-a-sidewalk first went live as part of ChicagoShovels.org in January 2012, and generated a bit of fanfare in local and national media:

  • New York Times: Snow Site Lets Chicago See if Plows Are Really in a Rut
  • ABC7 News: Mayor’s office launches ChicagoShovels.org
  • Chicagoist: City’s Adopt-A-Sidewalk Website Launches

Adopt-a-sidewalk saw moderate adoption, but quickly fell out of use due to a very mild winter, and the fast arrival of spring a few months later. In the fall of 2012, the City of Chicago asked the Smart Chicago Collaborative to assume the responsibility of hosting the application, and development responsibilities were handed over to the Code for America Chicago brigade.

To date, Adopt-a-sidewalk has seen very little adoption in Chicago. There are 557,793 individual sidewalk segments available for adoption, but only 75 registered users. 153 sidewalks have been claimed, either by volunteer shovelers, or people asking for help. That means that only 0.027% of all sidewalk segments in Chicago have been adopted. At its busiest, only 200 people visited the site in a given day.

There are three major issues that impact the usability and adoption of Adopt-a-sidewalk.

First, plainly speaking, the application is boring. In the case of a snow storm, there is a sense of urgency to responding and cleaning up the mess. The City deploys a fleet of snowplows to clear the streets, and neighborhoods are abuzz with residents scraping cars, shoveling steps, and snow-blowing their sidewalks and alleys. On Adopt-a-sidewalk, there is absolutely no perception of activity, urgency, or community. There is no mechanism to show users where activity is happening, or if there is a need for activity. On their first visit to the site, users are presented with a featureless, generic Google map of the city of Chicago, and no clear call to action. If the user does decide to register and adopt a sidewalk, there is little incentive to return or to refer friends to the site.

Second, the path to participating is laden with friction. Users must search using a real Chicago street address and register for an account before they may participate. Registering an account involves giving a name, email address, a password, and completing a captcha. There’s no mechanism to invite your neighbors to join you in shoveling, nor is there a mechanism to share your activity with your social network.

Third, the application is useless when there is no snow on the ground. Adopt-a-sidewalk is irrelevant in the summertime, and, for most of the winter spent between snow storms. There is no incentive to return to the site, and there is no meaningful action to take in between snow storms.

On a conceptual level, the premise of Adopt-a-sidewalk is flawed. Chicago residents are already expected to and, by ordinance, required to, shovel their sidewalks. Adopt-a-sidewalk provides no benefit to users who adopt the sidewalk in front of their house and dutifully shovel it each time snow falls. The steps to register and adopt their sidewalk is busy work.

The real work

Instead of asking users to do monotonous work, Adopt-a-sidewalk should focus on providing a real service: matching people in need of help with people willing to help. In that scenario, there are two key classes of users: people who cannot clear their sidewalks and people who are willing to help shovel sidewalks near them.

By shifting the interaction model from navigating a half million rectangles on a map to a focused, needs-based one, many of the core usability issues can be alleviated. It’s far easier to show activity, in the form of the most recent or most urgent requests for help, and the reward for participating is much more immediate and meaningful. Instead of highlighting what’s expected of people, the focus can be on enabling and rewarding people who want to help their neighbors.

The natural extension of this concept is to move beyond simple sidewalks and instead enable neighborhood adoption of any civic infrastructure. Adopting sidewalks could easily gave way in the spring and summer time to adopting parks and community gardens. In the fall, communities could band together to adopt a local school and fix it up before students return. A baseball team can adopt its ball field and organize events to maintain and improve it.

Fostering community around shared civic infrastructure is not a new concept. However, using technology, it is possible to integrate the real world thing with an online community, and the vast network of people and data that exists there. With the rise of open government data, not only is the civic infrastructure as physical object or place, it’s a continuous stream of data and interactions. The baseball diamond around the corner is not just a sandlot for shagging fly balls, it is a collection of data points: tweets, photos, and events created by community members, and crime reports, 311 requests, park facilities data from the local government.

I look forward to seeing where Adopt-a-sidewalk goes from here, especially if Code for America or one of the brigades takes some of the concepts from Adopt-a-sidewalk and pulls them back into the mainline repository. Adopt-a-sidewalk is, despite its flaws and low adoption, one very small step on a long path to building, enabling, and merging real life and online communities.

OpenGov Hack Night: Go2School and Business License Data

Christopher Whitaker

This is a new weekly feature that will highlight what’s happening at the Chicago OpenGov Hack Night. TheChicago OpenGov Hack Nights are weekly events where technologists and community members come together to work with open data and build tools that improve the civic experience. The events, run by Derek Eder and Juan-Pablo Velez, are held at 6:00 pm each Tuesday at 1871. As a founding member of 1871, the Smart Chicago Collaborative is proud to be able to provide space for this each week. 

This Week’s Presentation: GoToSchool by Tom Kompare

This week’s presentation is from Tom Kompare and his current app-in-progress GoToSchool. The app is help parents find directions to their kids school during those first few chaotic weeks. Tom is currently building the app hoping to have it released by the start of the next school year.

To use it, simply find your school by typing in the search bar. The app will try and help you by pulling up matching schools as you search. Once you select your school, you can state when you want to be there. Do you need to grab your kids after work? Plan for tomorrow morning? After you state when you need to be there, the app gives you three options on how you want to get there: walking, CTA/Metra, or by driving. It even gives you the number to call in case your kid is sick and can’t be at school.

The app is hosted on the Smart Chicago Collaborative servers and will be one of the first apps taking part in Civic User Testing.

Here’s how it works:

  • The site also uses Twitter bootstrap to make building the appearance of the app easier.
  • Tom used two separate data sets from CPS and placed those into Google Fusion Tables. The first is the school schedule and the second is school location data.
  • Transit directions are delivered through the Google Places API

Current Issues:

The app is still in development and has a couple of issues.

  • The data for start and end times for charter schools in incomplete
  • The “What time do you want to arrive” doesn’t look as good in Internet Explorer

Civic Developers and Designers! You Can Help Improve this app!

  • You can check out the app and submit pull request on the apps’ GitHub repository.

Dataset of the Week: Business License Data

This weeks’ dataset of the week is business license data. See that new construction across from your work? You can use the city’s business license data to pull up information on what is going into it. The city’s also built views that sort the data into different categories. For example, they have a view of the data that filters out everything but liquor licenses. You can turn that view into a heat map that where they are.

Socrata has a number of features that make exploring and viewing data easier. Once you register with the data.cityofchicago.org site you can make your own views and save them for later use.

Join us!

Are you interested in open data and civic innovation? Have something cool you’d like to show us? Register for the next OpenGov Hack Night here!