mRelief

SMART CHICAGO IS MOVING!!!

Kyla Williams

Good News!!! The Smart Chicago team is moving and now will be co-located with the City Digital Team at UI Labs. As such, our individual emails will be changing to:

Kyla Williams           

Sonja Marziano       

Denise Linn               

Leslie Durr               

Our new mailing address is 1415 N. Cherry Avenue Chicago, IL 60642 and general phone number is 312.281.6900.

Please check our website at smartchicagocollaborative.org or follow us on twitter @smartchicago for more updates.

We appreciate your patience during this time of transition.

LISC Chicago Begins Integrating Digital Skills into Financial Opportunity Centers

Denise Linn

Under Connect Chicago, LISC Chicago Financial Opportunity Centers are integrating digital skills training into their programming. This investment was launched on Saturday April 16th by Mayor Rahm Emanuel and other Chicago leaders.

LISC pic

This investment builds on an infrastructure of trusted community institutions that already provide income support, financial literacy training, and job training to residents. The blending of digital training components with traditional LISC programming has created pilot successes in the past. Under the Smart Communications Demonstration between 2011-2012, LISC Chicago Financial Opportunity Centers found that patrons who participated in digital skill training alongside other support services were 50% more likely to get a job than those that didn’t.

To assist with this integration, LISC Chicago hired Skill Scout. According to the Skill Scout website:

Skill Scout’s team is comprised of professionals with deep experience in workforce development, community organization, community collaboration, and the design and deployment of novel solutions in the market.

We came together through our work at gravitytank, on a design project to connect job seekers to employment in a more meaningful way.

There are 10 LISC Chicago Financial Opportunity Centers (formerly known as Centers for Working Families) where digital skills training will be integrated:

  • Center for Changing Lives
  • Central States SER
  • Chicago Commons
  • Instituto del Progreso Latino – Pilsen Site
  • Jane Addams Resource Corporation
  • Metropolitan Family Services
  • North Lawndale Employment Network
  • Preservation of Affordable Housing (POAH)
  • Safer Foundation
  • The Cara Program – Quad Communities.

We know that there are other community organizations that are seeking to integrate digital skills training into their work and missions. For instance, the Chicago Citywide Literacy Coalition has created a Technology Pilot Program to embed computer training in adult education and literacy work. LISC Chicago, under Connect Chicago, will not only integrate digital skills into their programming, but also share out their integration experience and lessons with others. By sharing, they can benefit institutions across Chicago as well as LISC Financial Opportunity Centers nationwide.

IMG_6860[1]

Skill Scout’s first task was to scan the field and assess existing Financial Opportunity Center services, resources, and challenges. On April 14th, Skill Scout convened representatives from Financial Opportunity Centers and shared preliminary findings from this first phase of work. A few themes arose:

It’s important to break down “training silos”

Digital skills shouldn’t always be taught as a completely separate track – rather, there should and can be multiple formal and informal ways of onboarding people into learning computer and technology skills.

For instance, a Resume Prep Class should leave students with a completed resume, but also leave them proficient in Microsoft Word and give them the knowledge they need to save and forward that resume electronically. Or, a financial coach assisting with a job search can teach a student how to use Google Maps – that job seeker can use the map to scan businesses in their neighborhood and see where they might want to apply first.

There is a need for more (or at least centralized) instructional tools

IMG_6864LISC Financial Opportunity Center staff took inventory of some of their favorite teaching tools and online resources: Blue Ocean Logic, Credit Karma, Kahn Academy, and Google forms. One trainer even mentioned that she likes to create mock online job applications through Google forms so that students can get comfortable with with forms and typing. Also, because it’s on a Google form that she controls, she can see their answers and give them feedback.

Staff at Financial Opportunity Centers mentioned that majority of their students have smartphones. Some want instruction to set up an email account on a smartphone, workshops coming up on taking professional pictures, or instructions to forward a resume. Also, students sometimes don’t know how to transfer skills from a smartphone to a computer. Some students even try to type up their resume on their smartphones. Creating a suite of tutorials to meet this demand for smartphone instruction is something that the Connect Chicago Meetup has discussed before.

Building in practice time is key

All agreed that all teaching tools and methods should be interactive and hands-on, if possible. When learning new skills, regular practice is just as important as learning. Since people in programs at Financial Opportunity Centers are very busy, it’s important to build purposeful practice time into existing lessons instead of filling the entire time with instruction.

IMG_6862

There is demand for more coordinated, standardized assessment

Trainers and staff agreed: regular, standardized pre and post instruction assessment that trainers could use across Financial Opportunity Centers would be ideal.

Northstar is an assessment that Chicago Commons recommended the Northstar assessment uses. The assessment gives them enough information to understand the needs of students. The Center for Changing Lives pointed out that all pre and post assessment done should be in the context of a concrete goal relevant to the student’s life.

Skill Scout Videos for LISC Chicago Financial Opportunity Centers

Skill Scout worked with LISC Financial Opportunity Center staff and trainers to develop short video tutorials to address some of the most common student challenges: uploading a resume, copying & pasting, twin accounts, how to understand your paycheck, and how to access benefits. For the “How to Access Benefits” tutorial, Skill Scout highlighted MRelief, an all-woman software development team that Smart Chicago has supported through our CivicWorks Project and tested through CUTGroup.

See them below:

mRelief Launches at Y-Combinator Demo Day

Smart Chicago

Smart Chicago partner mRelief launched a new tool to make public assistance more accessible. See more here on Tech Crunch. Here’s a rundown of the support we’ve provided mReleif over the years:

In November 2014, Smart Chicago provided $15,000 through our CivicWorks Project, which is funded by the Knight Foundation. Here’s a blog post about that and other investments. mRelief has gone on to receive a Knight prototype grantand a Knight News Challenge grant on their own.

Also since November 2014, we’ve hosted text-messaging for three of their programs: Early Learning, RTA Ride Free, andMedicaid, under our Developer Resources program.

In January 2015, we conducted a CUTGroup test for mRelief— see the results here. Rose Afriye of mRelief wrote a guest blog post about the experience: The Civic User Testing Group and Other Listening Strategies.

Smart Chicago partner Purple Binder created their first API in July 2013. Smart Chicago funded this work in order to serve our Smart Health Centers project.  In January 2015, mRelief integrated with this API, which implements the Open Referral standard.

In August 2015, we worked with mRelief to add the Chicago Early Learning Finder into their service and integrate it with our Chicago Early Learning Portal project.

mrelief home page

 

The Civic User Testing Group and Other Listening Strategies

Smart Chicago

Note: this is a guest post by Rose Afriyie of our partner mRelief.

How do you keep your finger on the pulse of user needs? At mRelief.com, a startup with non-traditional users — beneficiaries of public assistance — this is a question that we constantly ask ourselves. We are helping our users solve for long wait times by providing them an avenue to help them assess their eligibility for public assistance through text messaging conversations and online questionnaires that help them gauge whether it is worth it to complete extensive applications. These forms return response pages and text messages that help them determine their eligibility and local resources through a partnership that we have with Purple Binder.

Our users don’t have a lot of economic power in society. An average online mRelief user is paid $1,321 a month and those who text in to determine their eligibility make $150 less in earned income. When you have decreased purchasing power, technology is seldom built with your needs in mind. But in interviews and surveys, our users have shared that they are humbled by our willingness to learn how we can better serve them and provide relief to the process of asking for government help.

Since we launched in September last year in Chicago, we had to commit to some listening strategies— activities we engaged in to hear our users and meet them where they are. Considering that we had 134 percent online user growth between May and June  and that between June and July we almost tripled the number of text messages processed by our system, we think we are on to something. We would love to share one key listening strategy that contributed to getting us to this point: The CUTGroup.

Landing Page Before CUTGroup:

mRelief Homepage before CUTGroup

Landing Page After CUTGroup:

mRelief Homepage after CUTGroup

Civic User Testing Group (CUTGroup)

Since we launched mRelief, we conduct quarterly user surveys to get a sense of what makes our users tick. In 4th quarter of last year, the revelation was 82 percent of respondents didn’t pay for SMS which gave us the affirmation needed to launch our SMS strategy in November 2014.

But the most in-depth survey by far has been the CUTGroup test we participated in during Q1 of this year, an initiative from Smart Chicago to to help developers listen to the needs of their users. It combines observational analysis with insightful questions through surveys.

CUTGroup insights on our website usability combined with Google Analytics data on form completion and bounce rate were catalysts for redesigning our entire site with key leads on what should be areas of focus. Especially helpful was the notion that our icons on our pre-CUTGroup landing page were not clearly understood by 4 out of 6 of the users who mentioned our icons.

Other features that were the result of usability feedback led to rethinking our calculator by positioning a link to it near income questions and making all popovers/help text pop out as soon as a user enters data into a field. Based on typos, resulting from auto-correct and human error, we also revamped our SMS experience with more notices and additions that left users feeling like they weren’t penalized for mistakes. We helped users who texted in stay on the same text message if they made an error– all made possible through observational analysis in the CUTGroup.

Golden Nuggets for Future Consideration

I live 8 minutes from the Martin Luther King Community Service Center where we launched our first pilot involving case workers who served as navigators for our tools. There are times, on my way to work, that I will stop in and just wait with the folks we serve. I will listen. Observe folks — the phones they use, the questions asked about eligibility and surmise what the growing pain points are. For many startup co-founders, in-person surveys are time-intensive and are an “and” strategy combined with other world wide web magic. So, I also want to share two dope insights that we hope to integrate into listening strategies for the future:

  • Feedback Questions Integrated Within Your Tool – Cathy Deng at Data Made, a designer and developer we adore, has a listening strategy that integrates instant feedback on the tool itself. One contribution she made to the recently announced chicagosmilliondollarblocks.com was a feedback question seen here:

golden nugget

  1. Analytics, Analytics, Analytics – For those whose technology solution is primarily on web, listening with cutting edge analytics services is also crucial. Keen.io is one analytics as a service tool and Heapanalytics.com automatically captures hovering, scrolling, clicking and more that a user will engage in on your site.

So chime in, folks, tell us how are you listening?

See how we have integrated learnings into our site at www.mrelief.com

mRelief is also currently looking to pair with folks who have expertise in Angular JS. E-mail us at [email protected] if you are interested in supporting tools that modernize public benefits for all.

Chicago Early Learning Finder at ChiHackNight

Christopher Whitaker

mReliefChicagoEarlyLearningFor the August 11th #ChiHackNight, mRelief, the City of Chicago’s Cara Bader and our own Sonja Marziano talked about the new Chicago Early Learning Finder.

The Chicago Early Learning Finder lets Chicago residents see if they are eligible for Early Childhood Programs through either the mRelief website or by texting “Hello” to 773-377-8946.

The finder works by having residents answer short and simple eligibility questions. Once completed, the finder recommends three locations based on the resident’s eligibility and preference. When the resident selects the location they want it will connect them to the Early Childhood Learning Portal where they can kick off the enrollment process.

Smart Chicago has run the Early Learning Portal since 2012. In 2013, they worked with the Azavea team to integrate SMS messaging into the portal.

At the time, here’s what Azavea had to say about the project.

“An interesting challenge the design team will face in the upcoming work will be to refine the SMS interface to the application. During the usability tests and demos of the application, we’ve received a lot of excited feedback about this feature. It provides a way for users to access the data behind the application by sending and receiving text messages. There is a dearth of resources that describe good user experience (UX) design in the realm of SMS interfaces, so through the examination of existing SMS products and iterative redesign, we are looking forward to learning some of the tricks to creating a great SMS user experience.”

With mRelief’s experience with screening for social services, they were the perfect fit to help residents understand the different options for early learning childhood programs. This is one of the largest screeners that mRelief has ever built. Here’s the timeline that they used to build the Learning Finder.

productroadmap

The City of Chicago paid mRelief to do this work— one of the first civic startups in Chicago to conduct business directly with the city.

To find out more about mRelief, you can visit their website at mRelief.com.

CUTGroup #13 – mRelief

Sonja Marziano

Receiving a text message from mRelief on a SamsungFor our thirteenth Civic User Testing Group session, we tested mRelief, a website that helps residents check their eligibility for social services benefits in Chicago and Illinois. This in-person test took place at one of the Connect Chicago locations – Chicago Public Library King branch at 3436 S. King Drive in the Douglas community area.

The mRelief team was interested in testing these things:

  • Language: We heard that language is very important in understanding if someone is eligible for social services. We wanted to see if testers understood terms such as: gross income (what to and not to include), value of assets, household size, etc. The mRelief team also uses pop-overs and wanted to see if testers use them and if they were helpful.
  • SMS: We wanted to test the text-messaging platform to see if users found it easy to use and how long each session took to receive and send messages
  • Website usability: In addition to testing SMS, we wanted to see how testers liked the mRelief website and how easy it was to use.

Segmenting

On January 21, we sent out an email to all of our 834 CUTGroup participants. We wanted to know if they would be available for an in-person test on January 29 for about 30-45 minutes. We asked some screening questions to gather information. We wanted to focus on testers who were currently enrolled in or qualified for social services programs. We also wanted to focus on residents of the Southside of Chicago

We ended up having 11 testers participate in this test. In addition, 1 individual from the library wanted to be involved in the CUTGroup, and therefore, we tested with him, but did not include him in the results because he would not have qualified for the test.

Screening Questions

We heard form 71 CUTGroup participants through our callout for testers. We received a lot of good information just from the screening questions. Here is a look at what we learned:

  • 90% of CUTGroup respondents said they have a smartphone
  • 33% of respondents are currently enrolled in social services programs

Test Format

Due to the personal nature of discussing social services benefits with testers, we decided early on that we wanted to do one-on-one proctoring. We had a total of 7 people from Smart Chicago and mRelief help with welcoming testers and proctoring.

This was the first test that we were not only testing a website, but also wanted to test the text messaging platform. Testers were asked to go through both processes, and we wanted to see how they compared and if testers found one platform easier to use than the other.

Results

8 out of the 11 testers were currently enrolled in social services benefits, and all of these testers were enrolled in Food Stamp benefits. The 3 testers who were not currently enrolled seemed to have qualified for social services benefits based on their household size and income.

 Language

We learned from this CUTGroup test that testers found the form(s) when checking their eligibility to be a simple process. 7 out of 11 testers (64%) mentioned that the questions were easy to understand or straightforward. Although most of the language was very clear, however, only 1 tester clicked on a pop-over to find out more information. Therefore, any question that requires additional help text should be added after the question.

In addition, some testers did not notice that some questions might ask for monthly income vs others that ask for annual income. These are key pieces of information that should be made clearer in the question format.

SMS

Out of 11 testers, 10 testers (91%) do not pay per text message.

Only 5 out of 11 testers (45%) thought that the questions were “easy” or “straight-forward” when going through the text messages. The biggest thing we heard was that testers were “frustrated” or “annoyed” that they were unable to fix a mistake and had to start from the beginning.

Lastly, some testers received messages out of order, and some terms were split up between messages (ex: “Medicaid”).

1 of our testers, Simple One (#9), experienced a delay between text messages that lasted anywhere from 1 to 2 minutes. She thought it was “annoying” that she did know when the end of the questions would be. 3 testers thought the process was pretty fast!

Website usability

When reviewing the homepage, 6 out of 11 testers (55%) mentioned the graphics. 3 testers felt that the icons were very intuitive and understood that they connected to the social services benefits. The other 3 testers did not think the icons were as intuitive. 4 testers thought the graphics were icons that they could click on.

Most of the testers had a clear direction after reviewing the homepage. The majority (9 testers) clicked “Get Started” while the other 2 testers clicked on “Programs.” It was great to see that testers had a clear path in using the website that got them to the most important part — checking their eligibility for social services.

When checking eligibility, we noticed that a lot of testers liked the idea of the calculator to calculate income, but most testers did not use if because they had already answered the questions before coming across the calculator located at the bottom of the screen.

Documents & More Information

We wanted testers to tell us how important it would be for mRelief to provide additional information about documents that they might need to bring, or money that they would receive from the benefit. We hoped this would be a help for mRelief to decide what the next steps for the website might be. Here is what we learned:

How important is it for you to know what documents you need to prepare for an interview with a caseworker to receive public assistance?

5 Very Important      100% (11)
 4 Important                    0%
3 Neutral                         0%
2 Slightly Important     0%
1 Not Important            0%

Is it important for you to know why you do or don’t qualify for public assistance?

5 Very Important      82% (9)
 4 Important                     18% (2)
3 Neutral                          0%
2 Slightly Important      0%
1 Not Important              0%

How important is it for you to know how much money you will receive from public assistance before going to file with a caseworker?

5 Very Important      46% (5)
 4 Important                     9% (1)
3 Neutral                          9% (1)
2 Slightly Important      18% (2)
1 Not Important             18% (2)

How important is it for you to know how to file an application for public assistance?

5 Very Important      91% (10)
 4 Important                     0%
3 Neutral                          0%
2 Slightly Important      0%
1 Not Important              9% (1)

Final Report

Here is a final report of the results with the analysis of the questions we asked, followed by each tester’s responses, and copies of other questions we asked:


The raw test data can be found below with the written answers from every tester.