For our seventh Civic User Testing Group session, we wanted to get feedback on the EveryBlock iPhone app after EveryBlock relaunched on January 23, 2014. This in-person test took place at one of the Connect Chicago locations – Chicago Public Library Mayfair Branch at 4400 W. Lawrence Avenue in the Mayfair neighborhood.
EveryBlock lets you follow neighborhoods and get information about what is happening around you. From the EveryBlock Web site:
“Our goal is to help you be a better neighbor by giving you frequently updated neighborhood information, plus tools to have meaningful conversations with neighbors.”
We were interested in answers to these questions:
- Are there any problems using the iPhone app? Does the app crash?
- Do users have an easy time posting or commenting on the app? Do they find things they want to comment on?
- Do users like the different pages in the app?
- Would users use this app and in what scenarios might they post or comment? Would they use it “on the go”?
Segmenting
On February 5, we sent out an e-mail to 269 CUTGroup participants who said they had an iPhone as their primary or secondary device. We asked them if they would be willing to test a neighborhood app on February 10. We also asked some screening questions to gather extra information, and chose our group of participants based on a diverse selection of answers.
Here is a look at the distribution of iPhone users in our CUTGroup:
View iPhone Users for CUTGroup 7 in a full screen map
We were interested in having about 15 participants from different Chicago neighborhoods. We had 12 testers who came from neighborhoods across Chicago including Albany Park, Hermosa, Edgewater, Uptown, Logan Square, Auburn Gresham, and more. The furthest a tester traveled from their home location was 15.3 miles.
View CUTGroup 7 Participants in a full screen map
Test Format
This in-person test was the first opportunity we had for every tester to be paired with a proctor. In previous tests, some testers were paired with proctors, while others would complete answer questions about the Web site or app through an online form.
Smart Chicago and EveryBlock staff (including a developer who worked on the app) acted as proctors for this test. We understood that there might risks in having EveryBlock staff ask questions to testers since biased responses might be captured. We asked that all EveryBlock staff not introduce themselves as staff or indicate that they were associated in any way to the app.
We received both positive and negative feedback from testers even with this format. We value this opportunity for developers to be involved in the CUTGroup process. The open communication and active involvement between developers and residents are key in developing Web sites and apps to improve lives in Chicago.
Results
Are there any problems using the iPhone app? Does the app crash?
92% of testers liked the EveryBlock iPhone app, and most testers thought the app was easy to use. 4 out of 12 testers (33%) experienced a bug when using the app.
After signing up or opening the app, two testers experienced bugs on the first “Timeline” Page.
- Victoria P Test (#4) did not have anything populate on the “Timeline” page. Content appeared in mobile web but not in the app. Victoria P Test was able to see content on “Near You Now” page
- Software Breaker (#10) had an error message appear saying there is a problem connecting to EveryBlock
The app crashed when Amateur Student (#12) clicked on the New Post icon. In addition, the app crashed when Application Developer (#2) hit “Account Settings.” Once she made a post, the app did not redirect her to a new page, but instead went to a black page.
Do users have an easy time posting or commenting on the app? Do they find things they want to comment on?
New Comments:
- In regards to making a comment, 58% of testers found a post they would want to comment on. Most testers thought it was easy to make a comment and was similar to other apps
- 42% of testers did not want to make a comment. Some testers were not comfortable making comments, while others addressed privacy issues in making comments. One tester found a post that she might comment on, but the post was not formatted for comments
- One tester explained that they “feel like not much of a contributor.” For many testers, the decision not to make a comment is due to personal use of apps and Web sites. Two testers specifically were concerned about privacy on EveryBlock. A suggestion from a tester is clarity and options to change how names will appear on posts and comments
New Posts:
- 75% of testers liked the new post interface, and when asked what testers would want to post about half of the testers mentioned specific neighborhood news: neighborhood concerns, crime warnings, traffic, weather, favorite places, garage sales, missing pets, events
Do users like the different pages in the app?
67% of testers thought that “Timeline” was the most useful page. As mentioned before, two testers did experience problems with this page, and this number could be even higher if these issues did not appear.
- Testers liked this page the most because it showed the most relevant information. Testers were also interested in the “Near You Now” page when “on the go” to learn more about other neighborhoods
- One tester wanted to add more zip codes to follow through the app to create a more personalized experience
Would users use this app and in what scenarios might they post or comment? Would they use it “on the go”?
83% of testers liked the option of making posts on the go, 17% of testers were not sure if they would like it or thought they might use it rarely. The majority of testers thought this was a convenient option, and wants to comment on things while they are happening. One tester commented that she is a Twitter user and is used to this concept.
What do users want to see in the EveryBlock app?
While 75% of testers used EveryBlock, none of the testers ever used the iPhone app. Therefore, some testers had knowledge of the EveryBlock Web site and could compare these experiences. Testers who were familiar with the Web site were interested in having these features:
- Search feature
- Filters or an ability for users to choose what they want to see
- Bookmark feature to read posts later
- User Profile – more apparent and have incentive to contribute and “gain status” in the community
- More information on Community Guidelines & Privacy
- Address finder and map capabilities to help find locations
Here are some other suggestions:
- Testers who did not use EveryBlock in the past were still interested in personalizing their experience. These testers also wanted to see features like filtering, search bars, and an option to delete irrelevant posts from the feed
- Two testers would like to see dates and times on posts to see if posts are the most relevant
- 67% of testers were interested in sharing posts on social media
- More information on privacy is important to testers. There should be options to see how much personal information is being posted, and to change privacy features within the app
One tester was shocked to find the post above during the test about a break-in because it included the poster’s full name and the address. This tester was concerned for her own privacy and was interested in knowing how her information would appear before she posted or commented.
Conclusion
Testers generally had positive experiences using the EveryBlock iPhone app. The biggest takeaway from this test is that users are interested in more features to filter and find the news most relevant to them. Testers were able to point out features from the EveryBlock Web site that they would want to see on the iPhone app. Including these features would allow users to have an experience that matches their experience on the EveryBlock Web site.
Final Report
Here is a final report of the results with the key highlights from our CUTGroup test, followed by each tester’s responses, and copies of our e-mail campaigns and the questions we asked:
The raw test data can be found below with the complete answers from every tester.
