For our fourteenth Civic User Testing Group session, we tested Chicago Cityscape, a website that helps residents understand how, where, and when building development takes place in neighborhoods. This in-person test took place at one of the Connect Chicago locations – Chicago Public Library Logan Square branch at 3030 W. Fullerton Ave.
The developer, Steven Vance, wanted to learn these things from the test:
- Usability: Are there functions that are difficult to use or provide different information than expected? What do testers look at or do first when visiting the website? In what ways can Chicago Cityscape be improved?
- Desired features: Steven wanted a better understanding of the type of information that residents and homebuyers look for when researching neighborhoods or potential properties to live in. What are the pieces of information that residents/homebuyers most interested in? What pieces of information are they least interested in?
- Paid Services: Is there an interest in paying for these services at a low cost?
Segmenting
On January 30, 2015, we sent out an email to all of our 831 CUTGroup participants. We wanted to know if they would be available for an in-person test on February for about 30-45 minutes. We asked some screening questions to gather information. We were interested in getting residents who are involved and care about what happens in their neighborhood. We were also looking for people who were planning to buy a home. These homebuyers did not need to be as active in their community.
We ended up having 13 testers participate in this test.
Screening Questions
We heard form 62 CUTGroup participants through our callout for testers. We received a lot of good information just from the screening questions. Here is a look at what we learned:
- 60% of CUTGroup respondents are currently renting
- 37% of respondents are planning to buy a home in the next year
When we asked how involved testers are in their neighborhood, this is what we they said:
5 Very Involved 23% (14)
4 Involved 35% (22)
3 Neutral 31% (19)
2 Slightly Involved 3% (2)
1 Not at all Involved 8% (5)
Test Format
The format of this test was proctored sessions with proctors working one-on-one with the testers. Testers looked at the website using either their own laptops or laptops we provided while proctors asked questions and took down their responses. It was also the first time we used the software, ZoomText, which we added to one of the laptops for a tester who has low-vision.
Ever since the Roll with Me CUTGroup test, we have been trying to reach more people through the CUTGroup. Having a diverse set of individuals, such as low vision or blind testers, helps us better understand the structure and usability a website. For example, during this test, Architect Dropout (#11) tested using iPad voice-over commands. From that experience, Steven realized that the map failed because the voice-over commands selected each map tile individually and she could not select the specific pinpoints on the map. This led to a very detailed Github issue and conversation on accessibility improvements for Leaflet. This is a goal of CUTgroup — to not only uncover user experience issues on one website or app, but to help all developers build better websites that work for everyone.
Results
Information
12 out of 13 testers are interested in learning more about what is being built or torn down in their neighborhoods, and most look to community news outlets (such as DNA Info or Everyblock) for this type of information. When learning about a property, testers said that these are the most important pieces of information: property information, tax information, neighborhood data, and violations at that specific address.
When testers were asked to search for an address in their neighborhood, most of them searched for their home addresses and called out these pieces of data on the “Address Snapshot” page: demographics, Aldermen/ward information, property taxes, crimes, and then permits. While there was not a one-sided response to the “least useful information” question, a lot of testers had questions about the data they were seeing on the pages. Here are a couple of ways that testers would improve the Address Snapshot page:
- Change up the design elements: 5 out of 13 testers mentioned changing elements of the color and design as ways to improve the Address Snapshot page. HomeSweet (#12) says, “Visually the page is rather simplistic and not in a way that makes it easy to read. It’s not immediately clear that the ad is an ad.”
- Highlight or change information structure: 3 testers thought that information on the pages should change and different pieces of information should be highlighted. Testers also mentioned that there was a lot of information on these pages, and it would be better if they were organized with better definitions and descriptions.
When looking at the permit information, testers had an easier time navigating the permit information on the map versus the information listed on the community or neighborhood area. Testers again had a lot of questions about the data they were looking at, here are some examples:
“Are these permits already approved?” –Big on Community & IT Field #geekchic (#3)
“TIF district! I want to know what that is” –Non-Profit Pro (#6)
“What’s to be gained by showing the total estimated costs of the projects? What does change time mean? How do you get access to the records? Do you have a way to get it quickly?” – Construction Watcher (#2)
83% of testers thought that the permit information was useful or interesting. Two testers wanted to compare permit data in their neighborhood to other neighborhoods. In addition, one tester, Chicago Explorer (#10), thought that seeing the history of the neighborhood through permits is interesting. If she was starting a business or looking for a property it would be especially useful.
Usability
For the most part, testers found it very easy to navigate the website. There was some confusion when ads had a light background and matched Chicago Cityscape. In addition, testers had questions about the map icons and had questions about the data types or the timeline of what was being displayed.
When first exploring the website, most testers did not an overwhelmingly clear first step or action. 4 out of 13 testers (31%) viewed or clicked on the map on the homepage, 3 testers (25%) typed in an address, while the rest of the testers did a variety of other first actions. In addition, when testers were first reviewing the homepage, 5 out of 13 testers (31%) were not clear who the audience was.
Audience
Throughout the test, testers mentioned useful and not useful information on every page and often distinguished which information would be useful to one group but not to others. As mentioned before, testers had questions about the audience ever since the start of the test.
Architect Dropout (#11) says, “It doesn’t seem super targeted to your average Jane, seems targeted to a business, not very consumer targeted… doesn’t seem to want to sell me anything, feels more like an industry site.”
Both HomeSweet (#12) and Big on Community & IT Field #geekchic (#3) said that they thought this was a real estate/homebuyers website.
Chicago Explorer (#10) and First time looker (#9) both mentioned the paid subscriptions on the “Find Permits” community or neighborhood pages. First time looker (#9) thought the paywall in the description stifles here curiosity. She hasn’t used it enough to make a decision – there is no information on a trial readily available, no free content on the outset, which turns her off because she doesn’t even know if it’s valuable to her yet. Chicago Explorer (#10) added that “Subscribe– doesn’t appeal to me, but would if I were a mogul.”
Chicago Cityscape has a great opportunity to teach residents about building/property information and permits and then provide a platform for residents to learn more about what is happening in their own neighborhoods. The best way that can happen is to continue to include definitions and descriptions of terms.
It is also important to consider audience for this website and provide the proper pathways for individuals to get the information that they need. Whether they are an involved community member, home buyer, or a contractor. Defining the purpose of this site and how this data is useful to multiple groups of people in the forefront (on the homepage) will give individuals the grounding to use the data provided.
Final Report
Here is a final report of results with the analysis of the questions we asked, followed by each tester’s responses, and copies of other questions we asked:
The raw test data can be found below with the answers from every tester:
