For our eighth Civic User Testing Group session, we tested the Waitbot app, which is a place where you can find wait time estimates for all sorts of things, including transit, restaurants, airports, and more. This test had an in-person and a remote component to it. The in-person test took place at one of the Connect Chicago locations – Chicago Public Library Clearing Branch at 6423 W. 63rd Place in the Clearing neighborhood.
Through this test, we were interested in finding answers to these questions:
- What makes users download an app? Delete an app?
- Do users want to use Waitbot on a daily basis? Why or why not?
- What features do users want?
- What other wait categories would users want to see?
- Do users want to share wait time information on social media?
Segmenting
On March 5, we sent out an e-mail to all of our 749 CUTGroup participants. We asked them if they would be willing to test a wait time estimates app on March 12, 2014. We asked some screening questions to gather information, and chose our group of participants based on a diverse selection of answers and also device types.
We were interested in having about 15 participants from different Chicago neighborhoods, but only had 6 testers come to test in-person. A lot of testers could not come due to a combination of weather and distance, so we reached out to 4 more testers to do the test remotely.
In the end, we had 10 testers participate in the test, although 1 tester could not get the app to load and could not fully participate in the test.
Here is a look at which neighborhoods our testers came from:
View CUTGroup 8: Waitbot Participants in a full screen map
Test Format
For the in-person test, proctors were able to work with testers one-on-one. Testers looked at the app on their own devices and provided feedback, while the proctors wrote down notes. After the test, we sent out additional (and optional) questions to see if testers were using the app and how they liked the app in their own neighborhood.
For the remote test we asked testers to use the app on their own and provided questions to lead them through the test.
In the end, we got great responses from both types of tests.
Results
Before this test, testers generally associated apps that estimate wait times with transit apps. 9 out of 10 testers used Web sites or apps to check wait times. Some apps that testers use for wait times include CTA Bus Tracker, Transit Tracks, and Transit Stop.
Even during the test when testers were offered wait times for other establishments, most testers (7 out of 10) thought the Transit page was still the most useful, and the other three pages (Restaurants, Airports, Emergency Medical) were equally considered the least useful page.
In addition, after looking at the Transit page, testers were not drawn to do one thing over another, and there was not a logical course of action of what to do next.
What makes users download an app? Delete an app?
5 out of 10 testers mentioned that they would download or keep any app if they routinely used it and if it was easy to use. Other considerations included whether it was a low-cost or free, and if it had good recommendations or reviews from others.
Do users want to use Waitbot on a daily basis? Why or why not?
67% of testers (6) liked the Waitbot app and said they would use the app again after the test. Testers who did not like the app did not have many options populate for them and did not think they would use all of the features.
78% of testers (7) found the Transit page the most useful. In terms of using Waitbot on a daily basis, transit is a key way for users to use Waitbot on a daily basis.
What features do users want? How can Waitbot be improved?
- Only 1 tester liked the Map view option over the List view. The majority of testers liked the list view because it gave more information and was better organized than the map view. It was in general easier to use
- There was confusion regarding the color coding and what the colors meant in terms of wait time. 1 tester thought that the color referred to the CTA line
- Testers are interested in seeing more details and information for each page. In addition, besides the Transit page, testers did not have many options and a lot of places had “No report.” Testers were not immediately aware what this referred to
“Add Wait” was not particularly a well understood feature of the Waitbot app.
- Two testers chose it after looking at the “Transit” option because they were curious of what it did
- 4 out of 9 testers did not clearly understand what this feature would be used for. One tester saw this option as only for the establishments to add, not necessarily residents. Another tester clicked on “Add wait time” while on the Emergency Medical page and it gave him a wait time screen
- 3 out of 9 testers said it seems easy to use, and was straight-forward
A tester, I like Cacti (#4), said he would add wait times:
“If there’s enough data and it’s actually helping people. When I am at a restaurant and the line is really long, it’s a way to complain about it.”
What other wait categories would users want to see?
Of the categories “Coming Soon,” the top categories testers were interested in included:
- Government Services
- Tourist Attractions
- Barber Shop
- Nightlife
Other categories, testers wanted to see included:
- DMV
- Grocery Store lines
- Movies
- Gas Stations
Do users want to share on social media?
Most testers were not interested in sharing this type of information on social media. One tester would share on Facebook only if it was automatically connected, while another tester said he would not do it unless there was an incentive. Only 3 out of 10 testers would share on social media.
Conclusion
When testing the Waitbot app, testers liked the Transit page and the fact that it populated with nearby options. There was some confusion with color-coding and testers wanted added features such as route display. However, testers overwhelmingly liked this page.
One tester, My eyes are dried out (#10), explains why he doesn’t like the Waitbot app in general, but thought that the Transit page was the most useful:
“The Swiss army knife is useful and practical. Then the impostors ‘improved on it,’ making it bigger and more cluttered with useless features. Sometimes I feel app creators want to entice a large crown, instead of just perfecting one good thing.”
The Transit page is the one that testers would use consistently, while some testers might check Restaurant wait times. Airports and Emergency Medical categories are both categories that people would not base their decisions on before using. Lastly, testers said that the restaurant category did not have enough information about the establishments, or no wait times were represented. The majority of testers liked the app (67%), and thought it was an interesting concept, but without more wait time information or categories that users would check to base future decisions on, it is difficult to state if users would use Waitbot on a regular basis.
Final Report
Here is a final report of the results with key highlights from our CUTGroup test, followed by each tester’s responses, and copies of our e-mail campaigns and the questions we asked:
The raw test data can be found below with complete answers from every tester.
