For our fifteenth Civic User Testing Group session, we tested the Chicago Public Schools (CPS) website. This in-person test took place at one of the Connect Chicago locations – Chicago Public Library Roosevelt branch at 1101 W. Taylor Street.
Here is what the CPS website team said about why they wanted to do a CUTGroup test:
“We recently redesigned our website to be parent focused and mobile device friendly. We improved our search results, tagged content by topic and age level and created a district calendar. We are curious to know if the new design is meeting the needs of our parents.”
Ted Canji, Jay Van Patten, and Matt Riel, who redesigned the CPS website were interested in testing these things:
- Find School Information: Can parents find a school easily? Do they use the School Locator? Do they use the search feature? Are parents comfortable using search?
- Website usability: We wanted to discover if there are areas of the website that are challenging to use.
- Mobile devices: We wanted to learn about how easy it is to visit the CPS website on a mobile device. We also wanted to know if parents have visited the CPS website before and learn more about how they get information about schools.
Segmenting
On March 25, we sent out an email to all of our 868 CUTGroup participants. We wanted to know if they would be available for an in-person test on April 1 for about 30-45 minutes. We asked some screening questions to gather information. When choosing testers the main priority was to get parents of CPS students who had children in different grades. In addition, we wanted testers who had different types of mobile devices and some who have lived in Chicago for a long time, as well as parents who were newer to Chicago or their neighborhood.
We ended up having 10 testers participate in this test. 8 of these testers were from our CUTGroup community, but one tester brought in two friends who drove her to the test, and since they qualified as CPS parents, we allowed them to test. They also signed up to be part of the CUTGroup.
Screening Questions
We heard from 35 CUTGroup members during our callout for testers. Here is what we learned from our screening questions:
- 74% of CUTGroup respondents have children who go to Chicago Public Schools
- 68% of CUTGroup respondents have lived in Chicago for over 20 years
Test Format
For this test, each tester was paired up with a proctor. They were asked to bring their mobile device, and the proctor asked questions about the CPS website and then captured those responses. We not only wanted testers to review the website and do tasks to see how easy the site was to use, but we also wanted to understand how testers normally got information about their child’s school. We were curious if they have ever used the CPS website before or went online to search for information about different schools.
Results
Finding School Information
Past experiences
When we asked testers whether or not they ever searched online for information about the schools that their children attend, overwhelmingly (7 out of 10) testers said “No.”
Parent of Three Girls (#9) says, “I have looked before when she was first starting for information about the school. I’ve heard a lot of stuff about the school, but I like her teachers, and I go off of that.”
When asked whether or not they have ever visited the CPS website, 7 out of 10 testers said “Yes.” This is how these testers used the CPS website in the past:
- 3 testers used it to choose a new school or High School
- 2 testers used it to check the calendar of events
- 1 tester wanted to learn more about Local School Councils, and
- 1 tester wanted to find out about after-school programs
We asked testers how easy it is to find information about schools, whether online or in person. Half of the testers answered right in the middle about how difficult or easy it was to find school information:
5 Very Easy 20% (2)
4 Easy 10% (1)
3 Neutral 50% (5)
2 Difficult 20% (2)
1 Very Difficult 0%
We learned that a lot of our testers find out what is happening in their children’s schools from communication with the teachers either via phone, parent/teacher conferences, notes, email. Testers also learn about about what is happening in their school from other parents, weekly email newsletters and communications with teachers.
During the test
When searching for their child’s school, 6 out of 10 testers used the School Locator, but all testers thought it was an easy time searching. Testers were then asked to tell us the most important and least important information on the school’s profile pages. The most important information testers called out included: address/contact information, progress reports, number of students, test scores, admission and graduation rates. When asked about the least important information, 4 testers said that all of the information was important to them.
When all testers were asked to use the School Locator tool and search for an address, we saw that about half of the testers had some degree of difficulty using this feature. 4 testers had trouble with viewing the map and filters when holding their phones vertically; 2 of these testers were testing with their iPhones, while the other 2 were on Android devices. When holding their phone vertically, testers only saw the list of school names, but when clicking on the name it would open more information on the map which was not on view for the tester. The tester would have to hold their phone horizontally in order to see both the map and the list.
Usability of CPS.edu
We were very interested in learning more about how parents searched when finding information from their school or about other topics. The CPS website team wanted to see if parents were comfortable using the search features. Therefore, we asked testers to search for information that they were interested in learning more about. Half of the testers clicked on the appropriate topics to find out more information, while the other half of the testers used the search box. Only three testers had trouble finding the information they were looking for.
When testers were asked to search for their child’s school (in any way that they wanted), 6 out of 10 of the testers used the School Locator. All testers thought it was an easy experience to find their child’s school.
Active in the schools (#1) said, “It didn’t take but a hot second, I typed in Lane and it popped right up.”
A few testers experienced slow load times for the next pages. For example, Meekmeek (#10) downloaded a PDF of the calendar because the calendar did not show up on her screen, but the PDF was difficult to read since she had to zoom in and out. When clicking on “Calendar” it showed a white screen, and took a very long time to load.
A few testers reached PDFs during their tests and either they were not as easy to use such as in Meekmeek’s(#10) case, or they did not load. Parent of Three Girls (#9) chose a PARCC article in the headlines. When clicking on it, she was asked to download a PDF. After downloading, she clicks to open it but gets an error message saying that she cannot open the file.
As mentioned above, during this test we learned the biggest improvement that could be made was the School Locator and changing the view from a horizontal layout to a vertical layout. In general, all testers said they liked the CPS website and most of them had an easy time using the website.
“I just think the website is better than it used to be!” – High School Parent (#5)
“It is very easy to use. I like that they give you the address, phone, type, classification, programs offered… I like that they tell you a lot.” –Parent of k (#8)
Mobile devices
Normally for CUTGroup tests, we encourage testers to bring their own phones and devices for testing. It is important that we not only test mobile-friendly websites, but that we test them on their own devices. We not only learn about usability issues on different types of phones, but we also better understand how testers navigate tasks on their own device. 5 testers tested on laptops that we provided, and here is a look at the devices the other 5 testers used:
- iPhone 5
- iPhone 6
- Nexus 5
- Boost ZTE Max
- Android HTC Desire 10 Boost Mobile
Besides some of the issues we outlined above, most testers using a mobile device had an easy time navigating the new CPS website. We were excited to test a website that not only reaches so many parents in the city of Chicago, but are also excited as the CPS website team continues to redesign with mobile devices in mind. This CUTGroup opportunity was a great example of getting feedback directly from residents (and parents), on their own devices, while better understanding their own experiences when finding school information.
Changes to CPS.edu
Since this CUTGroup test, the CPS website design team has made enormous changes to the School Locator tool including instructional pop-overs to encourage parents to start their search, get more information, and provide details about the map legend and the tools available.
Other updates include users are now able to find schools quickly by using a new “Find Schools Around Me” button or by pressing and holding down on the map to drop a pin and see nearby neighborhood schools. The CPS website team also added Google street view functionality to have an interactive view of the school and its surroundings. Lastly, the “Compare Schools” feature allows users to compare different schools, so parents can find the best option for their child using information such as number of students, student performance, ratings, and more.
The biggest change we saw as a direct response from our CUTGroup testers having difficulties with the School Locator. The mobile layout of the School Locator page is now in a full-screen format, and users do not need to turn their mobile devices horizontally or move around the page to see all of the information they need. It is exciting when changes like this happen and the feedback directly from CUTGroup testers continues to be influential to websites and software that is so wide-reaching.
Final Report
Here is a final report of the results with the analysis of the questions we asked, followed by each tester’s responses, and copies of other questions we asked:
The raw test data can be found below with the written answers from every tester.
