Models for Leveraging Tech to Improve Public Service & Governance in Indonesia

Smart Chicago Collaborative guest blogger Jensi Sartin is a YSEALI State Department Fellow from Indonesia. He is working with the Smart Chicago Collaborative for a month to gain experience in Chicago’s public, philanthropic, and nonprofit sectors — especially on digital issues.

Rising incomes in populous Southeast Asian countries plus plunging smartphone prices have created an Internet boom. It is estimated that there will be 190 million smartphone users in Southeast Asia by end of 2015.  By the end of 2019, that number will be over 340 million! In my country of Indonesia we have the most mobile Facebook users in the world and the capital capital city, Jakarta, has more active Twitter users than any other city in the world.

This digital advancement affects the public, including citizens’ relationship with and communication with government. In Indonesia, you can tweet directly to the governor and, even if it’s an angry tweet, you’re sure to get reply or a retweet. The public also uses online petitions like Change.org to  spread awareness or even force government to shift positions on policies. We’ve even found that using  apps and crowdsourcing can help crackdown corruption in the government.

Organizing rally urging government to eradicate corruption and promote more meaningful transparency in the mining sector.

Organizing rally urging government to eradicate corruption and promote more meaningful transparency in the mining sector.

Despite the fact that Indonesia’s Internet quality is poorer compared to its Southeast Asian neighbor nations, most government agencies have at least two digital channels: a website and a social media account. Some agencies have even created digital platforms for e-Procurement, e-Budgeting, e-Tax, and many more “e-somethings”. For instance, the city of Jakarta, famous for its bad traffic jams, is using social media to make the city more livable.

Environmental damage by a mining company found by combining images from drone mapping with a government map on existing mining concession (the white color area is a lake that has been used to wash bauxite from soil)

Environmental damage by a mining company found by combining images from drone mapping with a government map on existing mining concession (the white color area is a lake that has been used to wash bauxite from soil)

Civic advocacy is also adopting technology in its work. Following the progress of Open Government Partnership and Indonesia’s Freedom of Information Law, a number of projects were launched to promote open data. Civic apps and websites were created to solve problems that are either rarely or inefficiently addressed by government. For instance, KawalPilkada is an app that educates the public  on provincial and city-level elections. Also, a local anti-corruption organization launched opentender.net to crosscheck the procurement process in the government projects.

Community brainstorming with fishermen, the marine tourism operators, and government to develop a joint-program to manage fisheries and marine tourism in the area. This region in east of Bali is one of main tourism destinations that is threatened by the increasing impact of climate change.

Community brainstorming with fishermen, the marine tourism operators, and government to develop a joint-program to manage fisheries and marine tourism in the area. This region in east of Bali is one of main tourism destinations that is threatened by the increasing impact of climate change.

Have all these “e-somethings” improved public service delivery in Indonesia or have they simply  spread the “you are not cool if you don’t have Twitter” feeling? Unfortunately, this question is rarely addressed. The most important thing in this work should be user (or expected user) opinion and experience. It’s about the people that use the technology!  A people-focused  organization, set of methods, and best practices should be established to ensure that new digital tools truly improve public life.

I think this might look like a Smart Chicago Collaborative model in Indonesia with missions in Internet access (like Connect Chicago), in digital skills and education (like Smart Health Centers), and in user-focused tools (like the CUTGroup).  This is why I’m here in Chicago — to learn more about the Smart Chicago model and see how I can apply it to districts in my country!

To learn more about me and follow my work, follow me on Twitter.

Jensi pic 1

Jensi Sartin is a YSEALI Professional Fellow for Legislative and Governance Process of  Department of State and he is hosted by American Council of Young Political Leaders.  YSEALI is President Barack Obama’s signature program to strengthen leadership development and networking in Southeast Asia. Based in Indonesia, Jensi serves as a program development manager of  Publish What You Pay Indonesia’s efforts to promote transparency and accountability in government especially related to governance of extractive sector (i.e. oil, gas, coal, mineral, forestry industry, etc.).  He also provides Reef Check Indonesia with  research and advisory support. Jensi holds a Master’s degree in Natural Resources Management from the James Cook University Australia and a Bachelor’s Degree in Marine Sciences from Diponegoro University.

Madonna Scholars + Smart Chicago

madonna-foundation-logoToday Smart Chicago and The Chicago Community Trust hosted a number of Madonna Scholars of the Madonna Foundation. The Madonna Foundation was established in 2001 by the Franciscan Sisters of Chicago, and is a public charity that increases access for young urban women to attend Catholic high schools in the Chicago area.

In addition to financial aid, the Foundation is dedicated to support the academic, psychological, spiritual and social needs of young women. These needs are addressed through a series of unique and innovative programs and service learning opportunities that provides young urban women the opportunity to build a meaningful, productive and successful life for themselves and leaders for future generations.

Today we talked about Smart Chicago, our founding partners, the CUTGroup, Connect Chicago, and Youth-Led Tech. Most of all, we’ll be listening to them and hearing how they use technology to make their lives better. 

This is just one of the ways we seek to strengthen ties between their neighborhoods and the robust public technology scene here in Chicago.

Here’s a set of pics from our day together and a group pic, below:

Madonna Scholars at Smart Chicago

 

New Cook County Data: Fiscal Year 2016 Executive Budget Recommendation

On Wednesday, the Cook County Department of Budget and Management Services released the Executive Recommendations for the Fiscal Year 2016. As with previous years the Budget is available as a series of PDFs.

As part of our work with Cook County Open Data we helped create and post three datasets as part of this year’s Executive Recommendation Budget release:

  1. Budget Summary By Object Classification: This dataset lists by department the Department Request and Executive Recommendation dollar amounts for each line item. (We helped publish this data last year too.)
  2. Budget Summary Of Positions By Business Unit: This dataset lists by department and business unit the proposed position counts and salaries.
  3. Capital Equipment: This dataset contains a list of capital equipment projects along with the requested dollar amount.

With these dataset releases we made several improvements over last year’s Executive Recommended Budget release:

  • Timeliness: This year we wanted to try to release open data on the same day as the PDFs were released. Over the past several weeks we have been working with staff in the Department of Budget and Management Services to make that possible for these datasets.
  • New Types of Datasets: Last year only the Budget Summary By Object Classification dataset was released. We wanted to identify additional data that is part of the budget process. In preparation for this year’s budget season we went through last year’s Budget Book PDFs to identify potential new datasets. From there, we worked with staff in the Department of Budget and Management Services to identify who is responsible for each type of data and to develop the final open dataset.
  • Data Lens: We wanted to go beyond just providing the data tables so we took this as an opportunity to try out Socrata’s Data Lens view that was launched earlier this year. The Data Lens views have interactive graphs and search functionality. They act as an alternative view to the spreadsheet style of the main dataset. We have Data Lens views for all 3 datasets:
    • Budget Summary by Object Classification
    • Budget Summary Of Positions By Business Unit
    • Capital Equipment Projects
  • Open Data in Context: While the Open Data Portal makes it easy to find these budget datasets if you are on the portal, we wanted to ensure that users going to the county’s website to learn about the proposed budget had easy access to the open data and Data Lens visualizations. To accomplish that, for the first time this year, links to the Data Lens views are included with the PDF links on the budget page.

Over the next several weeks there will be Departmental Reviews and Public Hearings on the Executive Recommendation Budget. See the Cook County Budget Calendar for the complete schedule including links to live streams and public hearing speaker registration.

Cook County Executive Recommendation Budget FY2016

Healthy Chicago 2.0: Health Action Plan Marshals Community, Data to Target Root Causes

Chicago’s public health goals are shifting toward battling crime, tenement housing and other stubborn social concerns. Nearly a year of data-driven community discussions have led the city’s health professionals to look beyond their traditional roles treating infections, substance abuse and other conditions.

“There are chunks of population in Chicago that are just suffering tremendously, and we just aren’t targeting our resources in the right way,” says Jaime Dircksen, deputy commissioner of the Chicago Department of Public Health. “We started this process with equity in mind, and with the goal of achieving equity across the city. I think having that lens really led to people feeling comfortable talking about some of the causes of these problems.”

Attacking these problems meant coming up with an approach other city departments would support in their own programs. Now the Healthy Chicago 2.0 plan is being circulated in City Hall for unveiling in late fall. Its priorities emerged in a community-driven process, developed for public health agencies with the federal Centers for Disease Control and Prevention.

“This plan is not the health department’s plan, it’s the city’s plan. Everyone plays a role in improving the health of the city,” Dircksen says. “We will be meeting with city department heads to make sure they understand what’s being put forth in the plan and will champion the plan. Then we’ll convene the interagency council of city agencies and share with them the draft of everything,. We’re identifying the opportunities where we can create synergy. “

Data to the people

Some 800 people contributed to the goal-setting process, a quarter of them in 10 working groups that set objectives and strategies. A data-intensive approach kept this potentially unwieldy goal-setting effort on track.

Percent of live births in which mother began prenatal care during 1st trimester, 1999 - 2009 (Chicago Health Atlas)

Percent of live births in which mother began prenatal care during 1st trimester, 1999 – 2009 (Chicago Health Atlas)

Public-health staffers gave work-group volunteers a thick stack of statistics on births and deaths, hospitalizations and personal habits. They mapped health outcomes by neighborhood, conducted survey research and adopted novel ways to probe the underlying causes of chronic diseases.

Finally, they faced down the realities of an austere 2016 city budget. The health department controls only $149 million directly, a 4 percent cut. Most of that is set aside for AIDS, women’s and children’s health, mental-health and emergency services.

“There’s a strong paradox constantly at work,” says Nikhil Prachand, the health department’s director of epidemiology and public health informatics. “It’s impossible to narrow down the priorities, but if you don’t have a lot of money it should be easy to narrow down the priorities. “

Smart Chicago will play a role in measuring the plan’s success. The city will use the Chicago Health Atlas website to mark progress toward goals for 2020.

“We will have a dashboard of indicators monitoring every action area,” Dircksen says. A website update will “really dive deep into community area data so that the community can see progress,” she says. “Community-based organizations can use it as a resource for funding opportunities and monitoring their own work.”

Planning began last year with surveys in English and Spanish, asking broadly about a healthy environment. Residents across the economic spectrum united around safety and access to healthy food as citywide needs. Yet there wasn’t much agreement on neighborhood needs.

In areas under economic stress, crime emerged as the top priority. In affluent areas – nearly half the sample – respondents were more concerned about the built environment as a local issue. The widest gulf was in access to education, based on agreement with statements like, ”Schools in my neighborhood have what they need to provide a high quality education.”

Tackling broader issues like safety, Dircksen argues, takes “understanding that people aren’t going to parks because they don’t feel safe, they’re not well lit, there’s trash all over the place, that’s where the gang violence happens — then thinking out how to respond to those issues.”

Percent of occupied crowded housing units, 2007-2011 (Chicago Health Atlas)

Percent of occupied crowded housing units, 2007-2011 (Chicago Health Atlas)

Root causes

Five panels probed more deeply into the equity questions. “They did a focus group with our hotline volunteers to hear the stories they’ve heard,” says John Bartlett, executive director of the Metropolitan Tenants Organization. “And they asked about their lives also, because many of them are tenants.“

University of Illinois at Chicago students scored the responses, along with content from a half-dozen StoryCorps oral histories. They found common themes – problems navigating mainstream society and a sense of powerlessness. Again, health issues were linked to larger social problems.

“For example, mold will trigger asthma,” Bartlett says. “We are continually counseling parents whose children have uncontrolled asthma, informing them of steps they have to take to get their landlords to make the environment safer for their kids. Oftentimes landlords can be recalcitrant about that.

“We will inspect units for things like paint dust, and if there is, work with the health department to get a city inspection and encourage families to get their kids tested,” he adds. “And bedbugs are definitely a stressor in people’s lives. They blame themselves, but it’s not anyone’s fault. These creatures are just hitchhiking all over the place.”

The next step was to share the results with local health advocates. Many were frustrated at the lack of money, equity, attention and political will to take on core issues. And they saw traps ahead for clients navigating Affordable Care Act enrollment and mental health clinic closures.

“We were happy because the city was making efforts to be accessible and to be inclusive of the disabilities community,” says Gary Arnold, spokesman for Access Living, which hosted one of the advocate forums.

Local Pubic Health System Assessment (Chicago Department of Public Health)

Local Pubic Health System Assessment (Chicago Department of Public Health)

Opportunities and threats

In one exercise, service providers scored the local health system using a CDC-approved framework. Working groups saw electronic health records posing opportunities for data sharing and monitoring, and threats from uneven adoption and stale information.

Health advocates saw new communication tools as potential threats, raising access barriers or triggering changes in brain development and socialization. But technology also was part of the solution: Ideas included wrist monitors, health provider networks and a 2-1-1 phone line to take health and human service calls.

“We ended up with 50 priorities, and they’re all very important,” Dirksen says. Grouping them yielded a more workable list of 16 themes, which were ranked by public and private stakeholders in the Partnership for Healthy Chicago. The city convened 10 expert panels this summer to draft objectives and strategies in key areas.

HEALTHY CHICAGO 2.0 ACTION AREAS

  1. Access to healthcare and human services
  2. Behavioral Health
  3. Chronic disease prevention and control
  4. Community development
  5. Data & Research
  6. Education equity
  7. Infectious Disease
  8. Maternal, Infant, Child and Adolescent Health
  9. Partnerships and Community Engagement
  10. Violence and Injury Prevention

“The first thing is laying out the roadmap then creating the will to fund it,” says Bartlett, who joined the community development team. “If we’re serious about having a healthier Chicago we need to look at prevention. All the departments dealing with housing should be on the same page looking at health as part of the decision-making process. How do we make sure Chicago housing is affordable and healthy? It’s not good to have only one without the other. “

Distributed network

The teams will reconvene next month to draft detailed plans. Eight final themes will mirror the action areas, with data and engagement as strategies throughout. “We can’t do any of this work without having the data to inform it, the research to gather additional data – and it’s an all-hands-on-deck effort,” Dircksen says.

Data will help make the case for funds, and track whether they’ve been spent wisely. “Community development is focused primarily on capital improvements – improving CTA stations, rehabbing schools, building structures,” Prachand says. “We have been able to assess the health of the city’s commercial areas and offered a number of metrics. We can monitor over time and give feedback whether these capital improvement projects and grand plans are having some impact on people. “

The plan calls for more community input, in projects such as locating new Divvy bike stations on the South Side. “How do you know where the next best place is? Not necessarily by looking at a map,” Dircksen says. “They have to talk to the community leaders and stakeholders. We’re talking about the public health planning and transportation planning worlds coming together, and working together to identify mutual benefit and priorities.”

The city will count on private agencies to take on some of the burden. “Funders are wholly committed to obesity, metal health, access to care, violence prevention,” Dircksen says. “They appreciate and understand housing is health care. But then they’re giving across the city, not making a great impact, and not necessarily using evidence-based strategies. How do we work with them to make sure they understand what the evidence is and what does work, and concentrate their efforts in places or with populations which we know need the most?”

“It’s our job to mobilize and motivate the community to be a part of this,” she adds. “By 2020 we expect to achieve all the things we’ve laid out. I think with this process we will have a lot of engagement come launch because people will have been involved throughout the process. There’s a lot of evidence that when you engage people from the very beginning, they’re more likely to buy in, they’re more likely to act.

“If we don’t address environment and community conditions and access to care, we’ll never be able to impact the lives of people,” she adds. “At the forefront we will focus on those root causes of why folks are overweight, why they’re smoking, why they aren’t caring for their chronic conditions or their mental illness, or why pregnant moms can’t get prenatal care or can’t deliver a healthy baby.”

Honorary Chicago + Documenters Program

Public Way!Tomorrow is the first of three sessions we’re running with Linda Zabors of Honorary Chicago to help improve the data on her website, which collects and displays “the who, where, and why of Chicago’s brown honorary street signs…. and other commemorative honors”.

We’re helping put together a crew of five people from our Documenters program who will review bound books of the Journal of the Proceedings of City Council at the Chicago Public Library. Linda will guide them through the sections to look for and the documenters will take pics and enter info into a spreadsheet so that it can be ported onto the website.

All of this is a part of digitizing the official records of the streets from City Council data not yet available in digital format to help complete Linda’s research of 30 years of honorary sign designations (more than 2,000 signs!).  Most of 1985-present has been collected, there are some gaps, especially in the 2003-2009 timeframe.

We’re going to help her with that. This is a perfect project for our Documenters program, for a number of reasons:

  • Focus is on city data and helping people understand a central record of our municipal government
  • Gets Chicago residents paid while learning and being civically engaged
  • Helps out a great actor in civic tech who has already done an enormous amount of work
  • We get to hang out at one of our central shared spaces— the public library!

We are looking to expand our corps of available documenters, so if you are someone you know is interested in joining us, complete this form. If you have further questions, contact Director of Operations Kyla Williams at .

Civic User Testing Group (CUTGroup): Remarks at Code for America 2015 Summit

Today, I will talk about the Civic User Testing Group (CUTGroup) during the 2015 Code for America Summit. In 2013, Dan O’Neil presented at the CfA Summit about the CUTGroup as a model for changing the relationship between government and residents. Since that summit two years ago, we have doubled the number of CUTGroup testers from 511 to over 1,000 testers, we have tested sixteen websites and apps, we have expanded to all of Cook County, and we continue to add processes to engage with people in the CUTGroup. The work is never done.

I now run the CUTGroup project for Smart Chicago. Here are my thoughts about how not only to run a CUTGroup, (we lay this out in detail in the CUTGroup book and blog posts) but how to sustain a CUTGroup by leading with community engagement.

CfASlideDeck-CUTGroup-Title-Slide1

The Civic User Testing Group (CUTGroup) is a new model for UX testing, digital skills, and community engagement. What makes the CUTGroup the CUTgroup is the merging of these three components into one experience. To build a CUTGroup, you need to devote time to all these components equally.

The CUTGroup is a central program for Smart Chicago because it cuts across our three areas of focus: access, skills and data. Access: we conduct the majority of our tests in public computer centers and libraries in the community. Skills: for the tester who is introduced to new technology, and for the developer who learns ways to design tests and engage residents. Data: we help improve existing technology and encourage the creation of better technology.

CfAdeck-CUTGroup-slide3

The CUTGroup is the community of people in Chicago and all of Cook County who come together in libraries and public computer centers to test and have conversations around technology. In Chicago, we have testers from every part of the city, all 50 wards and all 77 community areas. We are now reaching the rest of Cook County.

CUTGroup-slide-deckengagementslide2

These residents are paid to test websites and apps to help create better technology. We pay every CUTGroup member $5 for signing up, and then $20 when they participate in a test. As of today, we have done 19 tests that cover a wide range of topics such as schools, transportation, social services, neighborhood information, and more.

CfAdeck-CUTGroup-Slide5

The motto below is key to this work. It removes the idea that residents do not understand technology as well as technologists. Instead, it permits testers to participate, give their feedback and show that their ideas are a valuable part of the process.

CfADeck-CUTGroup-Slide6

We talk about the methods and processes in the CUTGroup book to help other cities run their own CUTGroup.

CfADeck-CUTGroup-Slide11

Asking how to start or run a CUTGroup is important, but I think the question below is more important.

CfAdeck-CUTGroup-Slide12

The answer is easier than we think.

CfADeck-CUTGroup-Slide13

So, when is it not a CUTGroup?

CfADeck-CUTGroup-Slide14

It’s not a CUTGroup when you are only testing civic apps. In other words, it’s not a CUTGroup if you put the tech first. The “civic” in Civic User Testing Group has to describe the group before it describes the user testing. At Smart Chicago, we are beginning to move away from the language of “testing civic apps” because there are more important criteria in determining what to test. We determine whether or not to test technology based on these criteria:

  • Interest and desire to do CUTGroup testing and talk with residents (commitment to be part of the process!)
  • The technology reaches a large and diverse group of residents and could have or has an impact on their lives
  • Willingness to listen and then respond to the feedback and make changes

Not everyone who seeks CUTGroup testing might see that the tech they created is a “civic app,” and that is ok.

Cfadeck-cutgroup-slide15

A CUTGroup needs to lead with engaging with people around technology and that becomes easier when you know that people want to help make tech better. They want their voices heard and they know so much about how they use technology and how they want technology to work for them.

CUTGroupSlide16

Slide18

Slide19

Invite

The first step to engaging through UX testing is inviting and recruiting people to join the CUTGroup and recruit from inside your networks as well as outside of it. It is crucial to open the CUTGroup to everyone and, then once they are in the group, to build one-on-one relationships with new testers. That way, no matter how they were recruited – by a friend, from a flyer in a library, from an organization – they are on the same level with all of the testers in the group. It does not matter how they get to you, but how they are included in the CUTGroup experience.

Slide21

I cannot emphasize the statements below more. Recruit anywhere and everywhere. If we only recruited from one place or one group, or stayed with the same method of recruitment, we would miss so many people who want to join and participate. The tech we test cannot define who is in our CUTGroup.

Slide22

Slide23

Here are a couple of very specific ways that we try to be inclusive in CUTGroup.

First, we use physical gift cards because it’s the closest thing to cash that we can get. We want to give testers something that they can use anywhere they normally would shop. Even though physical gift cards cost more than digital gift cards we use them because they are easier to use and can be used anywhere.  We also cannot assume that people shop online in their normal day-to-day.

Slide25

Recently, we incorporated text messaging into the CUTGroup process to reach people who do not have regular access to the internet. Out of our 1,000+ CUTGroup members today, 29% of our testers said their primary form of connecting to the Internet is either via public wifi or their phone with data plan. Testers will be able to sign up for CUTGroup and receive text notifications when new tests come up and being to respond to participate.

Slide26

Slide27

Ask

The next step centers on how we communicate with our testers and how we keep in contact with them. First, we never share information with our CUTGroup testers about anything other than CUTGroup. This maintains a relationship that only centers on the program that they signed up to be a part of.

We keep regular and open communication about upcoming tests, and explain the reasons behind why or why not we picked them for a specific test. If they are not selected for a test, we still want to share what the test was about so that they feel included in what is happening and can still interact with the website on their own.

Slide29

When testers sign up for CUTGroup, we  do not gather information about demographics, and prefer to ask questions that relate to their devices and the ways they connect to the Internet.  This keeps our sign-up form simple and easy to fill out. Also, we are more interested in learning about the tester’s non-technical and technical experiences, and we value learning about these experiences more than gathering demographic information. We do not have a database of testers’ age, race, income levels, or education. These things are just not as important.

Slide31

We do design the screening questions to gather better information about testers for that specific test. If necessary, this is when we capture information about demographics.

Slide32

This allows testers to voluntarily give us more information. If they do not feel comfortable with answering the questions for a specific test, they do not have to, and they can still be part of the CUTGroup. Letting people choose how they interact with us is important and we want to feel comfortable in these interactions.

Slide33

Listen

When it comes to the CUTGroup test, it is hard to listen and leave expectations aside. When you bring in developers and project managers, they have their own assumptions of how the test will go (and so will you).  However, the unexpected solutions that testers come up with can sometimes be the most valuable part of UX testing. It is important to step back, listen, and not tell testers why they are right or wrong  or what they should have done. This is hard because we want to help if they are getting stuck. What is better is to really focus on understanding how they are doing the task you asked of them.

Slide35

We generally test in branches of the Chicago Public Library because it allows us to visit new neighborhoods and reach new people. We are very lucky to have such a great resource in the Chicago Public Library since there is a library in every neighborhood. It would be really easy for us to test in our own offices, but meeting people in their community allows new people to participate in testing. We love going to them.

Slide36

It is important to talk about the person’s experiences not just about how they use technology. We want to ask how testers get information, how they do things in their normal day-to-day, and how they see technology fitting into those experiences.

Slide38

Once testing begins, we keep asking questions to learn more about what testers are doing and their expectation. We ask them why they clicked on that button/link or why they are using that feature. Their responses help us gather insight not only about the tech being tested but also tech in general. Their UX experience is influenced by other tech they know and learning about those expectations helps understand how to build tech better.

Slide40

We also always ask testers for very specific ways to improve the website or app. “Tell us anything and everything you can” is a phrase I use often because I never want testers to feel their improvements are too big or too small. I want to hear all of their ideas. We consistently ask this question across tests, and it gives testers to contribute to the creation.

Slide41

Another big part of our work is teaching developers how to incorporate user testing in the development of their technology. By requiring that the developer be part of the process, we put them in front of testers to see and hear for themselves how real people would use their tech.   

Slide37

Respond

The last piece of engaging with the CUTGroup is making the changes that the testers suggested.

Slide43

Once a test is completed, I spend a lot of time working on the analysis. I take a look at each question and provide quantitative and qualitative data. We want to see how easy it was for testers to finish a task as much as we want to understand their expectations of the technology and its functions. We also want to learn about whether or not they think this website is for them, if they like it, and if they would use it again.

Slide44

The last step is the hardest, and not always completely in our control, but my goal is always to find better ways to help developers make changes on their website or app. This is a process of collaboration and there needs to be a commitment made at the beginning of the process that the developers, project managers, or organization staff will make some changes based on the conversations they had with CUTGroup testers.

After that happens, I want to show testers what they helped to create and show them that we listened.

Slide45

It all comes back to making change happen because if we are inviting, asking, listening, but not responding — the test is hollow. We are not respecting our testers and their suggestions and it comes full-circle to our motto. If it doesn’t work for them… it doesn’t work.

Slide46

To follow my presentation at the summit, check out some of these hashtags #CUTGroup and #CfASummit. You can also follow me at @ssmarziano.

Here is my entire presentation: