select work
NYC mayor's office for economic opportunity

Establishing a consistent design process for the Civic Service Design studio of the Mayor's Office for Economic Opportunity

Client

The Civic Service Design Studio is a unit within the Mayor's Office for Economic Opportunity that uses principles in UX and Service Design to make public services more "effective, accessible and simple for all New Yorkers."

Challenge

The Studio deals with unnecessary hurdles and inefficiencies when conducting user-research interviews. I was honored to collaborate with the team to see how we could standardize an efficient process for doing user-research.

role

User research

  • usability testing
  • qualitative research
  • interaction design
impact

I delivered a participatory-designed framework for standardizing the process that the Studio can now use for researching, ideating, and designing solutions in the civic-space.

      With a revamped design process, The Studio began to take more actionable notes, incorporate diverse perspectives by working with stakeholders, and test well-informed prototypes.
      01.
      The Problem
      02.
      Research design
      03.
      Synthesizing data
      04.
      Designing a solution
      05.
      Evaluating impact
      06.
      Lessons learned
      The problem_

      Identifying the painpoints

      01/03. Roadblocks throughout the research process

      Duplications cause inefficiencies. The Studio's approach to user-research was filled with roadblocks that made collecting and managing user-data tedious and duplicative. For example, a common workflow after conducting a user-interview would be to tag key insights on sticky notes, record them on a cloud-based service (i.e. Google Docs), and then dispose of them in the bin afterwards. That's a lot of wasted post-its.

      02/03. Too many UX tools to choose from

      With so many moving parts throughout the research process, the Studio needed a tool to centralize all quotes, insights, and discoveries in one repository and share it with the rest of the team. However, there were too many software vendors to choose from and we needed to pick the right one that would solve for our specific needs.

      03/03. Not many actionable takeaways from weekly research-synthesis meetings

      Every week, the Studio would meet with key stakeholders on a project to discuss findings gathered during user-interviews. While it was important to have everyone at the table share quotes and insights, the impact of the meetings was limited by passive listening.

      The meetings were in need of a revamped framework that would enable our Studio team and stakeholders to do the following:

      • take actionable notes
      • ask informed questions
      • generate hypothesis
      • ideate solutions
      The Process: RESEARCH DESIGN_

      Planning the research

      Recruiting participants for qualitative interviews

      I reached out to four of the Studio's designers: two of whom worked full-time and two who were on a short-term fellowship. I liked that there was variation in how long they’ve been at the Studio. In my mind, the recent fellows would most likely have a different take on how things are done as opposed to the more established designers who may have grown accustomed to the same way of doing things. Fresh minds beget fresh perspectives.

      What did I want to understand?

      In order to understand which UX Tools would best facilitate the Studio’s research process, I had to understand how the Studio did research in the first place. All of my research efforts were focused on understanding three components: People, Processes, and Tools, and how they all affect each other.

      Research goals focused on people, processes, tools

      I wanted to center my research efforts on understanding my colleagues better: to know what their design process is like, which tools they use, and which possibilities they'd like to explore for solving their pain-points and accomplishing their goals.

      the process: synthesizing data_

      Making sense of the data

      After conducting the first round of user interviews, I noticed recurring pain-points that everyone in the team was having, as well as opportunities for solving them.

      Here's what the Studio team expressed as their shared pain-points:

      1. Lack of consistency

      “We need more structure on how to formalize our tools and processes. A lot is missing because a lot of people are involved on the research side. How we defined insights were varied.”

      -Studio_team_member-TR

      "Lack of consistency is what's most difficult about managing data."

      -Studio_team_member-TR
      2. Content overload and burnout

      "A lot of data, and really overwhelming"

      -Studio_team_member-LS

      "After the synthesis and capturing that synthesis, we were really exhausted because it took 4-6 hours a day. How do you take that and take a snapshot of it and have it synthesized for you?"

      -Studio_team_member-MK
      3. Many moving parts

      "One thing that was challenging for us- it was time-consuming to transcribe into note-taking form and then bringing it into post-its, that whole process was cumbersome"

      -Studio_team_member-LS

      "We had these 6 hour long days, and we were really tired and all this stuff would be up and hanging and we'd be like, 'oh we'll return to it tomorrow', but we'd have 800 other things so it would stay up and wait till the next week that'd we'd block and then we'd forget. We'd have to start over and we'd ask 'what is this again?'

      -Studio_team_member-LS

      Identifying opportunities to explore

      Throughout the interviews, I began to notice emerging themes around opportunities that would be worth exploring as potential solutions to our painpoints.

      Creating an ideal workflow

      "Having a designated space for your research- the space was such a restraint. This was the only space that allowed for everyone to use to do synthesis"

      -Studio_team_member-LS

      "Workflow wise, collaboration is super key- having another at least one or two people in the team working with you so that you can bounce ideas off, having that collaboration so that you're not doing everyting by yourself, even in the synthesis aspect."

      -Studio_team_member-MK

      "When possible, streamlining your notes and typing directly into a computer and then having time blocked off right after an interview to digitize."

      -Studio_team_member-EH

      "Have set synthesis days during a sprint, so you know that every week you have a 4 hour block for synthesis that you know you're coming to the session so you can prepare accordingly."

      -Studio_team_member-EH
      the process: Designing a solution_

      Bringing the team together to discuss our key findings

      After sorting the key-findings into painpoints and opportunities, I wanted to bring the Studio together for a workshop in which we could all offer input to solve for those pain-points and capitalize on opportunities.

      Validating our painpoints together along a matrix

      Next I presented the team's pain-points and issues. I wanted to see how often they came up in their work and how they felt about them. I asked the following two questions:

      • How painful are these issues?
      • How often do you encounter them?

      We came together as a team and placed our pain-points on a four-quadrant matrix along the lines of "intensity versus frequency". The vertical axis labelled the two extremes of frequency: “Rarely” and “Always”- and the horizontal one labelled the extremes of intensity: “not a pain” and “ouch, big pain.”

      "The most pressing and frequent issues were the following:"

      "Lack of consistent standards, processes, and defined outputs"

      "Onboarding new stakeholders and partners in the research process"

      "Physical exhaustion when sorting sticky-notes for hours when synthesizing data"

      "Rotating teams and positions within teams"

      What should we "stop, start, and continue" in our design processes?

      The team was then invited to deliberate on how to map out an ideal workflow that would ensure consistency and solve for the above pain-points by asking the following questions:

      • What do we want to stop doing?
      • What do we want to start doing?
      • What do we want to continue doing?
      "Here's how the team envisioned a standardized process going forward"
      1. Before we conduct a user-interview, we should...

      Have a team with diverse work-expertise so that various stakeholders can get involved

      Clarify and align on understanding between interview-lead and note-taker

      2. When we finish conducting an interview we should...

      Surface initial and most revelatory insights and quotes

      Generate fifteen immediate insights on sticky notes

      Take 5 minutes to debrief

      Interviewer should send a "thank you" note and follow-up to keep stakeholders invested

      3. When we sort through data we should...

      Have a transcript record of conversation

      Organize information and tag major themes/insights

      4. When we discuss our findings as a team we should...

      Create discussion-frameworks so that workshops and meetings are actionable, engaging, and not passive.

      Incorporate discussion points on patterns, insights, open questions, and hypotheses/ideas

      We ideated the following framework for taking actionable notes

      Each person in the meeting is given a ‘Notes Template’ to fill out as they are listening to the interviews being read aloud. The template will be divided into the following sections:

      Section 1: Patterns

      What emerging patterns are you noticing?

      Section 2: Insights

      Which quotes did you find novel/ interesting? why?

      Section 3: Open questions

      What do you still not know?

      Section 4: Ideating solutions

      What ideas are coming to your mind as potential solutions to explore?

      This note taking template evolved into This gave way to the "Field Logs" which the Studio now uses to journal each important milestone during the design-process for any given project.

      the process: Evaluating our solution_

      Putting our new design framework to the test

      The Studio was commissioned by the NYC Administration for Childrens' Services (ACS) to work on Pathways to Prevention, an initiative that would allow families to use their "voice and choice" to opt for social-services and thereby prevent their children from entering the foster-care system. We noticed the following improvements:

      01/04. Better, actionable notes called "Field Logs" during qualitative interviews and synthesis meetings

      The Studio created note templates for stakeholders to utilize which helped them:

      • ask insightful questions
      • identify emerging patterns and insights/trends
      • generate hypothesis and ideas for solutions

      The quality of follow-up questions was higher- helping the team explore solutions that addressed the needs of family stakeholders in a more meaningful way.

      02/04. The team was better informed when testing out prototypes

      By focusing on key emerging-trends gathered during interviews, the team was able to quickly prototype ideas that spoke to the needs of their stakeholders and receive validation from case-workers at ACS.

      03/04. Diverse groups of teams and stakeholders were incorporated in the research process

      We took our findings from our initial brainstorming session and decided to include diverse stakeholders throughout the research process.

      For the Pathways to Prevention project, the Studio invited families, ACS case-workers, and members from the Community Based Strategies team at ACS for more enriched perspectives.

      04/04. A centralized and digitized tool for tracking, sorting, and mining data

      Now with a revamped design process in place, we needed a software tool that could help fulfill our goals and address our existing pain-points.

      We narrowed our search to the following tools based on a decision-matrix:

      RealtimeBoard
      Pros
      • ask insightful questions
      • identify emerging patterns and insights/trends
      • generate hypothesis and ideas for solutions
      Cons
      • ask insightful questions
      • identify emerging patterns and insights/trends
      • generate hypothesis and ideas for solutions
      Trello
      Pros
      • ask insightful questions
      • identify emerging patterns and insights/trends
      • generate hypothesis and ideas for solutions
      Cons
      • ask insightful questions
      • identify emerging patterns and insights/trends
      • generate hypothesis and ideas for solutions
      Reframer
      Pros
      • ask insightful questions
      • identify emerging patterns and insights/trends
      • generate hypothesis and ideas for solutions
      Cons
      • ask insightful questions
      • identify emerging patterns and insights/trends
      • generate hypothesis and ideas for solutions
      Dovetail app
      Pros
      • ask insightful questions
      • identify emerging patterns and insights/trends
      • generate hypothesis and ideas for solutions
      Cons
      • ask insightful questions
      • identify emerging patterns and insights/trends
      • generate hypothesis and ideas for solutions
      The Studio decided to go with RealTimeBoard (Miro) as a UX tool to centralize and digitize all of our findings going forward.
      Reflection: Lessons Learned_

      Don't underestimate the design process

      After conducting the first round of user interviews, I noticed recurring pain-points that everyone in the team was having, as well as opportunities for solving them.

      To my excitement, it turned out to be much more than a Google search. After hearing my colleagues express their collective pain-points (i.e. working with limited office space, missing sticky-notes, fatigue from spending days sorting post-its)- I realized there were underlying issues that needed to be solved in a user-centered way.

      I took this as an invitation to learn about our Studio, my colleagues' design process, and what a collective vision looked like for standardizing our process in the future.

      The lesson for me was to never underestimate the need for thoughtful design in our everyday lives.

      Ask "why" to get to the root cause

      I learned that before engaging in any design work, it's essential to get to the root cause and not be misled into designing a "band-aid" solution for a problem that's deeper than what initially meets the eye.

      Consider the following dialogue:

      "We need a software tool to enhance the way we manage data."

      "Why?"

      "Because there's a lot of needless duplication and inefficiencies in the way we gather data."

      "Why?"

      "Because the way we in which we obtain data isn't consistent across projects"

      "Why?"

      "Because our design process isn't really defined."

      This method of questioning helped me uncover the Studio's true needs: it wasn't about software tools as much as it was about process-improvement and creating shared, best-practices for the entire team.
      01. The problem
      The problem_

      Identifying the painpoints

      01/03

      Roadblocks throughout the research process

      Duplications cause inefficiencies. The Studio's approach to user-research was filled with roadblocks that made collecting and managing user-data tedious and duplicative. For example, a common workflow after conducting a user-interview would be to tag key insights on sticky notes, record them on a cloud-based service (i.e. Google Docs), and then dispose of them in the bin afterwards. That's a lot of wasted post-its.

      02/03

      Too many UX tools to choose from

      With so many moving parts throughout the research process, the Studio needed a tool to centralize all quotes, insights, and discoveries in one repository and share it with the rest of the team. However, there were too many software vendors to choose from and we needed to pick the right one that would solve for our specific needs.

      03/03

      Not many actionable takeaways from weekly research-synthesis meetings

      Every week, the Studio would meet with key stakeholders on a project to discuss findings gathered during user-interviews. While it was important to have everyone at the table share quotes and insights, the format of the meetings was structured around passive listening and note-taking.

      The meetings were in need of a revamped framework that would enable our Studio team and stakeholders to do the following:

      • take actionable notes
      • ask informed questions
      • generate hypotheses
      • ideate solutions
      02. Research design
      the process: Research design_

      Designing the research

      Recruiting participants for qualitative interviews

      I reached out to four of the Studio's designers: two of whom worked full-time and two who were on a short-term fellowship. I liked that there was variation in how long they’ve been at the Studio. In my mind, the recent fellows would most likely have a different take on how things are done as opposed to the more established designers who may have grown accustomed to the same way of doing things. Fresh minds beget fresh perspectives.

      What did I want to understand?

      In order to understand which UX Tools would best facilitate the Studio’s research process, I had to understand how the Studio did research in the first place. All of my research efforts were focused on understanding three components: People, Processes, and Tools, and how they all affect each other.

      Research goals focused on people, processes, tools

      I wanted to center my research efforts on understanding my colleagues better: to know what their design process is like, which tools they use, and which possibilities they'd like to explore for solving their pain-points and accomplishing their goals.

      03. Synthesizing data
      The process: Synthesizing data_

      Making sense of the data

      After conducting the first round of user interviews, I noticed recurring pain-points that everyone in the team was having, as well as opportunities for solving them.

      Here's what the Studio team expressed as their shared pain-points:

      1. Lack of consistency

      “We need more structure on how to formalize our tools and processes. A lot is missing because a lot of people are involved on the research side. How we defined insights were varied.”

      -Studio_team_member-TR

      "Lack of consistency is what's most difficult about managing data."

      -Studio_team_member-TR
      2. Content overload and burnout

      "A lot of data, and really overwhelming"

      -Studio_team_member-LS

      "After the synthesis and capturing that synthesis, we were really exhausted because it took 4-6 hours a day. How do you take that and take a snapshot of it and have it synthesized for you?"

      -Studio_team_member-MK
      3. Many moving parts

      "One thing that was challenging for us- it was time-consuming to transcribe into note-taking form and then bringing it into post-its, that whole process was cumbersome"

      -Studio_team_member-LS

      "We had these 6 hour long days, and we were really tired and all this stuff would be up and hanging and we'd be like, 'oh we'll return to it tomorrow', but we'd have 800 other things so it would stay up and wait till the next week that'd we'd block and then we'd forget. We'd have to start over and we'd ask 'what is this again?' "

      -Studio_team_member-LS

      Identifying opportunities to explore

      Throughout the interviews, I began to notice emerging themes around opportunities that would be worth exploring as potential solutions to our painpoints.

      Creating an ideal workflow

      "Having a designated space for your research- the space was such a restraint. This was the only space that allowed for everyone to use to do synthesis"

      -Studio_team_member-LS

      "Workflow wise, collaboration is super key- having another at least one or two people in the team working with you so that you can bounce ideas off, having that collaboration so that you're not doing everyting by yourself, even in the synthesis aspect."

      -Studio_team_member-MK

      "When possible, streamlining your notes and typing directly into a computer and then having time blocked off right after an interview to digitize."

      -Studio_team_member-EH

      "Have set synthesis days during a sprint, so you know that every week you have a 4 hour block for synthesis that you know you're coming to the session so you can prepare accordingly."

      -Studio_team_member-EH
      04. Designing a solution
      the process: Designing a solution_

      Bringing the team together to discuss our key findings

      After sorting the key-findings into painpoints and opportunities, I wanted to bring the Studio together for a workshop in which we could all offer input to solve for those pain-points and capitalize on opportunities.

      Validating our painpoints together along a matrix

      Next I presented the team's pain-points and issues. I wanted to see how often they came up in their work and how they felt about them. I asked the following two questions:

      • How painful are these issues?
      • How often do you encounter them?

      We came together as a team and placed our pain-points on a four-quadrant matrix along the lines of "intensity versus frequency". The vertical axis labelled the two extremes of frequency: “Rarely” and “Always”- and the horizontal one labelled the extremes of intensity: “not a pain” and “ouch, big pain.”

      "The most pressing and frequent issues were the following:"

      "Lack of consistent standards, processes, and defined outputs"

      "Onboarding new stakeholders and partners in the research process"

      "Physical exhaustion when sorting sticky-notes for hours when synthesizing data"

      "Rotating teams and positions within teams"

      What should we "stop, start, and continue" in our design processes?

      The team was then invited to deliberate on how to map out an ideal workflow that would ensure consistency and solve for the above pain-points by asking the following questions:

      • What do we want to stop doing?
      • What do we want to start doing?
      • What do we want to continue doing?
      "Here's how the team envisioned a standardized process going forward"
      1. Before we conduct a user-interview, we should...

      Have a team with diverse work-expertise so that various stakeholders can get involved

      Clarify and align on understanding between interview-lead and note-taker

      2. When we finish conducting an interview we should...

      Surface initial and most revelatory insights and quotes

      Generate fifteen immediate insights on sticky notes

      Take 5 minutes to debrief

      Interviewer should send a "thank you" note and follow-up to keep stakeholders invested

      3. When we sort through data we should...

      Have a transcript record of conversation

      Organize information and tag major themes/insights

      4. When we discuss our findings as a team we should...

      Create discussion-frameworks so that workshops and meetings are actionable, engaging, and not passive.

      Incorporate discussion points on patterns, insights, open questions, and hypotheses/ideas

      We ideated the following framework for taking actionable notes

      Each person in the meeting is given a ‘Notes Template’ to fill out as they are listening to the interviews being read aloud. The template will be divided into the following sections:

      Section 1: Patterns

      What emerging patterns are you noticing?

      Section 2: Insights

      Which quotes did you find novel/ interesting? why?

      Section 3: Open questions

      What do you still not know?

      Section 4: Ideating solutions

      What ideas are coming to your mind as potential solutions to explore?

      This note taking template evolved into This gave way to the "Field Logs" which the Studio now uses to journal each important milestone during the design-process for any given project.

      05. Evaluating our solution
      the process: Evaluating our solution_

      Putting our new design framework to the test

      The Studio was commissioned by the NYC Administration for Childrens' Services (ACS) to work on Pathways to Prevention, an initiative that would allow families to use their "voice and choice" to opt for social-services and thereby prevent their children from entering the foster-care system. We noticed the following improvements:

      01/04

      Better, actionable notes called "Field Logs" during qualitative interviews and synthesis meetings

      The Studio created note templates for stakeholders to utilize which helped them:

      • ask insightful questions
      • identify emerging patterns and insights/trends
      • generate hypothesis and ideas for solutions

      The quality of follow-up questions was higher- helping the team explore solutions that addressed the needs of family stakeholders in a more meaningful way.

      02/04

      The team was better informed when testing out prototypes

      By focusing on key emerging-trends gathered during interviews, the team was able to quickly prototype ideas that spoke to the needs of their stakeholders and receive validation from case-workers at ACS.

      03/04

      Diverse groups of teams and stakeholders were incorporated in the research process

      We took our findings from our initial brainstorming session and decided to include diverse stakeholders throughout the research process.

      For the Pathways to Prevention project, the Studio invited families, ACS case-workers, and members from the Community Based Strategies team at ACS for more enriched perspectives.
      04/04

      A centralized and digitized tool for tracking, sorting, and mining data

      Now with a revamped design process in place, we needed a software tool that could help fulfill our goals and address our existing pain-points.

      We narrowed our search to the following tools based on a decision-matrix:

      RealtimeBoard
      Pros
      • ask insightful questions
      • identify emerging patterns and insights/trends
      • generate hypothesis and ideas for solutions
      Cons
      • ask insightful questions
      • identify emerging patterns and insights/trends
      • generate hypothesis and ideas for solutions
      Trello
      Pros
      • ask insightful questions
      • identify emerging patterns and insights/trends
      • generate hypothesis and ideas for solutions
      Cons
      • ask insightful questions
      • identify emerging patterns and insights/trends
      • generate hypothesis and ideas for solutions
      Reframer
      Pros
      • ask insightful questions
      • identify emerging patterns and insights/trends
      • generate hypothesis and ideas for solutions
      Cons
      • ask insightful questions
      • identify emerging patterns and insights/trends
      • generate hypothesis and ideas for solutions
      Dovetail app
      Pros
      • ask insightful questions
      • identify emerging patterns and insights/trends
      • generate hypothesis and ideas for solutions
      Cons
      • ask insightful questions
      • identify emerging patterns and insights/trends
      • generate hypothesis and ideas for solutions
      The Studio decided to go with RealTimeBoard (Miro) as a UX tool to centralize and digitize all of our findings going forward.
      06. Lessons learned
      Reflection: Lessons Learned_

      Don't underestimate the design process

      After conducting the first round of user interviews, I noticed recurring pain-points that everyone in the team was having, as well as opportunities for solving them.

      To my excitement, it turned out to be much more than a Google search. After hearing my colleagues express their collective pain-points (i.e. working with limited office space, missing sticky-notes, fatigue from spending days sorting post-its)- I realized there were underlying issues that needed to be solved in a user-centered way.

      I took this as an invitation to learn about our Studio, my colleagues' design process, and what a collective vision looked like for standardizing our process in the future.

      The lesson for me was to never underestimate the need for thoughtful design in our everyday lives.

      Ask "why" to get to the root cause

      I learned that before engaging in any design work, it's essential to get to the root cause and not be misled into designing a "band-aid" solution for a problem that's deeper than what initially meets the eye.

      Consider the following dialogue:

      "We need a software tool to enhance the way we manage data."

      "Why?"

      "Because there's a lot of needless duplication and inefficiencies in the way we gather data."

      "Why?"

      "Because the way we in which we obtain data isn't consistent across projects"

      "Why?"

      "Because our design process isn't really defined."

      This method of questioning helped me uncover the Studio's true needs: it wasn't about software tools as much as it was about process-improvement and creating shared, best-practices for the entire team.
      Next case study
      Administration for Childrens' Services
      Read Case Study