Repunzel

A web app that allows actors seeking representation to send personalized emails to verified lists of talent agents in a few clicks and then track who have viewed their materials.

Overview

Repunzel is a start-up company that develops B2C products designed to support actors with finding representation and tracking stats to grow their career. For this project, I led evaluative research for a web app that allows actors seeking representation to send personalized emails to verified lists of talent agents in a few clicks and then track who have viewed their materials.

Client

Repunzel

Team

Program Manager

Founder

UX Designer

Engineers

My Role

Sole UX Researcher

Methods

Stakeholder Interview

Usability Testing

User Interviews

Surveys

System Usability Scale (SUS)

Affinity Diagramming

Timeline

6 weeks

Background

The founder wanted to uncover his app’s usability by testing the beta version to uncover any problems so that the designer and developers can rapidly iterate before the product launch date. I interviewed the founder (also subject-matter expert) to clarify context and discovered that the product was already in the evaluative stage of its life cycle but did not test earlier designs and prototypes with target users. Nevertheless, we developed an evaluative research plan with goals and questions to guide our research moving forward.

Research Goals (RG)

#1: Evaluate if the app makes the process of submitting to agents easier than what users are currently doing

#2: Determine if app is easy and intuitive for users to use

#3: Determine if additional value is being provided through the app (ex: insights, data, etc.)

Research Questions

  • What is your experience with submitting to agents so far? (RG 1)

  • What parts of the app did users find confusing or felt lost in? (RG 1)

  • What features of the app were helpful or useful? (RG 1, 2, 3)

  • Were users able to complete the process of submitting to agents? (RG 2)

  • Do users have all the information they need to compose an email to agents? (RG 3)

  • What is the discoverability of the test email feature? (RG 3)

Methodology

  • Moderated Usability Testing (RG 1,3)

  • In-Depth Interviews (RG 1,3)

  • System Usability Scale (SUS) (RG 2)

To maintain product confidentiality and ensure that the sample group of users tested reflected the target user group, I selected moderated usability testing and interviews to gather behavioral and attitudinal data from first time users. SUS was used to provide benchmarking metrics and address the 2nd research goal. 

I conducted usability testing for the 1st round of beta testing and shared my findings with designers/developers. They rapidly iterated before the 2nd round of testing, which would follow up with users after they have used the app for a week with an interview. I developed the interview script for the moderated usability test independently (1st round) but invited our recently hired manager to co-create the interview script for the in-depth interviews (2nd round).

Participant Recruitment

Potential users were gathered as a result of the founder’s connections to managers in the acting industry. I developed a screener survey determine which set of participants would represent the most diverse and representative sample of target users based on the following criteria.

Age, experience/inexperience with submissions, gender, ethnicity, comfortability with technology use.

I wanted the sample of 7 users in each round of testing to be as similar in characteristics as possible. Unfortunately, one user dropped out for each round of testing and so we ended up with a sample size of 6 users each for moderated usability studies and interviews.

NOTE: Research documents were shared with the founder through Google Drive for instant access, including recorded usability testing and interview sessions. I also provided a quick summary for the team to follow up on while I synthesized my findings into a report.

Research Impact

From both usability testing and in-depth interviews I synthesized user data by affinity diagramming using Miro and provided insights and recommendations report to the CEO and manager for designers and developers to follow up prior to product launch. I invited the founder and manager to join me in a facilitated affinity diagramming session after the 2nd round of testing to increase research buy-in. Insights were categorized each time by priority (P0, P1, P2). I have summarized below only the most salient insights after each round of testing and connected it to the appropriate research goal.

Insights/Recommendations from Moderated Usability Studies

  • Need for improved discoverability of certain features (ex: sending a test email and locating/modifying agent list). I made a design recommendation to provide a faded preview of the agent list to increase discoverability which was iterated on before the next round of testing. (RG 2, 3)

  • System feedback needed for users when uploading headshots (ex: image size, spacing, headshot limit) so that images look how they want it. (RG 1)

  • Clearer signifiers needed around the send limit of 50 emails a day for user awareness and understanding so they are not caught off guard. (RG 2)

Insights/Recommendations from In-Depth Interviews

  • I made a recommendation to address user frustrations around the logistics and timing from the send limit. So that, instead of users being confined to a 24 hour wait cycle from when they sent their last email, it should restart at midnight of whatever time zone they are in. (RG 1)

  • Quality of life enhancements needed for Email Composer and Tracking Report. For example, many users wanted more analytics on the timestamps of when their email was opened in the Tracking Report. (RG 3)

  • We need ways to support inexperienced actors seeking submission (ex: cover letter examples as a package add-on) (RG 3)

System Usability Scale (SUS) Results

  • Post-Round 1 Testing: 66% of tested users rated usability as Excellent (RG 2)

  • Post-Round 2 Testing: 100% of tested users rated usability as Excellent (RG 2)

Overall: 34% increase in participants rating usability as Excellent.

Learnings

  • Conducting an early heuristic evaluation prior to the first round of testing could have yielded findings for an additional iteration. 

  • To track user behaviors over a week of usage, I would supplement the interview with a diary study to capture more insights users had while using the app instead of relying on user recall and notes. With enough of a research budget, we could also have a larger sample size of data to draw from.

  • At the foundational research stage, I would conduct contextual inquiries with target users to acquire qualitative and behavioral data and minimize costly time and resources for design and development.

  • The more involved stakeholders are in the research process, the more they are willing to act upon your insights and recommendations.

Challenges

From the initial meeting, we only had 6 weeks to do research and iterate quickly in time for launch. As the sole researcher on the team working part-time, I was only available to do usability testing after 3pm. Difference in time zones for some users compounded issues since some users wanted to avoid submitting past agency office hours, thus not completing a particular task.

Furthermore, most of the research was happening during holiday season which is non-ideal timing for submissions with agents. Research timeline shifted several times since iterations from design and development took longer than planned and led to a time crunch when conducting in-depth interviews (IDI’s). To address time constraints, I asked my manager to assist me in facilitation of IDI sessions.