From Chronological Clutter to Curated Content
Generated by Firefly
A series of research for Google to inform and evaluate a visionary personalized content experience, yielding game-changing insights that have guided product design and engineering innovations.
* The studies were redacted for confidentiality purposes.
Summary
Role
As the Lead Researcher, I directed the research lifecycle for a complex, multi-phase research initiative with deliverables tailored for design and engineering.
Goals
Discover what users value most in their tab* content and to gather feedback on a forward-looking concept to understand perceptions and mental models, informing the development of an MVP.
* Experience is associated with one of Gmail’s three default tabs.
Methods
Over two months, I orchestrated a two-part study, conducting parallel diary studies and subsequent in-depth interviews to generate user insights to shape a personalized experience.
Impact
“The value work, which I reference all the time, has established an excellent baseline and touchpoint for focus. It serves as a conversational backstop. It has turned something speculative into something intuitive.”
- Product Manager
Project Challenge
The product team responsible for one of Gmail's default tabs* embarked on an ambitious project aimed at transforming the core user experience from a generic, one-size-fits-all list into a personalized showcase tailored to user-valued content, rooted in foundational UXR and feature exploration.
Initially, the team requested a quick-turn study to meet a previously committed engineering key result (KR). However, as the project progressed, it became evident that a deeper understanding of how users assess the value and importance of their content was necessary. This led to the first systematic execution of such a user-value assessment within the organization.
I successfully advocated for postponing the concept evaluation and aligning the team on a more comprehensive plan. This strategic pivot not only deepened our understanding of users' core needs but also uncovered crucial insights that proved invaluable across various projects.
* Tabs were designed to automatically filter emails by specific content types. There are three default tabs, Primary, Promotions, and Social.
The Team
I was part of an exceptional cross-functional (xFN) team at Google. As an embedded qualitative UX Research consultant, I was tasked with leading a research project in partnership with a Product Manager, Lead UX Designer, and Lead LLM Engineer. Stakeholders also included xFN and cross product area (xPA) Director and VP-level leadership across two distinct verticals, highlighting the project's strategic impact.
Goals & Objectives
The primary goals were to assess the concept and inform a Minimum Viable Product (MVP) using semi-customized prototypes. Given the personalized nature of the conceived experience and the requirements from both design and engineering to advance the product, a two-part study was conducted.
1: Diary Study
To learn what types of content should be prioritized for inclusion and how users attribute value and relative value.
Content Prioritization: Determine which types of tab content users value most for product inclusion.
User Value Assessment: Explore how users assign value and importance to different email content.
Engagement Understanding: Analyze how perceived value influences engagement.
Preparation for Concept Evaluation: Use collected content to create semi-customized prototypes.
2: Concept Evaluation IDIs
To get users’ feedback on speculative categorical content design and experience.
Non-Chronological Perceptions: Learn how users think and feel about a subset of tab content being presented by user-value.
Visual Presentation Feedback: Assess user reactions to and preferences for a visually enhanced presentation.
Interaction & Satisfaction Exploration: Understand how the hypothetical changes affect user interaction and satisfaction with the tab, as well as expectations for non-included content.
Methodology
Who did we learn from?
We recruited a total of 45 participants across two user groups differentiated by tab usage patterns, to understand their needs and inform product growth strategies. All participants completed the diary study, and one-third were selected for follow-up IDIs.
How Did We Gather Insights?
Diary Study Methodology
We conducted two parallel diary studies, with durations and numbers of visits adjusted to reflect the user groups' visit behaviors. This approach allowed us to observe typical interactions with the tab. During the study, a mid-study synthesis helped establish a content value schema, which informed the design of the final diary activity.
_____________________
Data Collection
Screen recordings and saves of engaged with emails of interest and value.
Responses on interest and valuation.
Saving and ranking of top-valued emails.
_____________________
Activities
Introduction: Exercises prompting value articulation, coupled with screen recording practice.
Content Exploration: Recording and reflecting on their tab visits, highlighting, evaluating, and saving emails based on value.
Final: Answering ranking questions about content value based on schema derived through mid-study analysis and submitting their top five valuable emails with recorded explanations.
IDI Study Methodology
75-minute IDIs were conducted with select participants from each diary study group, using stimuli customized from content they engaged with during the diary studies to simulate a personalized experience.
_____________________
Study Stimuli
Participants evaluated two versions differing primarily in email format scale, both maintaining consistent user affordances and interface interactions. Counterbalanced setups introduced each version at a semi-arbitrary entry point to ensure unbiased evaluations of user preferences and prevent preconceived notions about navigation from influencing initial reactions
_____________________
Interview Flow
Initial Impressions: Gathering first impressions and overall perceptions.
Scenario-Driven Feedback: Exploring user actions, expectations, and desires within specific scenarios.
Interaction Probe: Investigating expectations and desires for interactions not previously explored.
Wrap-Up: Identifying preferred concept and a hypothetical query to assess considerations for adoption.
Examples of insights—redacted and reformatted
Insight Spotlight
What was learned from the diary study?
Semantic Expectations
User expectations for tab content closely align with their understanding of the tab label. This alignment creates strong mental models, and when content meets these expectations, it reinforces satisfaction and trust.
Precious Content
Within a vast landscape of Tab content, users consider a specific category the treasure trove, with the greatest potential for hidden gems that influence their decisions and critical online journeys. While other valued content types offer sustained inspiration, reliable references, and shareable value, it is the desire for the precious gems that drives user engagement and shapes their current experience.
“ I would have to say [x] and [y] emails, because we are a family of 5 and even an email that provides [benefit] makes a difference.”
Timely Orientation
Users appreciated the benefits of a personalized experience, but wanted the option to switch to reverse chronological order. The element of time provided orientation and context, helping to alleviate the fear of missing out (FOMO).
What was learned from IDIs?
Text < Photos
Users were more likely to overlook the text-only format and perceived it as hierarchical—attributing lower importance or value.
“[Concept] is focused mainly on [x], and I feel like that’s what [the Tab] is for...So this experience fits really well with the Tab.”
Impact & Learnings
Maximizing Value
To bridge the gap between user insights and engineering implementation, I fostered a collaborative environment where custom outputs were developed and refined in close partnership with the engineering team. This iterative process ensured that our research findings were actionable and directly applicable, empowering engineers through user insights and directly influencing model training and prioritization of engineering tasks.
Actionable Deliverables:
A comprehensive guide detailing nuanced value findings.
Audit tool that mapped and ranked these findings against existing LLM signals and features.
Interactive xFN Workshop: Two engaging two-hour workshops featuring games and brainstorming sessions designed to boost engagement and apply findings across the entire engineering and design teams.
“
Your ability to summarize and communicate insights helped me understand the value of small quantity, but “deep dive” qualitative research.”
Adaptive Insights
Despite a complex approval process, the research initially secured preliminary buy-in for the xPA product area. However, shifts in leadership and priorities halted advancement to live experimentation. The effective packaging and socialization of insights ensured they remained top of mind, enabling the team to pivot effectively. This led to the exploration of new solutions that continued addressing the same core user pain points, all while aligning with critical user journeys in the xPA sprint.
Key Learnings:
Early and Frequent xPA Sharing: Involving cross-product area (xPA) teams in UXR and UXD early fosters a holistic and aligned approach to product development.
Packaging Insights for Longevity: Embrace inevitable change and support pivots through strong socialization of research focused on core user needs (mental models & hierarchies) to ensure lasting relevance and utility of insights, even when immediate product outcomes shift.
Generated by Firefly
Museum Analogy: A Vivid Proxy for Prototypes
Since we cannot display the actual prototypes from the study, imagine this analogy that mirrors the transformative user experience we aimed to create. Picture a museum where exhibits are arranged in chronological order, predictable and overwhelming, versus one offering personalized tours based on individual interests. In the conventional museum, you might wander aimlessly, while in the tailored version, a knowledgeable guide leads you directly to exhibits that resonate with you, each enhanced by interactive labels for deeper engagement. This analogy serves to vividly conceptualize the complexity of the user-centric transformation we sought to evaluate and inform, capturing the extensive enhancements envisioned for the user experience.
