Experimentation & Change Framework

Experimentation & Change Framework

A practical approach to rapid testing, learning, and implementation.

A practical approach to rapid testing, learning, and implementation.

Implementation Timeline

Implementation
Timeline

Problem

One of the key challenges we faced after launching the Vaults feature was low purchase volume. Despite the appeal of high-value prizes like a PS5, users were skeptical about the legitimacy of the feature. Many perceived it as gimmicky or doubted whether anyone was actually winning, which significantly reduced trust and ultimately suppressed conversions.

Experimentation/Iteration

To address this, I ran a series of notification experiments aimed at building credibility and urgency by highlighting real user wins.

Copy example that was targeted to our Gen Z audience:

Header:
These lucky mfs!!!

Subtext:
Tap to see what these 4 users have won from a vault


Final Output

The notifications proved highly effective—each time we highlighted a real user win, vault purchases spiked by over 160% compared to our typical performance. This insight led to a lasting product improvement: we evolved the winning notifications into a persistent social proof ticker within the app. Now users can see real-time updates whenever someone wins a prize, reinforcing legitimacy and driving ongoing engagement through subtle, continuous trust-building.