Hacking Engagement: How I Improved Learning Product Engagement by 25%

Hey Guys,

Welcome back to my blog where I write about my learnings as a product manager and try contributing to others passionate about product management.

I was moved to a product Newton HeadStart since Jan 2025. It's the flagship Product at Newton School and I have had so many learnings through it. Recently I had a big win where I bumped up the engagement for the course by 25%.

Sharing it here.

Context

So Newton HeadStart is at this point (1 June 2025) India's largest Platform to learn coding for 11th and 12th Standard students. We have roughly 124,000 Users on the platform, learning courses like C++, Python, Web Dev and AI.

A few things to know before we get deeper.

How did we define Engagement?

We look at Video on Video Retention of user. A successful video watch was defined as a user spending at least 30 Secs on the video.

Why? Because our conversion equation can be built easily around video watch to converted. (Different story to tell, some other day).

The problem we faced:

• Video 1 → Video 2 had a 50% drop.
• Video 2 → Video 3 had a 65% drop.

And after video 3 the funnel operates constantly, users who watch upto 3rd video, 90–80% users watch the next video and it keeps happening till the end on the playlist.

Since our conversion is tied up with engagement, that engagement was our biggest lever to bring true conversion at scale.

So how did we solve it?

Started off with talking to users.

Spoke to users who had engaged till the end of the course and try to understand everything about them: why they came to the course, why did they start watching and what hooked them till the end?

There were multiple reasons but a common reason which was very prominent among some users was that they wanted the certificate that comes after completion of the course.

One that stood out the most was a user from Gujarat. He already knew Python but he still came to the platform, forcefully learnt Python again on HeadStart because he wanted to claim the certificate from the course.

Why? To show it, since it brought instant credibility on his profile. He hasn't even entered college yet and a certificate did hold so much value even at this point.

This gave me the confidence this would for sure bring a big bump if more users were educated and motivated to move closer to the certificate.

The Experiments

So we sent WA (WhatsApp) messages to a few not-engaged users. We said: "Your Certificate will expire in 17 Days, finish your course before it to claim your certificate."

This performed exceptionally better than the variant in A/B test. 17% CTR on link.

Got even more confidence on the idea and hence we designed 2 experiments.

1. Certificate Progress

Users would be shown their progress on how close are they to claim the certificate.

2. Certificate Expiry

Users would be shown the deadline before which they can claim the certificate.

Then after a lot of fights on stakeholder alignment and pushback we finally sent into development. We made it live on 13th June 2025.

The Results

Then comes the next day, we were building the dashboard for experiment and building cohorts that were running this experiment on.

And then when we see the video on video retention we were shocked. We cross checked it from multiple methods, if we're missing out on something. It bumped up the users video completion metrics by a crazy multiplier.

This is how the Certificate Expiry Experiment performed compared to default:

Overnight video on video retention bumped up and Completion till 5th Video bumped from 7.52% → 9.43%, a 25% jump in Video on Video Retention.

That's it for the blog. Hope I did add value to your time. Will share more such learnings in the future blogs.

Thanks for Reading. Bye.