Hoxhunt Leaderboards

At a glance

Title: Cyber security training leaderboards revamp

One-liner: A leaderboard that was pushing new joiners to give up transformed into a tool that brings fair and friendly competition to a cyber security training platform.

Role: Sole designer on project, responsible for UI, UX and discovery.

Team: Designer (me), PM, engineering pod of 6 – front/back-end

Timeframe: Autumn 2022

Context: Hoxhunt is an enterprise cyber security awareness training platform that uses a gamified training approach to increase employee engagement and improve learning outcomes.

Introduction

Users of the Hoxhunt platform earn points for reporting simulated phishing emails. One of the platform's key features is the leaderboard, which adds an element of gamification and competition to cyber security training. But the leaderboard ranked users by the absolute number of points they'd earned, so those who had been in the training longest were almost always at the top. If you were a new joiner, you had no chance of reaching the top of the leaderboard, no matter how hard you tried.

We were hearing through admin contacts that new joiners felt they had no chance to "win". This discouraged new employees from taking part, and our platform's adoption was suffering as a result. Hoxhunt's unique value proposition is that it makes cyber security training fun, so redesigning the leaderboard to make it fairer was a key initiative for my team when I joined the company.

Framing the problem

• How might we make leaderboard ordering logic fair?

• How might we enable healthy competition between countries, sites, business units, and departments?

• How might we make it easier to reward the best players?

• How might we make leaderboards a more positive experience for the average player?

How we measured success

Because the platform didn't support direct end-user research, we measured success through admin and CISO channels: a drop in complaints, qualitative feedback from points of contact, and leaderboard engagement over time.

Solution – a personal leaderboard with a twist

The first hard problem we hit while exploring solutions was this: if the leaderboard no longer measures the absolute number of points a user has earned, how do we measure success? Our first serious direction was a "Hall of Fame" – tiers of recognition, records like fastest report and longest streak, honorary mentions, country match-ups. It was an engaging way to represent the data across different dimensions. But even though it surfaced more users, it still only highlighted the top performers. What we kept hearing through CISOs was that users wanted to know where they stood in the organisation. So we turned away from the Hall of Fame idea and took a different direction.

[Image — Hall of Fame exploration]

We approached the problem from a different angle. Users resented old-timers having an advantage simply because they'd had more time to collect points – so why not reset the score every quarter? The old leaderboard was also fundamentally unfair, because not everyone received the same number of simulated phishing emails. So we based the score on a percentage system, where 100% is the best possible score you could have achieved in a given quarter. Resetting every quarter would also give new users – and people returning from holiday – a reason to re-engage. This was fairer: even though users don't all receive the same number of simulations, everyone is scored proportionally to their own potential.

This created another challenge. In larger organisations, many users receive the same number of simulations, so there were always ties. We broke ties with a secondary metric: time to report. It does disadvantage people who can't check their email as often, but reporting speed matters in the real world too, so we felt it was a fair basis.

The trade-off is that a pure ranked list is quite dry. By focusing on the individual, the bigger picture faded – the champions, and what a bare rank like 57th out of 283 actually means. To bring that meaning back, we borrowed ideas from the discarded Hall of Fame exploration and added contextual banners like "you're in the top 30% of the org". We also added different leaderboard groupings, so users could see themselves within the whole organisation, or just their own team or country.

Finally, to add delight and a "wow" factor, I refreshed the interface and worked with an illustrator to create a set of bespoke animated illustrations. The illustration shown depended on your score – the better the score, the fancier the illustration.

Outcome

After we released the new leaderboards, the feedback from our points of contact was positive, and the complaints about fairness stopped.

This project was a great opportunity to learn about SaaS gamification. It opened my eyes to how delicate a good game system can be, and to how much more users care about an otherwise dry product once these elements are added.

The biggest challenge throughout the project was getting input from day-to-day users, as opposed to CISOs and admins. As a small, bootstrapped B2B company, we weren't set up for user research at the time. The project taught me to plan for user research from the very start — especially when no research pipeline exists. </content> </invoke>