Design Top-N Leaderboard with Rolling Time Windows

Problem Statement

People regularly share articles on LinkedIn. Design a system to maintain a Top-N leaderboard of the most-shared articles over rolling windows of:

• 5 minutes
• 1 hour
• 24 hours

The leaderboard should be near real-time and support millions of share events per day.

Requirements

Functional:

• Track article share events
• Query top N articles for any of the three windows
• Results should be near real-time (acceptable lag: a few seconds)

Non-Functional:

• Handle 10M+ share events/day
• Leaderboard reads should be < 50ms
• Eventual consistency is acceptable (few seconds lag)
• Horizontally scalable

What the Interviewer Expects

1. Event ingestion:

   • Share events → Kafka topic (partitioned by article_id for ordering)
   • Consumers process events and update counters

2. Rolling window strategies:

   • 5-minute window: Redis sorted set with TTL. ZINCRBY on share, ZREVRANGE for top-N. Entries expire naturally.
   • 1-hour window: Time-bucketed counters (twelve 5-min buckets). Sum the buckets for current hour.
   • 24-hour window: Pre-aggregated hourly counts. Sum last 24 hourly buckets. Periodic rollup job.

3. Read path:

   • Cache the leaderboard results (refresh every 5-10 seconds)
   • Clients hit the cache, not the live computation
   • Different refresh rates for different windows (5-min refreshes every 5s, 24-hour refreshes every minute)

4. Accuracy vs performance trade-off:

   • Exact counting at LinkedIn scale is expensive
   • Approximate counting (Count-Min Sketch, HyperLogLog) for the 24-hour window
   • Exact counting for 5-minute window (smaller data volume)

5. Scaling:

   • Partition by article_id hash
   • Multiple Redis instances for different time windows
   • Pre-compute and serve from CDN for the 24-hour leaderboard

Follow-ups

1. What if an article gets "un-shared" (deleted)? How do you decrement across windows?
2. How would you handle spam/bot detection in the share events pipeline?
3. How would you extend this to personalized leaderboards (top articles in YOUR network)?
4. What's the memory footprint of maintaining sorted sets for millions of articles? How do you bound it?
5. How would you handle a viral article that gets 100K shares in 1 minute? (Hot key problem)