Adaptive Learning Engine with Real-Time Analytics

The Challenge

An online education platform serving 300,000+ students was delivering the same linear curriculum to every learner regardless of prior knowledge or learning pace. Completion rates sat at 34%, and student feedback consistently cited "too easy at the start, too hard later" as the primary frustration. Instructors had no real-time visibility into which concepts students were struggling with — they received weekly CSV exports that were typically 5 days stale by the time they reviewed them.

Our Approach

We built an adaptive learning engine that personalises content sequencing for each student using Bayesian Knowledge Tracing (BKT). The model maintains a per-student, per-concept mastery probability that updates with every interaction — correct answers increase mastery estimates, incorrect answers decrease them, and the model accounts for the forgetting curve over time.

Content Sequencing: When a student completes an exercise, the engine selects the next item from a pool of available content based on the student's current mastery profile. Items targeting concepts where mastery is between 0.3 and 0.7 (the "zone of proximal development") are prioritised. Fully mastered concepts are spaced using a modified SM-2 algorithm.

Event Pipeline: Every learning event (page view, video pause, quiz answer, hint request) is published to Apache Kafka and consumed by the BKT updater service. The pipeline processes 2M+ events daily with sub-second end-to-end latency. Events are also written to ClickHouse for analytics queries.

Instructor Dashboard: A real-time dashboard built with Next.js and Recharts shows per-class and per-student mastery heat maps, concept difficulty rankings, and engagement metrics. Instructors can identify struggling students within minutes rather than days.

Results