Day 21: Learning Path Generator - Intelligent Curriculum Sequencing

Aug 17, 2025

The Problem We're Solving

Ever wondered why some students excel while others struggle, even when studying the same material? The answer often lies in learning sequence. Traditional education follows a one-size-fits-all approach - everyone learns Topic A, then B, then C, regardless of their background, learning speed, or mastery level.
Key Pain Points:

Students get stuck on prerequisites they haven't mastered
Difficulty jumps are too steep, causing frustration and dropouts
No personalization based on individual progress patterns
Inefficient paths waste time on topics already understood
No guidance on optimal next steps in learning journey

Our Solution: An AI-powered Learning Path Generator that creates personalized curriculum sequences, just like how Netflix recommends movies or Spotify curates playlists - but for education.

What We're Building Today

Today we're implementing a Learning Path Generator that creates personalized learning sequences for quiz platform users. This system analyzes user knowledge, topic dependencies, and difficulty progression to generate optimal learning paths.

Key Components:

Path generation algorithm service
User progress analysis engine
Difficulty progression calculator
REST API endpoints for path retrieval
Integration with topic relationship data from Day 20

Core Concept: Adaptive Learning Sequences

Learning Path Generation transforms static curriculum into dynamic, personalized experiences. Instead of everyone following the same sequence, our system creates optimal paths based on individual progress and knowledge gaps.

Why This Matters: Netflix recommends movies, Spotify curates playlists - educational platforms need intelligent content sequencing too. Companies like Khan Academy and Coursera use similar algorithms to increase completion rates by 40-60%.

System Architecture & Data Flow

Our Learning Path Generator operates as a microservice that consumes topic relationships and user data to produce optimized learning sequences.

Core Components:

Path Algorithm Engine: Implements topological sorting for prerequisites
Difficulty Analyzer: Calculates optimal progression curves
User Progress Tracker: Maintains learning state and mastery levels
API Gateway: Exposes path generation endpoints

Data Flow:

User requests learning path via API
System fetches user's current progress and mastery data
Algorithm analyzes topic dependencies from relationship service
Difficulty progression engine calculates optimal sequence
Personalized path returned with next recommended topics

Algorithm Deep Dive

Topological Sorting for Prerequisites: Our system uses modified topological sorting to ensure prerequisite topics appear before dependent ones. Unlike simple dependency resolution, we weight edges based on knowledge transfer effectiveness.

Difficulty Progression Modeling: We implement adaptive difficulty curves that maintain learner engagement. Too easy = boredom, too hard = frustration. Our algorithm finds the "zone of proximal development" sweet spot.

Personalization Factors:

Current mastery levels per topic
Learning velocity patterns
Historical struggle points
Preferred content types

Real-World Implementation Patterns

Spaced Repetition Integration: Like Anki's algorithm, we schedule review topics based on forgetting curves and mastery confidence.

Multi-Objective Optimization: Balance multiple goals - completion time, retention rate, engagement level. Similar to how YouTube's recommendation algorithm optimizes for watch time AND user satisfaction.

Adaptive Branching: Dynamic path adjustments based on real-time performance, mimicking how video games adjust difficulty.

Component Integration in Quiz Platform

This service integrates seamlessly with our existing architecture:

Consumes topic relationships from Day 20's relationship mapping service
Provides learning sequences for quiz content delivery
Prepares ground for Day 22's caching layer optimization

API Design:

GET /api/learning-path/{user_id}
POST /api/learning-path/generate
PUT /api/learning-path/{user_id}/progress

Production Considerations

Scalability: Path generation can be CPU-intensive. We implement caching and async processing to handle millions of users.

Real-time Updates: As users complete topics, paths recalculate automatically. WebSocket connections push updated sequences to frontend.

A/B Testing Ready: Multiple algorithm implementations allow comparative effectiveness testing.

Implementation Guide

Github Link:

https://github.com/sysdr/aie/tree/main/day21/learning-path-generator

Prerequisites

Python 3.11+
Node.js 18+
Docker and Docker Compose
Basic understanding of graph algorithms

Step 1: Project Setup

Create the project structure and install dependencies:

bash

# Create project directory
mkdir learning-path-generator && cd learning-path-generator

# Setup Python backend
mkdir -p backend/{app,tests,config}
mkdir -p backend/app/{models,services,api,utils,database}
cd backend && python3 -m venv venv && source venv/bin/activate

# Install backend dependencies
pip install fastapi uvicorn sqlalchemy psycopg2-binary redis pydantic pytest networkx numpy scikit-learn

Step 2: Core Algorithm Implementation

Learning Path Generator Service (backend/app/services/path_generator.py):

python

import networkx as nx
import numpy as np
from typing import List, Dict, Optional

class LearningPathGenerator:
    def __init__(self, db: Session):
        self.db = db
        self.topic_graph = None
        self._build_topic_graph()
    
    def generate_personalized_path(self, user_id: int, target_topics: List[int], 
                                 max_difficulty_jump: float = 1.5) -> Dict:
        # Get user's current progress
        user_progress = self._get_user_progress(user_id)
        
        # Generate base path using topological sorting
        base_path = self._generate_prerequisite_path(target_topics)
        
        # Apply personalization and difficulty optimization
        personalized_path = self._personalize_path(base_path, user_progress, max_difficulty_jump)
        
        # Calculate success metrics
        path_metrics = self._calculate_path_metrics(personalized_path, user_progress)
        
        return {
            "topic_sequence": personalized_path,
            "estimated_duration": path_metrics["total_duration"],
            "completion_probability": path_metrics["success_probability"],
            "difficulty_progression": path_metrics["difficulty_curve"]
        }

Step 3: Database Models

Learning Path Models (backend/app/models/learning_path.py):

python

from sqlalchemy import Column, Integer, String, Float, DateTime, JSON, ForeignKey
from sqlalchemy.ext.declarative import declarative_base
from datetime import datetime

Base = declarative_base()

class Topic(Base):
    __tablename__ = "topics"
    id = Column(Integer, primary_key=True)
    name = Column(String, index=True)
    difficulty_level = Column(Float)  # 1.0 to 10.0
    estimated_duration = Column(Integer)  # minutes
    prerequisites = Column(JSON)  # List of topic IDs

class UserProgress(Base):
    __tablename__ = "user_progress"
    id = Column(Integer, primary_key=True)
    user_id = Column(Integer, ForeignKey("users.id"))
    topic_id = Column(Integer, ForeignKey("topics.id"))
    mastery_level = Column(Float)  # 0.0 to 1.0
    completion_status = Column(String)
    time_spent = Column(Integer)

Step 4: API Endpoints

Learning Path API (backend/app/api/learning_paths.py):

python

from fastapi import APIRouter, HTTPException
from pydantic import BaseModel
from typing import List, Optional

router = APIRouter()

class PathGenerationRequest(BaseModel):
    user_id: int
    target_topics: List[int]
    max_difficulty_jump: Optional[float] = 1.5

@router.post("/generate")
async def generate_learning_path(request: PathGenerationRequest):
    generator = LearningPathGenerator(db)
    result = generator.generate_personalized_path(
        request.user_id, 
        request.target_topics, 
        request.max_difficulty_jump
    )
    return result

@router.put("/{user_id}/progress")
async def update_progress(user_id: int, progress_data: dict):
    # Update user's learning progress
    # Recalculate path recommendations
    return {"message": "Progress updated successfully"}

Step 5: Frontend Dashboard

React Dashboard Setup:

bash

# Setup React frontend
cd ../frontend
npx create-react-app . --template typescript
npm install @mui/material @emotion/react @emotion/styled recharts axios react-router-dom

Main Dashboard Component (frontend/src/pages/Dashboard.js):

javascript

import React, { useState, useEffect } from 'react';
import { Container, Grid, Card, CardContent, Typography, LinearProgress } from '@mui/material';
import { LineChart, Line, XAxis, YAxis, CartesianGrid, Tooltip, ResponsiveContainer } from 'recharts';

function Dashboard() {
  const [userStats, setUserStats] = useState({});
  const [progressData, setProgressData] = useState([]);
  const [nextTopics, setNextTopics] = useState([]);

  useEffect(() => {
    loadDashboardData();
  }, []);

  const loadDashboardData = async () => {
    try {
      // Load user paths, progress, and recommendations
      const pathsResponse = await fetch('/api/learning-paths/1');
      const progressResponse = await fetch('/api/users/1/progress');
      const nextTopicsResponse = await fetch('/api/learning-paths/1/next-topics');
      
      // Update state with fetched data
    } catch (error) {
      console.error('Error loading dashboard data:', error);
    }
  };

  return (
    <Container maxWidth="lg">
      <Typography variant="h4" gutterBottom>Learning Dashboard</Typography>
      
      <Grid container spacing={3}>
        {/* Stats Cards */}
        <Grid item xs={12} sm={6} md={3}>
          <Card>
            <CardContent>
              <Typography variant="h6">Completion Rate</Typography>
              <Typography variant="h4">{userStats.completionRate}%</Typography>
            </CardContent>
          </Card>
        </Grid>

        {/* Progress Chart */}
        <Grid item xs={12} md={8}>
          <Card>
            <CardContent>
              <Typography variant="h6">Weekly Progress</Typography>
              <ResponsiveContainer width="100%" height={300}>
                <LineChart data={progressData}>
                  <CartesianGrid strokeDasharray="3 3" />
                  <XAxis dataKey="day" />
                  <YAxis />
                  <Tooltip />
                  <Line type="monotone" dataKey="mastery" stroke="#667eea" />
                </LineChart>
              </ResponsiveContainer>
            </CardContent>
          </Card>
        </Grid>
      </Grid>
    </Container>
  );
}

Step 6: Build and Test

Backend Testing:

bash

# Run unit tests
cd backend
python -m pytest tests/ -v

# Test API endpoints
python -m pytest tests/test_api_integration.py -v

Frontend Testing:

bash

# Run frontend tests
cd frontend
npm test

# Build production version
npm run build

Step 7: Docker Deployment

Docker Compose Configuration:

yaml

version: '3.8'
services:
  backend:
    build: ./backend
    ports:
      - "8000:8000"
    depends_on:
      - postgres
      - redis
    environment:
      DATABASE_URL: postgresql://postgres:postgres@postgres:5432/learning_paths
      REDIS_URL: redis://redis:6379

  frontend:
    build: ./frontend
    ports:
      - "5000:5000"
    depends_on:
      - backend

  postgres:
    image: postgres:15-alpine
    environment:
      POSTGRES_DB: learning_paths
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: postgres

  redis:
    image: redis:7-alpine

Start the system:

bash

# Build and run all services
docker-compose up --build

# Access the application
# Frontend: http://localhost:5000
# Backend API: http://localhost:8000
# API Docs: http://localhost:8000/docs

Quick demo:
http://localhost:5000/

Testing and Validation

Algorithm Testing

Test prerequisite ordering:

bash

curl -X POST http://localhost:8000/api/learning-paths/generate \
  -H "Content-Type: application/json" \
  -d '{"user_id": 1, "target_topics": [5, 3, 1, 4, 2]}'

# Should return topics in correct prerequisite order: [1, 2, 3, 4, 5]

Test difficulty progression:

bash

curl -X POST http://localhost:8000/api/learning-paths/generate \
  -H "Content-Type: application/json" \
  -d '{"user_id": 1, "target_topics": [1, 5, 9], "max_difficulty_jump": 1.0}'

# Should show gradual difficulty progression without large jumps

Performance Testing

Load test the API:

bash

# Install Apache Bench
sudo apt-get install apache2-utils

# Test 100 concurrent requests
ab -n 100 -c 10 -p test_data.json -T application/json \
   http://localhost:8000/api/learning-paths/generate

# Target: >50 requests/second, <100ms average response time

Integration Testing

Full user workflow test:

Create new user via API
Generate learning path through UI
Update progress via dashboard
Verify path recommendations update automatically
Check data persistence across sessions

Success Metrics

Path completion rates > 75%

Measure how many users complete their generated paths
Compare with traditional static curriculum approaches

Time-to-mastery reduction of 30%

Track how quickly users achieve topic mastery
Validate against control groups using fixed sequences

Algorithm response time < 100ms

Monitor API performance under load
Ensure real-time user experience

User engagement increase measured via session duration

Compare time spent learning before/after personalization
Track return visit frequency and platform stickiness

Assignment Challenge

Task: Extend the learning path generator to include collaborative filtering - recommend paths based on similar users' successful sequences.

Implementation Steps:

Cluster users by learning patterns using K-means
Analyze successful paths within each cluster
Weight recommendations by cluster similarity
Combine with existing prerequisite-based paths

Bonus: Implement path difficulty visualization showing progression curves and estimated completion times.

Solution Approach:

Use user progress data to identify learning velocity patterns
Apply clustering algorithms to group similar learners
Weight collaborative recommendations with individual preferences
A/B test collaborative vs. individual-only recommendations

This foundation prepares you for tomorrow's caching implementation, where we'll optimize these computationally expensive path calculations for production scale.

Discussion about this post

Ready for more?