Final Capstone: Build a Production NLP Intelligence System
What We’re Building Today
A multi-capability NLP Intelligence API that classifies intent, scores sentiment, and generates contextual responses — all served through a single REST endpoint.
A modular PyTorch-based pipeline where each NLP component is independently testable and swappable without touching the rest of the system.
A production-style deployment with Docker, a health-check endpoint, structured logging, and a minimal web demo interface.
Why This Matters
You’ve spent 176 days learning how individual pieces work — tensors, layers, training loops, embeddings, sequence models. What separates a student from a practitioner is the ability to compose those pieces into a system that solves a real problem end-to-end. Every major NLP product at companies like OpenAI, Cohere, or Hugging Face is not one model — it’s a pipeline of coordinated components: input normalization, classification, generation, post-processing, and serving. This capstone is your first time building that pipeline yourself, which means by the end of Day 180, you’ll have produced something you can legitimately show in a job interview or deploy for real users.
Core Concepts
1. The NLP Pipeline as a System, Not a Script
A pipeline is a directed graph of transformations. Each node takes in a well-defined input and produces a well-defined output. The moment you start building pipelines this way, your code becomes composable. You can swap a rule-based intent classifier for a neural one without touching the sentiment module. You can add a new component — say, entity extraction — without modifying the response generator.
In production systems at Stripe or Shopify, the ML inference path looks exactly like this: an API gateway hands the raw text to a normalization service, which passes the cleaned text to a classification model, which produces a structured prediction object, which routes to a generation or retrieval system, which formats the final response. Each handoff is a typed contract. Today you’re building that exact pattern.
2. Intent Classification with Lightweight Transformers
The dominant pattern for intent classification in production is fine-tuning a pre-trained transformer on a small labeled dataset. DistilBERT — a compressed version of BERT — gives you 97% of the accuracy at 40% of the size. This is the same trade-off that powers intent routing in voice assistants at Amazon and Google.
The key insight here is that the pre-trained model already understands language at a deep level. Your fine-tuning job is not to teach it language — it’s to teach it your categories. When you have only a few hundred labeled examples, this transfer learning approach outperforms training from scratch by a wide margin. Your final classifier is really just a linear layer sitting on top of a frozen or lightly-unfrozen transformer backbone.
3. Sentiment-Aware Response Generation
The response generation in this system is a hybrid: a retrieval layer picks a candidate response template based on intent and sentiment score, and a neural reranker scores each candidate against the original query context. This avoids the hallucination risk of pure generation while still producing contextually relevant responses.
This pattern — retrieval-augmented generation with a reranker — is exactly what systems like Perplexity AI and Bing Chat use at scale. The reranker is a cross-encoder transformer that takes (query, candidate) pairs as input and outputs a relevance score. You’re not building the full RAG stack today, but the architecture you implement mirrors the production pattern precisely.
4. API Design for ML Systems
Serving an ML model is fundamentally different from serving a database. Model inference has unpredictable latency, GPU resource contention, and serialization overhead. Three patterns matter most:
Request batching: Group multiple incoming requests together before sending to the model. A batch of 32 requests takes only 1.5× the time of a single request on a GPU, so throughput multiplies dramatically. Your system implements a simple synchronous batch endpoint.
Response caching: Identical or near-identical queries should return cached predictions. A simple Redis-backed LRU cache cuts GPU load significantly in production. The architecture includes a cache layer even if you implement it in-memory for the capstone.
Structured error envelopes: Every response — success or failure — should follow the same JSON schema. Clients must be able to rely on the response structure regardless of what happened internally. Your API will always return
{status, data, error, latency_ms}.
5. Component Architecture
The preprocessor runs first and its output is shared by both the intent classifier and the sentiment scorer — they run in parallel, then their outputs converge at the response selector. This parallel fan-out pattern keeps latency bounded to max(t_intent, t_sentiment) rather than t_intent + t_sentiment.
NLP Intelligence API
Input Text
|
v
[Preprocessor] --> normalize, tokenize, truncate
|
+-----------------------------+
v v
[Intent Classifier] [Sentiment Scorer]
(DistilBERT + linear) (LSTM / Day 176)
| |
+-------------+---------------+
v
[Response Selector]
(retrieve + rerank)
|
v
[JSON Response]
