The first production decision is the shape of the answer
A model that understands a document is not yet a production system. You must decide what the system should return: one label for the whole document, a label for every token, a syntactic link between words, or a structured fact assembled from several spans.
That output shape determines the model head, the annotation scheme, and the error metric. Sentiment can be one sequence label. Named entities require token or span boundaries. “Apple acquired Beats” becomes useful to a database only when the system returns entities plus an acquired relationship.
Modern transformers support all of these tasks, but the strongest architecture is not always the right deployment choice. A fast SpaCy pipeline can process more than 10,000 sentences per second on CPU when entities are well defined, while a transformer-backed model costs more and handles harder variation.
Tradeoff: a general model reduces task-specific engineering, but a narrow output head is easier to validate and often cheaper to serve. Start from the required artifact, not from the model you want to use.
Let labeled data determine the starting point
For binary or multiclass classification, a common transformer design places a linear head on the encoder’s CLS representation and trains with cross-entropy. For multilabel classification, labels are independent: use sigmoid outputs with binary cross-entropy, then tune a threshold for each class on validation data instead of assuming 0.5 is best for every label.
The data regime changes the recommended method. With more than about 1,000 examples per class, fully fine-tuning RoBERTa is a strong choice. With 8 to 100 examples per class, SetFit creates positive and negative contrastive pairs, adapts a sentence transformer, then trains logistic regression on the embeddings.
With no labeled examples, use an NLI model to test whether a document entails a label description, or prompt a decoder model with category definitions and examples. If interpretability or very fast inference matters most, TF-IDF plus logistic regression remains a sound baseline.
Tradeoff: zero-shot methods remove labeling work but can be sensitive to label wording. Full fine-tuning uses the task data most directly but needs enough examples. SetFit occupies the useful middle ground for 8–64 labeled examples per class.
Entity extraction is a boundary problem before it is a naming problem
Named Entity Recognition assigns a type such as person, organization, or location to spans in text. A token classifier can mark “John Smith visited Paris” as B-PER, I-PER, O, B-LOC. The B and I markers preserve the start and continuation of each entity instead of emitting disconnected token classes.
BIOES adds explicit single-token and end markers. Those extra states make boundaries clearer, especially for long entities, and the notes report a typical improvement of 0.5 to 1 F1 over IOB2 on span extraction.
A linear head predicts each token independently. A Conditional Random Field adds transition scores and uses Viterbi decoding to choose a globally consistent tag sequence, preventing an I-PER tag from following an incompatible B-ORG. The gain is modest on large encoders but more meaningful on smaller models.
Tradeoff: a CRF adds structural consistency and decoding complexity for roughly a 0.5 F1 gain on modern large encoders. Use it when invalid tag transitions matter or the encoder is small; otherwise a linear token head may be enough.
Some language structure does not fit one label per token
Span-based extraction scores candidate start-and-end ranges instead of forcing each token into one tag. This naturally represents nested entities, such as overlapping biological terms in “protein in cell nucleus,” which IOB cannot express cleanly.
Dependency parsing asks a different question: which token is the syntactic head of which other token? Biaffine attention parsers paired with transformer encoders produce dependency trees used by relation extraction, question answering, and grammatical analysis. Part-of-speech tags can support this analysis, though POS tagging is rarely a standalone product task now.
Coreference resolution links mentions that refer to the same entity, as in “John said he was tired.” It remains difficult because the answer can depend on distant context and competing spans. Transformer span representations followed by clustering are the dominant approach, with AllenNLP’s coreference model listed in the notes as a practical option.
Tradeoff: span enumeration supports nested entities but creates many candidate spans to score. Token tags are cheaper and simpler when entities never overlap.
Turn detected spans into facts you can validate
A classic relation-extraction pipeline first finds entities, then classifies candidate pairs. You can make the relationship explicit in the model input: “[E1] Apple [/E1] acquired [E2] Beats [/E2] for $3 billion.” An encoder then predicts a relation such as acquired for that pair.
Open information extraction removes the predefined relation list and returns subject, relation, object triples. A prompted language model can go further by filling a JSON schema, such as acquirer, target, price, and date for every acquisition event in a document.
In production, function calling or structured-output tools can enforce the schema and validate types. For a narrow, consistent document set, fine-tuning a smaller model such as Llama-3-8B with LoRA can beat repeated large-model prompting on cost, latency, and consistency.
Extraction is not ordinary generation. The system should return facts grounded in existing text, so its characteristic failures are false positives and bad span boundaries rather than merely awkward prose. Evaluate extraction with precision, recall, and F1 against annotated facts.
Tradeoff: pipelines expose entity and relation errors separately but let early NER mistakes propagate. Joint models can coordinate both predictions, while prompted schema extraction is quick to start but must be constrained against hallucinated facts.
Match the model to the artifact, data, and failure cost
Production NLP is a set of output contracts. Classification returns document labels. Sequence labeling and span models return grounded boundaries. Parsers return syntactic links. Coreference systems connect mentions. Relation and template extraction turn those pieces into typed facts.
Choose within each contract by evidence. Let labeled-data volume choose among full fine-tuning, SetFit, zero-shot methods, and classic baselines. Add CRF constraints only when consistent tag sequences justify the cost. Prefer span models for nested entities, and treat structured LLM output as extraction only when schema validation and grounding checks keep it tied to the source.
The remaining question is whether the result is good enough and cheap enough to run. In the final lesson, we will bring those concerns together: metrics must reflect the task’s real failures, and efficiency methods must preserve the quality those metrics are meant to protect.