Most localization teams discover quality problems after content ships. A reviewer flags an error in production, a support ticket arrives in a language no one on the team reads, or a regulatory audit surfaces inconsistent terminology across product versions. By then the cost of remediation has multiplied. The solution is to treat translation quality assurance as an automated, API-driven pipeline, one that runs alongside your existing machine translation and TMS workflows, not after them. This article maps the end-to-end architecture: from machine pre-translation and LQA task creation, through automated scoring with models like COMET and BLEURT, to CI-integrated acceptance gates that decide whether a segment ships or routes to a human reviewer. The goal is a measurable, audit-ready QA system you can stand up against any MT provider.
If your localization pipeline lacks programmatic quality controls, explore how Ollang’s API layer can close that gap.
Why Translation Quality Needs API Automation
Manual quality assurance does not scale. When you localize into dozens of languages across continuous release cycles, spreadsheet-based reviews create bottlenecks, introduce subjectivity, and leave no structured audit trail. API automation addresses each of these problems directly.
First, it makes quality measurement consistent. Every segment passes through the same scoring models, the same terminology checks, and the same threshold logic, regardless of which reviewer happens to be available. Second, it makes quality measurable over time. Structured scores stored per segment and per release let you track drift, compare MT engine performance, and justify investment in human post-editing where it actually matters. Third, it connects quality enforcement to your deployment pipeline. A failing quality gate in CI can block a release just as reliably as a failing unit test.
The economics are straightforward, too. Quality estimation API calls, whether to reference-based models or referenceless estimators, cost a fraction of full human review. The goal is not to eliminate human reviewers but to focus their time on the segments that genuinely need attention.
End-to-End QA Pipeline Architecture
A robust API-driven QA pipeline has five stages: machine pre-translation, LQA task creation, automated scoring, acceptance gating, and drift monitoring. Each stage communicates via API calls and webhooks, producing structured data that feeds the next.
Machine Pre-Translation and Segment Extraction
The pipeline begins when source content enters your TMS or content repository. An API call to your MT provider translates each segment, returning the target text along with metadata such as confidence scores, model version, and character counts.
A typical request to an MT endpoint looks like this:
{
"source_lang": "en",
"target_lang": "de",
"segments": [
{"id": "seg_001", "text": "Your subscription renews automatically."},
{"id": "seg_002", "text": "Cancel anytime from Settings."}
],
"glossary_id": "gloss_product_v3",
"model": "latest"
}
The response includes translated segments and any engine-reported quality signals. Store both source and target alongside the segment ID, you will need the pair for every downstream QA step.
At this stage, also extract and preserve any inline markup, placeholders, or ICU MessageFormat tokens. Placeholder corruption is one of the most common MT failure modes, and catching it early avoids expensive downstream debugging.
LQA Task Creation via TMS API
Once segments are pre-translated, create Linguistic Quality Assurance tasks programmatically. Most TMS platforms expose REST endpoints for task or job creation. The API call should specify the project, language pair, workflow step (e.g., lqa_review), assigned reviewer pool, priority, and due date.
{
"project_id": "proj_4821",
"task_type": "lqa_review",
"language_pair": "en-de",
"segments": ["seg_001", "seg_002"],
"priority": "high",
"due_date": "2025-07-20T12:00:00Z",
"mqm_schema": "mqm_v2",
"instructions": "Flag accuracy, fluency, and terminology issues per MQM."
}
The key detail here is the mqm_schema field. By specifying the MQM (Multidimensional Quality Metrics) schema at task creation time, you ensure reviewers annotate errors using a consistent taxonomy. This structured annotation feeds directly into your scoring and reporting layer.
Webhook-Driven Review Workflow
Rather than polling the TMS for task completion, register a webhook endpoint that receives events when reviewers submit annotations. A well-designed webhook payload includes the segment ID, the reviewer's MQM annotations, severity ratings, and a timestamp.
{
"event": "lqa_review_complete",
"task_id": "task_9930",
"segment_id": "seg_001",
"annotations": [
{
"category": "accuracy/mistranslation",
"severity": "major",
"comment": "Subscription translated as membership"
}
],
"reviewer_id": "rev_42",
"completed_at": "2025-07-18T14:32:00Z"
}
Your webhook handler should validate the payload, persist the annotations to your QA database, and trigger the next pipeline stage, automated scoring, for the affected segments.
Automated Quality Scoring via API
Human LQA annotations are essential but slow. Automated quality estimation models let you score every segment at machine speed, reserving human review for the segments that fall below threshold.
Quality Estimation with COMET and BLEURT
COMET and BLEURT are learned evaluation metrics that correlate more strongly with human judgments than traditional string-overlap metrics like BLEU. Both are available as self-hosted models or through API wrappers.
A quality estimation request typically takes a source segment, a machine translation, and optionally a reference translation:
{
"source": "Your subscription renews automatically.",
"translation": "Ihr Abonnement verlängert sich automatisch.",
"reference": "Ihr Abonnement wird automatisch verlängert.",
"model": "comet-22"
}
The response returns a score, COMET scores typically range from 0 to 1, with values above 0.85 generally indicating acceptable quality for most content types. BLEURT returns a similar continuous score.
For referenceless estimation (when no human reference exists), models like COMET-QE evaluate the source-translation pair alone. This is the more practical mode for continuous pipelines, since maintaining reference translations for every segment defeats the purpose of automation.
Store scores per segment alongside the model version and timestamp. You will need this data for threshold gating and drift analysis.
Back-Translation for Accuracy Verification
Back-translation, translating the target text back into the source language and comparing the result to the original, remains a useful heuristic, especially for high-stakes content such as legal or medical text.
The API flow is simple: send the target segment to your MT provider with reversed language direction, then compute a similarity score between the back-translation and the original source. Cosine similarity on sentence embeddings (from models like LaBSE or multilingual E5) outperforms string-level comparison for this purpose.
back_translation = mt_api.translate(source="de", target="en", text=translated_segment)
similarity = embedding_model.cosine_similarity(original_source, back_translation)
A similarity score below your threshold (commonly 0.80 for general content, 0.90 for regulated content) flags the segment for human review.
Back-translation is not infallible, symmetric errors can survive the round trip, but combined with quality estimation scores, it provides a valuable second signal.
LLM-Based Judges and Hallucination Detection
Large language models can serve as automated quality judges, evaluating translations against criteria you define in a prompt. This approach is particularly effective for detecting hallucinations, content in the translation that has no basis in the source, and for checking fluency in context.
A typical LLM judge prompt might instruct the model to evaluate a source-translation pair across accuracy, fluency, and completeness, returning structured JSON with per-dimension scores and explanations. The advantage over learned metrics is flexibility: you can add domain-specific criteria (tone, formality register, brand voice) without retraining a model.
The risk is cost and latency. LLM judge calls are substantially more expensive than COMET or BLEURT inference. Use them selectively, on segments that fall into an ambiguous score range, on high-visibility content, or as a tiebreaker when quality estimation and back-translation disagree.
For hallucination detection specifically, compare named entities, numbers, and dates between source and translation. An LLM judge can also be prompted to identify added information, claims or instructions present in the translation but absent from the source. In regulated domains, hallucinated content is not just a quality issue; it is a compliance risk.