Shipping translations at scale without a quality gate is a liability. Automated machine translation can produce fluent-sounding output that silently distorts meaning, drops legal nuances, or violates brand tone, problems that only surface after customers complain or regulators notice. The solution is not to slow everything down with manual review of every segment. Instead, modern localization teams build API-driven quality assurance pipelines that score translations automatically, flag risky segments, and route only the ones that need human judgment to qualified reviewers. This article walks through the architecture, endpoints, metrics, sampling strategies, and platform capabilities you need to build a measurable, auditable LQA pipeline. If your localization workflow already struggles with inconsistent quality or slow feedback loops, see how Ollang's review layer fits into your API pipeline.
How Translation Quality Review APIs Work
Translation quality review APIs expose the internal state of translated segments, draft, reviewed, approved, rejected, along with metadata like reviewer comments, error annotations, and confidence scores. A typical translation management system (TMS) provides REST endpoints that let you fetch segments by status, post corrections, attach annotations following error typologies like MQM, and transition segments through a review state machine.
The core workflow looks like this:
- Fetch segments in a specific state (e.g., translated or needs_review) from the TMS.
- Run automated quality metrics against each segment.
- Gate or route based on scores: auto-approve high-confidence segments, queue low-confidence ones for human review.
- Post corrections and annotations back to the TMS via API.
- Sync approved translations into translation memory for future leverage.
This loop runs continuously, triggered by webhooks when new translations land, or on a schedule for batch jobs. The API is the connective tissue, it replaces manual export-review-reimport cycles with programmatic, auditable state transitions.
Segment-Level Review Endpoints
Most TMS platforms expose segment-level review through endpoints that accept a project ID, file or job ID, and locale. A typical GET request to fetch segments pending review returns a paginated list of source-target pairs with metadata:
{
"segments": [
{
"id": "seg_4821",
"source": "Your account has been suspended.",
"target": "Votre compte a été suspendu.",
"status": "translated",
"confidence": 0.91,
"translator": "mt_engine_v3",
"created_at": "2025-01-15T08:32:00Z"
}
],
"pagination": {
"next_cursor": "eyJwYWdlIjoy",
"total": 347
}
}
You filter by status to pull only segments that need attention. After review, you transition the segment by posting a status update:
{
"segment_id": "seg_4821",
"status": "approved",
"reviewer": "reviewer_jdoe",
"comment": "Accurate and tone-appropriate.",
"annotations": []
}
When a correction is needed, the payload includes the revised target and an annotation referencing an error category:
{
"segment_id": "seg_4821",
"status": "corrected",
"revised_target": "Votre compte a été suspendu temporairement.",
"reviewer": "reviewer_jdoe",
"annotations": [
{
"type": "accuracy",
"severity": "major",
"note": "Original omitted 'temporarily' from source context."
}
]
}
These annotations follow the MQM error typology, which categorizes issues by dimension (accuracy, fluency, terminology, style) and severity (critical, major, minor). Structured annotations make quality data analyzable at scale, you can aggregate error rates by language pair, MT engine, or content type.
States, Comments, and Annotations
The segment lifecycle in a review-enabled TMS typically follows a state machine:
| State | Meaning | Transition Trigger |
|---|---|---|
| draft | Initial MT or TM output | Automatic on translation |
| translated | Ready for QA | Translator marks complete |
| needs_review | Flagged by automated QA | Score below threshold |
| in_review | Assigned to human reviewer | Reviewer picks up task |
| corrected | Reviewer edited the target | Reviewer submits edit |
| approved | Passes quality gate | Reviewer or auto-approve |
| rejected | Fails quality gate entirely | Reviewer flags for retranslation |
Comments and annotations attach to individual segments and persist as part of the audit trail. Good API design lets you query annotations across a project to surface systemic issues, for example, pulling all terminology errors for a given glossary term to determine whether the glossary entry itself needs updating.
Integrating Automated Quality Metrics via API
Manual review alone does not scale. Even with efficient human reviewers, evaluating every segment in a high-volume pipeline is cost-prohibitive. Automated quality estimation metrics let you pre-screen translations and focus human effort where it matters.
COMET, BLEURT, and TER Scoring Jobs
Three families of metrics dominate modern translation quality estimation:
- COMET (Crosslingual Optimized Metric for Evaluation of Translation) is a neural metric trained on human quality judgments. It takes source, MT output, and optionally a reference translation, producing a score that correlates strongly with human assessments. Research from Unbabel's COMET paper shows it outperforms surface-level metrics on segment-level correlation with human scores.
- BLEURT is Google's learned evaluation metric, also trained on human ratings. It handles paraphrases and meaning-preserving variations better than traditional n-gram metrics.
- TER (Translation Edit Rate) measures the number of edits needed to transform MT output into a reference translation. It is simpler and faster to compute but less nuanced, useful as a secondary signal or for legacy comparison.
You can run these as scoring jobs in your pipeline. A typical integration pattern uses an async job API:
curl -X POST https://qe-service.internal/v1/score \
-H "Authorization: Bearer $QE_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"segments": [
{
"id": "seg_4821",
"source": "Your account has been suspended.",
"target": "Votre compte a été suspendu.",
"reference": null
}
],
"metrics": ["comet", "bleurt"],
"callback_url": "https://your-app.com/webhooks/qe-results"
}'
The service processes segments asynchronously and posts results to your callback URL. Reference-free scoring (using only source and MT output) is essential for production pipelines where reference translations do not exist yet, that is the whole point of quality estimation.
Running Metrics as Async Batch Jobs
For high-volume projects, you batch segments into scoring jobs rather than scoring one at a time. Batching reduces API overhead and lets the scoring service optimize GPU utilization for neural metrics like COMET.
A practical batch workflow:
- Accumulate segments from TMS webhook events into a queue (e.g., SQS, RabbitMQ, or a Redis list).
- Every N minutes or when the queue reaches a size threshold, dequeue segments and submit a batch scoring request.
- The scoring service returns a job ID immediately. Poll for completion or register a webhook.
- On completion, iterate through scored segments and apply gating logic.
Batch sizes of 100-500 segments typically balance latency against throughput for neural metrics. Simpler metrics like TER can run synchronously in-process without a separate service.
Sampling Strategies for Scalable Review
Not every segment needs human eyes. Effective sampling strategies reduce review volume while maintaining statistical confidence in quality estimates.
Statistical Sampling and Risk-Based Selection
Two complementary approaches work well together:
- Statistical sampling selects a random subset of segments from each job or locale, sized to achieve a target confidence interval. For a project with 5,000 segments, reviewing a random sample of 200-300 gives you a statistically meaningful quality estimate at a 95% confidence level with roughly a ±5% margin of error. You can calculate exact sample sizes using standard formulas or tools like the ISO 2859 sampling tables adapted for translation.
- Risk-based selection oversamples segments that are inherently riskier:
- Legal, medical, or safety-critical content
- Segments with low automated QE scores
- New or rare language pairs where MT quality is less predictable
- Content containing numbers, dates, or proper nouns (high-consequence errors)
- Segments that did not match any translation memory entry
Combining both approaches means your random sample catches systemic issues while risk-based selection catches the segments most likely to contain critical errors.
Threshold-Based Gating Logic
Threshold gating automates the approve/route/reject decision. You define score thresholds that map to actions:
| COMET Score Range | Action | Destination |
|---|---|---|
| ≥ 0.90 | Auto-approve | TMS → approved state |
| 0.75, 0.89 | Route for review | Human review queue |
| < 0.75 | Flag for retranslation | Retranslation queue |
These thresholds are not universal, they should be calibrated per language pair and content type. A COMET score of 0.85 for German marketing copy might be acceptable, while the same score for Japanese legal text might warrant review. Calibration involves correlating automated scores with human quality judgments on a representative sample, then adjusting thresholds until the false-positive and false-negative rates are acceptable for your risk tolerance.
In code, the gating logic is straightforward:
def gate_segment(segment, score):
if score >= THRESHOLD_APPROVE:
tms_client.update_status(segment["id"], "approved")
elif score >= THRESHOLD_REVIEW:
review_queue.enqueue(segment["id"], priority=1 - score)
else:
retranslation_queue.enqueue(segment["id"])
Priority-based queuing ensures the worst-scoring segments get human attention first.