Localization quality debates are often subjective, slow, and unresolvable, and they stall the very scaling efforts they claim to protect. Teams argue over whether a translation is "good enough" without agreeing on what "good" means for a given content type, audience, or business outcome. The result is inconsistent review cycles, missed deadlines, and quality standards that exist only in the minds of individual reviewers.
This deep dive replaces opinion-driven QA with a measurable, auditable framework. Drawing on the Multidimensional Quality Metrics (MQM) model and Dynamic Quality Framework (DQF), it covers how to tier content, define target quality levels, build sampling plans, layer automated and human checks, handle AI-generated output fairly, and ultimately connect localization quality to business results. The goal: an LQA program with clear SLAs, dashboards, and corrective loops that scale.
Content Tiering and Target Quality Levels
Not all content deserves the same investment in quality assurance. A legal disclaimer for a regulated market demands near-zero tolerance for error, while an internal knowledge base article may tolerate minor stylistic inconsistencies. Treating everything equally either inflates costs or leaves high-risk content under-reviewed.
How to Set Thresholds Per Content Type
Start by classifying content into three to four tiers based on risk and visibility:
| Tier | Content Examples | Quality Target | Typical Error Tolerance |
|---|---|---|---|
| 1, Critical | Legal contracts, regulatory filings, safety labels | Near-zero defects | 0-1 minor errors per 1,000 words; no major or critical errors |
| 2, High | Marketing copy, product UI strings, customer-facing help | Publication-ready | ≤5 minor errors per 1,000 words; no critical errors |
| 3, Standard | Support articles, FAQs, community content | Fit for purpose | ≤10 minor errors per 1,000 words; no critical errors |
| 4, Utility | Internal docs, training drafts, gisting output | Comprehensible | Higher tolerance; reviewed only on escalation |
Thresholds should be expressed numerically using MQM severity weights. A common weighting scheme assigns 1 point to minor errors, 5 points to major errors, and 10 points to critical errors. A composite quality score is then calculated per thousand words, and the pass/fail gate is set per tier. For example, a Tier 1 asset might require a score below 5 per thousand words, while Tier 3 might allow up to 25.
The key discipline is documenting these thresholds before production begins and aligning all stakeholders, product managers, linguists, and vendor partners, on the same rubric. Without this alignment, every review cycle risks becoming an ad hoc negotiation.
Sampling Plans for Scalable QA
Full review of every translated word is neither economically viable nor necessary. Statistical sampling lets you make confident quality judgments without reviewing everything.
Designing a Sampling Strategy
A well-designed sampling plan answers three questions: how much content to sample, how to select it, and how often to adjust.
- Sample size. For high-volume projects, reviewing a random sample of 1,500-2,000 words per language per delivery batch provides a statistically meaningful snapshot. For smaller batches, a minimum of 10-15% of total word count is a practical floor.
- Selection method. Use stratified random sampling. Ensure the sample includes content from each tier, each translator or MT engine, and each subject domain represented in the batch. Pure random sampling can miss entire content types.
- Frequency. Sample every batch during onboarding of a new vendor or language. Once a vendor demonstrates consistent quality over three to five consecutive batches, shift to periodic sampling (e.g., every third delivery) with spot checks triggered by user complaints or metric anomalies.
Adjust sampling intensity dynamically. If a language pair starts failing gates, increase sampling back to 100% until root cause is resolved. This risk-based approach keeps QA effort proportional to actual quality risk.
Automated Checks: Terminology, Placeholders, and Locale Formats
Automation handles the categories of error that are unambiguous, high-frequency, and tedious for humans to catch consistently. These checks should run before any human reviewer sees the content.
What to Automate
- Terminology consistency. Validate translated terms against approved glossaries and termbases. Flag any deviation from the canonical translation of product names, feature labels, and regulated terms.
- Placeholder and variable integrity. Confirm that all placeholders (e.g., {username}, %d, {{count}}, ICU MessageFormat tokens like {count, plural, one {...} other {...}}) present in the source appear correctly in the target, with no missing, added, or corrupted variables.
- Locale-specific formatting. Check date formats, number separators, currency symbols, measurement units, and address structures against locale conventions. A price rendered as "$1,000.00" in a German locale (where "1.000,00 €" is expected) is a functional defect.
- Tag and markup validation. For HTML, XML, or XLIFF content, verify that tags are balanced, correctly nested, and not splitting translatable text in ways that break rendering.
- Length and truncation. Flag translations that exceed UI character limits or deviate significantly from source length ratios typical for the language pair.
- Spell check and basic grammar. Run language-specific spell checkers to catch typos that human reviewers might skim past.
These automated checks can catch a substantial share of all measurable defects in localized content, according to analyses by organizations like TAUS. Critically, they catch errors at machine speed, before human review begins, which means linguists spend their time on nuance rather than hunting for broken variables.
Linguistic Review Workflows
Automated checks handle the mechanical. Linguistic review handles meaning, tone, cultural fit, and the subtle errors that only a skilled human can evaluate.
Structuring the Review Loop
A lean but effective linguistic review workflow typically follows this sequence:
- Automated pre-check. All automated validations run. Content with critical automated failures is returned to the translator or post-editor before human review.
- Primary review. A qualified reviewer evaluates the sampled content against the MQM error typology, annotating each error with its category (accuracy, fluency, terminology, style, locale convention) and severity (minor, major, critical).
- Adjudication (when needed). If the translator disputes an error annotation, a second senior reviewer adjudicates. This step is essential for maintaining reviewer calibration and preventing subjective drift.
- Scoring and gate decision. The composite MQM score is calculated. If the batch passes the tier-specific threshold, it proceeds to release. If it fails, it enters a corrective loop.
- Feedback delivery. Error annotations, with explanations, are returned to the translator or MT post-editor within a defined SLA, ideally within 48 hours of delivery. Delayed feedback loses its corrective value.
The entire loop should be tracked in a centralized platform that records scores over time per language, vendor, content type, and error category. This data feeds the dashboards and SLAs that make the LQA program auditable.
For organizations managing complex multilingual workflows across content types, platforms like Ollang consolidate automated checks, review tracking, and scoring into a single execution layer. Ollang integrates with common TMSs and translation APIs to automate gates and surface dashboards alongside your existing workflows. You can book a demo with Ollang to see how this integrates with existing translation management systems: https://ollang.com/book-a-demo
Bias-Aware Evaluation for AI MT and Post-Editing
Machine translation output, whether from neural MT engines or large language models, introduces specific quality patterns that traditional QA rubrics may not fully address. Evaluating MT plus post-editing (MT+PE) output requires adapting both the error typology and the reviewer's mindset.
Addressing MT-Specific Quality Risks
MT engines can produce output that is fluent but inaccurate, a phenomenon sometimes called "fluent nonsense." A sentence may read naturally in the target language while subtly distorting the source meaning. Reviewers trained on human translation may be lulled by surface fluency and miss semantic errors underneath.
To counter this:
- Weight accuracy errors more heavily in MT+PE evaluation. Fluency errors in MT output are often minor and easily corrected; accuracy errors are where business risk concentrates.
- Evaluate for hallucinations. MT engines occasionally generate content not present in the source. Flag any added information as a critical accuracy error.
- Check for bias propagation. MT systems can amplify gender, cultural, or demographic biases present in training data. For example, gendered languages may default to masculine forms for professional titles. Include bias-related categories in the MQM typology and train reviewers to flag them.
- Calibrate reviewers on MT output specifically. Reviewers who primarily evaluate human translation may apply different (often harsher) standards to MT+PE output. Run calibration sessions where multiple reviewers score the same MT+PE sample, then align on consistent application of severity levels.
The goal is not to penalize MT for being MT, but to ensure the evaluation framework captures the error types that MT is most likely to produce.