12 Questions Every CTO Should Ask Before Buying an "AI Execution Layer" for Localization

Every vendor pitch deck in this category now uses the same three words: execution layer, orchestration, governance. The words are free. What separates a real infrastructure provider from a repackaged translation agency with a chatbot bolted on is whether you can point an API client at the system and watch it do the thing the deck describes, today, in a sandbox, before a contract exists.

This is not a comparison of feature checklists. It's a procurement instrument: twelve questions you can answer with a terminal window, not a follow-up call. If a vendor can't answer most of these with a working request-response cycle, you're not buying infrastructure. You're buying a services relationship with an API skin on top.

1. Can I get a health check with no authentication at all?

Before you hand over a key, you should be able to confirm the service is reachable. A vendor that makes you authenticate just to find out if their system is up is telling you something about how they think about operational transparency. Ollang's documentation exposes a health endpoint where a 200 with {"status":"OK"} confirms connectivity, and it notes that no API key is required to call it. Run this from your own network, not the vendor's demo environment, so you test your actual path to their infrastructure.

2. Can I authenticate, upload a file, and create an order without a sales call?

Ask for API key generation self-service, then run the upload-to-order sequence yourself. Ollang's model is straightforward: API-key authenticated requests against upload, order, and project endpoints, with the key passed as a header on every call. If getting a key requires a provisioning meeting, that's a signal the "self-serve API" is marketing language for a manual onboarding process wearing an API's clothes.

3. Does the order object tell me what's actually happening, or just "processing"?

Pull an order by ID mid-flight. You want status, language pairs, document references, and, critically, QC results attached to the same object, not scattered across a separate portal you have to log into by hand.

4. Can I trigger a standalone QC evaluation and get a structured score back?

This is the question that separates infrastructure from a black box. Ask the vendor to run quality control on a completed order through the API, not a PDF report emailed by a project manager three days later. Ollang's QC evaluation endpoint triggers an AI-powered assessment of the translation quality, evaluating criteria such as accuracy, fluency, tone, and cultural fit, and runs asynchronously with notification on completion. The call is scoped to work you've already received, it requires the order ID for which you want to run the evaluation, and the order must be in a completed or delivered state to be eligible. The response isn't a prose summary, it's a structured payload with an evaluation ID and processing status you can branch logic on programmatically. Run it on a real order in your sandbox and confirm you get back scores, not sentiment.

5. Can I scope the QC evaluation to what my compliance function actually cares about?

Not every workflow needs every dimension scored. A legal disclosure translation cares about accuracy far more than tone. Check whether you can toggle evaluation criteria per call rather than getting a fixed bundle. Ollang's QC request accepts individual boolean flags for accuracy, fluency, tone, and cultural fit, plus an optional custom prompt to guide the QC evaluation, which matters if your review criteria are domain-specific rather than generic.

6. What does the QC engine actually check per modality, and is it consistent?

Subtitles, dubbed audio, and structured JSON strings fail in different ways. Ask whether quality evaluation is a single generic score or whether it accounts for modality-specific failure modes, such as subtitle timing, dubbing sync, and schema integrity. A vendor should be able to describe a shared evaluation baseline with modality-specific validators layered on top, not one blunt instrument applied everywhere.

7. When I register a webhook, what happens if my endpoint is slow?

This is where "we support webhooks" claims usually collapse under actual load testing. Ask for the documented callback timeout in writing. Ollang's own troubleshooting guidance is explicit: callbacks have a 10-second timeout, and if your handler takes longer to respond, it's treated as a failure, so acknowledge quickly and process asynchronously. If a vendor can't give you a number here, assume their callback infrastructure hasn't been tested against your kind of downstream latency, and design your integration to acknowledge-then-process regardless.

8. What happens to the order if my callback fails?

A callback failure should never corrupt or stall the underlying job. Confirm this explicitly: callback failures do not affect the order itself. Then ask what your fallback path is. Ollang's documented answer is to fall back to polling Get Order by ID if you don't receive a callback within a reasonable window, which tells you the vendor expects webhooks to be the fast path, not the only path, and has built the polling endpoint to be reliable enough to serve as a backstop.

9. Are your callback handlers required to be idempotent, and why?

If the answer is "we guarantee exactly-once delivery," be skeptical, very few distributed systems actually do. The more credible answer treats duplication as expected and puts the burden of dedup on a documented key. Ollang's guidance is to treat callback handlers as idempotent, using the orderId plus completedAt as a dedup key, or upserting into your store keyed by orderId. It also recommends not trusting the callback body alone, pairing the callback with a GET order confirmation using your API key before acting on the contents. That's the kind of operational detail you only get from a vendor that has actually run this in production, not one reciting a webhook tutorial.

10. Can I add a human review gate to a single order without changing my plan tier?

This is the governance test. Many vendors force a binary choice: fully automated tier, or a premium "managed" tier where everything routes through humans regardless of need. Ask whether human review is an order-level decision. Ollang exposes this as a discrete API call, a request-human-review endpoint that triggers a manual review by a professional linguist on an existing order, used when you want a professional linguist to manually review the delivered content for quality assurance or compliance reasons. The platform frames this as a Level 1 review gate you can add to any order to route output to Ollang-managed linguists or your own LSPs and editors, your choice of reviewer, your choice of which order gets escalated, decided per call rather than baked into a contract tier.

11. Can I cancel a human review request if I change my mind mid-flow?

If human review is truly order-level and reversible, there should be a cancel path, not just a request path. Confirm both directions exist as separate, callable operations before you assume the review gate is flexible rather than one-way.

12. What are your documented rate limits and retry expectations, in writing, before launch?

This is the question most CTOs ask too late, after a product launch spikes volume and orders start failing silently. Don't accept "we'll scale with you" as an answer. Ask for the written throughput ceiling, the expected behavior on a 429, and whether the vendor's own documentation treats rate limiting as a first-class troubleshooting category. Ollang's troubleshooting reference groups authentication, uploads, callbacks, order state, and rate limits as the standard integration failure buckets, meaning rate limits are treated as an expected, documented condition rather than an edge case discovered in production. Get the numbers in writing and load-test against them in your sandbox before your launch date, not after.

What this checklist doesn't cover, and why that's the point

You'll notice nothing here asks about language coverage, pricing tiers, or roadmap commitments. Those are real considerations, but they're negotiable and vendor-narrated by nature. What isn't negotiable is whether the system does what the sales deck says when you're the one holding the API client. A vendor that welcomes this kind of adversarial testing, that has a documented reference detailed enough for you to run these twelve checks without a support ticket, shows how they built the product, for machines to call, not just for humans to click through a demo.

The category is shifting from localization-as-project to localization-as-infrastructure. Infrastructure gets evaluated the way you evaluate a database or a message queue, by its documented failure modes, its timeout behavior, its idempotency guarantees, not by how confidently someone says "orchestration" in a meeting. Run these twelve checks before you sign anything. If a vendor can't survive them, they're not selling you an execution layer. They're selling you a services contract with better slide design.