Enterprise Roadmap & Known Limitations
What is coming next — semantic caching, provider failover, A/B model testing, prompt versioning — and one honest consolidated table of known limitations.
On this page
TL;DR: This page is the single honest home for what is not yet available. Four larger capabilities are coming — tenant-isolated semantic caching, automatic provider failover, A/B model testing, and prompt versioning with rollback — and a consolidated table lists every other known limitation with its current state. Everything not listed here is delivered and documented in the walkthrough pages.
Coming soon
Tenant-isolated semantic caching
What it will do: serve semantically similar requests from a cache instead of re-billing the provider, with strict isolation so one project's cached responses can never leak into another's.
Current state: no semantic cache exists in the gateway today. Building one needs embedding infrastructure, similarity thresholds, and a tenant-isolation proof — a project in its own right.
Honest note: not yet available; today every request goes to the provider (provider-side prompt caching still applies where the provider offers it).
Automatic provider failover
What it will do: retry a failed or unhealthy provider on a configured secondary automatically, so a provider outage degrades latency rather than availability.
Current state: no failover exists in the gateway dispatch path — a route resolves to exactly one provider, and a provider error is relayed to the caller. The design needs secondary-provider fields on routes plus health- and error-triggered retry in the dispatch core.
Honest note: not yet available; today failover means an operator switching a route's provider, which is a configuration change.
A/B model testing
What it will do: split live traffic on a route between models by percentage, with stable experiment assignment, so model changes can be evaluated on real usage before a full switch.
Current state: route resolution is strictly first-match; no percentage split or experiment-assignment machinery exists.
Honest note: not yet available; comparisons today are done by switching routes for a cohort (for example, one project) and comparing the cost and usage reports.
Prompt versioning & rollback
What it will do: a first-class prompt-template registry with versions, parameters, pinning, and one-click rollback, distributed through the same signed catalog as skills and plugins.
Current state: no prompt registry exists; the nearest primitive (system-prompt overrides) keeps no version history. The signed distribution channel described in Tool Governance can carry prompt content, so the transport exists — the lifecycle object does not.
Honest note: not yet available; prompt content today is versioned the way the rest of your configuration is — in git.
OpenAI-compatible IDE endpoint
What it will do: a /openai/v1/chat/completions endpoint so OpenCode and VS Code Copilot route IDE traffic through platform governance to approved models (including GCP Vertex AI), with per-role model access and monthly budgets.
Current state: the gateway already speaks the OpenAI wire on its existing endpoints; the dedicated OpenAI-path endpoint, the Vertex AI provider, and per-role monthly budget policies are the build items.
Known limitations
| Limitation | Current state | Where it is discussed |
|---|---|---|
| SCIM provisioning | Deliberately deferred: neither AD FS (as IdP) nor Odoo pushes standards-based SCIM, so an endpoint would have no caller. Revisit if Okta or Entra fronts the instance. | User & Access Management |
| WORM audit immutability | Audit rows are append-only by convention, not mechanism — no WORM storage or hash-chaining. Planned hardening options: revoked UPDATE/DELETE grants, hash-chaining, or export to WORM storage. | Audit & Observability |
| Observability push egress | OTLP flows in and SSE streams out to connected clients, but nothing pushes audit data to Datadog, Splunk, or an external OTEL collector yet. | Audit & Observability |
| PHI taxonomy | PII scanning covers email, credit card, SSN, and phone; health-identifier categories are not yet in the set. | Content Safety & Guardrails |
| Per-key budgets | An API key or PAT inherits its owner's scope — it is not its own governance subject, so no per-key budget, rate, or model scope. | Model Gateway & Routing |
| Tab-acceptance metric | Not currently measurable: Claude Code emits no accept/reject signal and no manual-LOC baseline exists. Needs an IDE-level integration. | Analytics |
| Per-project budgets | Spend is clamped per user and instance-wide; a monthly cap per project (Commerce / Core) is not yet available. | Cost Management |
Verified evidence
Each roadmap item is tracked in the same automated test suite as everything delivered — as an explicitly skipped placeholder. The suite cannot pass a test for an unbuilt feature, so a test run visibly reports the gap rather than a false green; the day a capability ships, its placeholder becomes a real test that must pass.
Everything else
If a capability is not on this page, it is delivered — start from the walkthroughs: User & Access Management, Analytics, Cost Management, Model Gateway & Routing, Audit & Observability, Safety & Guardrails, and Tool Governance.