Every board wants AI in the product this year, and most existing systems were never designed to have a model bolted onto them. The gap between those two facts is where data leaks happen — customer records flowing to third-party APIs nobody reviewed, one tenant’s documents surfacing in another tenant’s answers, confidential data baked into a fine-tune forever. Here’s how to integrate AI and ML into an existing platform properly, as an engineering discipline rather than a demo.
Quick answer
Safe AI/ML integration into an existing system comes down to five disciplines: (1) treat every model — API or self-hosted — as an untrusted external system and put a gateway between it and your data; (2) minimise and transform what crosses that boundary (redact, tokenise, aggregate) instead of shipping raw records; (3) make retrieval permission-aware so the model can never see more than the requesting user could; (4) lock down vendor terms — training opt-outs, retention, region; and (5) roll out in shadow mode with evaluation and logging before anything touches customers. Data leaks are an architecture failure, not a model failure.
First, know which “data leak” you’re defending against
Three different failures get called a data leak, and they need different defences:
- Boundary leaks — your data leaving your control: sent to a model API under weak terms, logged by a vendor, retained in someone else’s infrastructure, or embedded into a model you don’t own.
- Authorisation leaks — your data crossing internal boundaries it shouldn’t: tenant A’s documents retrieved into tenant B’s context, a junior staff member’s chatbot quoting the CEO’s payroll file, an over-permissioned service account feeding a RAG index.
- Statistical leakage — the ML meaning: information from your test set or from the future contaminating training, so the model looks brilliant in evaluation and fails in production. Not a privacy problem, but the same root cause — sloppy data boundaries.
Most public AI incidents are the second kind. Almost all of them were preventable at design time.
Principle 1: The model is an untrusted external system
Whether you call a hosted API or run open-weights models on your own GPUs, architect as if the model endpoint is outside your trust boundary. That gives you one place to enforce everything:
- An AI gateway service — a single internal service through which every model call flows. No product code calls a model API directly. The gateway owns credentials, logging, redaction, rate limits and vendor failover.
- Egress control — production networks should only be able to reach approved model endpoints through the gateway. This is what turns “our policy says don’t send PII to AI tools” into something actually enforced — and it’s also your defence against shadow AI, where teams quietly wire a product to whatever API was easiest that week.
- Full audit logging — every prompt and response logged (with sensitive fields masked), attributable to a user, tenant and feature. When someone asks “what exactly did we send the vendor?”, you answer from your logs, not your memory.
Principle 2: Minimise what crosses the boundary
The single most effective control is boring: send less. Before a request leaves the gateway:
- Redact and tokenise — strip or placeholder direct identifiers (names, emails, account numbers) where the task doesn’t need them; re-substitute in the response on the way back. For many workflows the model genuinely doesn’t need to know who the customer is.
- Aggregate where possible — a model summarising support trends needs themes, not ten thousand raw tickets.
- Never send whole tables because it was convenient — the classic leak isn’t malicious; it’s a developer passing an entire customer object into a prompt because it was already in scope.
- Treat fine-tuning as permanent disclosure — anything in training data should be assumed unextractable-proof. Fine-tune on curated, reviewed datasets, never on raw production data.
Principle 3: Retrieval must inherit your permission model
RAG — retrieval-augmented generation — is where most real-world authorisation leaks happen, because teams build one big vector index over “the company’s documents” and let the model search it. The rule: the retrieval layer must enforce the same access control as the source systems, at query time.
- Store ACL metadata (tenant, team, sensitivity) alongside every chunk in the index, and filter on the requesting user’s entitlements before anything reaches the model context
- In multi-tenant systems, prefer hard partitioning (index-per-tenant or strict tenant filters enforced in one place) over trusting query-builders scattered through the codebase
- Re-index on permission change and deletion — a revoked document that lives on in an embedding index is a leak with a delay on it
- Remember that embeddings are data: they reconstruct meaning, so they inherit the source document’s classification and residency requirements
And because model output goes back to users: treat retrieved content as untrusted input too. Prompt injection — a document that instructs the model to ignore its rules and exfiltrate context — is a real attack class. Constrain what tools and data the model can reach; don’t rely on the model to police itself.
Principle 4: Vendor terms are part of your architecture
Before a model provider sees production data, the commercial layer has to hold:
- Training opt-out — your API data must not train the vendor’s models; on enterprise tiers this is standard, on consumer tiers often not
- Retention — how long prompts are stored, and whether zero-retention options exist for sensitive routes
- Region — where inference runs and logs live; for Australian regulated data, cross-border flows may need assessment under the Privacy Act (our Privacy Act reforms guide covers this)
- A DPA and sub-processor list — the same diligence you’d apply to any processor of customer data; your acquirer’s due-diligence team will ask
Self-hosting open-weights models trades these vendor questions for infrastructure ones — a legitimate choice when data cannot leave, but you inherit patching, scaling and evaluation burdens most teams underestimate.
Principle 5: Ship in shadow mode, measure, then expose
Integrating AI properly means admitting the output is probabilistic. The rollout pattern that works:
- Shadow mode first — the feature runs on real traffic, output logged but not shown; you learn failure modes at zero customer risk
- An evaluation set before launch — a few hundred representative cases with expected outcomes, run on every prompt or model change; without it, every tweak is a blind deploy
- Guardrails at the edges — schema-validated outputs, confidence thresholds, and human review on consequential actions (anything touching money, legal standing or customer communication)
- Monitoring for drift and cost — token spend, latency, refusal rates and quality metrics on dashboards, alarmed like any other production dependency
- A kill switch — feature-flag every AI capability so you can turn one off in seconds without a deploy
And the statistical kind: don’t leak the future into training
If you’re building predictive ML on your own data — churn, credit, fraud — the discipline is temporal: split train/test by time, never randomly across time; build features only from data available at prediction time; and validate on genuinely held-out periods. A model that “knows” the future in training will embarrass you in production. The same lesson applies to backtesting trading systems, where look-ahead bias is the classic version of this failure.
A sane sequence for an existing platform
- Weeks 1–2: inventory — what AI is already in use (including shadow AI), what data flows where, under what terms
- Weeks 3–4: stand up the gateway, egress rules and logging; fix vendor terms
- Weeks 5–8: first feature behind the gateway, permission-aware retrieval if RAG is involved, eval set built, shadow mode running
- Week 9+: graduated exposure with monitoring — and an architecture every subsequent AI feature reuses
This is exactly the kind of program a fractional CTO runs well: it needs senior judgement for a bounded period, across security, data and product at once. Our AI strategy service covers it, and if you’re weighing up whether the feature is worth building at all, our technical audit guide is the honest place to start.
Frequently asked questions
Is it safe to send customer data to a hosted model API?
It can be — with enterprise terms (training opt-out, defined retention, regional processing), a gateway enforcing redaction and logging, and data minimisation so the model sees only what the task needs. Unsafe is any product code calling a consumer-tier API directly with raw records.
Should we self-host models instead?
Self-host when data genuinely cannot leave your environment or scale economics demand it; otherwise hosted APIs with proper terms are usually faster and safer in practice. The architecture above works for both — the gateway just points somewhere else.
What’s the most common AI data leak you see?
Over-broad RAG: one index over everything, no per-user permission filtering. The second most common is shadow AI — teams wiring features to unapproved APIs before anyone reviews terms.
How do we retrofit this if we already shipped an AI feature?
In order: put a gateway in front of existing calls, fix vendor terms, add permission filtering to retrieval, then build the eval set. Retrofitting is routine — most of our AI engagements start from something already shipped.