⚡ You're viewing a live demo of ChimerAI. Data resets daily at midnight UTC.Get the CLI →
← Back to Demo

Taking RAG to Production: Multi-Tenant Isolation

The problem

The FAISS vector store in services/ai/services/vector_store.py is a single global index. Every document uploaded through /api/rag/upload and every query through /api/rag/chat reads and writes the same data/faiss_index file, regardless of which user or organization is making the request. There is no user_id/tenant_id on chunks, and no filtering by ownership at query time.

That's fine for this starter kit — one deployment, one knowledge base, no cross-user concerns. It stops being fine the moment you serve multiple customers or organizations from the same deployment: without isolation, User A's uploaded documents are retrievable (and visible as chat "sources") to User B.

If you're only ever running one instance per customer (e.g. one Docker container per client), you don't need any of this — the isolation already happens at the infrastructure level. This document is only relevant if a single running service must keep multiple tenants' data apart.

Option 1 — Metadata filtering on a shared index

Tag every chunk with a tenant_id at ingestion time (already easy: it just becomes another key in the metadata dict passed to add_documents), then filter similarity_search results by the caller's tenant_id before returning them.

  • Pros: smallest change. No new infrastructure, one index file, one process.
  • Cons: not real isolation — the search still runs across all tenants' vectors before filtering, so one tenant's data volume affects another's query latency. A bug in the filter step leaks data across tenants; there's no structural guarantee. FAISS itself has no efficient native pre-filtering for IndexFlatL2 (you filter after the fact, in Python).
  • When to use it: prototypes, internal multi-team tools where a filtering bug isn't a security incident, very small tenant counts.

Option 2 — One FAISS index per tenant

Instead of a single global vector_store, keep a dict of VectorStore instances keyed by tenant_id, each with its own index_path/metadata_path (e.g. data/faiss_index_{tenant_id}). init_vector_store() already exists as a factory — extend it to a get_or_create_vector_store(tenant_id) cache instead of a single module-level singleton.

  • Pros: real isolation — a tenant physically cannot query another tenant's vectors, because there's no shared search space. Still no new infrastructure (plain files on disk).
  • Cons: file count grows linearly with tenant count; each index has its own in-memory footprint once loaded, so memory scales with the number of active tenants, not just data volume. Works well up to a few hundred/thousand tenants on a single host; beyond that, disk I/O and memory pressure become a real constraint.
  • When to use it: most small-to-mid SaaS deployments. This is the natural next step from the current architecture — same tools, same mental model, just parameterized by tenant.

Option 3 — Managed vector database with native multi-tenancy

Migrate off local FAISS files to a vector database designed for this:

  • Qdrant — collections per tenant, or a single collection with payload-based filtering enforced server-side (not just in app code).

  • Pinecone — namespaces per tenant within an index.

  • Weaviate — native multi-tenancy on a class (tenant-aware sharding, built for this exact use case).

  • pgvector (Postgres) — a tenant_id column plus Row-Level Security, if you're already running Postgres and want one less moving part.

  • Pros: horizontal scaling, backups, replication, and access control are the vector DB's problem, not yours. Filtering enforced at the query layer, not in application code — much harder to accidentally leak data across tenants.

  • Cons: real infrastructure dependency, migration effort (re-embed or export/import existing vectors), potential hosting cost.

  • When to use it: you expect to scale past what a single host comfortably handles, need compliance guarantees (SOC2, data residency per tenant), or already have DevOps investment in one of these systems.

Recommendation

Start at Option 2 if you're moving beyond a single-tenant deployment — it's a small, mechanical change to the existing code (parameterize VectorStore construction by tenant, thread tenant_id through rag_service and the API routes) and gives you real isolation without new infrastructure. Move to Option 3 only when tenant count, compliance requirements, or operational scale actually demand it — don't pay that migration cost before you need to.

ChimerAI Docs · Back to Demo