← All case studies

AI platform engineering

RareDaysAI: a reusable RAG platform

I built the platform behind Ask Kindred so each new client is a config file and an ingest, not a new project. It ships as an npm SDK, a CLI and a hosted multi-tenant backend.

Role
Lead engineer and architect
Team
Led a team of 4–5 engineers and designers
Status
In production at Rare Days
Stack
TypeScript, Next.js, AWS Bedrock, Pinecone

Outcome

Three customer companies run on it, and a new client starts from a config file and an ingest.

  • 3customer companies on the platform
  • ~90%of repository commits authored by me
  • SDK + CLIweb, React Native and script-tag clients

Speed, uptime and cost

  • <5stime to first token
  • 99.99%availability
  • ~$15cost per client, per month

What I owned

Authored most of the platform, about 90% of the commits in the repository, from the retrieval pipeline to the SDK, CLI and admin tooling.

Designed the tenant model: each client gets isolated vector storage, its own tone and prompts, model routing and crisis handling, all declared in one config-as-code file.

Wrote the architecture decision record for moving from one stack per tenant to pooled regional cells with dedicated cells for compliance clients.

System at a glance

Clients use a React, React Native or script-tag widget, or the TypeScript SDK against the REST API.

  1. Clients embed a React, React Native or script-tag widget, or call the REST API with the TypeScript SDK.
  2. The CLI scaffolds a config, pushes it to the backend, merges live edits the way git does, and ingests a knowledge base.
  3. Each answer checks a semantic cache, embeds the query, searches the tenant's vector namespace, reranks, classifies the topic, routes to a model and streams a cited response.
  4. Input, output and session guards, per-tenant capacity caps and a region capacity gate sit in front of the model providers.

Key decisions

  • Compared hosted chatbot vendors against a custom Pinecone-backed build before committing, with the goal of a high-quality assistant at a fraction of vendor cost.
  • Moved live reranking from a self-hosted model to a managed one after measuring 10 to 11 seconds for real-size passages. The self-hosted model stays for eval runs only.
  • Proposed pooled regional cells over one stack per tenant, because a streamed answer holds a Lambda concurrency slot for 5 to 15 seconds while about 95% of that is waiting on the model. The record names the trade-off: shared quotas need per-tenant caps.
  • Made config the contract. A push that conflicts with live edits stops instead of overwriting them.

How I approached reliability

  • Deploys go to staging first, then production behind an approval.
  • Per-tenant caps on concurrent chats, requests per minute and monthly tokens stop one client from starving another.
  • On a model throttle the platform falls back from the larger model to the smaller one before failing the request.
  • A reranker failure never fails an answer. Retrieval order is the fallback.
  • Every stack resource is tagged, and model spend is attributed to the calling stack, so cost per client is a report, not a guess.

Technology focus

  • TypeScript
  • Next.js
  • AWS Bedrock
  • Pinecone
  • DynamoDB
  • SST
  • Turborepo
  • React Native
  • GitHub Actions