Get the latest tech news
RAG architecture for SaaS – Learnings from building an AI code assistant
Building a RAG-based AI SaaS isn't hard - but there are gotchas and decisions you need to be mindful of. Here's how to build a code assistant from scratch - including the design decisions and the implementation details.
Scalability: As the set of embeddings grows, we can move some tenants to new physical machines, which allows us to use more memory and CPU for our applications over time. When we get to the conversation phase, I'll show you how this data model allows me to find the relevant code snippets for a user question with a single query. This made it easy to load the data - I could clone the repositories I wanted to embed and work with local files, rather than figure out the Github APIs.
Or read this on Hacker News
