Get the latest tech news

What we've learned from a year of building with LLMs

From the tactical nuts & bolts to the operational day-to-day to the long-term business strategy.

While the technology is still rapidly developing, we hope that these lessons, the by-product of countless experiments we’ve collectively run, will stand the test of time and help you build and ship robust LLM applications. We like to use the following “intern test” when evaluating generations: If you took the exact input to the language model, including the context, and gave it to an average college student in the relevant major as a task, could they succeed? Reliability: 99.9% uptime, adherence to structured output Harmlessness: Not generate offensive, NSFW, or otherwise harmful content Factual consistency: Being faithful to the context provided, not making things up Usefulness: Relevant to the users’ needs and request Scalability: Latency SLAs, supported throughput Cost: Because we don’t have unlimited budget And more: Security, privacy, fairness, GDPR, DMA, etc, etc.

Get the Android app