Get the latest tech news
AWS now allows prompt caching with 90% cost reduction
AWS added Intelligent Prompt Routing and Prompt Caching to Bedrock in hopes of getting model usage prices down.
To answer customer demand, AWS announced two new capabilities on Bedrock to cut the cost of running AI models and applications, that are already available on competitor platforms. Simple yes-or-no questions like “Do you have a reservation?” are managed by a smaller model, but more complicated ones like “What vegan options are available?” would be routed to a bigger one. Luma CEO and co-founder Amit Jain told VentureBeat that AWS is the first cloud provider partner of the company to host its models.
Or read this on Venture Beat
