Something quietly changed in the RAG game at AWS and most people missed it.
AWS Bedrock now lets you point a Knowledge Base directly at S3. No vector database. No embedding pipelines. No Pinecone. No infrastructure overhead.
Drop your documents in S3. Bedrock handles chunking, embedding, indexing, retrieval. All managed. All serverless.
The traditional RAG stack has 6-7 moving parts. S3 Knowledge Base collapses it to two: your documents and Bedrock.
We were initially building with OpenSearch Serverless as our vector store. It works, but the cost adds up fast.
OpenSearch Serverless charges for OCUs (OpenSearch Compute Units) that run continuously, minimum 2 OCUs for indexing + 2 for search.
That is roughly $700-1000/month just to keep the vector store alive, before you even factor in embedding compute and data ingestion pipelines.
For a POC or early stage product, that is hard to justify. S3 Knowledge Base flips this completely. Storage cost is pennies. You pay per query, not per hour. For our use case with a few hundred documents and moderate query volume, we are looking at 80-90% cost reduction compared to the OpenSearch setup. And zero infra management.
We have been exploring this for an inbound conversational AI agent for one of our customers. Still in POC and testing, but early results are promising.
Retrieval quality is solid, latency is acceptable, and the operational simplicity is a massive win. No vector DB to babysit. Just update docs in S3 and the knowledge base stays current.
The interesting shift: the bottleneck moves from "how do we build RAG infrastructure" to "how do we structure our knowledge well." Which is where it should have been all along.
If you are building RAG on AWS and still running OpenSearch or a managed vector DB for small to mid-size knowledge bases, take a serious look at this. #AWS #AWSBedrock #GenAI #RAG #KnowledgeBase #OpenSearch #ConversationalAI #Serverless #AI
Sagar Thakare Swami Sivasubramanian Banjo Obayomi Antje Barth Adarsh Pandey Aravind M
Love what Ishan Mukherjee is building Rox