DEV Community: Wojtek Pluta

Vector Embeddings: How They Work, Where to Store Them, and Best Practices

Wojtek Pluta — Thu, 16 Apr 2026 15:51:46 +0000

Key Takeaways

Vector embeddings convert unstructured data into numeric representations that power semantic search, recommendations, and multimodal analytics beyond keywords.
Embedding success isn’t just about the model—it also depends on a data platform that can meet requirements for scale, low latency, security, and governance, including vector indexing/ANN search, access controls, encryption, and monitoring.
Oracle AI Database unifies native vector types and similarity search, enterprise-grade security, and integrated vector, structured, and unstructured data—so teams can build RAG, search, and analytics without piecing together multiple systems.

Semantic similarity search over vector space - Oracle Help Center

Introduction to Vector Embeddings

Vector embeddings have changed the way we interact with unstructured data such as text, images, audio, and code. By transforming this data into high-dimensional numeric vectors, we can use embeddings to process the semantic meaning and relationships within the data.

We can look at embeddings as task or domain-specific representations of vectors. The geometric relationships among them represent meaningful similarities between concepts in semantic space. The efficient storage and querying of vector embeddings enables capabilities such as semantic search, recommendations, and advanced analytics; and bridges the gap between unstructured and structured information.

What are Vector Embeddings? A Definition and Their Role

Vector embeddings are mathematical representations of objects—such as words, sentences, images, or audio—encoded as dense, high-dimensional vectors. Each vector encapsulates features that capture semantic meaning, context, or structure of the data. For example, similar words or images will have embeddings positioned closely in the vector space, enabling similarity-based operations. This allows for similar “things” to be grouped together under a distance metric.

The adoption of vector embeddings underpins many cutting-edge technologies:

Retrieval-augmented generation (RAG): Enhances large language models by retrieving relevant context using embedding similarity.
Semantic search: Finds documents with similar context, not just matching keywords.
Recommendations: Suggests products or content by comparing user or item embeddings.
Deduplication and anomaly detection: Identifies near-duplicates or outliers based on embedding distances.
Multimodal analytics: Links information across text, image, audio, and other domains.

This ability to bridge structured and unstructured data makes embeddings indispensable across modern data architectures.

How to Create Embeddings? Some Tools That Can Help

A variety of tools can encode text, images, and code as vector embeddings, enabling similarity search, retrieval workflows (including RAG), and other ML tasks:

OpenAI – provides hosted embedding APIs backed by task-optimized models, accessible with REST interfaces.
Hugging Face – offers a large catalog of pre-trained multimodal embedding models and libraries (such as the Transformers library), plus community benchmarks.
Oracle AI Database – provides a native vector memory store in Oracle Database, enabling storage, indexing (e.g., IVF/flat/HNSW), and retrieval of vector embeddings alongside relational data with SQL and PL/SQL integration; supports hybrid search (vector + metadata filters), enterprise-grade security, and governance for RAG and semantic search workloads
TensorFlow – enables building and serving custom embedding models using Keras, enabling easy integration into training pipelines.
PyTorch – provides flexible primitives to fine-tune or implement embedding models, and deploy them via TorchScript.

Benefits of Working With Vector Embeddings

The following are just a few of the benefits vector embeddings have brought to today's AI tech stack:

Vector embeddings are currently the best way to transform complex data into numerical units that reflect meaning, similarity and enable clustering and retrieval beyond keyword matching.
The limitations of keyword methods were particularly visible in areas such as synonym handling, typos, and paraphrasing, and are now absent in modern-day LLMs relying on vector embeddings.
Embeddings support multilingual and cross-modal experiences by aligning meaning across languages and modalities.

Other approaches, such as sparse lexical retrieval and symbolic/ontology-based methods, can be effective, but dense vector embeddings are often a better fit when you need semantic similarity matching (for example, paraphrases and synonyms) rather than exact keyword overlap.

Challenges in Working With Vector Embeddings

The following are some of the potential challenges you may face in working with vector embeddings, and potential ways to mitigate them:

Storage Volume and High Dimensionality

Storage challenges include:

Large embedding volumes: Billions of vectors require scalable storage and efficient indexing.
High dimensionality: Embeddings of 128, 512, or 1024+ dimensions need specialized data structures and optimized storage formats.

Performance and Latency Bottlenecks

Performance factors include:

Indexing and search speed: ANN techniques improve latency, but very large datasets demand optimized infrastructure.
Batch insertion and streaming: Efficiently handling ongoing ingestion of new embeddings.

Distributed System Complexities and Operational Overhead

At scale, sharding, replication, and consistency management become complex. Automated scaling, monitoring, and failover are desirable for production systems.

Cost Factors

Vector embeddings may affect operational cost:

Compute and storage requirements: High-dimensional data and fast search consume substantial resources.
Operational overhead: Consider cost of infrastructure, team expertise, and maintenance.

Encryption at Rest and in Transit

Securing embeddings is crucial as they can encode sensitive information:

Encryption at rest: Protects stored vectors using strong industry-standard algorithms.
Encryption in transit: Ensures vectors remain confidential when transmitted between systems or users.

Oracle AI Database enforces encryption by default and integrates with enterprise key management solutions.

Access Control and Authentication

Control who can access, modify, or query embeddings:

Granular permissions: Define user roles and table-level permissions.
Integration with SSO and identity providers: Streamlines enterprise authentication.
Audit trails: Track access and changes for compliance.

Data Sanitization and Monitoring

Reduce risk by implementing:

Sanitization: Remove or obfuscate sensitive or personal information in embeddings before storage.
Monitoring and anomaly detection: Detect unusual access patterns or potential misuse.

Advanced Cryptographic Techniques

For highly sensitive embeddings:

Homomorphic encryption or secure multi-party computation: Enables computation and search on encrypted embeddings, minimizing exposure.

Common Vector Embedding Use Cases

Embeddings open up a wide array of practical use cases:

Enterprise search and information retrieval: Improved accuracy and relevance in document and knowledge base searches.
Personalization and recommendation engines: Enhanced user experiences by surfacing relevant content or products.
Fraud and anomaly detection: Early identification of unusual patterns using embedding distances.
Data deduplication and clustering: Streamlined datasets and improved analytics through intelligent grouping.
Multimodal retrieval and analytics: Unified analysis over diverse data types, fostering deeper insights.

Storing Vector Embeddings and the Oracle Advantage

The following are a few key points related to the storage of vector embeddings, and how Oracle AI Database's native vector store capabilities can streamline and strengthen your stack with its native vector store capabilities.

Specialized Vector Databases

Dedicated vector databases are built for storing, indexing, and searching embeddings efficiently. These databases excel at large-scale similarity search with features such as:

High-dimensional indexing: Specialized data structures to support billion-scale embeddings.
Approximate search capabilities: Fast, scalable similarity queries using Approximate Nearest Neighbor (ANN) techniques.
RESTful APIs and SDKs: Developer-friendly interfaces for ingestion and search.

Popular examples include Pinecone, Weaviate, Milvus, and Vespa. Specialized databases are ideal for workloads with large volumes of embeddings and demanding similarity search requirements.

SQL/NoSQL Databases with Vector Support

Traditional databases are evolving to meet AI's demands by adding native vector data types and search capabilities:

SQL databases: PostgreSQL (with pgvector), Oracle AI Database, and others support vector columns and similarity search via extensions or built-in features.
NoSQL databases: MongoDB and Redis now offer basic vector search features, often using plugins or modules.

This integration enables seamless blending of embeddings with structured business data, supporting hybrid query scenarios.

Oracle AI Database Approach

From Oracle's viewpoint, AI databases must natively support vector data types, efficient similarity queries, and enterprise security for integrating embeddings across applications. Oracle AI Database is designed to address these needs at scale.

The Oracle AI Database offers a unified approach allowing developers to:

Store embeddings alongside structured and unstructured data.
Run similarity queries directly using SQL and specialized vector search operators.
Integrate with Oracle's rich security, high availability, and scalability features.
Combine vector search, filtering, ranking, and analytical queries in a single stack.

Example Procedures - Using Vector Embeddings in Oracle AI Database

The following examples are intentionally minimal and illustrative. They highlight how Oracle AI Database supports native vector storage and SQL-based similarity search.

CREATE TABLE documents (

 id NUMBER,

 content CLOB,

 embedding VECTOR

);

This example shows a minimal table definition using Oracle AI Database’s native VECTOR data type. In practice, embeddings are stored alongside structured or unstructured application data in the same database.

SELECT id, content

FROM documents

ORDER BY VECTOR_DISTANCE(embedding, :query_vector, COSINE)

FETCH FIRST 5 ROWS ONLY;

This example illustrates SQL-based similarity search in Oracle AI Database. The :query_vector placeholder represents the embedding generated from user input by an embedding model (inside or outside the database) and is used to rank the nearest matches.

Hybrid query pattern (semantic + relational filtering)

SELECT id, content

FROM documents

WHERE content IS NOT NULL

ORDER BY VECTOR_DISTANCE(embedding, :query_vector, COSINE)

FETCH FIRST 5 ROWS ONLY;

This hybrid pattern combines standard SQL filtering with semantic ranking in a single query. It is useful when semantic search must also respect metadata constraints, access controls, or business rules. This streamlines workflows and facilitates embedding-driven applications without moving data across siloed systems.

Using Oracle Autonomous AI Database in conjunction with langchain-oracledb, for example, we can simply generate embeddings, store, and interact with vectors directly from within the database – requiring no additional investment in another separate vector database.

Querying and Searching for Stored Vector Embeddings

The following are a few of the things you should keep in mind if your work involves querying and searching for stored vector embeddings:

Approximate Nearest Neighbor (ANN) Algorithms and Data Structures

Searching for similar embeddings at scale requires efficient algorithms:

ANN Techniques: Rather than exact search, algorithms like HNSW (Hierarchical Navigable Small World), IVF (Inverted File Index), and PQ (Product Quantization) yield fast, near-accurate results.
Data Structures: Use trees (KD-Tree, Ball Tree), graphs (HNSW), or hash-based indices (LSH) to organize and retrieve vectors efficiently.

ANN can deliver millisecond-latency searches over millions or billions of embeddings, making it essential for operational AI applications.

High-level retrieval workflow (generalized)

At a high level, semantic retrieval follows a simple and reusable pattern that applies across vector databases, frameworks, and application stacks:

Convert user input into a query embedding.
Compare it against stored embeddings.
Rank results by similarity.
Apply filters and business rules as needed.

This high-level workflow is framework- and language-agnostic. While the underlying implementation differs across platforms and tools, the conceptual flow remains the same for the most vector search and RAG-style applications.

Popular Libraries

Several tools make it easier to store, and search embeddings:

Vector search libraries: FAISS (Facebook AI Similarity Search), Annoy (Spotify), NMSLIB, ScaNN.

These libraries power both stand-alone vector stores and integrations within general-purpose databases.

How to Choose the Right Similarity Metrics

Selecting the right similarity metric is critical for effective search:

Cosine similarity: Measures the angle between vectors; ideal for text and semantic similarity.
Euclidean distance: Useful for geometric or spatial data.
Dot product: Common in deep learning models; efficient for high-dimensional comparisons.

Your choice depends on the nature of your data and the specifics of your application.

Oracle AI Database Capabilities

Oracle’s AI Database combines native vector capabilities, enterprise security, and proven scalability, making it a robust choice for organizations seeking a unified solution for traditional data and AI-enabled workloads.

Native vector data types and indexing: Supports efficient storage and retrieval of high-dimensional vectors.
Integrated similarity search: Enables querying and filtering based on vector proximity.
Enterprise-grade security: Encryption at rest, robust access controls, and activity monitoring.
Hybrid queries: Seamless combination of structured, unstructured, and vector data in complex analytical tasks.
High scalability: Handles massive volumes of embeddings without performance degradation.

Best Practices for Working With Vector Embeddings

The following are a few of the best practices for using vector embeddings to power semantic search, personalized recommendations, multimodal analytics (including anomaly detection), and domain-specific insights across enterprise applications.

Semantic Search and Information Retrieval

Semantic search with embeddings offers better context and intent recognition than keyword search. Querying an embedding retrieves documents or objects with similar meanings—crucial for legal, healthcare, customer support, and research applications.

Recommendation Systems and Personalization

Compare user and item embeddings to power personalized recommendations. This increases engagement, retention, and value in e-commerce, media, and B2B applications.

Multimodal Search and Anomaly Detection

Combine embeddings across text, image, and audio for multimodal analytics or use distance-based thresholds to flag anomalies and outliers in fraud prevention or system monitoring.

Domain-Specific Analytics

Specialized embeddings can be trained for particular industries—finance, healthcare, retail—and stored/retrieved for advanced analytics, predictions, or compliance monitoring.

How to Select Appropriate Tools and Architectures

Match your use case to the data platform (dedicated vector database vs. extended relational/NoSQL).
If you want both, Oracle AI Database is a good option.
Factor in scale, integration needs, security requirements, and budget.
Leverage proven libraries and frameworks to speed up development.

Security and Scalability Considerations

Encrypt embeddings, control access, and monitor usage.
Choose solutions that scale with data growth and user demand.
Balance security, performance, and cost based on enterprise requirements.

Architectural Patterns

Hybrid architecture: Combine vector storage/search with structured data in a unified database like Oracle AI Database.
Microservices: Separate ingestion, search, and analytics as independently scaling components if needed.
Cloud-native solutions: Consider managed vector databases for elasticity and reduced operational burden.

Tooling Reminders

Use specialized libraries (FAISS, Annoy, HNSWLib) for local development, prototyping, or custom solutions.
For production or enterprise use, rely on databases with native vector support and robust security, such as Oracle AI Database.

Frequently Asked Questions (FAQ)

What are vector embeddings and why do they matter?

Vector embeddings are dense, high-dimensional numeric representations of objects like text, images, audio, or code. They place semantically similar items near each other in a continuous space, enabling tasks like semantic search, recommendations, RAG, deduplication, and anomaly detection. Compared with keyword or symbolic methods, embeddings better capture meaning, handle synonyms/paraphrases, and are robust across languages and modalities.

What are the main challenges in storing and querying embeddings at scale?

Volume and dimensionality: Billions of vectors, often 128–1024+ dimensions
Performance: Fast indexing and low-latency search, efficient batch/stream ingestion
Distributed ops: Sharding, replication, consistency, monitoring, and failover
Cost: Compute, storage, and operational overhead
Security: Encryption at rest/in transit, access control, auditing, data sanitization, and advanced cryptographic techniques for sensitive data

Where should I store embeddings: a dedicated vector database or a database with vector support?

Two common patterns:

Specialized vector databases (e.g., Pinecone, Weaviate, Milvus, Vespa) for high-scale, low-latency similarity search with ANN, SDKs, and REST APIs.
SQL/NoSQL databases with vector support (e.g., Oracle AI Database, PostgreSQL with pgvector, MongoDB, Redis) for blending vectors with structured data and enabling hybrid queries. Your choice should consider scale, integration with existing data, security, cost, and operational complexity.

What does Oracle AI Database provide for embeddings?

Oracle AI Database offers native vector types and indexing, integrated similarity search in SQL, enterprise-grade security (encryption, granular access control, auditing), and high scalability. It supports hybrid analytical queries across structured, unstructured, and vector data. With Oracle Autonomous AI Database and libraries like langchain-oracledb, teams can generate, store, and query embeddings within one platform—avoiding data silos and extra operational overhead. Encrypt data, enforce access controls, and monitor usage to meet enterprise requirements.

Conclusion

Storing and querying vector embeddings is a critical enabler for next-generation AI and data applications. By leveraging the right databases, libraries, and best practices, organizations and engineers can unlock new value from unstructured content, while maintaining performance, scalability, and security.

Resources

Agent Memory: A Free Short Course on Building Memory-Aware Agents

Wojtek Pluta — Thu, 16 Apr 2026 12:13:17 +0000

Oracle and DeepLearning.AI have launched Agent Memory: Building Memory-Aware Agents, a free short course on DeepLearning.AI that teaches developers how to architect memory systems that give agents persistence, continuity, and the ability to learn over time.

"Memory turns a stateless LLM into an agent that learns over time. How to architect agentic memory is one of the most debated topics in AI right now. This course gives AI developers and engineers a comprehensive view of the most common memory patterns."

Andrew Ng, Founder, DeepLearning.AI

Most agents forget. Each new session starts from zero, accumulated context from previous interactions is discarded, and the agent has no mechanism to learn from what it has already done. As a result, AI developers often rely on workarounds: cramming everything into the context window, reloading conversation logs, or bolting on ad-hoc retrieval.

These approaches can work, but they don't provide a clear mental model for how information should live inside an agentic system boundary. This course treats memory as a first-class citizen in AI agents, and is built around that memory-first perspective.

"For the past few years, we have focused on prompt and context engineering to get the best results from a single LLM call. But engineering the right context for agents that need to work over days or weeks needs an effective memory system. This course takes that memory-first approach to building agents."

Richmond Alake, AI Developer Experience Director, Oracle

Beyond Prompt Engineering

You’ve heard about prompt engineering. You've probably heard about context engineering. This course introduces the next layer: memory engineering, treating long-term memory as first-class infrastructure that is external to the model, persistent, and structured.

The course covers the full memory stack across five hands-on modules, built on LangChain, Tavily, and Oracle AI Database:

Why AI Agents Need Memory: Explore failure modes of stateless agents and the memory-first architecture used throughout the course.
Constructing the Memory Manager: Design persistent memory stores across memory types, model memory data for efficient retrieval, and implement a manager that orchestrates read, write, and retrieval operations during agent execution.
Scaling Agent Tool Use with Semantic Tool Memory: Treat tools as procedural memory, index them in a vector store, and retrieve only contextually relevant tools at inference time using semantic search.
Memory Operations: Extraction, Consolidation, and Self-Updating Memory: Build LLM-powered pipelines that extract structured facts from raw interactions, consolidate episodic memory into semantic memory, and implement write-back loops that let an agent autonomously update and resolve conflicts in its own knowledge base.
Memory-Aware Agent: Assemble a stateful agent that initializes from long-term memory at startup, checkpoints intermediate reasoning states during execution, and persists learned context across sessions.

"The patterns we cover here are not theoretical. AI developers and engineers will walk through real implementations: building memory stores, wiring up extraction pipelines, and handling contradictions in memory. You leave with working code you can adapt for your own production agents."

Nacho Martinez, AI Developer Advocate, Oracle

Oracle AI Database as the Agent Memory Core

Oracle AI Database serves as the unified agent memory core throughout the course. Instead of treating a database as a passive store, the course demonstrates how Oracle AI Database functions as the active retrieval and persistence layer that makes each memory pattern work in production.

Oracle AI Database brings key retrieval strategies into a single engine, including vector search for semantic similarity and unstructured knowledge retrieval, graph traversal for relationship-aware reasoning across connected entities, and relational queries for structured, transactional memory that demands precision and consistency. This helps reduce complexity by avoiding separate systems for different data types.

The memory patterns taught in this course, such as semantic tool memory, self-updating memory, and memory consolidation, are the same patterns used to build production-grade agentic systems on Oracle AI Database. This course puts that architecture directly in the hands of AI developers and engineers.

Who This Course Is For

Agent Memory: Building Memory-Aware Agents is designed for:

AI developers and engineers building or evaluating agentic systems who need production-grade memory architecture
ML engineers integrating LLMs into multi-turn or multi-session workflows
Developers working with LangChain, LangGraph, or Tavily who want durable, structured memory
Technical leaders assessing Oracle AI Database for agent infrastructure at scale

Availability

Agent Memory: Building Memory-Aware Agents is available now on DeepLearning.AI. The course is free to access and requires no prior Oracle experience. Developers can enroll in the course.

About Oracle AI Database

Oracle AI Database is a converged database platform built for AI workloads. It provides native vector search, graph traversal, relational retrieval, and the persistence infrastructure required for production agent memory systems in a single database engine. This removes the fragmented infrastructure that can become a bottleneck for AI innovation. Oracle AI Database is used by developers and enterprises as the unified memory core for AI agents to build and deploy intelligent, secure, memory-aware systems.

A Practical Guide to Choosing the Right Memory Substrate for Your AI Agents

Wojtek Pluta — Thu, 16 Apr 2026 12:11:25 +0000

Key Takeaways

Don't conflate interface with substrate. Filesystems win as an interface (LLMs already know how to use them); databases win as a substrate (concurrency, auditability, semantic search).
For prototypes, files are hard to beat. Simple, transparent, debuggable—a folder of markdown gets you surprisingly far when iteration speed matters most.
Shared state demands a database. Concurrent filesystem writes can silently corrupt data. If multiple agents or users touch the same memory, start with database guarantees.
Semantic retrieval beats keyword search at scale. Grep performance degrades on paraphrases and synonyms. Vector search finds content by meaning, this is critical once your knowledge base grows.
Avoid polyglot persistence. Running separate systems for vectors, documents, and transactions means four failure modes. Oracle AI Database simplifies your memory architecture.

AI developers are watching agent engineering evolve in real time, with leading teams openly sharing what works. One principle keeps showing up from the front lines: build within the LLM’s constraints.

In practice, two constraints dominate:

LLMs are stateless across sessions (no durable memory unless you bring it back in).
Context windows are bounded (and performance can degrade as you stuff more tokens in).

So “just add more context” isn’t a reliable strategy due to the quadratic cost of attention mechanisms and the degradation of reasoning capabilities as context fills up. The winning pattern is external memory + disciplined retrieval: store state outside the prompt (artifacts, decisions, tool outputs), then pull back only what matters for the current loop.

There’s also a useful upside: because models are trained on internet-era developer workflows, they’re unusually competent with developer-native interfaces: repos, folders, markdown, logs, and CLI-style interactions. That’s why filesystems keep showing up in modern agent stacks.

This is where the debate heats up: “files are all you need” for agent memory. Most arguments collapse because they treat interface, storage, and deployment as the same decision. They aren’t.

Filesystems are winning as an interface because models already know how to list directories, grep for patterns, read ranges, and write artifacts. Databases are winning as a substrate because once memory must be shared, audited, queried, and made reliable under concurrency, you either adopt database guarantees or painfully reinvent them.

In this piece, we give a systematic comparison of filesystems and databases for agent memory: where each approach shines, where it breaks down, and a decision framework for choosing the right foundation as you move from prototype to production.

Our aim is to educate AI developers on various approaches to agent memory, backed by performance guidance and working code.

All code presented in this article can be found here.

Understanding Agent Memory and Its Importance

Let’s take the common use case of building a Research Assistant with Agentic capabilities.

You build a Research Assistant agent that performs brilliantly in a demo; in the current execution, it can search arXiv, summarize papers, and draft a clean answer in a single run. Then you come back the next morning, start from a clean run, and then prompt the agent: “Continue from where we left off, and also compare Paper A to Paper B.” The agent responds as if it has never met you because LLMs are inherently stateless. Unless you send prior context back in, the model has no durable awareness of what happened in previous turns or previous sessions.

Once you move beyond single-turn Q&A into long-horizon tasks, deep research, multi-step workflows, and multi-agent coordination, you need a way to preserve continuity when the context window truncates, sessions restart, or multiple workers act on shared state. This takes us into the realm of leveraging systems of record for agents and introduces the concept of Agent Memory.

The Stateless LLM Problem

Why your Research Assistant forgets everything between sessions?

What is Agent Memory?

Agent memory is the set of system components and techniques that enable an AI agent to store, recall, and update information over time so it can adapt to new inputs and maintain continuity across long-horizon tasks. Core components typically include the language and embedding model, information retrieval mechanisms, and a persistent storage layer such as a database.

Types of Agent Memory

Types of Agent Memory

In practical systems, agent memory is usually classified into two distinct forms:

Short-term memory (working memory): whatever is currently inside the context window.
Long-term memory: a persistent state that survives beyond a single call or session (facts, artifacts, plans, prior decisions, tool outputs).

Concepts and techniques associated with agent memory all come together within the agent loop and the agent harness, as demonstrated in this notebook and explained later in this article.

Agent Loop and Agent Harness

The agent loop is the iterative execution cycle in which an LLM receives instructions from the environment and decides whether to generate a response or make a tool call based on its internal reasoning about the input provided in the current loop. This process repeats until the LLM produces a final output or an exit criterion is met. At a high level, the following operations are present within the agent loop:

Assemble context (user request + relevant memory + tool json schemas).
Call the model (plan, decide next action).
Take actions (tools, search, code execution, database queries).
Observe results (tool outputs, errors, intermediate artifacts).
Update memory (write transcripts, store artifacts, summarize, index).
Repeat until the task completes or hands control back to the user.

Anthropic’s guidance on long-running agents directly points to this: they describe harness practices that help agents quickly re-understand the state of work when starting with a fresh context window, including maintaining explicit progress artifacts.

The agent harness is the surrounding runtime and rules that make the loop reliable: how you wire tools, where you write artifacts, how you log/trace behavior, how you manage memory, and how you prevent the agent from drowning in context.

To complete the picture, the discipline of context engineering is heavily involved in the agent loop and aspect of the agent harness itself. Context engineering is the systematic design and curation of the content placed in an LLM’s context window so that the model receives high-signal tokens and produces the intended, reliable output within a fixed budget.

In this piece, we implement context engineering as a set of repeatable techniques inside the agent harness:

Context retrieval and selection: Pull only what is relevant (via grep for filesystem memory, via vector similarity and SQL filters for database memory).
Progressive disclosure: Start small (snippets, tails, line ranges) and expand only when needed.
Context offloading: Write large tool outputs and artifacts outside the prompt, then reload selectively.
Context reduction: Summarize or compact information when you approach a degradation threshold, then store the summary in durable memory so you can rehydrate later.

The concepts and explanations above set us up for the rest of the comparison we introduce in this piece. Now that we have the “why” and the moving parts (stateless models, the agent loop, the agent harness, and memory), we can evaluate the two dominant substrates teams are using today to make memory real: the filesystem and the database.

Filesystem-first Agentic Research Assistant

A filesystem-based memory architecture is not “the agent remembers everything forever”. It is the agent that can persist state and artifacts outside the context window and then pull them back selectively when needed. This aligns with two of the earlier-mentioned LLM constraints: a limited context window and statelessness.

In our Research Assistant, the filesystem becomes the memory substrate. Rather than injecting a large number of tools and extensive documentation into the LLM's context window (which would inflate the token count and trigger early summarization), we store them on disk and let the agent search and selectively read what it needs. This matches with what the Applied AI team at Cursor calls “Dynamic Context Discovery”: write large output to files, then let the agent tail and read ranges as required.

Our FSAgent and demo is using valid filesystem-OS related operations (such as tail and cat to read the contents of files; but that this is a very "simplified" approach, with a limited number of operations for demonstration purposes, and the capabilities offered in the file system can be optimized (with other commands and implementations).

On the other hand, it's a great start for people to get familiarized with tool access and how file system memory is achieved.

Semantic memory (durable knowledge): papers and reference docs saved as markdown.
Episodic memory (experience): conversation transcripts + tool outputs per session/run.
Procedural memory (how to work): “rules” / instructions files (e.g., CLAUDE.md / AGENTS.md) that shape behavior across sessions.

What does this look like in tooling?

Before we jump into the code, here’s the minimal tool surface we provide to the agent in the table below. Notice the pattern: instead of inventing specialized “memory APIs,” we expose a small set of filesystem primitives and let the agent compose them (very Unix).

Tool	What it does	Output
arxiv_search_candidates(query, k=5)	Searches arXiv and returns a JSON list of candidate papers with IDs, titles, authors, and abstracts.	JSON string of paper candidates
fetch_and_save_paper(arxiv_id)	Fetches full paper text (PDF → text) and saves to `semantic/knowledge_base/<id>.md`. Avoids routing full content through the LLM.	File path
read_file(path)	Reads a file from disk and returns its contents in full (use sparingly).	Full file contents
tail_file(path, n_lines=80)	Reads the last N lines of a file (first step for large files).	Last N lines
read_file_range(path, start_line, end_line)	Reads a line range to “zoom in” without loading everything.	Selected line range
grep_files(pattern, root_dir, file_glob)	Grep-like search across files to find relevant passages quickly.	Matches with file path + line number
list_papers()	Lists all locally saved papers in `semantic/knowledge_base/`.	List of filenames
conversation_to_file(run_id, messages)	Appends conversation entries to one transcript file per run in `episodic/conversations/`.	File path
summarise_conversation_to_file(run_id, messages)	Saves full transcript, then writes a compact summary to `episodic/summaries/`.	Dict with transcript + summary paths
monitor_context_window(messages)	Estimates current context usage (tokens used/remaining).	Dict with token stats

This design directly reflects what the AI ecosystem is converging on: a filesystem and a handful of core tools, rather than an explosion of bespoke tools.

Progressive reading (read, tail, range)

The first memory principle implementation is simple: don’t load large files unless you must. Filesystems are excellent at sequential read/write and work naturally with tools like grep and log-style access. This makes them a strong fit for append-only transcript and artifact storage.

That’s why we implement three reading tools:

Read everything (rare),
Read the end (common for logs/transcripts)
Read a slice (common for zooming into a match)

The tools below were implemented in Python and converted into objects callable by a langchain agent using the @tool decorator from the langchain agent module.

First is the read_file tool, the “load it all” option. This tool is useful when the file is small, or you truly need the full artifact, but it’s intentionally not the default because it can expand the context window.

@tool
def read_file(path: str) -> str:
 p = Path(path)
 if not p.exists():
 return f"File not found: {path}"
 return p.read_text(encoding="utf-8")

The tail_file function is the first step for large files. It grabs the end of a log/transcript to quickly see the latest or most relevant portion before deciding whether to read more.

@tool
def tail_file(path: str, n_lines: int = 80) -> str:
 p = Path(path)
 if not p.exists():
 return f"File not found: {path}"
 lines = p.read_text(encoding="utf-8").splitlines()
 return "\n".join(lines[-max(1, n_lines):])

The read_file_range function is seen as the surgical tool that is used once you’ve located the right region (often via grep or after a tail), pulls in just the exact line span you need, so the agent stays token-efficient and grounded.

@tool
def read_file_range(path: str, start_line: int, end_line: int) -> str:
 p = Path(path)
 if not p.exists():
 return f"File not found: {path}"
 lines = p.read_text(encoding="utf-8").splitlines()
 start = max(0, start_line)
 end = min(len(lines), end_line)
 if start >= end:
 return f"Empty range: {start_line}:{end_line} (file has {len(lines)} lines)"
 return "\n".join(lines[start:end])

Again, this is essentially dynamic context discovery in a microcosm: load a small view first, then expand only when needed.

Grep-style search (find first, read second)

A filesystem-based agent should quickly find relevant material and pull only the exact slices it needs. This is why grep is such a recurring theme in the agent tooling conversation: it gives the model a fast way to locate relevant regions before spending tokens to pull content.

Here’s a simple grep-like tool that returns line-numbered hits so the agent can immediately jump to read_file_range:

@tool
def grep_files(
pattern: str,
root_dir: str = "semantic",
file_glob: str = "**/*.md",
max_matches: int = 200,
ignore_case: bool = True,
) -> str:

root = Path(root_dir)
if not root.exists():
return f"Directory not found: {root_dir}"

flags = re.IGNORECASE if ignore_case else 0
try:
rx = re.compile(pattern, flags)
except re.error as e:
return f"Invalid regex pattern: {e}"

matches = []
for fp in root.glob(file_glob):
if not fp.is_file():
continue
try:
with open(fp, 'r', encoding='utf-8', errors='ignore') as f:
for i, line in enumerate(f, start=1):
if rx.search(line):
matches.append(f"{fp.as_posix()}:{i}: {line.strip()}")
if len(matches) >= max_matches:
return "\n".join(matches) + "\n\n[TRUNCATED: max_matches reached]"
except Exception:
continue

if not matches:
return "No matches found."
return "\n".join(matches)

One subtle but important detail in our grep_files implementation is how we read files. Rather than loading entire files into memory with read_text().splitlines(), we iterate lazily with for line in open(fp), which streams one line at a time and keeps memory usage constant regardless of file size.

This aligns with the "find first, read second" philosophy: locate what you need without loading everything upfront. For readers interested in maximum performance, the full notebook also includes a grep_files_os_based variant that shells out to ripgrep or grep, leveraging OS-level optimizations like memory-mapped I/O and SIMD instructions. In practice, this pattern (“search first, then read a range”) is one reason filesystem agents can feel surprisingly strong on focused corpora: the agent iteratively narrows the context instead of relying on a single-shot retrieval query.

Tool outputs as files: keeping big JSON out of the prompt

One of the fastest ways to blow up your context window is to return large JSON payloads from tools. Cursor’s approach is to write these results to files and let the agent inspect them on demand (often starting with tail).

That’s exactly why our folder structure includes a tool_outputs/<session_id>/ directory: it acts like an “evidence locker” for everything the agent did, without forcing those payloads into the current context.

{
 "ts_utc": "2026-01-27T12:41:12.135396+00:00",
 "tool": "arxiv_search_candidates",
 "input": "{'query': 'memgpt'}",
 "output": "content='[\\n {\\n \"arxiv_id\": \"2310.08560v2\",\\n \"entry_id\": \"http://arxiv.org/abs/2310.08560v2\",\\n \"title\": \"MemGPT: Towards LLMs as Operating Systems\",\\n \"authors\": \"Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G. Patil, Ion Stoica, Joseph E. Gonzalez\",\\n \"published\": \"2024-02-12\",\\n \"abstract\": ...msPnaMxOl8Pa'"
}

Putting it together: the agent toolset

Before we create the agent, we bundle the tools into a small, composable toolbox. This matches the broader trend: agents often perform better with a smaller tool surface, less choice paralysis (aka context confusion), fewer weird and overlapping tool schemas, and more reliance on proven filesystem workflows.

FS_TOOLS = [
 arxiv_search_candidates, # search arXiv for relevant research papers
 fetch_and_save_paper, # fetch paper text (PDF->text) and save to semantic/knowledge_base/<id>.md
 read_file, # read a file in full (use sparingly)
 tail_file, # read end of file first
 read_file_range, # read a specific line range
 conversation_to_file, # append conversation entries to episodic memory
 summarise_conversation_to_file, # save transcript + compact summary
 monitor_context_window, # estimate token usage
 list_papers, # list saved papers
 grep_files # grep-like search over files
]

The “filesystem-first” system prompt: policy beats cleverness

Filesystem tools alone aren’t enough, you also need a reading policy that keeps the agent's token usage efficient and grounded. This is the same reason CLAUDE.md, AGENTS.md, and SKILLS.md matter: they’re procedural memory that is applied consistently across sessions.

Key policies we encode below:

Store big artifacts on disk (papers, tool outputs, transcripts).
Prefer grep + range reads over full reads.
Use tail first for large files and logs.
Be explicit about what you actually read (grounding).

Below is the implementation of an agent using the langchain framework.

fs_agent = create_agent(
 model=f"openai:{os.getenv('OPENAI_MODEL', 'gpt-4o-mini')}",
 tools=FS_TOOLS,
 system_prompt=(
 "You are a conversational research ingestion agent.\n\n"
 "Core behavior:\n"
 "- When asked to find a paper: use arxiv_search_candidates, pick the best arxiv_id, "
 "then call fetch_and_save_paper to store the full text in semantic/knowledge_base/.\n"
 "- Papers/knowledge base live in semantic/knowledge_base/.\n"
 "- Conversations (transcripts) live in episodic/conversations/ (one file per run).\n"
 "- Summaries live in episodic/summaries/.\n"
 "- Conversation may be summarised externally; respect summary + transcript references.\n"
 ),
)

What the memory footprint looks like on disk

After running the agent, you end up with a directory layout that makes the agent’s “memory” tangible and inspectable. In your example, the agent produces:

episodic/conversations/fsagent_session_0010.md — the session transcript (episodic memory)
episodic/tool_outputs/fsagent_session_0010/*.json — tool results saved as files (evidence + replay)
semantic/knowledge_base/*.md — saved papers (semantic memory)

That is exactly the point of filesystem-first memory: the model doesn’t “remember” by magically retaining state; it “remembers” because it can re-open, search, and selectively read its prior artifacts.

This is also why so many teams keep rediscovering the same pattern: files are a simple abstraction, and agents are surprisingly good at using them.

Advantages of File Systems In AI Agents

In the previous section, we showed what a filesystem‑first memory harness looks like in practice: the agent writes durable artifacts (papers, tool outputs, transcripts) to disk, then “remembers” by searching and selectively reading only the parts it needs.

This approach works because it directly addresses two core constraints of LLMs: limited context windows and inherent statelessness. Once those constraints are handled, it becomes clear why file systems so often become the default interface for early agent systems.

Pretraining‑native interface: LLMs have ingested massive amounts of repos, docs, logs, and README‑driven workflows, so folders and files are a familiar operating surface.
Simple primitives, strong composition: A small action set (list/read/write/search) composes into sophisticated behavior without needing schemas, migrations, or query planning.
Token efficiency via progressive disclosure: Retrieve via search, then load a small slice (snippets, line ranges) instead of dumping entire documents into the prompt.
Natural home for artifacts and evidence: Transcripts, intermediate results, cached documents, and tool outputs fit cleanly as files and remain human‑inspectable.
Debuggable by default: You can open the directory and see exactly what the agent saved, what tools returned, and what the agent could have referenced.
Portability: A folder is easy to copy, zip, diff, version, and replay elsewhere, great for demos, reproducibility, and handoffs.
Low operational overhead: For PoCs and MVPs, you get persistence and structure without provisioning extra infrastructure.

In practice, filesystem memory excels when the workload is artifact‑heavy (research notes, paper dumps, transcripts), when you want a clear audit trail, and when iteration speed matters more than sophisticated retrieval. It also encourages good agent hygiene: write outputs down, cite sources, and load only what you need.

Disadvantages of Filesystems In AI Agents

But, unfortunately, it doesn’t end there. The same strengths that make files attractive, simplicity, relatively low cost, and fast implementation, can quickly become bottlenecks once you promote these systems into production, where they are expected to behave like a shared, reliable memory platform.

As soon as an agent moves beyond single-user prototypes into real-world scenarios, where concurrent reads and writes are the norm and robustness under load is non-negotiable, filesystems start to show their limits.

Weak concurrency guarantees by default: Multiple processes can overwrite or interleave writes unless you implement locking correctly. Even then, locking semantics vary across platforms and network filesystems.
No ACID transactions: You don’t get atomic multi-step updates, isolation between writers, or durable commit semantics without building them. Partial writes and mid-operation failures can leave memory in inconsistent states.
Search quality is usually brittle: Keyword/grep-style retrieval misses meaning, synonyms, and paraphrases.
Scaling becomes “death by a thousand files”: Directory bloat, fragmented artifacts, and expensive scans make performance degrade as memory grows, especially if you rely on repeated full-folder searches.
Indexing is DIY: The moment you want fast retrieval, deduplication, ranking, or recency weighting, you end up maintaining your own indexes and metadata stores (which, being honest here…is basically a database).
Metadata and schema drift: Agents inevitably accumulate extra fields (source URLs, timestamps, embeddings, tags). Keeping those consistent across files is harder than enforcing constraints in tables.
Poor multi-user / multi-agent coordination: Shared memory across agents means shared state. Without a central coordinator, you’ll hit race conditions, inconsistent views, and an unclear “source of truth.”
Harder auditing at scale: Files are human-readable, but reconstructing “what happened” across many runs and threads becomes messy without structured logs, timestamps, and queryable history.
Security and access control are coarse: Permissions are filesystem-level, not row-level. It’s hard to enforce “agent A can read X but not Y” without duplicating data or adding an auth layer.

The core pattern is that filesystem memory stays attractive until you need correctness under concurrency, semantic retrieval, or structured guarantees. At that point, you either accept the limitations (and keep the agent single-user/single-process) or you adopt a database.

Database For Agent Memory

By this point, most AI developers can see why filesystem first agent implementations are having a moment. It is a familiar interface, easy to prototype with, and our agents can “remember” by writing artifacts to disk and reloading them later via search plus selective reads. For a single developer on a laptop, that is often enough. But once we move beyond “it works on my laptop” and start supporting developers who ship to thousands or millions of users, memory stops being a folder of helpful files and becomes a shared system that has to behave predictably under load.

Databases were created for the exact moment when “a pile of files” stops being good enough because too many people and processes are touching the same data. One of the most-cited origin stories of the database dates to the Apollo era. IBM, alongside partners, built what became IMS to manage complex operational data for the program, and early versions were installed in 1968 at the Rockwell Space Division, supporting NASA. The point was not simply storage. It was coordination, correctness, and the ability to trust shared data while many activities were happening simultaneously.

That same production reality is what pushes agent memory toward databases today.

When agent memory must handle concurrent reads and writes, preserve an auditable history of what happened, support fast retrieval across many sessions, and enforce consistent updates, we want database guarantees rather than best-effort file conventions.

Oracle has been solving these exact problems since 1979, when we shipped the first commercial SQL database. The goal then was the same as now: make shared state reliable, portable, and trustworthy under load.

On that note, allow us to show how this can work in practice.

Database-first Research Assistant

In the filesystem first section, our Research Assistant “remembered” by writing artifacts to disk and reloading them later using cheap search plus selective reads. That is a great starting point. But when we want memory that is shared, queryable, and reliable under concurrent use, we need a different foundation.

In this iteration of our agent, we keep the same user experience and the same high-level job. Search arXiv, ingest papers, answer follow-up questions, and maintain continuity across sessions. The difference is that memory now lives in the Oracle AI Database, where we can make it durable, indexed, filterable, and safe for concurrent reads and writes. We also achieve a clean separation between two memory surfaces: structured history in SQL tables and semantic recall via vector search.

The result is what we call a MemAgent, an agent whose memory is not a folder of artifacts, but a queryable system. It is designed to support multi-threaded sessions, store full conversational history, store tool logs for debugging and auditing, and store a semantic knowledge base that can be searched by meaning rather than keywords.

Available tools for MemAgent

Before we wire up the agent loop, we need to define the tool surface that MemAgent can use to reason, retrieve, and persist knowledge. The design goal here is similar to the filesystem-first approach: keep the toolset small and composable, but shift the memory substrate from files to the database. Instead of grepping folders and reading line ranges, MemAgent uses vector similarity search to retrieve semantically relevant context, and it persists what it learns in a way that is queryable and reliable across sessions.

In practice, that means two things.

First, ingestion tools do not just “fetch” content; they also chunk and embed it so it becomes searchable later.
Second, retrieval tools are meaning-based rather than keyword-based, so the agent can find relevant passages even when the user paraphrases, uses synonyms, or asks higher-level conceptual questions.

The table below summarizes the minimal set of tools we expose to MemAgent and where each tool stores its outputs.

Tool	What it does
arxiv_search_candidates(query, k)	Searches arXiv for candidate papers
fetch_and_save_paper_to_kb_db(arxiv_id)	Fetches paper, chunks text, stores embeddings
search_knowledge_base(query, k)	Semantic search over stored papers
store_to_knowledge_base(text, metadata)	Manually store text with metadata

FSAgent and MemAgent can look similar from the outside because both can ingest papers, answer questions, and maintain continuity. The difference is what powers that continuity and how retrieval works when the system grows.

FSAgent relies on the operating system as its memory surface, which is great for iteration speed and human inspectability, but it typically relies on keyword-style discovery and file traversal. MemAgent treats memory as a database concern, which adds setup overhead, but unlocks indexed retrieval, stronger guarantees under concurrency, and richer ways to query and filter what the agent has learned.

Aspect	FSAgent (Filesystem)	MemAgent (Database)
Search	Keyword and grep	Semantic similarity
Persistence	Markdown files	SQL tables + vector indexes
Scalability	Directory traversal	Indexed queries
Query Language	Paths and regex	SQL + vector similarity
Setup Complexity	Minimal	Requires database runtime

Creating data stores with LangChain and Oracle AI Database

Before we start defining tables and vector stores, it is worth being explicit about the stack we are using and why. In this implementation, we are not building a bespoke agent framework from scratch.

We use LangChain as the LLM framework to abstract the agent loop, tool calling, and message handling, then pair it with a model provider for reasoning and generation, and with Oracle AI Database as the unified memory core that stores both structured history and semantic embeddings.

This separation is important because it mirrors how production agent systems are typically built. The agent logic evolves quickly, the model can be swapped, and the memory layer must remain reliable and queryable.

Think of this as the agent stack. Each layer has a clear job, and together they create an agent that is both practical to build and robust enough to scale.

Model provider (OpenAI): generates reasoning, responses, and tool decisions.
LLM framework (LangChain): provides the agent abstraction, tool wiring, and runtime orchestration.
Unified memory core (Oracle AI Database): stores durable conversational memory in SQL and semantic memory in vector indexes.

With that stack in place, the first step is simply to connect to the Oracle Database and initialize an embedding model. The database connection serves as the foundation for all memory operations, and the embedding model enables us to store and retrieve knowledge semantically through the vector store layer.

def connect_oracle(user, password, dsn="127.0.0.1:1521/FREEPDB1", program="langchain_oracledb_demo"):
 return oracledb.connect(user=user, password=password, dsn=dsn, program=program)

database_connection = connect_oracle(
 user="VECTOR",
 password="VectorPwd_2025",
 dsn="127.0.0.1:1521/FREEPDB1",
 program="devrel.content.filesystem_vs_dbs",
)

print("Using user:", database_connection.username)

embedding_model = HuggingFaceEmbeddings(
 model_name="sentence-transformers/paraphrase-mpnet-base-v2"
)

Next, we define the database schema to store our agent’s memory and prepare a clean slate for the demo. We separate memory into distinct tables so each type can be managed, indexed, and queried appropriately.

Installing the Oracle Database integration in the LangChain ecosystem is straightforward. You can add it to your environment with a single pip command:

pip install -U langchain-oracledb

Conversational history and logs are naturally tabular, while semantic and summary memory are stored in vector-backed tables through OracleVS. For reproducibility, we drop any existing tables from previous runs, making the notebook deterministic and avoiding confusing results when you re-run the walkthrough.

from langchain_oracledb.vectorstores import OracleVS
from langchain_oracledb.vectorstores.oraclevs import create_index
from langchain_community.vectorstores.utils import DistanceStrategy

CONVERSATIONAL_TABLE = "CONVERSATIONAL_MEMORY"
KNOWLEDGE_BASE_TABLE = "SEMANTIC_MEMORY"
LOGS_TABLE = "LOGS_MEMORY"
SUMMARY_TABLE = "SUMMARY_MEMORY"

ALL_TABLES = [
 CONVERSATIONAL_TABLE,
 KNOWLEDGE_BASE_TABLE,
 LOGS_TABLE,
 SUMMARY_TABLE
]

for table in ALL_TABLES:
 try:
 with database_connection.cursor() as cur:
 cur.execute(f"DROP TABLE {table} PURGE")
 except Exception as e:
 if "ORA-00942" in str(e):
 print(f" - {table} (not exists)")
 else:
 print(f" [FAIL] {table}: {e}")

database_connection.commit()

Create the vector stores and HNSW indexes

For this section, it is worth explaining what a “vector store” actually is in the context of agents. A vector store is a storage system that persists embeddings alongside metadata and supports similarity search, so the agent can retrieve items by meaning rather than keywords.

Instead of asking “which file contains this exact phrase”, the agent asks “which chunks are semantically closest to my question” and pulls back the best matches.

Under the hood, that usually means an approximate nearest neighbor index, because scanning every vector becomes prohibitively expensive as your knowledge base grows. HNSW is one of the most common indexing approaches for this style of retrieval.

In the code below, we create two vector stores using the langchain_oracledb module OracleVS, one for the knowledge base and one for summaries, both using cosine distance.

Second, it builds HNSW indexes so similarity search stays fast as memory grows, which is exactly what you want once your Research Assistant starts ingesting many papers and running over long-lived threads.

knowledge_base_vs = OracleVS(
 client=database_connection,
 embedding_function=embedding_model,
 table_name=KNOWLEDGE_BASE_TABLE,
 distance_strategy=DistanceStrategy.COSINE,
)

summary_vs = OracleVS(
 client=database_connection,
 embedding_function=embedding_model,
 table_name=SUMMARY_TABLE,
 distance_strategy=DistanceStrategy.COSINE,
)

def safe_create_index(conn, vs, idx_name):
 try:
 create_index(
 client=conn,
 vector_store=vs,
 params={"idx_name": idx_name, "idx_type": "HNSW"}
 )
 print(f" Created index: {idx_name}")
 except Exception as e:
 if "ORA-00955" in str(e):
 print(f" [SKIP] Index already exists: {idx_name}")
 else:
 raise

print("Creating vector indexes...")
safe_create_index(database_connection, knowledge_base_vs, "kb_hnsw_cosine_idx")
safe_create_index(database_connection, summary_vs, "summary_hnsw_cosine_idx")
print("All indexes created!")

Memory Manager

In the code below, we create a custom Memory manager. The Memory manager is the abstraction layer that turns raw database operations into “agent memory behaviours”. This is the part that makes the database-first agent easy to reason about.

SQL methods store and load conversational history by thread_id
Vector methods store and retrieve semantic memory by similarity search
Summary methods store compressed context and let us rotate the working set when we approach context limits

from langchain.tools import tool
from typing import List, Dict

class MemoryManager:
 """
 A simplified memory manager for AI agents using Oracle AI Database.
 """

 def __init__(self, conn, conversation_table: str, knowledge_base_vs, summary_vs, tool_log_table):
 self.conn = conn
 self.conversation_table = conversation_table
 self.knowledge_base_vs = knowledge_base_vs
 self.summary_vs = summary_vs
 self.tool_log_table = tool_log_table

 def write_conversational_memory(self, content: str, role: str, thread_id: str) -> str:
 thread_id = str(thread_id)
 with self.conn.cursor() as cur:
 id_var = cur.var(str)
 cur.execute(f"""
 INSERT INTO {self.conversation_table} (thread_id, role, content, metadata, timestamp)
 VALUES (:thread_id, :role, :content, :metadata, CURRENT_TIMESTAMP)
 RETURNING id INTO :id
 """, {"thread_id": thread_id, "role": role, "content": content, "metadata": "{}", "id": id_var})
 record_id = id_var.getvalue()[0] if id_var.getvalue() else None
 self.conn.commit()
 return record_id

 def load_conversational_history(self, thread_id: str, limit: int = 50) -> List[Dict[str, str]]:
 thread_id = str(thread_id)
 with self.conn.cursor() as cur:
 cur.execute(f"""
 SELECT role, content FROM {self.conversation_table}
 WHERE thread_id = :thread_id AND summary_id IS NULL
 ORDER BY timestamp ASC
 FETCH FIRST :limit ROWS ONLY
 """, {"thread_id": thread_id, "limit": limit})
 results = cur.fetchall()

 return [{"role": str(role), "content": content.read() if hasattr(content, 'read') else str(content)} for role, content in results]

 def mark_as_summarized(self, thread_id: str, summary_id: str):
 thread_id = str(thread_id)
 with self.conn.cursor() as cur:
 cur.execute(f"""
 UPDATE {self.conversation_table}
 SET summary_id = :summary_id
 WHERE thread_id = :thread_id AND summary_id IS NULL
 """, {"summary_id": summary_id, "thread_id": thread_id})
 self.conn.commit()
 print(f" Marked messages as summarized (summary_id: {summary_id})")

 def write_knowledge_base(self, text: str, metadata_json: str = "{}"):
 metadata = json.loads(metadata_json)
 self.knowledge_base_vs.add_texts([text], [metadata])

 def read_knowledge_base(self, query: str, k: int = 5) -> str:
 results = self.knowledge_base_vs.similarity_search(query, k=k)
 content = "\n".join([doc.page_content for doc in results])
 return f"""## Knowledge Base Memory: This are general information that is relevant to the question
### How to use: Use the knowledge base as background information that can help answer the question

{content}"""

 def write_summary(self, summary_id: str, full_content: str, summary: str, description: str):
 self.summary_vs.add_texts(
 [f"{summary_id}: {description}"],
 [{"id": summary_id, "full_content": full_content, "summary": summary, "description": description}]
 )
 return summary_id

 def read_summary_memory(self, summary_id: str) -> str:
 results = self.summary_vs.similarity_search(
 summary_id,
 k=5,
 filter={"id": summary_id}
 )
 if not results:
 return f"Summary {summary_id} not found."
 doc = results[0]
 return doc.metadata.get('summary', 'No summary content.')

 def read_summary_context(self, query: str = "", k: int = 5) -> str:
 results = self.summary_vs.similarity_search(query or "summary", k=k)
 if not results:
 return "## Summary Memory\nNo summaries available."

 lines = ["## Summary Memory", "Use expand_summary(id) to get full content:"]
 for doc in results:
 sid = doc.metadata.get('id', '?')
 desc = doc.metadata.get('description', 'No description')
 lines.append(f" - [ID: {sid}] {desc}")
 return "\n".join(lines)

Then we instantiate it:

memory_manager = MemoryManager(
 conn=database_connection,
 conversation_table=CONVERSATION_HISTORY_TABLE,
 knowledge_base_vs=knowledge_base_vs,
 tool_log_table=TOOL_LOG_TABLE,
 summary_vs=summary_vs
)

Creating the tools and agent

The database-first agent follows a simple, production-friendly pattern.

Persists every conversation turn as structured rows, including user and assistant messages with thread or run IDs and timestamps, so sessions are recoverable, traceable, and consistent across restarts.
Persists long-term knowledge in a vector-enabled store by chunking documents, generating embeddings, and storing them with metadata, so retrieval is semantic, ranked, and fast as the corpus grows.
Persists tool activity as first-class records that capture the tool name, inputs, outputs, status, errors, and key metadata, so agent behavior is inspectable, reproducible, and auditable.

On top of that, the agent actively manages context: it tracks token usage and periodically rolls older dialogue and intermediate state into durable summaries (and/or “memory” tables), so the working prompt stays small while the full history remains available on demand.

Ingest papers into the knowledge base vector store

This is the database-first equivalent of “fetch and save paper”. Instead of writing markdown files, we do three steps:

Load paper text from arXiv
Chunk it to respect the embedding model limits
Store chunks with metadata in the vector store, which gives us fast semantic search later

from datetime import datetime, timezone
from langchain_core.tools import tool
from langchain_community.document_loaders import ArxivLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter

@tool
def fetch_and_save_paper_to_kb_db(
 arxiv_id: str,
 chunk_size: int = 1500,
 chunk_overlap: int = 200,
) -> str:
 loader = ArxivLoader(
 query=arxiv_id,
 load_max_docs=1,
 doc_content_chars_max=None,
 )
 docs = loader.load()
 if not docs:
 return f"No documents found for arXiv id: {arxiv_id}"

 doc = docs[0]

 title = (
 doc.metadata.get("Title")
 or doc.metadata.get("title")
 or f"arXiv {arxiv_id}"
 )

 entry_id = doc.metadata.get("Entry ID") or doc.metadata.get("entry_id") or ""
 published = doc.metadata.get("Published") or doc.metadata.get("published") or ""
 authors = doc.metadata.get("Authors") or doc.metadata.get("authors") or ""

 full_text = doc.page_content or ""
 if not full_text.strip():
 return f"Loaded arXiv {arxiv_id} but extracted empty text (PDF parsing issue)."

 splitter = RecursiveCharacterTextSplitter(
 chunk_size=chunk_size,
 chunk_overlap=chunk_overlap,
 )
 chunks = splitter.split_text(full_text)

 ts_utc = datetime.now(timezone.utc).isoformat()
 metadatas = []
 for i in range(len(chunks)):
 metadatas.append(
 {
 "source": "arxiv",
 "arxiv_id": arxiv_id,
 "title": title,
 "entry_id": entry_id,
 "published": str(published),
 "authors": str(authors),
 "chunk_id": i,
 "num_chunks": len(chunks),
 "ingested_ts_utc": ts_utc,
 }
 )

 knowledge_base_vs.add_texts(chunks, metadatas)

 return (
 f"Saved arXiv {arxiv_id} to {KNOWLEDGE_BASE_TABLE}: "
 f"{len(chunks)} chunks (title: {title})."
 )

We create two more tools below:

search_knowledge_base(query, k=5): Runs a semantic similarity search over the database-backed knowledge base and returns the top k most relevant chunks, so the agent can retrieve context by meaning rather than exact keywords.
store_to_knowledge_base(text, metadata_json="{}"): Stores a new piece of text into the knowledge base and attaches metadata (as JSON), which gets embedded and indexed so it becomes searchable in future queries.

import os
from langchain.tools import tool

@tool
def search_knowledge_base(query: str, k: int = 5) -> str:
 return memory_manager.read_knowledge_base(query, k)

@tool
def store_to_knowledge_base(text: str, metadata_json: str = "{}") -> str:
 memory_manager.write_knowledge_base(text, metadata_json)
 return "Successfully stored text to knowledge base."

Now we build the LangChain agent using the database-first tools.

from langchain.agents import create_agent

MEM_AGENT = create_agent(
 model=f"openai:{os.getenv('OPENAI_MODEL', 'gpt-4o-mini')}",
 tools=[search_knowledge_base, store_to_knowledge_base, arxiv_search_candidates, fetch_and_save_paper_to_kb_db],
)

Result Comparison: FSAgent vs MemAgent: End-to-End Benchmark (Latency + Quality)

At this point, the difference between a filesystem agent and a database-backed agent should feel less like a philosophical debate and more like an engineering trade-off. Both approaches can “remember” in the sense that they can persist state, retrieve context, and answer follow-up questions. The real test is what happens when you leave the tidy laptop demo and hit production realities: larger corpora, fuzzier queries, and concurrent workloads.

To make that concrete, we ran an end-to-end benchmark and measured the full agent loop per query—retrieval, context assembly, tool calls, model invocations, and the final answer—across three scenarios:

Small-corpus retrieval: a tight, keyword-friendly dataset to validate baseline retrieval and answer synthesis with minimal context.
Large-corpus retrieval: a larger dataset with more paraphrase variability to stress retrieval quality and context efficiency at scale.
Concurrent write integrity: a multi-worker stress test to evaluate correctness under simultaneous reads/writes (integrity, race conditions, throughput).

FSAgent vs MemAgent: End-to-End Benchmark (Latency + Quality)

From the result shown in the image above, two conclusions immediately stand out: latency and answer quality.

In our run, MemAgent generally finished faster end-to-end than FSAgent. That might sound counterintuitive if you assume “database equals overhead,” and sometimes it does.

But the agent loop is not dominated by raw storage primitives. It is dominated by how quickly you can find the right information and how little unnecessary context you force into the model, also known as context engineering. Semantic retrieval tends to return fewer, more relevant chunks (subject to tuning of the retrieval pipelines), which means less scanning, less paging through files, and fewer tokens burned on irrelevant text.

In this particular run, both agents produced similar-quality answers. That is not surprising. When the questions are retrieval-friendly and the corpus is small enough, both approaches can find the right passages. FSAgent gets there through keyword search and careful reading. MemAgent gets there through similarity search over embedded chunks. Different roads, similar destination.

And I think it’s worth zooming in on one nuance here. When the information to traverse is minimal in terms of character length and the query is keyword-friendly, the retrieval quality of both agents tends to converge. At that scale, “search” is barely a problem, so the dominant factor becomes the model’s ability to read and synthesise, not the retrieval substrate. The gap only starts to widen when the corpus grows, the wording becomes fuzzier, and the system must retrieve reliably under real-world constraints such as noise, paraphrases, and concurrency. Which it eventually does.

About the “LLM-as-a-Judge” metric

We also scored answers using an LLM-as-a-judge prompt. It is a pragmatic way to get directional feedback when you do not have labeled ground truth, but it is not a silver bullet. Judges can be sensitive to prompt phrasing, can over-reward fluency, and can miss subtle grounding failures.

If you are building this for production, treat LLM judging as a starting signal, not the finish line. The more reliable approach is a mix of:

Reference-based evaluation when you have ground truth, such as rubric grading, exact match, or F1-style scoring.
Retrieval-aware evaluation when context matters, such as context precision and recall, answer faithfulness, and groundedness. Tracing plus evaluation tooling so you can connect failures to the specific retrievals, tool calls, and context assembly decisions that caused them.

Even with a lightweight judge, the directional story remains consistent. As retrieval becomes more difficult and the system becomes busier, database-backed memory tends to perform better.

Large Corpus Benchmark: Why the gap widens as data grows

The large-corpus test is designed to stress the exact weakness of keyword-first memory. We intentionally made the search problem harder by growing the corpus and making the queries less “exact match.”

FSAgent with a concatenated corpus:
When you merge many papers into large markdown files, FSAgent becomes dependent on grep-style discovery followed by paging the right sections into the context window. It can work, but it gets brittle as the corpus grows:

If the user paraphrases or uses synonyms, exact keyword matches can fail.
If the keyword is too common, you get too many hits, and the agent has to sift through them manually.
When uncertain, the agent often loads larger slices “just in case,” which increases token count, latency, and the risk of context dilution.

MemAgent with chunked, embedded memory:
Chunking plus embeddings makes retrieval more forgiving and more stable:

The user does not need to match the source phrasing exactly.
The agent can fetch a small set of high-similarity chunks, keeping context tight.
Indexed retrieval remains predictable as memory grows, rather than requiring repeated scans of files.

The narrative takeaway is simple. Filesystems feel great when the corpus is small and the queries are keyword-friendly. As the corpus grows and the questions get fuzzier, semantic retrieval becomes the differentiator, and database-backed memory becomes the more dependable default.

The quality gap widens with scale. On a handful of documents, grep can brute-force its way to a reasonable answer: the agent finds a keyword match, pulls surrounding context, and responds.

But scatter the same information across hundreds of files, and keyword search starts missing the forest for the trees. It returns too many shallow hits or none when the user's phrasing doesn't match the source text verbatim. Semantic search, by contrast, surfaces conceptually relevant chunks even when the vocabulary differs. The result isn't just faster retrieval, it's more coherent answers with fewer hallucinated gaps. This is evident in our LLM judge evaluation on the large corpus benchmark, where FSAgent achieved a score of 29.7% while MemAgent reached 87.1%.

Concurrency Test: What production teaches you very quickly

We find that the real breaking point for filesystem memory is rarely retrieval. It is concurrency.

We ran three versions of the same workload under concurrent writes:

Filesystem without locking, where multiple workers append to the same file.
Filesystem with locking, where writes are guarded by file locks.
Oracle AI Database with transactions, where multiple workers write rows under ACID guarantees.

Then we measured two things:

Integrity, meaning, did we get the expected number of entries with no corruption?
Execution time, meaning how long the batch took end-to-end.

What we observed maps to what many teams discover the hard way.

Naive filesystem writes can be fast and still be wrong. Without locking, concurrent writes conflict with each other. You might get good throughput and still lose memory entries. If your agent’s “memory” is used for downstream reasoning, silent loss is not a performance issue. It is a correctness failure.

Locking fixes integrity, but now correctness is your job. With explicit locking, you can make filesystem writes safe. But you inherit the complexity. Lock scope, lock contention, platform differences, network filesystem behavior, and failure recovery all become part of your agent engineering work.

Databases make correctness the default. Transactions and isolation are exactly what databases were designed for. Yes, there is overhead. But the key difference is that you are not bolting correctness on after a production incident. You start with a system whose job is to protect the shared state.

And of course, you can take the file-locking approach, add atomic writes, build a write-ahead log, introduce retry and recovery logic, maintain indexes for fast lookups, and standardise metadata so you can query it reliably.

Eventually, though, you will realise you have not “avoided” a database at all.

You have just rebuilt one, only with fewer guarantees and more edge cases to own.

Conclusion: Is there a happy medium for AI Developers?

This isn’t a religious war between “files” and “databases.” It’s a question of what you’re optimizing for—and which failure modes you’re willing to own. If you’re building single-user or single-writer prototypes, filesystem memory is a great default. It’s simple, transparent, and fast to iterate on. You can open a folder and see exactly what the agent saved, diff it, version it, and replay it with nothing more than a text editor.

If you’re building multi-user agents, background workers, or anything you plan to ship at scale, a database-backed memory store is a safer foundation at that stage. At that stage, concurrency, integrity, governance, access control, and auditability matter more than raw simplicity. A practical compromise is a hybrid design: keep file-like ergonomics for artifacts and developer workflows, but store durable memory in a database that can enforce correctness.

And if you insist on filesystem-only memory in production, treat locking, atomic writes, recovery, indexing, and metadata discipline as first-class engineering work. Because the moment you do that seriously, you’re no longer “just using files”—you’re rebuilding a database.

One last trap worth calling out: polyglot persistence.

Many AI stacks drift into an anti-pattern: a vector DB for embeddings, a NoSQL DB for JSON, a graph DB for relationships, and a relational DB for transactions. Each product is “best at its one thing,” until you realize you’re operating four databases, four security models, four backup strategies, four scaling profiles, and four cascading failure points.

Coordination becomes the tax. You end up building glue code and sync pipelines just to make the system feel unified to the agent. This is why converged approaches matter in agent systems: production memory isn’t only about storing vectors—it’s about storing operational history, artifacts, metadata, and semantics under one consistent set of guarantees.

For AI Developers, your application acts as an integration layer for multiple storage engines, each with different access patterns and operational semantics. You end up building glue code, sync pipelines, and reconciliation logic just to make the system feel unified to the agent.

Of course, production data is inherently heterogeneous. You will inevitably deal with structured, semi-structured, unstructured text, embeddings, JSON documents, and relationship-heavy data.

The point is not that “one model wins”.

The point is that when you understand the fundamentals of data management, reliability, indexing, governance, and queryability, you want a platform that can store and retrieve these forms without turning your AI infrastructure into a collection of loosely coordinated subsystems.

This is the philosophy behind Oracle’s converged database approach, which is designed to support multiple data types and workloads natively within a single engine. In the world of agents, that becomes a practical advantage because we can use Oracle as the unified memory core for both operational memory (SQL tables for history and logs) and semantic memory (vector search for retrieval).

Frequently Asked Questions

What is AI Agent memory? AI agent memory is the set of system components and techniques that enable an AI agent to store, recall, and update information over time. Because LLMs are inherently stateless—they have no built-in ability to remember previous sessions—agent memory provides the persistence layer that allows agents to maintain continuity across conversations, learn from past interactions, and adapt to user preferences.
Should I use a filesystem or a database for an AI agent's memory? It depends on your use case. Filesystems excel at single-user prototypes, artifact-heavy workflows, and rapid iteration—they're simple, transparent, and align with how LLMs naturally operate. Databases become essential when you need concurrent access, ACID transactions, semantic retrieval, or shared state across multiple agents or users. Many production systems use a hybrid approach: file-like interfaces for agent interaction, with database guarantees underneath.
How do I build an AI agent with long-term memory? Start by separating memory types: working memory (current context), semantic memory (knowledge base), episodic memory (interaction history), and procedural memory (behavioral rules). Implement storage: a filesystem for prototypes and a database for production. Add retrieval tools that the agent can call. Build a summarization to compress the old context. Test with multi-session scenarios where the agent must recall information from previous conversations.
What are semantic, episodic, and procedural memory in AI agents? These terms, borrowed from cognitive science, describe different types of agent memory. Semantic memory stores durable knowledge and facts (like saved documents or reference materials). Episodic memory captures experiences and interaction history (conversation transcripts, tool outputs). Procedural memory encodes how the agent should behave—instructions, rules, files like CLAUDE.md, and learned workflows that shape behavior across sessions.
What is the best database for AI applications? The best database depends on your requirements. For AI agent memory specifically, you need: vector search capability for semantic retrieval, SQL or structured queries for history and metadata, ACID transactions if multiple agents share state, and scalability as your memory corpus grows. Converged databases that combine these capabilities—like Oracle AI Database—reduce operational complexity versus running separate specialized systems.

How I Added Memory to an AI Agent Using Spring AI and Oracle AI Database

Wojtek Pluta — Tue, 14 Apr 2026 18:17:32 +0000

Practical guide with a sample app for adding episodic, semantic, and procedural memory to an AI agent using Spring AI and a single Oracle AI Database instance.

This post shows how to build three types of persistent memory — episodic (chat history), semantic (domain knowledge via hybrid search), and procedural (tool calls) — using Spring AI and a single Oracle AI Database instance. Here's the code: GitHub repo

Key Takeaways

LLMs forget everything between sessions. Episodic, semantic, and procedural memory fix that — chat history, domain knowledge retrieval, and actionable tool calls, all persisted in the database.
One database handles it all. Oracle AI Database stores chat history, runs hybrid vector search, and hosts the application tables — no need to bolt on a separate vector database or search engine.
Hybrid search beats pure vector search. Combining dense embeddings with keyword matching (fused via Reciprocal Rank Fusion) means the agent finds documents by meaning and by exact terms like order IDs.
Embeddings stay in the database. A loaded ONNX model computes embeddings on insert — no external embedding API calls, no extra infrastructure.
Agent memory doesn't have to be complicated. Two advisors, six tools backed by real database tables, one database, and the LLM stops forgetting.

Why This Matters

Every LLM has the same problem: it forgets everything the moment the conversation ends, sometimes even during long conversations. Spend twenty minutes explaining your project setup, your constraints, your preferences — and it nails the answer. Close the tab, open a new session, and it greets you like a stranger. All that context, gone.

If you want to build an AI agent — one that remembers context, understands your domain, and can take action — you need to give it memory. Practical memory: capturing what users say, retrieving learned facts and executing real workflows backed by database queries.

This post walks through a proof of concept that does exactly that. Three types of memory, one database, and minimal code.

What You'll Learn

How to implement episodic, semantic, and procedural memory for an AI agent using Spring AI advisors and @Tool methods
How to use Oracle AI Database Hybrid Vector Indexes (vector and keyword search fused with Reciprocal Rank Fusion) for semantic retrieval
How to compute embeddings in-database with a loaded ONNX model — no external embedding API calls
How to wire it all together with one database, one connection pool, and minimal configuration

Architecture Overview

System architecture for a memory-enabled AI assistant using Streamlit, Spring Boot, Oracle AI Database 26ai, Ollama, and @Tool methods.

The agent runs on Spring Boot with Spring AI, with Ollama handling local chat inference (qwen2.5). Oracle AI Database 26ai stores all three memory types: a relational table for chat history (episodic), a hybrid vector index for domain knowledge retrieval (semantic), and application tables queried by @Tool methods (procedural). Embeddings are computed in-database by a loaded ONNX model (all-MiniLM-L12-v2), eliminating the need for external embedding API calls. A Streamlit frontend provides a simple web UI.

Both advisors and all six tools run on every request. The agent simultaneously remembers what you said, retrieves relevant knowledge, and executes tasks — all from a single Oracle Database instance. No second database. One connection pool, one set of credentials, one system to monitor.

Prerequisites

Java 21
Gradle 8.14
Oracle AI Database 26ai (container or instance)
Ollama with the qwen2.5 model pulled
Python 3.x with Streamlit (optional, for the web UI)
The ONNX model file (all_MiniLM_L12_v2.onnx) for in-database embeddings

Step-by-Step Guide

Step 1: Set Up the Oracle AI Database and Hybrid Vector Index

Start an Oracle AI Database instance, then run the one-time setup script to load the ONNX embedding model and create the hybrid vector index. This enables in-database embeddings and combined vector and keyword search.

-- Load the ONNX model for in-database embeddings
BEGIN
  DBMS_VECTOR.LOAD_ONNX_MODEL(
    directory  => 'DM_DUMP',
    file_name  => 'all_MiniLM_L12_v2.onnx',
    model_name => 'ALL_MINILM_L12_V2'
  );
END;
/

-- Create a hybrid index: vector similarity + Oracle Text keyword search
CREATE HYBRID VECTOR INDEX POLICY_HYBRID_IDX
ON POLICY_DOCS(content)
PARAMETERS('MODEL ALL_MINILM_L12_V2 VECTOR_IDXTYPE HNSW');

Once the index is created, embeddings are computed automatically on insert — no external embedding API calls required.

Step 2: Define Procedural Memory with @Tool Methods

Procedural memory is implemented as @Tool-annotated methods in a Spring component. These methods execute real database queries via JPA, which the LLM can call when it decides a task requires action, not just an answer. The @Tool description tells the LLM when to use each method, and @ToolParam defines the inputs.

@Tool(description = "Look up the status of a customer order by its order ID. " +
        "Returns the current status including shipping information.")
public String lookupOrderStatus(
        @ToolParam(description = "The order ID to look up, e.g. ORD-1001") String orderId) {
    // Fetches order from DB via JPA, returns formatted status string
}

@Tool(description = "Initiate a product return for a given order. " +
        "Validates the order exists, checks that it is in DELIVERED status, " +
        "and verifies the return is within the 30-day return window.")
public String initiateReturn(
        @ToolParam(description = "The order ID to return") String orderId,
        @ToolParam(description = "The reason for the return") String reason) {
    // Validates order exists, checks DELIVERED status and 30-day window, updates status via JPA
}

The full class has six tools: getCurrentDateTime, listOrders, lookupOrderStatus, initiateReturn, escalateToSupport, and listSupportTickets. The LLM decides when to act; the Java methods define how.

Step 3: Wire the Controller with Advisors and Tools

The controller builds a single ChatClient with two advisors and six tools. MessageChatMemoryAdvisor handles episodic memory by loading the last 100 messages for the current conversation from a relational table and persisting each new exchange. RetrievalAugmentationAdvisor, with a custom OracleHybridDocumentRetriever, handles semantic memory by calling DBMS_HYBRID_VECTOR.SEARCH to run vector and keyword search in parallel, fused with Reciprocal Rank Fusion (RRF). The tools are registered via .defaultTools(agentTools).

@RestController
@RequestMapping("/api/v1/agent")
public class AgentController {

    public AgentController(ChatClient.Builder builder,
                           JdbcChatMemoryRepository chatMemoryRepository,
                           JdbcTemplate jdbcTemplate,
                           AgentTools agentTools) {
        // Builds a ChatClient with:
        //   - MessageChatMemoryAdvisor (episodic: last 100 messages per conversation)
        //   - RetrievalAugmentationAdvisor + OracleHybridDocumentRetriever (semantic: hybrid search)
        //   - AgentTools via .defaultTools() (procedural: 6 @Tool methods)
        //   - System prompt defining the agent persona and tool usage rules
    }

    @PostMapping("/chat")
    public ResponseEntity<String> chat(
            @RequestBody String message,
            @RequestHeader("X-Conversation-Id") String conversationId) {
        // Sends message to ChatClient with conversation ID, returns LLM response
    }

    @PostMapping("/knowledge")
    public ResponseEntity<String> addKnowledge(@RequestBody String content) {
        // Inserts text into POLICY_DOCS table via JDBC (hybrid index handles embedding)
    }
}

All three memory types run on every request. The agent simultaneously remembers what you said, retrieves relevant knowledge, and executes tasks.

Step 4: Implement the Hybrid Document Retriever

The custom OracleHybridDocumentRetriever implements Spring AI's DocumentRetriever interface and calls DBMS_HYBRID_VECTOR.SEARCH via JDBC. It passes a JSON parameter specifying the hybrid index, the RRF scorer, and a keyword match clause, bypassing OracleVectorStore entirely for retrieval.

Why hybrid instead of pure vector search? Dense embeddings capture meaning — a query about "return policy" can match documents about refunds and exchanges. But they're weaker on exact terms: a query for "ORD-1001" performs poorly because embeddings encode semantics, not keywords. Hybrid search addresses both: the vector side captures meaning, the keyword side handles exact matches, and RRF merges the result sets by rank position.

Step 5: Run the Application

Start the Oracle DB container, install Ollama, pull the chat model, run the Spring Boot backend with the local profile, and optionally start the Streamlit UI.

The assistant recalls the customer’s name, prior issue, and order details to continue support without repeating context.

Optionally, quick test with cURL:

curl -X POST http://localhost:8080/api/v1/agent/chat \
  -H "Content-Type: text/plain" \
  -H "X-Conversation-Id: test-1" \
  -d "What orders do I have?"

The agent will remember your name and details, or use procedural memory (the listOrders tool) to query the database and return the demo orders. Try "What is your return policy?" to see semantic memory (hybrid search over policy documents) in action. Then type "My name is Victor" followed later by "What's my name?" to test episodic memory.

Frequently Asked Questions

Why does the agent need three types of memory instead of just chat history?
Chat history (episodic memory) only covers what was said in the conversation. Semantic memory lets the agent retrieve domain knowledge — like return policies or shipping rules — that was never mentioned in chat. Procedural memory lets it take actions, such as looking up an order or initiating a return, by calling tool methods backed by real database queries.

Why use hybrid search instead of plain vector similarity?
Pure vector search matches by meaning, which works well for natural-language questions but struggles with exact terms like product codes or order IDs. Hybrid search runs vector and keyword search in parallel and merges the results by rank position (Reciprocal Rank Fusion), so the agent finds relevant documents whether the match is semantic, lexical, or both.

Do I need a separate vector database to build this?
No. Oracle AI Database 26ai supports relational tables, hybrid vector indexes, and full-text search in a single instance. The POC uses one connection pool and one set of credentials for chat history, vector retrieval, and all application data.

How are the embeddings generated?
An ONNX model (all-MiniLM-L12-v2) is loaded directly into Oracle AI Database. Embeddings are computed automatically whenever a row is inserted into the indexed table — no external API calls and no separate embedding service required.

What are the limitations?
This is a proof of concept. There's no authentication, no rate limiting, and no streaming responses. It demonstrates the architecture and approach — production use would require hardening those areas.

Next Steps

Author

Victor Martin Alvarez – Senior Principal Product Manager, Oracle AI DatabaseBuilding AI-powered applications with Oracle AI Database and Spring AI.LinkedIn

Build an Ultra-Lightweight, Local-First AI Assistant with Persistent Memory

Wojtek Pluta — Tue, 14 Apr 2026 14:16:59 +0000

Key Takeaways

PicoOraClaw is a lightweight, offline AI assistant with local inference via Ollama.
Oracle AI Database stores sessions, memories, transcripts, prompts, and state with durable ACID-backed persistence.
Semantic recall happens in the database using ONNX embeddings and vector search, removing the need for an external embedding API.
The same project runs locally for development and can move to Oracle Cloud Infrastructure (OCI) when you need a managed deployment.

Local-First AI Assistant with Built-In Memory

If you want to build an AI assistant that runs locally, retains meaningful context, and can move to the cloud without rearchitecting the stack, PicoOraClaw is a strong starting point. It pairs a lightweight Go runtime with local inference via Ollama and uses Oracle AI Database as the persistent memory layer.

This matters for developers building edge AI systems, private assistants, or local-first prototypes. Instead of stitching together separate services for storage, embeddings, and retrieval, you can keep memory, state, and semantic recall within Oracle AI Database while still running a lightweight local runtime.

What is PicoOraClaw?

PicoOraClaw is a fork of PicoClaw that keeps the runtime lightweight, uses Ollama as the default inference backend, and adds Oracle AI Database for persistent memory and state.

PicoClaw is an independent open-source project initiated by Sipeed, written entirely in Go from scratch - not a fork of OpenClaw, NanoBot, or any other project.

The result is a developer-friendly architecture for assistants that retain meaningful context and retrieve it semantically, rather than relying on keyword matching. PicoOraClaw targets use cases such as edge AI, IoT, private assistants, and local-first developer workflows, where a small footprint and persistent context matter more than a cloud-only approach. See the PicoOraClaw repository.

Features

Lightweight Go runtime for local and edge-friendly assistant workflows
Oracle AI Database-backed memory, state, and semantic recall
Ollama as the default local inference backend
Support for multiple LLM providers including OpenRouter, Anthropic, OpenAI, Gemini, DeepSeek, Groq, and Zhipu
Default: Oracle AI Database Free with Oracle AI Vector Search for semantic memory
Optional Autonomous AI Database path for managed cloud deployment
Graceful file-based fallback when Oracle is unavailable

Why Choose PicoOraClaw vs. Standard PicoClaw?

If you're already familiar with PicoClaw, PicoOraClaw adds a more complete memory layer for developers who need durable context and semantic recall.

Oracle AI Database as the persistent backend for memories, sessions, transcripts, state, notes, prompts, and configuration
In-database ONNX embeddings and vector search for semantic memory using VECTOR_EMBEDDING() and VECTOR_DISTANCE()
Ollama as the default local LLM backend with no cloud dependency
One-click OCI deployment with Oracle AI Database Free, Ollama, and the PicoOraClaw gateway
Optional OCI Generative AI integration through the included oci-genai proxy
oracle-inspect CLI support for inspecting what the assistant stores without writing SQL

Features of PicoOraClaw

What PicoOraClaw enables:

Unified Memory Core - PicoOraClaw uses Oracle AI Database to store sessions, transcripts, notes, prompts, configuration, and long-term memories in a single persistent system. The database is the memory substrate for long-running, context-aware assistant behavior.
Build Fast with Modern APIs - Get started locally with a lightweight runtime, Ollama for local inference, and Oracle AI Database Free for semantic memory.
A Robust Scaling Path - Start locally, keep the same overall architecture, and move to OCI later when you need a managed environment.

Installation - Quick Start (in 5 minutes!)

For the fastest path to a working setup, use the PicoOraClaw one-command installer. It clones, configures, and runs the application in a single step:

curl -fsSL https://raw.githubusercontent.com/oracle-devrel/oracle-ai-developer-hub/refs/heads/main/apps/picooraclaw/install.sh | bash

To control the workspace path, clone the Oracle DevRel repository directly and build from the PicoOraClaw app directory.

Follow the steps below:

Prerequisites

Go 1.24+
Ollama
Docker (for Oracle Database Free)

Step 1: Build

Clone the Oracle DevRel repository, navigate to the PicoOraClaw app folder, and build the binary:

git clone https://github.com/oracle-devrel/oracle-ai-developer-hub.git
cd oracle-ai-developer-hub/apps/picooraclaw
make build

Step 2: Initialize

Initialize the application so it creates the local configuration and working directories:

./build/picooraclaw onboard

Step 3: Start Ollama and pull a model

Ollama is the default and recommended LLM backend for private local inference with no API keys and no cloud dependency.

# Install Ollama if needed: https://ollama.com/download
ollama pull qwen3:latest

Step 4: Configure for Ollama

Edit ~/.picooraclaw/config.json so PicoOraClaw points at your Ollama instance and model:

{
  "agents": {
    "defaults": {
      "provider": "ollama",
      "model": "qwen3:latest",
      "max_tokens": 8192,
      "temperature": 0.7
    }
  },
  "providers": {
    "ollama": {
      "api_key": "",
      "api_base": "http://localhost:11434/v1"
    }
  }
}

Step 5: Test semantic memory

Once the binary, config, and model are ready, start the assistant and test local conversations:

# One-shot
./build/picooraclaw agent -m "Hello!"

# Interactive mode
./build/picooraclaw agent

At this stage, you have a working local AI assistant with no cloud dependency.

The default LLM backend is Ollama, with an optional alternative for using OCI-hosted models. See oci-genai/README.md for related documentation.

The oci-genai module provides OCI Generative AI as an optional backend for PicoOraClaw. It runs a local OpenAI-compatible proxy that authenticates with OCI using your ~/.oci/config credentials and forwards requests to the OCI GenAI inference endpoint.

Deploying to Oracle Cloud (one-click procedure)

Click here to deploy to Oracle Cloud

This deployment provisions:

an OCI Compute instance
Ollama with a model preloaded for CPU inference
Oracle AI Database Free by default, with an optional Autonomous AI Database path
the PicoOraClaw gateway as a systemd service

You can start locally, keep the same overall architecture, and move to OCI when you need a managed environment.

After deployment, use these commands to verify setup, start chatting, and check gateway health:

# Check setup progress
ssh opc@<public_ip> -t 'tail -f /var/log/picooraclaw-setup.log'

# Start chatting
ssh opc@<public_ip> -t picooraclaw agent

# Check gateway health
curl http://<public_ip>:18790/health

Adding Oracle AI Vector Search

Oracle AI Database provides persistent storage, semantic memory and recall, and crash-safe ACID transactions, with an optional file-based storage mode.

Simply run the setup script:

./scripts/setup-oracle.sh [optional-password]

This script performs the following steps:

Pulls and starts the Oracle AI Database Free container
Waits for the database to be ready
Creates the picooraclaw database user with the required grants
Patches ~/.picooraclaw/config.json with the Oracle connection settings
Runs picooraclaw setup-oracle to initialize the schema and load the ONNX embedding model

This step gives the assistant durable semantic memory. Instead of relying on local files or ephemeral process state, PicoOraClaw persists and retrieves meaning-based context directly through Oracle AI Vector Search.

Expected output when setup is complete:

── Step 4/4: Schema + ONNX model ─────────────────────────────────────────
  Running picooraclaw setup-oracle...
✓ Connected to Oracle AI Database
✓ Schema initialized (8 tables with PICO_ prefix)
✓ ONNX model 'ALL_MINILM_L12_V2' already loaded
✓ VECTOR_EMBEDDING() test passed
✓ Prompts seeded from workspace

════════════════════════════════════════════════════════
  Oracle AI Database setup complete!
  Test with:
    ./build/picooraclaw agent -m "Remember that I love Go"
    ./build/picooraclaw agent -m "What language do I like?"
    ./build/picooraclaw oracle-inspect
════════════════════════════════════════════════════════

Test semantic memory

Use the following commands to test semantic memory:

# Store a fact
./build/picooraclaw agent -m "Remember that my favorite language is Go"

# Recall by meaning (not keywords)
./build/picooraclaw agent -m "What programming language do I prefer?"

The second command finds the stored memory via cosine similarity on 384-dimensional vectors rather than exact keyword matching.

Inspecting Oracle Data with oracle-inspect

A useful operational feature is oracle-inspect, a CLI tool that lets you inspect stored data without writing SQL.

picooraclaw oracle-inspect [table] [options]

These are the tables:

memories, sessions, transcripts, state, notes, prompts, config, meta

These are the options:

-n <limit> max rows (default 20), -s <text> semantic search (memories only)

To list all memories:

./build/picooraclaw oracle-inspect memories

You can also perform semantic search over memories:

./build/picooraclaw oracle-inspect memories -s "what does the user like to program in"

This is a meaningful developer benefit. Oracle-backed memory is inspectable, debuggable, and operationally visible. You can understand what the assistant stores without building a separate admin layer.

Overview dashboard

Run the following command to view an overview dashboard of stored data:

./build/picooraclaw oracle-inspect

Running the command with no arguments gives you a summary view across tables, recent memory entries, transcripts, sessions, state, notes, prompts, and schema metadata.

=============================================================
  PicoOraClaw Oracle AI Database Inspector
=============================================================

  Table                  Rows
  ─────────────────────  ────
  Memories                  20  ████████████████████
  Sessions                   4  ████
  Transcripts                6  ██████
  State                      8  ████████
  Daily Notes                3  ███
  Prompts                    4  ████
  Config                     2  ██
  Meta                       1  █
  ─────────────────────  ────
  Total                     48

  Tip: Run 'picooraclaw oracle-inspect <table>' for details
       Run 'picooraclaw oracle-inspect memories -s "query"' for semantic search

List all memories

Run the following command to list all stored memories:

./build/picooraclaw oracle-inspect memories

All Memories
─────────────────────────────────────────────────────────

ID: faffd019  Vector: yes
Created: 2026-02-19 04:13  Importance: 0.9  Category: preference  Accessed: 0x
Content: User prefers Oracle Database as the primary database. They work at Oracle
and prefer Oracle AI Vector Search for embeddings.

ID: 0e39036f  Vector: yes
Created: 2026-02-19 04:13  Importance: 0.8  Category: preference  Accessed: 0x
Content: Go is the user's primary programming language. They use Go 1.24 and target
embedded Linux devices (RISC-V, ARM64, x86_64).

Semantic search over memories

The following example shows how to perform semantic search over stored memories:

./build/picooraclaw oracle-inspect memories -s "what does the user like to program in"

Semantic Search: "what does the user like to program in"
─────────────────────────────────────────────────────────

[ 61.3% match]  ID: 383ff5d3
Created: 2026-02-16 06:13  Importance: 0.7  Category: preference  Accessed: 0x
Content: I prefer Python and Go for programming

[ 60.7% match]  ID: 0e74a94c
Created: 2026-02-18 02:20  Importance: 0.7  Category: preference  Accessed: 0x
Content: my favorite programming language is Go

For deeper inspection of sessions, transcripts, notes, config, prompts, and schema metadata, see the PicoOraClaw app in the Oracle DevRel repository.

Inspect sessions

You can inspect stored chat sessions using the following command:

./build/picooraclaw oracle-inspect sessions

Chat Sessions
─────────────────────────────────────────────────────────

Session: discord:dev-channel
Created: 2026-02-19 04:13  Updated: 2026-02-19 04:13  Messages size: 673 bytes

Session: cli:default
Created: 2026-02-16 06:12  Updated: 2026-02-18 06:07  Messages size: 2848 bytes

Inspect agent state

Inspect the agent's stored state:

./build/picooraclaw oracle-inspect state

Agent State (Key-Value)
─────────────────────────────────────────────────────────
agent_mode                     = interactive
last_channel                   = cli
last_model                     = gpt-4o-mini
total_conversations            = 42
user_name                      = jasperan

How Oracle Storage Works

The remember tool stores text along with a vector embedding using VECTOR_EMBEDDING(ALL_MINILM_L12_V2 USING :text AS DATA). The recall tool then uses VECTOR_DISTANCE() for cosine similarity search.

With Oracle-backed storage in place, PicoOraClaw supports the following LLM providers:

Ollama
OpenRouter
Zhipu
Anthropic
OpenAI
Gemini
DeepSeek
Groq

PicoOraClaw also supports OCI Generative AI as an optional LLM backend for enterprise models via the included oci-genai proxy.

CLI Reference

The following commands cover the core PicoOraClaw workflows:

picooraclaw onboard - initialize config and workspace
picooraclaw agent -m "..." - one-shot chat
picooraclaw agent - interactive chat mode
picooraclaw setup-oracle - initialize Oracle schema and ONNX model
picooraclaw oracle-inspect - inspect data stored in Oracle AI Database
picooraclaw oracle-inspect memories -s "query" - semantic search over stored memories
picooraclaw gateway - start the long-running service with channels enabled

Conclusion

PicoOraClaw is more than a lightweight assistant runtime. Combined with Oracle AI Database, it becomes a practical pattern for building assistants that retain context, retrieve facts semantically, and scale from local development to OCI without rearchitecting.

Start small, stay local, add durable semantic memory with Oracle AI Vector Search, and keep a clear path to a managed deployment model when you need it.

PicoOraClaw GitHub repository

Frequently Asked Questions (FAQs)

What hardware do I need to run PicoOraClaw?

PicoOraClaw runs on resource-constrained environments including x86_64, ARM64, and RISC-V platforms with a very small footprint. See the project repository for exact requirements.

How does PicoOraClaw remember information?

PicoOraClaw stores memories, sessions, and related state in Oracle AI Database. It uses in-database ONNX embeddings and vector search to retrieve memory by meaning rather than exact keyword matches.

Do I need an external embedding API?

No, the Oracle-backed memory flow uses in-database embeddings.

Can I run PicoOraClaw fully offline?

Yes. Ollama as the default backend enables fully local inference, making PicoOraClaw suitable for offline or privacy-sensitive workflows.

Can I deploy PicoOraClaw to Oracle Cloud?

Yes. The OCI deployment path provisions compute, Oracle AI Database Free, Ollama, and the PicoOraClaw gateway as a systemd service, with an optional Autonomous AI Database path. Deploy here.

Which LLM providers are supported?

Ollama (default), OpenRouter, Zhipu, Anthropic, OpenAI, Gemini, DeepSeek, Groq, and optional OCI Generative AI integration through the included proxy. See the PicoOraClaw repository for details.

Next Steps

Agent Reasoning: The Thinking Layer

Wojtek Pluta — Tue, 14 Apr 2026 13:08:46 +0000

Key Takeaways

Agent Reasoning is an open-source reasoning layer that adds planning, deduction, and self-correction to any Ollama-served LLM (e.g., gemma3, llama3), via plug-and-play Python or a proxy server.
Multiple proven reasoning strategies built-in (CoT, Self-Consistency, ToT, ReAct, Self-Reflection, Decomposition, Refinement) with a guided “start simple” path.
Practical tooling for teams: interactive CLI/TUI, Python API, and an Ollama-compatible gateway so existing apps gain reasoning without code changes.
Clear benchmark guidance: CoT delivers the best average accuracy; ToT shines for multi-step logic; ReAct leads when tools (search, calculator) matter.

Implementing Cognitive Problem-Solving in Open Source Models

From Nacho Martinez, Data Scientist Advocate at Oracle (and author of the A2A-based Multi-Agent RAG system) comes an open-source reasoning layer that can enable any open-source Large Language Model (LLM) such as gemma3 or llama3 to perform complex planning, logical deduction and self-correction. The layer wraps these models in a cognitive architecture built based on key research papers (CoT, ToT and ReAct).

We call this Agent Reasoning, and it is available open-source in this GitHub repository, alongside a Jupyter notebook.

Features of Agent Reasoning

Plug & Play: Use via Python Class or as a Network Proxy.
Model Agnostic: Works with any model served by Ollama.

Advanced Architectures:

Chain-of-Thought (CoT) & Self-Consistency: Implements Majority Voting (k samples) with temperature sampling.
Tree of Thoughts (ToT): BFS strategy with robust heuristic scoring and pruning.
ReAct (Reason + Act): Real-time tool usage (Web Search via scraping, Wikipedia API, Calculator) with fallback/mock capabilities. External grounding implemented.
Self-Reflection: Dynamic multi-turn Refinement Loop (Draft -> Critique -> Improve).
Decomposition & Least-to-Most: Planning and sub-task execution.
Refinement Loop: Score-based iterative improvement (Generator → Critic → Refiner) until quality threshold met.
Complex Refinement Pipeline: 5-stage optimization (Technical Accuracy → Structure → Depth → Examples → Polish).

Interactive Jupyter Notebook

We prepared an interactive Jupyter notebook to demonstrate the capabilities of agent reasoning.

This is a comprehensive demo covering all reasoning strategies (CoT, ToT, ReAct, Self-Reflection) with benchmarks and comparisons.

Architectures in Detail

For most users, start with Chain-of-Thought (CoT) — it has the best average accuracy and lowest latency cost. Use Self-Consistency when correctness is critical and you can afford 3–5× more inference time. Avoid ToT for knowledge-retrieval tasks (it underperforms baseline on MMLU) and reserve it for multi-step planning or logic puzzles.

Architecture	Description	Best For	Papers
Chain-of-Thought	Step-by-step reasoning prompt injection.	Math, Logic, Explanations	Wei et al. (2022)
Self-Reflection	Draft -> Critique -> Refine loop.	Creative Writing, High Accuracy	Shinn et al. (2023)
ReAct	Interleaves Reasoning and Tool Usage.	Fact-checking, Calculations	Yao et al. (2022)
Tree of Thoughts	Explores multiple reasoning branches (BFS/DFS).	Complex Riddles, Strategy	Yao et al. (2023)
Decomposed	Breaks complex queries into sub-tasks.	Planning, Long-form answers	Khot et al. (2022)
Recursive (RLM)	Uses Python REPL to recursively process prompt variables.	Long-context processing	Author et al. (2025)
Refinement Loop	Generator → Critic (0.0-1.0 score) → Refiner iterative loop.	Technical Writing, Quality Content	Inspired by Madaan et al. (2023)
Complex Refinement	5-stage pipeline: Accuracy → Clarity → Depth → Examples → Polish.	Long-form Articles, Documentation	Multi-stage refinement architecture

Accuracy Benchmarks

You can evaluate reasoning strategies against standard NLP datasets to measure accuracy improvements. The benchmark system includes embedded question sets from 4 standard datasets.

To run an accuracy benchmark:

Or using the Python API:

Charts are auto-generated after each run and save to benchmarks/charts/.

Dataset	Category	Questions	Format	Reference
GSM8K	Math Reasoning	30	Open-ended number	Cobbe et al. (2021)
MMLU	Knowledge (57 subjects)	30	Multiple choice (A-D)	Hendrycks et al. (2021)
ARC-Challenge	Science Reasoning	25	Multiple choice (A-D)	Clark et al. (2018)
HellaSwag	Commonsense	20	Multiple choice (A-D)	Zellers et al. (2019)

The following are the results of a full evaluation across all 11 strategies:

Strategy	GSM8K	MMLU	ARC-C	HellaSwag	Avg
Standard (baseline)	66.7%	90.0%	92.0%	90.0%	84.7%
Chain of Thought	73.3%	96.7%	88.0%	90.0%	87.0%
Tree of Thoughts	76.7%	63.3%	76.0%	90.0%	76.5%
ReAct	63.3%	86.7%	96.0%	90.0%	84.0%
Self-Reflection	66.7%	90.0%	88.0%	90.0%	83.7%
Self-Consistency	76.7%	96.7%	92.0%	—	66.3%
Decomposed	10.0%	60.0%	84.0%	—	38.5%

Key findings:

CoT achieves the highest average accuracy (87.0%), outperforming Standard on GSM8K (+6.6%) and MMLU (+6.7%)
Self-Consistency ties CoT on MMLU (96.7%) and GSM8K (76.7%) through majority voting
ToT excels on GSM8K math (76.7%, +10% over Standard) through branch exploration
ReAct achieves the highest ARC-Challenge score (96.0%) via tool-augmented reasoning

Accuracy statistics

This is the accuracy heat map per-strategy:

This is the average accuracy by strategy:

Benchmarks

Benchmarks charts are auto-generated after every benchmark run.

For a complete listing of sample output benchmarks (response latency, throughput etc.) please refer to the Agent Reasoning GitHub repository.

Quick start (3 commands)

uv sync && ollama pull gemma3:270m && uv run agent-reasoning

Installation

One-command, single-step install

curl -fsSL https://raw.githubusercontent.com/jasperan/agent-reasoning/main/install.sh | bash

You can also install agent-reasoning using either PyPi or directly from source.

Using PyPi

From Source using uv

Development

Configuring the large language model (LLM)

We use Ollama as an example for this procedure.

Ollama must be running locally, or you can connect to a remote Ollama instance.

ollama pull gemma3:270m    # Tiny model for quick testing
ollama pull gemma3:latest  # Full model for quality results

Configuring the remote Ollama endpoint

If you don't have Ollama installed locally, you can connect to a remote Ollama instance. Configuration is stored in config.yaml in the root directory of the repository.

Option 1: Interactive CLI configuration

agent-reasoning
# Select "Configure Endpoint" from the menu

Option 2: Server CLI Argument

agent-reasoning-server --ollama-host http://192.168.1.100:11434

Option 3: Direct Config File

Copy the example config and edit it:

cp config.yaml.example config.yaml

Or create config.yaml in the project root:

ollama:
  host: http://192.168.1.100:11434

Option 4: Python API

Usage

1. Interactive CLI

Use the rich CLI to access all agents, comparisons and benchmarks.

Timing Metrics: Every response shows TTFT, total time, tokens/sec
Session History: All chats auto-saved to data/sessions/ with export to markdown
Head-to-Head: Compare any two strategies side-by-side in parallel
Agent Info: Built-in strategy guide with descriptions and use cases
Benchmark Charts: Auto-generate PNG visualizations of benchmark results

Setup

Shortcuts

The CLI also provides useful shortcuts:

Interactive experience

2. Terminal UI

You can also use a Go-based terminal interface with a split-panel layout and arena grid view.

Split layout: agent sidebar + chat panel
Arena mode: 3x3 grid showing all agents running in parallel
Real-time streaming with cancellation support

The TUI automatically starts the reasoning server on launch. Requires Go 1.18+.

Keybindings for TUI

Chat View

The default chat view is a split-pane layout with a 16-agent sidebar, chat panel with live streaming, and a metrics bar showing TTFT, tokens/sec, and token count in real-time.

Press v to toggle structured visualization mode. Instead of raw text, you see the agent's reasoning process rendered live: tree diagrams for ToT, swimlanes for ReAct, vote tallies for Consistency, score gauges for Refinement, and more.

Press p to open the hyperparameter tuner. Adjust ToT width/depth, Consistency samples, Refinement score thresholds, and other agent parameters before running a query.

Press ? to invoke the strategy advisor. The MetaReasoningAgent analyzes your query and recommends the best strategy.

Modes of interaction

Arena Mode prompts all 16 agents to race simultaneously on the same query displayed using a 4x4 grid; a leaderboard bar updates as each agents finish:

Head-to-Head Duel prompts two agents to compete 1-1 on the same query.

There are plenty of other features to try, such as:

the Step-Through Debugger which enables pausing the agent between LLM calls and inspecting intermediate state
the Benchmark Dashboard which reads existing JSON benchmark files
the Session Browser which enables search and re-running of past conversations, with filtering options
the Agent Guide, which contains reference cards for all 16 agents, covering best-for, parameters, trade-offs, and research reference. Pressing Enter on any card initiates a chat with the agent.

3. Python API (for developers)

Use the ReasoningInterceptor as a drop-in replacement for your LLM client.

Using agents directly:

Using refinement agents for quality control:

4. Reasoning Gateway Server

Run a proxy server that impersonates Ollama. This allows any Ollama-compatible app, such as LangChain or Web UIs, to gain reasoning capabilities without any code changes whatsoever.

Then configure your app:

Base URL: http://localhost:8080
Model: gemma3:270m+cot (or +tot, +react, etc.)

API Endpoints

Troubleshooting

Model Not Found: Ensure you have pulled the base model (ollama pull gemma3:270m).
Timeout / Slow: ToT and Self-Reflection make multiple calls to the LLM. With larger models (Llama3 70b), this can take time.
Hallucinations: The default demo uses gemma3:270m which is extremely small and prone to logic errors. Switch to gemma2:9b or llama3 for robust results.

Extending the system further

You can add additional reasoning strategies.

Create a class in src/agent_reasoning/agents/ inheriting from BaseAgent.
Implement the stream(self, query) method.
Register it in AGENT_MAP in src/agent_reasoning/interceptor.py.

Conclusion

Thank you for reading, and we look forward to seeing what you build using Agent Reasoning!

Frequently Asked Questions (FAQs)

When should I use each strategy?

Start with Chain-of-Thought for best accuracy/latency trade-off; use Self-Consistency when correctness is critical; reserve Tree of Thoughts for complex multi-step reasoning; pick ReAct for fact-checks or calculations.

Do I need a specific model?

No. It’s model-agnostic for any model served by Ollama. Quality improves with larger models (e.g., gemma2:9b, llama3 vs tiny 270m).

How hard is setup?

Three-command quick start, one-line install script, and ready-to-run demos in a Jupyter notebook. A proxy lets existing Ollama apps adopt reasoning by just changing the base URL/model name.

How do I evaluate results?

Built-in benchmarks (GSM8K, MMLU, ARC-Challenge, HellaSwag) auto-generate charts, with side-by-side strategy comparisons and session histories for review.

Agent Memory Storage: A Practical Guide

Wojtek Pluta — Wed, 11 Feb 2026 23:01:08 +0000

Why this matters

Agentic AI promises autonomous systems that reason and act, but without persistent memory, they fail in real-world use. Agents forget conversation context after a few turns or repeat API errors due to stateless design. This tutorial shows how to build a hybrid memory system using Oracle AI Database, combining SQL for exact recall and vectors for semantic search, enabling production-ready agents that learn and remember across sessions.

What we’ll build

We'll create a Hybrid Memory Manager for AI agents that stores conversational history in SQL tables for ACID compliance and semantic knowledge in vector indexes for similarity search. Key features include automatic summarization to manage context windows, persistent tool retrieval, and integration with web search results via Tavily. This converged approach in Oracle AI Database eliminates data silos.

Hybrid Storage: SQL for episodic memory (chats), vectors for semantic memory (facts/tools).
Lifecycle Management: Summarize long threads to keep prompts efficient.
Learning Loop: Cache web search results in the knowledge base.
Requirements:
- Docker for Oracle AI Database.
- Python 3.10+ with libraries: oracledb, langchain-oracledb, langchain, sentence-transformers, tavily-python.

Setup

Install and configure

Start the Oracle AI Database container using Docker. This provides the converged engine for SQL and vector operations.

docker run -d \
  --name oracle-free \
  -p 1521:1521 -p 5500:5500 \
  -e ORACLE_PWD=OraclePwd_2025 \
  -v "$HOME/oracle/full_data:/opt/oracle/oradata" \
  container-registry.oracle.com/database/free:latest

Wait for the container to start (check with docker logs oracle-free). The database is ready when you see "DATABASE IS READY TO USE!" in logs.

Install Python dependencies for database connection, embeddings, and agent tools.

python -m pip install -U oracledb langchain-oracledb langchain \
  sentence-transformers tavily-python

Output:

Successfully installed oracledb-2.3.0 langchain-oracledb-0.1.0 ...

Prerequisites:

Product: Oracle AI Database.
Tools: Docker, Python 3.10+, VS Code or similar.
Skills: Basic SQL, Python, Docker.
Access: Run locally; no cloud signup needed for this guide.

Connect and verify

Connect to the database and verify the Oracle version. This Python script retries on startup delays.

import oracledb
import time

def connect_to_oracle(user="VECTOR", password="VectorPwd_2025", dsn="127.0.0.1:1521/FREEPDB1", max_retries=3):
    for attempt in range(1, max_retries + 1):
        try:
            conn = oracledb.connect(user=user, password=password, dsn=dsn)
            with conn.cursor() as cur:
                cur.execute("SELECT banner FROM v$version WHERE banner LIKE 'Oracle%'")
                print(f"Connected to: {cur.fetchone()[0]}")
            return conn
        except oracledb.Error as e:
            if attempt == max_retries:
                raise
            print(f"Retry {attempt}...")
            time.sleep(5)

conn = connect_to_oracle()

Output:

Connected to: Oracle Database ...

Core steps

Step 1: Design hybrid memory architecture

Match storage types to use cases: Use SQL for structured, exact-match episodic memory (e.g., chat history) to ensure auditability and transaction safety. Use vector embeddings for semantic search in knowledge bases and tools, retrieving similar content without exact keywords. In Oracle AI Database, store both in one schema to avoid sync issues and simplify queries.

This design prevents data drift and enables unified queries, like joining chat metadata with similar knowledge vectors.

Step 2: Implement episodic memory with SQL

Store conversation turns in a SQL table for reliable, queryable history. This supports filtering by thread_id and timestamp, essential for multi-turn agents.

Execute these SQL commands via SQLcl or the connection script (connect as SYS or VECTOR user with privileges).

-- Create episodic memory table
CREATE TABLE CONVERSATIONAL_MEMORY (
  id         VARCHAR2(100) DEFAULT SYS_GUID() PRIMARY KEY,
  thread_id  VARCHAR2(100) NOT NULL,
  role       VARCHAR2(50)  NOT NULL,
  content    CLOB          NOT NULL,
  timestamp  TIMESTAMP     DEFAULT CURRENT_TIMESTAMP,
  metadata   CLOB,
  summary_id VARCHAR2(100)
);

CREATE INDEX idx_conv_thread ON CONVERSATIONAL_MEMORY(thread_id);

Output:

Table created.
Index created.

Now, add a Python function to insert messages.

def write_conversation(conn, thread_id: str, role: str, content: str) -> str:
    with conn.cursor() as cur:
        out_id = cur.var(str)
        cur.execute(
            """INSERT INTO CONVERSATIONAL_MEMORY(thread_id, role, content, metadata)
               VALUES(:t, :r, :c, '{}') RETURNING id INTO :id""",
            {"t": thread_id, "r": role, "c": content, "id": out_id}
        )
    conn.commit()
    return out_id.getvalue()[0]

# Example usage
msg_id = write_conversation(conn, "thread123", "user", "Hello, agent!")
print(f"Inserted message ID: {msg_id}")

Output:

Inserted message ID: 123e4567-e89b-12d3-a456-426614174000

Pitfall: Always commit after inserts to persist data; use metadata CLOB for JSON extras like user ID.

Step 3: Build semantic memory with vectors

Use vector search for fuzzy retrieval of knowledge and tools. Oracle AI Database's vector index enables efficient similarity searches on embedded text.

First, create the vector table (via SQL).

-- Create semantic memory table with vector column
CREATE TABLE SEMANTIC_MEMORY (
  id         VARCHAR2(100) DEFAULT SYS_GUID() PRIMARY KEY,
  content    CLOB NOT NULL,
  embedding  VECTOR(384, FLOAT32),  -- Dimension matches model
  metadata   CLOB,
  source     VARCHAR2(100)
);

CREATE VECTOR INDEX idx_sem_vec ON SEMANTIC_MEMORY(embedding) VECTOR PARAMETERS('ORGANIZER=HNSW');

Output:

Table created.
Index created.

Now, integrate with LangChain for easy ingestion and search.

from langchain_oracledb.vectorstores import OracleVS
from langchain_community.embeddings import HuggingFaceEmbeddings

embedding = HuggingFaceEmbeddings(model_name="sentence-transformers/paraphrase-mpnet-base-v2")

kb_vs = OracleVS(client=conn, embedding_function=embedding,
                 table_name="SEMANTIC_MEMORY", embedding_col="embedding")

# Ingest sample knowledge
kb_vs.add_texts(
  texts=["A tablespace can be online or offline for maintenance.", "LOB segments are indexed implicitly for performance."],
  metadatas=[{"source": "admin_guide"}, {"source": "dev_guide"}]
)

# Search example
results = kb_vs.similarity_search("database maintenance", k=1)
print(results[0].page_content)

Output:

A tablespace can be online or offline for maintenance.

Pitfall: Match embedding dimensions to your model (384 for paraphrase-mpnet); re-embed if switching models to avoid drift.

Step 4: Unify with MemoryManager class

Encapsulate SQL and vector ops in a single class for clean agent integration. This abstraction simplifies the main loop, allowing agents to query memory without knowing the backend.

class MemoryManager:
    def __init__(self, conn, convo_table, kb_vs):
        self.conn = conn
        self.convo_table = convo_table
        self.kb_vs = kb_vs

    def read_convo(self, thread_id: str):
        with self.conn.cursor() as cur:
            cur.execute(
                f"SELECT role, content FROM {self.convo_table} WHERE thread_id=:t AND summary_id IS NULL ORDER BY timestamp",
                {"t": thread_id}
            )
            return cur.fetchall()

    def search_knowledge(self, query: str, k=3):
        return self.kb_vs.similarity_search(query, k=k)

    def write_convo(self, thread_id: str, role: str, content: str):
        return write_conversation(self.conn, thread_id, role, content)

# Example usage
mm = MemoryManager(conn, "CONVERSATIONAL_MEMORY", kb_vs)
history = mm.read_convo("thread123")
print(f"History length: {len(history)}")

knowledge = mm.search_knowledge("database tablespace")
print(knowledge[0].page_content)

Output:

History length: 1
A tablespace can be online or offline for maintenance.

This keeps the agent code focused on reasoning, not storage details.

Step 5: Add summarization for context management

Long conversations exceed LLM context limits and increase costs. Automatically summarize threads when they exceed a threshold (e.g., 10 turns), store the summary as a vectorized entry, and link originals via summary_id.

Extend MemoryManager with summarization (requires OpenAI or local LLM).

from langchain_openai import OpenAI  # Or use local model
from langchain.chains.summarize import load_summarize_chain

def summarize_thread(mm, thread_id: str, llm):
    history = mm.read_convo(thread_id)
    if len(history) < 10:
        return None

    # Format history for summarization
    docs = [f"{role}: {content}" for role, content in history]
    chain = load_summarize_chain(llm, chain_type="map_reduce")
    summary = chain.run(docs)

    # Store summary in semantic memory
    summary_doc = kb_vs.add_texts([summary], metadatas=[{"type": "summary", "thread_id": thread_id}])[0]
    summary_id = summary_doc.metadata.get("id", str(hash(summary)))

    # Mark originals
    def mark_summarized(conn, thread_id: str, summary_id: str):
        with conn.cursor() as cur:
            cur.execute("UPDATE CONVERSATIONAL_MEMORY SET summary_id=:s WHERE thread_id=:t AND summary_id IS NULL",
                        {"s": summary_id, "t": thread_id})
        conn.commit()

    mark_summarized(mm.conn, thread_id, summary_id)
    return summary

# Example (set OPENAI_API_KEY env)
llm = OpenAI(temperature=0)
summary = summarize_thread(mm, "thread123", llm)
print(f"Summary: {summary}")

Output:

Summary: The user greeted the agent, which responded helpfully about database topics.

Pitfall: Choose chain_type="stuff" for short threads; monitor token usage to avoid recursion in map_reduce.

Step 6: Integrate learning with Tavily search

Agents should persist external knowledge to reduce API calls and improve speed. Use Tavily for web search, then embed and store results in semantic memory.

Set TAVILY_API_KEY environment variable.

from tavily import TavilyClient

def learn_from_web(mm, query: str, max_results=3):
    client = TavilyClient(api_key="your-tavily-key")
    response = client.search(query, max_results=max_results)

    for result in response["results"]:
        content = result["content"]
        # Embed and store
        mm.kb_vs.add_texts([content], metadatas=[{"source": "tavily", "query": query, "url": result["url"]}])

    # Return for immediate use
    return mm.search_knowledge(query)

# Example
web_knowledge = learn_from_web(mm, "best practices for AI agent memory")
print(web_knowledge[0].page_content[:100] + "...")

Output:

Hybrid memory systems combine relational and vector databases for robust AI agents...

This creates a feedback loop: Search once, reuse forever. Security note: Sanitize web content before storage to avoid injection.

Pitfalls and patterns

Unbounded Growth: Implement summarization thresholds (e.g., every 10 turns) to prune history; monitor table sizes with SQL queries like SELECT COUNT(*) FROM CONVERSATIONAL_MEMORY.
Security: Encrypt PII in metadata CLOBs using Oracle Transparent Data Encryption. Use row-level security for multi-tenant agents. Audit access with Oracle Data Safe.
Embedding Drift: If updating the embedding model, add a version column and re-embed incrementally with a migration script.
Performance: Use HNSW index for vectors (default in Oracle); batch inserts for high-volume ingestion. Test query latency with EXPLAIN PLAN.
Error Handling: Wrap DB ops in try-except; retry on transient errors like connection timeouts.

Next steps

Explore the full implementation in the tutorial notebook. Star the repo. Share your agent builds in the comments!

Build a Scalable Multi-Agent RAG System with A2A Protocol, Oracle AI Database and LangChain

Wojtek Pluta — Thu, 22 Jan 2026 17:47:24 +0000

Why this matters

Multi-agent systems for retrieval-augmented generation (RAG) promise collaborative AI reasoning but often fail at scale due to resource contention and tight coupling. This tutorial shows how to build a distributed system using the Agent2Agent (A2A) Protocol, enabling independent agent scaling while integrating Oracle AI Database for vector storage and search through LangChain package (langchain-oracledb). You'll end up with a flexible RAG pipeline that handles document queries efficiently, suitable for AI developers facing production bottlenecks.

The outcome: A loosely coupled architecture where agents like planners, researchers, and synthesizers communicate via A2A, reducing latency and improving fault isolation in high-load scenarios.

What we’ll build

We'll create Agentic RAG, an intelligent RAG system with multi-agent CoT reasoning:

Planner, Researcher, Reasoner, Synthesizer agents communicating via A2A Protocol.
PDF/web/repo processing with Docling/Trafilatura/Gitingest.
Persistent vector storage in Oracle AI Database 26ai.
FastAPI API and Gradio UI for uploads/queries.
Local LLMs via Ollama (gemma3:270m default).

Alt: Architecture showing PDF/web processing, vector store, RAG agent, and A2A multi-agent CoT.

Requirements:

Oracle AI Database instance (Autonomous Database).
LangChain Integration for Oracle AI Vector Search - Vector Store
Python 3.10+, dependencies from requirements.txt.
Ollama installed and running.
Docling
Trafilatura
Gitingest

Repo: Oracle AI Developer Hub.

Setup

Install and configure

Clone the repo and install dependencies:

git clone https://github.com/oracle-devrel/oracle-ai-developer-hub.git
cd oracle-ai-developer-hub/apps/agentic_rag
pip install -r requirements.txt  # Includes docling, trafilatura, oracledb, fastapi, gradio, ollama and langchain-oracledb

Set up Ollama for local LLMs (default: gemma3:270m):

ollama pull gemma3:270m
ollama serve

Configure Oracle AI Database 26ai in config.yaml:

ORACLE_DB_USERNAME, ORACLE_DB_PASSWORD, ORACLE_DB_DSN. Use Oracle AI Database Free to store and retrive vector embeddings.

Connect and verify

Test Oracle connection (via tests/test_oradb.py):

python tests/test_oradb.py --stats-only

Or in Python (using oracledb):

import oracledb
connection = oracledb.connect(user="ADMIN", password="<pass>", dsn="<dsn>")
cursor = connection.cursor()
cursor.execute("SELECT * FROM v$version WHERE banner LIKE '%26ai%'")
print(cursor.fetchall())
connection.close()

Output:

[('Oracle Database 26ai ...',)]

Verify Ollama: curl http://localhost:11434/api/tags (should list gemma3:270m).

Core steps

Step 1: Process and ingest data into Oracle AI Database 26ai

Use built-in processors for PDFs (Docling), websites (Trafilatura), repos (Gitingest) to chunk text and generate vector embeddings, then store them in vector collections (PDFCOLLECTION, WEBCOLLECTION, REPOCOLLECTION, GENERALCOLLECTION).

Why vector embeddings: Embeddings capture semantic meaning, enabling efficient similarity search via Oracle AI Database's VECTOR_DISTANCE function. This supports intelligent query routing across diverse sources like PDFs and web content.

Focus on LangChain for embeddings: Integrate LangChain's embedding models (e.g., OllamaEmbeddings for local LLMs) to generate vectors before storing in Oracle Database. This allows seamless switching between embedding providers while leveraging Oracle's scalable vector storage.

Process a PDF:

python -m src.pdf_processor --input https://arxiv.org/pdf/2203.06605 --output chunks.json
python -m src.store --add chunks.json  # Adds to PDFCOLLECTION

For websites:

python -m src.web_processor --input https://example.com --output web_content.json
python -m src.store --add-web web_content.json  # Adds to WEBCOLLECTION

In code (src/store.py equivalent, with LangChain embeddings):

from langchain.embeddings import OllamaEmbeddings
from oracle_ai_vector_search import OracleVectorStore  # Compatible with langchain-oracledb

# Initialize embeddings with LangChain
embeddings = OllamaEmbeddings(model="gemma3:270m")

# From src/store.py - initialize OracleVS
store = OracleVectorStore(
    connection=connection,
    collection="PDFCOLLECTION",
    embedding=embeddings  # Pass LangChain embeddings
)
store.add_texts(chunks, metadatas=metadata)
results = store.similarity_search("query", k=5, embedding=embeddings)

Expected output:

Processed 10 chunks from PDF.
Generated embeddings with OllamaEmbeddings.
Added to vector store: PDFCOLLECTION (total chunks: 15).

Pitfall: Configure config.yaml for DB creds; large PDFs may need chunk_size adjustment in LangChain's text splitter. Ensure Ollama is running for local embeddings.

LangChain Integration for RAG Orchestration

LangChain simplifies building RAG pipelines by providing chains for retrieval, question-answering, and conversational memory. In this system:

Use RetrievalQA or ConversationalRetrievalChain to query the Oracle vector store.
Integrate with A2A agents for multi-step reasoning: LangChain's tool-calling agents can invoke A2A endpoints as custom tools.
Example: Wire OracleVectorStore to a RetrievalQA chain for hybrid search (vector + keyword).

Code snippet (in src/local_rag_agent.py):

from langchain.chains import RetrievalQA
from langchain.embeddings import OllamaEmbeddings
from oracle_ai_vector_search import OracleVectorStore

embeddings = OllamaEmbeddings(model="gemma3:270m")
store = OracleVectorStore(connection=connection, embedding=embeddings, collection="PDFCOLLECTION")

qa_chain = RetrievalQA.from_chain_type(
    llm=llm,  # Ollama LLM
    chain_type="stuff",
    retriever=store.as_retriever(search_kwargs={"k": 5})
)

result = qa_chain.run("Explain DaGAN in Depth-Aware GAN")
print(result)

Why it fits: LangChain's modular design complements A2A's distributed agents, enabling scalable CoT while offloading vector ops to Oracle AI Database.

Step 2: Implement A2A agent cards and discovery

A2A Protocol enables JSON-RPC communication for agent discovery, task management, and distributed CoT.

Why: Supports interoperable, scalable multi-agent workflows with capability advertisement.

Agent card example (from agent_card.py):

{
  "agent_id": "planner_agent_v1",
  "name": "Strategic Planner",
  "version": "1.0.0",
  "description": "Breaks queries into actionable steps",
  "capabilities": ["agent.query"],
  "inputs": {"query": "string", "step": "optional"},
  "outputs": {"plan": "array"},
  "endpoint": "http://localhost:8000/a2a"
}

Discovery via curl:

curl -X POST http://localhost:8000/a2a \
  -H "Content-Type: application/json" \
  -d '{
    "jsonrpc": "2.0",
    "method": "agent.discover",
    "params": {"capability": "agent.query"},
    "id": "1"
  }'

Output:

{"jsonrpc":"2.0","result":{"agents":[{"agent_id":"planner_agent_v1","url":"http://localhost:8000/a2a"}]},"id":"1"}

Deploy: Update config.yaml with AGENT_ENDPOINTS for distributed (e.g., planner_url: http://server1:8001).

Pitfall: Ensure A2A server runs (python -m src.main).

Step 3: Build the multi-agent pipeline with A2A and CoT

Use local_rag_agent for RAG queries; enable --use-cot for distributed multi-agent reasoning (Planner → Researcher → Reasoner → Synthesizer via A2A).

Why: Provides structured CoT for complex queries, with transparent steps and sources.

CLI example:

python -m src.local_rag_agent --query "Explain DaGAN in Depth-Aware GAN" --use-cot --model gemma3:270m

In code (from src/local_rag_agent.py):

# Simplified orchestrator
from a2a_handler import A2AHandler
handler = A2AHandler()

def run_cot_pipeline(query: str):
    # Step 1: Planner via A2A
    plan = handler.call_agent("planner_agent_v1", {"query": query})
    # Step 2-4: Delegate to researcher/reasoner/synthesizer
    research = handler.call_agent("researcher_agent_v1", {"query": query, "plan": plan})
    reasoning = handler.call_agent("reasoner_agent_v1", {"context": research})
    final = handler.call_agent("synthesizer_agent_v1", {"steps": [plan, research, reasoning]})
    return final

result = run_cot_pipeline("What is A2A?")
print(result)

Output:

Step 1: Planning - Break down query...
Step 2: Research - Gathered from PDFCOLLECTION...
...
Final Answer: A2A is an open protocol for agent communication...
Sources: document.pdf (pages 1-3)

Pitfall: CoT increases latency (2-5x); use for complex queries only. Ensure all agents registered.

Step 4: Launch Gradio UI and API for interaction

Run Gradio for UI (includes model management, document processing, chat with CoT/A2A tabs).

Why: Provides user-friendly interface for uploads, queries, and A2A testing.

python gradio_app.py  # Starts at http://localhost:7860; auto-starts A2A server

API endpoints (FastAPI at http://localhost:8000):

# Upload PDF
POST /upload/pdf
Content-Type: multipart/form-data
file: <pdf-file>

# Query with CoT
POST /query
Content-Type: application/json
{
    "query": "Your question",
    "use_cot": true
}

In code:

import requests
response = requests.post("http://localhost:8000/query", json={"query": "Test RAG", "use_cot": True})
print(response.json()["answer"])

Output:

{
  "answer": "Response with CoT steps...",
  "sources": ["PDFCOLLECTION"]
}

Pitfall: Port conflicts; use --port flag. Gradio requires gradio installed.

Practical Benefits of using A2A

It's Free: all LLMs are open-source, so you only have to deploy them and start talking free of charge.
Operational Clarity: With Agent Cards and discovery, your ops team knows exactly what agents are available, what they can do, and how loaded they are. Monitoring becomes straightforward - track task completion rates per agent type, identify real bottlenecks, and scale intelligently.
Fault Isolation: When one researcher agent crashes, others continue working. When a planner agent goes down, you can quickly discover an alternative or restart it without disrupting the entire pipeline.
Flexibility: Need better document analysis? Swap your researcher agent for one using a different model or provider. A2A doesn't lock you into a specific implementation.
Enterprise Compliance: Each agent can enforce its own security policies, authentication schemes, and audit logging. A2A supports JWT, OIDC, and custom authentication at the agent level.

Next steps for the project

I'd like to implement a few things into this project - and we're looking for contributors to get involved! Give us a star on our GitHub repository.

Couple of things on our roadmap are:

The ability to create custom agents, not only the pre-defined pipeline I created (planner -> researcher -> reasoner -> synthesizer)
Fully decouple the LLMs in the current pipeline: I'd like to test another architecture where agents work independently on parts of the answer instead of having a cascading or sequential mechanism (what we have more or less right now, as the synthesizer agent has to wait for the other agents to finish their tasks first)

Conclusions

The evolution from monolithic Agentic RAG to A2A-based distributed systems is well underway, moving away from ‘deploy the whole pipeline more times’ to a position of deploying the right number of the right agents.
The beauty of A2A adoption is that it's open-source and standardized (and it's always nice to have it developed and maintained by Google). For organizations building serious agentic systems, this is the time where you can get ahead of the rest and start building with Oracle AI Database, A2A Protocol and LangChain Oracle AI Vector Search Integration!