Gen AI with Java & Spring Boot — Agents, RAG & Multimodality
A hands-on, production-shaped reference application that explores the full surface of
Spring AI on Java 25 and Spring Boot 3.5.
It brings together multi-provider chat, retrieval-augmented generation (RAG), multimodality,
tool-calling agents with human-in-the-loop approval, the Model Context Protocol (MCP), and a
full observability stack.
Production-shaped Spring AI reference app (Java 25, Spring Boot 3.5): multi-provider LLM chat (OpenAI, Vertex AI, Hugging Face, Ollama), advanced RAG over pgvector with Cohere reranking, multimodality, a human-in-the-loop tool-calling Security-Review Agent, MCP client+server, and full Prometheus/Grafana/OpenTelemetry observability.
Status: learning / reference project. It is intentionally broad — each package is a
self-contained demonstration of one Spring AI capability, wired so the pieces can be combined.
Pluggable providers behind Spring AI's ChatClient: OpenAI, Google Vertex AI Gemini,
Hugging Face (dedicated inference endpoints), Ollama (local), and the
Docker Model Runner (OpenAI-compatible local endpoint).
Streaming responses, structured output (POJO / List / Map binding), and a "raw" OpenAI
Java client comparison.
Conversational memory two ways: JDBC-backed message-window memory and
semantic memory backed by a pgvector VectorStore.
Retrieval-Augmented Generation (RAG)
A simple path using QuestionAnswerAdvisor over an in-memory SimpleVectorStore.
An advanced path using RetrievalAugmentationAdvisor with:
domain-synonym query transformation,
multi-query expansion,
pgvector similarity retrieval,
Cohere reranking, neighbour-chunk stitching, and citation-header post-processing.
PDF ingestion (page/paragraph readers) plus a filesystem watcher that ingests new PDFs
dropped into a directory at runtime.
An "internal Q&A over docs" scenario and a "mini employee handbook" scenario.
Multimodality
Image → text: product-image description.
Text → image: marketing-asset / campaign image generation with DALL·E 3.
Speech → text: meeting-notes transcription (Whisper) with structured summarization.
Text → speech: ticket-status voice responses (TTS).
Agents & Tools
A Security-Review Agent that plans and executes a multi-step review of a system, calling
tools for: threat-model diagram extraction (draw.io / image), RAG over internal security
docs, web/OWASP lookups (via Google Vertex AI Search), and a security-posture lookup.
Human-in-the-loop checkpoints (approve diagram extraction, approve final report), with
durable review state persisted to Postgres.
A vision variant and a deterministic chain-workflow variant of the agent.
MCP: the main app is an MCP client; the posture-service is a standalone MCP server
exposing posture tools.
Cross-cutting
Custom ChatClient advisors: system-prompt injection, validation, error-wrapping,
SafeGuardAdvisor (sensitive-term guard), content sanitization, and a simple logger.
Full observability: Micrometer + Prometheus metrics, OpenTelemetry tracing to
Jaeger/Tempo, Loki/Promtail logs, and provisioned Grafana dashboards.
spring.ai.* — per-provider model names (e.g. gpt-4o-mini, gemini-2.0-flash, mistral:7b).
spring.datasource.* — points at Postgres on localhost:5433 (matches the compose file).
Heads-up — machine-specific paths. A few settings currently point at absolute local
directories (app.agent.upload-dir, app.rag.pdf.path under /Users/Shared/...) and the Vertex
project-id is hard-coded. Update these to your own paths/project before running the agent and
filesystem-watcher features. See Notes & Known Rough Edges.
Running Locally
1. Infrastructure (Postgres + pgvector)
docker compose -f docker/postgres/pgvector.yml up -d
Brings up pgvector/pgvector:pg16 on host port 5433. The vector tables (rag_vector_store,
vector_store) and chat-memory schema are auto-created on first run (initialize-schema: true).
2. Observability stack (optional)
docker compose -f docker/observability/observability-compose.yml up -d
Ingestion (RagIngestionService + PdfWatcherService) reads PDFs, splits them with a
TokenTextSplitter, embeds with OpenAI, and upserts into the rag_vector_store table. Drop a new
PDF into the configured watch directory and it is ingested without a restart.
The Security-Review Agent
aiagent/ implements an agent that reviews a system design for security issues:
Plan the review (Plan / PlanStep records).
Extract a threat-model diagram from an uploaded draw.io / image file (DiagramTools).
Pause for human approval of the extracted model.
Gather evidence via tools — internal-docs RAG (RagTools), web/OWASP lookups
(WebTools, backed by Vertex AI Search), and security posture (PostureTools over MCP).
Draft a report and pause for human approval before finalizing.
Review state (ReviewState, checkpoints, pending approvals, stored messages) is persisted to
Postgres so a review can be resumed. Two variants exist: SecurityReviewAgentVision (image input)
and SecurityReviewAgentChainWorkflow (deterministic chained steps, toggled by
app.agent.use-chain-workflow).
ContentSanitizerAdvisor scrubs tool/document content before it reaches the model.
chat/openai/jailbreak/demo (with BankingTools) demonstrates a jailbreak scenario and the
guardrails that mitigate it.
These are educational demonstrations, not a substitute for a real security review.
Testing with the Postman Collection
Import gen-ai-with-java-spring.postman_collection.json
into Postman. It contains ready-to-run requests for every endpoint above, grouped by capability.
The project was built iteratively with heavy manual and Postman-driven testing across each
provider and RAG configuration.
Notes & Known Rough Edges
Machine-specific paths: app.agent.upload-dir and app.rag.pdf.path point at
/Users/Shared/...; the Vertex project-id in application.yml is hard-coded. Change these for
your environment (or externalize them to env vars) before using the agent / PDF-watcher features.
Java 25 + Spring Boot 3.5.8 + Spring AI 1.1.0 are aligned and current; if you build on an
older JDK, adjust <java.version> and the compiler <release> in pom.xml.
pgvector dimensions are fixed at 1536 (OpenAI text-embedding-3-small/ada-002). If you
switch embedding models, update spring.ai.vectorstore.pgvector.dimensions and rebuild the store.
app.rag.force-rebuild: true re-ingests on startup — set to false once your store is warm.
Some provider endpoints (Hugging Face dedicated endpoint URL, Vertex data store) are placeholders
and must be pointed at your own resources.
License
No license file is currently included. Add one (e.g. MIT) before publishing or reusing.
S
Spring Ai Agents Rag
omonuj·3 ดาว·
Production-shaped Spring AI reference app (Java 25, Spring Boot 3.5): multi-provider LLM chat (OpenAI, Vertex AI, Hugging Face, Ollama), advanced RAG over pgvector with Cohere reranking, multimodality
Production-shaped Spring AI reference app (Java 25, Spring Boot 3.5): multi-provider LLM chat (OpenAI, Vertex AI, Hugging Face, Ollama), advanced RAG over pgvector with Cohere reranking, multimodality, a human-in-the-loop tool-calling Security-Review Agent, MCP client+server, and full Prometheus/Grafana/OpenTelemetry observability.
ตรวจสอบล่าสุดเมื่อ 2026-10-28 Production-shaped Spring AI reference app (Java 25, Spring Boot 3.5): multi-provider LLM chat (OpenAI, Vertex AI, Hugging Face, Ollama), advanced RAG over pgvector with Cohere reranking, multimodality, a human-in-the-loop tool-calling Security-Review Agent, MCP client+server, and full Prometheus/Grafana/OpenTelemetry observability.