A comprehensive multi-agent AI orchestration platform featuring secure Retrieval-Augmented Generation (RAG), advanced research capabilities, voice interactions, MCP integration, and sophisticated memory management using Mastra AI framework.
Secure Retrieval-Augmented Generation (RAG) with role-based access control using Mastra AI orchestration, with JWT Secure, also Deep Research built with Cedar & Next.js. Gemini, OpenAI, Openrouter, & Gemini CLI usable. Also exposes tools, agents as MCP Server Also A2A Agent Cards
星數
19
工具
—
可用率
—
Overview
A comprehensive multi-agent AI orchestration platform that combines secure Retrieval-Augmented Generation (RAG) with advanced research capabilities, web scraping, content analysis, voice interactions, MCP integration, and sophisticated memory management. Built with enterprise-grade security featuring hierarchical RBAC, document classification, and multi-agent security pipelines to ensure users only access authorized corporate knowledge while enabling powerful AI-driven research and analysis workflows.
Document Classification: Public/internal/confidential with tag-based filtering
Multi-Agent Orchestration: 20+ specialized agents for research, analysis, content creation, and security
Advanced Research: Web scraping, multi-phase research workflows, and content evaluation
Voice Integration: Google Gemini Live voice capabilities for conversational AI
MCP Integration: Model Context Protocol support for enhanced agent capabilities
Secure RAG: Enterprise-grade retrieval with access controls and audit trails
Advanced Memory: Persistent memory with semantic recall and working memory templates
Content Analysis: Learning extraction, evaluation scoring, and quality assessment
Audit-Ready: Citations, logs, and compliance validation throughout
Architecture
flowchart TD
%% Frontend Layer
subgraph "🎨 Frontend Layer"
A[Next.js App Router<br/>React 19 + TypeScript]
B[Cedar OS Integration<br/>Product Roadmap UI]
C[React Components<br/>Chat, Auth, Indexing]
D[Shared Libraries<br/>JWT, MDX, Hooks]
end
%% Backend Layer
subgraph "⚙️ Backend Layer"
E[Mastra Core<br/>Orchestration Engine]
F[AI Agents<br/>27+ Specialized Agents<br/>5 Domains]
G[Workflows<br/>10+ Multi-step Orchestration<br/>Complex Tasks]
H[Tools & Services<br/>50+ Tools<br/>Business Logic]
I[API Routes<br/>Chat & Indexing<br/>Streaming]
end
%% Data Layer
subgraph "💾 Data Layer"
J[(PostgreSQL + PgVector<br/>Vector Embeddings<br/>Document Storage)]
K[(Content Corpus<br/>Markdown Files<br/>RAG Source)]
end
%% External Services
subgraph "🔗 External Services"
L[Google Gemini<br/>AI Models]
M[OpenAI<br/>LLM Services]
N[Data Sources<br/>Alpha Vantage, Finnhub<br/>SerpAPI, Polygon]
O[Academic Sources<br/>ArXiv, Research APIs]
end
%% Agent Categories (detailed breakdown)
subgraph "🤖 Agent Categories"
P1[Governed RAG Agents<br/>identity, retrieve, rerank<br/>answerer, verifier]
P2[Domain Agents<br/>research, copywriter<br/>cryptoAnalysis, compliance]
P3[Specialized Agents<br/>productRoadmap, salesIntelligence<br/>stockAnalysis, voiceAgent]
P4[Orchestration Agents<br/>a2aCoordinator, assistant<br/>editor, evaluation]
end
%% Workflow Categories
subgraph "🔄 Workflow Categories"
Q1[Chat Workflows<br/>Multiple Versions<br/>Real-time Conversation]
Q2[Analysis Workflows<br/>Financial Analysis V3<br/>Research Workflow]
Q3[Content Workflows<br/>Content Generation<br/>Report Generation]
Q4[RAG Workflows<br/>Governed RAG Answer<br/>Governed RAG Index]
end
%% Tool Categories
subgraph "🛠️ Tool Categories"
R1[Data Tools<br/>Alpha Vantage, Finnhub<br/>Polygon, SerpAPI]
R2[Processing Tools<br/>Document Chunking<br/>PDF Conversion, Web Scraper]
R3[Analysis Tools<br/>Competitive Intelligence<br/>Compliance Check, Sales Analysis]
R4[Specialized Tools<br/>JWT Auth, Vector Query<br/>Graph RAG, Roadmap Tool]
end
%% Connections
A --> I
B --> I
C --> I
D --> I
I --> E
E --> F
F --> G
G --> H
H --> J
H --> K
J --> H
K --> H
F --> L
F --> M
G --> L
G --> M
H --> N
H --> O
F --> P1
F --> P2
F --> P3
F --> P4
G --> Q1
G --> Q2
G --> Q3
G --> Q4
H --> R1
H --> R2
H --> R3
H --> R4
%% Styling for Dark Mode
classDef darkMode fill:#1e1e1e,stroke:#ffffff,stroke-width:2px,color:#ffffff
class A,B,C,D,E,F,G,H,I,J,K,L,M,N,O,P,Q,R darkMode
class P1,P2,P3,P4 darkMode
class Q1,Q2,Q3,Q4 darkMode
class R1,R2,R3,R4 darkMode
class sys,web,api,db darkMode
class sys,web,api,db,auth,sec darkMode
class auth,policy,retrieve,rerank,answerer,verifier,research,report,copywriter,evaluation,learning,product,editor,self,assistant,starter,voice,mcp,template,network,rag,content darkMode
Research & Analysis Pipeline:
flowchart TD
%% Research Input
A[Research Query] --> B[Research Agent<br/>Query Decomposition<br/>Multi-phase Planning]
%% Initial Research Phase
B --> C[Web Scraping Tools<br/>Site Mapping, Content Extraction<br/>Batch Processing]
%% Analysis Phase
C --> D[Content Analysis<br/>Learning Extraction<br/>Evaluation Scoring]
%% Follow-up Research
D --> E[Iterative Research<br/>Follow-up Queries<br/>Depth Analysis]
%% Synthesis Phase
E --> F[Content Synthesis<br/>Report Generation<br/>Validation]
%% Output
F --> G[Research Report<br/>With Citations & Sources]
%% Supporting Tools
subgraph H["Research Tools"]
I[Web Scraper<br/>Content Extraction]
J[Evaluator<br/>Quality Assessment]
K[Learning Extractor<br/>Insight Mining]
L[Graph RAG<br/>Knowledge Graph]
end
%% Tool Integration
B -.-> I
C -.-> J
D -.-> K
E -.-> L
%% Styling for Dark Mode
classDef darkMode fill:#1e1e1e,stroke:#ffffff,stroke-width:2px,color:#ffffff
class A,B,C,D,E,F,G,H darkMode
Secure RAG Pipeline:
flowchart TD
%% Input Layer
A[User Query + JWT] --> B[Identity Agent<br/>JWT Validation & User Context]
%% Security Layer
B --> C[Policy Agent<br/>Role-Based Access Control<br/>Document Classification Filtering]
%% Retrieval Layer
C --> D[Retrieve Agent<br/>PgVector Similarity Search<br/>Security-Filtered Results]
%% Processing Layer
D --> E[Rerank Agent<br/>Relevance Scoring<br/>Context Ranking]
%% Generation Layer
E --> F[Answerer Agent<br/>Secure Response Generation<br/>Citation Assembly]
%% Verification Layer
F --> G[Verifier Agent<br/>Compliance Validation<br/>PII Detection<br/>Policy Enforcement]
%% Output Layer
G --> H[Secure Response<br/>With Citations & Metadata]
%% Multi-Agent Enhancement
I[Research Content Network<br/>Multi-Source Analysis] -.-> F
J[Governed RAG Network<br/>Cross-Agent Coordination] -.-> G
%% Supporting Components
subgraph K["Available Tools (12 total)"]
L[Vector Query Tool<br/>Secure Similarity Search]
M[Document Chunking Tool<br/>Intelligent Text Splitting]
N[Web Scraper Tool<br/>Content Extraction]
O[JWT Auth Tool<br/>Token Validation]
P[Content Tools<br/>Copywriter, Editor, Evaluation]
Q[Data Tools<br/>Weather, Roadmap, File Manager]
end
%% Tool Usage
D -.-> L
D -.-> M
F -.-> P
I -.-> N
I -.-> Q
B -.-> O
%% Styling for Dark Mode
classDef darkMode fill:#1e1e1e,stroke:#ffffff,stroke-width:2px,color:#ffffff
class A,B,C,D,E,F,G,H,I,J,K,L,M,N,O,P,Q darkMode
Getting Started
Prerequisites
Node.js >=20.9.0
Docker and Docker Compose
Git
OpenAI API key
Quick Setup
Clone and install:
git clone https://github.com/ssdeanx/governed-rag-ai.git
cd mastra-governed-rag
npm install
Configure environment:
cp .env.example .env
# Edit .env with your API keys and other settings:
# - OPENAI_API_KEY or GOOGLE_GENERATIVE_AI_API_KEY (required for AI models)
# - SERPAPI_API_KEY (optional, for web search tools - get from https://serpapi.com/manage-api-key)
# - SUPABASE credentials (required for authentication)
# - DATABASE_URL (required for PostgreSQL)
Set up authentication (Supabase):
This application uses Supabase for user authentication. You need to:
# Only needed if using local PostgreSQL
docker-compose up -d db
Index documents:
npm run cli index
Start development:
npm run dev # http://localhost:3000
AI & Research Capabilities
Multi-Agent Orchestration: 20+ specialized agents for research, analysis, content creation, and security
Advanced Research Workflows: Multi-phase research with web scraping, content evaluation, and learning extraction
Secure Vector Search: Filtered retrieval based on user permissions with PgVector
Content Analysis: Automated learning extraction, quality evaluation, and insight mining
Web Search & Data Collection: SerpAPI-powered Google Search, News, Trends, Shopping (Amazon, Walmart, eBay, Home Depot), Academic (Google Scholar), Finance, and Local Business (Yelp) searches
Web Scraping Tools: Comprehensive web content extraction and processing
Contextual Reranking: Relevance scoring with security constraints
Citation Generation: Source attribution with access verification
Voice Integration: Google Gemini Live voice capabilities for conversational AI
MCP Integration: Model Context Protocol support for enhanced agent tools
Advanced Memory: Persistent memory with semantic recall and working memory templates
Quality Assurance: Automated evaluation scoring and compliance validation
Streaming Responses: Real-time answer generation with SSE
AI Agents
This system employs a comprehensive multi-agent architecture with 20+ specialized agents:
Core RAG Agents
Identity Agent - Validates user authentication and permissions
Policy Agent - Enforces access control and security policies
Retrieve Agent - Performs intelligent document retrieval with security filtering
Rerank Agent - Ranks and scores retrieved documents for relevance
Answerer Agent - Generates secure responses with citations
Verifier Agent - Validates responses for compliance and accuracy
Specialized Agents
Research Agent - Conducts in-depth research and analysis
Report Agent - Generates structured reports and summaries
Copywriter Agent - Creates marketing and communication content
Evaluation Agent - Assesses content quality and relevance
Learning Extraction Agent - Identifies and extracts key learnings
Product Roadmap Agent - Analyzes product strategy and roadmaps
Editor Agent - Reviews and improves content quality
Self-Referencing Agent - Maintains context across conversations
Assistant Agent - Provides general AI assistance
Starter Agent - Handles initial query processing
Voice Agent - Manages voice-based interactions and audio processing
MCP Agent - Integrates with Model Context Protocol tools
Template Reviewer Agent - Reviews and validates templates and workflows
Network Agents
Research Content Network - Orchestrates multi-agent research workflows
Governed RAG Network - Manages secure RAG operations across agents
Each agent follows a single-tool-call policy, ensuring predictable and auditable AI behavior while maintaining security governance throughout the entire pipeline.
Research & Analysis Features
Beyond secure RAG, this system provides comprehensive research and analysis capabilities:
Multi-Phase Research Workflows
Query Decomposition: Breaking down complex research questions into focused search queries
Web Content Extraction: Advanced web scraping with site mapping and content cleaning
Content Evaluation: Automated assessment of information quality and relevance
Learning Extraction: AI-powered identification and extraction of key insights
Iterative Research: Follow-up research based on initial findings
Report Synthesis: Structured report generation with citations and sources
Advanced Analysis Tools
Graph RAG Queries: Complex knowledge graph-based retrieval and analysis
Content Quality Scoring: Automated evaluation using LLM-based scorers
Semantic Memory: Persistent context with working memory templates
Compliance Validation: PII detection and policy enforcement
Multi-Source Synthesis: Combining information from diverse sources
Research Agent Example
// Research agent with multi-phase workflow
const researchAgent = new Agent({
name: 'Research Agent',
instructions: `
PHASE 1: Deconstruct topic into focused search queries
PHASE 2: Web scraping and content extraction
PHASE 3: Content evaluation and learning extraction
PHASE 4: Follow-up research and synthesis
`,
tools: {
webScraperTool, evaluateResultTool, extractLearningsTool,
graphRagQueryTool, mdocumentChunker
},
memory: pgMemory,
scorers: {
relevancy: createAnswerRelevancyScorer(),
safety: createToxicityScorer()
}
})
AI Tools
The system includes 12 specialized tools that agents can invoke to perform specific operations:
Type-Safe Development: Full TypeScript with Zod schema validation
Hot Reload: Concurrent development for frontend and backend
Comprehensive Testing: Vitest framework with service and workflow tests
Docker Integration: Containerized development environment
Extensive Documentation: Complete API references and guides
Project Structure
mastra-governed-rag/
├── app/ # Next.js routes and API endpoints
├── components/ # React UI components
├── cedar/ # Cedar OS interactive components
├── corpus/ # Sample documents for indexing
├── docs/ # Comprehensive documentation
├── src/
│ ├── mastra/ # Core Mastra implementation
│ │ ├── agents/ # AI agents (20+ specialized agents for research, analysis, security)
│ │ ├── workflows/ # Orchestrated agent workflows (10 workflows for research, RAG, content)
│ │ ├── tools/ # Reusable agent tools (12 tools for web scraping, analysis, search)
│ │ ├── services/ # Business logic and integrations (13 services)
│ │ ├── networks/ # Multi-agent orchestration networks
│ │ ├── schemas/ # Data validation schemas (Zod)
│ │ ├── config/ # Configuration and external services
│ │ └── policy/ # Access control policies and ACL
│ └── cli/ # Command-line interface
├── lib/ # Shared utilities and client libraries
└── docker-compose.yml # Development services
Usage Examples
Research Query with Multi-Phase Analysis
// Advanced research with web scraping and analysis
const researchRequest = {
question: "What are the latest developments in AI agent orchestration frameworks?",
researchDepth: 3, // Multi-phase research
includeWebSources: true,
evaluationRequired: true
}
// Response includes:
// - Multi-source web research
// - Content evaluation scores
// - Extracted learnings and insights
// - Citations and source validation
// - Structured synthesis report
Secure RAG Query with Authentication
// Generate JWT for a finance viewer
const jwt = generateDemoJWT('finance')
// Query the system
const response = await fetch('/api/chat', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
jwt,
question: 'What is the expense reimbursement policy?',
}),
})
const result = await response.json()
// Returns secure answer with citations
Role-Based Access Demonstration
# Test different user roles
npm run cli query "$(npm run jwt:finance)" "What are expense approval thresholds?"
npm run cli query "$(npm run jwt:hr)" "What is executive compensation policy?"
npm run cli query "$(npm run jwt:public)" "What is our company mission?"
[!NOTE]
Each role sees different results based on their access level. See Demo Roles for complete examples.
API Reference
Core Endpoints
POST /api/chat - Secure RAG queries with streaming responses
POST /api/index - Document indexing with classification
GET /api/auth/* - Authentication endpoints
Request/Response Format
// Chat request
{
"jwt": "eyJhbGciOiJIUzI1NiIs...",
"question": "What is the company policy on X?"
}
// Streaming response
data: {"content": "According to the policy..."}
data: {"done": true, "citations": [...]}
We welcome contributions! Please see our Contributing Guide for details.
License
This project is licensed under the MIT License - see the LICENSE file for details.
Built with ❤️ by Mastra Community. Questions? Open an issue.
S
Secure Rag Multi Agent
ssdeanx·19 顆星·
Secure Retrieval-Augmented Generation (RAG) with role-based access control using Mastra AI orchestration, with JWT Secure, also Deep Research built with Cedar & Next.js. Gemini, OpenAI, Openrouter, &
Secure Retrieval-Augmented Generation (RAG) with role-based access control using Mastra AI orchestration, with JWT Secure, also Deep Research built with Cedar & Next.js. Gemini, OpenAI, Openrouter, & Gemini CLI usable. Also exposes tools, agents as MCP Server Also A2A Agent Cards
上次檢查時間:2026-10-28。 Secure Retrieval-Augmented Generation (RAG) with role-based access control using Mastra AI orchestration, with JWT Secure, also Deep Research built with Cedar & Next.js. Gemini, OpenAI, Openrouter, & Gemini CLI usable. Also exposes tools, agents as MCP Server Also A2A Agent Cards