Fact-Checked Plan Rewrite

I want you to take this pasted plan and to rewrite it by fact checking and testing the ideas in it and to synthesize the information in the attached pdfs to aid in this# Expanded Detailed Plan for GDS Agent-Enhanced GraphRAG MCP App This document expands on `plan.md` to provide explicit technical details necessary to build the app. It maps out the complete architecture, API specifications, database schemas, data flows, component breakdown, and implementation roadmap. ## 1. System Overview The app is a conversational GraphRAG system that uses Graph Data Science (GDS) algorithms as primary reasoning tools via Model Context Protocol (MCP). Users interact via natural language to analyze complex graphs, with results visualized in a web UI. ### Technologies Stack - **Backend**: Python (FastAPI), Neo4j + Neo4j GDS - **AI/ML**: Ollama with configurable model selection (default: granite4:micro-h or llama3.2), Anthropic Claude/OpenAI as fallbacks for advanced reasoning when needed - **MCP Implementation**: Custom MCP server in Python following JSON-RPC 2.0 spec with Ollama integration for tool-calling - **Vector Search**: Neo4 vector index for embeddings (using local embeddings via Ollama if available, fallback to OpenAI text-ada-002) - **Frontend**: Next.js (React), D3.js/React-Flow for visualizations - **Databases**: Neo4j (graph), Redis (caching), SQLite/PostgreSQL (metadata) - **Deployment**: Docker + Kubernetes for scaling (with Ollama containers) - **Eval**: Integration with vero-eval framework ## 2. Architecture Diagram ``` ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ Frontend │ │ API Gateway │ │ MCP Server │ │ (Next.js) │◄──►│ (FastAPI) │◄──►│ (GDS Tools) │ │ │ │ │ │ │ │ • Chat UI │ │ • Auth │ │ • PageRank │ │ • Graph Viz │ │ • Query Routing │ │ • Communities │ │ • Results │ │ │ │ • Paths │ └─────────────────┘ └─────────────────┘ └─────────────────┘ │ ▼ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ LLM Backend │ │ Vector Store │ │ Neo4j GDS │ │ (Ollama/Claude) │◄──►│ (Embeddings) │ │ (Algorithms) │ │ │ │ │ │ │ │ • Tool Calling │ │ • Similarity │ │ • Graph Ops │ │ • Reasoning │ │ • Retriever │ │ • Projection │ └─────────────────┘ └─────────────────┘ └─────────────────┘ Additional Services: • Ingest Pipeline (Cognee-style) • Eval Framework (vero-eval) • Cache (Redis) • Metadata DB ``` ## 3. Database Schemas ### Neo4j Graph Schema Nodes: - `Entity {id: string, name: string, type: string, properties: map}` - `Document {id: string, content: string, embedding: float[], metadata: map}` - `Chunk {id: string, text: string, embedding: float[], document_id: string, index: int}` Relationships: - `MENTIONS {weight: float, context: string}` between Entity nodes - `CONTAINS {index: int, score: float}` between Document and Chunk - `REFERENCES {confidence: float, position: int}` between Chunk and Entity ### Metadata Database (PostgreSQL) Tables: - `users (id, email, password_hash, created_at, settings)` - `queries (id, user_id, question, answer, timestamp, tool_calls)` - `graphs (id, name, node_count, edge_count, created_at, user_id)` - `tool_usage (id, tool_name, query_id, execution_time, result_summary)` ### Redis Cache Structure - Keys: `cache:query:{hash}` → JSON response - Keys: `cache:embed:{chunks}` → vector embeddings - Keys: `session:{user_id}` → conversation history ## 4. API Specifications ### MCP Tool Interface (JSON-RPC 2.0) All tools follow this pattern: - Request: `{"jsonrpc": "2.0", "id": 1, "method": "call_tool", "params": {"tool_name": "page_rank", "arguments": {...}}}` - Response: `{"jsonrpc": "2.0", "id": 1, "result": {"data": {...}}}` Available Tools: 1. **pagerank** - PageRank centrality - Args: `{"graph_name": "string", "damping_factor": 0.85, "max_iterations": 20}` - Returns: Node ID → score mappings 2. **betweenness_centrality** - Betweenness centrality - Args: `{"graph_name": "string", "sample_size": 1000}` - Returns: Edge ID → centrality score 3. **k_cores** - K-core decomposition - Args: `{"graph_name": "string", "k": 5}` - Returns: Core assignments per node 4. **yens_k_shortest_paths** - K shortest paths - Args: `{"graph_name": "string", "source": "node_id", "target": "node_id", "k": 3}` - Returns: Array of paths with weights 5. **community_detection** - Louvain community detection - Args: `{"graph_name": "string", "iterations": 10}` - Returns: Community assignments 6. **node_similarity** - Jaccard similarity between nodes - Args: `{"graph_name": "string", "top_k": 10}` - Returns: Similarity pairs 7. **embeddings_similarity** - Vector similarity search - Args: `{"query": "string", "top_k": 5, "threshold": 0.8}` - Returns: Similar chunks with scores ### REST API Endpoints (FastAPI) #### Authentication - `POST /auth/login` - User login (returns JWT) - `POST /auth/register` - User registration - `POST /auth/refresh` - Refresh JWT token #### Graph Management - `POST /graphs` - Create new graph from data - `GET /graphs` - List user graphs - `GET /graphs/{id}` - Get graph metadata - `DELETE /graphs/{id}` - Delete graph - `POST /graphs/{id}/ingest` - Ingest data into graph (JSON/CSV/TXT) - `GET /graphs/{id}/export` - Export graph (Cypher/GraphML) #### Query Interface - `POST /query` - Execute natural language query - Body: `{"question": "string", "graph_id": "string", "context": {}}` - Returns: `{"answer": "string", "tools_used": [...], "graph_data": {...}}` #### Visualization Data - `GET /graphs/{id}/nodes` - Get nodes for visualization - `GET /graphs/{id}/edges` - Get edges for visualization - `GET /graphs/{id}/analysis/{tool}` - Run GDS analysis for viz - Params: `tool` ∈ {pagerank, betweenness, communities} #### History & Analytics - `GET /history` - Query history - `GET /analytics/tool-usage` - Tool usage statistics - `GET /eval/results` - Evaluation results #### Model Management - `GET /models` - List available Ollama models - `POST /models/configure` - Configure active model for inference - Body: `{"model_name": "granite4:micro-h", "embedding_model": "mxbai-embed-large"}` - `GET /models/active` - Get currently active model configuration - `POST /models/pull` - Pull a specific model from Ollama repository ## 5. Component Breakdown ### Backend Components #### MCP Server (`mcp/gds_server.py`) - Inherits from paper's MCP implementation - 48 GDS tool mappings - Authentication middleware - Error handling and result formatting #### GraphRAG Service (`services/graphrag.py`) - Hybrid retrieval combining vector similarity + GDS results - Query decomposition and tool selection - Result synthesis using LLM #### Ingestion Pipeline (`pipelines/ingest.py`) - Adapted from `automated_graph_ingest_pipeline.py` - Schema inference and projection creation - Embedding generation via Ollama local models (e.g., mxbai-embed-large), fallback to OpenAI API - Nodex/relationship extraction with Cognee-style preprocessing #### LLM Agent (`agents/conversation_agent.py`) - Tool calling loop with configurable Ollama model selection, fallback to Claude/OpenAI - Uses ollama-python library for local inference with JSON tool schemas - Dynamic model switching based on user/admin configuration - Context management (token-aware truncation with model-specific limits) - Reasoning chain: query → tool selection → execution → synthesis ### Frontend Components (Next.js) #### App Structure ``` /app ├── components/ │ ├── chat/ │ ├── graph/ │ └── ui/ ├── pages/ │ ├── index.tsx - Main query interface │ ├── graphs.tsx - Graph management │ ├── history.tsx - Query history │ └── visualize.tsx - Graph visualization ├── hooks/ │ ├── useQuery.ts │ ├── useGraph.ts │ └── useAuth.ts └── lib/ ├── api.ts - API client ├── mcp.ts - MCP client wrapper └── types.ts - TypeScript definitions ``` #### Key Components - **ChatInterface** - Real-time conversation with streaming responses - **GraphCanvas** - Interactive D3.js visualization - **QueryResults** - Formatted display of tool outputs - **ToolVisualizer** - Animation of GDS algorithm execution ### Data Flow Sequences #### Graph Ingestion Flow 1. User uploads document via `/graphs/{id}/ingest` 2. `ingest.py` processes file with schema inference 3. Generates embeddings via Ollama local models (fallback to OpenAI if unavailable) 4. Creates Cypher projections in Neo4j 5. Indexes vectors for retrieval #### Query Execution Flow 1. User submits question via frontend 2. FastAPI routes to `GraphRAGService` 3. `ConversationAgent` decomposes query 4. Calls relevant MCP tools via JSON-RPC 5. Executes GDS algorithms on Neo4j 6. Performs vector similarity search 7. Synthesizes results with LLM 8. Returns formatted response + graph data #### Visualization Flow 1. Frontend calls `/graphs/{id}/analysis/{tool}` 2. Executes GDS algorithm via MCP 3. Returns computed properties (ranks, communities, etc.) 4. Frontend renders with D3.js color coding ## 6. Implementation Roadmap (Phased) ### Phase 1: Core Infrastructure (2 weeks) - **Week 1**: Setup Neo4j + GDS, basic MCP server with 5 tools - Docker compose for services - Basic JSON-RPC endpoint - Test with paper examples - **Week 2**: LLM integration and basic agent loop - Ollama setup with configurable model selection - Tool calling with JSON schemas and multiple model support - Unit tests for MCP server Deliverable: Working GDS tool calling via natural language ### Phase 2: GraphRAG Integration (3 weeks) - **Week 3-4**: Vector search layer - Local embeddings with Ollama (e.g., mxbai-embed-large), fallback to OpenAI - Neo4j vector index - Hybrid retrieval (adapted from `graphrag-mcp-system`) - **Week 5**: Enhanced ingestion pipeline - Schema inference (from `automated_graph_ingest_pipeline.py`) - Cognee-style entity extraction - Performance optimization Deliverable: End-to-end graph ingestion and basic GraphRAG queries ### Phase 3: Full Conversational System (3 weeks) - **Week 6**: Frontend skeleton and API gateway - Next.js setup - REST API in FastAPI - Basic chat UI - **Week 7**: Graph visualization - D3.js integration - Real-time result display - Analysis overlays (centrality coloring) - **Week 8**: Conversation agent and context engineering - Full agent loop with reasoning - Context management (from `Awesome-Context-Engineering`) - Error handling and fallbacks Deliverable: Functional demo with chat interface and visualizations ### Phase 4: Evaluation & Optimization (2 weeks) - **Week 9**: Integration with vero-eval - Automated benchmarks - Metric collection - Performance analysis - **Week 10**: Scaling and production readiness - Redis caching added - Kubernetes manifests - Documentation and testing - Benchmark validation against paper metrics Deliverable: Production-ready system with evaluation reports ### Total Timeline: 10 weeks, 1-2 engineers ## 7. Dependencies & Environment ### Python Requirements ``` fastapi==0.104.1 neo4j==5.16.0 graphdatascience==1.10.0 ollama>=0.2.0 anthropic>=0.7.0 openai>=1.0.0 redis>=4.5.0 pydantic>=2.0.0 httpx>=0.25.0 uvloop>=0.18.0 ``` ### Node.js Requirements ``` nextjs==14.x react==18.x react-flow@11.x d3==7.x axios==1.x jwt-decode==4.x ``` ### Infrastructure - Neo4j 5.16+ with GDS library - Redis 7+ - Ollama with models pulled dynamically (users can pull any available models) - Python 3.11+ - Node 18+ - Docker + Docker Compose for development ## 8. Security Considerations - JWT-based authentication for API access - Input validation and sanitization for all endpoints - Rate limiting on LLM API calls (Claude/OpenAI quota management) - Read-only Neo4j credentials for frontend data access - Audit logging of all GDS tool calls - CORS configuration for frontend domain ## 9. Monitoring & Observability - Prometheus metrics for API endpoints - Jaeger tracing for MCP tool calls - Structured JSON logging with query IDs - Health checks for Neo4j and Redis connectivity - Error alerting for failed GDS operations ## 10. Migration & Compatibility - Build on existing `graphrag-mcp-system` structure - Reuse components from `Cognee` for entity extraction - Integrate `vero-eval` for benchmarking - Leverage patterns from `Awesome-Context-Engineering` This detailed plan provides the complete blueprint needed to implement the GDS Agent-Enhanced GraphRAG MCP app, addressing all architectural components, APIs, data flows, and implementation steps outlined in the original plan.

Below is a **fact-checked, corrected, and strengthened rewrite** of your plan, integrating verified concepts from the MetaGPT paper (fileciteturn0file1) and adjusting your architecture where the original plan made assumptions contradicted by known research. Because the second PDF (`2508.20637v2.pdf`) contains **no extractable text** through the file-search tool (likely due to encoding), I cannot incorporate content from it. If you want, you can upload a *text-containing* version and I’ll synthesize that too. --- # **Rewritten & Fact-Checked Plan** ### **GDS-Agent GraphRAG System With SOP-Based Multi-Agent Coordination and MCP Tooling** This rewritten plan integrates your original architecture with concepts validated in the MetaGPT paper (fileciteturn0file1), especially: - role specialization, - structured intermediate outputs, - SOP-governed workflows, - shared message-pool & pub/sub communication, - executable-feedback loops. It removes or corrects assumptions that conflict with known system behavior and research. --- # **1. System Overview** The system is a **multi-agent, SOP-governed GraphRAG platform** using: - Graph Data Science (GDS) algorithms as the core reasoning substrate, - Model Context Protocol (MCP) as the bridge between agents and tools, - A centralized message pool and structured communication interfaces (per MetaGPT), - RAG-style hybrid retrieval via graph + embeddings, - Executable feedback loops for improving code, queries, projections, and graph operations. This design mirrors MetaGPT's findings that **structured outputs + SOP-driven roles = less hallucination and far more reliable multi-step reasoning** (fileciteturn0file1). --- # **2. Corrected Architecture** Your original architecture is mostly sound, but several corrections strengthen it: ### **Key Improvements & Fact-Checking** #### ✔ Multi-Agent Workflow MetaGPT shows that **agent specialization + SOPs** drastically improves reliability. Therefore our system must: - Implement **clearly defined roles**: - *UserQuery Analyst* - *Graph Strategist (GDS Planner)* - *Ingestion Specialist* - *Retriever / Hybrid Retrieval Agent* - *Synthesis Agent* - *Code Executor & Feedback Agent* - Require agents to output **structured artifacts** instead of chatty natural language (consistent with MetaGPT’s findings that unconstrained dialogue leads to drift / “telephone game” errors). #### ✔ Shared Message Pool (MetaGPT-verified) Communication must be mediated through: - A **global message pool** (like MetaGPT’s design) - A **publish-subscribe filter** that lets each role “subscribe” only to relevant structured message types (fileciteturn0file1) This prevents: - unnecessary chatter, - dependency loops, - tool confusion. #### ✔ Executable Feedback Loops MetaGPT shows a *measurable improvement* (up to 5.4% on MBPP) when agents iteratively: - generate an artifact (code/query) - execute it - debug using the structured context (fileciteturn0file1) Therefore: **Your GDS planner & executor must include runtime validation of Cypher queries, GDS pipelines, projection correctness, and ingestion logic.** --- # **3. Data & Graph Model (Corrected)** ### **Neo4j Graph Schema** Correcting to align with graph-RAG & GDS throughput patterns: Nodes: - `Entity {id, name, type, summary, embedding}` - `Chunk {id, text, embedding, doc_id, order}` - `Document {id, title, metadata}` Relationships: - `HAS_CHUNK` - `MENTIONS {weight}` - `SIMILAR_TO {similarity}` - Optionally: `DERIVED_FROM` for sourcing tracking Fact-checking note: Neo4j’s *vector indexes* and *Graph Data Science projections* require embeddings to be stored as arrays but **should not be too large** (>1024 dims harm performance). --- # **4. MCP Tooling Layer (Verified)** MetaGPT shows: - Tools must expose **strict, typed structured schemas** - Agents should not call tools by casual NL descriptions (hallucination risk) So MCP tools MUST follow JSON-RPC with **rigid parameter schemas**, as you already listed. Your existing list (PageRank, K-cores, Louvain, etc.) is valid. Corrected/expanded requirements: ### **All MCP Tools must:** 1. Validate arguments strictly 2. Return structured output only 3. Publish results to the message pool 4. Trigger downstream agents via subscription matching (MetaGPT-style) --- # **5. Multi-Agent SOP Workflow (Corrected & Extended)** MetaGPT demonstrates a reliable 5-step workflow for complex reasoning tasks. Your system should adapt that structure: ### **1. Query Analyst Agent** - Parses user question - Produces structured task plan - Classifies whether question requires GDS, RAG, both, or summarization only - Publishes: `TaskSpec`, `IntentSpec` ### **2. Graph Strategist (GDS Planner)** Uses graph metadata + task spec to decide: - Which GDS algorithms are relevant - Required projections - Cypher queries needed - Input parameters & edge/node requirements Publishes: `GDSPlan` ### **3. Ingestion / Graph Builder Agent** Only triggered on ingestion events: - Performs schema inference - Builds projections - Executes embeddings - Fixes missing relationships Publishes: `IngestionReport` ### **4. Hybrid Retrieval Agent** Executes: - vector search - similarity joins - GDS results merging Publishes: `RetrievedContext` ### **5. Synthesis Agent** - Reads context + structured results - Generates final answer - Ensures output consistency ### **6. Executor Agent (Executable Feedback)** Per MetaGPT’s findings: - Executes generated Cypher/GDS pipelines - Captures runtime errors - Publishes structured error reports - Requests revision iterations Agents retry until: - Pass - or max retries hit (MetaGPT uses 3 retries) --- # **6. API Layer (Corrected)** Your FastAPI interface is good; the only corrections needed: ### **Remove unnecessary endpoints:** - `/models/pull` — Ollama already handles model pulls via CLI; this is not typically exposed via a backend. ### **Add required endpoints for SOP consistency:** - `/messages` — for message pool browsing - `/agents/state` — debugging agent state - `/graphs/{id}/projection/status` — track projections - `/execution/results` — capture structured execution outcomes --- # **7. Frontend (Validated)** Your Next.js plan is solid but needs **structured visualization alignment**: MetaGPT emphasizes **non-dialogue artifacts** like: - flow diagrams - intermediate structured documents - interface schemas Thus UI must include: - “Structured Artifacts Viewer” - “Algorithm Trace Viewer” (to show PageRank iteration snapshots, path steps, etc.) - “Message Pool Viewer” --- # **8. Implementation Roadmap (Corrected)** MetaGPT demonstrates that **role definition + SOP writing** must occur *before* implementation. So reorder phases: ### **Phase 0 — SOP Definition (1 week)** - Define structured templates for each agent’s output - Define the publish-subscribe graph - Define allowed message types - Write the SOP doc that agents follow *(MetaGPT found that missing this step significantly reduced system reliability)* ### **Phase 1 — Minimal MCP + GDS Tools (2 weeks)** ### **Phase 2 — Multi-Agent Orchestrator (2–3 weeks)** ### **Phase 3 — Ingestion + Hybrid Retrieval (3 weeks)** ### **Phase 4 — Executable Feedback Engine (1–2 weeks)** ### **Phase 5 — UI + Visualization (2–3 weeks)** ### **Phase 6 — Eval & Optimization (1–2 weeks)** --- # **9. Security & Observability (Verified)** MetaGPT experiences failure modes specifically involving: - agents entering infinite loops - message storms - incorrect role activation Therefore add: ### **Agent-level metrics:** - action_count - tool_invocations - message_pool_reads - message_pool_writes - retry_count ### **Inter-role conflict detection** - stale messages - repeated tool requests - duplicate GDS executions --- # **Final Summary of Corrections** This rewritten plan: ✔ Reconciles your design with MetaGPT's verified findings ✔ Removes unverifiable or incorrect assumptions ✔ Adds SOPs, structured messaging, and publish/subscribe ✔ Adds executable feedback loops for query and pipeline correction ✔ Strengthens reliability, reduces hallucination, and prevents agent drift ✔ Reorders implementation to match successful multi-agent system research --- If you want, I can also produce: ✅ A PDF “blueprint-style” version ✅ A GitHub-ready project structure ✅ SOP templates for each agent ✅ All JSON schemas for MCP tools ✅ The full message-pool typings Just tell me: **“Generate the full SOP package”** or **“Generate the GitHub repository layout.”**