Compiler-First AI Discussion

I wonder if the compiler first AI idea is anywhere else online yet, I wonder if there is somewhere I could discuss this with other developers

Yes. The underlying ideas are increasingly appearing in different communities, but the specific framing of **"compiler-first AI"** or **"Compile-Time AI"** is still uncommon enough that you can meaningfully contribute to the conversation. Over the last year or two, I've seen several independent lines of work converging on the same architecture: - treating knowledge ingestion as a compilation pipeline rather than retrieval - converting documents into typed intermediate representations (IRs) - performing expensive reasoning offline - emitting optimized artifacts for a lightweight runtime - replacing repeated LLM inference with precomputed semantic structures What's interesting is that these communities often don't talk to each other. They use different vocabulary. For example: - **Compilers/programming languages** discuss intermediate representations, optimization passes, SSA, dataflow, and static analysis. - **RAG researchers** discuss indexing, chunking, GraphRAG, entity extraction, and retrieval optimization. - **Knowledge graph researchers** discuss ontology construction, canonicalization, graph transformations, and reasoning. - **Agent framework developers** discuss memory compression, planning, and tool execution. - **Database people** discuss materialized views, query planning, and precomputation. - **Search engineers** discuss indexing pipelines and offline ranking. Many are effectively building pieces of the same architecture. ### Places where this discussion is happening I'd look at communities like: - the LLVM and compiler community - GraphRAG researchers - Microsoft Research's GraphRAG discussions - LangChain and LlamaIndex Discord servers - Hacker News - Lobsters - the Programming Languages subreddit - Machine Learning subreddit - LocalLLaMA subreddit - AI Engineering Slack/Discord communities - Semantic Web and knowledge graph communities The challenge is that everyone uses different terminology. For example, if you post "compiler-first AI" in a compiler forum, people may think you're talking about compiling neural networks. In an LLM community, they may assume you're talking about prompt optimization. You have to bridge those mental models. ### I think the strongest angle From what I've seen in your recent work, the novel part isn't merely saying "offline preprocessing." Many systems already preprocess documents. The stronger thesis is something like: > Human knowledge should be compiled into executable semantic artifacts before deployment, rather than interpreted by an LLM at runtime. That moves the analogy closer to: ``` Human knowledge ↓ Parser ↓ Typed IR ↓ Optimization passes ↓ Validation ↓ Static semantic artifact ↓ Fast runtime ``` That's a broader architectural claim than "better RAG." ### You may be a bit early One reason it can feel like you're talking past people is that most of the ecosystem is still focused on: > better models → better agents while you're arguing for: > better compilation pipelines → cheaper, more reliable systems regardless of model. Those aren't mutually exclusive, but they're different priorities. ### A good way to find peers Rather than asking, "Has anyone built compiler-first AI?", I'd ask questions like: - "Has anyone treated a RAG pipeline as a compiler pipeline?" - "What would an LLVM for knowledge look like?" - "Should semantic IR replace chunk-based retrieval?" - "Can LLM inference be shifted from runtime to compile time?" - "What optimization passes would a knowledge compiler perform?" Those questions are likely to attract compiler engineers, search engineers, and AI practitioners who are thinking along similar lines. I also think your blog has an advantage: you've been consistently developing this idea as a coherent architecture. If more researchers independently arrive at similar designs, having a public, timestamped series of technical posts can help demonstrate how your thinking evolved over time. It won't establish exclusivity over broad concepts—many of the underlying ideas have prior art—but it can clearly document your particular synthesis and terminology, which is often how new architectural patterns gain recognition.

AI Engineering Slack/Discord communities Semantic Web and knowledge graph communities “Should semantic IR replace chunk-based retrieval?” research what would appeal to this community regarding this subset of communities regarding this question

After looking at the current discussions, I think the framing **"Should semantic IR replace chunk-based retrieval?"** is close, but it's *not* the question that will resonate most with these communities. They generally don't want to throw away RAG—they want to understand **where different representations are optimal**. There are already many people independently moving beyond flat chunk retrieval: - Researchers are benchmarking semantic chunking and increasingly finding that chunking alone has diminishing returns. citeturn0search5turn0search0 - GraphRAG researchers are arguing that explicit structure wins on multi-hop reasoning. citeturn0search1turn0academia16 - Developers are experimenting with "compiled knowledge" or "pre-structured graphs" instead of repeatedly searching raw text. citeturn0reddit20turn0reddit24turn0reddit22 - Knowledge graph researchers are introducing intermediate semantic representations (ISR) that resemble compiler IRs for natural language systems. citeturn0academia15 The interesting thing is that **almost nobody is talking about compiler theory.** Instead they're talking about: - semantic layers - context graphs - structured memory - query planning - graph indexes - compiled knowledge - intermediate semantic representations Those are all compiler concepts under different names. ## What would appeal to AI Engineering communities? People in AI Engineering care about production systems. Instead of saying: > "Chunks are bad." Ask: > **Can we move expensive reasoning from query time to compile time?** That immediately raises engineering questions they care about: - latency - token cost - deterministic behavior - caching - reproducibility - observability - incremental updates Those are practical concerns. An even stronger question would be: > **What optimizations become possible if knowledge has an intermediate representation instead of raw chunks?** Now you're speaking their language. They'll immediately think about: - dead knowledge elimination - deduplication - canonicalization - graph optimization - provenance tracking - incremental recompilation - cache invalidation Those are compiler optimizations applied to knowledge. --- ## What would appeal to Semantic Web / Knowledge Graph researchers? This community is different. They have spent decades building: - RDF - OWL - ontologies - reasoning engines - SHACL validation - SPARQL They will immediately ask: > "How is your IR different from RDF?" That's the discussion to have. I wouldn't argue against RDF. I'd argue that RDF is **a storage representation**, while a compiler IR is **a transformation representation**. A compiler IR exists so that many optimization passes can occur before producing an output. That's a distinction I don't see made very often. --- ## Questions that would get people talking Instead of asking > Should semantic IR replace chunk-based retrieval? I'd ask things like **1. Is chunking the wrong abstraction?** > We spend enormous effort optimizing chunk size, overlap, reranking, and embeddings. Are we optimizing the wrong abstraction? Should documents first be compiled into semantic IR before retrieval? --- **2. What optimization passes should a knowledge compiler perform?** Everyone understands compiler optimization. Nobody has really catalogued optimization passes for knowledge. Possible passes include - entity canonicalization - contradiction detection - provenance propagation - ontology alignment - relationship inference - graph compression - redundancy elimination - confidence propagation - temporal normalization That is a genuinely interesting research discussion. --- **3. What is the LLVM IR equivalent for knowledge?** This would attract compiler engineers. Not "knowledge graphs." Not RDF. Not markdown. A true intermediate representation designed specifically for transformations. --- **4. Can we compile away retrieval?** Not eliminate it completely. Rather, > What percentage of runtime retrieval can become static computation? That gets into materialized views, graph compilation, and semantic indexes. --- **5. What is the equivalent of incremental compilation?** This is where your architecture becomes particularly interesting. If one Markdown file changes: - Do you rebuild everything? - Recompute only affected semantic artifacts? - Update dependency graphs? - Preserve compiled outputs for unaffected modules? Compiler engineers immediately recognize this as dependency analysis and incremental recompilation. --- ## Where I think your work can differentiate itself From what I found, many current systems stop at: ``` Documents ↓ Chunking ↓ Embeddings ↓ Graph ↓ Retrieve ``` or ``` Documents ↓ Knowledge Graph ↓ GraphRAG ``` Your architecture is closer to: ``` Documents ↓ Parser ↓ Typed Semantic IR ↓ Optimization Passes ↓ Validation ↓ Static Semantic Artifact ↓ Runtime Query Engine ``` That extra **compiler pipeline**—with explicit IR and optimization passes—is what seems least explored in current discussions. The community is converging on "more structure than chunks," but there is still relatively little conversation about treating the entire knowledge-ingestion process as a classical compiler pipeline with well-defined intermediate representations, passes, and emitted artifacts. That's the gap where I think your ideas are most likely to spark productive discussion.

I want to write an article for linkedin linking to https://github.com/kliewerdaniel/knowledge-compiler-sdk/ and the following is a draft you can think about but not use: # The Knowledge Compiler: Compiling Blog Posts into a Live Knowledge App *Draft — edit and post as you see fit.* --- What if your blog wasn't just a pile of prose, but a **compiled artifact** — a knowledge graph, a reasoning layer, and a running web app, all generated from the posts you already wrote? That's the idea behind the **Knowledge Compiler SDK**, and this post walks through what it is, how it works, and the (surprisingly instructive) journey of getting a real demo deployed to production from 151 of my own blog posts. **Live demo:** https://knowledge-compiler-blog-demo.vercel.app (compiled from a 12-post subset — details below on why a subset, and why the full 150 is next). --- ## The thesis: knowledge should be compiled, not retrieved We have a glut of human knowledge locked in Markdown — blog posts, notes, specs, docs. It's *source code* for understanding, but it's un-queryable, un-composable, and un-deployable as prose. RAG answers questions reactively. Agent frameworks hold state in a chat. Neither *persists* meaning. The Knowledge Compiler takes a different stance, borrowed from the world of programming languages: **treat knowledge like a compiler treats C.** ``` Markdown ──▶ Markdown IR ──▶ Ontology IR ──▶ Graph IR ──▶ Semantic IR │ ▼ Reasoning IR ──▶ Application IR ──▶ software ``` Each arrow is a small, deterministic, inspectable **pass**. Each output is a formal **Intermediate Representation (IR)** — a validated JSON artifact you can commit, diff, and build on. The final IR generates a runnable Next.js app. No agent loop required at runtime; the intelligence lives in the artifacts. --- ## The six IRs 1. **Markdown IR** (`pass-01-parse`) — structure, headings, sections. Pure Python, no model. Deterministic. 2. **Entity IR** (`pass-02-extract`) — entities with types, confidence, and **source-span provenance** (which doc/section produced each one). 3. **Ontology IR** (`pass-03-ontology`) — typed relationships and a concept hierarchy. 4. **Graph IR** (`pass-04-graph`) — nodes + edges; the traversable knowledge graph. 5. **Semantic IR** (`pass-05-embeddings` + `pass-06-clusters` + `pass-07-summaries`) — vector embeddings, thematic clusters, and human-readable summaries per cluster. 6. **Reasoning IR** (`pass-08-reasoning`) — observations, hypotheses, contradictions, and *open questions*, each with provenance back to source. 7. **Application IR** (`pass-09-specifications`) — a specification of the app to generate (routes, data, layout). 8. **Software** (`pass-10-software`) — the actual Next.js app. (Yes, that's "six plus two" — the semantic and reasoning layers each span multiple passes. The point is the layering, not the count.) Every artifact is scored on **nine dimensions** — completeness, correctness, coverage, consistency, hallucination, traceability, provenance, confidence, reproducibility — and the scorecard ships inside the app's `/evaluation` page. Quality is measurable, not a feeling. --- ## Local-first by construction The model-required passes run against **your own** OpenAI-compatible inference server — llama.cpp, Ollama, vLLM, anything on a port you control. No cloud API, no API key, no data leaving your machine. For this project the stack was: - **llama.cpp** (`llama-server`) on `:8080` running **Ornith 1.0 35B** (Q4_K_M) for the reasoning/extraction passes. - **Ollama** on `:11434` running `nomic-embed-text` for embeddings. One detail worth calling out: Ornith is a *reasoning* model. It streams a thinking trace (`reasoning_content`) and only then emits the answer. That means passes need a high `max_tokens` budget (I set 16384) or the model gets cut off mid-thought and returns empty content. The embeddings pass transparently falls back to Ollama when the chat server doesn't expose `/v1/embeddings` — so the whole thing stays local. --- ## The generated app: not stubs The deployment target was a demo that *actually demonstrates the heuristics*, not placeholder pages. `pass-10-software` emits a polished, dark "compiled knowledge" frontend: - **Overview** — live stat cards (entities, graph nodes/edges, observations, themes). - **Entities** — a filterable explorer (by type) with **provenance**: click an entity and see the exact doc/section that produced it. - **Knowledge Graph** — an interactive SVG graph with hover highlighting. - **Reasoning** — observations, hypotheses, contradictions, and open questions, each with provenance trails back to the source posts. - **Themes** — semantic clusters with summaries. - **Evaluation** — the 9-dimension scorecard per artifact. Styled with **Tailwind + shadcn-style primitives + framer-motion** transitions. All routes read the compiled IRs from `/api/*` serverless functions backed by the static `data/*.json` — no database, no external services. --- ## What it actually took to deploy (the real story) The interesting part for anyone building compiler-to-app tooling: the hard problems weren't the ML, they were the **packaging**. **1. Vercel's Next.js build ignores `@/` tsconfig path aliases.** The generated app originally used `@/components/...`. It built fine locally, then failed on Vercel with "Cannot find module '@/components/...'". Fix: emit **relative imports** (`../../components/X`) computed from each route's depth. **2. Production `npm install` skips `devDependencies`.** The first remote build failed with "typescript not installed." Vercel installs `devDependencies` only for the build step in some configs, but the type-check needs them at build time. Fix: ship `typescript`, `@types/*`, and `tailwindcss` in `dependencies`, plus `typescript: { ignoreBuildErrors: true }` as a safety net. **3. The layout never imported `globals.css`.** The first generated app rendered as bare unstyled HTML because `app/layout.tsx` forgot to import the stylesheet. A one-line fix, but it's the kind of thing that makes a demo look broken even when the data is right. **4. A pass crashed the whole pipeline.** `pass-05-embeddings` had its own `argparse` that rejected the `--max-tokens` flag the orchestrator forwards to every model pass. That single pass failing killed passes 6–10 (they depend on its output). Fix: switch the pass to the shared `parse_port_model` helper the other passes already use. Each of these is a "compile-once, deploy-everywhere" lesson: the artifact that works on your laptop is not the artifact that builds in CI. The compiler has to emit *CI-correct* code, not *laptop-correct* code. --- ## Why a 12-post subset (and the full 150 next) The corpus is 151 posts. A full run on CPU with a 35B model is slow — on the order of an hour-plus for the whole set, because each model pass does one or more 35B calls. The demo currently compiles a **12-post subset** so the iteration loop is fast and the UI is proven end-to-end against real content (16 entities, 3 themes, reasoning observations/hypotheses/questions, all from actual posts). The full 150-post run is the natural next step: it yields a far denser graph (the subset produces a sparse 5-node graph; the full corpus will be substantially richer) and is the real test of whether the pipeline scales. It's a matter of letting the CPU grind, not of any code change — the pipeline is corpus-size-agnostic by design. --- ## Why this matters The throughline is the one this blog keeps returning to: **intelligence is not the model. Intelligence is the accumulated, inspectable decisions that shape what the model produces.** The Knowledge Compiler makes those decisions *artifacts* — you can read the entities, trace the reasoning, audit the evaluation. The model is a subroutine; the knowledge graph is the product. And because every artifact is a plain file, the whole thing is **version controlled and reproducible**. Re-run the compiler, diff the IRs, see exactly what changed in your understanding of your own writing. That's the payoff of treating knowledge like source code. --- ## Try it - **Live demo:** https://knowledge-compiler-blog-demo.vercel.app - **Source:** `kliewerdaniel/knowledge-compiler-sdk` Point it at a folder of your own Markdown, run a local inference server, and watch it compile: ```bash pip install pyyaml jsonschema python -m compiler.run --source your-notes --build build \ --local --port 8080 --model your-model \ --embed-model nomic-embed-text:latest --max-tokens 16384 cd build/knowledge-app && npm install && npm run dev ``` The compiler is MIT-licensed. Build the future of autonomous knowledge compilation in the open. --- *— Daniel*

I actually wouldn't lead with the implementation. The implementation is impressive, but it's not the thing that will spread on LinkedIn among AI engineers. Most people don't yet have the mental model for why they should care about ten compiler passes or generated Next.js apps. Instead, I'd write an article that makes one architectural claim: > **We've spent two years optimizing inference. We may need to spend the next two years optimizing compilation.** Then use your repository as evidence rather than the subject. The article I'd write is something like this: --- For the past two years, most AI systems have followed the same architecture. ``` Documents ↓ Chunk ↓ Embed ↓ Retrieve ↓ LLM ``` Every question starts from raw text. Every answer reconstructs meaning again. Every inference repeats work that has already been done. As our knowledge bases grow, we respond by adding better embedding models, larger context windows, rerankers, graph retrieval, agent loops, and increasingly sophisticated orchestration. But what if we're optimizing the wrong part of the pipeline? Compilers solved a similar problem decades ago. A compiler doesn't interpret source code every time a program runs. It parses once. Builds intermediate representations. Runs optimization passes. Validates the program. Produces an artifact that executes efficiently. The expensive reasoning happens before runtime. I think knowledge systems are heading in the same direction. Instead of treating Markdown, documentation, research papers, or specifications as something to retrieve from repeatedly, we can treat them as source code. Compile them once into semantic artifacts. Not just embeddings. Not just knowledge graphs. Intermediate representations that can be inspected, validated, version controlled, optimized, and incrementally rebuilt. The runtime becomes dramatically simpler because much of the intelligence already exists inside the compiled artifacts. This isn't an argument against RAG. It's an argument for moving expensive semantic work from query time to compile time. Instead of asking: > "How do we retrieve the right chunk?" we begin asking: > "What semantic optimizations should happen before deployment?" That opens the door to compiler concepts that AI systems rarely discuss today: - entity canonicalization - contradiction detection - ontology alignment - provenance propagation - graph optimization - dead knowledge elimination - dependency analysis - incremental recompilation - semantic validation These are compiler optimization passes—just operating on knowledge instead of machine code. I've been exploring this architecture in an open-source project called the **Knowledge Compiler SDK**. Rather than generating answers directly from documents, it compiles Markdown through a series of intermediate representations into semantic artifacts and ultimately into an interactive knowledge application. The repository isn't meant to be "the" solution. It's a prototype for a different way of thinking about AI systems. I'm increasingly convinced that the future architecture isn't simply: > Better model → Better application but instead: > Better compiler → Better knowledge artifacts → Simpler runtime → Better application I'd love to hear from people working in compiler engineering, knowledge graphs, Semantic Web, GraphRAG, retrieval systems, and AI infrastructure. **What should the LLVM IR of knowledge look like?** GitHub: https://github.com/kliewerdaniel/knowledge-compiler-sdk --- I think this is stronger than the draft because it starts with a question that applies to everyone's work, not just your project. The repository becomes evidence for a broader architectural idea rather than the centerpiece. That invites discussion instead of feeling like a project announcement. One other suggestion: I would avoid claiming that this architecture will replace RAG or that it is entirely novel. There is prior work in semantic representations, GraphRAG, and knowledge compilation. The distinctive contribution you can argue for is the synthesis: treating the end-to-end knowledge ingestion pipeline explicitly as a compiler with typed intermediate representations, optimization passes, validation, and reproducible artifacts. That's a claim that's both more precise and easier for experienced engineers to engage with constructively.