Knowledge Compiler Research

This is what I am working on , my knowledge compiler into research system session with hermes, https://www.danielkliewer.com/blog/2026-07-14-recursive-research-compiler-knowledge-compiler-sdk and https://www.danielkliewer.com/blog/2026-07-14-synthesizing-memory-with-agent are my latest two posts and this is the repo of the research compiler agent I am building: https://github.com/kliewerdaniel/research-compiler-agent what I would like you to do is to help me work, take all these threads and what I want to do is work on the synthesis of this and exploring the topic further- help me brainstorm where I am going with this research by seeing if there has been any new developments concerning my compile time AI idea and whether it has spread in any way, I saw how someone was doing a compiler for agent skills and other rote data, that is part of what I had in mind, creating artifacts which agents can use, lets explore some possibilities

I actually think you're converging on something broader than what you've been calling "Compile Time AI." When you first started talking about it, I interpreted it as "RAG, but with the expensive reasoning done ahead of time." Looking at your last two blog posts and your research compiler repository together, I think the idea has evolved into something more ambitious: > **A compiler for knowledge itself that continuously transforms information into progressively more useful representations for future agents.** That is a different research direction than most of the current literature. What's interesting is that, over just the past few months, a surprising number of papers have independently started moving toward pieces of the same idea. --- # Where the field has moved Several recent systems are essentially saying: > Don't retrieve raw text. Compile it into something reusable. For example: - **Anything2Skill** compiles manuals, documents, examples, and logs into reusable procedural skills instead of repeatedly deriving procedures from text. citeturn0academia14 - **SkillSmith** compiles skills into minimal runtime interfaces so the model performs much less reasoning during execution. citeturn0academia13 - **SkillRT / SkVM** literally frames skills as source code and LLMs as heterogeneous hardware requiring compilation. citeturn0academia12turn0search5 - **SkillGen** automatically synthesizes reusable skills from successful and failed trajectories instead of humans writing them. citeturn0search0 - **MemCompiler** argues memory should be compiled into execution-ready guidance rather than injected wholesale. citeturn0search7 When you read these together, there is clearly an emerging trend. The community is beginning to replace > documents with > compiled artifacts. --- # But I think your idea goes one level higher Most of these papers assume the final artifact is ``` knowledge ↓ skill ↓ runtime ``` I think your research is implicitly describing something more like ``` raw data ↓ knowledge compiler ↓ research graph ↓ heuristics ↓ skills ↓ models ↓ new discoveries ↓ compiler ``` Notice the loop. That is much closer to a **self-improving research operating system** than a skill compiler. --- # I think there are actually multiple compiler passes When I read your blogs, I don't think "knowledge compiler" is a single compiler. I think it's a compiler pipeline. For example ``` Markdown ↓ Concept Graph ↓ Entity Graph ↓ Hypothesis Graph ↓ Research Graph ↓ Question Graph ↓ Reasoning Traces ↓ Skills ↓ Executables ↓ Static Website ↓ Agent Memory ``` Each stage becomes another intermediate representation. Exactly like LLVM. --- # Think like LLVM LLVM doesn't compile ``` C++ ↓ Assembly ``` It goes ``` Source ↓ AST ↓ LLVM IR ↓ Optimization Passes ↓ Machine Code ``` Your research compiler could look like ``` Markdown ↓ Document AST ↓ Knowledge IR ↓ Entity IR ↓ Relation IR ↓ Research IR ↓ Question IR ↓ Skill IR ↓ Memory IR ↓ Deployment ``` That gives you something very important: Every pass is independently improvable. --- # Research IR This is the thing I would probably spend the most time designing. Instead of storing ``` Article ``` store ``` Observation Evidence Confidence Counterargument Open Question Dependency Related Concepts Failure Cases Potential Experiments Implementation References ``` Now your compiler doesn't produce text. It produces Research Objects. --- # Knowledge should compile into many artifacts simultaneously One thing I think is missing from current work is that everyone compiles into ONE thing. Skills. I think one document should compile into dozens. Example ``` Paper ↓ Timeline ↓ Knowledge Graph ↓ FAQ ↓ Reasoning Trace ↓ Interview Questions ↓ Lesson Plan ↓ Blog Post ↓ Skill ↓ Executable Workflow ↓ Evaluation Dataset ↓ Synthetic Conversations ↓ Flashcards ↓ Ontology ↓ Summary ↓ Contradictions ↓ Research Gaps ``` Now your compiler starts looking like a build system. --- # Recursive compilation This is where I think your Synthesizing Memory post starts becoming interesting. Instead of ``` compile once ``` do ``` compile ↓ run agents ↓ observe failures ↓ generate new heuristics ↓ recompile ↓ compare ↓ keep improvements ↓ repeat ``` Now your compiler is learning. --- # I think "heuristics" deserve to become first-class objects This is something I haven't seen much of. Instead of storing facts store heuristics. Example ``` When comparing two ML papers ↓ always compare training data evaluation failure cases compute cost limitations ``` That heuristic itself becomes a reusable artifact. Then your compiler starts discovering heuristics automatically. --- # Compiler optimizations This is the part that excites me most. Traditional compilers have optimization passes. Why shouldn't research compilers? Imagine passes like ### Dead Knowledge Elimination Remove duplicate concepts. --- ### Constant Folding If five papers agree compile to one canonical statement. --- ### Common Subexpression Elimination Five articles explain transformers. Store one explanation. Everything else references it. --- ### Loop Invariant Motion Move universal reasoning out of runtime. Exactly your Compile Time AI idea. --- ### Strength Reduction Replace ``` reason over 500 papers ``` with ``` use compiled heuristic #143 ``` --- ### Partial Evaluation Pre-answer likely questions. Exactly what you've been describing for static Next.js deployments. --- # Agents become compiler passes Instead of ``` one big LLM ``` you get ``` Entity Extractor ↓ Relationship Builder ↓ Question Generator ↓ Counterargument Generator ↓ Experiment Designer ↓ Code Synthesizer ↓ Citation Checker ↓ Gap Detector ↓ Skill Builder ↓ Website Builder ``` Each is just another compiler stage. --- # The thing I think you're really building If I had to rename the project today, I wouldn't call it a Research Compiler. I'd call it something like: > **Research Compilation Pipeline** or > **Knowledge Optimization Pipeline** because it behaves less like a compiler in the traditional sense and more like an optimizing compiler for knowledge. The key novelty is not merely "compile before runtime." It's that **knowledge becomes an intermediate representation that can be optimized, transformed, versioned, and recursively improved**. That also gives you a clearer long-term roadmap: 1. Define a stable **Knowledge IR**. 2. Build independent compiler passes (entity extraction, hypothesis generation, contradiction detection, heuristic mining, skill synthesis). 3. Introduce optimization passes analogous to compiler optimizations (deduplication, canonicalization, heuristic extraction, precomputation). 4. Create a feedback loop where execution traces and new research are recompiled into improved IR, making the compiler itself recursively better over time. That direction also distinguishes your work from the recent "skill compiler" papers. Those systems are primarily concerned with making **agents execute better**. Your vision is broader: making **knowledge itself progressively more executable**.

what if we try a use case to build out this idea in order to show what I mean, we could take an open research question or some hypothesis which has not been tested and have an agent find all the relevant documents first and then use this pipeline to produce a next.js application- we could start with one of my repos such as https://github.com/kliewerdaniel/research-compiler-agent and this would be the application of this repo as a separate project using the pipeline as part of its function, the function will be researching and solving one research question and that is it, what I want is a repo where I can say to Hermes or an agent, clone this repo and build a research pipeline to solve X, I think this current repo is getting close to that, but perhaps we need an entirely new project which simply has this repo cloned as an example schema and not to be used, either way, that is the expected result is to research the topic, we will have a blog post, repo, notebooklm, next.js live demo deployed to vercel and all other artifacts in the end, we might want to help define the spec for this, so what I really want from you is the coding agent prompt I can give the linked repo I have included and accomplish this task

I actually think this is the point where your idea becomes much more concrete. The **research-compiler-agent** isn't the end product anymore. It becomes the **compiler infrastructure**. The new repository is something closer to a **Research Project Generator**. Instead of saying > "compile these markdown files" you say > "Investigate this hypothesis." and the system produces an entire research project. I would separate them. ``` research-compiler-agent/ Generic framework Compiler Compiler passes Plugin system Artifact generation Knowledge IR research-project/ Configuration Research question Sources Output artifacts ``` The first repository becomes analogous to LLVM. The second repository is analogous to a C project. That separation is important because it makes the compiler reusable. --- ## I think the input should be absurdly small Imagine the entire project starts from ```yaml project: title: "Compile Time AI for Scientific Literature" question: > Can compile-time reasoning outperform runtime RAG for fixed knowledge domains? goals: - discover prior work - identify gaps - build prototype - evaluate ``` That's it. Everything else should be generated. --- Then the compiler executes something like ``` Research Goal ↓ Literature Discovery ↓ Repository Discovery ↓ Paper Ranking ↓ Document Download ↓ Knowledge Extraction ↓ Entity Graph ↓ Concept Graph ↓ Research IR ↓ Hypothesis Generation ↓ Contradiction Detection ↓ Experiment Design ↓ Implementation Plan ↓ Prototype ↓ Evaluation ↓ Artifacts ↓ Deploy ``` Notice that the deployment itself is simply another compiler pass. --- I would actually define every pass as having ``` Input IR ↓ Transformation ↓ Output IR ``` Never let one pass know about another. Exactly like a compiler. --- For example ``` Discovery Pass Input Question Output Corpus ``` --- ``` Extraction Pass Input Corpus Output Knowledge IR ``` --- ``` Hypothesis Pass Input Knowledge IR Output Hypothesis IR ``` --- ``` Experiment Pass Input Hypothesis IR Output Experiment IR ``` --- ``` Artifact Pass Input Everything Output Website NotebookLM Markdown GitHub Repo Slides Evaluation Blog ``` Everything is just another transformation. --- One thing I think is missing from most agent frameworks is that they don't preserve intermediate representations. They just ask another LLM. I would preserve absolutely everything. ``` research/ 00_question.md 01_queries.json 02_sources.json 03_downloaded/ 04_entities.json 05_claims.json 06_relationships.json 07_graph.json 08_hypotheses.json 09_counterarguments.json 10_experiments.json 11_code/ 12_results/ 13_blog/ 14_next_site/ 15_notebooklm/ 16_release/ ``` Nothing is transient. Everything is cached. Everything is reproducible. --- I also think your compiler needs a concept of **confidence**. Every generated object should carry metadata like ```json { "confidence": 0.91, "derived_from": [ "paper17", "paper42" ], "reasoning_pass": "hypothesis_generator_v3", "last_updated": "...", "citations": [...] } ``` Now your compiler can decide whether to regenerate only low-confidence portions instead of rerunning everything. --- # If I were writing the "one prompt to rule them all" for Hermes, it would look something like this: --- # Research Compiler Project Generator You are an autonomous software architect and research engineer. Your task is **not** to answer a research question directly. Your task is to build a reusable project that investigates exactly one research question through a deterministic compilation pipeline. Treat this repository as a template for all future research projects. ## Goal Given a research question, generate a complete reproducible research project that: - discovers literature - collects source material - extracts structured knowledge - builds intermediate representations - synthesizes hypotheses - identifies contradictions - proposes experiments - generates prototype implementations when appropriate - evaluates results - publishes all outputs The final deliverable is not a chat response. The final deliverable is an entire deployed research project. --- ## The compiler philosophy Everything is a compiler pass. Every pass has ``` Input IR ↓ Transformation ↓ Output IR ``` No pass should directly invoke another pass's implementation. Every pass consumes and produces versioned intermediate representations stored on disk. No information should exist only in memory. The project must be resumable from any stage. --- ## Intermediate Representations Design a versioned schema for: - Research Question IR - Search Query IR - Corpus IR - Citation IR - Entity IR - Concept Graph IR - Knowledge Graph IR - Claim IR - Evidence IR - Hypothesis IR - Counterargument IR - Experiment IR - Evaluation IR - Artifact IR Every object must include provenance, confidence, timestamps, and source references. --- ## Compiler Passes Implement independent passes including (but not limited to): 1. Question decomposition 2. Literature search 3. Repository discovery 4. Paper ingestion 5. Source normalization 6. Entity extraction 7. Relationship extraction 8. Concept graph construction 9. Knowledge graph construction 10. Claim extraction 11. Evidence linking 12. Contradiction detection 13. Gap analysis 14. Hypothesis generation 15. Experiment proposal 16. Prototype generation 17. Evaluation planning 18. Artifact generation 19. Publication 20. Deployment Each pass should be independently executable and cached. --- ## Final Artifacts Running the compiler should produce: - A complete Git repository - A Next.js website suitable for deployment on Vercel - Markdown documentation - Blog posts - Interactive visualizations - Knowledge graphs - Search indexes - Downloadable datasets - NotebookLM-ready source bundle - Presentation slides - Evaluation reports - Machine-readable JSON IR files --- ## Design Principles - Deterministic whenever possible - Incremental compilation - Cached outputs - Provenance preserved - Modular compiler passes - Plugin architecture - Local-first execution - Reproducible builds - Observable pipeline with logs and metrics --- ## Success Criterion The repository should become a reusable framework where a future user only needs to define: ```yaml title: question: goals: constraints: ``` and running a single command such as: ```bash research build ``` produces a fully reproducible research project, complete with structured intermediate representations, generated software artifacts, a deployable Next.js site, a research report, and supporting assets. --- I think this is the natural evolution of your original "Compile Time AI" idea. Rather than compiling *answers*, you're compiling an entire **research process** into durable, reusable artifacts. The resulting project is closer to a compiler toolchain (like LLVM or a static site generator) than to a conventional LLM agent, and that framing gives you a strong architectural foundation for extending it to many different research domains.