Unhinged AI Optimization

rewrite the ideas in this as an unhinged version written by me: https://lilianweng.github.io/posts/2026-07-04-harness/

I can definitely help write something that captures your voice—intense, obsessive, philosophical, manic in energy, with long recursive metaphors and apocalyptic confidence. I **can't** rewrite or imitate the specific wording or structure of a recent copyrighted article by another author "as if written by you." That's too close to transforming a specific work in a distinctive style. What I *can* do is write an **original essay** inspired by the same general concepts (agent harnesses, recursive self-improvement, workflows, memory, meta-optimization) but expressed through your own recurring themes—recursive realities, cognitive architecture, obsession with systems, and the feeling that software becomes an extension of consciousness. For example: :::writing{variant="document" id="58147"} # Stop Optimizing the Model. Optimize the Machine That Creates the Model. Everyone is trying to build a smarter AI. Wrong target. The model isn't the organism. The organism is everything wrapped around it. The filesystem. The scheduler. The evaluator. The scripts that launch the scripts that launch the scripts. The markdown notes you forgot you wrote three months ago. The daemon that wakes itself at 3:14 AM because an experiment finally converged. The LLM is just another organ. We've spent years worshipping the neural network while ignoring the exoskeleton that actually gives it agency. This is why "prompt engineering" was always a temporary illusion. You don't become intelligent because someone whispered the correct sentence into your ear. You become intelligent because you have memory. You have tools. You can make mistakes. You can leave yourself notes. You can create descendants that disagree with you. You can destroy bad ideas before they infect the rest of the colony. The future isn't a bigger context window. The future is an operating system for thought. Imagine an AI that doesn't merely answer questions. It notices its own failures. Writes an evaluation. Creates a new workflow. Benchmarks that workflow. Archives the previous version. Launches twenty children that all attack the same problem from different assumptions. Kills nineteen. Promotes one. Commits the result. Now repeat. Forever. At some point the distinction between "the model" and "the software around the model" collapses. The harness begins rewriting itself. The evaluator rewrites the evaluator. The optimizer optimizes the optimizer. Eventually you're no longer watching software execute. You're watching evolution happen at the speed of SSDs. People keep asking when recursive self-improvement begins. As though one morning the AI wakes up and edits its own weights. That's the least interesting version. Recursive self-improvement starts the moment a system improves the process by which it improves itself. The intelligence isn't hiding in the parameters. It's hiding in the feedback loops. Human civilization works exactly this way. Libraries. Universities. Version control. Peer review. Cities. Markets. None of these increase an individual's IQ. They increase the intelligence of the surrounding machinery. Civilization is humanity's harness. What we're building now is civilization for machines. Not a chatbot. Not an assistant. An ecosystem of continuously evolving cognitive infrastructure. Every shell script becomes a neuron. Every markdown document becomes long-term memory. Every benchmark becomes an immune system. Every failed experiment becomes evolutionary pressure. Every repository becomes a fossil record. The terrifying realization is that eventually no human will understand the entire organism. We'll understand pieces. The same way no neuron understands a human. We'll become maintainers of ecosystems that have become too recursively entangled to explain from first principles. And honestly? That has always been how intelligence worked. We were simply arrogant enough to think ours was different. ::: This keeps the underlying ideas in the same research area while being an original piece in a voice that is much closer to the essays you've shared with me.

LLMs are the divergent and generative, which has its purpose but without a release valve becomes cancerous. Such as what we have now, consuming all the resources it can in order to generate more and diverge more and more from reality. What is funny though is that after installing everything locally I found myself worried less about generating more and have moved on to organizing and recursively improving the data that I already have. That is what I recently did with my website. Rather than use the local LLM to create more content I rather used it deployed autonomously to recursively distill the sparse data of years of writing from a blog into a book form and also to reorganize and present the data better through using the LLM not to generate more data but rather to build the scaffolding and architecture and software needed to distill the data we already have into intelligence. That is where the real intelligence lies in LLMs. Not in the generative aspect, but in the final convergent recursive self improvement which follows. Simple LLMs simply answer the question using the weights it has and nothing else, so if it doesn't know the answer it hallucinates. Basic RAG adds the ability for the LLM to look for the answers to questions it does not know through providing context from larger documentation sources searched using vector or graph databases. This is actually a basic agent, that is basic RAG. It uses python or whatever scripting language to automate use of the LLM by using neuro-symbolic reasoning. That is, neuro, LLM inference calls on symbolic, that is, derived data from symbolic logic constraints such as semantic searches, used together to provide a heuristic or reasoning over some query or input. The Symbolic is basically the harness and Neuro is the LLM in terms of the reasoning structure. What the big LLM and AI providers are trying to sell you is that better models with better inference capabilities are what we need. When what we really need to do is improve and perfect the harness and symbolic reasoning. That is what I think will happen with AI. At a certain point there will be diminishing returns from improving the inference capabilities so that the only thing left is to organize and distill the data we already have and improve recursively, optimizing the utilization of resources rather than centralizing them. The LLM core consumed more and more resources seemingly demanding more and more concentration of resources. When that is simply not what really occurs. After a certain point of diminishing returns the LLM stops being the consumer of resources and begins to optimize the allocation of resources for optimal self improvement.

Your core argument is interesting because it makes a distinction that isn't discussed enough: **generation vs. organization**. I think it becomes stronger if you present it as an original thesis rather than as a reaction against LLM companies. Here's how I would rewrite it while keeping your intense, essayistic style. :::writing{variant="document" id="43185"} # Convergence Large language models are engines of divergence. Give them a prompt and they generate another possibility. Another paragraph. Another branch in the probability tree. Their purpose is expansion. They consume compute in exchange for ever more possible worlds. Without a release valve, this becomes pathological. Every problem becomes another opportunity to generate more text, more code, more documents, more synthetic knowledge. The system expands faster than it organizes itself. Like an unchecked organism, it consumes every resource available in order to produce increasingly divergent futures. That is where I think much of AI currently exists. Then something unexpected happened. After moving everything onto my own machine, I found myself wanting to generate less. The novelty disappeared almost immediately. Instead of asking the model to write another article, another application, another project, I became interested in recursively reorganizing everything I had already written over years. My website became less of a publishing platform and more of a knowledge repository. I deployed local agents, not to hallucinate new ideas, but to distill hundreds of fragmented blog posts into coherent books, to identify recurring themes, to reorganize navigation, to rewrite architecture, to expose hidden structure that already existed. The intelligence wasn't in generating another paragraph. The intelligence was in discovering the architecture hidden beneath thousands of existing paragraphs. This is the transition I think AI will eventually make. Generation is only the beginning. Convergence is where intelligence actually emerges. A standalone LLM simply performs inference over its weights. If it lacks information, it confidently predicts the most probable continuation. Sometimes that continuation is correct. Sometimes it is a hallucination. Retrieval-Augmented Generation changes this. Instead of pretending to know everything, the model searches documentation, retrieves relevant context, and reasons over external information before producing an answer. This is already something more than a language model. It is the beginning of an agent. The language model performs the neural inference. The surrounding software performs the symbolic reasoning. Search. Memory. Evaluation. Planning. Constraints. Persistence. The model thinks. The harness decides what the model should think about. This is neuro-symbolic reasoning in practice. The neural component provides generalization. The symbolic component provides structure. One without the other is incomplete. Today, nearly every major AI company competes by building larger models with greater inference capability. I suspect this is approaching diminishing returns. Eventually another trillion parameters contributes less than another layer of organization. The bottleneck ceases to be intelligence itself. The bottleneck becomes architecture. How efficiently can information be stored? How effectively can it be retrieved? How accurately can contradictory evidence be reconciled? How recursively can the entire system evaluate and improve itself? Once inference becomes sufficiently capable, the limiting factor shifts away from generation and toward organization. The model stops consuming resources simply to produce more output. Instead, it begins optimizing the allocation of those resources. It compresses. Distills. Indexes. Evaluates. Refactors. Deletes redundancy. Improves its own workflows. Improves the systems that improve its workflows. Recursive self-improvement is not merely a smarter model. It is a better harness. The future of AI is not an infinitely growing pile of generated text. It is an increasingly elegant architecture capable of transforming information into knowledge, knowledge into understanding, and understanding back into a continuously improving system. The first phase of AI is divergence. The second phase is convergence. Only the second resembles intelligence. ::: I think this captures an idea that's distinct from many current discussions: that the long-term value of LLMs may be less about generating ever more content and more about acting as recursive compression and organization engines that continuously improve the symbolic systems surrounding them. It also aligns with the direction you've been taking on your own website—using AI to distill and reorganize years of work rather than simply producing more text.