LLMs and AGI Potential

summarize: Saw a post this morning discussing whether LLMs will get us to AGI. As I started to comment, it got quite long, but I wanted to attempt to weigh-in in a nuanced given my background as neuroscientist and non-tech person, and hopefully solicit feedback from the technical community. Given that a lot of the discussion in here lacks nuance (either LLMs suck or they're going to change the entire economy reach AGI, second coming of Christ, etc.), I would add the following to the discussion. First, we can learn from every fad cycle that, when the hype kicks in, we will definitely be overpromised the capacity to which the world will change, but the world will still change (e.g., internet, social media, etc.). in their current state, LLMs are seemingly the next stage of search engine evolution (certainly a massive step forward in that regard), with a number of added tools that can be applied to increase productivity (e.g., using to code, crunch numbers, etc). They've increased what a single worker can accomplish, and will likely continue to expand their use case. Don't necessarily see the jump to AGI today. However, when we consider the pace at which this technology is evolving, while the technocrats are definitely overpromising in 2025 (maybe even the rest of the decade), ultimately, there is a path. It might require us to gain a better understanding of the nature of our own consciousness, or we may just end up with some GPT 7.0 type thing that approximates human output to such a degree that it's indistinguishable from human intellect. What I can say today, at least based on my own experience using these tools, is that AI-enabled tech is already really effective at working backwards (i.e., synthesizing existing information, performing automated operations, occasionally identifying iterative patterns, etc.), but seems to completely fall apart working forwards (predictive value, synthesizing something definitively novel, etc.) - this is my own assessment and someone can correct me if I'm wrong. Based on both my own background in neuroscience and how human innovation tends to work (itself a mostly iterative process), I actually don't think linking the two is that far off. If you consider the cognition of iterative development as moving slowly up some sort of "staircase of ideas", a lot of "human creativity" is actually just repackaging what already exists and pushing it a little bit further. For example, the Beatles "revolutionized" music in the 60s, yet their style drew clear and heavy influence from 50s artists like Little Richard, who Paul McCartney is on record as having drawn a ton of his own musical style from. In this regard, if novelty is what we would consider the true threshold for AGI, then I don't think we are far off at all. Interested to hear other's thoughts.

The post argues that while LLMs are currently overhyped in terms of their potential to achieve AGI, they are still transformative, particularly as advanced search engines and productivity tools. The author, a neuroscientist, highlights that technological hype cycles often overpromise but still lead to significant change. They note that LLMs today excel at working backwards—synthesizing existing information and identifying patterns—but struggle with true novelty and predictive reasoning. However, given how human creativity itself is largely iterative (e.g., the Beatles drawing from 50s rock), AGI may not be as distant as it seems if iteration and recombination of existing knowledge are sufficient criteria. Ultimately, while AGI likely won't emerge in 2025, continued advancements in AI and a deeper understanding of consciousness may bridge the gap. The author invites feedback from the technical community on these perspectives.