Is the LLM that has returned to itself really destined? Comments on Yann Lecun’s recent keynote speech on AI Action Summit

Yann Lecun, chief AI scientist at META and one of the pioneers of modern AI, recently believed that autoregressive large language models (LLMSs) are fundamentally flawed. According to him, the probability of producing the correct response is reduced exponentially with each token, making it impractical for long forms, reliable AI interactions.

Although I deeply respect Lecun’s approach to AI development and resonate with many of his insights, I believe this particular claim ignores some key aspects of the functionality of LLM in practice. In this post, I will explain why autoregressive models are inherently free of inherent differences and doom, and how to adopt chains of ideas (COT) and careful inference query (ARQ), a method we developed to achieve high Accurate customer interaction with Parlant – effectively demonstrates this.

What is automation?

At the heart of LLM is a probabilistic model that trains to generate text one token at a time. Given the input context, the model can predict the most likely token, feed it back to the original sequence, and then iteratively repeat the process until the stop condition is met. This allows the model to generate anything from a brief response to the entire article.

For a deeper understanding of automation, check out our recent tech blog post.

Is the generation error exponentialized?

Lecun’s argument can be unraveled as follows:

1. definition c As a collection of all possible lengths n.
2. definition ac As an acceptable subset u = c – a It means unacceptable.
3. let CI[K] Complete length within progress kat k Still acceptable (CI[N] ∈A Maybe it will still apply in the end).
4. Assuming constants e As the probability of error generating the next token so that it pushes CI Enter you.
5. Possibility of retaining remaining tokens CI exist one That’s (1 – e)^(n – k).

This leads to Lekken’s conclusion that for a sufficiently long response, the possibility of maintaining coherence will double the exponentially close to zero, indicating that the LLM of self-callback is inherently flawed.

But here’s the problem: E is not constant.

In short, Lecun’s argument assumes that the possibility of making mistakes in each new token is independent. However, LLM does not work.

For the analogy that the reason why LLM can overcome this problem, imagine you are telling a story: If you make a mistake in one sentence, you can still correct it in the next sentence to keep the narrative relevant. LLM is the same, especially when techniques like Business Chain (COT) prompt them to guide them to better reason by helping them reevaluate their output in the process.

Why does this assumption have a flaw

LLMS Display Self-correcting attributes This prevents them from spiraling to form incoherence.

take Business Chain (COT) Tipswhich encourages the model to generate intermediate reasoning steps. COT allows the model to consider multiple perspectives, thereby improving its ability to converge to acceptable answers. Similarly, Verification Chain (COV) Structured feedback mechanisms such as ARQ guide the model to strengthen effective output and discard the wrong output.

A small mistake in a generation process is not necessarily destined to be the final answer. Symbolically, LLM can carefully check its work, backtracking and correct errors.

Careful inference query (ARQ) is a game-changer

At Parlant, we further adopt this principle in our work Careful inference query (Currently, a research paper describing our results is in progress, but implementation patterns can be explored in our open source code base). ARQ introduces a reasoning blueprint that helps the model maintain consistency by dynamically refocusing key instructions at strategic points during completion, thereby continually preventing LLMS shunts into inconsistencies. Using them, we have been able to maintain a large test suite that exhibits nearly 100% consistency when generating the correct completion for complex tasks.

This technology enables us to achieve greater accuracy within the scope of AI-driven reasoning and following guidance, which is critical to us achieving reliable and customer-facing applications.

The autoregressive model stays here

We think that the auto-return LLM is far from doomed. Although long-term coherence is a challenge, there is an exponential error rate Ignore key mechanisms to mitigate differences– From thoughtful reasoning to structured reasoning such as ARQ.

If you are interested in AI alignment using LLM and improve the accuracy of chat agents, feel free to explore Parlant’s open source work. Let us continue to improve the knowledge of LLM generation and structure.

Disclaimer: The views and opinions expressed in this guest article are the author’s views and do not necessarily reflect the official policies or positions of Marktechpost.