<p>Okay, sorry, I just need a hole to scream into. I’ll be done in a minute.</p> <p><em class="stage-direction">[inhales]</em></p> <p>If you read about the current crop of “artificial intelligence” tools, you’ll eventually come across the word “hallucinate.” It’s used as a shorthand for any instance where the software just, like, makes stuff up: An error, a mistake, a factual misstep<span class="w"> —</span> a lie.</p> <ul> <li><a href="https://old.reddit.com/r/cursor/comments/1jyy5am/psacursornowrestrictsloginstoa_single/">An “AI” support bot informs users about a change to a company’s terms of service<span class="w"> —</span> a change that didn’t actually happen</a>? A hallucination.</li> <li><a href="https://www.theverge.com/news/666443/judge-slams-lawyers-ai-bogus-research">Some law firms used “AI” to file a brief riddled with “false, inaccurate, and misleading” citations</a>? A hallucination.</li> <li><a href="https://apnews.com/article/elon-musk-grok-ai-south-africa-54361d9a993c6d1a3b17c0f8f2a1783c">A chatbot on a rightwing social media website decides to start advancing racist conspiracy theories, even when nobody asked</a>? A hallucination.</li> </ul> <p>I have a semantic quibble I’d like to lodge.</p> <p>Everything<span class="w"> —</span> <em>everything</em><span class="w"> —</span> that comes out of these “AI” platforms is a “hallucination.” Quite simply, these services are slot machines for content. They’re playing probabilities: when you ask a large language model a question, it returns answers aligned with the trends and patterns they’ve analyzed in their training data. <sup class="footnote-ref"><a class="footnote" href="#fn-tuning" id="fnref-fn-tuning" aria-labelledby="title-footnotes">1</a></sup> These platforms do not know when they get things wrong; they certainly do not know when they get things right. Assuming an “artificial intelligence” platform knows the difference between true and false is like assuming a pigeon can play basketball. It just ain’t built for it.</p> <p>I’m far from the first to make this point. But it seems to me that when we use a term put forward by the people subsidizing and selling these so-called tools<span class="w"> —</span> people who would very much like us to believe that these machines can distinguish true from false<span class="w"> —</span> we’re participating in a different kind of hallucination.</p> <p>And a far worse one, at that.</p> <hr /> <div class="footnotes flow"> <h2 class="hed-footnotes has-dash" id="title-footnotes">Footnote</h2> <ol><li id="fn-tuning" class="footnote-item"><p>Well, taking into account any subsequent <a href="https://www.ibm.com/think/topics/fine-tuning">“fine-tuning”</a> of the model that humans may have performed. <a href="#fnref-fn-tuning" class="reversefootnote">↩︎</a></p> </li> </ol> </div>
<hr /> <p>This has been “<a href="https://ethanmarcotte.com/wrote/hallucinating/">Hallucinating.</a>” a post from <a href="https://ethanmarcotte.com/wrote/">Ethan’s journal.</a></p> <p><a href="mailto:listener+rss@ethanmarcotte.com?subject=Reply%20to:%20“Hallucinating.”">Reply via email</a></p>