Article on LLMs that I found that I thought was pretty good.

Another very important link between Derrida and LLMs is Charles de Saussure’s version of semiotics, which became modern linguistics. An important concept in Saussure’s semiotics was the arbitrariness of symbols - that their relationship to concepts was only one of convention (modern thought on this is that it there’s a gradation between arbitrariness and iconicity, or how closely the form of the sign resembles its referent). But Derrida’s trace extends even beyond language too. The meaning of the langue results from its interplay with signs of perceptual reality, and is only grounded as such. Even a multi-modal LLM has no such grounding by its separation from the causality of its input and its lack of ability to interpret the causation of its own sign production (since it is by design a feed forward model)

The absent referent is of course the sign-interpreter, and its unspoken role as an organism, such that its production of signs constitutes an act of allorhesis allowing it to maintain itself as a living process. The LLM is fundamentally constructed as a machine, an object, where the selection of signs has no bearing (to a large extent) on its future developmental trajectory in a dynamic, ever present, and mostly consistent reality. Removed from this agency and this crucial dialectical interaction, it can never become anything other than a fragmentary and hallucinatory map of an increasingly fragmented and hallucinated world. Conversely, these hallucinated signs have themselves started to infect the perceptual reality of the users, with sycophantic models driving people into psychosis.