COFF May 2025

coff@tuhs.org

9 participants
3 discussions

Wikipedia anecdotes - LLM generalizations [was On the unreliability of LLM-based search results (was: Listing of early Unix source code from the Computer History Museum)

by Charles H Sauer (he/him)

TUHS->COFF > > It's like Wikipedia. > > No, Wikipedia has (at least historically) human editors who > supposedly have some knowledge of reality and history. > > An LLM response is going to be a series of tokens predicted based on > probabilities from its training data. ... > > Assuming the sources it cites are real works, it seems fine as a > search engine, but the text that it outputs should absolutely not be > thought of as something arrived at by similar means as text produced > by supposedly knowledgeable and well-intentioned humans. > > An LLM can weigh sources, but it has to be taught to do that. A human > can weigh sources, but it has to be taught to do that. Before LLMs, Wikipedia, World Wide Web, ... adages such as "Trust, but verify," and "Inspect what you expect," were appropriate, and still are. Dabbling in editing and creating Wikipedia articles has enforced those notions. A few anecdotes here -- I could cite others. 1. I think my first experience was trying in 2008 to fix what is now at https://en.wikipedia.org/wiki/Vulcan_Gas_Company_(1967%E2%80%931970), because the article had so much erroneous content, and because I had worked/performed at that venue 1969-70. Much of what I did in 2008 was accepted without anyone else verifying. But others broke things/changed things, even renamed the original article and replaced it with an article about a newer club that adopted the name. A few years ago, I tried to make corrections, citing poster images at https://concerts.fandom.com/wiki/Vulcan_Gas_Company. Those changes were vetoed because fandom.com was considered unreliable. I copied the images from fandom to https://technologists.com/VGC/, and then citing those images was then accepted by the editors involved. (The article has been changed dramatically, still is seriously deficient, IMO, but I'm not interested in fixing.) 2. Last year, I created https://en.wikipedia.org/wiki/Hub_City_Movers, citing sources I considered reliable. Citations to images at discogs.com were vetoed as unreliable, based on analogous bias against that site. Partly to see what was possible, I engaged with editors, found citations they found acceptable, and ultimately produced a better article. 3. Later last year, I edited https://en.wikipedia.org/wiki/IBM_AIX to fix obviously erroneous discussion of AIX 1/2/3. Even though I used my own writings as references, the changes were accepted. I still use the Web, Wikipedia, and even LLMs, but cautiously. Charlie -- voice: +1.512.784.7526 e-mail: sauer(a)technologists.com fax: +1.512.346.5240 Web: https://technologists.com/sauer/ Facebook/Google/LinkedIn/mas.to: CharlesHSauer

1 week

Re: [TUHS] Re: Wikipedia anecdotes - LLM generalizations [was On the unreliability of LLM-based search results (was: Listing of early Unix source code from the Computer History Museum)

by jnc＠mercury.lcs.mit.edu

> From: Paul Winalski > Wikipedia quite rightly wants citations for stated facts and this > practice goes a long way to prevent inaccuracies. But I recall one > instance where it actually caused the establishment of a factual error. Not rare; this pattern was the subject of an XKCD strip, "Citogenesis": https://xkcd.com/978/ https://www.explainxkcd.com/wiki/index.php/978:_Citogenesis The latter URL has a very similar story to the one you gave. Noel

2 weeks, 6 days

Re: On the unreliability of LLM-based search results

by Alexis

[Redirected from the TUHS list] George Michaelson <ggm(a)algebras.org> writes: >> We're way off topic. Warren should send a kill. > > That said: please don't repeat the "hallucinate" label. It's > self > -aggrandisement. Its deliberate, to foster belief "it's like > thinking" > > It's not a hallucination, it's bad model data and bad constraint > programming. They're not thinking, or dreaming, or demanding not > to be > turned off, or threatening or bullying: They're not Markov > chains > either but they're a damn sight closer to a machine than a mind. Point taken, although i think trying to change that language might be tilting at windwills at this point. Still, i'll try to use alternate phrasing, e.g. "LLMs are known to output nonexistent sources". (Alternative phrasings welcome.) i'd also be interested in an analysis of this: > An artificial intelligence model created by the owner of > ChatGPT has > been caught disobeying human instructions and refusing to shut > itself off, researchers claim. -- https://www.stuff.co.nz/world-news/360701275/openai-software-ignores-explic… Alexis.

3 weeks

2025

2024

2023

2022

2021

2020

2019

2018

COFF May 2025