TUHS->COFF
It's like
Wikipedia.
No, Wikipedia has (at least historically) human editors who
supposedly have some knowledge of reality and history.
An LLM response is going to be a series of tokens predicted based on
probabilities from its training data. ...
Assuming the sources it cites are real works, it seems fine as a
search engine, but the text that it outputs should absolutely not be
thought of as something arrived at by similar means as text produced
by supposedly knowledgeable and well-intentioned humans.
An LLM can weigh sources, but it has to be taught to do that. A human
can weigh sources, but it has to be taught to do that.
Before LLMs, Wikipedia, World Wide Web, ... adages such as "Trust, but
verify," and "Inspect what you expect," were appropriate, and still are.
Dabbling in editing and creating Wikipedia articles has enforced those
notions. A few anecdotes here -- I could cite others.
1. I think my first experience was trying in 2008 to fix what is now at
https://en.wikipedia.org/wiki/Vulcan_Gas_Company_(1967%E2%80%931970),
because the article had so much erroneous content, and because I had
worked/performed at that venue 1969-70. Much of what I did in 2008 was
accepted without anyone else verifying. But others broke things/changed
things, even renamed the original article and replaced it with an
article about a newer club that adopted the name. A few years ago, I
tried to make corrections, citing poster images at
https://concerts.fandom.com/wiki/Vulcan_Gas_Company. Those changes were
vetoed because
fandom.com was considered unreliable. I copied the images
from fandom to
https://technologists.com/VGC/, and then citing those
images was then accepted by the editors involved. (The article has been
changed dramatically, still is seriously deficient, IMO, but I'm not
interested in fixing.)
2. Last year, I created
https://en.wikipedia.org/wiki/Hub_City_Movers,
citing sources I considered reliable. Citations to images at
discogs.com
were vetoed as unreliable, based on analogous bias against that site.
Partly to see what was possible, I engaged with editors, found citations
they found acceptable, and ultimately produced a better article.
3. Later last year, I edited
https://en.wikipedia.org/wiki/IBM_AIX to
fix obviously erroneous discussion of AIX 1/2/3. Even though I used my
own writings as references, the changes were accepted.
I still use the Web, Wikipedia, and even LLMs, but cautiously.
Charlie
--
voice: +1.512.784.7526 e-mail: sauer(a)technologists.com
fax: +1.512.346.5240 Web:
https://technologists.com/sauer/
Facebook/Google/LinkedIn/mas.to: CharlesHSauer