Can an AI guess what it knows?
Last New Year’s Eve I asked ChatGPT a question that has been rattling around my head: based purely on your training data, if you had to guess (without looking anything up), is “Wentworth Valley Campground” notable enough to have a wikipedia page? That is not really a question about a campground. It is a question about whether a model can look inward and estimate the density of its own knowledge. Every hallucinated citation I have ever been handed is a failure of exactly this skill. ...