Epistemology? How we know that a machine knows

This is the entry that looks furthest from the subject and is closest to it. Artificial intelligence is, strictly speaking, a knowledge technology: it produces claims about the world from evidence, and we decide when to accept them. Epistemology is the discipline that has been dealing with that decision for twenty-five centuries, and it has not solved the problem. Worth knowing before delegating it.

Notice what just happened in the previous paragraph. I asked you to accept that the topic matters on the strength of nothing but my say-so. If I now ask how you know what you know, several possible answers are “I heard it”, “I trust the source”, “I looked it up”. That is the whole problem.

Observation

We tend to think the best source of knowledge is direct experience, the kind the senses give, and to treat it as the most objective thing we can aspire to. That idea took work to establish: for centuries the opposite view prevailed, that the senses deceive and it is better to reason than to go and look.

The conventional history places the change in the seventeenth century, when observational facts become the basis of science and the authority of Aristotle and the Bible stops being enough. The emblem is Galileo dropping objects from the tower of Pisa. (That experiment probably never happened —the source is a biography written decades later by a disciple— and it still does the pedagogical work, which is itself an interesting epistemological fact about how knowledge circulates.)

This matters for thinking about AI because we face a new form of observation: the kind that starts with the generation of a datum. And there are so many data available that it is easy to come to believe the data are the world.

With sensor-mediated observation the same thing happens as with observation plain, only more so: we see what we see according to what we are looking to see. Observing is not receiving stimuli. Formulating and accepting an observational statement requires a conceptual framework and prior knowledge. The classic example is a child learning to recognise apples: without corrections from someone who already knows what makes something an apple, they will call apples things that are not.

Exactly that happens to a machine learning system, with two differences. The corrections come from a dataset labelled by people whose conceptual framework gets built in without being declared. And there is nobody there to ask why.

Induction

Deriving a general regularity from observational statements is called induction. You observe that metals A, B and C expand when heated, and induce that all metals do.

Inductivism holds that making good inductions is all it takes to have good knowledge. The trouble is that, unlike deduction, induction does not guarantee the truth of the conclusion even when the premises are true. The requirements proposed for a good inductive inference come apart on close inspection:

• Many observations supporting the generalisation. It is not clear how many.
• Great variety of conditions. There is no criterion for deciding which variations are relevant.
• No known exceptions. Most scientific laws have exceptions.

There is no satisfactory characterisation of induction, and it cannot be justified by appeal to deductive logic or to more induction without circularity. There are more sophisticated versions based on probability and statistics, but they inherit the same problems, with the aggravation that these end up concealed among the assumptions made to get the calculation working.

The conclusion is not sceptical: it is that facts and theories support each other. You need the existing theory about what you want to know in order to judge which inductive generalisations are acceptable. That only looks like a problem if you were expecting an ultimate truth.

And it describes with uncomfortable precision what a trained model does: generalise from cases, with no theory of the domain, on sufficiency criteria set by whoever assembled the dataset.

Deduction and falsification

Falsificationism starts from the fact that knowledge changes with time and with available resources, and so puts the problem of progress at the centre: explaining why one theory is better than another without any of them being true in an ultimate sense, because a better one can always come along.

The canonical example is physics from Aristotle to Newton to Einstein. Aristotelian physics explained a great many phenomena and was still falsified in several respects. Newton’s was such a large advance that for two centuries it looked definitive. Relativity turned out better still.

What matters here is the displacement: what is at stake is not truth but the process —formulating hypotheses, testing them, falsifying them. Observation and experience are fundamental, but so are imagination and creativity in producing the hypotheses to be tested, and that remains the open challenge for any artificial system. There is a good deal more imagination in human intelligence than the theories centred on full rationality wanted to admit.

Technological knowledge

Worth dismantling in passing a very entrenched idea: that technology is applied science. The linear model running from scientific discovery to practical application became associated with Vannevar Bush’s 1945 report Science, the endless frontier, and with the science policy it organised for decades.

It was criticised on two counts. First, because it ignores the social, political and economic factors that shape development. Second, because the process is neither linear nor one-directional: technology has a logic of its own, responds to practical needs and market forces, and frequently runs ahead of the scientific explanation of why it works.

Deep learning is a textbook case. It works far better than the available theory can explain, and a good part of the research consists of working out afterwards why it worked.

Social epistemology: whom we trust

Epistemology is the study of how we know what we know. Social epistemology deals with how we know it through others, which is by far the most common way of knowing, science included.

A situation: someone tells you a rumour. Do you believe it? You can weigh how reliable that person is, or you can look for more information. Either move takes a position on the social nature of knowledge.

Defending a belief by what we know of its source’s reliability is a position called reliabilism. Trust works as a shortcut: instead of verifying every detail, you rely on sources that have proved dependable. It is efficient and it is dangerous for the same reason.

Two things complicate the picture:

• The social mechanisms of verification were built when publishing was expensive. Printing a newspaper or running a television channel implied an organisation, and that organisation was what you were assessing when you trusted. Digital technologies took that cost to zero without replacing the mechanism.
• Cognitive biases are shortcuts. If a friend usually gives me accurate information, she has earned my trust. But what makes her reliable? Where does she get what she says? Could she be saying what she reckons and, since nobody challenges her, appear to be right?

With generative AI this gets acute. When we interact with a language model, the output can be as polished as an expert’s, and it comes without any of the signals we use to calibrate trust: no source, no track record, nobody to answer for it. If you are used to checking something with a search, it is worth asking what or whom you are trusting exactly.

And it is not only an individual problem. When generated text starts circulating as a source for other generated text, what degrades is not a particular belief but the system by which a community decides what it takes as established. That is the underlying subject of the epistemic commons, one of the axes running through the blog and much of Philosophy of AI.

That closes the conceptual floor of the chapter. What follows is the practical part —where to learn all this, in what order and by what criteria— and it is worth walking in with the question in hand. None of the courses ahead asks it, and it is the one that makes all the others useful.

Further thinking

Chalmers, A. F. (2013). What is this thing called science? (4th ed.). Hackett.

Cutcliffe, S. H. (2000). Ideas, machines, and values: An introduction to science, technology, and society studies. Rowman & Littlefield.

Lawler, D. (2020). Los estándares como artefactos. Filosofia Unisinos, 21(1), 24-35. https://doi.org/10.4013/fsu.2020.211.03

Parente, D., Berti, A. and Celis Bueno, C. (Eds.). (2022). Glosario de filosofía de la técnica. La Cebra.

This entry revises and updates Epistemology? from v1, available in Spanish.