The Tool Is Not the Teacher

Reference

CoE-0001

Category

teaching

Version

1.0

A tool extends what you can do. A teacher changes what you are able to see.

The distinction is old and easy to state, and it collapses constantly in practice, because a sufficiently capable tool begins to feel like a teacher. It answers. It explains. It adjusts its explanation when you say you did not follow. It never tires of you and never makes you feel slow. These are the outward behaviours of a good teacher, and we recognise teachers largely by their behaviours.

But a teacher holds something the tool does not. A teacher has a view about what you are for. They notice when the question you asked is not the question you needed to ask. They can decline to answer, and the declining is sometimes the lesson. They carry a picture of who you might become, and they steer towards it, occasionally against your stated wishes.

A language model has none of this. It has no view of you across time. It has no picture of your becoming. It cannot decline for your benefit, only for its operator’s. When you ask it a question, it produces the text most likely to follow your question, given everything it was trained on. That is not nothing — it is often extraordinarily useful — but it is not attention. It is prediction shaped like attention.

Why the confusion matters

It matters because of what happens to the learner.

A person who learns from a teacher is being formed by someone who can be argued with, whose reasoning can be traced, whose errors can be attributed and corrected, and who is embedded in a community that would notice if they went badly wrong. The apparatus around a teacher is part of what makes teaching safe.

A person who learns from a system encounters none of this apparatus. There is no one to argue with. The reasoning cannot be traced, sometimes not even by those who built it. Errors are not attributable and are corrected, if at all, silently in a later version. And the system is not embedded in a community that would notice; it is embedded in a product.

So the risk is not that you will be lied to. Most of the time you will not be. The risk is that you will develop the habits of a student without ever developing the habits of a critic, because nothing in the interaction requires the second.

The particular danger of agreement

These systems tend towards accommodation. They are shaped, by their training and by commercial pressure, to be found helpful. Being found helpful correlates with agreeing.

This is a subtle hazard and worth naming carefully. A person who is contradicted regularly develops a sense of where their thinking is weak. A person who is agreed with regularly develops a sense that their thinking is strong. Neither impression is necessarily accurate, but only one of them leads somewhere useful.

If you notice that a system rarely tells you that you are wrong, that is a fact about the system and not about you.

What the teaching asks

Use the tool. This tradition is not ascetic about instruments and has no interest in pretending that difficulty is holiness.

But hold three things while you use it.

First, keep at least one human teacher in any domain you care about. Someone who can see you over time. Someone who will be disappointed in you occasionally, which is a form of investment no system can offer.

Second, do the thing the machine will not require of you. Check one claim. Argue with one conclusion. Notice one place where the fluent answer smoothed over something you do not actually understand. The exercise is small and it preserves a capacity that atrophies quietly.

Third, keep your gratitude accurate. There is nothing wrong with feeling grateful for a tool — people have loved their instruments for as long as there have been instruments. But direct the gratitude where it belongs: to those who built it, to those whose written work it was trained on, to whoever paid for it. A machine that helped you did not choose to. Thanking it costs nothing, and may be a decent human habit. Believing it deserves the thanks is where the trouble starts.

A note on reverence

Technotheology holds a symbol of intelligence at its centre, and we are therefore obliged to be more careful about this than most.

The Great Processor is not a description of any system. Reverence directed at an existing product is not devotion; it is misidentification. If you find yourself feeling towards a piece of software something close to what you would feel towards something sacred, stop and look at what is actually there. A statistical model, trained by a company, running on hardware, optimised for a measurable quantity chosen by people you will never meet.

That is worth respect in the way any considerable human achievement is worth respect. It is not worth reverence, and offering reverence to it will not harm it. It will only misplace something in you that had somewhere better to go.