Why I am bored by “Machine Learning” (and statistics).

It is easy to teach computers facts. We call it “‍uploading.‍” You say “‍remember this‍” and they do, until you tell them to forget or they die.

It is also easy to teach computers skills. We call it “‍programming‍”. Computers never make mistakes, never get bored, and work really fast. As long as you fully understand the skill in question the computer can learn it as fast as you can describe it.

But teaching computers anything else is… well, it might be possible… So far, the other thing we can teach them is the skill of noticing groupings and trends in facts.

Machine Learning is a hot topic right now. The “‍Big Data‍” buzzword suggests that computers ought to “‍learn‍” what is interesting in a set of data with minimal supervision or direction. But learning is a skill you have to teach the computer and therein lies the rub. We don’t know how to teach computers to learn much.

Machine learning essentially solves just one problem, though many variations on it exist. Humans provide the computer with a family of models and a lot of data and the computer finds a model in that family that is the most like the data. For example, the human might say “‍Find the line that best approximates these five points‍”—the line is the family of models and the five points are the data.

What machines don’t do that humans do do is understand anything. There are no “‍Ah-ha‍” moments, no integrating a model into a world view, no world view to integrate anything into. The joy of teaching and of learning, the progress of understanding, is totally missing.

A classic “‍triumph‍” of machine learning is machine translation. You can translate with understanding (text to meaning to text) or by correlation (text directly to text based on a bilingual corpus). We’ve tried to teach computers both styles. When the understanding-based version works it works very well, but we have to understand how to understand text in order to teach the computer that skill and handling every corner case is difficult. The zero-understanding machine-learning version works reasonably well all of the time and with minimal effort on the human’s part. Correlational translation contributes nothing to our understanding of language, but it does get the translation job done.

Machine learning is a hot topic right now. I do work in artificial intelligence, a field whose interesting portion has been almost completely ousted by the machine learning community. I read a lot about machine learning I know enough to know that ML aficionados will baulk at my above description of ML. I am elided details for the sake of clarity. , I am conversant in its details, I understand the difficult problems it contains, I even understand that you can sometimes “‍learn‍” interesting things. But try as I might I still find the whole field void of interest.

theory Much of science can be split into three camps: observational science discovers what is; theoretical science postulates why it is as it is; and experimental science creates observations to affirm or refute theories. Machine learning, along with most of the field of statistics and just about every article that includes the phrase “‍studies show‍”, is almost entirely observational science.

I have two things against observational science. The first is that it bores me. This is purely a personal distaste; I want people to do observational science, I just don’t want to be one of them; but it bears mentioning since it probably biases this entire article. But the other is that it is outstripping its theoretical and experimental peers. Balance breeds understanding. It is nice to know what is, but more valuable if you also know why it is. Machine learning, like with most statistics and “‍studies‍”, doesn’t include a why component.

I’ve written before about understanding; in that article, the “‍very clean universe‍” hypothesis would suggest that machine learning could be used to generate understanding. But even if we do live in such a universe, we are far from learning good theories by observation now. Unless we can and until we do get to that point, I’ll keep working on understanding the universe and leave looking at it in myriad ways to others.