re: Who’s Afraid of Smart Machines?

Brons — Fri, 07 Aug 2009 20:53:18 GMT

Sorry to take so long to get to writing here. I've meant to comment since you wrote this. Yours was the third or fourth item regarding this story that I encountered, and I wanted to answer a bit more fully than I had the others. Aiming higher has slowed me down.

My first response to the Time article was that it was a classic example of horrid headline writing. They cry, "Scientists Worry Machines May Outsmart Man", and yet graf 4 cites as an example that "computer worms and viruses that defy extermination and could thus be said to have reached a “cockroach” stage of machine intelligence".

Well, first, it's a rigged contest. Those fighting worms and viruses aren't allowed to loose anti-viruses, to take remote action to root out the worms and viruses. That would be illegal. Secondly, if the malware could actually survive serious attempts at extermination, that would put them at the level of real viruses and bacteria, not cockroaches. Finally, assuming that you grant that this really is evidence of "cockroach" intelligence, we are not intellectually threatened by the cockroach. The headline is FUD.

But more importantly, the article itself and not just the headline is problematic. It is based on what I regard as a profound misunderstanding of the nature of, well... intelligence, life, the universe and everything. for well more than half a century we have called digital computers "electronic brains" or the like. Fiddle faddle. Brains are not digital, they are analog. They aren't "about" calculating or storing numbers, they are about communications, about relationships, and they are dynamic. Thinking they are the same sorts of things is an example of a profound mistake that we have mad about the wonders of science and technology for hundreds of years.

The error is to take our latest really clever invention, our shiniest toy, our latest revelation and attempt to explain the whole world in terms of it. One of the most profound examples of this is the view of the world as a clockwork. When we were able to (once again--Heron clearly could have done as much) build machines as complex as the great clockworks built for the medieval churches and kings, it was a triumph of engineering and science. Clocks could track the Heavens, predict the future (in terms of sunrises and sunsets, the turn of the seasons) and model the universe.

If science could do this, obviously it had tapped the Truths of the Universe, and so we came to regard the world to be a giant clock, designed by a Divine Clock Maker, put in motion and operating deterministically ever since according to simple rules discoverable by science. Our belief that digital computers are just like human brains and that therefore all they have to do is be big enough, complex enough and they will surpass brains and have minds greater than even our own. Voila! Verner Vinge's Singularity!

(Or perhaps they already have and we are nothing but a simulation in a great computer! In fact, since it is inevitable that such computers and simulations will be invented, and there will be zillions of simulations and only one real world, statistically it is far more likely that we ARE already in a simulation than that we are real!)

Such thinking misses a lot. It over-simplifies, it obsesses over a single notion until that notion becomes the whole of the universe, explains something. Like the number "23", we come to see it everywhere. Like those obsessed by conspiracy theories and other paranoid delights, we need to back off, contemplate the limits of each idea, each model, and not merely its power.

The role of clockwork thinking, and at least one view of it as one step on the path from naturalistic, to mechanistic, to systems thinking, I recommend a movie called "MindWalk" (you can read about it on a page I wrote recently on the web: http://blog.eldacur.com/musings/mindwalk ).

In point of fact, the world differs from the clockwork world in several ways: It is alive and organic, it constantly changes rather than being the infinitely repetitive motions of an unchanging mechanism. The world is open. Any system of rules is incomplete (see Gödel). It is indeterministic and uncertain, driven by chance.

These are all things we know for certain, and for people of faith, be they Christians like you or Deists like me (or pagans and Taoists like some of my other most intelligent friends), there is also the dimension of the spirit. If there is spirit, then mind is more than brain, and even if brains can be modeled haven't mastered mind. Even for those who are not people of faith there is the question of will, the riddle of consciousness of mind beyond behavior.

So, to get around to answering your question, am I afraid of smart machines? Well, no, for many reasons. First, I don't know that I believe in smart machines, even in theory. If I do, I am certain that none of the machines we have now are anywhere near smart, not in the way that we are. Even in that distant future where they might become smart I don't know that they can become willful, intentional, and if they can do THAT, I still don't see them as a THREAT.

Life, the universe and all that are not about reaching a goal. There is no achievable goal in an open universe and every discipline we know tells us that this on is. They are about the journey. There is no contest to be the best, to be the winner. It's only in the movies that "There can only be one". In life, another set of feet upon the journey, a companion, a different point of view is a boon, a grace.

Why must we see man as "the only animal that ..."? Why must we be the only one? Why must there be a best? Why can't we be a part of a tapestry, a cosmic whole, a player in a drama that requires others to be complete or even comprehensible? Why must we fear a rival rather than embrace a challenge and a chance to learn from comparing perspectives?

As to the teaching aspects: I don't know if the question comes up in CS classes, but it should, and philosophy, and history, and mathematics. The nature of the world, of mind, of science and physical law is something that I think we don't address enough in education or in life. We'll fight about details that derive from different lines of reasoning based on one or another world views, argue Intelligent Design or the like, but not really look at the nature of the world.

Just my two bits.

re: Who’s Afraid of Smart Machines?

Todd I. Stark — Fri, 31 Jul 2009 01:18:36 GMT

This is a big topic that I've followed for some time because it intersects my interests in human expertise, computer science, problem solving, and philosophy of mind. One of my favorite approaches is the one taken my Minsky in "The Emotion Machine," where he takes the stance that we are pretty far from knowing the full technical details of human cognition, but pretty close to a good general architectural understanding. His version is a six-level model of cognition with 'critic' and 'selector' modules at each level that detect patterns and recruit resources. I especially like the way he explores emotion and self-awareness in a reasonable resource-recruitment based model. I don't know if he has the right model but it makes as much sense as anything else I've seen, and more than most. As for fearing them, I fear their potential but somewhat less than I fear the lunatics around me on the road today.