The Chatbot From 6 Decades Ago

In the 1960s, computer scientist Joseph Weizenbaum created a “chatterbot” called ELIZA. The program was meant to parody the era’s psychotherapists with its pat replies, but instead, it ended up fooling many users into thinking it had real intelligence. Many users of  ELIZA attributed a surprising degree of understanding to even its repetitive diction. Weizenbaum had intended ELIZA to be a parody but it captivated users.

Today’s large language models, pioneered by Google, were popularized by researchers at OpenAI who noticed that the larger they made these models, the better the models scored on performance benchmarks. Ask ChatGPT to give alternative theories to dark matter, or to summarize the movie Bambi, or tell you who you are and you will get an almost instantaneous answer.

When they train these language models they force them to guess. They hide random words from content so that the programs learn to guess which words are likely to fill these gaps. Doing that it learns the complex web of dependencies that drive language. While they are doing that, they really don't care about truthfulness. That is scary.

One paper on all this was titled “ChatGPT is bullshit,” because they were using philosopher Harry Frankfurt’s definition of bullshit as “speech intended to persuade without regard for truth.” 

ELIZA would accept user input and break it into sentences. It scanned each sentence for keywords or phrases. Then, ELIZA used a set of predefined rules (scripts) to match these keywords with "decomposition patterns," which helped it understand the structure of the sentence.

Based on the matched pattern, ELIZA would generate a response using "reassembly patterns." These responses often involved transforming the user's input into a question or statement.

For example, if a user said, "I feel sad," ELIZA might respond with, "Why do you feel sad?" This approach gave the illusion of understanding, even though ELIZA didn't truly comprehend the conversation. It was crude and limited, but people were amused or even fascinated.

I created a version of ELIZA when I was working at NJIT using the most famous ELIZA script called "DOCTOR" that simulated a Rogerian psychotherapist, reflecting the user's statements back to them in a non-directive manner. As of this writing, one version of that old program still exists at   https://web.njit.edu/~ronkowit/eliza.html 

Perceptron

Mark I Perceptron displayed at the Smithsonian museum

Perceptron as supposed to be a machine, but it began as an algorithm and is still used in that context. It is a piece of tech history from the early days of artificial intelligence (AI).

The perceptron algorithm was invented in 1957 at the Cornell Aeronautical Laboratory by Frank Rosenblatt, funded by the United States Office of Naval Research.

Its first implementation was in software for the IBM 704, and it was subsequently implemented in some custom-built hardware known as the "Mark 1 perceptron." It was one of the first artificial neural networks to be produced.

The Mark 1 perceptron was designed for image recognition and used 400 photocells connected to the "neurons." Still to this day, perceptrons are used generically as a basic neural network.

The algorithm's use was not without controversy. At a 1958 US Navy press conference, some of Rosenblatt's comments opened a controversy among the fledgling AI community. The New York Times picked up on this and reported that the perceptron would be "the embryo of an electronic computer that [the Navy] expects will be able to walk, talk, see, write, reproduce itself and be conscious of its existence."

This fear about "machine learning" was, and still is, feared by many people. Giving computers the ability to learn on their own without being explicitly programmed strikes some people as technology gone wild.

AI has met with ethical and technical hurdles. Steven Pinker has said that trying to duplicate the way the human brain works shows that for computers "Hard is easy. Easy is hard." Computers can do very complex calculations. They can calculate the depth of the ocean and project the outcomes of complicated experiments. But they have trouble answering a question such as "Can a shark play baseball?" which even a child could answer.

Deeper Learning

A computing system developed by Google researchers in Great Britain has beaten a top human player at the game of Go. That game is an ancient Eastern contest of strategy and intuition that has far more possible plays and strategies than chess. Chess masters were beaten by computers early on, but Go has been the grail for artificial intelligence (AI) experts for decades.

My friend Steve tried to get me into Go years ago. I tried. I bought a board. But it didn't click for me. First off, I hate board games. I hate most games. I never got into or owned any computer or video gaming systems.

I did have some interest in Scrabble, Trivial Pursuit and Jeopardy-style games, but not enough to sustain hours of play. I liked Othello and Pente which are very simplified variations on Go play. I learned not to play well but that I lack the brain that strategizes. Even when I taught my young sons how to play chess, they were able to beat me very soo. I don't think far enough ahead.

Perhaps, this gaming helped me realize that I am a natural Zen Master - living in the moment. Tomorrow? Far away. Three moves ahead? I don't see it. I still play the Scrabble-silly Words with Friends on my phone with a few people and lose almost all the time. I just put in words I know (none of these oddball words that work but no one has ever seen before)and I'm not clever enough or interested enough to use the triple letter and double word opportunities. I don't look to see what letters remain or calculate the opponent's rack of letters. My one son won't even play against me. Too easy to win to be any fun for him.

But a computer beating a master of this 2,500-year-old game was big news. It's not the end of the world or the rising of the machines quite yet, but it is an important event for AI.

It was researchers at DeepMind (a company Google acquired in 2014) that set up the machine-versus-man contest. Their system, called AlphaGo, went up against Fan Hui, Fan Hui (reigning European champion) and the machine last October went undefeated in five games.

This month, AlphaGo defeated Korean grandmaster Lee Sedol, finishing the best-of-five series with four wins and one loss. One for the humans!

Of course, the research behind this isn't designed to win Go matches. Google, Facebook, Microsoft and the rest of the gang are interested and already using deep learning to identify images, recognize spoken words, and understand natural language.

DeepMind is said to combine deep learning with a technology called reinforcement learning and other methods. They are looking for ways for that autonomous vehicle or robot to learn to perform physical tasks and respond to their environment.

Computers have always been better than humans at sorting through lots of data very quickly. They are not so great at figuring out what to do with it. Deep learning will probably be best used in research as a supplement to human researchers.

Lee Se-dol won that single game by doing something humans have done to computers for years in sci-fi stories from Asimov to Star Trek - confused them.  He said after the win that he "tricked" AlphaGo computer with a series of unorthodox moves.

Perhaps, Steve and others will enjoy reading the four-page, move by move explanation of how the machine was beaten, but the dummy version is that Lee went against his own best instincts as a player to create confusion for the computer.

He "lured" the AlphaGo in an aggressive position and forced it to neglect precision in order to maintain dominance. It made a serious mistake on move 79, and then "confused" (we do like to make the machine sound more human, even with our language) it continued to make errors as it attempted to correct itself.

I almost feel bad for the computer, but must root for mankind.  I've read too many sci-fi stories. I know what is coming.