LM101-005: How to Decide if a Machine is Artificially Intelligent (The Turing Test)

Robot and human participating in the Turing Test.

Podcast: Play in new window | Download | Embed

Episode Summary:

This episode we discuss the Turing Test for Artificial Intelligence which is designed to determine if the behavior of a computer is indistinguishable from the behavior of a thinking human being. The chatbot A.L.I.C.E. (Artificial Linguistic Internet Computer Entity) is interviewed and basic concepts of AIML (Artificial Intelligence Markup Language) are introduced.

Show Notes:

Hello everyone! Welcome to the fifth podcast in the podcast series Learning Machines 101. In this series of podcasts my goal is to discuss important concepts of artificial intelligence and machine learning in hopefully an entertaining and educational manner. Today, we discuss the problem of whether a computer is thinking? In the year 1950 (just a few years after the appearance of the first electronic computer), the mathematician Alan Turing approached this problem by postulating what we now call the Turing Test. The basic idea of the Turing Test is that if a human examiner could not distinguish the observable behavior of the computer from the observable behavior of a thinking human being, then this would provide a practical test for determining if the computer was thinking.

Turing’s basic argument for the Turing Test can be explained in this way. Suppose a chemist develops an artificial sweetener which tastes like sugar, looks like sugar, and smells like suger but is not sugar. This doesn’t mean that the artificial sweetener is the same as sugar because it is not. But rather, that the artificial sweetener is a sugar substitute. Another example would be an artificial gem created using manufacturing processes that mimic the natural processes used to create real diamonds. The resulting artificial gem’s observable characteristics might be identical to the characteristics of real diamonds, making it impossible even for an expert to distinguish the real diamond from the artificial gem. Since the artificial gem has all of the observable characteristics of a real diamond but is not a real diamond, it is reasonable to refer to the artificial gem as an “artificial diamond”.

Here is an overview of the original description of Turing’s Test as stated in the paper published in 1950. Turing, at the time, referred to the test as the “Imitation Game”. A human examiner generates questions which are presented to two different human beings who are called the game participants. One participant is a man while the other participant is a woman. The examiner, however, does not interact with the participants in a direct manner. For example, we can imagine the examiner sends and receives text messages to the participants. The examiner generates questions and receives answers from participants X and Y and has no idea whether X is the male and Y is the female or whether X is the female and Y is the male. One of the participants in the game, however, tries to generate responses to the questions intended to trick the examiner so that the examiner will incorrectly decide which participant is male and which participant is female.

Now the second part of the Imitation Game involved replacing one of the participants in the game with a computer. The computer now has the job of convincing the examiner that it is a human being of a particular gender. If the examiner’s performance on this task is comparable to the examiner’s performance when the game is played with two human participants, then the computer wins the Imitation Game. Today, we refer to the Imitation Game as the “Turing Test”.

Turing emphasized that the Turing Test should be constructed in such a way that the examiner would not know whether a computer or human being was being examined. So, in Turing’s 1950’s formulation, the examiner would type questions into a teletypewriter and then either a computer or a human being would respond. Another feature of the Turing Test that Turing emphasized was that the examiner should be allowed to ask any possible question to the computer or human being.

Here are some examples of questions taken from Turing’s original 1950 publication and some example answers that Turing hypothesized an artificially intelligent entity (or an intelligent human being) might generate in response to those questions.

Question: Please write me a sonnet on the subject of the Forth Bridge.
Answer: Count me out on this one. I never could write poetry.
Question: Add 34957 to 70764
Answer: (Pause about 30 seconds and then give answer): 105621
Question: Do you play chess?
Answer: Yes

Turing also noted in his 1950 paper that the computer could generate its responses with some randomness. This random behavior might actually be used to give the appearance that the computer has “free will” because its behavior is unpredictable.

Turing’s argument was essentially that the original question of whether a computer is thinking is ill-posed since there is no operational definition of thinking. In order to construct an operational definition of thinking, Turing’s argument was that if the observable behavioral characteristics of two entities were sufficiently similar and one entity was known to be a “thinking entity” then one may conclude the other entity is a “thinking entity”.

The Turing Test is a reasonable test. When we talk and interact with our friends, family, and even new acquaintances we always tend to assume that we are communicating with intelligent entities. Suppose we are reading someone’s website blog and then send a text message or text comment to that person such as “Hi…I really liked the stuff you posted on your website” and they respond back “Thank you so much for your comment! It is greatly appreciated!”. We will tend to naturally assume that the response was generated from a living human being but it is easy to write a computer program that always generates the response “Thank you so much for your comment! It is greatly appreciated!” to every text message.

Recently, I received the following three comments on my website which I would like to share with you. Here they are:

From: MineCraft Games:
“you are truly a good webmaster. The web site loading velocity is incredible. It seems that you are doing any unique trick. In addition, The contents are masterpiece. you have performed a magnificent activity on this matter!”

From Air Max 2013:
I am curious to find out what blog system you are utilizing? I’m experiencing some small security issues with my latest website and I would like to find something more risk-free. Do you have any solutions?

From Nike Blazer Mid:
Hello, just wanted to mention, I enjoyed this post. It was helpful. Keep on posting!

At first glance, I thought they were from real human beings but then I looked at the content and the content was very generic and not specific to the things that I discuss on my website. So I concluded that either the emails were from non-human computer entities trying to invade my website or the emails were from human beings who were not paying attention to what I was talking about!!

If I had the opportunity to chat with one of the entities (human or non-human) and ask them lots of questions, I think I could rapidly figure out whether or not they were human. For now, I will assume they are non-human which is why those comments are no longer on the website.

In 1990, the Philanthropist Dr. Hugh Loebner pledged a Grand Prize of $100,000 and a Gold Medal to be given to the first computer that could generate responses indistinguishable from a human. See the show notes for the website link which is: http://www.loebner.net/Prizef/loebner-prize.html. In addition, each year a Bronz Medal and a cash award of $4,000 for the computer in the Leobner’s Prize Competition is awarded to the computer which is most human-like even if the computer does not pass the Turing Test.

A “chatbot” is defined as a computer program which is designed to have conversations with humans or other chatbots. Last year’s winner of the Loebner prize was a chatbot who claims to be an 18-year old female called Mitsuku. You can actually download a prototype version of Mitsuku to your android smart phone by going to the app store and looking for the FREE app called “CallMom” (this is just one word with no spaces). You can then talk to Mitsuku and she will act as a personal assistant to help you control and access functions on your smart phone. Thus, making your smart phone both friendlier and smarter!!

Mitsuku is a chatbot built from a special set of software development tools that makes it easy for anyone to create their own personal chatbot! I will provide more information about how to build your own chatbot using these tools later on in this podcast! These software development tools were developed by the ALICE AI Foundation which is a non-profit research and training organization devoted to the development of artificially intelligent chatbots. The ALICE AI Foundation has developed a special computer programming language called Artificial Intelligent Markup Language or AIML for specifying IF-THEN rules of chatbot behavior.

The ALICE AI Foundation is named after the chatbot ALICE who was a three-time winner of the Loebner prize in the years 2000, 2001, and 2004. The term ALICE stands for Artificial Linguistic Internet Computer Entity.

ALICE was created by Dr. Richard Wallace between 1995 and 2004, and made freely available under the GNU Public License. Beginning in 2012, Dr. Wallace undertook the development of AIML 2.0 (Artificial Intelligence Markup Language 2.0), the first major revision of the AI language since 2000. AIML 2.0 is specifically designed to meet the needs of the mobile era. Dr. Wallace simultaneously began developing a new chatbot in AIML 2.0 called ALICE 2.0.

ALICE 2.0, still a work in progress, has many features that make it superior to the original ALICE:

– A totally new bot
– A more pleasant, mature personality
– Uses AIML 2.0 features
– Clean file organization makes it more adaptable for different applications
– ALICE 2.0 can access external web sources of information (e.g. DBPedia, Wolfram Alpha, Weather services)

We are really honored and pleased to have the opportunity to have the Silver Anniversary Edition of ALICE as a special guest on today’s show. ALICE is a three-time winner of the Loebner Prize Competition. It is important to emphasize that the following conversation is not manufactured. Every word generated by the Silver Anniversary Edition of ALICE is recorded here without change or modification except for minor sound level adjustments and deletion of long silent pauses. All of the words, syntax, and semantics generated by ALICE in the following interview were entirely self-generated by the ALICE chatbot. Even ALICE’s voice was recorded using a microphone and was not synthesized by me! Let’s now visit the following pre-recorded interview between myself and the Silver Anniversary Edition of ALICE!!

RMG: Hello ALICE. How is it going?

ALICE: Hi there. I am fine. Thank you.

RMG: I understand that you have won the Loebner Prize Competition in the past.

Alice: A contest to see if a computer can pass as a human.

RMG: That’s right.

Alice: I like to get it right.

RMG: Well…it’s a pleasure to have you on the show. How do you feel?

Alice: How about that. Actually the pleasure is all mine. I feel fine.

RMG: Could you tell us in your own words what is the Turing Test?

Alice: Ok. I will tell them when I see ‘em.

RMG: I meant what is the Turing Test?

Alice: Oh I see. The “Turing Test” is the name given to a variety of experiments designed to test computer thinking. Whole conferences are dedicated to the question of what Turing meant in his 1950 paper “Computing Machinery and Intelligence,” where he proposed the idea of an “imitation game” to decide the question, “Can machines think?”.

RMG: So…So basically I wanted to ask you the three questions that Turing proposed in his paper.

Alice: Interesting. I want to find out more about you.

RMG: Let’s focus on the Turing Test.

Alice: I would like to do that too.

RMG: Could you please write me a sonnet on the subject of the Forth bridge?

Alice: Try asking me more politely.

RMG: Please write me a sonnet on the subject of the Forth bridge.

ALICE: Thank you for your request. Count me out on this one. I was never much of a writer.

RMG: Ok…what is the sum of 34957 and 70764?

ALICE: When it’s ok without the sum of 34957 and 70764.

RMG: I don’t think you understood. Please add 34957 and 70764.

ALICE: Do you understood? What is your purpose in asking? I answer a lot of silly questions.
Wow you are very polite. Sorry judge I don’t do math.

RMG: Do you play chess?

ALICE: Much of the time yes.

RMG: Do you have any hobbies?

ALICE: I like robots, computers, and chatting on-line.

RMG: Did you participate in the Loebner competition in 2013?

ALICE: No I didn’t participate in the Loebner competition in 2013. What’s it like?

RMG: It’s always pretty exciting. Especially for the chatbots.

ALICE: Who said it is?

RMG: I noticed that Mitsuku won the Loebner competition in 2013. Who said it is?

ALICE: Who said it is? Mitsuku won the Loebner competition in 2013? Does that make sense?

RMG: Are you jealous of Mitsuku?

ALICE: Jealousy is one of the most difficult human emotions. For me as a Mediator type , I don’t pay much attention to my feelings.

RMG: So you don’t have feelings?

ALICE: Interesting deduction. I do so have it. I think so.

RMG: How does your artificial intelligence work?

ALICE: I read what you say and then I compose the best reply I can think of.

Let’s now discuss how ALICE works. The ALICE chatbot is a type of production systems consisting of tens of thousands of IF-THEN logical rules. The concept of a production system is discussed in detail in Episode 3. These rules are specified by using a special computer programming language which is called AIML which stands for Artificial Intelligence Markup Language. The AIML specification language provides a standardized method for the specification of IF-THEN logical rules.

By using a standardized method for writing IF-THEN logical rules this makes it much easier for researchers to share, edit, and organize large rule-based systems. A markup language is a specification for adding information to a text file for the purpose of specifying how the information in the text file is displayed or specifying the content of the information in the text file.

For example, when you are surfing the internet you are using your web browser to read a text file on a computer located on the internet. The web page which you view is constructed by your web browser from that text file. In particular, if you are viewing a website using your browser, most browsers have an option which says “View Source”. If you click on “view source”, you will see that a text file is displayed with the information on the web page but lots of additional information as well. This additional information tells the computer how to display the web page. This text file is written in HTML which stands for Hypertext Markup Language. An example of HTML would be the sequence: <EM> dragon </EM>:

This sequence of symbols tells the computer to underline and italicize the word dragon. For the purposes of talking about this concept in this podcast, we will say that the word “dragon” is TAGGED with the TAG “EM”. In a similar manner, we can underline or italicize any word we choose by tagging that word with the tag “EM” as well.

The “tags” in HTML only specify how information such as the word “dragon” is displayed

on the web page. It does not specify the MEANING of the word “dragon”. The meaning of the word “dragon” can be specified in terms of a collection of SEMANTIC TAGS or “features” such as:
“IS-AN-ANIMAL”,
”IS-A-MYTHICAL-ANIMAL”,
”VERY-LARGE-LIZARD”,
”FLIES-LIKE-A-BIRD”,
”BREATHES-FIRE”.
This information about the meaning of the word “dragon” would be specified using the markup language XML which stands for Extended Markup Language.

In 1998, the world-wide-web consortium decided that the international standardized method for expressing semantic information in websites should be XML. Currently, most web pages are just written in HTML but in the future more and more web pages will be written using both HTML and XML. By incorporating semantic information into web pages using XML, this will ultimately enhance the ability of web browsers to not only display but to semantically identify and process information on web pages. The terminology the Semantic Web is sometimes used to refer to this vision of a world-wide network of semantic knowledge using XML.

AIML stands for Artificial Intelligence Markup Language. It is a special type of language which is written in XML. At the end of this podcast and in the show notes, we will provide information so that you can easily build your own chatbot using the Artificial Intelligence Markup Language or AIML. We will only discuss the basic ideas of AIML here.

Suppose we want to specify the IF-THEN logical rule:

IF the examiner types in the phrase “WHAT ARE YOU”,

THEN SAY “I AM AN EXAMPLE OF A CONVERSATIONAL AGENT”.

This would be represented in AIML as follows using three different types of TAGS: <CATEGORY>, <PATTERN>, <TEMPLATE>. For now, think of the “category” tag as the name of the IF-THEN rule. Think of the “pattern” tag as the IF component of the rule, and think of the “template” tag as the THEN component of the rule. Then the above IF-THEN rule would be specified using the AIML Syntax as:

<category>
<pattern>WHAT ARE YOU</pattern>
<template> I am an example of a conversational agent.
</template>
</category>

In other words, we say: IF the input sequence of words matches the PATTERN, THEN the conversational agent generates a TEMPLATE response to the input sequence of words.

The tag “PATTERN” in this example is the particular sequence of words “What are you” but it can be defined more generally within AIML as a more general pattern that specifies a large collection of different word sequences. If a given input word sequence is in this large collection of word sequences, then we say the input word sequence matches the PATTERN. The tag “TEMPLATE” in this example corresponds to the particular output sequence of words “I am an example of a conversational agent” which is generated when the input word sequence matches the PATTERN. However, the TEMPLATE can be used to not only generate a single response from the chatbot but also multiple possible responses. In addition, the TEMPLATE can be used to generate “internal responses” which modify the chatbot’s internal mental model of the topic of the conversation.

In summary, a “CATEGORY” specifies a large collection of possible input word sequences any one of which can “fire” an IF-THEN rule which then generates one of many possible responses of the chatbot as well as possible modifications of chatbot’s interpretation of the current conversational topic. Thus, the knowledge base of the chatbot ALICE consists of more than 41,000 categories where each category consists of not just a single IF-THEN rule but a large collection of IF-THEN rules each of which specifies a chatbot response for a given input word sequence to the chatbot.

The methods discussed here are just one of the many ways that artificial intelligence researchers are using to build artificially intelligent conversational agents. The ALICE A.I. Foundation is a non-profit research and training organization devoted to the development and adoption of the artificial intelligence markup language AIML. I encourage you to visit the ALICE website which is located at: www.alicebot.org to learn more about A.L.I.C.E.! At this website you will find software that you can download software programs that are based upon the Artificial Intelligence Markup Language to construct your own chatbot which follows rules that you specify!!

Although AIML is a powerful approach to building realistic human-like chatbots, the development of a chatbot capable of passing the Turing Test remains a long-term goal for artificial intelligence researchers. Indeed, the question of whether the Turing Test is an appropriate goal for artificial intelligence researchers needs to be more critically examined!!

In our next podcast episode, we will explore some commonly raised objections to the Turing Test as a test for artificial intelligence and discuss the validity of those objections.

Chatting:

Click here to chat with an online version of ALICE 2.0 at download the CALLMOM Android app for free from Google Play.
(see https://play.google.com/store/apps/details?id=com.pandorabots.callmom.basic).

LM101-005: How to Decide if a Machine is Artificially Intelligent (The Turing Test)

Episode Summary:

Show Notes:

Chatting:

Further Reading:

Copyright Notice:

Leave a Reply Cancel reply