AI test by Alan Turing
AI Test_Header

The Turing Test: How up-to-date is the Artificial Intelligence Test in times of Chat GPT & Co.

In this first part of our series 'AI Rap with Eva Eggeling', Eva Eggeling, Center Leader for Data Driven Design at Fraunhofer Austria and Head of the Innovation Center for Digitization and Artificial Intelligence KI4Life, dives deep into the topic of the Turing Test.
Table of contents: The Turing Test: How up-to-date is the Artificial Intelligence Test in times of Chat GPT & Co.

Can a computer think like a human or not? In the age of Chat GPT, questions like these are more topical than ever. However, pioneering work in the field of computers was already being carried out in the 1940s and 1950s. machine learning achieved. And even back then, researchers were looking for answers to questions like these.

The British logician, mathematician, cryptanalyst and computer scientist Alan Turingpresented a test in his essay "Computing Machinery and Intelligence" that was intended to serve as a research method in the field of artificial intelligence.

Alan Turing
Alan Turing was a British logician, mathematician, cryptanalyst and computer scientist. Credit: Wikipedia

What is the Turing test?

The Turing test is designed to test the Intelligence of machines demonstrate. The test is passed if the computer can imitate human reactions under certain conditions.

The predecessor of the Turing test is the so-called "Imitation Game". Three human participants sit in three separate rooms, each of which is connected via a screen and keyboard. A male participant sits in one room, a female participant in the second room and a male or female "referee" in the third room.

The female participant now tries to convince the neutral referee that she is the male participant by the way she answers some questions. The referee has the task of finding out who the male participant is.

How does the Alan Turing test work?

The further development of the imitation game for Alan Turing's Turing test now brings an AI into play. A computer terminal is operated by one computer, while humans sit in front of the other two. The referee must question the participants on a specific topic in a given format and context.

After a certain time and a certain number of questions, the referee must decide which computer will be operated by the AI. This procedure is repeated many times. If the referee does not recognize the AI in half or fewer of the test runs, the AI can be classified as intelligent, as it is perceived by the referee as "just as human" as the human respondent.

Criticism of the Turing test

A large Problem with AI tests is certainly the fact that Alan Turin himself did not formulate any requirements for the questioner. This makes the test difficult to interpret. Scientists such as Gary Marcus, Professor of Psychology and Neuroscience at New York University, point out the disadvantages of the test. For example, the test with the chatbots says nothing about the intelligence of the people who also take part in the Turing test. In addition, the AI is not able to answer complex questions and does not act intuitively.

Another point of criticism is that in the past, the type of question had to be restricted in order for a computer to demonstrate human-like intelligence. For a long time, a computer only achieved a high score in the Turing test if the questioner asked yes or no questions and the topics related to a narrow field of knowledge. If open questions were asked, the probability that the AI could deceive the human was lower.

To this day, it is questionable whether the Turing test can provide credible proof of artificial intelligence at all. Does the deceptively genuine imitation of human communication actually indicate an independent artificial intelligence or is it nothing more than a deceptively genuine imitation?

KI-Test_1

Which AI passes the Turing test?

To mark the 60th anniversary of Alan Turing's death in 2014, the University of Reading in London organized a competition in which a Chatbot named Eugene Goostman simulated a 13-year-old boy. In the eyes of some scientists, the chatbot passed the test as it fooled 33 percent of the judges.

Others argue that there were not enough jurors and that the test was invalid because it only lasted five minutes. Another argument is that programs like Eugene Goostman are primarily aimed at misleading the questioners. The AI responds to questions that it does not understand or to which it does not have an answer ready with counter-questions and evasive maneuvers. The more successful this is, the more people believe they are communicating with a human counterpart.

A second example is the program Google Duplexa project that enables users to make reservations via smartphone. In 2018, Google Duplex successfully made an appointment with a hairdresser over the phone in front of 7,000 people. The receptionist did not know that she was not talking to a real person. This experiment is also considered by some scientists to be a successful Turing test, although it is not based on Alan Turing's original format.

ChatGPT and other modern AIs based on a Large Language Model (LLM) now regularly pass the Turing test. This is another reason why it is considered obsolete in science.

Three areas of application of the Turing test

Although Alan Turing's AI test has been widely criticized, the basic idea - the deceptively realistic imitation of human communication - still plays a role in digitalization today.

  • Human Interaction Proof (HIP): The CAPTCHA query can be seen as a negative Turing test. As a human interaction proof, it has the function of distinguishing humans from machines and effectively keeping bots away from a website through automated text and image queries. The name CAPTCHA stands for "Completely Automated Public Turing test to tell Computers and Humans Apart".
  • Bots: Bots are electronic tools that perform positive or negative tasks depending on their application. For example, they are used as chatbots to efficiently automate customer service inquiries. On the other hand, they are also used as social bots or spam bots to spread false information or malware. In both cases, variants of the Turing test help to further develop bots and make them more human-like.
  • Voice assistants: Of all technologies, voice assistants probably come closest to Alan Turing's original concept. Voice assistants such as Alexa or Siri are based on a question-and-answer system and are designed to automate everyday tasks and user needs. Even if today's applications do not yet reach the level to pass the Turing test, they are constantly being improved through machine learning and the analysis of user behavior and are becoming more and more similar to humans.

Is Alan Turing's test still relevant today?

Eva EggelingCenter Director for Data Driven Design at Fraunhofer Austria and Head of the Innovation Center for Digitalization and Artificial Intelligence KI4Life says: "I think it is particularly topical now, especially since the great popularity of ChatGPT for the general population."

Despite much debate about the relevance of the Turing test today and the validity of competitions based on it, the test is still a philosophical starting point for the discussion and exploration of AI. As we continue to make advances in AI and better understand and map how the human brain works, the Turing test remains fundamental to defining intelligence and is a basis for debate about what we should expect technologies to be thinking machines.

Also of interest

AI in practice
AI in practice
It is both an economic necessity and a strategic goal: AI should give European industry a competitive edge. And indeed, more and more companies are turning to the technology of the future. Examples of AI in practice show the possible applications.
Play Video
AI Insights from Fraunhofer Austria
Welcome to the third part of our series "AI Rap with Eva Eggeling"! In this video we dive deep into the world of generative AI and learn how creative algorithms work.
Artificial-intelligence-benefit
How AI works
AI is everywhere - but where is the journey heading for industrial applications? We highlight how different players - from global tech giants to small startups - are using artificial intelligence to drive innovation and optimize work processes.
Play Video
Experts talking about AI
Klaus Paukovits, Head of Business Development at WEKA Industrie Medien, talks to Sebastian Schlund, Center Manager for Sustainable Production and Logistics, and Eva Eggeling, Center Manager for Data Driven Design, at Fraunhofer Austria about innovations in AI in industry.
Sepp-Hochreiter_Header
AI Research
AI researcher Sepp Hochreiter has made another remarkable breakthrough. A language model that works more efficiently and faster than ChatGPT and similar systems. However, the research lead painstakingly earned in Austria is in danger of fizzling out.
Regulation-KI_Header
KI Regulation
In mid-June 2023, the EU Parliament settled on a system for regulating AI. What exactly is in the AI regulation, how companies are affected in the development and use of AI systems, who is in favor of it and what criticisms are being voiced, you can read here.

Multimedia

Blog

Read here about strategies, possible applications and technologies in the manufacturing industry.

Videos

We bring experts and personalities from science, research and industry together at one table and discuss ...

en_USEnglish
Powered by TranslatePress
WordPress Cookie Plugin by Real Cookie Banner