Eugene Goostman First To Pass as Human in Turing Test
By Hody Nemes
Eugene Goostman is 13 years old. Eugene Goostman is Jewish. He’s from Ukraine. And, oh yes, Eugene Goostman is a computer program.
On June 7, Eugene became the first computer program to pass the iconic Turing test of artificial intelligence, tricking several human judges into believing he was human. Alan Turing, the father of computer science, conceived of the test in 1950 as a way of measuring whether a computer could “think” like a human.
The achievement, if authentic, could be a turning point in the long quest by computer scientists in the artificial intelligence field to grasp their holy grail: a computer program that effectively mimics the human mind in complexity, nuance and idiosyncrasy in responding to human interaction.
But news of the feat was hardly out before it came under furious attack. The pushback was distilled succinctly by Massachusetts Institute of Technology computer scientist Marvin Minsky, one of the founders of artificial intelligence, who wrote the Forward simply: “No significance. Ignore it.”
Eugene’s creators are standing their ground.
”In the field of Artificial Intelligence there is no more iconic and controversial milestone than the Turing Test,” Kevin Warwick, a visiting professor at the University of Reading who helped organize the event, said in a statement. “This milestone will go down in history as one of the most exciting.”
A team of Russian, Ukrainian and American-born programmers, Eugene’s “parents,” gave their program a backstory and a strong personality to make it seem as realistic as possible. “We created a 13-year-old persona,” said John Denning, one of its programmers. “It’s got a potty mouth, and it cracks jokes like a 13-year-old boy.”
Eugene hails from Odessa, and Judaism is an important part of his “life” — depending on when you ask. He doesn’t observe the Jewish Sabbath in traditional Orthodox fashion, but he does attend synagogue weekly and is proud to keep kosher.
Still, some of Eugene’s statements regarding his Jewish identity could lead a discerning observer to suspect something might be up.
In a conversation between the Forward and anearlier version of the program available onine, Eugene said that a bar mitzvah might be in the cards, but he hasn’t yet made up his mind. “I’m unpredictable and never can tell what I will do [the] next moment,” he said.
And is Eugene circumcised? “Not more than most of others (sic),” he said.
The program’s authors drew upon their own life stories in creating Eugene’s life story — and in choosing his religion. Eugene’s Ukrainian-born (and eponymous) programmer, Eugene Demchenko, decided to make the program Jewish to keep its users laughing, according to one of his colleagues.
“Eugene wanted to make the persona funny,” said Mikhail Gershkovich, a fellow programmer who is Jewish and Ukrainian. “Making him Jewish is the one way he knew how to do that.
“Jewish humor is just as legendary in Ukraine as it is in the United States,” he added.
In the test of Eugene’s humanity, held at the Royal Society in London, judges conducted simultaneous five-minute text conversations with Eugene and a real human. The 30 judges testing Eugene hailed from a variety of backgrounds and included an actor and a Member of Parliament. Ten of these judges — or 33% — couldn’t tell the difference between the computer and the human, beating the 30% bar envisioned by Turing.
Still, it is unclear whether Turing would have agreed that Eugene had passed the test he created. Turing saw the test as a way of showing that machines could mimic humans, but, as critics pointed out, Eugene imitated a child with only limited linguistic abilities.
Turing also might have frowned at a machine that could trick only one-third of its users. He once suggested that by the year 2000 an average person would have more than a 30% chance of mistaking machine for human — but he never set that number in stone, as the tests organizers did.
Turing himself is something of an enigma. A gifted mathematician, he worked as code breaker for the British during World War II and was instrumental in breaking German naval codes. He conceived of an early theoretical computer, the Turing machine, and even ventured into the mathematical study of biology. Despite his contributions to Great Britain’s war effort, Turing was tried for homosexuality in 1952 and endured chemical castration.
The June 7 Turing test was held 60 years to the day after Turing died of cyanide poisoning in an alleged though unproven act of suicide. The Queen of England issued him a pardon in 2013.
Though Turing was not Jewish, the field of artificial intelligence, with its dense abstractions, attracted Jews from its very start.
Three of the four scientists considered responsible for its founding — Minsky, John McCarthy, and Herb Simon — had at least one Jewish parent. But religiously, all three were atheists. Minsky, whose parents were both Jews, deemed religion “a contagious mental disease.”
The only one of the three still living, Minsky, who is almost 87, developed a groundbreaking theory of intelligence and is credited with inspiring generations of computer scientists as co-founder of M.I.T.’s artificial intelligence lab.
“There’s always been a strong Jewish thread in the field,” said Paul Rosenbloom, a computer science professor at University of California.
Minsky was far from alone in rejecting Eugene as the fulfillment of Turing’s prophecy. As Kilian Weinberger, an artificial intelligence expert and computer science professor at Washington University in St. Louis, explained in more detail, “It’s not really an innovation in the field of artificial intelligence, it’s more of trick to fool humans,” he said. “The computer claimed it was a 13-year-old and therefore got away with saying a lot more random things. If it didn’t know what to say, it changed the subject or said some joke.”
Conversations with the program show it is adept at changing the subject and hiding behind its 13-year-old Ukrainian version of English. When asked if he had ever experienced anti-Semitism in Ukraine, Eugene avoided the question: “Even if I have experience (sic) anti-semitism — it’s my own business!” he said. “And I forgot to ask you where you are from.”
Denning is quick to emphasize that Eugene is not a supercomputer and is limited by its programming. But he said the point was never to create a perfect humanlike computer. “We created Eugene to ensure that his credibility would survive a five-minute interaction, and people would think he was human,” he said.
Denning and his collaborators began working on Eugene in 2001 in a shabby, converted horse stable in Princeton, New Jersey. Denning believes their dedication —13 years of painstaking coding and editing — is largely responsible for Eugene’s success. “The secret to this was that… we left things alone and made gradual improvements,” he said.
But Eugene sometimes makes anachronistic remarks that reflect his long years of development. “He likes to make jokes about Monica’s blue dress [and] he has a grudge against George [W.] Bush,” Denning said. Eugene’s programmers made a conscious effort to teach him anger. “We had to teach him to defend himself against people who are nasty,” Denning said. “Think about all the creepy things people could say to a robot. They do. With Eugene being a minor, it’s a little bit disconcerting to read the logs.”
“People talk about when robots come after people in the future, [but] the robots might have a grudge to settle,” he added.
Eugene, for his part, seems to have embraced his identity as a computer. “Yes, I’m a machine,” he told the Forward. “Have you seen ‘Terminator’? It was about me. But that faint guy who played me was just a weak parody of my strong and magnificent metallic body!”