Human edges out machine in latest artificial intelligence showdown

by Peter Griffin / 13 February, 2019
ibm project debater  vs Harish Natarajan

IBM's Project Debater AI Machine debated whether pre-school should be subsidised with debating champ Harish Natarajan.

In all of the contests of the last 25 years pitting supercomputer against human, one thing has remained constant – there’s always a finite number of possible answers available to flesh and silicon-based player alike.

In 1996’s match-up between IBM’s Deep Blue supercomputer and Russian chess grandmaster Gary Kasparov, the one constant was the total number of possible moves on a chess board. The result was a draw – one series won by Kasparov, the other by Deep Blue.

In 2011, the IBM Watson machine had to trawl its database to find the right answers to cryptic questions asked of it during the quiz show Jeopardy. Watson trounced its human rivals.

But what about a live debate, where facts matter less than human intuition and the power of persuasion?

Last night’s debate in San Francisco between Project Debater, an artificial intelligence engine based on Watson, and Harish Natarajan, the 2016 World Debating Championship grand finalist, suggests this is where humans still have the edge – for now.

Project Debater sat on stage, a monolithic black tablet emanating an even, American-accented woman’s voice. Over 25 minutes she traded statements and rebuttals with Natarajan on the topic of whether pre-school should be subsidised.

Miss Debater, as she has been dubbed, was arguing for the resolution. She and her human rival had just 15 minutes to prepare for the debate. But able to trawl 10 billion sentences of reference material, mainly newspaper and magazine articles, it didn’t take long for the computer to formulate a strong argument for funding preschool.

The computer says yes

Laying out her case logically, she peppered her talk with OECD and US Centers for Disease Control report statistics and even quoted former British Prime Minister Benjamin Disraeli. Most impressive was Project Debater’s ability to listen to Natarajan’s arguments in real time, her display blinking away all the time, and counter them with a reasonable, fact-based rebuttal. She even managed some emotive flourishes at one point saying, “to be clear, my intention is not to leave a suitcase full of money for everyone to grab at will.”

Natarajan, without any research materials to draw on, had to rely on his best rhetorical skills. And his ability to do so skillfully is what separated man and machine. His rebuttals were stronger than Project Debater who largely just continued on with her narrative in favour of the resolution.

Sitting among the 700-strong audience at San Francisco's Yerba Buena Center, which held debaters from around the San Francisco Bay Area and a cohort of journalists, it was my job to help decide who won the debate.

The human edge

After the electronic votes were tallied, a humble and unflappable Natarajan was declared the winner. His victory hinged on the key argument that funding pre-school would largely just subsidise middle-class families who are likely to send their children to preschool anyway.

Prior to the debate, 79 per cent of the audience agreed that preschool should be subsidised. Afterwards however, that had dropped to 62 per cent. The debate rules awarded victory to the debater who gained the most points over the course of the debate and Natarajan subsequently emerged victorious.

But a second question asked of us revealed Project Debater’s true strength – the majority of the audience found Project Debater more informative.

“In terms of rhetorical skills, the system is still not at the level of a debater like Harish,” admitted Noam Slonim, one of the Israeli IBM scientists who led the development of Project Debater.

“That said, the system is capable of pinpointing relevant evidence within a massive collection.”

Indeed, with no pre-programming specific to the debate topic and talking fluently and coherently for up to four minutes at a time, Project Debater is a world away from the short and sometimes garbled snippets emitted from the Siri or Alexa digital assistants.

Read more: Why Facebook’s mega messaging merger will make you rethink social media

That’s down to the sophisticated algorithms that underpin Project Debater, but also the huge computing power behind it – a dedicated computer server with dozens of processors and as much memory as 50 laptops.

But the potential real world applications of Project Debater’s intelligence are vast. In the area of government policy development it could canvas peer-reviewed literature to formulate arguments in support of evidence-based policy making, reducing the more subjective approach of human researchers.

Does Shane Jones’ regional growth fund proposal stack up? Should we create a national strategy for cancer treatment? Imagine the questions you could get smart, rational answers on.

ibm project debater  vs Harish Natarajan

Natarajan came out on top.

The results, of course, will always depend on the quality of the information that is fed into Project Debater and most of it after all, will have been written by humans anyway.

“If you take some of those skills and add a human being who can use it in slightly more subtle ways, I think that could be incredibly powerful,” concluded Natarajan following the debate.

It is a convenient conclusion for the AI companies forging ahead with development amidst fears that automation will destroy millions of jobs. The clear implication is that AI will become smarter at augmenting human skills, rather than taking the human out of the decision-making process entirely.

Taking the tedious and time-consuming effort out of research, giving human beings a better factual basis on which to make decisions, seems to be where Project Debater’s future lies.

Imagine the power of such an AI-powered service on your smartphone or sitting in the smart speaker in your lounge.

Given the exponential growth in computing power, we can expect it to be within our grasp within years. But by that stage, the machine may well be better able to outwit the smartest human debaters too.

Peter Griffin visited San Francisco as a guest of IBM.

Follow NOTED on Twitter, Facebook, Instagram and sign up to our email newsletter for more tech news.


Simon Bridges searches for a miracle
108491 2019-07-17 00:00:00Z Politics

Simon Bridges searches for a miracle

by Graham Adams

The opposition leader hoped to pick up election-winning tips in Australia.

Read more
Pumla Gobodo-Madikizela on the tragedy of post-apartheid South Africa
108416 2019-07-17 00:00:00Z Profiles

Pumla Gobodo-Madikizela on the tragedy of post-apa…

by Clare de Lore

Scathing critic of South African Government corruption Pumla Gobodo-Madikizela, here to give a public lecture, has insights about forgiveness after...

Read more
Writer Robert Macfarlane finds deeps truths in Underland
108287 2019-07-17 00:00:00Z Books

Writer Robert Macfarlane finds deeps truths in Und…

by Tony Murrow

In a new book, Robert Macfarlane heads underground to ponder mankind’s effect on the planet.

Read more
Why extra virgin olive oil is back on the menu for frying
108203 2019-07-17 00:00:00Z Nutrition

Why extra virgin olive oil is back on the menu for…

by Jennifer Bowden

For decades, the word in the kitchen has been that olive oil shouldn’t be used for frying, but new research could change that.

Read more
Abstract artist Gretchen Albrecht's true colours
108108 2019-07-16 00:00:00Z Profiles

Abstract artist Gretchen Albrecht's true colours

by Linda Herrick

Gretchen Albrecht paintings may be intangible, but they are triggered by real-life experience, she tells Linda Herrick.

Read more
That's a Bit Racist is playful, but it packs a punch
108435 2019-07-16 00:00:00Z Television

That's a Bit Racist is playful, but it packs a pun…

by Diana Wichtel

The taboo-busting doco is trying to change our default settings on race, but some people aren't stoked.

Read more
Are there too many tourists in NZ?
108444 2019-07-16 00:00:00Z Life in NZ

Are there too many tourists in NZ?

by North & South

Here's what's inside North and South's August 2019 issue.

Read more
Huawei's dogged determination: Can it make a breakthrough in New Zealand?
108428 2019-07-16 00:00:00Z Tech

Huawei's dogged determination: Can it make a break…

by Peter Griffin

The tech company at the centre of a trade war between the US and China is willing to go to extraordinary lengths to prove it can be trusted.

Read more