AI Talk: Augmented arguments

Jan. 31, 2020 / By V. “Juggy” Jagannathan, PhD

This week’s AI Talk…

We have all heard of data mining, but have you heard of the term argument mining? Turns out it is an active area of research in natural language processing! You can see the latest paper that IBM Research has published in this area here. I came across reference to this work in an article from MIT Technology Review last week. What exactly is the IBM research team trying to do here? They are addressing the question of how to marshal arguments for and against any motion. A motion can be any, possibly controversial, statement such as: Blood donation should be mandatory. And, given a motion, their system does a search for sentences in a large corpus of documents to find both positive (in favor of the motion) and negative (against the motion) examples. They are further classified as being of two types of evidence: one that is supported by research (Study) or by expert opinion (Expert).

How does this work? Well, all sentences retrieved must mention the topic (in the above example, blood donation), source of information (for example, a study), and a reference to an action (ex. mandatory, optional, etc.). The retrieved sentences are classified using a neural-net classifier. The classifier itself is trained using state-of-the-art neural-net solutions such as BERT and Bi-directional LSTM. Intuitively, the machine learns the contextual patterns that frame the pros and cons of arguments. So, given any topic, the sifting of millions of sentences yields the ones that fit these patterns.

IBM Project Debater developed under the leadership of Israeli researcher Noam Slonim, who published the above research paper, and has a good history of showcasing this particular line of their work. Last spring, in an experiment similar to Jeopardy and Watson, they orchestrated a human vs machine debate in San Francisco. The human contestant was Harish Natarajan, a world debating champion. You can view this entire debate here. The topic of the debate? We should subsidize preschool. Project Debater here displays its impressive prowess of speech recognition, summarization and depiction of humor. You can see stylistic and content differences between the arguments put forth by computer and human contestants. The computer was backing its arguments with lots of factual information whereas Harish arguments were nuanced, and policy-based. Harish won the debate, but it is clear the computer can marshal arguments.

The follow-up for the above debate was a full-on debate on the topic of “AI will bring more harm than good,” in the hallowed halls of Cambridge. The site was chosen well, as it happens to be the venue where innumerable debates have taken place over two centuries—under the auspices of the Cambridge Debating Society. This debate, where Project Debater skills were used, was different than a typical debate. This blog by IBM Research explains the debate in detail. Here, the two teams arguing for and against the proposition regarding AI both got a lift from the IBM Project Debater. That is, Project Debater supplied arguments to both teams arguing for and against the proposition. The arguments themselves were crowd sourced and the role of Debater here was to select, classify, prioritize, synthesize and present. Take a look at the arguments, both pro and con, for the debating topic that were synthesized by a Debater. They are indeed relevant and appropriate—both pro and con! That’s the message from this research team: AI will augment our arguments! The actual debate, with its 200-year-old voting system (people walk through different doors to signify their vote!), resulted in almost a tie! Which goes to show, people are quite ambivalent about the role of AI.

Now back to the original paper that we talked about! This is indeed interesting work and there is still much work that needs to be done–like handling bias, selection that does not necessarily depend on explicit mention of topics, etc. However, if IBM succeeds and releases a tool that will summarize arguments for and against controversial topics, it has the potential to raise the quality of discourse happening in our society. It may transform an argument from meaningless shouting in echo chambers to a civil discourse. One can dream.

I am always looking for feedback and if you would like me to cover a story, please let me know. “See something, say something”! Leave me a comment below or ask a question on my blogger profile page.

V. “Juggy” Jagannathan, PhD, is Director of Research for 3M M*Modal and is an AI Evangelist with four decades of experience in AI and Computer Science research.