AI Talk: “Genius Makers” book review

April 23, 2021 / By V. “Juggy” Jagannathan, PhD

Genius Makers by Cade Metz

Last month the book, Genius Makers – The mavericks who brought AI to Google, Facebook, and the world was released. The title was quite intriguing, so I bought the book and read it. It was another page turner. The book covers the resurgence of neural networks and deep learning over the past decades. I have been in AI for the past four decades, but as the author points out, the majority of that time I was part of the “symbolic AI tribe.” I have been following the deep learning revolution for the past five years—that’s how long I have been blogging on AI—but I was completely unaware of the story behind how the transition to deep learning really happened. The book is full of characters and events, I will just highlight a few that caught my attention. Metz masterfully weaves together different threads in the deep learning revolution in this, his first book.

The main protagonist of the book is Geoffrey Hinton, a Brit and a descendant of mathematician George Boole of “Boolean logic” fame. The book starts off with a little-known fact about Geoff Hinton: As of 2006, he has lived with a back condition that does not allow him to sit down, forcing him to stand all day long or lie down. In 1986, he co-authored a landmark paper on back-propagation that is at the core of deep learning network’s ability to learn. That discovery languished without adoption by the broader AI community until last decade. Hinton and a few others, notably Yann LeCun and Yoshua Bengio (who together won the Noble-equivalent Turing Price a few years ago), persisted in working on fine tuning and developing neural networks the entire time.

During the December 2012 NeurIPS (Neural Information Processing Systems) conference, Hinton decided to conduct a silent auction. What was auctioned? A fledgling company he created with two of his students a few months earlier called “DNNResearch.” The bidders? Chinese firm Baidu, Google, Microsoft and then independent Deep Mind. Hinton accepted a bid of $44 million from Google! The impetus for this landmark scramble to acquire neural network talent was triggered by the paper published by Hinton and his students in the summer of 2012 about how well a neural network performed in recognizing images (it blew by the state of the art (SOTA) results at that time in 2012). This auction and its conclusion signified that Big Tech woke up to the potential of deep learning in early 2013 and then the race was on.

In spring of 2014, Google realized that their datacenter that fielded billions of search queries needed to change. Deep learning researchers put in a petition to acquire 40,000 GPUs—the processor that is at the core of neural computing. Though the management balked initially, Google invested $130 million and acquired the 40,000 GPUs! That investment completely changed the trajectory of innovation and deployment of deep learning at Google. The other tech firms were not far behind.  

Google went on to acquire Deep Mind—with its stated goal of developing Artificial General Intelligence (AGI). That of course, is an elusive goal and we are not anywhere close to realizing that dream of human-like intelligence. Open AI was created with a similar goal, with funding from the likes of Elon Musk. Meanwhile, deep learning has become a mainstream tool adopted by most companies around the world. The book also touches on the various ethical quagmires that swirl around AI today.

Hinton’s life has been marked by multiple tragic events. His first and second wife both succumbed to cancer, but he continues to do seminal work well into his seventies. Just recently he published a new manifesto on what he believes will be the next evolution of AI. He calls this system “GLOM” (a play on the word “agglomerate”) on how to tease out part-whole relationships in what is being perceived. Hinton joked, in an interview with MIT Technology Review, that it is a clever acronym for “Geoff’s Last Original Model.” Given his track record, I wouldn’t bet on it being his last! Whether this work will turn out to be as impactful as the back-propagation algorithm, only time will tell, but his advice to researchers that he gave during his Turing award presentation cements his reputation as a maverick:  “If you have an idea and it seems to you it has to be right, don’t let people tell you it’s silly,” he said. “Just ignore them.”

Read the Washington Post review of the book.

I am always looking for feedback and if you would like me to cover a story, please let me know! Leave me a comment below or ask a question on my blogger profile page.

V. “Juggy” Jagannathan, PhD, is Director of Research for 3M M*Modal and is an AI Evangelist with four decades of experience in AI and Computer Science research.

Listen to Juggy Jagannathan discuss AI on the ACDIS podcast.