AI Talk: Data woes and Perceiver

Aug. 6, 2021 / By V. “Juggy” Jagannathan, PhD

AI models for COVID-19

I was skimming through MIT Technology Review and I saw one article that stopped me cold. It proclaimed that hundreds of AI models were developed to detect COVID-19 during the pandemic but NONE helped. Wow! We are all still living through this nightmare pandemic, but one thing that has buoyed our spirits is the rapid development of vaccines—thanks to advances in genetic technology. So what about all those AI models developed by hundreds of researchers all over the world? The verdict is in and it is quite sad. Multiple detailed retrospective studies have pretty much confirmed the same conclusion: AI models were created too rapidly using bad and biased datasets, thus the models were practically useless.

The Alan Turing Institute released a report last month summarizing a workshop that was held late last year to assess the role of data science and AI for COVID-19. The goal was to figure out how to make these processes more effective. The report highlights the lack of robust, unbiased, timely data. It also highlights the culture of creating proprietary and non-replicable models.

The Biomedical Journal released a similar report, co-authored by roughly 40 researchers from around the world. These researchers have a website where evaluations of 238 models are ongoing. You can look at their assessments here. They have found that, out of the 238 models they are reviewing, 228 have a high degree of bias, six of them they are unsure of, leaving only four that have potentially low bias.

In a very interesting lecture from a few months ago, Andrew Ng makes the argument that to create good models, one needs to pay attention to ensuring clean data. Ignore that advice at your own peril.

The fundamental problem highlighted by the MIT Technology Review article is that academic researchers have no incentive to share datasets. This is compounded by the fact that privacy rules forbid sharing in any meaningful fashion. Hence, no one else can verify or validate the approach. The World Health Organization is now seriously considering instituting data sharing policies in the future. Let’s hope we don’t have to wait until the next pandemic to develop solutions that are aimed at advancing the common good.

Perceiver

Now, another research advancement from the DeepMind team, the one that solved the protein folding problem. Their latest system is dubbed “Perceiver,” a deep learning model that is agnostic to the type of input it is dealing with—be it images, audio or video. The name for their model is inspired by biological systems, specifically our perception system that processes all sensory inputs seamlessly. So, the question is can you develop a model that can process different modalities of input? The DeepMind researchers have managed to architect a Transformer model that pays iterative attention to successive portions of arbitrary inputs. They show that they achieve better than state of the art on a variety of standard datasets for audio, video and images using this approach. If you are interested in understanding the technical details of their work, you can read their paper. If you want to get a better understanding of what they have done, this YouTube video provides a great summary. Another small step towards artificial general intelligence.

Cyberattack perception

The latest ACM TechNews had an interesting story about cyberattacks. Yang Cai, a senior researcher at Carnegie Mellon University (CMU), has converted network traffic data into music! Cai, who is not a musician, collaborated with two CMU alumni who are both musicians—one a composer and the other a harp player. There is an ocean of cybersecurity data and monitoring that data to recognize abnormal network traffic is quite difficult. Cai converts the change in the network traffic when a cyberattack occurs to music and the change in pitch is instantly recognizable to even non-musicians. There is a nice clip of music created from network traffic you can listen to here. Cai’s vision is to eventually support the use of virtual reality goggles bringing about a multi-modal perception of abnormal cyber events. More power to him.

I am always looking for feedback and if you would like me to cover a story, please let me know! Leave me a comment below or ask a question on my blogger profile page.

V. “Juggy” Jagannathan, PhD, is Director of Research for 3M M*Modal and is an AI Evangelist with four decades of experience in AI and Computer Science research.