The keynote The inaugural Google I/O 2023 was plagued with announcements about artificial intelligence. Especially those linked to PaLM 2, its new massive language model. There we were able to take a brief look at Med-PaLM 2, a chatbot similar to Bard or ChatGPT but specifically applied to medicine. and the company now has begun testing it in different hospitals in the United States.
As reported The Wall Street Journal, Google is testing Med-PaLM 2 in multiple health services, including the renowned Mayo Clinic. There, the platform is used to answer medical questions, and it is also applied to more tedious tasks, such as organizing health information or summarizing documentation.
Google claims that it does not gain access to the personal information or medical records of customers already testing Med-PaLM 2. This is because the data is encrypted so that it cannot be accessed by the California-based firm, which also prevents them from being used to train the AI chatbot.
In any case, that the platform based on the same Bard technology is already being tested in hospitals, does not mean that its mass launch is just around the corner. Those of Sundar Pichai for now have refused to mention when Med-PaLM 2 could be enabled in more health centers.
Med-PaLM 2 wants to raise the bar for generative AI applied to medicine
When Google gave a sample of the capabilities of Med-PaLM 2, you could see how the chatbot interpreted an injury from the image of an x-ray. This made it clear that those from Mountain View want their AI to become, sooner or later, a kind of virtual assistant for doctors. However, that will probably still be several years away.
The answers that Med-PaLM 2 is giving to the customers who are trying it are under permanent review by real doctors. And while Google’s technology has already proven its worth, delivering quality returns on most benchmarks, it can’t escape the specter of other AI-based apps: hallucinations.
What does this mean? That just as ChatGPT or Bard can invent answers in seconds when asked something about which they are not trained, Google’s medical chatbot can do something similar. Experts have found inaccuracies and irrelevant content in diagnoseswhen comparing them with those offered by real doctors in the same situation.
Google researchers said they haven’t yet been able to develop a method to test for flaws in Med-PaLM 2 results. So don’t expect the chatbot to start being deployed as a foolproof tool for complex diagnostics any time soon. The AI has delivered very interesting initial results, but it is still far from reaching a level of reliability worthy of a science fiction movie.
And the latter is not surprising. when google revealed the plans for Med-PaLM 2, warned that still had a lot to learn about how to apply it to the field of medicine:
“The impressive performance of Med-PaLM 2 on medical exam-style questions is a promising development, but we need to learn how it can be leveraged to benefit healthcare workers, researchers, administrators, and patients. In building Med-PaLM 2, we have focused on safety, fairness, and unfair bias evaluations. Our limited access for select Google Cloud customers will be an important step forward in furthering these efforts, bringing additional expertise across the life sciences and healthcare ecosystem.” .
Google.