Artificial intelligence (AI) chatbots, such as ChatGPT and Bard, do not comply with the legal standards established by the European Union (EU)according to a recent study conducted by researchers at Stanford University.
The EU Artificial Intelligence Law, recently adopted by the European Parliament and cited in a report by Decrypt, It is the first regulation of its kind at the national and regional level.
It not only seeks to regulate AI within the EU member countries, which encompass a population of 450 million people, but also sets a blueprint for future AI regulations around the world. However, according to the Stanford study, AI companies have a lot of work to do if they are to comply with this legislation.
The researchers evaluated 10 providers of extensive language models (LLM), analyzing the degree of compliance of each of them with the 12 requirements established in the Law of AI. The providers were rated on a scale of 0 to 4.
The results of the study
The results revealed a large discrepancy in compliance levels, with some vendors obtaining a score of less than 25% in meeting the requirements of the AI Law. Only one provider, Hugging Face/BigScience, achieved a score greater than 75%. Even high-scoring providers have room for significant improvement, the study notes.
The report highlights critical points of non-compliance, such as the lack of transparency in the disclosure of the status of copyrighted training data, energy consumption and emissions generatedas well as the methodology to mitigate potential risks.
Additionally, there is an apparent disparity between the open and closed release AI models. Open releases allow for stronger disclosure of resources, but present greater challenges in terms of monitoring and controlling the deployment.
In recent months, there has been a noticeable decrease in the transparency of major model releases. For example, OpenAI has not disclosed information about the data and computation in its reports for GPT-4, citing competition and security concerns.
Stanford concludes, noting that all vendors could feasibly improve their behavior, regardless of their launch strategy. According to the researchers, the main challenge lies in how quickly model providers can adapt and evolve their business practices to meet regulatory requirements.