Stanford University's Percy Liang Spearheads AI Transparency Initiative

blockchain.news:

In the rapidly evolving landscape of artificial intelligence, the emergence of foundation models like GPT-4 and Llama 2 has transformed numerous sectors, influencing decisions and shaping user experiences on a global scale. However, despite their widespread use and impact, there is a growing concern about the lack of transparency in these models. This issue is not limited to AI; it echoes the transparency challenges faced by previous digital technologies, such as social media platforms, where consumers grappled with deceptive practices and misinformation​​​​.

The Foundation Model Transparency Index: A Novel Tool for Assessment

To address this critical issue, the Center for Research on Foundation Models at Stanford University, along with collaborators from MIT and Princeton, developed the Foundation Model Transparency Index (FMTI). This tool aims to rigorously assess the transparency of foundation model developers. The FMTI is designed around 100 indicators, spanning three broad domains: upstream (covering the ingredients and processes involved in building the models), model (detailing the properties and functionalities), and downstream (focusing on distribution and usage). This comprehensive approach allows for a nuanced understanding of transparency in the AI ecosystem​​​​.

Key Findings and Implications

The FMTI’s application to 10 major foundation model developers revealed a sobering picture: the highest score was a mere 54 out of 100, indicating a fundamental lack of transparency across the industry. The average score was just 37%. While open foundation model developers, allowing downloadable model weights, led the way in transparency, closed model developers lagged, particularly in upstream issues like data,