Identification of Gaps in AI Governance
We need to focus on real world AI deployment
Paper: Real-World Gaps in AI Governance Research
Researchers from Social Science Research Council, University College London and O’Reilly Media are interested in understanding AI and its performance and impact in real-world.
Hmm..What’s the background?
Corporate AI research is found to increasingly concentrate on pre-deployment areas like model alignment and testing & evaluation, while attention to deployment-stage issues such as model bias has decreased. This focus on pre-deployment laboratory settings through model alignment and testing neglects user, system, and society-level impacts.
Growing corporate concentration in AI research is seen as potentially exacerbating these deficiencies, as the commercial 'AI race' prioritizes an engaging user experience over broader societal impacts. Furthermore, those in the best position to monitor and understand these risks (corporations) have economic and reputational incentives to underplay them rather than conduct transparent research.
So what is proposed in the research paper?
Here are the main insights:
The authors constructed a large dataset of 1,178 AI safety and reliability governance papers from a total of 9,439 generative AI papers published between January 2020 and March 2025
They used OpenAI’s o3-mini to classify papers into eight sub-categories and also conducted keyword searches for high-risk deployment domains and capabilities
Only 4% of Corporate AI papers tackle high-stakes areas like persuasion, misinformation, medical & financial contexts, disclosures, or core business liabilities (IP violations, coding errors, hallucinations), despite emerging lawsuits showing these risks are material
Corporate AI tends to frame bias as a pre-deployment model personality issue rather than a post-deployment statistical issue
What’s next?
The central recommendation is the need for structured access for external researchers and auditors to data on AI systems operating in real-world environments. Ultimately, bridging the gap requires not just incident tracking, but continuous, structured observability of AI in the wild for governance through tiered public research, governance, and audit access.
We need to focus on real world AI deployment
Learned something new? Consider sharing it!