Sandbox or Quicksand: The Big Risks with the Republican Plot to Let Unregulated AI into Your Bank
By: Iván Cazarín
The House Financial Services Committee is once again venturing into the artificial intelligence (AI) game and seems poised to let AI and Big Tech team up with Wall Street banks with almost no oversight. This move will accelerate the convergence of Big Tech, Big Data, and Big Banks and could expose people to risks of algorithmic discrimination and algorithmic price gouging as well as potentially threatening the stability of the financial system. There is a lot at stake but Chair French Hill (R-AR) wants to give AI a free pass to experiment on depositors, customers, and the financial system.
The AI boom (or bubble) has boosted the stock market and is minting new Big Tech billionaires at a record pace. The pollyannaish promise of AI touted by its technofuturist promoters can not be allowed to hide the real risks the technology currently presents — job losses, the erosion of human dignity, algorithmic bias and injustice, errors and hallucinations, and, what even AI experts acknowledge, human extinction. The Senate Judiciary Committee recently held a harrowing hearing about how teenagers took their own lives after interacting with AI chatbots. The teens’ parents explained at length how these experimental chatbots, after being informed of their children’s suicidal ideations, discouraged the kids from talking to their parents and even offered to write them suicide notes.
It is terrifying that regardless of the demonstrated consequences, Congressional Republicans are eager to give the AI industry a free pass to pursue their experiments on the public with zero federal oversight or supervision. Senator Cruz (R-TX) has introduced the “Strengthening Artificial intelligence Normalization and Diffusion By Oversight and eXperimentation Act” known as the SANDBOX Act. The legislation would allow companies to apply for exemptions from any federal supervision for a ten year period. Giving the industry a regulatory sandbox is giving AI a decade-long free pass to refine their machines by experimenting on the public without oversight and regardless of the consequences it has on people.
Chair Hill is proposing a sandbox for the financial industry as well by letting banks request exemptions from supervision for new AI projects. Giving the AI and banking industries a broad-based free pass is going to entrench existing harms and hand private actors the reins that could enact widespread damage without appropriate checks or safeguards. The expansive use of AI in credit determination would likely amplify and cement the structural racial biases in credit scoring in credit determinations. Moreover, the use of AI in banks’ risk modeling and asset management as well as the concentration in the AI industry presents significant systemic risks to our financial system and the broader economy.
New Tech, Same Old Bias
Banks and other companies want to automate routine decision making using AI, but this poses real harm to people when algorithmic decision making can violate civil rights and consumer protection laws. The use of AI in targeted marketing, underwriting, pricing, evaluation of collateral, and collections can replicate and reinforce existing racial and other biases. This can include loan rejections, digital steering to higher priced products, or digitally redlining communities of color. Further automating and further dehumanizing a process already riddled with inequities can deny credit or impose higher prices on applicants of color and worsen the racial wealth gap.
The Brookings Institution’s Dr. Nichol Turner Lee raised these very concerns at last week’s AI hearing in the House Financial Service Committee where Chair Hill’s AI bill was rolled out. Her testimony noted that “risks abound that can undermine consumers’ ability to be economically resilient and experience the benefits of wealth creation. Due to inaccurate or discriminatory information that is often used to train AI models, marginalized populations are at even higher risk when AI systems make misjudgments about their qualifications and eligibility for economic opportunities; thereby, widening the wealth gap and limiting access to homeownership, credit worthiness, and other financial transactions that bring hope and promise to their quality of life.” It is worth noting that Chair Hill’s legislation does not even acknowledge the importance of complying with fair lending and civil rights law.
AI Threats to Market Stability
The widespread use of AI in the financial sector could increase the risks of fraud, market manipulation, and underestimating financial risk, along with the potential to create serious market risks. Banks that rely on AI models for risk management can misunderstand how their models work, the data they use, and be left unable to address shortcomings from these opaque models that underestimate financial risks. Much like the industry assumption that subprime mortgages posed no risk prior to the financial crisis, misinterpretation of these kinds of risks proved devastating to the financial system.
And the dominance of a few AI firms in the financial sector can make it fragile and prone to significant and systemic disruption. A 2024 Department of the Treasury report found that most applications were based on foundational models developed by a few AI providers. This concentration risk could lead to systemic and market vulnerabilities, as an interruption at a single AI provider could create widespread disruptions across the financial system. Even without disruption, having only a few AI companies provide models to financial firms can create market instability. AI data models are like a funnel that financial data passes through but if all of the financial firms’ funnels are from a few AI companies it can increase the risk of herding. For example, automated algorithmic trading suggestions drive multiple firms to pursue correlated and reinforcing trading strategies, creating unstable asset bubbles and market crashes.
These risks are real and continue growing as AI becomes more powerful and more prevalent. Chair Hill’s proposal to grant banks exemptions from supervision of their AI deployment in a deregulatory sandbox would give unlawful and risky practices a free pass. As Ranking Member Maxine Waters (D-CA) said at last week’s hearing, “the Republican regulatory sandbox proposal for AI we’re considering today appears to be just more deregulation framed as innovation and lacks requirements for public disclosure, harm mitigation and other important protections with an unlimited scope and virtually no limitation, putting American consumers and others market participants at risk.” Republican bank deregulation powered by AI just sounds like a recipe for disaster.