Miriam Vogel
More posts from Miriam Vogel
The National Security Commission on Artificial Intelligence (NSCAI) issued a report last month delivering an uncomfortable public message: America is not prepared to defend or compete in the AI era. It leads to two key questions that demand our immediate response: Will the U.S. continue to be a global superpower if it falls behind in AI development and deployment? And what can we do to change this trajectory?
Left unchecked, seemingly neutral artificial intelligence (AI) tools can and will perpetuate inequalities and, in effect, automate discrimination. Tech-enabled harms have already surfaced in credit decisions, health care services and advertising.
To prevent this recurrence and growth at scale, the Biden administration must clarify current laws pertaining to AI and machine learning models — both in terms of how we will evaluate use by private actors and how we will govern AI usage within our government systems.
The administration has put a strong foot forward, from key appointments in the tech space to issuing an executive order on the first day in office that established an Equitable Data Working Group. This has comforted skeptics concerned both about the U.S. commitment to AI development and to ensuring equity in the digital space.
But that will be fleeting unless the administration shows strong resolve in making AI funding a reality and establishing leaders and structures necessary to safeguard its development and use.
Need for clarity on priorities
There has been a seismic shift at the federal level in AI policy and in stated commitments to equality in tech. A number of high profile appointments by the Biden administration — from Dr. Alondra Nelson as deputy of OSTP, to Tim Wu at the NEC, to (our former senior advisor) Kurt Campbell at the NSC — signal that significant attention will be paid to inclusive AI development by experts on the inside.
The NSCAI final report includes recommendations that could prove critical to enabling better foundations for inclusive AI development, such as creating new talent pipelines through a U.S. Digital Service Academy to train current and future employees.
The report also recommends establishing a new Technology Competitiveness Council led by the vice president. This could prove essential in ensuring that the nation’s commitment to AI leadership remains a priority at the highest levels. It makes good sense to have the administration’s leadership on AI spearheaded by Vice President Harris in light of her strategic partnership with the president, her tech policy savvy and her focus on civil rights.
The U.S. needs to lead by example
We know AI is powerful in its ability to create efficiencies, such as plowing through thousands of resumes to identify potentially suitable candidates. But it can also scale discrimination, such as the Amazon hiring tool that prioritized male candidates or “digital redlining” of credit based on race.
The Biden administration should issue an executive order to agencies inviting ideation on ways AI can improve government operations. The order should also mandate checks on AI used by the USG to ensure it’s not spreading discriminatory outcomes unintentionally.
For instance, there must be a routine schedule in place where AI systems are evaluated to ensure embedded, harmful biases are not resulting in recommendations that are discriminatory or inconsistent with our democratic, inclusive values — and reevaluated routinely given that AI is constantly iterating and learning new patterns.
Putting a responsible AI governance system in place is particularly critical in the U.S. Government, which is required to offer due process protection when denying certain benefits. For instance, when AI is used to determine allocation of Medicaid benefits, and such benefits are modified or denied based on an algorithm, the government must be able to explain that outcome, aptly termed technological due process.
If decisions are delegated to automated systems without explainability, guidelines and human oversight, we find ourselves in the untenable situation where this basic constitutional right is being denied.
Likewise, the administration has immense power to ensure that AI safeguards by key corporate players are in place through its procurement power. Federal contract spending was expected to exceed $600 billion in fiscal 2020, even before including pandemic economic stimulus funds. The USG could effectuate tremendous impact by issuing a checklist for federal procurement of AI systems — this would ensure the government’s process is both rigorous and universally applied, including relevant civil rights considerations.
Protection from discrimination stemming from AI systems
The government holds another powerful lever to protect us from AI harms: its investigative and prosecutorial authority. An executive order instructing agencies to clarify applicability of current laws and regulations (e.g., ADA, Fair Housing, Fair Lending, Civil Rights Act, etc.) when determinations are reliant on AI-powered systems could result in a global reckoning. Companies operating in the U.S. would have unquestionable motivation to check their AI systems for harms against protected classes.
Low-income individuals are disproportionately vulnerable to many of the negative effects of AI. This is especially apparent with regard to credit and loan creation, because they are less likely to have access to traditional financial products or the ability to obtain high scores based on traditional frameworks. This then becomes the data used to create AI systems that automate such decisions.
The Consumer Finance Protection Bureau (CFPB) can play a pivotal role in holding financial institutions accountable for discriminatory lending processes that result from reliance on discriminatory AI systems. The mandate of an EO would be a forcing function for statements on how AI-enabled systems will be evaluated, putting companies on notice and better protecting the public with clear expectations on AI use.
There is a clear path to liability when an individual acts in a discriminatory way and a due process violation when a public benefit is denied arbitrarily, without explanation. Theoretically, these liabilities and rights would transfer with ease when an AI system is involved, but a review of agency action and legal precedent (or rather, the lack thereof) indicates otherwise.
The administration is off to a good start, such as rolling back a proposed HUD rule that would have made legal challenges against discriminatory AI essentially unattainable. Next, federal agencies with investigative or prosecutorial authority should clarify which AI practices would fall under their review and current laws would be applicable — for instance, HUD for illegal housing discrimination; CFPB on AI used in credit lending; and the Department of Labor on AI used in determinations made in hiring, evaluations and terminations.
Such action would have the added benefit of establishing a useful precedent for plaintiff actions in complaints.
The Biden administration has taken encouraging first steps signaling its intent to ensure inclusive, less discriminatory AI. However, it must put its own house in order by directing that federal agencies require the development, acquisition and use of AI — internally and by those it does business with — is done in a manner that protects privacy, civil rights, civil liberties and American values.
Comment