July 22, 2020
July 22, 2020
Contributor: Ben Hertzberg
Public concerns about AI dangers are warranted. An external AI ethics board can help embed representation, transparency and accountability into AI development decisions.
After widespread protests against racism in the U.S., tech giants Microsoft, Amazon and IBM publicly announced they would no longer allow police departments access to their facial recognition technology. The concern? Artificial intelligence (AI) can be prone to errors, particularly in recognizing people of color and those in other underrepresented groups.
Such ethical concerns don’t end at facial recognition software. Any organization developing or using AI solutions needs to be proactive in ensuring that AI dangers don’t jeopardize their brand, draw regulatory actions, lead to boycotts or destroy business value.
Microsoft President Brad Smith was widely quoted as saying his company wouldn’t sell facial-recognition technology to police departments in the U.S., “until we have a national law in place, grounded in human rights, that will govern this technology.”
So, in the absence of highly rigorous institutional protections against AI dangers, what can organizations do themselves to guard against them?
After all, AI could lead any number of negative outcomes, from eliminating stable, well-paying jobs to determining prison sentences and medical benefits through unaccountable algorithms. Anything fully automated can abuse and be abused.
The first, and perhaps the most critical, step to prevent these outcomes: Install an external AI ethics board to prevent — not just mitigate — AI dangers.
The COVID-19 pandemic has raised specific concerns about how to apply data ethics to decision-making processes when collecting, using and sharing data about employees and the health of individuals. But every organization now has to guard against AI dangers if it hopes to leverage the many opportunities AI offers to the business.
The perceived and actual dangers most often stem from the ability of AI to make decisions and take action with little or no human intervention. Often publicized are the risks that AI poses to privacy, jobs, relationships and equality.
Read more: 3 Ways to Embrace Proactive Data Ethics
Although public mechanisms do exist to mitigate against AI dangers, they aren’t sufficient. Market forces have encouraged companies to develop AI to meet, for example, government needs for facial recognition and surveillance — yet there hasn’t been an associated boost in their commitment to ethical use.
Regulators often aren’t prepared or knowledgeable enough about AI to codify adequate oversight. Courts, focused on avoiding a difficult-to-reverse legal outcome, tend to be more or less stringent than is necessary to ensure public well-being.
It’s hardly surprising, then, that AI solution developers are concerned that their AI will become warped during development or be misused once in the market. Microsoft’s concerns about the use of facial recognition technology are certainly warranted, given that the advanced technologies, while designed to protect communities, can also negatively impact those very same communities.
And that burden can all too easily fall on people of color, those from the LGBTQ community and others who already tend to be underrepresented in many of the institutions that develop and provide oversight of such technologies.
Axon, a law enforcement technology solution provider, established an external ethics board to help mitigate the shortcomings of the mechanisms meant to protect the public. Specifically, the company created a board that enabled greater transparency, accountability and representation in the AI development process. The following are among the lessons learned.
Your primary customers are the buyers of your products, but who are the consumers? In the case of policing technologies, law enforcement agencies might be the customers but the communities they serve are the consumers — and they are directly impacted by the use of the technologies.
Make sure to overindex voices that understand your critical consumers and can provide insight into the potential impact of your AI technologies. This helps to filter out poor AI product proposals before they go to market.
Be totally transparent with your AI ethics board about every AI project and your AI roadmap. An uninformed, unaware board is not a useful board. Make sure your board members have all the details needed to make confident recommendations on developing projects. This kind of transparency and credibility will help you to acquire and keep worthy board members.
Once the board has made its recommendations, senior leaders need to respond to them — publicly. This demonstrates that the organization is committed and accountable.
Axon also took discrete steps to empower the AI ethics board and its members:
An effective external AI ethics board embeds in the organization’s culture AI development practices that drive competitive advantage and business resilience and make the organization more attractive to talent.
Ben Hertzberg is a director of research in the Gartner Data & Analytics (D&A) practice, and drafted this view along with Ariel Silbert, senior specialist. The Chief Data & Analytics Officer Research Team, of which Ben and Ariel are a part, helps chief D&A officers achieve their mission-critical priorities by developing peer-sourced best practice recommendations.
Join your peers for the unveiling of the latest insights at Gartner conferences.
Recommended resources for Gartner clients*:
Case Study: Ethical AI With an External Board (Axon)
AI Ethics: Use 5 Common Guidelines as Your Starting Point
*Note that some documents may not be available to all Gartner clients.