During\u00a0last Wednesday\u2019s congressional hearing\u00a0about Twitter transparency, Twitter CEO Jack Dorsey was forced to take accountability for the damaging cultural and political effects of his company. Soft-spoken and contrite, Dorsey provided a stark contrast to Facebook\u2019s Mark Zuckerberg, who seemed more confident\u00a0when he appeared before Congress\u00a0in April. In the months since, collective faith in the fabric of the internet has been anything but restored; instead, consumers, politicians, and the tech companies themselves continue to grapple with the aftermath of what social platforms hath wrought.<\/p>\n
During the hearing, Representative Debbie Dingell asked Dorsey if Twitter\u2019s algorithms are able to learn from the decisions they make\u2014like who they suggest users follow, which tweets rise to the top, and in some cases what gets flagged for violating the platform\u2019s terms of service or even\u00a0who\u00a0gets banned\u2014and also if Dorsey could explain how all of this works.<\/p>\n
\u201cGreat question,\u201d Dorsey responded, seemingly excited at a line of questioning that piqued his intellectual curiosity. He then invoked the phrase Explainable AI, or \u201cexplainability,\u201d a field of research he said Twitter is currently investing in, though he noted it\u2019s early days yet.<\/p>\n
Dorsey didn\u2019t elaborate further, and on the hearing went. But this wasn\u2019t the first time the CEO mentioned explainability. In August,\u00a0Dorsey appeared on Fox News Radio<\/a>\u00a0to talk about the shadow bans Twitter allegedly enacted against conservatives. \u201cThe net of this is we need to do a much better job at explaining how our algorithms work. Ideally opening them up so that people can actually see how they work,\u201d he said. \u201cThis is not easy for anyone to do. In fact there\u2019s a whole field of research in AI called \u2018explainability\u2019 that is trying to understand how to make algorithms explain how they make decisions in this criteria.\u201d Dorsey said Twitter is \u201csubscribed to that research,\u201d saying the company is helping fund and lead the charge in this new field.<\/p>\n Most simply put, Explainable AI (also referred to as XAI) are artificial intelligence systems whose actions humans can understand. Historically, the most common approach to AI is the\u00a0\u201cblack box\u201d<\/a>\u00a0line of thinking: human input goes in, AI-made action comes out, and what happens in between can be studied, but never totally or accurately explained. Explainable AI might not be necessary for, say, understanding why Netflix or Amazon recommended that movie or that desk organizer for you (personally interesting, sure, but not necessary). But when it comes to deciphering answers about AI in fields like\u00a0health care<\/a>,\u00a0personal finances<\/a>, or the\u00a0justice system<\/a>, it becomes more important to understand an algorithm\u2019s actions.<\/p>\n I contacted\u00a0x.ai<\/a>, a company that makes AI-powered assistants, to ask about Explainable AI. CEO Dennis Mortensen says that while explainability and \u201cthe black box\u201d are at opposite ends of the algorithm spectrum, there is a sliding scale that exists between the two. That said, he believes that even just beginning to ask \u201cwhy?\u201d is important and overdue. \u201cHand over heart, the whole industry hasn\u2019t spent much time if any on the \u2018why\u2019 part, even when it\u2019s technically feasible,\u201d says Mortensen.<\/p>\n Why now then? For one, it\u2019s the age of the algorithm. \u201cWe\u2019ve seen an explosion of the use of algorithms in decision-making,\u201d says Mortensen. Where most important decisions\u2014like who gets a home loan or what a prison sentence will be\u2014were once made by humans or humans assisted by machines, now many are entirely algorithm-based. \u201cI think we always had some sort in inherent belief of humans at least trying to do the right thing,\u201d he says. People aren\u2019t ready to inherently trust a machine the same way. Which brings up the more urgent reason Explainable AI has become a pressing issue: Consumers are witnessing algorithms backfire. Facebook\u2019s fight against misinformation and Twitter\u2019s war on bots are only two of the recent instances in which algorithms spectacularly failed. Until very recently, Mortensen says, it was acceptable for a technology company to defend a failure by pointing out that it wasn\u2019t caused by a person or a team, but by an algorithm: \u201cSo it\u2019s not that your employees are evil, it\u2019s the machine that\u2019s evil.\u201d That doesn\u2019t fly anymore. \u201cThese companies now understand it\u2019s not an excuse and if anything, it might even be worse,\u201d he says. \u201cIt doesn\u2019t matter if a person does it or a machine does it, you need to be able to explain it.\u201d<\/p>\n One very important element of Explainable AI is determining who the algorithm\u2019s decisions are explainable to. There is no collective answer, but Mortensen believes it should be the average user. \u201cIt\u2019s all good and fine that developers and data scientists understand it, but people should understand it,\u201d he says. He points out that while General Data Protection Regulation (an EU regulation on individual data protection) primarily focuses on privacy, it also\u00a0requires increased algorithmic transparency<\/a>. So what do those explanations look like? In some cases, it\u2019s something as simple as a right-click option that opens a window containing a few sentences explaining why a user is being presented with certain information. In a health care scenario, a doctor would want a detailed, thorough analysis of why a machine suggested a diagnosis.<\/p>\n It seems obvious why developers would opt for an XAI system. The problem is that by their nature, XAIs have limitations. \u201cThe more complex a system is, the less explainable it will be,\u201d artificial intelligence researcher John Zerilli told me via email. \u201cIf you want your system to be explainable, you\u2019re going to have to make do with a simpler system that isn\u2019t as powerful or accurate.\u201d<\/p>\n Zerilli is part of a group of researchers from the University of Otago who recently published a paper about Explainable AI. Titled\u00a0\u201cTransparency in Algorithmic and Human Decision-Making: Is There a Double Standard?,\u201d<\/a>\u00a0the paper asks whether it\u2019s fair to require algorithms to explain themselves when humans have trouble doing so. Setting up some sort of regulation for explainability in AI \u201ccould end up setting the bar higher than is necessary or indeed helpful.\u201d (The other serious limitation is that exposing a company\u2019s AI could also expose its proprietary technology.) Instead of changing the entire architecture of the internet\u2019s algorithms, the group argues for using intentional stance explanations, which means giving consumers more information about the data points used by an algorithm in coming to a decision.<\/p>\n While the group wants to dissuade regulators from overtasking AI systems with explaining themselves, they do believe social platforms will benefit from being more transparent about their algorithms. Alistair Knott, another of the paper\u2019s writers, pointed out that Google and Facebook are beginning to do this with the\u00a0\u201cWhy am I seeing this ad?\u201d explanations<\/a>. However, Knott pointed me to\u00a0new research<\/a>\u00a0that shows those explanations might have unexpected effects. \u201c[It] suggests that they \u2018backfire\u2019 as tools to gain user acceptance if the data supporting the ad choice appear to come from some distant unknown source,\u201d Knott wrote. When these explanations are either vague about where or how they gathered data or there isn\u2019t enough information about where it even came from in the first place, users are more likely to distrust it. \u201cA perfectly targeted ad can be rendered ineffective if unsavory practices that underlie it are exposed,\u201d the paper concludes.<\/p>\n