Computer Says No: Part 1 Algorithmic Bias and Part 2 Explainability

These two (of a planned three) posts take an interesting approach to the ethical problems of algorithmic decision making, resulting in a much more optimistic view than most who write on this. It’s very much worth reading even though the arguments don’t seem quite as strong as they are made to appear.

Part 1 essentially side steps the problem of bias in decision making by asserting that automated decision systems don’t actually make decisions (humans still mostly do that), but should instead be thought of as prediction systems – and the test of a prediction system is in the quality of its predictions, not in the operations of its black box. The human dimension is a bit of a red herring as it’s not hard to think of examples where in practice the prediction outputs are all the decision maker has to go on, even if in theory the system is advisory. More subtly, there is an assumption that prediction quality can easily be assessed and an assertion that machine predictions can be made independent of the biases of those who create them, both of which are harder problems than the post implies.

The second post goes on to address explainability, with the core argument being that it is a red herring (an argument Ed Felten has developed more systematically): we don’t really care whether a decision can be explained, we care whether it can be justified, and the source of justification is in its predictive power, not in the detail of its generation. There are two very different problems with that. One is that not all individual decisions are testable in that way: if I am turned down for a mortgage, it’s hard to falsify the prediction that I wouldn’t have kept up the payments. The second is that the thing in need of explanation may be different for AI decisions from that for human decisions. The recent killing of a pedestrian by an autonomous Uber car illustrates the point: it is alarming precisely because it is inexplicable (or at least so far unexplained), but whatever went wrong, it seems most unlikely that a generally low propensity to kill people will be thought sufficiently reassuring.

None of that should be taken as a reason for not reading these posts. Quite the opposite: the different perspective is a good challenge to the emerging conventional wisdom on this and is well worth reflecting on.

Data as photography

Ansel Adams, adapted by Wesley Goatley

“A visualisation is usually looked at – seldom looked into.” – Ansel Adams “The sheer ease with which we can produce a superficial visualisation often leads to creative disaster.” – Ansel Adams “There's nothing worse than a sharp visualisation of a fuzzy concept.” – Ansel Adams “You don't collect a data set, you make it.” – Ansel Adams “There are always two people in every data visualisation: the creator and the viewer.” – Ansel Adams “To make art with data truthfully and effectively is to see beneath the surfaces.” – Ansel Adams “A great data visualisation is a full expression of what one feels about what is being visualised in the deepest sense, and is, thereby, a true expression of what one feels about life in its entirety.” – Ansel Adams “Data visualisation is more than a medium for factual communication of ideas. It is a creative art.” – Ansel Adams “We must remember that a data set can hold just as much as we put into it, and no one has ever approached the full possibilities of the medium.” – Ansel Adams “Data art, as a powerful medium...offers an infinite variety of perception, interpretation and execution.” – Ansel Adams “Twelve significant data points in any one year is a good crop.” – Ansel Adams

The idea that the camera does not lie is as old as photography. It has been untrue for just as long.

The exposure of film or sensor to light may be an objective process, but everything which happens before and after that is malleable and uncertain. There are some interesting parallels with data in that: the same appearance – and assertion – of accurately representing the real world, the same issues of both deliberate and unwitting distortion.

This tweet simply takes some of the things Ansel Adams, the great photographer of American landscapes, has written about photography and adapts them to be about data. It’s neatly done and provides good food for thought.

Don’t believe the hype about AI in business

Vivek Wadhwa – VentureBeat

If you want to know why artificial intelligence is like teenage sex, this is the post to read. After opening with that arresting comparison, the article goes on to make a couple of simple but important points. Most real world activities are not games with pre-defined rules and spaces. And for businesses – and arguably still more so for governments – it is critically important to be able to explain and account for decisions and outcomes. More pragmatically, it also argues that competitive advantage in the deployment of AI goes to those who can integrate many sets of disparate data to form a coherent set to which AI can be applied. Most companies – and, again, perhaps even more so most governments – are not very good at that. That might be the biggest challenge of all.

YouTube, the Great Radicalizer

Zeynep Tufekci – New York Times

This article has been getting extensive and well-deserved coverage over the last few days. Essentially, it is demonstrating that the YouTube recommendation engine tends to lead to more extreme material, more or less whatever your starting point. In  short, “YouTube leads viewers down a rabbit hole of extremism, while Google racks up the ad sales.”

The reason for including it here is not because of the specific algorithm or the specific behaviour it generates. It is because it’s a very clear example of a wider phenomenon. It’s a pretty safe assumption that the observed behaviour is not the result of a cabal of fringe conspirators deep in the secret basements of Google setting out a trail to recruit people into extremist groups or attitudes. The pretty obvious motivation is that what they are actually trying to do is to tempt people into spending as long as possible watching YouTube videos, because that’s the way they can put most advertising in front of most eyeballs.

In other words, algorithmic tools can have radically unintended consequences. That’s made worse in this case because the unintended consequences are not a sign of the intended goal not being achieved; on the contrary, they are the very means by which that intended goal is being achieved. So it is not just the case that YouTube has some strong incentives not to fix the problem, the problem may not be obvious to them in the first place.

This is a clear example. But we need to keep asking the same questions about other systems: what are the second order effects, will we recognise them when we see them, and will we be ready to – and able to – address them?

A roadmap for AI: 10 ways governments will change (and what they risk getting wrong)

Geoff Mulgan – NESTA

This is a great summary of where AI stands in the hype cycle. Its focus is the application to government, but most of it is more generally relevant. It’s really helpful in drawing out what ought to be the obvious point that AI is not one thing and that it therefore doesn’t have a single state of development maturity.

The last of the list of ten is perhaps the most interesting. Using AI to apply more or less current rules in more or less current contexts and systems is one thing (and is a powerful driver of change in its own right). But the longer term opportunity is to change the nature of the game. That could be a black box dystopia, but it could instead be an opportunity to break away from incremental change and find more radical opportunities to change the system. But that depends, as this post rightly concludes, on not getting distracted by the technology as a goal in its own right, but focusing instead on what better government might look like.

UK police are using AI to make custodial decisions – but it could be discriminating against the poor

Matt Burgess – Wired

In abstract, AI is a transformational technology. It may bring perfect and rigorous decision analysis, sweeping away human foibles. Or it may displace human sensitivity and judgement – and indeed the humans themselves – and usher in an era of opaque and arbitrary decision making.

This article, which focuses on the introduction of AI to Durham Constabulary, is a good antidote to those caricature extremes. Reality is, as ever, messier than that. Predictability and accountability are not straightforward. Humans tend to revert, perhaps unwisely, to confidence in their own judgements. It is not clear that some kinds of data are appropriately used in prediction models at all (though the black boxes of human brains are equally problematic). In short, the application of AI to policing decisions isn’t simple and clear cut, it is instead a confused and uncertain set of policy problems. That shouldn’t be surprising.

How AI will transform the Digital Workplace (and how it already is)

Sharon O’Dea – Intranetizen

AI is often written about in terms of sweeping changes resulting in the wholesale automation of tasks and jobs. But as this post sets out, there is also a lower key version, where forms of AI appear as feature enhancements (and thus may not be apparent at all). Perhaps self-generating to do lists are the real future – though whether that will be experienced as liberation or enslavement is very much a matter of taste. Either way, AI won’t be experienced as robots, breaking into the building to take our jobs; instead tasks will melt away, enhanced in ways which never quite feel revolutionary.

10 Principles for Public Sector use of Algorithmic Decision Making

Eddie Copeland – NESTA

This is a really interesting attempt to set out a a set of regulatory principles for the use of algorithms in the public sector. It brings what can easily be quite an abstract debate down to earth: we can talk about open data and open decision making, but what actually needs to be open (and to whom) to make that real?

The suggested principles mostly look like a sensible starting point for debate. Two of them though seem a little problematic, one trivially, the other much more significantly. The trivial one is principle 9, that public sector organisations should insure against errors, which isn’t really a principle at all, though the provision of compensation might be. The important one is principle 5, “Citizens must be informed when their treatment has been informed wholly or in part by an algorithm”. On the face of it, that’s innocuous and reasonable. Arguably though, it’s the equivalent of having a man with a red flag walking in front of a car. Government decisions are already either based on algorithms (often called “laws” or “regulations”) or they are based on human judgements, likely to be more opaque than any computer algorithm. Citizens should absolutely be entitled to an explanation and a justification for any decision affecting them – but the means by which the decision at issue was made should have no bearing on that right.

Will We Own AIs, or Will They Own Us?

Tim Weber – Techonomy

Neither, seems to be the answer to the question posed by the title. But if one thing AIs do is filter the world for us, the question of who does the filtering and in whose interest they do it becomes very important. As with other free services, free algorithms will be provided in expectation of a benefit to somebody, and that somebody may very well not be the end user. So far so unexceptional (and putting it under the heading of AI doesn’t change the substance of an issue which has been around a good while). But if this is a problem, what are the pressures and processes which will work to relieve it rather than reinforce it? Here, the argument rather fades away: we are told we need clear laws and well-accepted procedures to regulate AI, but there is little suggestion here about what they would say or how we would get to them. It’s slightly unfair to single this piece out for what is quite a common problem: when challenges are technology driven, but solutions need to be socially driven, it’s a lot easier to talk about the first than the second.

AI in a week

Laura Caccia – Oxford Insights

This post is just a teaser, but a teaser for something interesting in both style and substance. Starting next Monday, we are promised a five day intensive course on artificial intelligence. AI is for many people at the stage where there is lots of fragmented insight and understanding, but little which brings the fragments together to form a coherent whole. So there is a gap here well worth filling – and this looks to be a neat low key way of filling it. Watch that space.

The black box is a state of mind

Kathrin Passig – Eurozine

The idea of the black box pervades a lot of thinking and writing about AI. Mysterious algorithms do inscrutable things which impinge on people’s lives in inexplicable ways. That is alarming in its own right, but doubly so because this is new and uncharted territory. Except that, as this post painstakingly points out, it’s not actually new at all. People have been writing software about which they could not predict the outputs from the inputs since pretty much since they have been writing software at all – in a sense, that’s precisely the point of it. And if you want to look at it that way, the ultimate black box is the human brain, where the evidence that we don’t understand the reasons for our own decisions, never mind anybody else’s, is pretty overwhelming.

The need for precision at one level – software doesn’t cope well with typos and syntax errors – doesn’t translate into precision at a higher level, of understanding what that precisely written software will actually do. That thought came from Marvin Minsky in 1967, but people had been writing about black boxes for years before that, when the complexity of software was a tiny fraction of what is normal now.

The fact that this is neither new nor newly recognised doesn’t in itself change the nature of the challenge. What it does perhaps suggest, though, is that strategies developed for coping with these uncertainties in the past may well still be relevant for the future.

The Latest Data Privacy Debacle

Zeynep Tufekci – The New York Times

Issues of data aggregation and de-anonymisation are hardly new, but there’s nothing like a good example to make an issue more visible – and secret US bases revealed through aggregated data from fitness trackers are about as good as it gets.

The real issue though is less such revelations and more the implications for data and privacy more generally. This article argues powerfully that to see this as an issue of individuals and clickthrough privacy policies is to miss a very important point. People can’t consent to the ways their personal data will be used and the risks that carries, because service providers don’t and can’t understand those things themselves, and so can’t explain them in a way which makes consent meaningful. That has some important data policy implications, including much stronger liability for data breaches, and keeping the amount of data captured and held to a minimum in the first place. Those are not new suggestions, of course, so as ever the real question is not how the risks could be managed better, but how incentives can be aligned to ensure that the risks are in fact managed. And that is a political and social problem, not a technical one.

Artificial Intelligence is a Horseless Carriage

Terence Eden

A whimsical twitter thread of etymological onion peeling, now crystallised into a blog post, results in a splendid definition of AI. Starting with ‘complicated algorithms running on very fast computers’, we end up with AI helpfully described as

The method by which an old Persian magician uses counting stones, to move other stones, by way of amber resin, such that a casual observer thinks the stones are moving themselves.

Personal Data Representatives: An Idea

Tom Steinberg – Medium

In dealing with digital services – indeed in dealing with organisations generally – power is very asymmetric. Amazon does not invite you to negotiate the terms on which it is prepared to sell you things (though of course you retain the power not to buy). Digital services and apps give the illusion of control (let’s think about whether to accept these cookies…) but have developed a habit and a reputation or helping themselves to data and making their own judgement about what to do with it. That’s not necessarily because individual consumers can’t control permissions, but is also because the cost and complexity of doing so make it burdensome. Tom Steinberg brings a potential solution to that problem: what if we had somebody negotiating all that on our behalf, could that asymmetry be addressed? Typically, he recognises the difficulties as well as the potential, but even if the answers are hard, the question is important.

How to Build Self-Conscious Artificial Intelligence

Hugh Howey – Wired

The title of this article is a bit of a false flag, since it could easily have continued ‘… and why that would be a really bad idea’. It is though an interesting – though considerably longer – complement to the argument that the idea of general artificial intelligence is based on a false analogy between human brains and computers. This article takes the related but distinct approach that self-consciousness exists as compensation for structural faults in human brains, particularly driven by the fact that having a sophisticated theory of mind is a useful evolutionary trait and that it would be pointless (rather than impossible) to replicate that – because perhaps the most notable thing about human introspection about consciousness is how riddled it is with error and self-contradiction. That being so, AI will continue to get more powerful and sophisticated. But it won’t become more human, because it makes no sense to make it so.

204 examples of artificial intelligence in action

Chris Yiu

This is a page of links which provides over two hundred examples of artificial intelligence in action – ranging from mowing the lawn, through managing hedge funds and sorting cucumbers all the way to writing AI software. Without clicking a single one of the links, it provides a powerful visual indicator of how pervasive AI has already become. There is inevitably a bit of a sense of never mind the quality, feel the width – but the width is itself impressive, and the quality is often racing up as well.

There is a linked twitter account which retweets AI-related material – though in a pleasing inversion, it shows every sign of being human-curated.

Ethics and ethicists

Ellen Broad

This is a short tweet thread making the point that ethics in AI – and in technology generally – needs to be informed by ethical thinking developed in other contexts (and over several millenia). That should be so obvious as to be hardly worth saying, but it has often become one of those questions which people doing new things fall into the trap of believing themselves to be solving for the first time.

Why the robot boost is yet to arrive

Tim Harford – the Undercover Economist

One of the problems with predicting the future is working out when it’s going to happen. That’s not quite as silly as it sounds: there is an easy assumption that the impact of change follows closely on the change itself, but that assumption is often wrong. That in turn can lead to the equally wrong assumption that because there has been limited impact in the short term, the impact will be equally limited in the long term. As Robert Solow famously put it in 1987, ‘You can see the computer age everywhere but in the productivity statistics.’ In this post, Tim Harford updates the thought from computers to robots. The robot takeover isn’t obviously consistent with high employment and low productivity growth, but that is what we can currently observe. The conclusion – and the resolution of the paradox is disarmingly simple, if rather frustrating: wait and see.

Don’t believe the hype: work, robots, history

Michael Weatherburn – Resolution Foundation

This post introduces a longer paper which takes the idea of understanding the future by reflecting on the past to a new level. The central argument is that digital technologies have been influencing and shaping the industry sectors it examines for a long time, and that that experience strongly suggests that the more dramatic current forecasts about the impact of technology on work are overblown.

The paper’s strengths come from its historical perspective – and, unusually for this topic, from being written by a historian. It is very good on the underlying trends driving changing patterns of work and service delivery and distinguishing them from the visible emanations of them in web services. It does though sweep a lot of things together under the general heading of ‘the internet’ in a way which doesn’t always add to understanding – the transformation of global logistics driven by ERP systems is very different from the creation of the gig economy in both cause and effect.

The paper is less good in providing strong enough support for its main conclusion to justifying making it the report’s title. It is true that the impacts of previous technology-driven disruptions have been slower and less dramatic to manifest themselves than contemporary hype expected. But the fact that hype is premature does not indicate that the underlying change is insubstantial – the railway mania of the 1840s was not a sign that the impact of railways had peaked. It is also worth considering seriously whether this time it’s different – not because it necessarily is, but because the fact that it hasn’t been in the past is a reason to be cautious, not a reason to be dismissive.

What we talk about when we talk about fair AI

Fionntán O’Donnell – BBC News Labs

Courtesy of Laura Amaya from the Noun Project

This is an exceptionally good non-technical overview of fairness, accountability and transparency in AI. Each issue in turn is systematically disassembled and examined.  It is particularly strong on accountability, bringing out clearly that it can only rest on human agency and social and legal context. ‘My algorithm made me do it’ has roughly the same moral and intellectual depth as ‘a big boy made me do it’.

I have one minor, but not unimportant, quibble about the section on fairness. The first item on the suggested checklist is ‘Does the system fit within the company’s ethics?’ That is altogether too narrow a formulation, both in principle and in practice. It’s wrong in practice because there is no particular reason to suppose that a company’s (or any other organisation’s) ethics can be relied on to impose any meaningful standards. But it’s also wrong in principle: the relevant scope of ethical standards is not the producers of an algorithm, but the much larger set of people who use it or have it applied to them.

But that’s a detail. Overall, the combination of clear thinking and practical application makes this well worth reading.