This is the video of a conference talk by Ed Felten, which is fascinating for a number of reason. He has been thinking hard about technology and the policy consequences of technology for a very long time, and doing so with deep technical expertise (on the explicability of algorithms, to take just one example).
But he also has been at the heart of the intersection of technology and public policy – a one man One Team Government – including a couple of years in the Obama White House. This talk is primarily about how machine learning lands in a public policy context and is immediately addressed to an audience at a big AI conference, whose perspective can be assumed to be technical.
Given that, the starting point is to underline a critical difference in perspective. At least in principle, science and engineering are about a search for truth. Democracy is not just not a search for truth, it is not really a search for anything. And that difference is simultaneously obvious, a strength and a source of deep confusion and misunderstanding
Democracy is not a search for truth; it is an algorithm for resolving disagreements
But this talk is interesting not just to an audience of technologist having the world of public policy explained by one of their own who has ventured into a strange and distant land. Given the importance of AI and machine learning – and indeed technology change more generally – to almost every aspect of policy, it is jut as important for policy makers and players in the democratic process to understand how their world is perceived. And from that perspective, this is a fascinating account of a strange world by a participant-observer who has retained his distance and brings a distinct professional perspective.
There’s no getting away from the fact that parliamentary procedure is pretty arcane and that modelling that procedure adds a still more arcane overlay. But this is a beautifully reflective post which wears deep expertise very lightly to share thinking which is relevant well beyond the immediate parliamentary context.
Two points which should resonate far beyond the Palace of Westminster are worth pulling out. One is that parliamentary processes may have some extreme characteristics, but they also have some characteristics which people involved with other kinds of information flows will instantly recognise. It may or may not be possible to express definitively how the system should work; for different reasons it may or may not be possible to capture in detail how it does work, particularly if that is in some circumstances indeterminate. But taking an almost anthropological approach to understanding systems is both an art form and an investment which needs to be made.
The second is that for all the power of starting with user needs, that is necessarily limited if some kinds of needs come into being only as a result of building a system which satisfies them. In a nice nod to George Box, the post ends with a bold claim for the art of system modelling:
The models are only ever maps, but if they’re good enough to be useful they can be useful in ways the map designers never considered. No amount of requirements gathering or user research will ever compensate for omitting the work on modelling, because user needs are emergent from use and emergent from materials.
Jerry Fishenden – ntouk
Don’t be misled by the title, this isn’t really a post about payday loans. Instead, it explores the fascinating contrast between the approaches HMRC (for tax) and DWP (for benefits) have taken to opening their services to third parties. The basic story is pretty simple: HMRC has a long pre-internet history of working with third party intermediaries which it carried forward into its thinking abuot online services (at one stage their ambition was not directly to offer an online tax return service at all); DWP’s history is much more about direct delivery, and that tradition similarly has been carried forward into the online world. The post makes no pretence to neutrality on the central question of which was the better choice, HMRC is clearly seen to have won that argument hands down.
The post is good on the advantages of the open method and the opportunities that could create (including the ethical payday loans of the title). But it doesn’t address the fairly central question of whether there is a reason for the difference. After all, HMRC’s administration of tax credits, which are a benefit in everything but name, didn’t get the same open treatment as their revenue-raising lines of business. The question of whether a version of HMRC’s trust model for taxpayers and their agents could be translated to the benefits system is one well worth further reflection.
Catherine Howe – Curious?
An odd thing about many large organisations is that change is seen as different from something called business as usual. That might make a kind of sense if change were an anomalous state, quickly reverting to the normality of stasis, but since it isn’t, it doesn’t.
If change is recognised as an essential element of business as usual, then lots of other ideas drop easily into place. One of the more important ones is that it allows and encourages better metaphors. The idea of change as something discrete which starts and stops, which has beginnings and ends, encourages mechanical parallels: like a machine, it can be turned on and off; like a machine, controlling the inputs will control the outputs. But if change permeates, if organisations and their environments are continually flexing, then metaphors naturally become more organic: the pace of change ebbs and flows; organisations adapt as a function of their place in a wider ecosystem; change is just part of what happens, not some special extra thing.
From that perspective, it’s a small step to recognising that there is real power in thinking about organisational change in terms of systems. But it’s a small step with big consequences, and those consequences are what this post is all about.
The world of system change provides a different framing of organisational change and a way of seeing it as part of an organic process and not something that is bolted onto an organisation. The simple but powerful shift from process to purpose is something that can make a profound difference to how you go about engaging the networks that already exist within your organisation. Once we acknowledge and bring to fore the networks that make up our organisations and the system they create can we ever really deny that all change is system change?
This is a short, perhaps even slightly cryptic, note on the purpose of organisations. Having had the unfair advantage of being part of the conversation which prompted it, my sense is that it captures two related, but distinct, issues.
The first is that not everything has a purpose at all, in any terribly useful or meaningful sense. We can observe and describe what elements of a system do, but that does not mean that each such element has a purpose, still less that any purpose it might have relates to the behaviour of the wider system of which it is part. Not being careful here can lead to spectacular errors of reverse causation – the purpose of noses is not, as Pangloss argued, to support the wearing of spectacles.
The second is that it is easy to look at human-made systems and assume that they have a purpose, and that that purpose can be both discerned and – should we wish it – amended. That’s an understandable hope, but not necessarily a realistic one. Organisations of any size are both complex systems in their own right and components of larger and yet more complex systems. What they do and how they do it cannot be reduced to a single simple proposition. That’s not, I take it, a nihilistic argument against trying to understand or influence; it is a recognition that we need to recognise and respect complexity, not wish it away.
Abbe Marks – NZ Digital government
The idea that it should be possible to capture legislative rules as code and that good things might result from doing so is not a new one. It sounds as though it should be simple: the re-expression of what has already been captured in one structured language in another. It turns out though not to be at all simple, partly because of what John Sheridan calls the intertwingling of law: the idea that law often takes effect through reference and amendment and that the precise effect of its doing so can be hard to discern.
There is interesting work going on in New Zealand experimenting with the idea of law and code in some limited domains, and this post is prompted by that work. What makes it distinctive is that it is written from a policy perspective, asking questions such as whether the discipline of producing machine consumable rules is a route to better policy development. It’s still unclear how far this approach might take us – but the developments in New Zealand are definitely worth keeping an eye on.
Alix Dunn – Medium
One of the unavoidable problems in curating a site like Strategic Reading is that lots of the posts and articles end up slightly blurring into one another. That’s a good thing in many ways: ideas build on each other, views of the world coalesce, but it can sometimes feel as though there isn’t much really new thinking.
This post is different. It is deliberately disruptive and challenging and provides some useful insights into a problem which has existed for a long time, but has been largely overlooked. What counts as the right kind of knowledge to understand and use technology effectively? It isn’t in itself technical knowledge – telling everybody to learn to code is no more effective than addressing transport management problems through the medium of car maintenance classes. And it isn’t stepping away, leaving such issues to the priesthood of the initiated. The limitations of that model are increasingly obvious in a world where big companies refuse to acknowledge or understand the sociology of technology.
The answer suggested here is something called ‘technical intuition’ (which is a slightly odd label, since it’s about being knowledgeable rather than intuitive), which allows people who are not technically expert to imagine, to inquire, to decide and to demand. That’s then brought to life in an example about an individual deciding whether to sign up for a supermarket loyalty card. That’s fair enough in its own terms. Personal understanding of the implications of technology related decisions is really important (and closely related to what Rachel Coldicutt calls digital understanding).
But that leaves us with two essential questions, which are left implied but not addressed. The first is where this intuition is to come from. If it is a body of knowledge, how is it to be assembled, conveyed and absorbed – all of which are preconditions to its being acted on. The second is how it then scales and aggregates – both in terms of where the social (as opposed to individual) acceptability of loyalty cards comes from, and, very differently, how that leads to confidence in other kinds of decision making. What is the technical intuition we should expect supermarket executives to display in designing loyalty cards in the first place? And in some ways, that is the most important question of all.
It is not every post you will read which links the choice of web domain name to the results of the civil war, but it is characteristic of this one that that’s exactly what it does. If you are interested in the arcane minutiae of parliamentary structure, this post is for you. But behind the specific points, there is something much more generally significant which should interest everybody, including those who, inexplicably, are not fascinated by parliamentary minutiae.
Computers crystallise systems. That’s fine for as long as the crystallised form remains valuable – and sometimes that can be quite a long time. But it’s not at all fine for systems which need to retain flexibility and adapt to changing circumstances – and that’s quite a lot of systems.
The lack of a fixed, exhaustive ruleset means Parliament is open to exaptation and adaptation. It is evolutionary by design. It is not brittle. It can sway in winds. Computers on the other hand are really not like this. They tend to prefer defined rulesets. They are deterministic. They are dumb. They are brittle.
So the question becomes whether we can get brittle computers to support systems which are not brittle. And that’s a question which matters much more widely than just for parliament.
Eddie Copeland – NESTA
This simple and powerful set of slides does an extraordinarily good job of summarising the key issues in digital transformation, not least in being really clear about the extent to which all of this is a technology issue (not as much as it looks) as opposed to an everything else issue (much more than it first appears). The section on ‘deciding how you want to work’ gets twice as many slides as ‘thinking about your technology needs’, which is a pretty strong indicator of the approach being taken.
It’s certainly possible to challenge some of the details. The arresting assertion that ‘we can broadly take for granted that technology can do whatever we want it to do’ perhaps has more power than precision – though the slightly lesser claim the technology needed to support government processes already exists is indeed a useful reminder that appeals for technological exceptionalism are very likely to be misguided. The insistence that agile projects can’t succeed in organisations which retain traditional approaches to funding and governance is both wrong and unhelpful: wrong because there are plenty of example of where it has succeeded, and unhelpful because every organisation has to start somewhere, and if agile can’t work at all unless everything is agile, there is effectively no way of making that start.
Overall, though, the strengths far outweigh the weaknesses – and this is a beautiful example of doing the hard work to make things simple. As a further bonus, the slides are open for comments, and have already sprouted a rich set of observations from Matthew Cain.
Martin Stewart-Weeks – The Mandarin
If it is hard to think and act systemically about the long term, it’s also worth reflecting on patterns of behaviour which get in the way even of the attempt. The rhetoric of innovation, of openness, of fearless honesty runs into a reality which seems designed to punish and constrain precisely those behaviours. And of course ‘design’ is precisely the wrong word here: these characteristics are emergent rather than intended (which does not, of course, mean that it would be impossible to design them to be different). There are many reasons why that is an unfortunate state of affairs, one which is rightly given some emphasis is that it risks crowding out the strategic and the systemic:
The real dilemma is that we’re so busy honing the efficiency of the pieces that we’ve failed to work out how to put the puzzle together or work out what the puzzle is or should be.
Pia Waugh – Pipka
This is a characteristically excellent post, examining in some detail both what it takes for change to succeed and, perhaps even more importantly, how to scale it.
The short answer is that if you want to change the system, you have to change the system. And to do that on the fifty plus year scale which is the level of ambition behind this post, requires rigour and discipline. Five questions are set out, including the two which are the most critical: what future do you want? And what are you going to do today?
Scaling from an idea of the future to systematic government and national level change can’t be done by exhortation – and simple observation suggests can only with the greatest difficulty be done at all. The recommendations here are an intriguing mixture of the very slow burn (supporting long term varied career development, to reduce aversion to new thinking) to the much more immediate (mandating the use of user research in funding bids).
All that still leaves the question of how best to start this whole process, but this is a manifesto of what should be done, or rather how it should be done; it doesn’t purport to be a set of instructions for making it happen.
Gavin Starks – NESTA
This is an interesting report which asks almost the right question. Government is at little risk of losing its mind, or its short term memory. The two better questions – which in practice are the ones this report starts to address – are whether government can stop losing its longer term memory, and how the power of the government’s mind can be enhanced by better ways of connecting and amplifying its component parts.
Those are important questions. It’s already all too easy to forget how long we have been worrying about the ease of forgetting things. Aspects of the problem have been recognised, and solutions attempted, since the earliest days of what we no longer call electronic government. None has been more than partially successful.
The two questions are also closely related. People are reluctant to incur even the smallest overhead in storing things for the benefit of posterity, so the benefit needs to come from improving the effectiveness of current work. Conversely, tools which facilitate collaboration, sharing and serendipity will often (though with some very important exceptions) also be tools which facilitate the storage and discovery of what has been created. That was one of the key themes of a series of blog posts I wrote a couple of years ago, which covered some (though by no means all) of the same ground – including the observation, echoed in this report, that the web was invented to be an intranet and knowledge management tool; the world wide bit came rather later.
Where this report adds to the debate is in its more explicit recognition not just that we need to be thinking about texts rather than documents, but that a lot of what we need to be thinking about isn’t conventional text in the first place, making the paginated document an even less useful starting point for thinking about all this.
And there is a delicious irony that this blog – and my blogging generally – exists in large part to serve as my outboard memory, now with well over a decade of material created in part as protection against the weaknesses of institutional knowledge preservation.
Beyond even the bonus points for talking about laws being ‘intertwingled’, this is an important and interesting post at the intersection of law, policy and automation. It neatly illustrates why the goal of machine-interpretable legislation,such as the recent work by the New Zealand government, is a much harder challenge than it first appears – law can have tacit external interpretation rules, which means that the highly structured interpretation which is normal, and indeed necessary, for software just doesn’t work. Which is why legal systems have judges and programming languages generally don’t – and why the New Zealand project is so interesting.
Nadia Webster – NZ Digital Government
The rather dry title of this post belies the importance and interest of its content. Lots of people have spotted that laws are systems of rules, computer code is systems of rules and that somehow these two fact should illuminate each other. Quite how that should happen is much less clear. Ideas have ranged from developing systems to turn law into code to adapting software testing tools to check legislative compliance. This post records an experiment with a different approach again, exploring the possibility of creating legislative rules in a way which is designed to make them machine consumable. That’s an approach with some really interesting possibilities, but also some very deep challenges. As John Sheridan has put it, law is deeply intertwingled: the meaning of legislation is only partly conveyed by the words of a specific measure, which means that transcoding the literal letter of the law will never be enough. And beyond that again, the process of delivering and experiencing a service based on a particular set of legal rules will include a whole set of rules and norms which are not themselves captured in law.
That makes it sensible to start, as the work by the New Zealand government reported here has done, with exploratory thinking, rather than jumping too quickly to assumptions about the best approach. The recommendations for areas to investigate further set out in their full report are an excellent set of questions, which will be of interest to governments round the world.
Jennifer Guay – Apolitical
To describe something as ‘policy based evidence making’ is to be deliberately rude at two levels, first because it implies the use of evidence to conceal rather than illuminate, but secondly because it implies a failure to recognise that evidence should drive policy (and thus, though often less explicitly, politics).
Evidence based policy, on the other hand, is a thing of virtue for which we should all be striving. That much is obvious to right-thinking people. In recent times, the generality of that thought has been reinforced by very specific approaches. If you haven’t tested your approach through randomised controlled trials, how can you know that your policy making has reached the necessary level of objective rigour?
This post is a thoughtful critique of that position. At one level, the argument is that RCTs tell you less than they might first appear to. At another level, that fact is a symptom of a wider problem, that human life is messy and multivariate and that optimising a current approach may at best get you to a local maximum. That is of course why the social sciences are so much harder than the so-called hard sciences, but that is probably a battle for another day.
Paul Clarke – honestlyreal
On a morning where transport is disrupted across the UK by snow and cold winds, it’s worth returning to this post from a few years ago, which explains why small amounts of snow here are so much more disruptive than the much larger amounts which are easily managed elsewhere. In short, the marginal cost of being ready for severe weather, when there isn’t very much of it, isn’t justified by the benefits from another day or two a year of smooth operations. That is a very sensible trade off – the existence of which is immediately forgotten when the bad weather arrives.
It’s a trade off with much wider application than snow-covered railway tracks. Once you start looking, it can be seen in almost every area of public policy, culminating in the macro view that everybody (it is asserted) wants both lower taxes and better services. Being more efficient is the way of closing the gap which is simultaneously both clearly the right thing to do and an excellent way of ducking the question, but at best shifts the parameters without fundamentally changing the nature of the problem. Hypothecation is a related sleight of hand – let’s have more money, but only for virtuous things. In the end, though, public policy is about making choices. And letting the trains freeze up from time to time is a better one than it appears in the moment to the people whose trains have failed to come.
The Yorkshire Ranter
This is a follow up to a post covered here a few days ago which looked critically at outsourcing, starting from the fundamental question first posed by Coase on what organisations should do and what they should buy. This second post is at one level a short summary of the first one, but it’s also rather more than that. It puts forward a slightly different way of framing the question, making the point that time and uncertainty are relevant to the decision, as well as pure transaction cost narrowly defined.
There are transactions, which are in the moment, and imply no further commitment or relationship. There are contracts, which are a commitment to future transactions, and depend on shared assumptions about the future conditions in which those transactions will happen. And there are organisations, which exist in the space beyond contractual precision and certainty.
To complete the hat trick, there is also a separate post applying this thinking to Capita. Even for those less interested in the company, it’s worth reading to the end to get to the punch in the punchline:
In important ways, this is the service that Capita provided and still provides: the ability to blame problems on computers and computer people, while ignoring the physical reality of policy
The Yorkshire Ranter
Organisations, including governments, follow fashions. Some of those fashions change on short cycles, others move more slowly, sometimes creating the illusion of permanence. The fashion for outsourcing, for buying rather than making, has been in place in government for many years, but there are some interesting signs that change may be coming. One immediate cause and signal of that change is the collapse of Carillion, but that happened at point when the debate was already beginning to change.
This post goes back to the roots of the make or buy choice in the work of Ronald Coase on the nature of the firm. The principle is simple enough, that it makes sense to buy things when the overhead of creating and managing contracts is low and to make them when the overheads are high. The mistake, it is argued here, is that organisations, particularly governments, have systematically misunderstood the cost and complexity of contract management, resulting in the creation of large businesses and networks of businesses whose primary competence is the creation and management of contracts.
One consequence of that is that it becomes difficult or impossible to understand the true level of costs within a contractual system (because prices quickly stop carrying that information) or to understand how the system works (because tacit knowledge is not costed or paid for).
All very thought provoking, and apparently the first in a series of posts. It will be worth looking out for the others.