Goals and outcomes

Analysing goals and potential outcomes

Six separate principles concern the analysis of the goals and potential outcomes we expect from projects to develop and deploy technology:

  • Question desirability
  • Clarify externalities
  • Require peer reviews
  • Involve multiple perspectives
  • Analyse the whole system
  • Anticipate fat tails

Question desirability

The principle of “question desirability” starts with the recognition that, just because we believe we could develop some technology, and even if we have some desires to develop that technology, that’s not a sufficient reason for us actually to go ahead and develop it and deploy it.

After all, once a project starts, with the goal of developing some technology, the project sometimes develops momentum that can carry it forward almost of its own accord:

  • People involved tell themselves and each other that there’s already a commitment to continue the project.
  • They may feel they’re in a race with competitors.
  • They may feel a sense of loyalty to the project.
  • They may feel an obligation to fellow team members, or to bosses, or to others who are assumed to be waiting for the technology.
  • They may desire to ship their product to the world, to show their capabilities.

But the result of this inertia could be outcomes that we later bitterly regret.

Therefore, the principle of “question desirability” urges that we take the time to write down what we assume are the good outcomes we will obtain from the technology to be developed. Once these assumptions have been written down, it allows for a more thoughtful and considered review.

The principle also urges that we consider other methods for achieving these intended outcomes – other methods apart from the particular technology that has somehow caught our fancy.

This separation in analysis of desired outcomes, also known as “requirements”, from possible solutions, is a vital step to avoiding unintended consequences of technologies. We may find better, safer, more reliable ways of achieving the outcomes that we have in mind.

The principle of “question desirability” also recommends that we should in any case challenge assumptions about which outcomes are desirable, and we should be ready to update these assumptions in the light of improved understanding.

Finally, we should also avoid taking for granted that agreement exists on what will count as a good outcome.

That takes us to the next principle, “Clarify externalities”.

Clarify externalities

Recall that an externality is an effect of a project, or the effect of an economic transaction, that is wider than the people directly involved.

Examples of externalities include noises, smells, pollution, resource depletion, cultural chaos, and a general loss of resilience.

The principle of “clarify externalities” draws attention to possible wider impacts (both positive and negative) from the use of products and methods, beyond those initially deemed central to their operation, and seeks to ensure that these externalities are included in cost/benefit calculations.

Therefore we should not just consider metrics such as profit margin, efficiency, time-to-market, and disabling competitors. We need to consider broader measures of human flourishing.

What makes this analysis possible is the effort taken, in line with the “Question desirability” principle, to write down the intended outcomes of the technology to be developed. What makes this analysis more valuable are the principles of “Require peer reviews” and “Involve multiple perspectives”, to which we now turn.

Require peer reviews

The alternative to requiring peer reviews is that we trust the people who are behind a particular project. We may feel they have a good track record in creating technologies and products. Or that they have outstanding talent. Therefore no peer review would be required.

That may be acceptable for projects that are sufficiently similar to those undertaken in the past. However, new technologies have a habit of bringing surprises, especially when used in novel combinations.

That’s why independent peer reviews are required, involving external analysts who are not connected with the initial project team. These analysts should ask hard questions about the assumptions made by the project team.

Involve multiple perspectives

What’s more, this analysis phase, into the desirability of products and methods, in pursuit of the goals that have been stated, should involve people with multiple different skill sets and backgrounds.

This should include expertise in law, economics, and human factors, as well as designers, scientists, and engineers.

To be clear, these independent analysts won’t necessarily have a veto over decisions taken by the project team. However, what is required is that the project team, along with their sponsors, take proper account of questions and concerns raised by independent analysts.

That proper account needs to observe two further principles: analyse the whole system, and anticipate fat fails.

Analyse the whole system

What’s meant by “the whole system” is the full set of things that are connected to the technology that could be developed and deployed – upstream influences, downstream magnifiers, and processes that run in parallel. It also includes human expectations, human beliefs, and human institutions. It includes aspects of the natural environment that might interact with the technology. And, critically, it includes other technological innovations.

When analysing the potential upsides and downsides of using the new technology that we have in our mind, we need to consider possible parallel changes in that wider “whole system”.

For example, rather than simply analysing how a piece of new artificial intelligence might behave in the environment as it exists today, we should consider possible complications if other pieces of new artificial intelligence, including adversarial technology, or new forms of hacking, are introduced into the environment as well.

This analysis might lead to the conclusion that the new technology would, after all, be more dangerous to deploy than was first imagined. Or it could lead to design changes in the new technology, so that it would remain beneficial even if these other alterations in the environment took place.

Anticipate fat tails

The principle of “Anticipate fat tails” urges us to remember that not every statistical distribution follows that of the Normal curve, also known as the Gaussian bell curve.

In many circumstances, once we observe the mean of a set of observations, often denoted by the Greek letter mu, and also the standard deviation of these observations, known as sigma, we can be confident that new measurements more than three standard deviations away from the mean will be unlikely. They’ll be seen only around three times in a thousand. And for a new measurement that is more than six standard deviations away from the mean, you would have to wait on average more than one million years, if a new measurement was made every single day.

However, our initial observations of the data might lead us astray. The conditions for the distribution of results being Normal might not apply. These conditions require that the outcomes are formed from a large number of individual influences which are independent of each other. When, instead, there are connections between these individual influences, the distribution can change to have what are known as “fat tails”: outcomes that are more than six sigma away from the previously observed mean – or even more than twenty sigma away from it – can arise, taking everyone by a horrible surprise.

That possibility would change the analysis from “how might we cope with significant harm”, such as a result three sigma away from the mean, to “could we cope with total ruin”, such as a result that is, say, twenty sigma distant.

In practical terms, this means our plans for the future should beware the creation of monocultures that lack diversity – cultures in which all the variations can move in the same direction at once.

We should also beware the connection of hidden connections, such as the shadow links between multiple different financial institutions that precipitated the shock financial collapse in 2008.

If there are reasons to foresee such outcomes, it means we need to rethink our plans for the new technology. Otherwise, the world might experience a shock outcome from which there is no prospect of any recovery – perhaps for generations, perhaps indefinitely.

Recent Posts

RAFT 2035 – a new initiative for a new decade

The need for a better politics is more pressing than ever.

Since its formation, Transpolitica has run a number of different projects aimed at building momentum behind a technoprogressive vision for a better politics. For a new decade, it’s time to take a different approach, to build on previous initiatives.

The planned new vehicle has the name “RAFT 2035”.

RAFT is an acronym:

  • Roadmap (‘R’) – not just a lofty aspiration, but specific steps and interim targets
  • towards Abundance (‘A’) for all – beyond a world of scarcity and conflict
  • enabling Flourishing (‘F’) as never before – with life containing not just possessions, but enriched experiences, creativity, and meaning
  • via Transcendence (‘T’) – since we won’t be able to make progress by staying as we are.

RAFT is also a metaphor. Here’s a copy of the explanation:

When turbulent waters are bearing down fast, it’s very helpful to have a sturdy raft at hand.

The fifteen years from 2020 to 2035 could be the most turbulent of human history. Revolutions are gathering pace in four overlapping fields of technology: nanotech, biotech, infotech, and cognotech, or NBIC for short. In combination, these NBIC revolutions offer enormous new possibilities – enormous opportunities and enormous risks:…

Rapid technological change tends to provoke a turbulent social reaction. Old certainties fade. New winners arrive on the scene, flaunting their power, and upturning previous networks of relationships. Within the general public, a sense of alienation and disruption mingles with a sense of profound possibility. Fear and hope jostle each other. Whilst some social metrics indicate major progress, others indicate major setbacks. The claim “You’ve never had it so good” coexists with the counterclaim “It’s going to be worse than ever”. To add to the bewilderment, there seems to be lots of evidence confirming both views.

The greater the pace of change, the more intense the dislocation. Due to the increased scale, speed, and global nature of the ongoing NBIC revolutions, the disruptions that followed in the wake of previous industrial revolutions – seismic though they were – are likely to be dwarfed in comparison to what lies ahead.

Turbulent times require a space for shelter and reflection, clear navigational vision despite the mists of uncertainty, and a powerful engine for us to pursue our own direction, rather than just being carried along by forces outside our control. In short, turbulent times require a powerful “raft” – a roadmap to a future in which the extraordinary powers latent in NBIC technologies are used to raise humanity to new levels of flourishing, rather than driving us over some dreadful precipice.

The words just quoted come from the opening page of a short book that is envisioned to be published in January 2020. The chapters of this book are reworked versions of the scripts used in the recent “Technoprogressive roadmap” series of videos.

Over the next couple of weeks, all the chapters of this proposed book will be made available for review and comment:

  • As pages on the Transpolitica website, starting here
  • As shared Google documents, starting here, where comments and suggestions are welcome.

RAFT Cover 21

All being well, RAFT 2035 will also become a conference, held sometime around the middle of 2020.

You may note that, in that way that RAFT 2035 is presented to the world,

  • The word “transhumanist” has moved into the background – since that word tends to provoke many hostile reactions
  • The word “technoprogressive” also takes a backseat – since, again, that word has negative connotations in at least some circles.

If you like the basic idea of what’s being proposed, here’s how you can help:

  • Read some of the content that is already available, and provide comments
    • If you notice something that seems mistaken, or difficult to understand
    • If you think there is a gap that should be addressed
    • If you think there’s a better way to express something.

Thanks in anticipation!

  1. A reliability index for politicians? 2 Replies
  2. Technoprogressive Roadmap conf call Leave a reply
  3. Transpolitica and the TPUK Leave a reply
  4. There’s more to democracy than voting Leave a reply
  5. Superdemocracy: issues and opportunities Leave a reply
  6. New complete book awaiting reader reviews Leave a reply
  7. Q4 update: Progress towards “Sustainable superabundance” Leave a reply
  8. Q3 sprint: launch the Abundance Manifesto Leave a reply
  9. Q2 sprint: Political responses to technological unemployment Leave a reply