Evolution and enforcement

The final area of the Singularity Principles covers how the overall set of recommendations is itself likely to evolve over time, and how the recommendations will be applied in practice rather than simply being some kind of wishful thinking. There are four principles in this area:

• Build consensus regarding principles
• Provide incentives to address omissions
• Halt development if principles not upheld
• Consolidate progress via legal frameworks

Since they bridge what could be a yawning gulf between aspiration and actuality, these principles can be seen as the most important in the entire set.

This chapter gives an initial description of these four principles. The remainder of the book explores various questions of practicality and enforcement that arise.

Build consensus regarding principles

The principle of “Build consensus regarding principles” urges that this set of principles be discussed widely, to ensure broad public understanding and buy-in, with conformance in spirit as well as in letter.

In this way, the principles can become viewed as being collectively owned, collectively reviewed, and collectively endorsed, rather than somehow being imposed from outside.

Indeed, society should be ready to update these principles if discussion makes such a need clear – provided the potential change has been carefully reviewed beforehand. There is no assumption of “tablets of stone”.

We can be guided in this discussion by applying many of the Singularity Principles, which were initially about the development of technology, to the principles themselves.

This includes the Singularity Principles about goals and outcomes:

• Question desirability (of each principle)
• Clarify externalities (for each principle)
• Require peer reviews (for each principle)
• Involve multiple perspectives (for each principle)
• Analyse the whole system (for each principle)
• Anticipate fat tails (for each principle)

It also includes the Singularity Principles about the characteristics that are desirable in solutions and methods:

• Reject opacity (for any of the principles)
• Promote resilience (for each principle)
• Promote verifiability (for each principle)
• Promote auditability (for each principle)
• Clarify risks to users (for each principle)
• Clarify trade-offs (for each principle)

It’s still comparatively early days in this consensus-building discussion.

Provide incentives to address omissions

The principle “Provide incentives to address omissions” states that, where any of this set of principles cannot be carried out adequately, measures should be prioritised to make available additional resources or suitably skilled personnel, so that these gaps can be filled.

This may involve extra training, provision of extra equipment, transfer of personnel between different tasks, altering financial incentive structures, updating legal rules, and so on.

However, if the gap grows too large, between the recommendations of the principles, and prevailing industry practices, something more drastic is needed.

That brings us to the principle of “Halt development if principles are not upheld”.

Halt development if principles are not upheld

In case any of these Singularity Principles cannot be carried out adequately, and measures to make amends are blocked, any further development of the technology in question should be halted until such time as the principles can once again be observed.

This may be viewed as a shocking principle, but it was applied very successfully as part of the revolutionary lean manufacturing culture developed in Toyota in Japan from the 1930s onward. Toyota executives realised it was actually to the competitive advantage of their company if each and every employee on their production line was able, on noticing a significant problem with the production, to pull a cord to temporarily halt production of that product. The brake meant that wide attention was quickly brought to bear on whatever quality issue had been noticed. Production throughput slowed down in the short term, but quality throughput and reliability increased in the medium and longer term.

But as just noted, adopting this principle means a slow down in production in the short term, which some companies may consider to put them at a fundamental disadvantage against faster-moving competitors. Moreover, some companies, out of their own bias toward thinking their solutions have very special qualities, may not agree with the appropriateness of some of the principles given earlier. They may wish to rush on regardless. As in the famous phrase that was Facebook’s motto for many years, they are prepared to “Move fast and break things”, thinking that if things do get broken, they will be easy enough to fix afterward.

That thought – “move fast and break things” – is opposed by an idea expressed at a lecture in 1923 at Cambridge University by biologist JBS Haldane, entitled “Daedalus, or Science and the Future”. Haldane reflected on the growing power of science. Observing the progressive increase in these powers, he suggested that these powers are “only fit to be placed in the hands of a being who has learned to control himself”. He went on to say, in a phrase that others have subsequently often repeated, that “man armed with science is like a baby armed with a box of matches”.

No one likes to be told they are “like a baby”, especially if they have a past history of developing remarkable technology. However, from a certain point of view, we all lack sufficient forethought and control to simply “move fast and break things”, when what we might break has such explosive potential.

So how do we ensure that production is halted, if necessary, before it reaches an explosive phase? The answer is in the final principle in the entire set.

Consolidate progress via legal frameworks

The principle “Consolidate progress via legal frameworks” states that we should embed aspects of these principles in legal frameworks, to make it more likely that they will be followed.

There needs to be appropriate penalties for violating these frameworks, just as there are already penalties in place when companies violate any of a range of existing regulations on health and safety, or on truthfulness in advertising, or on the presentation of financial information.

These legal frameworks will need to have trenchant backing from society as a whole. After all, some of the companies that are rushing ahead to create more powerful technologies have huge financial motivations to evade legal restrictions. These companies are receiving extensive investments, from banks or venture capitalists, under the assumption that they can produce and maintain a decisive competitive advantage. They are motivated to keep many of their plans under tight reins of secrecy. Via the extensive budgets at their command, they can purchase the support of tame politicians. As such, they form what might appear to be an irresistible force. And as such, they will need to be challenged by an equally strong counterforce.

That counterforce is politics – or, better said, democratic politics. History teaches us that governments can, on occasion, build sufficient public support to impose a change of direction on major corporations. For example, anti-trust legislation in the US from the 1890s onward trimmed the power of large conglomerates or cartels in railways, steel, tobacco, oil, and telecommunications, helping to prevent monopoly abuse. Other legislation restricted widespread fraudulent or unsafe practice in fields such as food preparation and the distribution of supposed medicines (which were often “snake oil”).

Of course, just as there can be serious anti-social consequences of over-powerful corporations, there can be serious anti-social consequences of over-powerful politicians. Just as there are well-known failure modes of free markets, there are well-known failure modes of political excess. Just as corporations need to remain under the watchful eye of society as a whole, the political framework also needs to be watched carefully by society.

That’s why I said that the counterforce to dominant corporations should be, not just politics, but democratic politics – politics that (when it works well) responds quickly to the needs and insights of the entire population.

Moreover, just as the content of the Singularity Principles needs to be subject to revision following public debate, the corresponding legal statutes likewise need to be subject to prompt revision, whenever it becomes clear, following appropriate public review, that they need amending. In other words, the legal frameworks need to combine both strength and adaptability.

None of this will be easy. It will require high calibre politics to ensure it works well. It will also require high calibre geopolitics, to ensure a suitable level playing field on the international stage.

Is it credible to look forward to “high calibre politics” and “high calibre geopolitics”? For discussion of these and related questions, let’s consider in the next chapter the key success factors for the Singularity Principles.