Evolution and enforcement

The final area of principles covers how the overall set of recommendations might itself evolve further over time, and how the recommendations will be applied in practice rather than simply being some kind of wishful thinking. There are four principles in this area:

• Build consensus regarding principles
• Provide incentives to address omissions
• Halt development if principles not upheld
• Consolidate progress via legal frameworks

Build consensus regarding principles

The principle of “Build consensus regarding principles” urges that this set of principles be discussed widely, to ensure broad public understanding and buy-in, with conformance in spirit as well as in letter.

We should also be ready to update these principles if discussion makes such a need clear, provided the potential change has been widely reviewed beforehand.

We can be guided in this discussion by applying many of the Singularity Principles, which were initially about the development of technology, to the principles themselves.

This includes the Singularity Principles about goals and outcomes:

• Question desirability
• Clarify externalities
• Require peer reviews
• Involve multiple perspectives
• Analyse the whole system
• Anticipate fat tails

It also includes the Singularity Principles about the characteristics that are desirable in solutions and methods:

• Reject opacity
• Promote resilience
• Promote verifiability
• Promote auditability
• Clarify risks to users
• Clarify trade-offs

It’s still comparatively early days in this consensus-building discussion.

Provide incentives to address omissions

The principle “Provide incentives to address omissions” states that, where any of this set of principles cannot be carried out adequately, measures should be prioritised to make available the additional resources or suitably trained personnel that can fill these gaps.

This may involve extra training, provision of extra equipment, transfer of personnel between different tasks, altering financial incentive structures, updating legal rules, and so on.

However, if the gap grows too large, between the recommendations of the principles, and prevailing industry practices, something more drastic is needed.

That brings us to the principle of “Halt development if principles are not upheld”.

Halt development if principles are not upheld

In case any of these singularity principles cannot be carried out adequately, and measures to make amends are blocked, any further development of the technology in question should be halted until such time as the principles can once again be observed.

This may be viewed as a shocking principle, but it was applied very successfully as part of the revolutionary lean manufacturing culture developed in Toyota in Japan from the 1930s onward. Toyota executives realised it was actually to the competitive advantage of their company if each and every employee on their production line was able to pull a cord, on noticing a significant problem with the production, to temporarily halt production of that product. The break meant that wide attention was quickly brought to bear on whatever quality issue had been noticed. Production throughput slowed down in the short term, but quality throughput and reliability increased in the medium and longer term.

But as just noted, adopting this principle means a slow down in production in the short term, which some companies may consider to put them at a fundamental disadvantage against faster-moving competitors. Moreover, some companies, out of their own bias towards thinking their solutions have very special qualities, may not agree with the appropriateness of some of the principles given earlier. They may wish to rush on regardless. As in the famous phrase that was Facebook’s motto for many years, they are prepared to “Move fast and break things”, thinking that if things do get broken, they will be easy enough to fix afterward.

That thought – “move fast and break things” – is opposed by an idea expressed at a lecture in 1923 at Cambridge University by biologist JBS Haldane, entitled “Daedalus, or Science and the Future”. Haldane reflected on the growing power of science. Haldane noted the progressive increase in these powers. He suggested that these powers are “only fit to be placed in the hands of a being who has learned to control himself”. He went on to say, in a phrase that others have subsequently often repeated, that “man armed with science is like a baby armed with a box of matches”.

No one likes to be told they are “like a baby”, especially if they have a past history of developing remarkable technology. However, from a certain point of view, we all lack sufficient forethought and control to simply “move fast and break things”, when what we might break has such explosive potential.

So how do we ensure that production is halted, if necessary, before it reaches an explosive phase? The answer is in the final principle in the entire set.

Consolidate progress via legal frameworks

The principle “Consolidate progress via legal frameworks” states that we should embed aspects of these principles in legal frameworks, to increase the chance that they will be followed.

There should be appropriate penalties for violating these frameworks, just as there are, for example, penalties in place when companies violate any of a range of existing regulations on health and safety, or on truthfulness in advertising, or on the presentation of financial information.

We should also be ready to update these legal statutes promptly if it becomes clear that they need amending, following appropriate review.

This is no easy task. It will require high calibre politics, to ensure it works well. It will also require high-calibre geopolitics, to ensure a suitable level playing field on the international stage.