Table of contents
This page lists the main section headings for all the chapters in this book.
- Confusion, turbulence, and peril
- This time it’s different
- The scope of the Principles
- Collective insight
- The short form of the Principles
- The four areas covered by the Principles
- What lies ahead
1. Background: Ten essential observations
- Tech breakthroughs are unpredictable (both timing and impact)
- Potential complex interactions make prediction even harder
- Changes in human attributes complicate tech changes
- Greater tech power enables more devastating results
- Different perspectives assess “good” vs. “bad” differently
- Competition can be hazardous as well as beneficial
- Some tech failures would be too drastic to allow recovery
- A history of good results is no guarantee of future success
- It’s insufficient to rely on good intentions
- Wishful thinking predisposes blindness to problems
2. Fast-changing technologies: risks and benefits
- Technology risk factors
- Prioritising benefits?
- What about ethics?
- The transhumanist stance
2.1 Special complications with artificial intelligence
- Problems with training data
- The black box nature of AI
- Interactions between multiple algorithms
- Self-improving AI
- Devious AI
- Four catastrophic error modes
- The broader perspective
- The gorilla problem
- Examples of dangers with uncontrollable AI
- Proposed solutions (which don’t work)
- The impossibility of full verification
- Emotion misses the point
- No off switch
- The ineffectiveness of tripwires
- Escaping from confinement
- The ineffectiveness of restrictions
- No automatic super ethics
- Issues with hard-wiring ethical principles
- Asimov’s Three Laws
- Ethical dilemmas and trade-offs
- Problems with proxies
- The gaming of proxies
- Simple examples of profound problems
- Humans disagree
- No automatic super ethics (again)
- Other options for answers?
- No guarantees from the free market
- No guarantees from cosmic destiny
- Planet B?
- Humans merging with AI?
- Approaching the Singularity
- Breaking down the definition
- Four alternative definitions
- Four possible routes to the Singularity
- The Singularity and AI self-awareness
- Singularity timescales
- Positive and negative singularities
- Tripwires and canary signals
- Moving forward
3.1 The Singularitarian Stance
- AGI is possible
- AGI could happen within just a few decades
- Winner takes all
- The difficulty of controlling AGI
- Superintelligence and superethics
- Not the Terminator
- Recap
- Opposition to the Singularitarian Stance
3.2 A complication: the Singularity Shadow
- Singularity timescale determinism
- Singularity outcome determinism
- Singularity hyping
- Singularity risk complacency
- Singularity term overloading
- Singularity anti-regulation fundamentalism
- Singularity preoccupation
- Looking forward
3.3 Bad reasons to deny the Singularity
- The denial of death
- How special is the human mind?
- A credible positive vision
- Factors causing AI to improve
- 15 options on the table
- The difficulty of measuring progress
- Learning from Christopher Columbus
- The possibility of fast take-off
5. The Singularity Principles in depth
5.1 Analysing goals and potential outcomes
- Question desirability
- Clarify externalities
- Require peer reviews
- Involve multiple perspectives
- Analyse the whole system
- Anticipate fat tails
5.2 Desirable characteristics of technological solutions
- Reject opacity
- Promote resilience
- Promote verifiability
- Promote auditability
- Clarify risks to users
- Clarify trade-offs
5.3 Ensuring development takes place responsibly
- Insist on accountability
- Penalise disinformation
- Design for cooperation
- Analyse via simulations
- Maintain human oversight
- Build consensus regarding principles
- Provide incentives to address omissions
- Halt development if principles are not upheld
- Consolidate progress via legal frameworks
- Public understanding
- Persistent urgency
- Reliable action against noncompliance
- Public funding
- International support
- A sense of inclusion and collaboration
7.1 Measuring human flourishing
- Some example trade-offs
- Updating the Universal Declaration of Human Rights
- Constructing an Index of Human and Social Flourishing
- Moore’s Law of Mad Scientists
- Four projects to reduce the dangers of WMDs
- Detecting mavericks
- Examples of trustable monitoring
- Watching the watchers
- Uplifting regulators
- The central role of politics
- Toward superdemocracy
- Technology improving politics
- Transcending party politics
- The prospects for political progress
- Top level areas of the Vital Syllabus
- Improving the Vital Syllabus
- Global action against the creation of AGI?
- Possible alternatives to AGI?
- A dividing line between AI and AGI?
- A practical proposal
7.6 Measuring progress toward AGI
- Aggregating expert opinions
- Metaculus predictions
- Alternative canary signals for AGI
- AI index reports
7.7. Growing a coalition of the willing
- Risks and actions