By Dana Edwards and Alexander J. Karran

Abstract

Accelerating social complexity in combination with outstanding problems like attention scarcity and information asymmetry contribute to human error in decision making. Democratic institutions and markets both operate under the assumption that human beings are informed rational decision makers working with perfect information, situation awareness, and unlimited neurological capacity. We argue that, although these assumptions are incorrect, they could to a large extent be mediated by a process of cyborgization, up to and including electing cyborgs into positions of authority.

Introduction

In the modern information age governing bodies, business organisations and adaptive systems are faced with ever increasing complexity in decision-making situations. Accelerating rates of technological and social change further compound this systemic complexity. In this complex environment the effects of human cognitive bias and bounded rationality become issues of great importance, impacting upon such domains as political policy, legislature, business practice, competitiveness and information intelligence.

In this text we shall use regulatory capture as an illustration of how human cognitive bias and conflicts of interest interact in the politico-economic space to create disproportionate advantage. We shall also hypothesize a novel potential solution to human cognitive bias in the form of human-machine hybrid decision support.

In broad terms regulation encompasses all forms of state intervention in economic function, and more specifically intervention with regard to the control of natural monopolies. The term “regulatory capture” is used to explain a corruption of the regulatory process. Regulatory capture has both narrow and broad interpretations. The broad interpretation is that it is a process through which special interest groups can affect state intervention ranging from the levying of taxes to legislation affecting the direction of research and development [i].The narrow interpretation places the focus specifically on the process through which regulated monopolies exert pressure to manipulate state agencies to operate in their favour[ii].

What these interpretations express is that regulatory capture generally involves two parties: the regulated monopoly and the state regulatory agency. The process of regulatory capture can be two way: just as corporations can capture government regulation agencies, the possibility exists for government agencies to capture corporations. As a result of this process, government regulatory agencies can fail to exert financial and ethical boundaries if they are captured, while corporations can fail strategically and financially if they are captured.

Regulatory capture takes two forms, materialist and non-materialist capture. In materialist capture, which is primarily financially motivated, the mechanism of capture is to appeal to the self-interest of the regulators. Materialist capture alters the motives of regulators based on economic self-interest, so that they become aligned with the commercial or special interest groups which are supposed to be regulated. This form of capture can be the result of bribes, political donations, or a desire to maintain government funding. Non-materialist capture also called cognitive or cultural capture happens when the regulator adopts the thinking of the industry being regulated. Status and group identification both play a role in the phenomena of regulators identifying with those in the industry they are assigned to regulate[iii].

Given the current socio-political climate of accelerating technological and social change, consideration should be given to how organizations are formed. Organizations should be structured to resist or otherwise minimize any service disruption caused by regulatory capture, so that if the process of normative regulation fails i.e. in situations where the balance of the relationship between the two entities has become corrupted, the service which required regulation in the first place can remain available after the failure.

One example of potential government regulatory failure due to a captured agency is the Environmental Protection Agency (EPA) hydraulic fracking scandal of 2004. The EPA released a report[iv] in which they stated that hydraulic fracturing posed “little or no threat” to drinking water supplies. Whistle-blower Weston Wilson disputed[v] this conclusion of the EPA publicly and exposed five of the seven members of the peer review panel as having conflicts of interest. These conflicts of interest allowed elements within the administration to apply pressure, and become involved in discussions about how fracking would eventually be portrayed in the report. Due to this pressure the EPA may have unable to publish a genuine conclusion about the safety of fracking. This reveals a potential failure of the EPA to protect the public interest due to regulatory capture.

Another example of regulatory capture concerns a dramatic failure of regulatory oversight for the British National Foundation (BNF), which is one the UK’s most influential institutes on diet and health. The BNF, established more than 40 years ago, advises government, schools, industry, health professionals and the public, and exists solely to provide “authoritative, evidence-based information on food and nutrition”[vi]. Its ability to provide independent evidence-based advice however has been called into question given its apparent bias towards promoting the views of the food industry and the organization’s lack of transparency when reporting funding sources.

This comes as no surprise when 39 members of its funding membership come from the food industry[vii]. For example, In October 2009, when a television commercial for a member company’s probiotic yoghurt product was banned, the BNF spoke out in support of the product (and thus the company) by claiming that there is “growing evidence that a regular intake of probiotics may positively influence our health”. As a result while appearing to take a stance on the grounds of public health, it would appear as though the BNF were protecting its own interests and those of a member company under the guise of regulatory oversight.

Factors that affect human decision making within complex adaptive systems

The examples of regulatory capture described above highlight some of the issues associated with human cognitive bias, specifically within a complex adaptive system (such as a government or corporation) where rational choice is bounded by self-interest combined with overarching organizational goals. In information saturated environments such as these, human cognitive limitations can become a factor that leads to poor rational decision making, requiring the individual or organisation to rely on shortcuts which may lead to human error. A number of psychological and social factors such as “attention scarcity”, “information asymmetry”, and “accelerating societal complexity” contribute to poor rational decision making within complex organisational structures. Awareness has been rising that human attention has become a scarce resource in the information age, and attention scarcity ultimately relates to the economics of attention.

Attention scarcity relates to a human cognitive limitation which determines the amount of information a human can digest and attend to in a given period of time (also referred to as an information economy). Simply put, “attention is a resource-a person only has so much of it”[viii]. Thus, in a low information economy any item brought to the attention of decision makers is perceived by its economic properties which are deemed decisive for its profitability. In contrast, in a high information economy, the diversity of items mean perception is limited and only choices that expose decision makers to sufficiently strong signals are viable.

Attention scarcity is a weakness of human cognition which can be purposefully exploited. For example, consider the U.S. Affordable Care Act, which has over 9000 pages of rules. It is likely that most voters lacked sufficient “attention” to read through and digest each page at the time when the act was being debated. Due to the complexity of legislative law, even if a team of “netizens” formed to crowd source the reading and analysis of a new law, it is unlikely that they would be able to interpret and understand it within the available timeframe to object if needed.

The effects of attention scarcity are observed in the poor public understanding not only of legal documents, but also of complex open source software. We see in open source software situations where the developers allow anyone to read the source code but in which the source code has so many lines of opaque obfuscated code that very few users or even other software developers understand how it works. We can see how attention scarcity produces information asymmetry between the open source developers who can decipher the source code and everyone else who may or may not choose to use the software.

Information asymmetry is a serious factor intrinsic to cognitive bias in human decision making, and concerns decisions in transactions where one party has the perception of, or is in possession of, more or better quality information than the other. This potentially creates an imbalance in the transaction power dynamic which may lead to future failure and a collapse of trust, causing a kind of market failure in a worst case scenario.

Accelerating societal complexity refers to the structural and cultural aspects of our institutions whose practices are identified by the “shrinking of the present”, a decreasing time period during which expectations based on past experience reliably match the future[ix]. When combined with accelerating technological progress this “shrinking” appears to flow ever faster, making decisions based on belief or the perception of better information problematic.

All of these individual factors can influence the human decision making process; in combination they potentially create a decision space that becomes more fluid, with a self-reinforcing feedback loop which requires better decisions to be made in shorter spaces of time with incomplete or asymmetric information. Indeed, by all accounts humans make errors all of the time, but as society gets ever more complex, these errors have lasting and increasingly dangerous consequences (such as in the example of hydraulic fracking discussed above). In order to get a clearer picture of a possible basis for this error effect, some discussion of human cognitive limitations is warranted.

The impact of human cognitive limitation

As we have discussed previously, information asymmetry in complex adaptive systems allows for decision error to appear within the system, as the better informed parties possess a marked information advantage which allows them to exploit the ignorance of other parties. This can occur in any field of human endeavour, such as law, science, commerce or governance, where new knowledge will be easier to grasp by those with previous knowledge, given that knowledge is self-referential and compounds on itself[x]. As organizations grow larger and the decision requirements become ever more complex, attention scarcity and information asymmetry can form a feedback loop that – at scale – slows the rate of innovation/knowledge diffusion, as individuals and organisations vie for supremacy in transactions.

Research in the area of cognitive neuroscience suggests that the cognitive abilities of an individual are limited to five core systems (objects, agents, number, geometry and social) [xi], each with its own set of limitations. An example of limitation within the social system is “Dunbar’s number”, first proposed by British anthropologist Robin Dunbar[xii], who posited that the number of social group members a primate can track is limited to the volume of the neocortex, and while this theory is hotly disputed[xiii], it has yet to be disproven with any certainty. This limitation, if taken to its logical conclusion and scaled to match an average complex adaptive system (such as regulatory or corporate bodies) highlights that the decision making abilities of an average individual could be impaired significantly, when not augmented by technology or genetic engineering.

This impairment of decision making ability was remarked upon in Herbet A. Simon’s theories of bounded rationality[xiv]. These theories were concerned with rational behaviour in the context of individuals and organisations and individuals within organisations, which he stated were indistinguishable under the “theory of the firm”. In this theory the given goals and the given conditions (of the organization) drive “rational” decision making based on two functions: the demand function (the quantity demanded as a function of price) and the cost function (the cost of production as a function of the quantity produced). These two rules when applied to complex adaptive systems, such as regulatory or governing bodies, demonstrate the vast scope in which human cognitive bias can affect outcomes at the macro scale while appearing to be a series of micro decisions made by individuals.

Nowhere can this asymptotic synergy of information, human cognitive ability and bounded rationality be seen more clearly, than in the case of law. A truism often used in this context is that Ignorance of the law excuses no one, but the complexity of law confuses everyone. In a world where few if anyone in society knows the law it may well become necessary for people to supplement their own cognitive capacities with “apps” to protect themselves from the complexity of the law. “Lawfare” is said to describe a form of asymmetric warfare which allows for the exploitation of the esoteric and complex nature of the law to damage political opponents. Just as complex words on an ingredient list can be used to hide undesirable ingredients from customers, the law and its potential use as a weapon also remain hidden from most citizens.

The current analogue forms of government have their basis in a complicated combative bureaucracy (necessary to support representative forms of democracy). Accelerating technological progress, however, shows that this approach may not scale particularly well as society becomes orders of magnitude more complex in the coming decades. It is our analysis, that unless a Transhumanist approach is adopted to enhance the existing human decision processes by merging with technological decision support, catastrophic failures may occur.

In this socially complex future, it is likely that our politicians may have to rely increasingly on information technologies, to the point that they essentially become cyborgs, merging fact checking and recommendation engines – based on rational rulesets – to keep pace with accelerating societal change and allow them to fully encompass monolithic social structures. In addition, citizens may also out of necessity need to adopt similar technologies, in order to understand the decisions made by these new “enhanced” politicians and to adapt to and effectively participate in an increasingly complex and fast changing society. In addition the institutions of the future will likely have to adopt human error tolerant designs which use the latest decision support technology to help mitigate and dampen the consequences of human error.

The Cyborg Citizen: A transcendent solution?

In order to avoid confusion we first have to properly define what we mean by a cyborg citizen. Andy Clark[xv] in his book Natural-Born Cyborgs: Minds, Technologies, and the future of Human Intelligence, argues that human beings are by nature cyborgs, claiming that human neural plasticity and a propensity to build and utilise tools in everyday life (from handwriting to mobiles phones), produces a species that thinks and feels most effectively only through the use of its technologies. Ray Kurzweil[xvi] goes one step further to predict that, by 2030, most humans will choose to be cyborgs:

Our thinking then will be a hybrid of biological and non-biological thinking. We’re going to gradually merge and enhance ourselves. In my view, that’s the nature of being human – we transcend our limitations.

In order to understand what a “cyborg citizen” means in today’s information and technology driven society, we must expand upon this definition to include current technological and social developments. Indeed, we will have to recognize that each individual today, and more so in the future, will have a digital, virtual, and physical self[xvii]. Thus, a cyborg is a person who is (singly or in combination) enhanced by or dependent upon, robotic, electronic, or mechanical devices such as artificial hearts, pacemakers, portable dialysis machines or even mobile / cloud computing which employs storage, search, retrieval and analysis (SSRA) capabilities such as Google, Amazon etc.

Corporations also appear to be taking advantage of technologies to enhance human decision making as a way to adapt to increasing business and market complexity. Venture capitalist firm Deep Knowledge Ventures named to their board of directors[xviii] an algorithm called VITAL, which they intend to someday evolve into a full-fledged artificial intelligence. This move may represent one of the initial forays in what may become a trend toward human-machine run corporations. Indeed, some are going much further, to call for complete replacement of humans within complex organisations (such as government) with artificial intelligence[xix]. However, arguments about the inevitable rise of artificial general intelligence aside, we push for a “human-in-the-loop” approach through the merger or bonding of human ethical and moral “instinct” with a bounded rational decision support engine, existing in either digital space or embedded into the human central nervous system via implants.

So what would such a citizen cyborg look like? Below is a list of a number of hypothetical decision support systems which are presently borderline (in that they exist, but are not as yet fit for purpose), which could exist in digital space and employ SSRA capabilities to allow for enhanced human-machine hybrid decision making.

• Intent casting: Intent casting, originally described by Doc Searls, allows consumers to directly express their wants and needs to the market. This could allow for the digitization of intent and for agent-based AI to shop on behalf of customers.
• Algorithmic democracy: Algorithmic democracy in theory, would allow voters to delegate their voting decisions (and thus agency) to an algorithm, which could be referred to as a digital voting agent (DA). Examples of digital agents today include Siri, Amazon Echo, and Cortana. As these DA’s become more capable, it is possible that voters could rely on their DA to inform them as to how they should vote in accordance to their specific interests and preferences.
• Digital decision support consultants: These are intelligent decision support systems that would help professionals make better decisions. It is likely that there will be apps for different professions such as IBM’s WellPoint for doctors, legal assistant apps, and real-time fact checkers[xx] These apps may be decentralized collaborative applications with human and robot participation or they may be software agent based AI. This category would also include algorithms such as Deep Knowledge Ventures VITAL and agents to track relationships and the flow of information between groups within a complex organization or brokers between two transaction parties.

Examples of algorithms that hypothetically speaking, could run on physiologically embedded technology, directly accessible by the human brain to provide decision support:

• Generate and test search: a reinforcement learning, trial and error algorithm which can search through a limited solution space in a systematic manner to find the best solution[xxi]. In operation this algorithm would generate possible solutions to a set problem and test each until it finds the solution which passes a positive threshold, whereupon the solution is relayed to the human cognitive process for a potential decision and reinforcement. This kind of technique can be used to take advantage of simulation testing and solve problems which have a limited solution space, such as those presented by the “free market” or those requiring a quick human decision in a “lesser of two evils” scenario.
• Global optimization search: evolutionary algorithms which are inspired by the biological mechanisms of global optimization search, such as mutation, crossover, natural selection and survival of the fittest[xxii]. These algorithms can search a solution space and compare each solution to a desired fitness criteria. In the case where human input is necessary to evolve a solution then an interactive evolutionary algorithm could allow the human to be the solution selector, while the algorithm is the solution generator. The algorithms can go through a similar process and be generated and evolved for improved fitness.
• Markov decision processes: an experimental framework for decision making and decision support. A Markov decision process automates finding the optimal decision for each state while taking into account each action’s value in comparison to the others, essentially an idealised decision output for a given problem state. With human decision selection driving the process, the ramifications of each decision selection at each stage of the problem analysis can be carefully considered and accepted or rejected based on rational choice.

This list is by no means exhaustive and there may be other borderline hypothetical decision support systems and algorithms which are not mentioned here. However, this list gives a general idea of how embedded or digital artificial decision support agents can improve decision quality in certain sectors of human society. By improving decision quality through technology and semi-autonomous agencies we may be able to reduce the frequency of poor decisions which result from nothing more than human error and or human ignorance.

Discussion: Checks and Balances

We do not propose that cyborgization makes for a perfect solution to the problem of human cognitive limitation and decision error in complex social systems. Indeed, decision support systems already exist in one form or another. However, they are still in an early stage of development and not ubiquitous, thus technology such as VITAL benefits only large corporations and perhaps the intelligence establishment. It is a situation similar to the early stages of computer development or the Internet, both of which existed, but the benefits were limited to certain domains, back in the 1960s during the Cold War.

We believe the widespread adoption of decision support technology, be it embedded or digital, could provide the tools necessary for individuals to comprehend the entirety of complex organizations, model the decision-consequence space and select ethical decisions. These tools would essentially enable decision makers to take into account individual need and motivation, and provide ethical solutions which afford the greatest good for the greatest number, without creating asymmetric information economies.

An example of a beneficial application of cyborg technology would be the doctor who utilises WellPoint[xxiii] to make diagnoses based on a combination of learned skillset and a digital health agent with a broad specialist evidence based knowledge base. Alternatively, in a quantified-self context an individual could upload health data gathered from wearable sensor technology, and receive information of potential health issues which could be treated with alacrity in their early stages by doctors able to access this information and review treatment options.

However, such technology and its application would not come without limitation or risk. The widespread use of these technologies could lead to a form of information “cold war”, in which human and machine agents (singly or in combination) attempt to create a state of “perfect information” to gain a competitive advantage. They may seek a form of perfect regulatory capture where one party seeks always to have an advantageous position in any transaction, be it in the free market or in the policy, legislative or intelligence domains. Arguably, such an information cold war already exists between various governments, intelligence services and corporate entities and while the “battle ground” as it were, is in so called cyberspace, it is primarily an analogue concern where agency is biological i.e. human as opposed to A.I.

It is a sad reflection upon humanity that one “positive” aspect of this cold war scenario, is that competition (war) leads to innovation, as opposing sides race to gain the information advantage. This impetus this would accelerate the development of the technologies required to create a “true” cybernetic individual or generally intelligent artificial agent. It is a matter for debate whether this would result in a situation that would be to the benefit of humanity in general or lead to a totalitarian dystopia; in which one entity or organisation exists in a near perfect state of “knowing”, stifling the development of both technology and society.

It is our opinion that the potential benefits of cyborgization outweigh the potential risks. As our technological systems and culture grow ever more complex, we must consider the risk of human error, of bad decisions, of ignorance combined with advanced technologies, in the light of a technology so pregnant with possibility.

We realize cyborgization is a controversial subject, however we see it an unavoidable and unstoppable trend. Indeed, Ginni Rometty (Chairman and CEO of IBM) stated recently that:

In the future, every decision that mankind makes is going to be informed by a cognitive system like Watson, and our lives will be better for it[xxiv]

This is a statement is very much in accordance with our notion of keeping the human-in-the-loop during decision making. Furthermore, an argument could be made that given the current reliance by vast numbers of the world population on mobile phones and internet search engines, rather than becoming cyborgs at some specific point in time (as in the prediction of Kurzweil), we have always been cyborgs (as per Clarke’s argument) and it is merely a matter of time and technology, until the line between what is human and what is our technology becomes non-existent.

Conclusion

Just as search engines allow for human beings to find the relevant information meeting their “criteria”, the adoption of decision support engines could allow autonomous digital agents and human-machine hybrids alike to find the most ethical decision within a given consequence-decision space. This approach would allow for “what if” hypothesis testing[xxv] of many decision types such as policy determination, legislative impact, market transactions and global consequence. The dawn of ethical computing is fast approaching and it is in this area requiring our fullest attention. Transhumanism provides a socially progressive framework that if adopted can allow us to transcend our human cognitive limitations, so that we can become more effective and ethical decision makers. We believe that developing the technology which can facilitate our arrival at the cyborg stage of human leadership should be a top priority, especially in this time of accelerating developments in Artificial Intelligence, which if left unsupervised could surpass us to become the apex decision maker for our entire species.

Footnotes

[i] Stigler, G. (1971), “The Theory of Economic Regulation.”, Bell Journal of Economics and Management Science, 2, 3–21

[ii] Peltzman, S. (1976), “Toward a More General Theory of Regulation.”, Journal of Law and Economics, 19 , 211–48.

[iii] Carpenter, D., & Moss, D. A. (Eds.). (2013). “Preventing regulatory capture: special interest influence and how to limit it.” Cambridge University Press.

[iv] Environmental Protection Agency, “Study of Potential Impacts of Hydraulic Fracturing of Coalbed Methane Wells on Underground Sources of Drinking Water.” Office of Groundwater and Drinking Water report, June 2004 – accessed May 2015.

[v]http://www.ucsusa.org/our-work/center-science-and-democracy/promoting-scientific-integrity/oil-extraction.html#.VX7TjkbFdlx

[vi] Chamberlain & Laurance (2010). “Is the British Nutrition Foundation having its cake and eating it too?” http://www.independent.co.uk/life-style/food-and-drink/news/is-the-british-nutrition-foundation-having-its-cake-and-eating-it-too-1925034.html – accessed May 2015.

[vii] Chamberlain & Laurance (2010). “Is the British Nutrition Foundation having its cake and eating it too?” http://www.independent.co.uk/life-style/food-and-drink/news/is-the-british-nutrition-foundation-having-its-cake-and-eating-it-too-1925034.html – accessed May 2015.

[viii]Crawford, Matthew B. (March 31, 2015). “Introduction, Attention as a Cultural Problem”. The World Beyond Your Head: On Becoming an Individual in an Age of Distraction (hardcover) (1st ed.). Farrar, Straus and Giroux. p. 11.

[ix] Rosa, H.: “Social Acceleration: A New Theory of Modernity.” Columbia University Press, New York (2013)

[x] Klein, S. B., & Kihlstrom, J. F. (1986). “Elaboration, organization, and the self-reference effect in memory.” Journal of Experimental Psychology: General, 115(1), 26-38. doi:10.1037/0096-3445.115.1.26

[xi] Kinzler KD, Spelke ES. Core systems in human cognition. Progress in Brain Research. 2007;164:257–264

[xii] Dunbar, R. I. M. (1992). “Neocortex size as a constraint on group size in primates”. Journal of Human Evolution 22 (6): 469–493. doi:10.1016/0047-2484(92)90081-J

[xiii] Wellman, B. (2012). “Is Dunbar’s number up?” British Journal of Psychology 103 (2): 174–176; discussion 176–2. doi:10.1111/j.2044-8295.2011.02075.x

[xiv] Simon, H.A. (1972). Theories of bounded rationality. In C.B. McGuire and R. Radner (Eds.), Decision and organization: A volume in honor of Jacob Marschak (Chap. 8). Amsterdam: North-Holland

[xv] Andy, Clark. (2004) “Natural-Born Cyborgs: Minds, Technologies, and the Future of Human Intelligence.”, Oxford; Oxford University Press.

[xvi] Guia Del Prado “Google Futurist Ray Kurweil thinks we’ll all be cyborgs by 2030” http://uk.businessinsider.com/ray-kurzweil-thinks-well-all-be-cyborgs-by-2030-2015-6?r=US – accessed june-2015

[xvii] The digital and virtual while similar are distinct in their differences. To make clear the distinction, something is virtual if it will only exist contained within a virtual world while if something is digital it is known to exist in the physical world just in digitized form. The distinction is between digital and virtual space in which digital space is a subset of what people consider to be part of the physical world while virtual space isn’t directly referring to a part of the physical world

[xviii]Wile, R. (2014, May 13). “A Venture Capital Firm Just Named An Algorithm To Its Board Of Directors – Here’s What It Actually Does.” Retrieved June 5, 2015, from http://www.businessinsider.com/vital-named-to-board-2014-5#ixzz31dVwrSEo

[xix] http://ieet.org/index.php/IEET/more/pellissier20150612

[xx] Ciampaglia GL, Shiralkar P, Rocha LM, Bollen J, Menczer F, Flammini A (2015) Computational Fact Checking from Knowledge Networks. PLoS ONE 10(6): e0128193. doi:10.1371/journal.pone.0128193

[xxi] Kaelbling, L. P., Littman, M. L.,.and Moore, A. W., (1996) “Reinforcement Learning: A Survey.”, Journal of Artificial Intelligence Research, Volume 4, pages 237-285

[xxii] Weise, T. “Global Optimization Algorithms – Theory and Application.” Germany: it-weise.de (self-published), 2009. [Online]. Available: http://www.it-weise.de/ -accessed 06-2015

[xxiii] http://www.ibm.com/smarterplanet/us/en/ibmwatson/assets/pdfs/WellPoint_Case_Study_IMC14792.pdf

[xxiv] http://www.businessinsider.in/The-CEO-of-IBM-just-made-a-jaw-dropping-prediction-about-the-future-of-artificial-intelligence/articleshow/47289655.cms

[xxv] Winfield, A. F., Blum, C., & Liu, W. (2014). “Towards an ethical robot: internal models, consequences and ethical action selection.” In Advances in Autonomous Robotics Systems (pp. 85-96). Springer International Publishing

Footnote

The article above features as Chapter 5 of the Transpolitica book “Envisioning Politics 2.0”.

The image is an original design by Alexander Karran.