Whats the big deal?
Machine ethics and robot rights are quickly becoming hot topics in artificial intelligence/robotics communities. Using this literature review I will argue that the attempts to allow machines to make ethical decisions or to have rights in the workplace are misguided. Instead a new science of safety engineering for intelligent artificial agents is proposed. In particular we issue a challenge to the scientific community to develop intelligent systems capable of proving that they are in fact safe even under recursive self-improvement. (Hall, J)
Introduction to Artificial Intelligence:
Artificial intelligence (AI) is the aptitude demonstrated by machines or software. In academia it studies the goal of creating intelligence. Researchers in the field and textbooks define this field as “the study and design of intelligent agents”, where an intelligent agent is a system that perceives its environment and takes actions that maximize its chances of success. John McCarthy, who coined the term in 1955, defines it as “the science and engineering of making intelligent machines”. (McCarthy, 1955)
AI research is technical and specialized, and is severely divided into subfields that often fail to converse with each other. Some of the separation is due to social and cultural factors: subfields have grown up around specific organizations and the work of individual scholars. AI research is also divided by several technical issues. Some subfields focus on the explanation of specific problems. While other fields focus on one of several possible approaches or on the use of a particular tool or towards the accomplishment of particular applications.
The central goals of AI research include rational, knowledge, planning, learning, natural language processing, perception and the ability to move and control objects. General intelligence is still among the field’s long term goals. Currently popular approaches include statistical methods, computational intelligence and traditional symbolic AI. There are a large number of tools used in AI, including versions of search and mathematical optimization, logic, methods based on probability and economics, and many others. The AI field is interdisciplinary, in which a number of sciences and professions converge, including computer science, mathematics, psychology, linguistic, philosophy, and neuroscience, as well as other specialized fields such as artificial psychology. (McCarthy, 2006)
Recent developments in artificial intelligence are allowing an increasing number of decisions to be passed from human to machine. Most of these to date are operational decisions such as algorithms on the financial markets deciding what trades to make and how. However, the ranges of such decisions that can be compute are increasing, and as many operational decisions have moral consequences, they could be considered to have a moral component.
Imagine, in the near future, a company using a machine learning algorithm to recommend workers compensation for approval. An unsatisfied workers comp. applicant brings a lawsuit against the company, alleging that the algorithm is discriminating racially against workers compensation applicants. The company replies that this is impossible, since the algorithm is intentionally blinded to the race of the applicants. Indeed, that was part of the companies’ rationale for implementing the system. Even so, data show that the company’s approval rate for black applicants have been plummeting. Submitting ten seemingly equally qualified genuine applicants’ shows that the algorithm accepts white applicants and rejects black applicants. What could possibly be happening?
Finding a solution may not be easy. If the machine learning algorithm is constructed on a complicated neural network, or a genetic system produced by directed development, at this point it may show things almost impossible to understand why, or even how, the algorithm is judging applicants based on their race. Additionally a machine learner based on decision trees or Bayesian networks is much more transparent to programmer inspection (Yampolskiy. 2012), which may enable an inspector to discover that the AI algorithm uses the address information of applicants who were born or previously resided in predominantly in stricken areas where nuclear disasters had taken place.
AI algorithms play an increasingly large role in modern society, though usually not labeled “AI”. The scenario described above might be transpiring even as you read. It will become increasingly important to develop AI algorithms that are not just powerful and scalable, but also transparent to inspection, to name one of many societal concerns with safety of constantly advancing machines. Unfortunately the perceived abundance of research in intelligent machine safety is misleading. The great majority of published papers are purely philosophical in nature and do little more than reiterate the need for machine ethics and argue about which set of moral convictions would be the right ones to implement in our artificial progeny. (Yampolskiy. 2012) However, since ethical norms are not universal, a “correct” ethical code could never be selected over others to the satisfaction of humanity as a whole.
Artificial Intelligence Safety Engineering:
Even if people are successful at designing machines capable of passing a Moral Test, human-like performance means some dishonest actions, which should not be acceptable from the machines we design and employ. In other words, we don’t need machines which are Full Ethical Agents debating about what is right and wrong, we need our machines to be inherently safe and law abiding. As Robin Hanson has sophisticatedly put it: “ In the early to intermediate era when robots are not vastly more capable than humans, you’d want peaceful law-abiding robots as capable as possible, so as to make productive partners. … Most important would be that you and they have a mutually-acceptable law as a good enough way to settle disputes, so that they do not resort to predation or revolution. If their main way to get what they want is to trade for it via mutually agreeable exchanges, then you shouldn’t much care what exactly they want. The later era when robots are vastly more capable than people should be much like the case of choosing a nation in which to retire. In this case we don’t expect to have much in the way of skills to offer, so we mostly care that they are law-abiding enough to respect our property rights. If they use the same law to keep the peace among themselves as they use to keep the peace with us, we could have a long and prosperous future in whatever weird world they conjure. … In the long run, what matters most is that we all share a mutually acceptable law to keep the peace among us, and allow mutually advantageous relations, not that we agree on the “right” values. Tolerate a wide range of values from capable law-abiding robots. It is a good law we should most strive to create and preserve. Law really matters.” (Yampolskiy. 2012)
Consequently, we propose that purely philosophical discussions of ethics for machines be supplemented by scientific work aimed at creating safe machines in the context of a new field we will term “AI Safety Engineering.” Some real work in this important area has already begun . A common theme in AI safety research is the possibility of keeping a super intelligent agent in a sealed hardware so as to prevent it from doing any harm to humankind. Such ideas originate with scientific visionaries such as Eric Drexler who has suggested confining trans human machines so that their outputs could be studied and used safely. Similarly, Nick Bostrom, a futurologist, has proposed an idea for an Oracle AI (OAI), which would be only capable of answering questions. Finally, in 2010 David Chalmers proposed the idea of a “leakproof” singularity. He suggested that for safety reasons, AI systems first be restricted to simulated virtual worlds until their behavioral tendencies could be fully understood under the controlled conditions. (Yampolskiy. 2012)
The Challenges Ahead:
As for the challenge of AI safety engineering, I believe that developing safety mechanisms for self-improving systems will be the most difficult. If an artificially intelligent machine is as capable as a human engineer of designing the next generation of intelligent systems, it is important to make sure that any safety mechanism incorporated in the initial design is still functional after thousands of generations of constant self-improvement without human interference. Ideally every generation of self-improving system should be able to produce a verifiable proof of its safety for external inspection. It would be disastrous to allow a safe intelligent machine to design an inherently unsafe upgrade for itself resulting in a more capable and more dangerous system.
Some have argued that this challenge is either not solvable or if it is solvable one will not be able to prove that the discovered solution is correct. As the complexity of any system increases, the number of errors in the design increases proportionately or perhaps even exponentially. Even a single bug in a self-improving system (the most complex system to debug) will violate all safety guarantees. Worse yet, a bug could be introduced even after the design is complete either via a random mutation caused by deficiencies in hardware or via a natural event such as a short circuit modifying some component of the system.
My Two Cents:
We would like to offer some broad suggestions for the future directions of research aimed at counteracting the problems presented in this paper. First, the research itself needs to change from the domain of interest of only theoreticians and philosophers to the direct involvement of practicing computer scientists. Limited AI systems need to be developed as a way to experiment with non-anthropomorphic minds and to improve current security protocols. Some preliminary work has begun to appear in scientific venues which aim to specifically address issues of AI safety and ethics, if only in human-level-intelligence systems. One of the most prestigious scientific magazines, Science, has recently published on the topic of Roboethics and numerous papers on Machine Ethics and Cyborg Ethics have been published in recent years in other prestigious journals.
With increased acceptance will come possibility to publish in many mainstream academic venues and we call on authors and readers of this volume to start specialized peer-reviewed journals and conferences devoted to the AI safety research. With availability of publication venues more scientists will participate and will develop practical algorithms and begin performing experiments directly related to the AI safety research. This would further solidify AI safety engineering as a mainstream scientific topic of interest and will produce some long awaited answers.
In the meantime we are best to assume that the AGI may present serious risks to humanity’s very existence and to proceed or not to proceed accordingly. A quote from a paper by Samuel Butler which was written in 1863 and amazingly predicts the situation in which humanity has found itself :
“Day by day, however, the machines are gaining ground upon us; day by day we are becoming more subservient to them; … Every machine of every sort should be destroyed by the well-wisher of his species. Let there be no exceptions made, no quarter shown; let us at once go back to the primeval condition of the race. If it be urged that this is impossible under the present condition of human affairs, this at once proves that the mischief is already done, that our servitude has commenced in good earnest, that we have raised a race of beings whom it is beyond our power to destroy, and that we are not only enslaved but are absolutely acquiescent in our bondage.”
These challenges may seem visionary, but it seems predictable that we will encounter them; and they are not devoid of suggestions for present‐day research directions.
Allen, C., Wallach, W., & Smit, I. (2006). Why Machine Ethics? IEEE Expert / IEEE Intelligent Systems. doi:10.1109/MIS.2006.83
Hall, J. S. (2007). Self-improving AI: an Analysis. Minds and Machines. doi:10.1007/s11023-007-9065-3
International Conference in Information, Communication, and Automation Technologies 2011 : Sarajevo, Bosnia Herzegovina. (2011). 2011 XXIII International Symposium on Information, Communication and Automation Technologies, ICAT 2011: 27-29 October 2011, Sarajevo, Bosnia Herzegovina. Piscataway, NJ: IEEE.
International Symposium on Neural Networks (8th : 2011 : Guilin Diqu, China). (2011). Advances in neural networks–ISNN 2011: 8th International Symposium on Neural Networks, ISNN 2011, Guilin, China, May 29-June 1, 2011, proceedings. Berlin: Springer.
McCarthy, J., Minsky, M., Rochester, N., & Shannon, C. E. (2006). A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence, August 31, 1955.
Muller, Vincent C. (2013). Philosophy and theory of artificial intelligence. Berlin: Springer.
Yampolskiy, R. V., & Govindaraju, V. (2012). Artificial Intelligence Safety Engineering: Why Machine Ethics Is a Wrong Approach. doi:10.1504/IJBM.2008.018665