An Article by Dirk Helbing (ETH Zurich and TU Delft)
Artificial Intelligence (AI) can help us in many ways. Particularly when combined with robotics, AI can make our everyday life more comfortable. It can perform hard, dangerous and boring work for us. It can help us to save lives and cope with disasters more successfully. So far, however, any technology came along with side effects and risks. As I will show, people may lose self-determination and democracy, companies may lose control, and nations may lose their sovereignty, if we do not pay attention. In the following, I describe a worst-case and a best-case scenario to illustrate that our society is at a crossroads. It is crucial now to take the right path.[1]
According to Ray Kurzweil, a technology guru in the Silicon Valley working for the Google Brain project, computers will surpass the capacity of the human brain before 2030. Such forecasts have long been considered science fiction. But now there are deep learning algorithms, and AI can learn by itself, making explosive progress.
Since a number of decades already, computers are better at playing chess. In the meantime, they are better in almost all strategic games people like to play. Now, IBM's Watson computer is able to win game shows - and not only this. Watson is also better in coming up with many medical diagnoses. Moreover, about 70 percent of all financial trades are performed by autonomous computer algorithms. Soon, we may use self-driving cars that drive better than humans. Algorithms come also increasingly close to human abilities in recognizing handwritings, listening to languages, translating them, and identifying patterns. As 90 percent of today's jobs are based on these abilities, it will soon be possible to replace all routine jobs by computer algorithms or robots, which will perform better, never get tired, never complain, and will not have to pay social insurance or taxes.
Jim Spohrer at IBM thinks: While AI will be our tool in the beginning, robots may soon be our teammates, and then our coaches. Steve Wozniak, the co-founder of Apple, put it like this:[2] "... I agree that the future is scary and very bad for people. If we build these devices to take care of everything for us, eventually they'll think faster than us and they'll get rid of the slow humans to run companies more efficiently."
Are people ready for this? No, we are not, but we need to get ready for the impending challenges as soon as possible. Let me start with a worst-case scenario, before I discuss a best-case one.
A worst-case scenario
In the past, whenever people have raised concerns that AI may take over the world, experts said we could always pull the plug, and this would solve the problem. Unfortunately, this is not true. My main concern is not that AI might take over the world. It is rather that a few people might try to use AI technology to take over the world.
Justified by terror threats, large amounts of personal data are now collected about every single citizen, and these are fed into AI systems, which learn how every citizen behaves.Moreover, we are literally all experimental subjects of companies such as Google and Facebook,[3] who are performing millions of automated experiments with us every day. AI systems learn how we respond to them and how these stimuli can be used to trigger certain behavioral responses.
Therefore, the trend goes from programming computers to programming people.[4] This manipulation is often so subtle that we would not even notice it. A technology, initially developed to personalize advertisements and make them more effective, has now become a tool of politics, too. "Big nudging",[5] the combination of the "nudging" approach from behavioral economics with "big data" about all of our behaviors, is being used, for example, to manipulate public opinions and election outcomes.
Ever more effective feedback mechanisms such as personalized prices are being developed. The "citizen score",[6] as it is currently implemented in China, is a further step. Here, everything people do gets plus or minus points: the shopping behavior as well as the links clicked in the Internet. The political opinion is evaluated as well as the behavior of one's social network. The citizen score will determine credit conditions, the jobs one can get, and whether one is allowed to travel to certain countries or not.
Most likely, big nudging and citizen score technologies have not only been deployed in Singapore and China. According to "nudging pope" Richard Thaler, no less than 90 countries have established "nudging units" in recent years. So far, little is publicly known about these units. Would the Singaporean or Chinese model be applied in the aforementioned 90 states, democracies worldwide might be in great danger.[7]
The problem is that a digital power grab is easily possible and hard to reverse. For example, whoever has access to a big nudging infrastructure may be able to determine the result of an election.[8] Furthermore, terror attacks or other events that traumatize the public may be used to restrict democratic principles. Given the considerable potential to inflict harm and violate basic human rights by using the technologies discussed above, we urgently need initiatives to implement the following measures as quickly as possible:
- The above instruments should be democratically controlled by the parliament.
- It also makes sense to give opposition parties access to such information systems in order to ensure a reasonable balance of power.
- The use of these tools should be based on democratic mandate and scientific principles. They should be operated by interdisciplinary teams of leading scientists. These groups need to be open to international exchange and report about their activities at public international conferences.
- Ethical oversight should also be ensured.
- Personal data should be anonymized and breaches of privacy punished.
- Transparency about on-going activities would be important. Opt-out (at least from scoring and big nudging) should be offered to ensure informational self-determination. (Note that this will also promote trustworthy uses of these methods.)
- If social experiments have caused undesirable side effects, victims should be properly compensated.
Secret services would probably want to have separate access to these information systems, but some principles should nevertheless apply:
- The use of these tools should be recorded. Large-scale nudging should be forbidden.
- The same applies to mass surveillance. Deanonymization should be limited to a small number of people and democratically controlled.
Private companies would have to follow the new European General Data Protection Regulation, and the government would have to enforce compliance not only of big IT companies, but also of the often relatively unknown companies trading with our personal data.
Democracy certainly deserves a "digital upgrade",[9] but it would be a disaster for the future of our planet, if democracy would be extinct, i.e. if we didn't have a competition between different political systems anymore
In conclusion, the self-determination of people is currently at stake, which is a major concern. Big nudging, citizen scores, and implants could lead to digitally enabled slavery. However, this is not only endangering the freedom of people. It is also endangering the sovereignty of companies and entire countries. Whoever has the most powerful AI system might be able to control all other AI systems and, in this way, all the companies, institutions and people manipulated by them. In other words, technologies intensify the race to control the world and its resources. Today, 62 people are said to control as much capital as 50 percent of people on this planet.[10] This concentration process is likely to accelerate.
I believe we should distance ourselves from "big nudging", "citizen scores" and other approaches, which may be used to control millions, perhaps billions of people in a centralized and top-down way. They could easily end in the most totalitarian regime ever. We should rather engage in a cooperative AI paradigm— human-machine symbiosis.
A best-case scenario
We are just about to step into a new era of history - the digital society and economy 4.0. If we want this transformation to succeed, it is important that we create opportunities for everyone: business, politics, science, and citizens alike. With new information and communication technologies, this can now be accomplished more easily than ever. If we manage to create a suitable legal framework, everyone could have a prosperous life. The digital economy provides almost unlimited possibilities because intangible goods can be reproduced as often as we likeand used in millions of different ways.
In fact, if we want to master the challenges humanity is faced with, our economy and society will (have to) be organized in entirely new ways. In particular, we need a resilient design and operation of our society in order to be able to flexibly adapt to unexpected events. This requires a sufficient amount of decentralization and diversity.
In the digital age, we must also reinvent innovation, from research to publication to teaching:we need to generate knowledge in real time (as much as this can be done) and share our reflections, judgments and insights more adequately, faster, and worldwide.This requires a new framework that I like to call "Plurality University".
Furthermore, we must increase innovation rates dramatically.For this, we need a fundamentally new approach to innovation that puts more emphasis on open innovation - in order to create and offer all the products and services that are currently not provided by large companies. In fact, citizen science, so-called Fablabs (public centers for communities of digital hobbyists), as well as initiatives to mobilize civil society are becoming increasingly important. The key word is co-creation, which means that citizens can augment information, knowledge, services and products in a largely open information and innovation ecosystem. Therefore, I expect that collective intelligence and co-evolution will be the organizational principles of the participatory digital society of the future, which will be based on a highly diverse and networked economy.
In this connection, the OpenAI initiative, which was recently started with a donation of 1 billion dollars, is quite remarkable. Initiator Elon Musk formulated the goals as follows: "AI should be an extension of individual human wills and, in the spirit of liberty, as broadly and evenly distributed as possible." In particular, we must design and teach AI systems to act morally and socially. In John Brockman's book "How to think about machines that think" I conclude: "[On the long run,] Intelligent machines would probably learn that it is good to network and cooperate, to decide in other-regarding ways, and to pay attention to systemic outcomes..."
We further need to think more about ways to foster the spirit of experimentation. We should encourage radically new ideas, sometimes referred to as "disruptive innovations". But the question is: how can we ensure that such innovations will lead to sustainable products that do not harm our society and environment? For this, we need to measure and price externalities. These refer to the external costs or benefits associated with products, services and interactions. Interestingly, the Internet of Things (IoT) makes it increasingly possible to do this. In addition, a multi-dimensional incentive and exchange system allows one to introduce local feedbacks that can support self-organizing systems. I call this system "finance 4.0".
In many cases, the self-organization of a complex dynamical system (such as the society, economy, or traffic flow) based on suitable interactions of its components (e.g. cars) can increase the resource efficiency by 30 to 40 percent. For such reasons, it makes sense to empower citizens by means of information and communication technologies. This puts them in a position to make better decisions and contribute more to business and society. If set up well, enabling users, customers, and citizens will lead to better services, better products, better businesses, better neighborhoods, smarter cities, and smarter societies. For example, personal digital assistants can help people to behave in a healthier and environmentally friendly way.
I believe that modern information technology can also help us to reduce conflict in the world, namely by mitigating the competition for scarce resources. This can be achieved by the combination of several measures. First, resources need to be used more efficiently, as it can be reached by considering externalities. Second, recycling techniques could be considerably advanced. Third, the principles of the sharing economy could be applied to an increasing number of areas of social and economic life, including how urban space is managed and used.
Last but not least, engaging in a "Culturepedia Project" could achieve a better understanding of the success principles, on which different cultures of the world are built, and their innovative application. The greatest social and economic potential of this approach lies directly at today's cultural fault lines. For example, suitable reputation mechanisms can promote coordination and cooperation in multi-cultural settings, and particular qualification, competition, communication and matching mechanisms are promising as well.
Some of these cultural success mechanisms will be built into the "Nervousnet" platform,[11] so that its "data for all" approach will lead to responsible use. Nervousnet (see nervousnet.info) is an open and participatory, citizen-run Internet-of-Things platform that will support (1) real-time measurements of the world around us, (2) its scientific understanding, (3) awareness of the implications of various decision alternatives, (4) real-time feedback to support self-organization, and (5) collective intelligence. The project takes informational self-determination seriously.
It is high time that we start to work on this best-case scenario. First of all, we are forced to be much more innovative than today. Second, it would bring great benefits to everyone to implement the above proposals.
It seems that the USA have already started to invest in a new strategy. They are betting on reindustrialization on the one hand, and citizen science and combinatorial innovation on the other. Even Google has embarked on a new strategy with the founding of Alphabet, which aims to make the company less dependent on personalized advertising. And Apple has recognized the value of privacy as a competitive advantage.
We are entering a digital age that increasingly frees itself of material limitations. This is absolutely fascinating. People start to realize that the digital economy is not a zero-sum game. In the area of the Internet of Things, Google has engaged in open innovation. Tesla Motors has opened up many of its patents, and many billionaires have recently promised to donate large sums of money for good. So, we see many signs of change. The only question is how long it will take until Europe will make full use of all the fantastic opportunities that the digital revolution is offering us now.
Note: Read here the full article.
[1] For further thoughts see https://www.researchgate.net/publication/291832442 and http://futurict.blogspot.ch/2016/02/how-internet-of-things-can-make.html, accessed February 15, 2016
[2] see http://www.computerworld.com/article/2901679/steve-wozniak-on-ai-will-we-be-pets-or-mere-ants-to-be-squashed-our-robot-overlords.html, accessed on January 24, 2016
[3] see, for example, http://www.wsj.com/articles/furor-erupts-over-facebook-experiment-on-users-1404085840 and http://www.pnas.org/content/111/24/8788.full, accessed on January 24, 2016
[4] see https://www.youtube.com/watch?v=KlWeuK46_nA and https://www.youtube.com/watch?v=pplhyw-vEWg, accessed on January 24, 2016
[5] http://www.spektrum.de/news/big-nudging-zur-problemloesung-wenig-geeignet/1375930, accessed on January 24, 2016
[6] http://www.computerworld.com/article/2990203/security/aclu-orwellian-citizen-score-chinas-credit-score-system-is-a-warning-for-americans.html, accessed on January 24, 2016
[7] Moreover, as the Stanford Prison Experiment has shown, any system that creates too much difference in power between those who decide and those who have to obey will sooner or later turn bad and get out of control.
[8] http://www.pnas.org/content/112/33/E4512.abstract, accessed on January 25, 2016
[9] D. Helbing and E. Pournaras, Build Digital Democracy, Nature 527, 33-34 (2015): http://www.nature.com/news/society-build-digital-democracy-1.18690
[10] http://m.spiegel.de/wirtschaft/a-1072576.html, accessed on January 24, 2016 and http://www.forbes.com/forbes/welcome/#version:realtime on January 18, 2016
[11] D. Helbing and E. Pournaras, Build Digital Democracy, Nature 527, 33-34 (2015): http://www.nature.com/news/society-build-digital-democracy-1.18690,
http://futurict.blogspot.ch/2014/09/creating-making-planetary-nervous.html,
http://futurict.blogspot.ch/2015/08/smart-data-running-internet-of-things.html, http://futurict.blogspot.ch/2016/01/nervousnet-towards-open-and.html, all accessed January 24, 2016
