Battle of the Minds : 無料・フリー素材/写真
Battle of the Minds / jurvetson
| ライセンス | クリエイティブ・コモンズ 表示 2.1 |
|---|---|
| 説明 | Both books describe the AI revolution and technology’s ability to shift geopolitical power, but draw nearly opposite conclusions.Suleyman argues for robust AI containment, rife with dystopian fears and surveillance solutions. He is CEO of Microsoft AI, and formerly Google Deep Mind (where at both companies employee protests shut down work on military systems).Karp, in stark contrast, argues that restraint is misguided, and America should “without delay commit to launching a new Manhattan Project in order to retain exclusive control over the most sophisticated forms of AI for the battlefield.” He is CEO of Palantir. Here are the excerpts that summarize their AI arguments, starting with similar premises but driving to divergent recommendations:The Technological RepublicOpening line: “SILICON VALLEY HAS LOST ITS WAY.”“A generation of founders cloaked themselves in the rhetoric of lofty and ambitious purpose — indeed their rallying cry to change the world has grown lifeless from overuse — but often raised enormous amounts of capital and hired legions of talented engineers merely to build photo-sharing apps and chat interfaces for the modern consumer. A skepticism of government work and national ambition took hold in the Valley. Startup after startup catered to the whims of late capitalist culture without any interest in constructing the technical infrastructure that would address our most significant challenges as a nation.” (9)Why? “The most capable generation of coders have never experienced a war or genuine social upheaval.” (10) “The current generation of spectacularly talented engineering minds has become unmoored from any sense of national purpose or grander and more meaningful project.” (11)“The causes of this turn away from defending the American national project, we argue, include the systematic attack and attempt to dismantle any conception of American or Western identity during the 1960s and 1970s. The dismantling of an entire system of privilege was rightly begun. But we failed to resurrect anything substantial, a coherent collective identity or set of communal values, in its place.” (13)“In this book, we make the case that the technology sector has an affirmative obligation to support the state that made its rise possible.” (11)The Software Century “The newest forms of artificial intelligence, known as large language models, have for the first time in history pointed to the possibility of artificial general intelligence (AGI) — that is, a computing intellect that could rival that of the human mind when it comes to abstract reasoning and solving problems. It is not clear however that the companies building these new forms of AI will allow them to be used for military purposes. We make the case that one of the most significant challenges that we face in this country is ensuring that the U.S. Department of Defense turns the corner from an institution designed to fight and win kinetic wars to an organization that can design, build, and acquire AI weaponry — the unmanned drone swarms and robots that will dominate the coming battlefield.” (12)“The United States since its founding has always been a technologic republic, one whose place in the world has been made possible and advanced by its capacity for innovation.” (15)“An unwinding of the skepticism of the American project will be necessary to move forward. We must bend the latest and most advanced forms of AI to our will, or risk allowing our adversaries to do so as we examine and debate, sometimes it seems endlessly, the extent and character of our divisions. Our central argument is that—in this new era of advanced AI, which provides our geopolitical opponents the greatest opportunity since the last world war to challenge our global standing—we should return to that tradition of close collaboration between the technology industry and the government. It is that combination in pursuit of innovation with the objectives of the nation that will not only advance our welfare but safeguard the legitimacy of the democratic project itself.” (15)“We have now, nearly eighty years after the invention of the atomic bomb, arrived at a similar crossroads in the science of computing, a crossroad that connects engineering and ethics, where we will again have to choose whether to proceed with the development of a technology whose power and potential we do not yet fully apprehend.” (18) “It is not at all clear—not even to the scientists and programmers who build them—how or why the generative language and image models work.” (19)“The risks of proceeding with the development of artificial intelligence have never been more significant. Yet we must not shy away from building sharp tools for fear they might be used against us. The potential integration of weapons systems with increasingly autonomous AI software necessarily brings risks, which are only magnified by the possibility that such programs might develop a form of self-awareness and intent. But the suggestion to halt the development of these technologies is misguided. It is essential that we redirect our attention toward building the next generation of AI weaponry that will determine the balance of power in this century, as the atomic age ends, and the next.” (26)“This next era of conflict will be won or lost with software. One age of deterrence, the atomic age, is ending, and a new era of deterrence built on AI is set to begin. The risk, however, is that we think we have already won.” (28)“The decisive wars of the future will be driven by artificial intelligence, whose development is proceeding on a far different, and faster timeline than in the past. A fundamental reversal of the relationship between hardware and software is taking place. For the 20th century, software has been built to maintain and service the needs of hardware, from flight controls to missile avionics, and fueling systems to armored personnel carriers. With the rise of AI and the use of large language models on the battlefield to metabolize data and make targeting recommendations, however the relationship is shifting. Software is now at the helm, with the hardware—the drones on the battlefields of Europe and elsewhere—increasingly serving as the means by which the recommendations of AI are implemented in the world.” (45)“Yet the level of investment in such technologies, and the software systems that will be required for them to operate, is far from sufficient (at 0.2% of the defense budget). The U.S. government is still focused on developing a legacy infrastructure—the planes, ships, tanks, and missile—that delivered dominance of the battlefield in the last century but will almost certainly not as central in this one.” (45)“Other nations, including many of our geopolitical adversaries, understand the power of affirming shared cultural traditions, mythologies, and values in organizing the efforts of people. They are far less shy than we are about acknowledging the human need for communal experience.” (217)“What we need is more cultural specificity—in education, technology and politics—not less. The vacant neutrality of the current moment risks allowing our instinct for discernment to atrophy. We must now take seriously the possibility that it will be the resurrection of a shared culture, not its abandonment, that will make possible our continued survival and cohesion.” (216)----------------------------And from The Coming Wave:“Having been up close to this unfurling revolution over the last decade and a half, I am convinced that we’re on the cusp of the most important transformations of our lifetimes.” (16) “We really are at a turning point in the history of humanity.” (78)“At the heart of the coming wave lie two general purpose technologies of immense promise, power, and peril: artificial intelligence and synthetic biology.” (17)“Four key features that explain why this isn’t business as usual: these technologies are inherently general and therefore omni-use, they hyper-evolve, they have asymmetric impacts, and in some respects, they are increasingly autonomous.” (17)“The foundation of our present political order will be further weakened by a series of shocks amplified by the wave: the potential for new forms of violence, a flood of misinformation, disappearing jobs, and the prospect of catastrophic accidents.” (17)“The coming wave of technologies threatens to fail faster and on a wider scale than anything witnessed before. Containment is not, on the face of it, possible. And yet, containment must be possible. (19)“Proliferation of new technology is the default. Civilization’s appetite for useful and cheaper technologies is boundless. This will not change.” (31)“History tells us that technology diffuses, inevitably, eventually to almost everywhere, from the first campfires to the fires of the Saturn V rocket. Incentives are overwhelming. Capabilities accumulate; efficiencies increase. Waves get faster and more consequential.” (34)“Technology’s unavoidable challenge is that its makers quickly lose control over the path their inventions take once introduced to the world.” (35) “Thus, technology’s problems have a tendency to escalate in parallel with its capabilities, and so the need for containment grows more acute over time.” (36)HAVE WE EVER SAID NO?“Unhappy at the prospect of unregulated mass production of knowledge and culture, the Ottoman empire tried to ban it. Istanbul did not possess a sanctioned printing press until 1727, nearly three centuries after its invention.” (38)“Technologies are ideas, and ideas cannot be eliminated.” (41) “For most of history, the challenge of technology lay in creating and unleashing its power. That has now flipped; the challenge of technology is about containing its unleashed power, ensuring it continues to serve us and our planet.” (48)A CAMBRIAN EXPLOSION“Technology is a set of evolving ideas. New technologies evolve by colliding and combining with other technologies. Invention is a cumulative, compounding process. It feeds on itself.” (56) “General-purpose technologies are accelerants. Invention sparks invention.” (92)“Of course all programs in the future will ultimately be written by AIs, with humans relegated to, at best, a supervisory role.” (69)(He then gives a cursory nod to robotics, quantum computing, and fusion energy as amplifiers of the mega wave)“The coming decades will be defined by the convergence of biology and engineering. At the center of this wave sits the realization that DNA is information, a biologically evolved encoding and storage system.” (79) “Already genetically engineered organisms account for 2% of the U.S. economy through agricultural and pharmaceutical uses.” (88)ASYMMETRY: A COLOSSAL TRANSFER OF POWER“Never before have so many had access to such advanced technologies capable of inflicting death and mayhem.” (106)“Over time, technology tends toward generality. What this means is that weaponizable or harmful uses of the coming wave will be possible regardless of whether this was intended. Simply creating civilian technologies has national security ramifications. What’s different about the coming wave is how quickly it is being embedded, how globally it spreads, how easily it can be componentized into swappable parts, and just how powerful and above all broad its applications could be. It unfurls complex implications for everything from media to mental health, markets to medicine. This is the containment problem supersized.” (112)“A paradox of the coming wave is that its technologies are largely beyond our ability to comprehend at a granular level yet still within our ability to create and use. In AI, the neural networks moving toward autonomy are, at present, not explainable.” (114) (I have described this process of creation as more akin to parenting than programming.)“In China, Go wasn’t just a game. It represented a wider nexus of history, emotion, and strategic calculation. AlphaGo [AI-built with RL, beating a human champion] helped focus government minds even more acutely on AI. Today, China has an explicit national strategy to be the world leader in AI by 2030 ‘making China the world’s primary innovation center’ from defense to smart cities” (120)“In terms of volume of AI research, Chinese institutions have published a whopping 4.5x more AI papers than U.S. counterparts since 2010, and comfortably more than the U.S., U.K., India and Germany combined.” (121)“China installs as many robots as the rest of the world combined. It built hypersonic missiles thought years away by the U.S. In 2014, China filed the same number of quantum technology patents as the U.S.; by 2018 it filed twice as many.” (122)“Shortly after becoming President in 2013, Xi Jinping made a speech with lasting consequences. ‘Advanced technology is the sharp weapon of the nation state,’ he declared. ‘Our technology still generally lags that of developed countries, and we must adopt an asymmetric strategy of catching up and overtaking.’” (123) “Any world leader could make the same point. Technology has become the world’s most important strategic asset” (124)“In the Manhattan Project, America had conducted an arms race against phantoms, bringing nuclear weapons into the world far earlier than other circumstances." (126)FRAGILITY AMPLIFIERS:"Technology is ultimately political because technology is a form of power. Wherever power is today, it will be amplified. Whether it’s commercial, religious, cultural, or military, democratic or authoritarian, every possible motivation you can think of can be dramatically enhanced by having cheaper power at your fingertips. This will be the greatest, most rapid accelerant of wealth and prosperity in human history.” (164)“The cost of military drones has fallen 1000x over the last decade. AI-enhanced weapons will improve themselves in real time. AI cyberweapons will continuously probe networks, adapting themselves autonomously to find and exploit weaknesses… a worm that improves itself using reinforcement learning, experimentally updating its code with each network interaction” (166)“Now powerful, asymmetric, omni-use technologies are certain to reach the hands of those who want to damage the state. The nature of the features favors offense: this proliferation of power is just too wide fast and open.”“A Carnegie Mellon study analyzed more than 200 million tweets discussing COVID-19 at the height of the first lockdown. 82% of influential users advocating for ‘reopening America’ were bots. This was a targeted propaganda machine, most likely Russian, designed to intensify the worst public health crisis in a century.” (172)“More than half of all jobs could see many of their tasks automated by machines in the next seven years. Automation is unequivocally another fragility amplifier.” (179)“The history of humanity is, in part, a history of catastrophe. Pandemics feature widely. Two killed up to 30% of the world population” (205) “We know what a lab leak might look like in the context of amplifying fragility… the omnicron variant of COVID infected a quarter of Americans within 100 days of first being identified.” (209)THE DYSTOPIAN TURN“Technology has penetrated our civilization so deeply that watching technology means watching everything. With the architecture of monitoring and coercion being built in China and elsewhere, the first steps have arguably been taken. If zombielike states will sleepwalk into catastrophe, their openness and growing chaos a petri dish for uncontained technology, authoritarian states are already gladly charging into this techno-dystopia, setting the stage, technologically if not morally, for massive invasions of privacy and curtailments of liberty. And on the continuum between the two, there is a chance of the worst of all worlds: scattered but repressive surveillance and control apparatuses that still don’t add up to a watertight system. Catastrophe and dystopia.” (217)“Make no mistake: standstill spells disaster. I think it’s easy to discount how much of our way of life is underwritten by constant technological improvements. A moratorium on technology is not a way out; it’s an invitation to another kinds of dystopia, another kind of catastrophe. Even if it were possible, the idea of stopping the coming wave isn’t a comforting thought. Maintaining, let alone improving, standards of living needs technology. Forestalling a collapse needs technology. The costs of saying no are existential. And yet every path from here brings grave risks and downsides. This is the great dilemma.” (221)“For progress there is no cure. Any attempt to find automatically safe channels for the present explosive variety of progress must lead to frustration.” — John von Neumann in 1955CONTAINMENT MUST BE POSSIBLE“On paper, regulation looks enticing, even obvious and straightforward; suggesting it lets people sound smart, concerned, and even relieved. It’s a simple way to shrug off the problem. It’s also the classic pessimism-averse answer. As we have seen, governments face multiple crises independent of the coming wave—declining trust, entrenched inequality, polarized politics, to name a few.” (226)After this thoughtful discussion of the problems facing us, the final 40 pages on containment remedies rang hollow to me, more wishful thinking than implementable solutions to the grand dilemma. Rather than quote them all, I will list:Narrow AI instead of general systems that are harder to contain. An “Apollo program for technical safety.” More safety researchers. Automating alignment research. Resource caps on training compute. Crypto-protecting model weights limiting how widely they could be copied. Bulletproof off switch. “Audits are critical to containment.” “Keeping close tabs on significant data sets that are used to train models.” KYC for AI API access. Scan for harmful code. “Encrypted back doors” (!) Buy time with choke points: “China spends more on importing chips than it does on oil.” (249) “Skills too are a choke point: the number of people working on all the frontier technologies discussed in this book is probably no more than 150,000.” (251) A new generation of corporations. Heavier government involvement: “I think the government needs to get way more involved, back to building real technology, setting standards, and nurturing in-house capability.” (259) but… he then suggests that the government “above all needs to log all the ways technology causes harm—tabulate every lab leak, every cyberattack, every language model bias, every privacy breach—in a publicly transparent way so everyone can learn from failures and improve.” (260) Licensing labs to restrict access. Overhaul taxation “to fund security and welfare as we undergo the largest transition of value creation¬—from labor to capital—in history. If technology creates losers, they need material compensation.” (261) A new tax on robots and autonomous systems. UBI. New world government entities – a “World Bank for biotech or a UN for AI” Precautionary principles: “pause before building, pause before publishing” a “Pandemic Test Ban Treaty to stop working with pathogens or gain-of-function research.“Technology is not a niche; it is a hyper object dominating human existence.” (236)“The wave and its central dilemma need containment, need an intensified, unprecedented, all-too-human grip on the entire technosphere. It will require epic determination over decades across the spectrum of human endeavor.” (286)“Looking at the myriad paths forward, it seems containment fails in many of them. The narrow path must be walked forever from here on out, and all it takes is one misstep to tumble into the abyss. The blunt challenge of containment is not a reason to turn away; it’s a call to action, a generational mission we all need to face.” (278) |
| 撮影日 | 2025-03-09 14:32:41 |
| 撮影者 | jurvetson , Los Altos, USA |
| タグ | |
| 撮影地 | |
| カメラ | iPhone 14 Pro Max , Apple |
| 露出 | 1/5000 sec |
| 開放F値 | f/1.8 |