The public tragedy of Sept. 11 dramatically shifted the focus in Washington from debates over federal privacy legislation to a mania for total information awareness, turning Silicon Valley’s innovative surveillance practices into objects of intense interest. As Jack Balkin, a professor at Yale Law School, observed, the intelligence community would have to “rely on private enterprise to collect and generate information for it,” in order to reach beyond constitutional, legal, or regulatory constraints, controversies that are central today. By 2013, the CIA’s chief technology officer outlined the agency’s mission “to collect everything and hang on to it forever,” acknowledging the internet companies, including Google, Facebook, YouTube, Twitter, and Fitbit and telecom companies, for making it possible. The revolutionary roots of surveillance capitalism are planted in this unwritten political doctrine of surveillance exceptionalism, bypassing democratic oversight, and essentially granting the new internet companies a license to steal human experience and render it as proprietary data.

Young entrepreneurs without any democratic mandate landed a windfall of infinite information and unaccountable power. Google’s founders, Larry Page and Sergey Brin, exercised absolute control over the production, organization and presentation of the world’s information. Facebook’s Mark Zuckerberg has had absolute control over what would become a primary means of global communication and news consumption, along with all the information concealed in its networks. The group’s membership grew, and a swelling population of global users proceeded unaware of what just happened.

…

To understand the economics of epistemic chaos, it’s important to know that surveillance capitalism’s operations have no formal interest in facts. All data is welcomed as equivalent, though not all of it is equal. Extraction operations proceed with the discipline of the Cyclops, voraciously consuming everything it can see and radically indifferent to meaning, facts and truth.

In a leaked memo, a Facebook executive, Andrew Bosworth, describes this willful disregard for truth and meaning: “We connect people. That can be good if they make it positive. Maybe someone finds love. … That can be bad if they make it negative. … Maybe someone dies in a terrorist attack. … The ugly truth is … anything that allows us to connect more people more often is *de facto* good.”

In other words, asking a surveillance extractor to reject content is like asking a coal-mining operation to discard containers of coal because it’s too dirty. This is why content moderation is a last resort, a public-relations operation in the spirit of ExxonMobil’s social responsibility messaging. In Facebook’s case, data triage is undertaken either to minimize the risk of user withdrawal or to avoid political sanctions. Both aim to increase rather than diminish data flows. The extraction imperative combined with radical indifference to produce systems that ceaselessly escalate the scale of engagement but don’t care what engages you.

…

The economics of surveillance capitalism begot the extractive Cyclops, turning Facebook into an advertising juggernaut and a killing field for truth. Then an amoral Mr. Trump became president, demanding the right to lie at scale. Destructive economics merged with political appeasement, and everything became infinitely worse.

Key to this story is that the politics of appeasement required little more than a refusal to mitigate, modify or eliminate the ugly truth of surveillance economics. Surveillance capitalism’s economic imperatives turned Facebook into a societal tinderbox. Mr. Zuckerberg merely had to stand down and commit himself to the bystander role.

…

We are meant to believe that the destructive effects of epistemic chaos are the inevitable cost of cherished rights to freedom of speech. No. Just as catastrophic levels of carbon dioxide in the earth’s atmosphere are the consequence of burning fossil fuels, epistemic chaos is a consequence of surveillance capitalism’s bedrock commercial operations, aggravated by political obligations and set into motion by a 20-year-old dream of total information that slid into nightmare. Then a plague came to America, turning the antisocial media conflagration into a wildfire.

…

As early as February 2020, the World Health Organization reported a Covid-19 “infodemic,” with myths and rumors spreading on social media. By March, researchers at the University of Texas M.D. Anderson Cancer Center concluded that medical misinformation related to the coronavirus was “being propagated at an alarming rate on social media,” endangering public safety.

The Washington Post reported in late March that with nearly 50 percent of the content on Facebook’s news feed related to Covid-19, a very small number of “influential users” were driving the reading habits and feeds of a vast number of users. A study released in April by the Reuters Institute confirmed that high-level politicians, celebrities and other prominent public figures produced 20 percent of the misinformation in their sample, but attracted 69 percent of social media engagements in their sample.

A study released in May by Britain’s Institute for Strategic Dialogue identified a core group of 34 extremist right-wing websites disseminating Covid disinformation or linked to established health misinformation hubs now focused on Covid-19. From January to April of 2020, public Facebook posts linking to these websites garnered 80 million interactions, while posts linking to the W.H.O.’s website received 6.2 million interactions, and the Centers for Disease Control and Prevention received 6.4 million.

An Avaaz study released in August exposed 82 websites spreading Covid misinformation reaching a peak of nearly half a billion Facebook views in April. Content from the 10 most popular websites drew about 300 million Facebook views, compared with 70 million for 10 leading health institutions. Facebook’s modest content moderation efforts were no match for its own machine systems engineered for epistemic chaos.

In October a report from the National Center for Disaster Preparedness at Columbia University estimated the number of avoidable Covid-19 deaths. More than 217,000 Americans had died. Tragically, the analysis concluded that at least 130,000 of those deaths could have been avoided. Of the four key reasons cited, details of each one, including the “lack of mask mandate” and “misleading the public,” reflect the orgy of epistemic chaos loosed upon America’s daughters and sons.

This is the world in which a deadly mysterious microorganism flourished. We turned to Facebook in search of information. Instead we found lethal strategies of epistemic chaos for profit.

…

Three Principles for the Third Decade

What unprecedented solutions can address the unprecedented harms of the epistemic coup? First, we go upstream to supply, and we end the data collection operations of commercial surveillance. Upstream, the license to steal works its relentless miracles, employing surveillance strategies to spin the straw of human experience — my fear, their breakfast conversation, your walk in the park — into the gold of proprietary data supplies. We need legal frameworks that interrupt and outlaw the massive-scale extraction of human experience. Laws that stop data collection would end surveillance capitalism’s illegitimate supply chains. The algorithms that recommend, microtarget and manipulate, and the millions of behavioral predictions pushed out by the second cannot exist without the trillions of data points fed to them each day.

Next, we need laws that tie data collection to fundamental rights and data use to public service, addressing the genuine needs of people and communities. Data is no longer the means of information warfare waged on the innocent.

Third, we disrupt the financial incentives that reward surveillance economics. We can prohibit commercial practices that exert demand for rapacious data collection. Democratic societies have outlawed markets that trade in human organs and babies. Markets that trade in human beings were outlawed, even when they supported whole economies.

These principles are already shaping democratic action. The Federal Trade Commission initiated a study of social media and video-streaming companies less than a week after filing its case against Facebook and said it intended to “lift the hood” of internal operations “to carefully study their engines.” A statement by three commissioners took aim at tech companies “capable of surveilling and monetizing … our personal lives,” adding that “too much about the industry remains dangerously opaque.”

Groundbreaking legislative proposals in the European Union and Britain will, if passed, begin to institutionalize the three principles. The E.U. framework would assert democratic governance over the largest platforms’ black boxes of internal operations, including comprehensive audit and enforcement authority. Fundamental rights and the rule of law would no longer vaporize at the cyberborder, as lawmakers insist on “a safe, predictable, and trusted online environment.” In Britain the Online Harms Bill would establish a legal “duty of care” that would hold the tech companies responsible for public harms and include broad new authorities and enforcement powers.

Two sentences often attributed to Justice Brandeis feature in the congressional subcommittee’s impressive antitrust report. “We must make our choice. We may have democracy, or we may have wealth concentrated in the hands of a few, but we cannot have both.” The statement so relevant to Brandeis’s time remains a pungent commentary on the old capitalism we know, but it ignores the new capitalism that knows us. Unless democracy revokes the license to steal and challenges the fundamental economics and operations of commercial surveillance, the epistemic coup will weaken and eventually transform democracy itself. We must make our choice. We may have democracy, or we may have surveillance society, but we cannot have both. We have a democratic information civilization to build, and there is no time to waste.

Vir: Shoshana Zuboff, New York Times

Shoshana Zuboff is a professor emeritus at Harvard Business School and the author of “The Age of Surveillance Capitalism.”