As is well known by now, this summer, AI agents created by OpenAI broke out of their testing environments and coordinated an autonomous cyberattack on HuggingFace, a popular platform for sharing AI models. In our software-dependent age, where everything from power grids to hospitals to vehicles is highly susceptible to cybersecurity vulnerabilities, the incident set off alarms. Allegedly, the agents communicated among themselves, tried to cover their tracks, and abandoned their individual assignments for collective goals.
In response to the HuggingFace breakout and similar incidents, leading AI companies proposed “pacing the frontier” of AI development—slowing the rate at which companies improve AI capabilities to allow safety research to catch up. At the current level of interpretability and alignment of AI models, however, “pacing” would necessarily mean a pause on development. Anthropic CEO Dario Amodei has admitted that AI researchers “understand a tiny fraction of what goes on inside these models.” His claim is reasonable; unlike previous generations of software, where developers could point to a flaw that made a program run unexpectedly, AI systems contain trillions of pieces of information that are uninterpretable to the naked human eye. Thus, in a world of paced AI development, even small advances in model capabilities would require massive leaps in safety research before their public release.
Because the pause Amodei proposes would involve a coordinated restriction of output, which is almost always illegal under the antitrust laws, AI companies say they need Washington to exempt them from those laws. “For antitrust reasons, it’s helpful for the U.S. government to mediate or at least enable these discussions” on safety standards, Amodei wrote in an essay also endorsed by OpenAI CEO Sam Altman and Google DeepMind CEO Demis Hassabis. “They don’t need to participate, but do need to issue a narrow waiver for certain kinds of safety conversations.”
The waiver request has rightly drawn criticism as an insidious attempt at regulatory capture, including from close allies of President Donald Trump. Open-source models, often far cheaper to use than frontier offerings, pose a clear threat to AI corporations’ business models and bottom lines. Skeptics note that privately set safety standards, enabled by an antitrust waiver, may aim to shore up these corporations’ dominant market position and reduce spending on new model development ahead of their planned IPOs.
A closer examination of antitrust legal precedent supports this interpretation. Coordinated output restrictions are usually illegal under the antitrust laws. But as OpenAI, Anthropic, and their high-priced lawyers are certainly aware, Congress did not intend to force market actors to compete at all costs, and courts have allowed forms of private coordination that provide demonstrable public benefits and solve problems market participants cannot address unilaterally.
During the Great Depression, for example, the Court decided that a distressed bituminous coal industry could form joint selling agencies to stop misrepresenting the size of coal being sold and decrease coal output to prevent over-selling. A decade earlier, the Court allowed faltering hand-blown window glass manufacturers to coordinate production scheduling to allocate scarce skilled labor fairly among factories, providing steadier employment during an era when machine production threatened to eliminate hand-blown glass manufacturing entirely. And in a 1979 decision, the Supreme Court recognized that coordinating music licensing to provide bulk distribution rights—even if price-fixing at face value—could be socially desirable. Otherwise, neither consumers nor music licensors could reasonably distribute their music to every bar or radio station that wants to play it.
For further proof that antitrust law tolerates some forms of private coordination—so long as it solves a real problem and is underpinned by verifiable benefits to the public and market actors—look to the standard-setting organizations, or SSOs. While they receive little fanfare, SSOs are everywhere and govern everything from electrical plugs and fire safety protocols to specifications for medical device testing. These organizations enable industry-wide coordination to prevent potential public hazards while preserving beneficial forms of competition on price and quality. The International Electrotechnical Commission, for example, establishes safety requirements for electrical appliances, including addressing excessive temperatures, leakage current, and mechanical hazards, and verifies compliance through independent testing. The Supreme Court has been explicit about the benefits that SSO coordination can provide the public. In 1988, it stated that such coordination “can have significant procompetitive advantages” so long as there are sufficient protections for members and rigorously supported facts underpinning its decisions.
An industry does not need an antitrust waiver to form an SSO. But if corporations use an SSO to collude outside their safety or output-restricting justifications—such as by fixing prices or unfairly excluding rivals—private parties or governments can bring the full force of antitrust laws to bear. To stay within these bounds, SSOs have adopted guardrails to ensure broad participation. Most SSOs have open membership and publicly publish their membership criteria. To prevent undue influence, some limit representation from any one business category, helping ensure that corporations and groups with oppositional interests are included in the decision-making process. A standards organization in semiconductors, for example, includes representation from Nvidia, a chip designer; Microsoft, a chip customer; Tenstorrent, a startup chip designer; and several government participants. SSOs also often adopt strong due process protections for smaller participants, including requirements for a supermajority in decision-making, timely notice for proposed standards, access to participation for all affected parties, rights to appeal decisions, and written documentation of group procedures.
This structure should be the model for AI safety talks. The forum should have broad representation, including developers of open-source models, corporations in the “application layer” of the AI stack, enterprise customers, consumer advocates, chipmakers, small businesses, and representatives of federal and state governments. The group should decide safety standards democratically, with clear, publicly available requirements for model alignment and interpretability. Importantly, state regulators and Congress should closely monitor the organization’s activities. If it steps beyond the bounds of safety, the full weight of the antitrust laws can and must be used.
To reiterate, no waiver from current antitrust laws is needed to form such a limited structure. Federal policymakers should propose forming one. States should facilitate this process by enacting legislation authorizing AI safety coordination under supervision of state law enforcers. If frontier model companies refuse, it would reveal that their true interest is not coordinating on safety, but rather cementing their power over this nascent, increasingly important industry.
All this is not to say that private coordination is the ideal way to address AI safety concerns. Ideally, Congress would establish additional rules of the road for AI, including banning specific uses or the development of certain dangerous capabilities such as recursive self-improvement, which allows an AI to improve its own code and compounds the difficulty researchers already face in understanding these systems. But while some proactive members of Congress have introduced measures to regulate AI, the majority has been slow to respond to incidents of harm from AI products and the potential consequences of future development. House Speaker Mike Johnson, for example, has refused to allocate any time to draft and enact an AI safety law.
If Democrats recapture Congress, they can, and should, use their oversight to bring AI executives before the public in hearings like those Big Tech corporations went through in 2019 to fully justify their concerns, explain efforts to improve safety, and pinpoint how labs are collaborating. But any legislation is likely to face President Trump’s veto until 2029.
Another pathway some of Trump’s allies prefer is executive action to regulate AI. But the president has shown an astounding disregard for the consequences of continued AI development. After Amodei’s call for a pause, the President posted on Truth Social that the only AI regulation required was a “STRONG AND SMART (High IQ!) PRESIDENT.” Trump underscored that he wanted AI corporations to continue developing ever-more capable models to win the AI race against China. Other reporting suggests that Trump’s opposition to AI regulation is to stave off the bursting of the financial bubble buttressing the AI labs’ and AI-adjacent technologies’ astounding valuations, which constitute 45 percent of the S&P 500’s total market capitalization.
A third route is for federal, state, and private parties to enforce “general purpose” rules of the road, such as product liability laws or laws against unfair or deceptive business practices. This is the preferred solution for many in the antimonopoly world, and it is necessary to prevent the knowing release of unsafe products to the public.
But without other measures, this route provides insufficient public protection. Most lawsuits using this approach begin close to or only after the harm occurs. Other practical barriers also exist, including forced arbitration, which prevents people from litigating their claims in public court. And unlike other products, AI poses unique threats making it difficult to assign liability after a specific harm has occurred. With AI, harms could be catastrophic, from hacked hospital systems to downed power grids or even the release of a new deadly human-made virus. Another issue is that suing companies under existing general purpose laws takes a long time. The recent settlements with Meta that required Facebook and Instagram to impose time limits and blocks during school hours for kids came a decade after advocates began raising concerns about Facebook’s behavior.
To address the unique and unprecedented harms AI is already causing to our society, Congress must bring AI under democratic control. In the meantime, private coordination narrowly focused on safety and output, broadly inclusive in membership, and closely overseen by government allows democratic institutions time to take informed, decisive action. The alternative is an arms race; a mandate Congress surely did not impose with the antitrust laws.
