At least 113 current and former employees of leading AI companies have signed an open letter in support of SB 1047, the bitterly contested AI safety bill authored by state Sen. Scott Wiener. The letter, published early Monday, revealed that more than three dozen signatories openly contradict their employers’ official stance against the bill.
SB 1047 would establish liability for developers of AI models that cause a catastrophe if the developer didn’t take appropriate safety measures. The legislation would apply only to developers of models that cost at least $100 million to train and do business in California, the world’s fifth-largest economy.
The bill recently passed both houses of California’s Legislature with strong majorities. It now sits with Gov. Gavin Newsom, who has until Sept. 30 to sign or veto it.
SB 1047 has exposed rifts in San Francisco and the wider tech world, scrambling the usual political coalitions. Opponents include Speaker Emerita Nancy Pelosi, San Francisco Mayor London Breed, AI “godmother” Fei-Fei Li, and Trump-supporting venture capitalists like Marc Andreessen and Ben Horowitz.
Proponents include Elon Musk (who called Wiener a “pedophile-apologist” in July), AI “godfathers” Geoffrey Hinton and Yoshua Bengio, and the progressive youth advocacy group Encode Justice.
A common complaint from critics is that SB 1047 is well-intentioned but not informed by experts. One prominent venture capitalist wrote in a Financial Times op-ed that “the bill is a fundamental misunderstanding of the technology.”
The employee letter — which was signed by heavy hitters, including Hinton, the former Google Brain researcher and Turing Award winner; Anthropic co-founder Chris Olah; former OpenAI researcher Jan Leike; and OpenAI whistleblowers Daniel Kokotajlo and William Saunders — represents a major challenge to this claim.
The brief letter warns “that the most powerful AI models may soon pose severe risks, such as expanded access to biological weapons and cyberattacks on critical infrastructure.” The signatories argue that reasonably safeguarding against these harms is “feasible and appropriate” for frontier AI companies. They conclude that SB 1047 “represents a meaningful step forward” and recommend that Newsom sign it into law.
Thirty-seven signatories are employees of companies opposing the bill: 19 work for Google DeepMind, 15 are at OpenAI, and three are at Meta. In total, 70 people who signed the letter are staff or alumni of companies opposing SB 1047, with 13 from Google DeepMind and five from OpenAI; 25 are anonymous.
Perhaps the biggest surprise is the willingness of 10 OpenAI employees to come out openly in opposition to their company’s stance. In spite of CEO Sam Altman’s past rhetorical support for AI regulation, OpenAI has taken draconian steps to ensure silence from its employees and alumni. In May, Vox reported that the company held outgoing employees’ vested equity hostage in exchange for lifetime NDA and non-disparagement agreements. Following external and internal outcry, Altman apologized and claimed ignorance, and OpenAI later scrapped the agreements.
While staff from all the top AI companies signed a May 2023 letter stating that AI poses an existential risk, Monday’s letter may be the first time AI employees have publicly supported a concrete piece of legislation formally opposed by their employers.
Nearly half of the letter’s signatories were Anthropic employees, nine of whom previously worked at OpenAI. In the fallout from Altman’s short-lived November firing and the equity clawback revelations, OpenAI has experienced an exodus of safety researchers.
Anthropic, creator of the Claude family of chatbots, has been the most common destination for these departing researchers. This trend mirrors the origin of the company, which was founded by an earlier round of safety-concerned OpenAI defectors.
The first major crack in the AI industry’s near-uniform resistance to SB 1047 came in late August, when Anthropic published an open letter stating that the revised bill’s “benefits likely outweigh its costs.” (SB 1047 was amended to address many of the company’s earlier concerns.)
Musk’s xAI hasn’t taken a formal position on the bill, but the founder and CEO broke his silence with an endorsement in late August.
It remains to be seen if the letter will be enough to sway Newsom, who was seated between Pelosi and Congresswoman Zoe Lofgren at the Democratic National Convention in August. Lofgren, who represents parts of Silicon Valley, organized a letter against SB 1047 signed by seven other California House Democrats. AI investors opposed to the bill have also hired lobbyists with deep ties to the governor.
Pelosi’s opposition to SB 1047, combined with her strange pattern of endorsements in San Francisco elections, has fueled speculation that she is attempting to create momentum for her daughter Christine, who is rumored to be weighing a run against Wiener for Pelosi’s congressional seat upon her retirement. The former House speaker has vigorously disputed this narrative, and her office declined to comment.
Copyright for syndicated content belongs to the linked Source link
