• California debates the dangers of artificial intelligence.
• Companies not to be liable if AI at fault in emergencies.
• Bill SB 1047 vetoed by state governor.

An attempt by Californian legislators to force AI companies to implement guardrails into their products has been vetoed by the state’s governor, Gavin Newsom.

Newsom said that the proposed bill would have given Californians, “a false sense of security about controlling this fast-moving technology,” in a statement following his decision.

The bill was originally proposed by Senator Scott Wiener, and would have required firms to have been able to turn off their platforms if they were instrumental in causing major disruption to essential services such as power generation. With the dangers of artificial intelligence front of mind, the bill would also have forced developers of AI systems to submit details of their safety plans to the state attorney general. Additionally, developers would have been held liable in the event of a machine learning model causing harm or significant threat to public safety.

Although the bill received the support of the Californian state Assembly and passed with a majority, an overturn of Newsom’s veto would require a two-thirds majority by legislators, which is considered to be unlikely.

Opponents of bill cited the possibility that large technology companies would incorporate outside of California, thereby limiting innovation born and nurtured in the place regarded as the cradle of technology in the US. The bill was opposed by national Democratic congressional leaders as well as companies that included Meta and OpenAI. Support for the law came from sources both expected and unlikely, including Elon Musk, AI company Anthropic, and pressure groups such as the Center for AI Safety.

Speaking to the LA Times after the bill had been passed by the California state Assembly, and before the veto, Professor Yoshua Bengio from the Univerité de Monteréal, said, “I worry that technology companies will not solve the significant risks associated with AI on their own because they’re locked in their race for market share and profit maximisation.”

Governor Newsom’s statement on his veto said “the bill applies stringent standards to even the most basic functions […] I do not believe this is the best approach to protecting the public from real threats posed by the technology.”

In lieu of the measures proposed by the legislation, Newsom announced on Sunday that he has enlisted industry leaders to help address the dangers of artificial intelligence by creating protective measures for the technology’s use, in an exercise that to many will seem like requesting ideal chicken coup measurements from neighbourhood foxes. Newsom’s statement did not mirror the concerns of many of the bill’s opponents, which cited concerns about the stifling of innovation. Rather, he concentrated on the fact that the bill’s contents would impact smaller AI model creators in addition to those of the big players.

“[It] does not take into account whether an AI system is deployed in high-risk environments, involves critical decision-making or the use of sensitive data,” he said in the statement.

What seems to be likely, therefore, is that AI models may well be used inside the state, for instance, as part of control systems for critical infrastructure. Should the AI happen to misfire or be thought to be in some way responsible for a serious incident, the makers of the AI in question would not be held accountable.

In an interview at Dreamforce, a San Francisco technology conference held in September, Newsom said, “We’ve been working over the course of the last couple years to come up with some rational regulation that supports risk-taking, but not recklessness.”

Dario Amodei, the CEO of Anthropic, said in a letter to Newsom dated 21 August, that he thought the bill’s pro’s outweighed its cons, and that it “addresses real and serious concerns with catastrophic risk in AI systems.”

While Amodei did not state his outright support for the legislation, coded SB 1047, Elon Musk was more clear. In a post on the platform he owns, he said: “This is a tough call and will make some people upset, but, all things considered, I think California should probably pass the SB 1047 AI safety bill. For over 20 years, I have been an advocate for AI regulation, just as we regulate any product/technology that is a potential risk to the public.”

Newsom’s desire for guardrails that support risk-taking but not recklessness seem to imply that the risks from inaccurate data surfacing and the occasional hallucination are acceptable, but it would be reckless to rely on AI for anything particularly important, such as controlling a US state’s water supply. With luck, anyone tempted to deploy an AI platform in mission critical systems will remember his words.

Want to learn about the IoT from industry leaders? Check out IoT Tech Expo taking place in Amsterdam, California, and London. The comprehensive event is co-located with other leading events including Cyber Security & Cloud Expo, AI & Big Data Expo, Intelligent Automation Conference, Edge Computing Expo, and Digital Transformation Week.

Explore other upcoming enterprise technology events and webinars powered by TechForge here.