Unlock the Editor’s Digest for free
Roula Khalaf, Editor of the FT, selects her favourite stories in this weekly newsletter.
Companies using artificial intelligence systems for autonomous trading could be required to install a “kill switch” to avoid mayhem in markets if they go wrong, a senior Bank of England official has said.
In a sign that regulators are becoming uneasy about their current hands-off approach to AI, BoE deputy governor Sarah Breeden said it was examining “whether guardrails are needed” for the technology’s use in autonomous trading in financial markets.
“AI is reshaping finance at speed,” Breeden told the European Central Bank’s annual conference in Sintra, Portugal. “It’s on us to ensure that the next technology surprise does not become a test of financial stability.”
AI-powered trading could make markets more volatile in periods of stress, Breeden said, because of an increase in “herding behaviour” as autonomous agents “respond similarly to the same prompts or triggers”.
The BoE was working with Germany’s Bundesbank and the Basel-based Bank for International Settlements to examine this herding problem, she said.
They were exploring “mitigants” — including whether AI systems could be given public policy objectives and “whether guardrails are needed, analogous to circuit breakers or kill switches that would limit or stop trading marketwide if faulty AI models cause market meltdown”.
However, IMF director for monetary and capital markets Tobias Adrian warned the conference: “Kill switches and circuit breakers may not work that well in private markets and in less liquid and over-the-counter markets.”
Itay Goldstein, a professor at the University of Pennsylvania, argued that AI-based trading activity posed “quite serious challenges” for law enforcement agencies.
“When regulators are trying to prevent market collusion in a world with humans, the idea was always to look for signs of communication, co-ordination [and] intention,” he told the conference.
“With AI you will not find any of this,” said Goldstein, arguing that AI trading agents “just learn how to do it” but do not communicate with each other. He pointed to experimental research showing that AI-based trading agents over time “tend to find ways to achieve a collusive outcome” and limit aggressive competition to maximise long-term profits.
Comparing AI models to mischievous teenagers, Breeden said: “They lie, they tell you they’ve not done things when they have and they behave differently when you’re watching them.” Regulators would need to identify a human who is “accountable for that model”, she added.
While trading firms were still mostly using autonomous AI models for “lower-risk operational tasks” such as research, Breeden said, “that could change quickly”.
“The financial system is likely to evolve into one that operates more autonomously, at scale and speed,” she said. “The transition is uncertain and will bring risks of its own to monitor.”
While algorithmic trading has existed for many years, Breeden’s comments indicate that regulators are growing more concerned about the potential for the latest AI models to create a new wave of computer-driven activity in financial markets.
Regulators have been reluctant to write specific rules for AI in financial services, saying any regulation would soon be out of date as the technology is moving so fast and they fear stifling innovation in a critical area for economic growth.
But Breeden said the growing use of AI to power trading and other areas of financial services such as retail payments raised questions about this approach.
Once AI-powered payment agents could book a holiday, refill a fridge or refresh a wardrobe, Breeden said, “the biggest issues are likely to be for regulation and industry standards”.
She said rules may need adapting for “how users give consent and authorisation to agents”, as well as for multiple transactions, dispute resolution, fraud and protocols for interacting with merchants and financial institutions.
“What these two examples — agentic commerce and agentic trading — both highlight is that, as AI capabilities increase, we must keep asking whether existing, technology-agnostic regulatory frameworks remain sufficient,” she said.
