AI is “not open in any sense,” the battle over encryption is far from won, and Signal’s principled (and uncompromising) approach may complicate interoperability efforts, warned the company’s president, Meredith Whittaker. But it’s not all bad news.
(Actually, it is all bad news, because I wrote up the good news separately.)
Speaking onstage with me at StrictlyVC LA, Whittaker called out a resurgence of legislative attacks on encryption as “magical thinking.”
“We’re seeing a number of, I would say, parochial and very politically motivated pieces of legislation often indexed on the idea of protecting children. And these have been used to push for something that’s actually a very old wish of security services, governments autocrats, which is to systematically backdoor strong encryption,” said Whittaker. “Often, I believe, pushed by well-meaning people who just don’t have the knowledge or education to understand the implications of what they’re doing, that could, you know, fundamentally eliminate the ability to communicate privately digitally.”
Ironically, or perhaps cynically, one of the animating factors has been a decade of calls for tech companies to take more responsibility.
“The overall theme I’m seeing is a deep desire for accountability in tech, which we saw sort of animated mid-2010s. That, then, has been weaponized, and I think we’re seeing surveillance wine in accountability bottles,” she said.
“‘Accountability’ looks like more monitors, more oversights, more back doors, more elimination of places where people can express or communicate freely, instead of actually checking on the business models that have created, you know, massive platforms whose surveillance advertising modalities can be easily weaponized for information ops, or doxing, or whatever it is, right? There’s an unwillingness to hit at the root of the problem. And instead, what we see is effectively proposals to extend surveillance into government and NGO sectors in the name of accountability.”
One such proposal comes via the Investigatory Powers Act in the United Kingdom, under which the government there threatens to prevent any app updates — globally — that it deems a threat to its national security.
“[The IPA] is effectively claiming for the U.K. the ability to demand that any tech company, across all jurisdictions, check in with the U.K. government before you ship a security patch, because they may be exploiting that patch somewhere for some business they want to keep doing. It’s a form of, again, parochial, magical thinking here,” said Whittaker.
“It’s very dangerous because we are being threatened to a return before the liberalization of encryption in 1999, kind of an early ’90s paradigm where the government has a monopoly on encryption and the right to digital privacy. And where the ability to deploy encryption or privacy updates or anything that would secure and harden your service becomes something you have to get permission from the government to do.”
“And honestly,” she added, “I think we need the VC community, and the larger tech companies more involved in naming what a threat this is to the industry, and pushing back.”
One bit of regulation that might seem to make sense is the messaging interoperability mandate being pursued in the EU via the Digital Markets Act. But this, too, has hidden perils.
“I think the spirit makes a lot of sense. But of course Signal can’t interoperate with another messaging platform, without them raising their privacy bar significantly,” even ones like WhatsApp that support end-to-end encryption and already partly utilize the protocol. “Because we don’t just encrypt the contents of messages using the Signal protocol. We encrypt metadata, we encrypt your profile name, your profile photo, who’s in your contact list, who you talk to, when you talk to them. That would need to be the level of privacy and security agreed across the board with anyone we interoperated with before we could consent to interoperate.”
There’s a risk, she explained, that the opposite would happen, watering down security and privacy in the name of convenience. “It could actually drop the standard of privacy, creating kind of an interoperating monolith that further relegates those who are demanding a standard of privacy with a lot of integrity to a more marginal position.” (Incidentally, she ridiculed the idea of Apple getting a pass and leaving any such regime hopelessly fragmented.)
In the private sector, Whittaker was quick to call the ascendant Nvidia a monopolist.
“It’s the chip monopoly — and the CUDA monopoly,” she said, referring to the proprietary computational architecture at the heart of much high-performance computing today.
How much are Nvidia’s rivals investing in startups? We investigated
I asked if she thinks the company has become dangerous in its accumulation of power.
“I mean, we have a lot of Spider-Men pointing at each other, right? I’m seeing Microsoft pointing fingers at Nvidia now, and saying, if you’re worried about monopoly, do not look to poor Microsoft, look to Nvidia, they’re the ones, and you also look to Google. Google put out this sort of PR missive last week, kind of their AI access principles, and they talked about Google being the only vertically integrated company from app store to chips. And that’s true, right? But then Google published a couple of days later, like Microsoft is actually the monopoly because it has the OpenAI and sort of the Azure monopoly, right?
“So like, no one is innocent here. There’s a lot of like, ‘We’re all trying to find the guy who did this …’” (i.e., the famous ‘hot dog guy’ meme lifted from “I Think You Should Leave with Tim Robinson”).
“I think we need to recognize that AI is dependent on Big Tech. It requires Big Tech resources. It is not open in any sense,” she said. “We can be honest that, if you need $100 million for a training run, that is not an open resource, right? If you need $100 million to deploy at scale for a month, that is not open, right? So we need to be honest about how we’re using these terms. But I don’t want the deflection toward Nvidia as the culprit of the week to detract from what we’re dealing with this massively concentrated power.”
You can watch the full interview below.
Comment