Online abuse in its many forms on social media is in the spotlight these days as it has never been before: Not only can strong and hateful words be directly harmful to individuals, but their use fans the flames around hate groups and other extremists, fomenting even stronger (and often tragic) responses offline, too. Today, a startup called Sentropy that believes it has developed the most sophisticated, yet easy to implement, system yet to help identify, track and ultimately purge online abuse by way of its AI-based platform, is launching to take on the issue.
The idea behind Sentropy is as much philosophical as it is technical: John Redgrave, the CEO who co-founded the startup with Michele Banko, Taylor Rhyne and Ethan Breder, said that it was about using what he and his co-founders knew about how to manipulate and “read” unstructured data, to apply it to online abuse, which, finally, really needed to be taken seriously.
“Every week I was watching the impact that online conversations were having on the real world,” he said in an interview. “They were creating a lasting impact. There is a link between URL and IRL. For the younger generation, the difference between their digital and physical selves will be indistinguishable. That is powerful, but also problematic.”
The startup is coming out of stealth mode with $13 million in funding from a pretty illustrious list of backers — they include Alexis Ohanian, the co-founder of Reddit, his VC firm Initialized Capital, King River Capital, Horizons Ventures and Playground Global, founders and executives from Riot Games, Nextdoor, OpenAI, Twitch, Twitter and a “former head of government” (whose name is not disclosed).
Although Sentropy is emerging into the open today, it has actually been active for a while, working with a number of tech companies while in a private beta since June for the first of its products, an API-based abuse detection engine branded Defend. That is coming out as a generally available product, along with an out-of-the-box version called Detect. Redgrave would not disclose which tech companies it has been working with so far, although it’s notable that until just this week, Ohanian still sat on the board of Reddit and would have been an obvious strategic connection into working with it, and of course Reddit is a prime example of the kind of platform that faces the kind of abuse that Sentropy is tackling.
“I’ve seen first-hand the difficulty of manually moderating online communities,” said Ohanian, in a statement. “The breadth and depth of this issue require serious resources and machine learning chops. Sentropy has built the tech that is much needed across social media communities. User safety has become a competitive differentiator for those willing to take a stand against abuse.”
Another reason Sentropy has likely had some early and strong interest in its product is because of the track record of the founders. Their previous company, Lattice Data, was a specialist the larger general area of unstructured, dark data, albeit for different ends. It was acquired by Apple in 2017, with several on the current team going on to work there after the sale. Redgrave said that the idea of tackling online abuse as an unstructured data problem was something that they had actually started to identify while Lattice was still a startup, although it was not something that they developed there.
Sentropy is sold both as an API-based product for developers to implement and customise and as one that can be used out of the box to manage communities online, respectively branded Detect and Defend. Both implementations today are based on providing tools to humans to ultimately make moderation decisions for their respective platforms, although over time there could be a service added that will automate more of the work.
There are a number of problems today when dealing with online abuse and the task of trying to stem and moderate it, Redgrave pointed out. The sheer volume of comments on popular sites makes the task of human triage very complicated and practically an insurmountable (and potentially harmful in its own right) task. Meanwhile, a lot of the current systems for automating and flagging keywords and phrases are often not sophisticated enough to catch the right things. This leads to a small dent in the problem, but in actual fact some 75% of online harassment goes unreported, the company said, citing Pew Research.
Rhyne said that the “brain” that underpins the service was built by ingesting many, many pages of conversations from across the open web, the grey web and the dark web. As you would expect with an AI system, Sentropy also “learns” how specific customers use it to adapt to them. In turn, those learnings are also fed into the bigger brain to continue teaching it as well. (The way that Sentropy has parsed all that data to make its own reading of it more “intelligent” is of course the secret sauce here.)
There are a lot of questions and untapped areas that will likely need to be addressed over time, even with the headway it’s making here. For one, Sentropy is making its start by focusing on text. For now, there is no product that parses audio or video content, which would need to today be run through a transcriber in order to be “read” by Sentropy’s algorithms. Over time, given the prevalence of video apps and the popularity of sites like YouTube and video on other platforms like Facebook, this is an area it will have to reckon with.
Similarly, there are question marks still over how much customising a site might want to make around certain terminology and how and if that might be abused at some point. Rhyne notes that for now, Sentropy decides the terms so it can’t itself be abused and used as a tool of political censorship, as one example. It raises interesting questions, of course, about what is universally “right” and “wrong.”
“We do have a choice of who our customers are,” said Redgrave. “It’s not open source technology, and they have to interact with us and we will learn specific use cases.”
Comment