The success of large language models like GPT has sparked a frenzy of developers eager to make AI-powered applications. But building AI services can be tricky, especially due to the shortage of skilled developers to meet the rising demand these days.
That’s where Chaoyu Yang, an early software engineer at the data mega-unicorn Databricks, comes in. Along with his co-founders, he’s built the AI development framework BentoML, which just announced a seed financing round.
Yang, in an interview with TechCrunch, explained that today’s AI services are often built on multiple machine learning models, making their management and operation complicated. Many programmers entering the fray are coming from a full-stack or application development background, meaning they often lack the skills to build the required AI infrastructure, resulting in a prolonged development process.
A demo AI app like Microsoft’s Visual ChatGPT, an upgrade to the chatbot that allows it to produce responses from both text and image prompts, for example, can take at least three to six months to make it production-ready, Yang said.
While tech behemoths like Microsoft enjoy the financial prowess and human capital to train AI models and put them to use in the real world, smaller businesses, in Yang’s words, are collecting “valuable data that can benefit tremendously from AI” but “lack the resources to build the infrastructure for development.”
BentoML, which provides a high-level API that abstracts away the details of the infrastructure needed for running AI models on the cloud, belongs to a camp of tools like SageMaker that wants to smooth the path for developing AI services. It’s a so-called AI application framework, a set of tools that make it easier to build, ship and scale AI applications, like a construction tool kit one uses to build a house.
Specifically, BentoML is targeting data scientists who train AI models, DevOp engineers who manage their lifecycle and developers who actually build applications on top of the models.
With BentoML, developers can make Visual ChatGPT scalable and cost-efficient for production use in as short as two days, said Yang. Users have also used the framework to run the art generator Stable Diffusion and open source LLMs on the cloud.
Yang compared his company to Vercel, which focuses on serving front-end developers and was last valued at over $1 billion. BentoML aims to be the Vercel for AI, he said.
While Yang predicted that AI would eventually become more production-ready, he admitted he didn’t think the AI application wave would arrive so soon. The founder expects AI app developers to account for over 90% of the platform’s users in the future.
“If you ask me a year ago, I’d say that probably 90% of the companies would be training their own models, but the foundation models that have recently emerged are so powerful that they can perform well even given a dataset it has never seen before,” he said.
“Rather than focusing on model training, developers now only need to work on finetuning and product engineering, which in themselves present a bottleneck because of the shortage of AI-focused developers.”
BentoML was open sourced in 2019 and later introduced a self-hosted SaaS version to enterprise customers. It’s been acquiring users organically through its open source community, which quadrupled its membership to more than 3,000 over the past year, with Japanese messaging giant Line and South Korean internet conglomerate Naver being among its early adopters.
Yang declined to disclose the company’s revenue size.
Investors are taking note of BentoML’s traction in the developer community. The startup recently raised $9 million from its seed financing round led by DCM Ventures, with Bow Capital also participating. DCM’s general partner, Hurst Lin, has joined BentoML’s board following the round.
The exuberant AI market has been a boon to BentoML, but the rapidly changing industry also makes it tricky for the team to juggle short- and long-term goals, Yang admitted.
“You might have to build things that ride the current trend, but in the long term, we of course want to have our own moat. The question is how we balance our time and human resources between the two.”
Update (June 28, 2023): The article previously stated that Line was a South Korean company. Line is headquartered in Japan and is a subsidiary of Z Holdings, a joint venture between Japan’s SoftBank and South Korea’s Naver.
Vercel raises $150M Series D as it looks to build an end-to-end front-end development platform
Comment