Today, SUSE announced that it is creating a hard fork of Red Hat Enterprise Linux (RHEL) and that it will develop and maintain an RHEL-compatible distribution. SUSE says that it will invest $10 million into this project over the coming years. One major open source company forking another major open source company’s project is equivalent to going nuclear. But there’s a reason SUSE is doing this now, and that it will likely be championed by many in the open source community. It’s a complicated story.
In its early days, open source was a movement — maybe even a religion. For the longest time, it was a playground of geeks and activists, building software in the open and arguing about licenses. But over the course of the last decade, it morphed into a business model, and ever since, there has been this tension between commercial interests — often driven by some of the world’s largest companies — and the more community-driven, activist side of open source. For the most part, both sides have remained in balance over the years and found ways to co-exist, even as the open source movement commercialized, in large part because the default open source licenses ensure an even playing field.
Every now and then there are skirmishes, as we saw a few years ago when a number of open source companies changed their licenses after Amazon started launching its own products based largely on open source projects. Over the course of the last few weeks, we’ve seen the beginnings of a new open source fight. The players are very different — Red Hat and a number of Linux distributions and vendors that offer Red Hat Enterprise Linux-compatible distributions — but the question at its core isn’t all that dissimilar from the previous arguments.
To understand what’s happening here, we need to go back a few years. In late 2020, Red Hat made a crucial change to CentOS Linux (the Community Enterprise Linux Operating System). For the longest time, CentOS was essentially the free (as in beer) version of Red Hat Enterprise Linux (RHEL), Red Hat’s flagship distribution. Red Hat acquired CentOS in 2014 after a lot of turmoil in the CentOS community and gained a permanent majority on the CentOS board.
“The CentOS project was in trouble,” Gunnar Hellekson, Red Hat’s VP and GM for Red Hat Enterprise Linux, told me. “At the same time, we needed a way to collaborate with other communities — OpenStack in particular at the time. And we said, well, here’s an opportunity! We can take the CentOS project. Now we have something that is freely available and close enough to RHEL to do the development on — and then that gives us a way to work in the community. And then when customers move into production, they can go on to Red Hat Enterprise Linux.”
Red Hat already published the Fedora distribution, too, but with its six-month release cycle, that was moving too fast for an enterprise-centric distribution.
For many years, all of this worked quite well, but in 2020, Red Hat changed things up. Instead of regular point releases that mostly coincided with RHEL versions, it launched CentOS Stream, a “‘rolling preview’ of what’s next in RHEL,” Red Hat’s Chris Wright explained at the time. That made CentOS 7 the last CentOS version with long-term support (which ends in 2024).
The idea, Red Hat said at the time, was to shorten the feedback loop between developers in the RHEL ecosystem. “Say we have a complicated networking feature we want to add,” he explained. “The community would do the work in CentOS, which is a good start. But all the decisions about the product had already been made through CentOS. Remember, we’re an upstream-first company, everything we do goes in the upstream first and then it flows through the system. This meant that changes we made in the CentOS downstream now had to go into the upstream communities, flow down into Fedora and then flowed down into Red Hat Enterprise Linux and then into CentOS. So it actually slowed us down considerably and we couldn’t move as quickly as we wanted.”
But it also meant that CentOS and RHEL wouldn’t be 100% compatible anymore. This move created quite a stir in the community and a number of CentOS contributors split off and launched new distributions: Rocky Linux and Alma Linux.
Alma Linux was founded by CloudLinux, which provides commercial support for it, but among those two, the Rocky Linux team was probably the most outspoken about these changes. Founded by Gregory Kurtzer, one of the co-founders of CentOS, Rocky Linux aimed for bug-to-bug compatibility with RHEL, ensuring users that they could use Rocky as a drop-in replacement for Red Hat’s distribution. Kurtzer’s startup CIQ, which raised $26 million in funding in 2022, provides enterprise support for Linux (sound familiar?). CIQ provides enterprise support for Rocky Linux (and CIQ is the largest sponsor of Rockly Linux).
When Red Hat changed how CentOS is being developed, it suddenly became much harder for Rocky and Alma to get access to the RHEL source code, which is open source.
“I read the Red Hat press release and I said, ‘what does that actually mean?’ It was really confusing,” Kurtzer told me when I asked him what he thought when he first saw Red Hat’s announcement. “It either said that CentOS is going to continue to be the definitive location for all Red Hat sources — but Red Hat’s not putting their sources there anymore. I couldn’t — it was hard to follow. It was hard to understand. So the first thing that we did is we all just kind of talked internally within the Rocky team leads and groups, and what they said is: ‘well, this is kind of weird, because what we noticed was updates in RHEL are not hitting CentOS Stream anymore.”
This whole thing has become such a mess that even Oracle, which hasn’t always been seen as a champion of open source and which bases its Oracle Linux on RHEL, is coming out of this looking pretty good.
Unsurprisingly, Red Hat has a very different take on all of this. Hellekson noted that there are three ways to get RHEL. The first is through CentOS Stream, “because whatever goes into red Enterprise Linux ends up in CentOS stream.” Then there’s the Red Hat customer portal, where Red Hat customers can get it, and, until the recent changes, there was also git.centos.org. “There we were publishing the source RHEL 7, CentOS 8 — and this was — I will describe this as an accident of history because that used to be how we disclosed the source code to the downstream CentOS Linux project. And it seemed for us innocuous enough to keep that running. We were releasing to these two other places so why not.”
Then, at some point, Red Hat released that the likes of Rocky and Alma were building their downstream Linux distros and claiming bug-for-bug compatibility. Red Hat obviously has commercial interests here. I don’t think anyone is going to dispute that what Rocky and Alma were doing was legal and within their rights, given how Red Hat licenses its code, Hellekson argues that if this were happening with another open source project people wouldn’t really think Red Hat was doing anything wrong by making it harder for others to use their work.
“Let’s just treat this as two open source projects,” he said. “I’m running Red Hat Enterprise Linux, someone else comes along, takes my open source project, claims bug-for-bug compatibility, and thereby promising not to do any innovation on it at all, not to improve it in any way. Put their own logo on it, and then go actively recruit my users to go use their version instead of mine. In the open source community, this is bad behavior. It’s legal but it’s frowned upon. It’s counterproductive and not good for the ecosystem.”
I asked Kurtzer what he thought of Red Hat’s original statement about this, which used relatively strong language.
“I’ve seen a lot of this online as well. I don’t know where it originated from. But some of the terms like ‘freeloaders’ have been coming out,” he said. “And I think it’s just a negative, derogatory way of looking at it. I mean, this is open source. You never know who is going to just be downloading and using the code today, who is going to become a very valuable member of that community tomorrow. You never know this. As a matter of fact, the amount of people that I know, that have thanked me because their introduction to Linux and to open source and to contributing and becoming a developer was CentOS. It’s really important, I think, to recognize that part of the reason that open source is so popular and so powerful is not that it’s just it’s about the contributions. It’s definitely a big piece of that, but it’s about the community at large and it’s about supporting our community and being good stewards of that community.”
Kurtzer argues that he didn’t originally want to become a Linux vendor. He simply wanted to use CIQ to fund Rocky Linux. “It quickly became very clear quickly that a lot of very large organizations, a lot of very large companies with IT shops — they all need help. And the help that they’re getting from existing models that are out there are not exactly impactful for the customers.”
Couldn’t Rocky and Alma simply fork CentOS 7 or 8? Kurtzer argues that CentOS Stream simply isn’t an enterprise-ready operating system. He said that early on, the Rocky community argued over whether it wanted to simply fork CentOS. “The community was pretty responsive in saying let’s not create another standard. As much as we can, let’s try to be as compatible. And so that was our goal: to be as compatible to that source and to what the community needs. Now, CentOS Stream is not exactly that — it’s close but it’s not exactly that. It’s also not maintained for 10 years, it’s only maintained for five years. So it does mean that it would still be a little bit difficult to base on that.”
Meanwhile, Red Hat decided that it didn’t really see a reason to make life easier for its competitors. Hellekson noted that there’s a question of open source ethics but he also acknowledged Red Hat’s commercial interests. “You add a set of business concerns on top of that and that’s when we realized we actually have to, we feel obligated to intervene here,” he said. “If they want to go build an open source distribution, they are obviously welcome to do that. That’s what open source is about. And if you want to do that, the right place to do that is in CentOS Stream, where you can take all the same bits that we have and pull all these things together and create a distribution.”
He argues that there are plenty of ways today to use RHEL for free, including its free Developer Subscription for Teams that allows up to 25,000 people in an organization to use RHEL, as well as free subscriptions for individual developers and programs for schools.
Hellekson acknowledged that he knew this move would always upset a portion of the community. “What I think we didn’t anticipate was that the way in which the GPL, for example, and the open source licenses interact with our enterprise agreement — I think that a lot of people learned a lot about how those two things connect,” he explained. “We feel comfortable that we’ve got a business model and that is: we’re an enterprise software company with an open source development model. And we’re very proud of the way in which we are able to be a very successful enterprise software company and still be a contributor back to the open source community.” He also argued that this started a very important conversation in open source about what is — and isn’t — okay.
For Rocky, Alma and others, there are still ways to get the RHEL sources, including through the Red Hat Universal Base Images, which remain available but don’t contain all of RHEL. It’s just a lot more complicated. Indeed, Kurtzer noted that things could’ve been worse for Rocky.
There have been some questions about whether Red Hat was pressured to take this approach by IBM, which acquired the company for $34 billion in 2019. Hellekson vehemently denied this. “This was a Red Hat decision. I’ve heard all kinds of theories about why this happened and the truth is that this was a Red Hat decision.”
But Red Hat’s decision created an opening and nature abhors a vacuum, so about 2,000 words later, we’re back to SUSE.
SUSE CEO Dirk-Peter van Leeuwen today argues that SUSE is wading into this because of its belief that “becoming more proprietary should not be the basis for competition between open source companies. We have all contributed to the open source community — just as in the same way we have all benefited from it. It’s something bigger than the sum of our parts.”
It’s no surprise that one of the larger Linux vendors stepped in here. With maybe the exception of Oracle, the smaller vendors don’t have the resources to maintain a hard fork and create a community around that. SUSE has been around for more than 30 years, and while it has seen its ups and downs (and owners), it’s a known and trusted entity — and it’s doing quite well for itself these days.
“This collaborative effort demonstrates SUSE’s deep-rooted commitment to fostering innovation and nurturing community-driven development, and it reinforces the fundamental values of open source software. We invite the community to actively engage and collaborate in shaping the future of this essential software,” said SUSE CTO and CPO Thomas Di Giacomo. “We firmly believe this new RHEL-compatible Linux distribution, together with SUSE’s portfolio, will help the community and customers navigate unprecedented advancements in enterprise Linux, cloud computing, containerization, edge, AI/ML and other emerging technologies.”
What does Red Hat have to say about all of this? “A fork of Red Hat Enterprise Linux is a testament to the fact that we’re following the spirit of open source and that RHEL’s source code remains freely available to any who want it,” Hellekson told me today in a written statement. “The world is rich with Linux distributions, and this is the open source model working as intended to strengthen the community overall.”
Bonus: if you’ve made it this far, you deserve some levity. Enjoy.
Comment