Red Hat has changed the way it distributes Enterprise Linux kernel code in an effort to prevent Oracle and Novell from stealing its customers, making it more difficult for these competitors to understand which patches have been applied where. Some have speculated that the change is designed to make it harder for Oracle as well …
Downvote me, my pants are down.
It's easy to argue that overall Redhat are good for future of linux. But I am sorry to say they are suffering because of a potential folly in GPL: It's very easy for a big giant to eat into a niche created by a smart but small innovator whose entire product is open source, whether the smaller guy is a willing partner or not. Redhat's patches and products are well understood by competition and they can easily eat into red hat's accounts. Also, redhat can't easily do direct-to-customer patch delivery because of GPL's terms.
OTOH, even if redhat had money, could they do that to Oracle customers ? Most likely no, because redhat wouldn't have the same level of expertise with the Oracle's software or understand the patches in the same great detail. Most 3rd party service providers depend on a revenue share agreement with the original product vendor to deliver patches. (Eg: if IBM services arm is installing patches on customer's Sun/Oracle box, it's because their agreement with Sun/Oracle covers it).
Lot of Linux distros are simple re-branded redhats. They make money by investing on 3 unbilled engineers worth of money where as redhat has a 300-400 engineer staff which is part of R&D and not billed to any customers.
I think original purpose of GPL - **why I like it**- it that it allowed a customer to tweak the product they bought as per their whims and fancy. Things are a lot different 30 years down the line.
No more free-loading for Larry.
"Also, redhat can't easily do direct-to-customer patch delivery because of GPL's terms."
Wrong. GPL allows that. By GPL, you only need to provide source to customers (while customers might chose to pass along). Red Hat does this. Red hat in fact provide full source to everyone, except now they only provide knowledge about patches (documentation) to customers.
Before this move, all was up for grabs to Oracle and every patch was marked with number and it came with corresponding explanation and documentation that is easy to understand even to non-programers. Oracle just hired bunch of monkeys and they were offering support just based on this documentation, without any real expertise. They didn't write their own documentation, and they "unbreakable kernel" is just stuffed with crap from staging tree (which is called "crap" by Linux devs; low quality code goes there until it is cleaned up enough to be good for mainline).
But now, Red Hat's knowledge base exists only for RH customers (and I see they added some stuff that are very convenient). Oracle will not get it anymore, they will get full sources like everyone else which is off course readable and clean but Oracle will need experts to see what is new and what it does. In effect Oracle will need to hire real devs and understand what they are supporting. They will need to write their own documentation. They will need to maintain their own bug database. And they will need to really *support* customers and not just claim they are cheaper than Red Hat while blindly using Red Hat's instruction manual when someone calls them for support.
And when they do all this and really put some investment behind their OEL, they will either not be so cheap, or they'll kiss goodbye to profits and say hello to loses.
No more free lunch for Larry. Smart move by Red Hat.
So oracle always used a different kernel put stoled it
And "crap" being 99% stability patches that will pop up in RHEL a year later after "QA".
RHEL6 / OEL6 will seem fresh for 1-2 years now, and after that their users will again go from bug report to bug report looking at things that were patched years ago in mainline, while their RHEL boxes still crash from it.
> Wrong. GPL allows that. By GPL
I didn't say impossible. I said "not easy" = difficult, coz they had to change their original process.
> No more free lunch for Larry. Smart move by Red Hat.
is that business?
This exactly the reason I am sticking with Windows Servers, they work and they work very well. Red had and the rest of the Linux crowd are trying to make money by dishonesty it seems, is that by making something wrong on purpose so you need their support. Reading this it won't surprise me. And I don't quite get it, I never had to ask for any support with Windows software from Microsoft directly, windows forums are great. Why do people need support directly from Redhat or Oracle for that matter. I can only think there is something wrong with their software.
It doesn't seem a very appetising place to eat basically.
I think you might have to go back to your Quick and Easy Free and Open Source for Microsoft Engineers book. It doesn't say "Free software is about hacking together a mess of code in order to extract ever greater amounts of money from support". It says something along the lines of "The economics of code distribution, like most other creative works, has fallen to zero. If you still would like to earn money you can do a couple of things: a) lock your product up and milk your customers like cattle (Microsoft, Oracle, Apple), b) Get paid for performing code creation (Independent SMEs) or c) Get paid for supporting the software in an business context, with a note that you should probably push a bit of that money into development in order to keep your customers."
Sometimes I think Microsoft people learn about FOSS on the back of cornflake packets.
What are you talking about?
> Windows Servers, they work and they work very well
If that were true, my bank manager would be very upset.
Windows servers need much more coddling. Linux Just Works.
> Red had and the rest of the Linux crowd are trying to make money by dishonesty
That is entirely untrue. They are extremely honest.
> by making something wrong on purpose so you need their support.
This is not happening. No-one is making anything "wrong on purpose". You appear to have read a different article to everyone else.
What RH are doing is ceasing to lay out in enormous detail every single patch to the sources that they make, rolling them up into is single source tarball instead. If you think you'll get something like that out of Microsoft, you're in cloud cuckoo land.
The difficulty is that RH are now offering less than they previously did, not that what they're offering now isn't still rather wonderful.
> I never had to ask for any support with Windows software from Microsoft directly
Big deal. If you don't want support - don't buy it. But others *do* want it - and the market is there for exactly that. It's not a compulsory charge, it's an offer.
> Why do people need support directly from Redhat
They don't. But if they want it, it's there.
> I can only think there is something wrong with their software.
You have clearly never worked in a commercial environment. How are the GCSEs going?
> I didn't say impossible. I said "not easy"
But it is easy. You just distribute under section 3(a), rather than 3(b).
Somewhat pointless, but not at all difficult.
Re: is that business
I run Windows and Open SuSE servers. Both just work, but I find the SuSE ones need a lot less attention than the Windows ones and take up a lot less of my time. They probably need a bit more knowledge to get started, but once you know what you are doing, you can ssh in and administer things much more quickly using the command line than you can with the nice easy-to-use graphical tools that Microsoft provides over rdp. Of course SuSE provides the nice easy-to-use graphical tools as well and you can use them over nx.
Are you certain you even understand what this article is about?
Red Hat makes around 800 million in revenues per year. How much of this is not support then? Thinking as an IT customer why do I need support (unless I am stupid to throw my money away). Maybe because I don't feel very confident with the software, or there isn't enough documentation, either directly from the company or from web forums or I am charitable!. I would never get support if I know I can fix it myself in the unlikely case something is wrong. Any explanations why a company would need support if they are looking after their money?
Quoteth: Windows servers need much more coddling. Linux Just Works.
Er, not exactly. Windows and Linux have been neck and neck as far as OS stability for years. The main issue you run into when it comes to stability is the application running on top of it.
Now sure, you can make the case that developing on windows is faster and looser than on linux, hence there are more flaws. But after the decade I've been in this industry, I'd say both OSes suffer from proprietary vendor oddness. Both are susceptible to stability issues due to application level glitches.
> How much of this is not support then?
It's all support.
> Thinking as an IT customer why do I need support
You don't *need* support. You could decide to use community knowledge on the web and do it yourself - just like you intimated you would with Windows.
But as a commercial entity, that might not make any sense. You might decide it is cheaper and less risky to buy support from a specialist vendor than to try to cover everything in-house.
> (unless I am stupid to throw my money away).
See, it's comments like that that convince me you've never seen a commercial environment.
Users want support. That might be in-house, or it might be bought in. But leaving a bunch of users alone to support themselves is a recipe for disaster except in an environment where all users are sufficiently skilled to be support people if they so chose.
So support is a fact of life, whatever OS you're on. I have *far* more Windows support customers than I do Linux support customers, despite the fact that I am targeting the latter.
> Maybe because I don't feel very confident with the software, or there isn't enough
> documentation, either directly from the company or from web forums or I am charitable!.
No. It's because management want to make sure that, should anything tricky come along, the entire company won't be stuck twiddling its thumbs while someone investigates the problem.
A huge number of support contracts are never actually used...
> I would never get support if I know I can fix it myself in the unlikely case something is wrong.
Good for you. But you won't be managing any significant IT resources in the near future. When hundreds of engineers require the IT to work so that they can do anything, every hour of downtime costs you a *lot* of money. Having to trawl through the link-farms on Google to find an obscure fix to a gnarly problem is not a cost-effective way of providing that support function. So you either train up your in-house support staff to be experts in every package they have to see, or you train them to be good enough to deal with the bulk of the workload, and you have a specialist support company on contract to pick up the phone. Guess which of these makes commercial sense once the application count starts to rise...
You'll notice that the above argument doesn't discriminate between OSes, and doesn't mention the quality of the code or its documentation; it's a simple business case that says you mitigate risk by buying support.
> Any explanations why a company would need support if they are looking after their money?
Plenty. You'll see it in action once you start seeing IT in industry.
 Even then, it will often go horribly wrong because you get competing policies.
Almost like closed source but with far more politics
So all this talk means they don't tell you which patches they have added to the stock Linux kernel? You'll have to try and figure out yourself from the mangled up source code?
I think this is bad for everyone, not just the leeches. I like knowing what extra bits goes in. If this is true I'll make sure we really reconsider continuing our two support licences when they come near expiration date. Never actually used much support from Redhat apart from updates anyway so I think we'll survive with CentOS.
I can understand they need the cash but this is not what I expect from a Linux distribution. Maybe they regret not taking BSD, at least they could be going closed source with it.
I dont get all the hubub
Really? You read all the kernel patches?
Try 'diff -Naur <pristene kernel> <red hat>'
Sheesh, the way everyone has complained about this you would think it wasn't open souce
anymore. Perhaps everyone is just worried for poor defenseless Oracle.
RE: I dont get all the hubub
Whilst we're a RH customer and I'm a big RH fanboi, I'm still left a bit unhappy about this new approach. We've already discussed it with RH and they have assured us - as customers - we can look at the source any time we like, that they will explain any and all patches and help us determine whether a patch is good for us or not. All this we have come to expect from the excellent RH support staff and is the reason we chose RH in the first place - the support. By trying to edge out Oracle and Novell, RH is basically trying to tie up the support market, which is their buisness model. I can understand why they would want to do this, it makes commercial sense, but it still feels like it goes against the whole spirit of Linux (not a very good business argument, I know), and I also worry that it reduces the ability of the experienced people in the community to look at a RH patch and spot an error before it gets compiled into the kernel. Whilst I'm sure RH are so good at their job that's unlikley to happen much, it still worries me.
I dont' read all the kernel patches that's why this is bad news for me.
Previously the individual patch files had a reasoning behind each one, which is what I saw. Now all I get is a long list of diff output with no context at all.
Sure it's still open source, but much less so and if you don't have the resources of a large company to go through then it might as well be closed.
So yeah downvote me, I've been using Linux since kernel 0.98.1 and I'm disgusted, although not surprised, by this "new way". Whatever's next really, redhat exclusive binary blobs? All seems fair in the "fight" against Oracle...
Yeah, like Oracle cannot hire 4-5 people to follow RedHat Bugzilla and LKM
Oracle would no longer be able to freeload that's all.
However the investment not to freeload is so marginal that it is not even worth discussing. We are talking what? half an M per annum or so? And that half an M will be paying back because it will now have a kernel development and support team of its own. It can in turn use that to improve its own patches and its own distro's tree as well. Which in tun is not in RedHat's favour. Forcing your competitor to commit resources to something that actually improves their offering in the long run is not a good idea.
"""Forcing your competitor to commit resources to something that actually improves their offering in the long run is not a good idea."""
But then Oracle's kernel development team would also be pushing some stuff back upstream, which should in turn benefit Red Hat, as well as the rest of us. The point is that investment in Linux (theoretically) benefits everyone who uses it, so fair play to Red Hat for trying to force a bit more of it?
re: Oracle ... would also be pushing some stuff back upstream
You trust them to do that, do you?
It's not about trust
> You trust them to do that, do you?
They don't have a whole lot of choice. The licence requires them to release their changes under GPL.
Whether they submit any such patches upstream is irrelevant; they *have* to release them downstream (or lose their ability to ship the code at all), and anyone downstream has the right to redistribute under the GPL (e.g. by submitting the patches back to the mainstream development).
The exact definition of "downstream" depends slightly on the method by which source distribution is achieved, but since the code is freely distributable under the GPL, it tends towards being "everyone who wants it".
[Waiting for more downvotes to pour in because I've said something that makes FOSS look reasonable again]
Not A Problem
This seems like more of a problem for people (Oracle, Novell) who want to patch a kernel someone else (Red Hat) built, not their own RHEL derived kernel (CentOS, Scientific). Russ knows what he talking about, and I trust his judgment on the matter.
We use some other Oracle products which have been acquired by the giant... We usually find ourselves sorting out production support issues days or weeks before Oracle support get back to us with their useless mumbo-jumbo. Oracle is starting to look like some kind of monger willing to sell just about anything. For those contemplating Oracle support, don't... Stick to the REAL support from Redhat.
Refund time ala class action style
I am a self-employed tech in the "support business", I do this to support my family. I quite legitimately paid for a Red Hat Linux Engineers course, studied my ass off, and passed the exam. I was then able to support linux in the field, which finally enabled me to start paying off my credit cards and house and car loans.
Now that Red Hat has decided to covertly introduce flaws in their system that prevents anybody else from supporting Red Hat, not only is my RHCE useless, but I will no longer be able to support my family.
I think Red Hat should be forced to compensate every RHCE for the invalidation of the certificate as well as loss of income.
Thanks Dead Rat.
Spare us the defenseless widow with children spiel!
How are Red Hat Sharp Practices [tm] going to impede your income?
Maybe you are confused about what a RHCE is all about. It means you can handle a Red Hat System. It doesn't mean you can do all the support that comes with a Red Hat license yourself. If in doubt, ask for Red Hat support, which is presumably being paid for by your customer.
Are you kidding me? Your RHCE is irrelevant because now you can't see individual patchsets in the kernel? Your RHCE certifies your knowledge of how to administer a server, not how to figure out what line in the kernel was changed to what and why.
@AC class action...
> Red Hat has decided to covertly introduce flaws in their system...
Covertly? Flaws? Forgot to take your pill today?
RHCE is a cert so even monkey can do BASIC sys admin job. As such RH's change doesn't really concern you and if you claim you can no longer do your job, I feel sorry for your family already.
Umm- simple question
Did you actually communicate with RH about this? Part of the RH product sphere needs people like you, so I cannot imagine RH shooting itself in both feet here. Maybe you ought to talk to RH first..
I smell troll...
> Now that Red Hat has decided to covertly introduce flaws in their system
It has done no such thing.
This change merely wraps up all the individual patches into one super-patch, and they no longer put all the info into the changelog. It's the same code as it ever was - just without all the explanation RH have historically put into their releases.
It's a shame RH has done this, but I can see their point.
Re AC Refund
You can still support and maintain just about any other distribution with ease and to transfer your
family to another distribution should be a doddle. In fact maintaining paid for RH installations will still be really easy.
Now consider if you'd done the same thing with MS - you'd have had to do 5 or 6 courses every couple of years just to work out where they'd hidden your particular subset of experience in the menu this time.
So why are you not able to apply your engineering skills all of a sudden?
I don't believe you
Trolling / shilling / astroturfing - it's all the same in the end.
Nice try, MS munchkin
I happen to be an RHCE, and I don't see how this move could possibly hurt me. If anything, it will make more demand for RHCEs.
I think you don't even know what RHCE is and what it means, and this is lame attempt to throw dirt on RH. Stop embarrassing yourself. Go debug your BSoDs instead.
RE: Refund time ala class action style
"....I think Red Hat should be forced to compensate every RHCE for the invalidation of the certificate...." I thought the RHCE was aimed at certifying you to install and configure standard RH releases and then working as a sysadmin in conjunction with RH support, not a certification in how to undercut RH's support. Try reading the small print, maybe?
A pragmatic solution
...if you ask me.
They could easily have decided that this OSS business is not for them anymore and closed source a lot of their value-added software. Good on them for sticking with GPL and sticking two fingers up at Oracle.
Pragmatism is bad for FLOSS
I support this decision of Red Hat and I won't insult it by calling it pragmatism. It is smart move. Not pragmatism.
Pragmatism is worst dogma ever. These days, pragmatism means being a an idiot and a jerk ,and backstabbing everyone. And doing what Oracle does. Pragmatism is evil. Pragmatic decision for Red Hat would be to let Oracle acquire them. Or go closed source.
Maybe pragmatism before had some other meaning, but idiots disfigured the meaning through common usage of the word. They used it to justify every evil they do. Now it is pretty much same as quislingism.
So down with pragmatism.
This is a pain
RHEL distributions are long-lived, and the kernel shape is the same at the end of life as it was at launch - so anything RHEL4-based started with a kernel that looked like 2.6.9, and still does to this day. But they are very far from obsolete - RH back-ports much new kernel development into these old frames to give something that is updated and fresh, whilst maintaining compatibility with what went before. That's a Good Thing(tm).
However, if you want to find out if a certain kernel incorporates a certain feature, you have to go through the patchset. The availability of the changelog, with patch documentation, is a boon here. Red Hat's removing it will be a royal pain in the arse to those of us who use their code.
I can't blame them. They are operating well within the GPL, and give what Oracle (in particular) has been doing, I can't say I'd have done anything differently. But it is properly annoying...
You know what
I think I'll just buy red hat's corporate spin for once. Not because I like them, or their product (the memory of 2.95.4 still smarts, yes, I know that's a long time ago). More because the CentOs guys don't seem to care, who're pretty much the closest to the red hat product you can get without working for, or being red hat. The fact I had a couple beers with them and think they're cool even if I care nothing for either product has nothing to do with it, honest. *cough*
Anyway. Red Hat is a corporate player and found themselves getting undercut by bigger guns with less brains (as in, quality support) parasiting on their efforts, so they pretty much had to do something. This something is about as elegant as it gets, given the circumstances. I really can't blame them for, you know, making sure they don't get sucked dry by a cheapskate competitor.
Oracle stealing customers?
Why doesn't Red Hat sue Oracle for stealing customers and using Oracles support stuff without consent? They (Oracle) set the precedent by suing SAP for $1B for 'Corporate Theft'.
Re: Oracle stealing customers? → #
Redhat are doing the best they can given the situation. Suing is not an option coz unlike proprietary code (basis of SAP lawsuit), customers are not property of the vendor!
Oracle has enough money to become a shill customer of Red Hat's, entitling it to the patches and documentation. Way cheaper than figuring it out for itself. BACK TO THE GOOD OLD DAYS.
For Linux virus writers - and other "security professionals" (well, they are, in a way) - who can make a good living exploring and exploiting unpatched vulnerabilities in corporate Linux installations.
"He insists that the change does not violate either the letter or the spirit of RHEL's GPL open source license."
Like hell it does! If you aren't releasing source code in useable form, then you aren't releasing source code.
Maybe I should just stick to using the GParted Live CD from now on...
Please read the article...
> If you aren't releasing source code in useable form, then you aren't releasing source code.
They *are* releasing source in a usable form.
What they are no longer doing is breaking it down into a bazillion tiny patches, with explanation for each and every one of them.
But I suspect someone will.
"Like hell it does! If you aren't releasing source code in useable form, then you aren't releasing source code."
They still release the source code.
All of the source code they releases compiles and runs.
Anyone with the relevant programming capabilities can read it and work with it.
AFAIK, nowhere in either the text or the mission of the GPL does it require that patches be maintaned separately from major release code.
So exactly what are your criteria for "useable" code that RH is now failing to meet? Where in the GPL are your criteria espoused?
What is usable for you?
"If you aren't releasing source code in useable form, then you aren't releasing source code"
As a long time developer and occasional contributor to a few FOSS projects, none of them in the kernel, I have to ask this question once and for all: why releasing a simple source code file is not usable?
Last time I contributed code to a FOSS project, I just downloaded the sources, examined them, changed whatever I wanted, created the patch and submitted to the maintainers.
I've a very hard time believing that you need the individual patch history to the code to be usable for development. Perhaps Linus or some other genius is capable of directly creating patches instead of working on the source files. Even if you're a developer of such caliber, you are going to test by applying the patch to the source and recompiling. So by no means a list of patches is usable for software development.
The history with support is different. There is this common practice called "bisecting", that takes a source code tree and applies patches one by one, testing for the existence of a bug after applying each. It is a faster way of finding where the problem is.
So Red Hat is removing the ability to bisect individual changes to the kernel. So your support becomes more difficult, but by no means kernel development is more difficult. Guess what? Red Hat lives and pays all the kernel, x, Gnome, etc, developers by selling.... support.
It's not nice for others to piggyback on the work created by Red Hat and make customers pay for what they've got for free. Bad, Oracle.
It's not nice, it's nasty, to brand and sell something "Unbreakable" simply by taking the works of others and package it under a new name. Bad, Oracle.
The free lunch is over, Larry, now it's time to explain to your "Unbreakable" customers that you were simply using other people work without paying for it.
The point of open source,
is to enable someone other than the source code author to compile, debug, maintain the software - subject to legalities such as copyright. I maean, I could publish an entire program of my own and forbid it being used by anyone. (Diifficult to enforce, of course.)
And the absolute entire point of the GPLs is that your contributions and additions to a GPL work are not your exclusive property. They are surrendered to the community of users and potential users. It's what the GPL is for.
Let's have independent security researchers publish information about Red Hat vulnerabilities the same way that Red Hat are publishing patches, and see how Red Hat likes it then. All right, this is dangerously close to treating hacking as legitimate, and I don't want to do that. But I think that, while wrong, it wouldn't be unfair.
The point of your post...
is what, exactly?
1. Red Hat is still releasing their source code in compileable, debuggable, maintainable form. They're still giving the code back to the community. The only difference with this policy change is that they're rolling patches into the main code tree rather than applying them separately.
2. Actually, independent security researchers publish information about vulnerabilities in different forms depending on the audience. They generally provide the vendor with an overview of the issue and a tool and/or the source code for a tool that will exploit the vulnerability. This is similar in scope to what Red Hat is doing.
I ask again: what EXACTLY are YOUR criteria for useability which you believe Red Hat are violating with this policy change? How will this impinge on your ability to use their patches? How do you use their patches, anyway?
Re: The point of open source
> is to enable someone other than the source code author to compile, debug, maintain the software
Yes. And you can still do that. The source code is published in its entirety.
> subject to legalities such as copyright
Well, that's the real point of Free Software - while the copyright stays with the author, a licence to use and redistribute comes with the code - and such redistribution will leave the recipient with an identical licence to use and redistribute. So al song as you stick to the licence conditions - which is really easy - you can pass the code around to your heart's content.
> the absolute entire point of the GPLs is that your contributions and additions to
> a GPL work are not your exclusive property.
That is incorrect.
Any code you write *is* your property. That's how copyright works.
What the GPL does is to ensure that anyone distributing variants of your code has to pass on the same rights as he got - so others (including the original authors) can get their hands on the source and re-use it.
> They are surrendered to the community
No they are not. This is critical; the GPL does *not* require you to surrender any copyrights.
> Let's have independent security researchers publish information about Red Hat vulnerabilities
Errr - they already do.
> see how Red Hat likes it then
Red Hat would like it very much, I should imagine. They already publish the bugs they know about in public. It's called Bugzilla...
And they wonder why...
... they can't win Slackware users.
- Microsoft: We're hiking UK cloud prices 22%. Stop whining – it's the Brexit
- Thanks, IoT vendors: your slack attitude will get regulators moving
- And so we enter day seven of King's College London major IT outage
- Capita's head of tech solutions resigns
- AWS budget tool update gives suits the control they crave