What Happens To -AC (And Other) Kernel Mods? 164
Note: Here's what Alan passed on in response to this question. As usual, things aren't quite as simple as they first appear. -T.
Alan Cox: Probably the first thing to explain is the Red Hat kernel. That actually isn't something I am responsible for. Arjan van de Ven is the keeper of the distribution kernel, and has the unenviable task of getting a kernel together that will actually pass all the brutal QA testing. Arjan is perfectly entitled to (and sometimes does) throw out bits of -ac changes.
You'll see Red Hat patches being merged into -ac and Linus trees when appropriate, often from Arjan or Pete Zaitcev. Many of the other patches in the RH tree are considered "fixups" - they are workarounds for problems but not generalised or clean enough to feed into the main tree without further work. Others are RH specific patches for things like packaging.
With the -ac tree I try and do rapid rolling releases, sucking in new code to test it and also its interactions with other new code. By doing releases every few days I get a high number of people testing and reporting bugs before there are too many possible causes. This is how Linus trees used to work long ago, and I still think its the better technique.
At regular intervals I take stuff from the -ac tree and feed it to Linus. Sometimes Linus doesn't want to take other changes in case they confuse other things being done, sometimes they just vanish and fairly often they get applied.
I'm actually limited in the rate I can forward patches because I need to feed Linus blocks that are debuggable. Thus I don't want to feed Linus both file system and disk driver changes at once or I won't know which to blame if there are corruption reports.
I also don't feed Linus code that has active maintainers unless the maintainer has asked me to do so. Thus the USB diverges quite a lot because Johannes Erdfelt has chosen not to feed chunks of the USB and input changes on. Similarly, the user-mode-linux port in -ac has not been fed on to Linus because Jeff Dike wishes to improve it further before submitting it.
I have been concentrating on getting the driver code and some architectures synchronized with Linus, and that is now mostly done. The next big challenge is getting all the file system work on to Linus, and Al Viro has begun that and fed Linus the first blocks of the superblock handling cleanup.
Finally we have changes that are down to fundamental disagreements, perhaps in part stemming from the fact my background is real production systems rather than OS design work. Linus decided to update the 3D support without keeping back compatibility - I kept both. Linus I suspect will never accept a patch to do that. Secondly he decided that he didn't wish to allocate new device major numbers but look for a saner solution over time. Laudible, but not in the middle of a stable release. The -ac tree has drivers allocated "non-Linus" major numbers that are recognized by LANANA and thus common across vendors. These drivers like the HPT370 and Promise IDE raid will thus always be part of the -ac tree only.
The -ac tree also tries hard to avoid any incompatibilities. Having applications that require -ac or Linus trees is simply not an acceptable situation. The only specific exception for that right now for 2.4.x is deep at the system level and is for quota tools. That one was unavoidable to get 32bit uid quota working.
Post-Mortem debugging of multithreaded processes (Score:5, Interesting)
Considering that Solaris has had this (what seems to be BASIC) functionality for years, why do we see the continued insistence on keeping this functionality out of the production kernel? Are we waiting for the gdb team to catch up?
Until this is fixed, multithreaded programming under Linux will remain a black art - only developers willing to apply hordes of -ac patches to a homegrown development kernel have a change of successfully developing a multi-threaded application under Linux. Considering that many commercial software development packages (RogueWave, for instance) won't even support you if you're not using a RedHat released kernel, this puts multi-threaded development "out-of-bounds" for many.
Merge the -ac kernel mods!
Re:Post-Mortem debugging of multithreaded processe (Score:3, Informative)
Re:Post-Mortem debugging of multithreaded processe (Score:5, Informative)
In the mean time, if you're desperate, I can give you a patch that provides this capability to any Linus tree.
Re:Post-Mortem debugging of multithreaded processe (Score:2)
Re:Post-Mortem debugging of multithreaded processe (Score:2, Informative)
I had no idea so much still needed work (Score:1, Insightful)
Re:I had no idea so much still needed work (Score:1)
Yeah, I know it's bogus - 2.3.x should never have been dubbed 2.4 until it was stable (esp. with the rotten VM), but there you have it.
I tried 2.4 and got bitten by the VM, so am thinking (being lazy) of switching to SuSE who have large file support merged into the 2.2 kernel.
Re:I had no idea so much still needed work (Score:2)
Re:I had no idea so much still needed work (Score:1)
Re:I had no idea so much still needed work (Score:2)
Linus and company may know how to write code, but it's very obvious they were never taught to manage all of their code. Talk about a black art. Linus' hatred of CVS doesn't make things any easier.
Re:I had no idea so much still needed work (Score:2)
Re:I had no idea so much still needed work (Score:1)
Re:I had no idea so much still needed work (Score:1)
If Linus hates CVS, why not set up a Perforce server for him? I was recently looking at their web site [perforce.com], and saw that Open Source developers can obtain licenses for FREE!
It's more powerful than CVS, and has a great user community to help solve problems.
I just downloaded it, installed it, and started submitting files in about an hour -- and half that was the download (modems suck, especially after having had ADSL, 1-way cable, and 2-way cable).
No relation to Perforce, just very pleased with their interface and their attitude.
Re:I had no idea so much still needed work (Score:1)
Re:I had no idea so much still needed work (Score:2)
A lot of things could be done differently. They, of course, won't because Linus hates CVS. Personally, I'd prefer something along the lines of ClearCase, but I'm paying for the licenses
Anyone who knows anything about code management knows what a "code freeze" is. They also know to begin working within new branches (long) before freezing previous branches. 2.5 should have been open for development when 2.3 was "frozen". And certainly parts of the current 2.4 tree don't belong there -- bug fixes and isolated back ports from the development branch are all that should be going into the 2.4 line. Additionally, it's hard to construct a schedule when you have no clear direction -- a list of features to be in X and which features are push further down the line.
Of course, I don't what the kernel versions to start looking like Cisco IOS tags
Re:I had no idea so much still needed work (Score:1)
You should mention to Linus that Perforce [perforce.com] is giving free licenses to Open Source developers; the GNU GPL v2 or higher is sufficient.
Perforce is very flexible, a great system. Plus there's a great user community.
One advantage Perforce has over CVS is it groups a number of files into a "submit". A submit is atomic: if anything goes wrong it rolls everything else back. Change numbers describe a submission, then, not a single file. This is very useful -- you can see which files a person touched at once, either to see how he was thinking while working, or if maintenance is required, it's easy to find all the changes that went into his contribution to the source tree -- there all described by a single change number.
Re:I had no idea so much still needed work (Score:2, Interesting)
Without these alternate kernel trees, nothing would ever get done. the -ac trees really aren't a tree I'd recommend for a production server unless it has a fix or driver that the server desperately needs... the -ac's are to test and impliment things in advance to KEEP the "stable" tree "stable", and keep Linus happy that the patches he's putting in the mainstream tree have been tested.
And no, I don't feel the 2.4 series deserves to be called stable, but I damn well use it anyway on my primary desktop box
Linus, GIVE US 2.5 TO PLAY WITH!
Re:I had no idea so much still needed work (Score:1)
Linux is quite the maelstrom. I cannot fathom why anyone would use Linux on production boxes when other, more stable and reliable OSes are available.
~g.
Re:I had no idea so much still needed work (Score:1)
>
> Linux is quite the maelstrom. I cannot fathom
> why anyone would use Linux on production boxes
> when other, more stable and reliable OSes are
> available.
And what are those OSes? FreeBSD, Solaris, s/390?
And in what situations are they more stable?
As a firewall/fileserver/desktop?
I've been using Linux for Prodution boxes and for my personal use for about 3 years in so many ways.
And this far the only kernel crashes I've had is with the AgpGart and NVdriver kernel modules.
I'm not saying that *BSD and other OSes aren't more stable in some situations.
I'm just saying that you made a pointless generalitation!
Re:I had no idea so much still needed work (Score:1)
If you experience crashes/oopses that seem to be related to the kernel, PLEASE PLEASE PLEASE report them with all possible information to the linux-kernel list.
Re:I had no idea so much still needed work (Score:1)
if you're going to troll by calling a Windows system more usable than linux, at least use something PLAUSABLE, like win98, in your examples...
Re:I had no idea so much still needed work (Score:2, Insightful)
Now I don't want to start a flame war here, but when someone says "Linux" what do they mean?
There are so many distributions, each with little kernel tweaks (RedHat IMHO is especially bad) and different userland applications. So, I have kernel 2.4.9 with what patches, and what userland apps, what compiler, etc.? I'm totally confused. What *is* Linux?
If I go to kernel.org and download a 30MB tarball of the kernel.... now what about the userland apps? Where do those come from? Who wrote "ps"? What version of "netstat" should I use? Which gcc compiler should I use?
However, with an OS like FreeBSD... "FreeBSD 4.3-RELEASE" refers to one and only one kernel & userland. No questions. No confusion. If there are problems with a particular version, I can retrieve a CVS snapshot of the kernel and userland sources at any point in time since the beginning of the project. Very impressive.
~g.
Linux distributions (Score:1)
Therefore, Red Hat 7.1 is a complete OS with a well defined (perhaps only internally documented) list of utilities/applications at well known individual version stages. Then you can compare RedHat 7.1 with FreeBSD 4.3-RELEASE to find out how they compare.
To answer your question: Linux is the kernel used by compatible OSes, known as "Linux distributions".
Yeah but (Score:1)
When is the MAX_MAP_COUNT fix going to make it into the mainstream kernel?
Re:Yeah but (Score:5, Interesting)
Wait a minute (Score:2)
Perhaps he meant the unstable series Linus releases. I sure as hell would NOT like to see a new "stable" kernel release every few days. The current faux-schedule of a new release every couple of weeks seems a bit too quick for decent testing to me, to tell you the truth.
Re:Wait a minute (Score:4, Informative)
You test continually, you test each changeset, and then every so often you run a several day shakedown test.
You are right that you can't QA a kernel to vendor production grade in two weeks. Some of the RH test runs take several days per run for example.
Re:Wait a minute (Score:2)
Sure, but sometimes you have to test all at once.
I've worked on projects before where the hardware was only partially stable, and the rom code was changed daily and normally couldn't boot. Those doing higher level work on the system were forced to write code for the specs, and debug latter. It isn't the best, but it works. (Yes there is simulation, but simulation tends to comment out all the hardware interaction, which changes things drasticly)
Re:Wait a minute (Score:2)
It's crazy to have stable releases coming out as often as we do. There is no way that you could change enough in the kernel to justify a new stable release 2 weeks (or less) later unless it was a to fix a major oversight (read: bug) in the previous stable release. You just don't have time to test things out.
Re:Wait a minute (Score:1)
With most changes, for example, a driver fix, you don't need to test out the whole kernel, just things that deal specificaly with that driver.
And yes, there are bugs in every "stable" release, there are bugs in every "stable" release of all software.
Go monitor the changelog and the linux-kernel list for a couple weeks, and you'll understand why stable kernels are released within 2-4 weeks of eachother.And no, monitoring the KernelTraffic summaries DOES NOT COUNT as monitoring linux-kernel, far more goes on than KT summarizes.
Re:Wait a minute (Score:1)
Unit testing? (was Re:Wait a minute) (Score:1)
What if the direction was set to "require" developers to submit unit tests for their modules along with them? In not too long it would be possible to *shudder* regression test the entire kernel.
How is the QA testing of the Red Hat Kernel done now? WOuld it be easier or harder to integrate a test suite into the kernel source itself?
Alan misunderstood the question (Score:1)
from the cyfrifiadurol dept... (Score:4, Informative)
And before anyone says it, yes, computers have reached Wales now...
Re:from the cyfrifiadurol dept... (Score:1)
Better not run Harpoon [harpoon4.com] on them, or Greenpeace [greenpeace.org] will be pissed.
Re:from the cyfrifiadurol dept... (Score:3, Funny)
yeah but the cost of classified ads in the paper is prohibitive when they're looking for programmers in llyncyrfdlywrfldycrlycywwcrynrfrwnr...
i'd like to buy a vowel.
(oh crap i think i just called someone's mother a really nasty name)
Re:from the cyfrifiadurol dept... (Score:1)
Re:from the cyfrifiadurol dept... (Score:1)
Yeah. I don't know Welsh either, but my grandparents do, and it's not half as unpronouncable as it looks. I guess we're getting offtopic here, but whatever.
This is the first time in a while there was an ask slashdot that was both interesting (compare to say the Dorm Storm one a while back), and informative (Alan Cox r00lz, though he may not appreciate me saying it in such a manner).
Re:from the cyfrifiadurol dept... (Score:1)
Re:from the cyfrifiadurol dept... (Score:1)
Re:from the cyfrifiadurol dept... (Score:1)
Argh... you "GUESS that the Y *is* a vowel"? Guess again.
Man, are you american or what? An second question: Has America some sort of IQ-restriction for every citizen. Like, say, a maximum of 5? Even dry bread is smarter than you guys.
C. M. Burns
Re:from the cyfrifiadurol dept... (Score:1, Informative)
I'm not sure what you mean here, but it's obvious that you're a real moron. Some letters have different pronunciations in different languages, and 'y' is one of those. In English it is either vowel or consonant depending on the context, which is actually an orthographic mistake in my opinion. This usage comes from Middle French, though. (French has dropped the 'y' in most places, compare old "roy" with modern "roi.") In Latin, as far as I understand it was always a vowel which was used to represent the 'u' sound like in French (like the 'ue'/u+umlaut sound in German). (That's a front high rounded vowel, for the phonetically informed.) This came from Greek and was used only in writing Greek words, since no native Latin word uses the sound. The same goes for old English. In a popular transcription system for Russian, 'y' represents the letter whick looks roughly like Ib in Russian (high middle unrounded vowel?). In many newer writing systems, such as Wolof (to take one that I speak), 'y' is always a consonant, a palatal glide like in "yes."
As a more logical alternative, many languages, such as Dutch, German and Polish, have chosen to use 'j' for the sound equivalent to the English consonant 'y'. This is really more sensible, because 'j' is derived from 'i', which in Latin was used for both a consonant and a vowel, like 'y' in modern English. The practice orginated among scribes wishing to make Latin texts more clear, so they added a little curl at the bottom of the i's that were consonants. People in Central Europe picked this up when developing writing systems for their own vernacular langauges.
So, although I can't say I know any Welsh, I do know a thing or two about languages and alphabets. Why don't you think before you open your fool mouth next time?
Ask Slashdot (Score:1, Offtopic)
I think (Score:2)
Re:I think ... it ate my journal entry! (Score:2)
Give us an update, CT!
Oh, and ignore the 'game client' link below -- if it is still there. Yesterday it pointed to my journal entry.
How about getting XFS in -ac? (Score:1)
Re:How about getting XFS in -ac? (Score:1)
He has commented about this before on linux-kernel. Search, read, be enlightened.
For now it's... no, with good reason. IMHO XFS is the best thing sense sliced bread. In very informal testing it works best out of all the jornaling FSs that I have played with.
Re:How about getting XFS in -ac? (Score:1)
I have heard that it has to do with the fact that XFS changes some other things in the kernel outside of the actual filesystem code, but I don't know if that's actually the problem.
I just hope we don't end up with one or more forks of the kernel because of this.
Re:How about getting XFS in -ac? (Score:1)
Hmm, well. It was a topic last week. maybe it has not hit the lists yet. basicly it (XFS) changes more then just a few things, and more then a new FS should. I know it makes changes to the VM, and scheduling.
There was more info then that, but I hate para-phrasing other people.
Don't you wish there was a built in spell check in slashcode? But then again it keeps me from being too verbose. :>
Re:How about getting XFS in -ac? (Score:1)
Only the first three patch sets are required to get the core XFS functionality. The other four are nice to have but not mandatory. SGI have repeatedly sent the first three sets to Linus, without any response.
If any kernel distributor (including -ac) wants the individual XFS patches, just ask. SGI do not make them generally available because they are snapshots which are only created as required. The normal method of getting XFS is from CVS or as a release, both of which include all the patch sets.
Re:How about getting XFS in -ac? (Score:2)
I really am very impressed with XFS; it seems like solid, proven code. I think XFS has the best chance at being the heavy-duty file system for Linux.
Re:How about getting XFS in -ac? (Score:1)
I can appreciate the problem (Score:4, Interesting)
That's one of the reasons I started that project, in the first place. Because it's mind-numbingly tedious to massage patches from different groups together. If you can get the whole thing in one gigantic gloopy splodge, life would be much easier.
Unfortunately, I've discovered a number of things along the way:
That's not to say that FOLK is a disaster. Quite the opposite! I'm learning a huge amount about the Linux kernel, for a start, and the sheer complexity of juggling hundreds of patches is really giving my C coding skills a workout and a half!
My hat is off to Alan Cox who not only manages his patch set with far more grace than I ever could, but actually keeps it so that it runs!
I know the Royal Web Admin uses Linux (cos that was on an interview, some time ago), so if he's reading & has any influence, I honestly think Sir Cox would not be an undeserved title for his amazing computing skills and his contribution to both computing and Britain.
Re:I can appreciate the problem (Score:2)
Thanks. I'll keep my thoughts to myself from now on. Between this Innovation® and the new unavoidable link to the front page of the site that started to afflict "older stuff" I'm about ready to fugged about
Re:I can appreciate the problem (Score:1)
Comments not being accepted for lameness, or because I type too quickly, or a missing field or some ass error.
Who let this new slashcode into a production environment? Was it ever really tested?
/me thinks it wasn;t
Re:I can appreciate the problem (Score:1)
Promise IDE RAID (Score:2)
Re:Promise IDE RAID (Score:3, Informative)
Re:Promise IDE RAID (Score:1)
Re:Promise IDE RAID (Score:1)
Yeah, same thing with the Abit KT7A-RAID board. Actually IIRC that's a promise board too, just built into the motherboard.
Those extra ATA/100s are really nice for doing software RAID, though.
Re:Promise IDE RAID (Score:2, Informative)
All I can really say is that it's not quite perfect. I'm using an on-board HPT370 [it's part of my Abit VP-6], and under Win2K I don't think I've had any crashes [well other than some reproducable ones that are things that I've done]
But I've used the -ac patches in 2.4.6-2.4.8 and so far there is still a couple of times when my machine will just lock up. It seems to be related to disk access, but it also only happens when I'm running X. Without X, I haven't had any problems [although I can't run Mozilla or XMMS, etc without X]
In general, though, it recognizes and runs fine. I haven't had any general data inconsistency [I run ReiserFS on the RAID partitions]
Again, this is for the HPT370, not the Promise IDE RAID, but since they are in the same kernel patch, I figured their results would be similar.
Re:Promise IDE RAID (Score:2)
I had a HPT366 in my Abit BE6 and have had nothing but problems with it. I won't buy another board that uses a highpoint - they are junk.
HPT366 needs hdparm for stability (Score:2)
/sbin/hdparm -c 3 -m 16 -u 0
cheers
Re:Promise IDE RAID (Score:1)
Don't Feed the Linus (Score:2, Funny)
I also don't feed Linus code that has active maintainers unless the maintainer has asked me to do so.
So Linus eats code. Everything is so clear now...
Re:Don't Feed the Linus (Score:2, Funny)
Re:Don't Feed the Linus (Score:2, Funny)
So does that mean Alan Cox is really Ice-T? [shudder]
Ahem. (Score:1)
"At regular intervals I take stuff from the -ac tree and feed it to Linus."
Take that out of context and think about it.
A Canonical Response (Score:2)
> Take that out of context and think about it.
Just because Alan Cox is east of the US doesn't mean he is a snake.
And why are all of you handing me my coat?
Geoff
Why aren't you guys using CVS? (Score:1)
What is holding AC and Linus (aside from having to learn how to use CVS)?
Re:Why aren't you guys using CVS? (Score:1)
Its worth noting that several subsystems of linux are maintained under their own CVS repositories, with patches being seperated out into functional units and sent to Alan or Linus to sync the project up.
Re:Why aren't you guys using CVS? (Score:1)
cvsup -g ${SUPFILE}&& cd
Linux kernel development is organized in a way to encourage these maverick kernels.
Re:Why aren't you guys using CVS? (Score:1)
Re:Why aren't you guys using CVS? (Score:1)
-AC (Score:1)
OMG!!!! (Score:1)
All my confidence in Linux is lost forever (Score:4, Funny)
What Happens To -AC (And Other) Kernel Mods?
I'm sorry, but if the kernel has a bunch of modifications done by people who find it necessary to be referred to as the initials for Anonymous Coward then how can we trust the security of the kernel?
They get modded down on /. but then get merged into the kernel source? Let's make a stand and stick to it!
Oh, and I copied these comments to a text file so I can repost it in the event that /. pukes up it's guts again.
Re:All my confidence in Linux is lost forever (Score:1)
Re:All my confidence in Linux is lost forever (Score:1)
AC patches: who needs 'em? (Score:1)
Nate
(I'm sorry; I had to do it....)
The BSDs solved this long ago (Score:1)
ac tree... (Score:1)
Re:ac tree... (Score:1)
You can find Alan's kernel patches Here [kernel.org]
Athlon CPU support should (EXPERIMENTAL) (Score:1)
Re:Athlon CPU support should (EXPERIMENTAL) (Score:1, Informative)
Avoiding the streaming memory fetches will help on most of those boards (ie optimise for PII or K6)
Re:Athlon CPU support should (EXPERIMENTAL) (Score:2)
Re:Athlon CPU support should (EXPERIMENTAL) (Score:1)
I've had NO trouble with Athlon optimizations in 2.4.7 and 2.4.8 and now 2.4.8-ac9.
Most of the Athlon problems reported in the past have been problems with VIA's chipsets, which I label as "devil-spawn".
-stable vs -current ? (Score:1)
Instead of having only one tree, there are two: the -stable and the -current. -current is the tree with the newest features and active development going on, the tree which might, or might not, compile, the tree which might, or might not, break your system. The -stable tree is the tree in which everything works, which has no real new active development (all development is done in -current), only merges from the -current track are coming back into it.
For more information, see http://www.freebsd.org/doc/en_US.ISO8859-1/books/
Re:-stable vs -current ? (Score:1)
It's allready there. There isn't any real active development allowed on 2.4, period. It's all fixes and clean ups. Once 2.5 hits then we'll see nifty stuff like a new console subsystem, making everything happy with hotplugging, maybe a few new FS's being merged and other subsystem reorganisations.
- RustyTaco
Re:-stable vs -current ? (Score:1)
those -preXX kernel versions are the "unstable" releases (so to speak) of the vanilla kernel tree. the odd-numbered kernels are development kernels that don't even come close to being stable enough to be called unstable.
the -ac kernels are not simply -stable vs -current, a severely unstable patch that might break things isn't going to intentionaly make it into -ac kernels... you could think of -ac kernels as a testbed for future intigration into the mainstream kernel, but that doesn't make it unstable on the level you're talking about with FreeBSD's -current, that level of instability is reserved for the odd-numbered kernel series (i.e. 2.3 and the forthcoming 2.5).
One does wonder though what clueless businesses that already know odd=unstable for the kernel will think when 3.0 arrives
Re:-stable vs -current ? (Score:1)
you can do WHAT?!?! (Score:1)
Alan's Kernel is becoming more "official" (Score:2, Interesting)
This is very interesting:l /0108.2/0416.html [indiana.edu]
http://www.uwsg.indiana.edu/hypermail/linux/kerne
It seems Alan Cox is considering his -ac kernel tree to be a legitimate alternative to the official "Linus" tree, rather than a playground for testing patches. He's actually perpetuating a difference between -ac and the official tree, in a way that breaks source compatibility between the two (albeit in a very small way.) The fact that RedHat's kernels are all based on -ac now bears this out.
Alan has forked the Linux kernel.
::meyhem::
I think he has Linus' blessing in this though. Reading between the lines, I think Alan has been taking on more and more work in the past year or so that had previously been Linus'. Linux is ten years old now; I suppose Linus is burning out.
And Alan works for RedHat too, which is one of the two distributions that I *know* will be around in ten more years -- they have a solid business plan. Alan is voracious, just tireless, and RedHat would hire the entire core kernel development team if they had to. Linux will not die for lack of a maintainer if Linus gets hit by a bus tomorrow.
(OT)Slashdot does not censor. (Score:1)
There's a big double yellow line with orange traffic cones between improving signal-to-noise and censorship. Slashdot is slalom-skating it.
Ever since CmdrTaco and friends created the moderation system, Slashdot has intentionally deleted exactly one comment, and it was a flagrant copyright infringement [slashdot.org]. If you browse Slashdot at -1, you'll see everything.
Re:(OT)Slashdot does not censor. (Score:1, Offtopic)
Re:(OT)Slashdot does not censor. (Score:1)
Re:(OT)Slashdot does not censor. (Score:1)
Re:Something I'd like to see... (Score:1)
Re:Don't be a playa hata (Score:1)
Re:Random Crap (Score:1)
Thank you for you assistance in my (not so) scientific experiment.