Linux 2.6.17 Released 444
diegocgteleline.es writes "After almost three months, Linux 2.6.17 has been released. The changes include support for Sun Niagara CPUs, a new I/O mechanism called 'splice' which can improve the performance greatly for some applications, a scheduler domain optimized for multicore machines, driver for the widely used broadcom 43xx wifi chip (Apple's Airport Extreme and such), iptables support for the H.323 protocol, CCID2 support for DCCP, softmac layer for the wireless stack, block queue IO tracing, and many other changes listed at the changelog"
Re:Question for the masses. (Score:5, Informative)
Re:Go Linux! (Score:5, Informative)
some highlights from the changelog (Score:5, Informative)
Block queue IO tracing support (blktrace). This allows users to see any traffic happening on a block device queue. In other words, you can get very detailed stadistics of what your disks are doing. User space support tools available in: git://brick.kernel.dk/data/git/blktrace.git
New
Introduce the splice(), tee() and vmsplice() system calls, a new I/O method.
The idea behind splice is the availability of a in-kernel buffer that the user has control over, where "splice()" moves data to/from the buffer from/to an arbitrary file descriptor, while "tee()" copies the data in one buffer to another, ie: it "duplicates" it. The in-buffer however is implemented as a set of reference-counted pointers which the kernel copies around without actually copying the data. So while tee() "duplicates" the in-kernel buffer, in practice it doesn't copy the data but increments the reference pointers, avoiding extra copies of the data. In the same way, splice() can move data from one end to another, but instead of bringing the data from the source to the process' memory and sending back to the destination it just moves it avoiding the extra copy. This new scheme can be used anywhere where a process needs to send something from one end to another, but it doesn't need to touch or even look at the data, just forward it: Avoiding extra copies of data means you don't waste time copying data around (huge performance improvement). For example, you could forward data that comes from a MPEG-4 hardware encoder, and tee() it to duplicate the stream, and write one of the streams to disk, and the other one to a socket for a real-time network broadcast. Again, all without actually physically copying it around in memory.
Re:Go Linux! (Score:4, Informative)
Re:Video Editing? (Score:1, Informative)
Microkernel anyone? (Score:4, Informative)
This is the essence of the Microkernel debate. http://en.wikipedia.org/wiki/Microkernel/ [wikipedia.org] The truth is that the Microkernel model probably is a better design, but in terms of when the Linux kernel was starting out - its implementation simply wasn't pratical. It didn't help that the people who thought they knew how to build a better kernel decided to try and intellectually brow-beat Linus into doing it instead of implementing it themselves and putting it under the GPL. This led to a lot of bitterness and resentment between the two camps. The HURD http://en.wikipedia.org/wiki/Hurd [wikipedia.org] project is a GPL microkernel project, but it simply wasn't managed as well as Linus managed Linux.
I think over time, things eventually will move to a microkernel model even though there are other ways to emulate some of their security and flexability benefits - like xen http://en.wikipedia.org/wiki/Xen [wikipedia.org]
Re:Question for the masses. (Score:3, Informative)
As far as network drivers in particular go, the layers that use them, such as IP, live in the kernel, so it's rather annoying for them to talk to drivers that are up in user space. Specific network cards, especially wireless, might have bits that live up in user space, such as user interfaces for loading in crypto keys, but the bulk data transfer applications normally belong down in or near the kernel.
Why are there a whole pile of network card drivers in the kernel when you'd normally use only one or two? Same reason there are a whole pile of drivers for other devices in the kernel, when you've normally only got one graphics card and one sound card. If you're shipping a pre-compiled kernel, you want it to support as many different users as possible, and all it costs you is some RAM to store the code you're not using, or if you can handle them as loadable modules, it only costs you the work to keep track of those. But if you want to compile your own kernel for specific machines, and leave out the drivers you don't want, and while you're at it compile all your applications programs with the level of optimization your hardware supports, get a copy of Gentoo Linux [gentoo.org] and have fun learning lots more detail about Linux internals.
Re:Video Editing? (Score:5, Informative)
not like that (Score:5, Informative)
Linus refused the FreeBSD-style zero-copy because it is often a lose on SMP and with modern hardware. Page table and TLB updates have huge costs on modern hardware.
If you do like the Microsoft way, use Red Hat's kernel. The in-kernel server works very well.
Re:support for the h.323 protocol, quite unlikely (Score:5, Informative)
obtw: your pedant bit is apparently stuck high. just a fyi -- didn't know if you realized it.
Re:Go Linux! (Score:5, Informative)
That is exactly why it was done. More information about can be found at kerneltrap: here [kerneltrap.org], and here [kerneltrap.org]. It was also previously on slashdot [slashdot.org], although you would be best to skip that - it has more misinformation than the other kind.
In short, all the known ways of implementing zero-copy within the existing API's cause the most common usage cases of those API to be slower than they are now. Therefore, it made more sense to export this new API for the applications where speed is critical.
In the the first kernaltrap article, Linus also explains why splice is different from sendfile, contrary to the posts here claiming they are essentially the same.
Re:Go Linux! (Score:3, Informative)
Programs that want to be backwards compatible can rely on getting an error result of ENOSYS when they try, and can then fall back to using the traditional method.
Re:Had to exchange a motherboard (Score:2, Informative)
One may find it difficult to fit an AMD CPU on an Intel motherboard. Pesky competition.
As the grand-parent said, he had to exchange a motherboard. That means he wasn't intentionally upgrading, thus expecting to continue to use the rest of his hardware (memory, disk, CPU). Were this a system upgrade rather than a replacement of a faulty part then it may be useful to suggest he look at an Intel-based solution. As it is, he obviously already had an AMD CPU and so he needed another motherboard that would work with said CPU.
Re:Missing driver? (Score:5, Informative)
try hitting '/' on make menuconfig, type ov511 hit enter. That's a hot tip that's saved me quite a bit of time...
It'll find it if it's there.
Re:Question for the masses. (Score:2, Informative)
Re:Where is 2.7? (Score:5, Informative)
So instead, the 2.6 goal is to have development/stable parts of the cycle, rather than seperate branches. Roughtly: patches that could break things get submitted at the beginning of the cycle, and -pre1/-pre2 tarballs are released. If you want bleeding edge, you go here. Release candidates are released, where developers get chance to fix bugs etc in the code. Then, any code that's still [known to be] buggy gets dropped for the final release (eg, 2.6.17). The developer can work on it, and try add it again during subsequent cycles. When it works, it can be included in a final release.
During this cycle, security and other urgent bug fixes take place in the ultra-stable branch, with version such as 2.6.16.1, 2.6.16.2.
(This is the rough idea I believe, there could be some slight inaccuracies in how it actually takes place, I haven't followed it 100%, but this should be close enough to get the right idea).
Re:support for the h.323 protocol, quite unlikely (Score:4, Informative)
Unlikely or not, that's what it appears to be. h.323 conntrack nat helper [netfilter.org]
This patch (or module, actually) comes with an H.323 decoding library that is based on H.225 version 4, H.235 version 2 and H.245 version 7. It is extremely optimized for Linux kernel and decodes only the absolutely necessary objects in a signal. ... The total size of code plus data is less than 20 KB.
Doesn't look like a gatekeeper or anything, that looks like an honest-to god ipconntrack nat implementation.
For the other responder to my initial post. I have taken your offer into consideration but have decided to decline.
lol.
Re:Where is 2.7? (Score:5, Informative)
The stable kernels aren't remotely on the bleeding edge; they contain only features which have been tested over the past three months, after being filtered out of the bleeding-edge development as being things that have already stabilized and stand a good chance of being proven in three months. It's effectively very similar, except the development series isn't left known-broken and the stabilization process happens on a quick schedule, with stuff that isn't ready pushed off to the next cycle rather than delaying the current cycle. Also, the version numbers change by less (development gets -mm, -rc, or -git; stable series change the third digit by one instead of the second by two; and bugfix releases change the fourth digit instead of the third).
Re:Video Editing? (Score:4, Informative)
Re:"splice" - because Microsoft did it? (Score:3, Informative)
So even though the whole operation can't be reduced to a splice() or sendfile(), a substantial portion of it can. And the speed improvement you take isn't just that you avoid copying -- as "zero copy" implies; you also avoid unnecessary cache dirtiness.
I wonder what your sample size is for "usual" there. As far as I can tell, you discuss only one case where this is so: Windows. And since we know that Windows has lots of other architectural problems leading to crashes, and in any event has an architecture entirely different from Linux's, we know that case to be irrelevant.All in all, your analysis is critically, embarrassingly bad. There is no "serving Web pages from the kernel" going on here. There is simply an optimization for a common case, with no degradation to the less-common cases -- that's why it's implemented as a separate system call.
Re:Question for the masses. (Score:2, Informative)
but we know what you meant...
Re:not like that (Score:3, Informative)
I think you are wrong. Splice'd data are not processed by userland at all: they are piped from one file to the other at kernel level by page copying.
That and (Score:5, Informative)
C is the best compramise. While assembly might give you the theoritical best code, it'll big a giant mess to try and totally unmaintainable. Might actually be slower and larger for it. C is pretty good because it's easy enough to generate deceant code in, but it isn't much higher up the abstraction chain so it compiles quite efficient.
You have to remember that object orientation and such are all human creations. Processors don't think in objects, for that matter they don't really even think in functions. They think in memory locations, and jumps to those locations. Doing OO code means a whole messy layer the compiler has to go through to translate that in to something the processor actually understands.
Broadcom 43xx HOWTO: (Score:5, Informative)
I'm not an Ubuntu guy, but this reference [ubuntuforums.org] might be useful to anybody trying to make the new Broadcom Wifi driver work in Linux. Very easy steps, and most non-Ubuntu users should find it easy to adapt for their specific distros.
Re:Go Linux! (Score:4, Informative)
There is no overloading going on here. Overloading is to create a new function with the same name, but taking different parameters.
Ahem. The original function, sendfile(2), was rewritten to call splice() instead of doing something else.
Everybody that wrote code that used the old function now has to deal with splice() running instead of the old function's logic.
Just to hammer it home:
Old - app -> sendfile(2) -> some logic -> return to app
New - app -> sendfile(2) -> splice() -> splice's logic -> return to sendfile(2) -> return to app
With the Linux kernel, as this exepmlifies, you can improve the original code and get everyone (well, those to lazy to revert the changes) to use it. In this case you have a fixed API (sendfile(2) which is well known and published) so you don't just want to tell everybody to recompile with called to splice().
See the difference? Feel the difference.
The kernel is GPL and thus the actual source code used to compile the binary kernel you use is available to you. With a closed source kernel you might be able to purchase an SDK with linkable binaries and some (probably undocumetned) header files. Programmers in this situation need things like function overloading and class inheritence just to do anything. One way of looking at the history of languages like C++ is as a technical solution to the ethical problem of closed source programming. Those languages focus on extending on the outside. With OSS you can usually replace, fix and improve on the inside. BSD and GNU differ on a the point of GNU wanting everyone to share the source to those fixes if they share the resulting binaries. But I digress.
And I can't wait to see if this breaks something.
Re:support for the h.323 protocol, quite unlikely (Score:1, Informative)
firewall.
This module supports RAS, Fast-start, H.245 tunnelling, RTP/RTCP
and T.120 based data and applications including audio, video, FAX,
chat, whiteboard, file transfer, etc. For more information, please
see http://nath323.sourceforge.net/ [sourceforge.net].
Re:module shotguns (Score:5, Informative)
Re:not like that (Score:3, Informative)
Re:"splice" - because Microsoft did it? (Score:3, Informative)
WHat is this nonsense ? The khttp in-kernel web server was implemented on Linux first, then copied by MS.
IIRC it isn't even in 2.6 kernels anymore.
So now Linux has a comparable "zero copy" facility
Linux already had a zero-copy facility, splice is just a new improved one.
What are you talking about ?
"Zero copy" tends to be overrated. It makes some benchmarks look good, but it's only useful if your system is mostly doing very dumb I/O bound stuff. In environments where web pages have to be ground through some engine like PHP before they go out, it won't help much.
The usual effect of adding "zero copy" to something is that the performance goes up a little, the complexity goes up a lot, and the number of crashes increases
Not in this case. Linux made it right.
Re:linux (Score:3, Informative)
Re:Video Editing? (Score:4, Informative)
Re:Broadcom 57xx (Score:2, Informative)
That's Broadcom's wired gigabit interface. Has been *long* supported by the tg3 driver. Heck, Broadcom even has an alternative GPL'd driver for this interface downloadable from their website.
Re:module shotguns (Score:5, Informative)
If you're contributing a driver, GREAT. It'll compile against the currently installed kernel just fine.
Untrue I'm afraid. If your modules aren't in-tree then they *will* break every so often because the kernel API is not stable. Especially under the 2.6 development model - under the previous 2.4/2.5 model you were pretty much guaranteed that API breakages would only be happening in the 2.5 tree, now they happen at any point in the 2.6 tree. (Yes, I do know this stuff - I work on out-of-tree kernel code).
There is some arguement that all drivers should be in-tree, and for common hardware it is definately a Good Thing to have the drivers in the tree - as the API changes then the person implementing the API change will fix up all the in-tree code that uses that API.
For very specialist and expensive hardware it poses a problem though: the person who does the API change won't have the hardware to test with, and probably all the people who use that hardware are using enterprise distributions so breakages to the module won't be spotted for a long time. It's hard for the hardware vendor to track these kinds of updates and perform the necessary regression testing.
Re:Go Linux! (Score:3, Informative)
There's no functional difference between using an overloaded name f(a), f(x, y), f(p, q, r) or three separate ones f_a(a), f_xy(x, y), f_pqr(p, q, r).
If you want default arguments C has them, and if you want polymorphism then C has it too (function pointers).
Re:S-ATA hotplug (Score:2, Informative)
echo "scsi remove-single-device 3 0 0 0" >/proc/scsi/scsi
echo "scsi add-single-device 3 0 0 0" >/proc/scsi/scsi
It has to be watched through dmesg, sometimes the drive migrates from
Re:Video Editing? (Score:3, Informative)
There are valid uses for a GOTO (Score:5, Informative)
but, at least in C, there are times when using GOTO is the most natural and,
unequivically, the best choice.
Off the top of my head, I can think of two situations where using a GOTO is
the best solution:
1. breaking out of nested loops. In C, the break command can only break
out of a single loop level. If you need to break out of 2 or more loops, you
can play an ugly game of setting and checking state flags at each level
of looping or you can simply create a label at the exit point and use
GOTO to get there. (sometimes you can wrap your loops as a function call,
but that's often the ugliest solution)
2. shared cleanup code. In a function with multiple exit points, instead
of doing cleanup at each exit point, it is often clearer to set your
return value and then GOTO a label that handles all cleanup before
returning.
Be cautious when using GOTO, but don't be afraid of it. Learn to
recognize when GOTO is appropriate and when it should be avoided.
Re:S-ATA hotplug (Score:3, Informative)
It's not just a matter of enabling it, it's about making it reliable.
More here: http://linux-ata.org/features.html [linux-ata.org]
Re:OK, so where are they? (Score:1, Informative)
<M> Generic IEEE 802.11 Networking Stack
[*] Enable full debugging output
<M> IEEE 802.11 WEP encryption (802.1x)
<M> IEEE 802.11i CCMP support
<M> IEEE 802.11i TKIP encryption
<M> Software MAC add-on to the IEEE 802.11 networking stack
[*] Enable full debugging output
Device Drivers --> Network device support --> Wireless LAN drivers (non-hamradio) & Wireless Extensions
<M> Broadcom BCM43xx wireless support
[*] Broadcom BCM43xx debugging (RECOMMENDED)
Re:There are valid uses for a GOTO (Score:5, Informative)
If that leads to clearer code, then in the cases where you can do that, fine. Do that.
However, there are situations when a condition doesn't make sense until you've already
entered the nested loops at least once (for example, when allocating lots of chuncks of memory,
you can't test to see if you've successfully allocated memory until after you've tried to
allocate memory). Also, if there are several conditions that might require a break, but
they can all be handled the same (at least until after you break out of your loops),
do you really want each one to be tested at every loop test? Think how big and confusing that
would make your continuation test for your outer loops.
2) Use a clean-up function. It will return to the correct place without all the spagetti code.
There's nothing wrong with using cleanup functions if they are convienent for your
particular purpose, but if you have to free 11 objects before returning, then you'll
need to pass all 11 to the cleanup function each time you call it. I don't know about
you, but I usually find functions with 5+ arguments to be ugly. I would rather simply have
a 'goto cleanup' that jumps to a label that does all the cleanup in place. An acceptable
compromise would be to define a macro that does the cleanup in place but hides it from casual
code inspection, thus keeping the code clear, but avoiding the use of GOTO.
Using GOTO in the manners I've described will not lead to speghetti code since the flow of control
will be clear and uni-directional (the antithesis of speghetti code). In case (1), the use
of GOTO is equivalent to raising an exception in Java, C++, or Python from within the loop and
capturing the exception outside the loop (idioms commonly accepted in all three communities).
In case (2), the use of GOTO maps multiple exit points to a single exit point. If you feel
that these techniques qualify as speghetti code, then I would suggest that you've never
seen real speghetti code.
When Djikstra wrote "Goto considered harmful", he was talking about using GOTO to jump outside
the scope of the current function, something not possible in with C's goto (C's goto can only
jump to a label within the current function). See BASIC and PASCAL (I think) for examples of GOTO that
can jump anywhere in the program.