Should You Pre-Compile Binaries or Roll Your Own? 301
Jane Walker writes "The completion of pre-compiled packages and maximizing machine performance are two powerful incentives for Windows admins to use Linux and compile an OSS package." TechTarget has an article taking a look at some of the "why" behind rolling your own. What preferences have other Slashdot users developed, and why?
Eh? (Score:5, Insightful)
Was I the only one who found that this article didn't really shed too much light on whether or not you should compile your software from source?
By the way, I know the benefits of compiling from source, but how this made slashdot, I don't know.
Advantages to both (Score:5, Insightful)
Conversely, if programmers sufficiently document their binaries, that's not as much of a problem. URLs for other binaries, or instructions for checking/compiling dependencies, can speed up that process.
Of course, binaries are a huge advantage to non-experts and beginners, who just want a program to be there. They don't care about maximizing efficiency, they care about getting their work/fun done.
So really, it entirely depends on the application. For programs that are typically for the hardcore programmer/user crowd, source-only makes sense -- those people who use the program are going to know how to compile and check everything already. But for programs like, say, a CD burning program? I definitely think both should be provided, and installation for both should be well documented. Given how easy it is to provide a binary, even if it's "fat," there's no reason why a more popular program can't provide both versions.
As usual, the answer is... (Score:4, Insightful)
If the time required to compile (plus trouble) is less than the time/trouble saved through performance gains (assuming you actually compile it correctly to get said gains), then compile. Else download and install.
But then again, if you just have a lot of time on your hands, it doesn't really matter what you do.
TEN PERCENT! (Score:5, Insightful)
"Absolute best performance" fallacy (Score:5, Insightful)
I am Between Self Compiling and Gentoo (Score:4, Insightful)
A co-worker introduced me to Gentoo late last year and I have to say I am very impressed. It's much faster than the optimizations I was using. Of course I didn't compile everything in RedHat or Fedora by hand. That's why Gentoo really rocks. You CAN hand compile everything from the ground up! I also used to use Linux From Scratch. And YES, I do use this stuff on production machines. You just can't get any better performance or security by doing it all yourself. The only reason to use package managers is if you are new to Linux or just don't want to learn much. But if you don't dig in, then you're at risk of thinking that something in your installation is broken, when it's not. I've seen many people throw up their hands saying, "I have to re-install my Linux box dammit!" when all they really needed to do was fix a symlink, edit a config file or downgrade something for compatibility reasons.
For example, on a laptop at home I decided I wanted to use the Xdmx server from X.org, so I hand compiled X.org. After that, I kept having this problem where the acpid (tracks the laptop's battery life among other things) daemon wasn't starting and would produce an error message whenever I logged into Gnome. I dug around on the net for quite a while and finally found out that the version of X.org (a devel version, not a stable version) grabs ACPI if you are using the RedHat graphical boot screen. The fix? Stop the RHGB from running by removing the RHGB kernel option. I think a lot of people would have assumed they hosed their installation and reinstalled if that problem really bothered them. It's not hard to find solutions to most problems in Linux no matter how obscure. That's why only knowing how to use pre-compiled binaries is a detriment if you're serious about using Linux.
deployment? (Score:5, Insightful)
They are not only wrong in their conclusion, but the article barely scratches the surface of the question.
Put simply, compiling software on your own is fine for a one off, or your desktop, or your hobby machine.ou.. or if you either a) need the latest wizbang features (and maybe can put up with the latest gobang bugs) or b) need really specific version control or c) can't find a precompiled package with the right ompile time options set.
Other than that, you should pretty much always use pre-built.
Sure, you can build your entire system from scratch if you like, libc on up. Why? The performance increase will be so minor that you will have to run benchmarks if you even want to be able to tell there was a change. You will then have to recompile everything you ever decide to update.
This strategy is a nightmare as the systems get more diverse and complex.
it also has nothing to do with deployment. Deployment is what you do after you have decided what you want and made it work once. Deployment is when you go and put it on some number of machines to do real work.
I would love to see them talk more about the differences in deployment. With precompiled packages from the os vendor, ala debian or redhat, its easy. You use apt-get or rpm and off you go. Maybe you even have a redhat network satalite or a local apt repository to give yourself more control. Then you can easily inject local packages or control the stream (no, sorry, I am NOT ready to upgrade to the new release)
but should you compile "most of the time"? hell no!
It is, in fact, the vast minority of software where you really need the latest features and or special compile options. Its the vast minority of the time where the performance increase will even be perceptable.
Why waste the time and cpu cycles? Takes over a day to get a full gentoo desktop going, and for what? I can have ubuntu installed and ready to go with a full desktop in maybe 2 hours.
Lets take the common scenario.... new openssl remote root exploit comes out. The email you read just basically said, in no uncertain terms, that half your network is now just waiting to be rooted by the next script kiddie who notices. Thats lets say... 50 machines.
Now your job is to deploy a new openssl to all these machines.
You could notice that the vulnerability came out in such a time frame that they allowed the OS vendors like debian to release fixes (often happens, if not they are usually out within a very reasonable time frame)... so you hit apt-get update && apt-get upgrade
or maybe you just promote the packgae into your repository, and let the automated updates deploy it for you. You go home, have a coffee, and be ready to log in remotly if anything goes tits up.
Now if you are hand compiling what do you do? Compile, test. And then um.... ahh... scp the dir to 49 machines and ssh in and do a make install on each?
How is this better than using a package manager again? Now you ahve dirs sitting around, you have to hope that your compile box is sufficiently similar to the other boxes that you didn't just add a library requirement (say because configure found that yoru dev box has libfuckmyshit.so installed and this neat new bits of openssl can make use of it)
How about when a box crashes and burns and you now need to take your lovingly handcrafted box, and rebuild it, and put all that lovingly compiles software back on it.
Fuck all that... give me a good binary distro any day of the week. I will hand compile when I HAVE to... and not before.
-Steve
The real issue is dependencies (Score:5, Insightful)
If you're compiling your own for performance reasons, don't bother. There's a few packages that can benefit from being compiled for specific hardware, the kernel and libc, for example, and things like math libraries that can use special instructions when compiled for specific hardware. For the most part, though, your apps aren't going to be limited by the instruction set, they'll be limited by things like graphics and disk I/O performance and available memory. IMHO if you're trying to squeeze the last 1% of performance out of a system, you probably should look at whether you need to just throw more hardware horsepower at the problem.
The big benefits and drawbacks of custom-compiled vs. pre-built packages is in the dependencies. Pre-built packages don't require you to install development packages, compilers and all the cruft that goes along with a development environment. You can slap those packages in and go with a lot less installation, and you can be sure they're built with a reasonable selection of features. On the other hand, those pre-built packages come with dependencies. When you install a pre-built package you pretty much have to have the versions of all the libraries and other software it depends on. By contrast, when you compile your own packages they'll build against the versions of other software you already have, using all the same compiler options, and they'll probably auto-configure to not use stuff you don't have installed. This leads to a more consistent set of binaries, fewer situations where you need multiple versions of other packages installed (eg. having to install Python 2.3 for package X alongside Python 2.4 for package Y) and overall a cleaner package tree.
Where the cut-off is depends on your situation. If there's only a few instances of dependency cruft, it may not be an issue. If you have a lot of dueling dependencies, it may be worth while to recompile to reduce the number of different versions of stuff you need. If you've got externally-dictated constraints (eg. only one version of OpenSSL is approved for your systems and everything must use it or not use SSL at all) then you may have no choice but to compile your own if you can't find a pre-built package that was built against the appropriate versions of libraries.
hardware, ease of distribution, latest version (Score:4, Insightful)
Another missed point is it's usually easier for the developers and hardware vendors. Easier to distribute source code than maintain dozens of binaries. Just another advantage of open source. Many projects have a few binaries for the most popular platforms, and source code for those who want to use something less common.
Latest version and leading edge. I've been trying Gentoo. In most cases the extra performance isn't much, and isn't worth the hassle. And Gentoo defeats one of the attractions of roll your own: the leading edge. There is always a lag between when a project releases a new version, and a distributor can get around to turning the new version into a new package. Article didn't mention that aspect either. If your distro is slow, you can bypass them and go straight to the source, or, if available, the source's binaries. Ubuntu Linux 5.10, for instance, is still stuck on Firefox 1.0.7. I'm not talking about checking out the very latest CVS source tree, just the latest release.
Re:I am Between Self Compiling and Gentoo (Score:4, Insightful)
You learn not to.
Yes, there are times when a source based distro is good. There are times when you NEED the level of control that it gives you.
Most jobs, in most environments don't. In fact, most sysadmins that I have seen just don't have the resources to exert that much control, or put that much love and care into every system.... nor should they.
I have said before, in specific cases (research computing clusters where you essentially have 3 machines, one of which is copied n times as "nodes" that can be reimaged from a new image pretty much at will - is the one case where I have really seen it used to good effect) source based distros are great. Of for your hobby machine, or you r laptop.
As soon as you start talking less about "running linux" and more about "deploying services", the focus shifts. Source based distros are a management nightmare in any manner of large or hetrogeneous environment.
Frankly, the majority of systems that I have had responsibility for havn't even had a compiler on them, never mind a full blown development environment, usually not even library headers.
Why? Because we don't want admins compiling code on just any box and tossing it in place. So, why make it easy for them to do so? Nothing should EVER EVER EVER be compiled on the production mail server, or the web server.... it should be compiled on the compiler machine, the dev box.
When you start making distinctions between the roles of boxes like that, as you should do in any larger environment, then you start to see the benefits of a source based distro melt away, and the real power of package management come into full effect.
Most linux users, the home user, will never see this. I know I didn't understand it until I had been doing it for a living for a few years.
-Steve
Re:I am Between Self Compiling and Gentoo (Score:4, Insightful)
The performance can be debated, but you have got the security argument backwards. If you use pre-packaged binaries, you can get security updates quickly and automatically because any responsible Linux distributor will provide updated packages in a timely manner. This is especially useful if you maintain a large number of machines and want to make sure that the latest security patches are quickly applied to all of them.
On the other hand, compiling your own software requires you to pay attention to all security announcements and apply the security patches yourself. It may be possible to automate that to some extent (e.g., Gentoo provides some help for security patches), but not as easily as with the pre-packaged binaries.
From a security point of view, I am glad that some others are doing most of the work for me. The issue of performance can be debated because there are many pros and cons, but security is usually better for the pre-packaged binaries. Note that I am talking about security from the point of view of applying the latest security patches and staying ahead of the script kiddies, not in terms of initial configuration of the system. And by the way, I am not against compiling software on my own because I compile several packages from sources (directly from cvs or svn, especially for the stuff that I contribute to). But then I am aware that the security may actually be weaker for those packages than for the ones that I can update automatically.
Yes, pre-compile - but not for the obvious reason (Score:5, Insightful)
However, I don't consider performance to be a particularly big benefit. Once you've got gcc doing it's basic -O3 or whatever, anything else is marginal at best.
There are, however, two things I get out of it which are useful to me. These should not be ignored:
Of course, there are drawbacks:
I guess it's just a case of "know the tool you're using".
Re:Don't waste your time. (Score:1, Insightful)
And it's been posted before, but http://funroll-loops.org./ [funroll-loops.org.]
ps: you can make Debian do the same thing, you just don't have the USE flags.
Another great thing about managed code... (Score:3, Insightful)
In platforms like
No need to break out gcc or try and resolve dependencies. Ahh... I loved the managed world.
Comment removed (Score:5, Insightful)
Re:Gentoo? (Score:1, Insightful)
Well, you have some very strong points against Gentoo. Also, you're right on all accounts. But there are some remarks here.
First of all, it's not that big of a speedup, quite honestly, I use Gentoo because I just really like the way it works. I get more done with less. It's not the build-from-source part that I like, it's just the entire system as a whole, I love using it.
Second of all, those USE flags, they need some upstream support to build in or strip out features. Fortunately, there are a bunch of packages that support this. Some USE flags for example do some work in Apache to enable or disable features, that does help. Same thing goes for PHP.
The etc-update feature of Gentoo does help a lot. It allows for upgrading to packages to keep up with new features. For example, Shorewall recently got upgraded in Portage. This changed a bunch of the config files. I suppose Debian has some post install script in place to fix that. Though I tend to not look at an update all the time. This way, I get the updated configs later.
I suppose a postinstall script works fine with a binary package, short install time. But the etc-update works better for Gentoo, longer install time.
The definite speed advantages of Gentoo are arguable, but that's not always the reason for the switch.
On a different note. I'm glad to say that my parents are happy Ubuntu users.
A Linux distro should fit your personal needs. I like Gentoo as my parents are just content with Ubuntu. But that's what FOSS is about, having a choice.
Re:Gentoo? (Score:2, Insightful)
How to get best of both worlds (Score:2, Insightful)
Just compile speed-critical programs yourself and get everything else from a stable distribution.
Re:Yeah, that will make people REALLY want to adop (Score:3, Insightful)
Re:Gentoo? (Score:2, Insightful)
Does a custom-compiled compiler create different binaries to a pre-packaged compiler? I was under the impression that it might compile the application faster, but the resulting linked-up, ready-to-run binary is no different. So "it does accumulate" doesn't add up to me...
Just nit-picking, sorry...
It's not really about performance (Score:5, Insightful)
If there is one thing that appeals to me about Gentoo (as I understand it), it's the concept of meta-configuration at build time. Unfortunately, lots of options in packages get configured at BUILD TIME, so either the binary packages have those options on, or they don't. When this is the case, if the distro doesn't provide multiple binary packages with all of the useful configurations, then you end up having to build from source. (IMO, building from source means compiling from source packages whenever possible.)
So I like the concept of saying in one place, "build for KDE", "build with ssl support", "build with ldap support", etc. Maybe someday everything will be runtime configurable, but until that day, I'll be wishing for that meta-level of configuration...
Having said all of that, check out apt-source, apt-build, and their ilk if you're interested in "Debian from source". Debian binary packages solve about 97% of my problems so I'm not usually looking for "make world" kind of functionality.
Enough rambling for now.
Re:Gentoo vs. Debian (Score:1, Insightful)
Unfortunate for people who don't care about freedom -- but if that's the case, why aren't you using Red Hat or SUSE?
Re:"Absolute best performance" fallacy (Score:1, Insightful)
Re:Gentoo? (Score:2, Insightful)