SourceForge Announces Compile Farm 91
HeUnique sent us the NewsAlert press release regarding SourceForge's new compile farm. For the projects hosted there, it means that they will be able to do test compiles on both Linux and *BSD systems.
Re:1st (Score:1)
Re:Entirely off topic... (Score:1)
it started to display slashdot, then noticed, hey, I can't see this image. Rather than just displaying a broken image, I'm going to redirect your browser window to an cryptic error message pointing this out. And when you hit the back button, I'll try to load it again, and then bounce you again.
Typical microsoft 'we know what is good for you'...
Re:Sparc / Alpha ? (Score:1)
Re:Other OS's (Score:1)
MS could contribute a Windows NT terminal server, and SCO could contribute Tarantella (Tarantella works well even if you have 9,600 baud modem - I tried that) - that way you'll get a windows screen with your browser, and you can do stuff remotely with other users..
I really don't know if MS, nor SCO will pick the glove here, and I donno how much an NT terminal server can hold users who are compiling stuff on 1 or some machines..
Again - just an idea.
Re:Writing Portable Software (Score:1)
I believe he was referring to glib [gtk.org], the low-level portability library for GTK (and gimp), not the GNU libc libraries.
From what I understand, it is a very nice, generalized, portability layer. (IANAC ;)
Nice to hear (Score:2)
--
Re:Other Unices? (Score:2)
Increasingly, the quality of applications can be linked to the number of supported platforms and the number of geeky libraries required. The more platforms and fewer libraries, the better.
I'm sorry, but that's simply untrue. Libraries are there so everyone and their mother doesn't have to include the same freaking code in their programs. Libraries are to simplify things and allow code sharing. Maybe you're thinking about libraries from hell which enforce a particular coding style on the users (you know what I'm talking about.) I'm sorry, but I like to see my programs that use JPEG imaging linking with libjpeg, just as I like to see programs using compression streams linking with zlib. You cannot say anything about a program's quality because the author linked it with important support libraries.
1. The GNU version of tar
2. The GNU version of make
I think you meant to say here that programs shouldn't be using GNU extensions of anything. I can agree with that for the most part, but...
3. Any particular kind of C compiler beyond specifying ANSI compliance or other standards compliance
The only real standards for inline assembly usage are in GCC. You're also kidding yourself if you think that most C compilers are nearly as closely standard-compliant as GCC is with ANSI. When something requires machdep code, it has to be done reasonably.
Also, you really need to note that many things won't compile on some extremely esoteric operating systems because they are simply not a good enough facsimile of the Unix API. Don't pretend for one minute that anyone's going to stop using the Unix API for portable software.
--
Re:Nice to hear (Score:3)
Sorry for forgetting this link:
http://bento.FreeBSD.org [freebsd.org] is the moniker for the actual port-building cluster. Satoshi's documented some very good ideas there, including trying each port under its own clean chroot to make sure dependencies are correct.
It seems you've already gotten the idea of putting multiple distros in chroots on the machine, so it could be trivially extended to have a tarball containing that environment that could be unpacked and chrooted to for testing :) That's how the building for 3.X-STABLE is done, on top of a 4.0-CURRENT base.
It might be really nice to have a ports-type tree to do this, with more simplistic Makefiles which have targets for configuring, compiling, installation, and cleaning. Full SourgeForge "tree" building could be done that way =)
--
no, of course not (Score:3)
:)
The GNU configure and build system (Score:2)
--
Whether you think that you can, or that you can't, you are usually right.
Re:Sparc / Alpha ? (Score:3)
Re:Other Unices? (Score:3)
When we had 'free software' it compiled on pretty much everything that smelt of Unix and often on things that didn't.
Now, much of the latest and greatest 'open source' software seems to think that cross platform means RPMS for Red Hat AND SuSE. It really annoys me. I see shit like "This was designed as a cross platform project, so it should work on Linux and *BSD".
Increasingly, the quality of applications can be linked to the number of supported platforms and the number of geeky libraries required. The more platforms and fewer libraries, the better.
My definition of cross platform does NOT include requirements such as:
1. The GNU version of tar
2. The GNU version of make
3. Any particular kind of C compiler beyond specifying ANSI compliance or other standards compliance
At work I have a very well looked after Sparc-solaris 2.7 box. Free software that doesn't build on it, and whose readmes only talk about Linux goes straight in the bin. Yes, I do email the authors so they can fix it. No I don't immediately sign up as the official Solaris porter for the project and fix it myself. Yes I do like to point out that if they'd written it in Perl* they wouldn't have these problems
The traditional argument has been that the developers don't have access to the wide range of platforms to test on. Cynics like me might point out that SHURELY wonderful amazing open source software will never suffer from such problems because it has 2.6 billion potential developers who have every known flavour of every hardware to test it on. Sure...
So, it would be nice to see a place with some HP-UX machines, some RS/6000 gear, and some Irix boxes so that people can actually write cross platform code. Or, to be more accurate, so that people would no longer have an excuse when they didn't write cross platform code.
*Substitute Java for Perl if you feel that way inclined
VA, top linux company (Score:1)
that sell linux boxes, I have come to the following conclusion.
VA is worth the premium*.
I say this because they have the best product support for hardware/software computability, the sales staff is wonderful and they give back to the community.
What other company can claim to provide such a strong base and also be generous and give back to the
movement that enabled them to be?
Thanks VA.
*note: These are my views, not necessarily that of my manager or my company.
Re:Features, Current and Future (Score:1)
~luge
Re:Long Overdue (Score:1)
have a compiler that can compile the standard that's over 10 years old. The only other language
I know of that still caters to pre current-standard stuff is Fortran. Ada, Eiffel, Perl, most other languages just don't have the porting problems that C does.
Re:Writing Portable Software (Score:1)
Re:Writing Portable Software (Score:1)
As for Cosm, interesting concept but I do nto think it will help. I am working with a chat clone (http://www.james-howard.com/display.html?page=pa
Re:Writing Portable Software (Score:1)
Re:Writing Portable Software (Score:1)
Writing Portable Software (Score:4)
Re:Writing Portable Software (Score:2)
Re:Other Unices? (Score:1)
Use a portable language (Score:2)
Cheers,
Ben
Security concerns (Score:3)
Another problem is that some software packages assume that "make install" will always be executed by root and include things like "install -o root" which will fail for non-root users. This is the most common reason for source RPMs which cannot be built by a non-root user.
----
Wrong, an S/390 is perfect for this. (Score:1)
If you missed it here is a quick overview. The S/390 runs virtual machines. You can run (I think 14000 linux machines) on a S/390.
All machines are indepent of each other, and if you mess one machine up, just install from a image file. (Sweet)
The S/390 was a perfect solution to someone that runs a server farm. (Hint, CompileFarm)
Its also cost effective when you consider 1000 i386 boxes or 1 s/390. (Hint, 5Nine anyone?)
Missed the S/390 being used in the article, but sounds like a good idea.
-Brook Harty
Re:Telnet Access to Different *x flavors? (Score:1)
Re:Features, Current and Future (Score:1)
What a dork!
Re:Features, Current and Future (Score:1)
There's already efforts to make some open source programs available on Solaris here [sunfreeware.com]. Make the commitment to at least a few platforms that VA Linux does not sell, so we know you are serious and that this is not just a scheme to market your hardware and that you actually intend to make this the thing you claim it to be. Also, will you commit to having SourceForge on early Itanium [intel.com] machines as soon as you can get them from Intel [intel.com]?
I'm sure there are a lot of issues you have to work with, security being the most critical. For example, what if the project requires root access (some programs need to be SUID root for users, and some are tools for system administration). I know it won't be easy. So get a few Sparc and Alpha boxes, put them behind a tight firewall which prevents people from getting out execpt via their own SSH tunnel, put BSD, Linux, and Solaris up as appropriate, and just let it go as a little "glass world" experiment so you can at least see what the issues are you'll have to deal with.
IndyBox is still better (Score:1)
Re:Features, Current and Future (Score:1)
Lost of people and lots of businesses can put together several machines and run a few different Linux distros and a few BSDs. Far fewer can make a true universal platform farm (the better term is "lab"). A company like VA Linux has the resources to not just to the corner cases, but also fill in all the gaps. If they are true to their word they will, and they won't have any doubts about it up front.
Re:VA, top linux company (Score:1)
I usually dont respond to trolls, but, Ill make an exception here for Mr. Anoymous.
I happened to be the guy heading up that "volunteer project". I cant speak for the 11 other people who were a part of that project, but I can certainly speak for myself.. If you or any other anonymous troll has something to say to me, you can email me. My address is right at the top of this response, and I generally check my mail 2-3 times per day.
If I really had anything to say about the whole mess that happened between VA and System 12 last year, I would have already said it. Infact, I've probably done a few key people at VA a favor by not telling more people what happened. I saw what happened. And yes, I have my suspicions. I choose to keep those to myself these days.
As for what other people may say or think about VA, sorry, that really isnt my concern anymore. I stopped waving the flag for VA once I found out for myself what sort of company they are.. I learned a very important lesson through it all, and I felt other people might somehow benefit from hearing it, I still couldnt say anything, or else get slapped with a libel suit. However, I'm kinda glad a few people know, if you want my honest opinion. Life has a happy way of finding an equilibrium between right and wrong, good and bad. What comes around goes around, I guess. VA isnt exempt from that.
Have a good one,
Bowie J. Poag
Project Founder, PROPAGANDA For Linux (http://propaganda.themes.org [themes.org])
Re:VA, top linux company (Score:1)
Agreed. I dont think you guys (the four of you, specifically) were to blame. You were doing the same as us, basically, working in ultraprivate. However, theres at least one person at VA I know of who cant enjoy a position of innocence, and he knows who he is. He's also the one who has to live with it on his conscience.
Thats enough for me.
Bowie J. Poag
Project Founder, PROPAGANDA For Linux (http://propaganda.themes.org [themes.org])
Re:Security concerns (Score:2)
Anything that uses -o root in the install is broken. That means I can't install things to e.g. a home directory. (There might be exceptions to this rule, e.g. dpkg, but those are very few and far between)
The usual problem with SRPM's, however, are install -o user commands written directly into the
Security? No problemo (Score:2)
What would pose some problems, however, is bandwidth. I don't doubt VA has reams of it, but several hundred simulataneous X sessions (particularly of fancy Gnome/KDE apps with heavy high-color graphics) could bring even a
There will probably have to be a bandwidth policy, to keep things working smoothly. It would be completely reasonable.
(And as an aside, to VA: THANKS!!! The Compile Farm is an excellent idea! I look forward to testing and building my apps there. HP/UX and Alpha binaries, here I come!)
Re:Features, Current and Future (Score:2)
Because even if VA is doing this to help move more hardware, they still deserve the kudos. The compile farm (even if all x86) will allow developers to catch subtle bugs brought on by distribution quirks, as well as allow them to create RPM's *and* DEB's easily from one spot. It's a win-win situation, and one of those wins is ours.
But I definitely agree that other architectures in the farm, including a plethora of non-free Unices, will be a boon to everyone. Each non-GNU system has its own eccentricities and bits of brain-deadness; testing code in such enviroments provides first-rate insight into its robustness and portability. The disadvantage to Linux/gcc is simply the sheer quality and capability of the platform. When you write for it, you're writing for a better Unix than "Unix" itself. (Hence you see all these apps that won't build on commercial Unices, and-- in the worse cases-- even *BSD).
I know when I compile stuff under IRIX (for example), the compiler gives me warnings I don't see under gcc, some header files (libintl.h for one) are nowhere to be found, the paths for X binaries and libraries are completely different, etc. You have to be aware of things like that to write solid configure scripts, and design the code accordingly. Myself, I'm lucky enough to have access to computer labs lined with Sun and SGI workstations. But many developers don't. And even I can't build/test anything on HP/UX, Tru64, AIX, etc. MIT doesn't support those {:-(
What you're saying, however-- having each and every Linux/BSD/etc. on each and every platform that they runs on-- is not worthwhile. Sure, that covers a lot of possible end-user setups, but keep in mind:
- Do you want to test and build for 30+ slightly different OS/platform combos?
- Heck, forget using; who here wants to administrate that??
- Is it even necessary? Sure, Linux distros have some incompatibilities, but IT'S NOT THAT BAD!!! This isn't UNIX, after all...
- In the end, when you're building something with gcc, the platform doesn't make all that much of a difference anyway. The GNU libraries are well-designed, and work around a lot of those issues. (Endianness isn't even that much of a problem, if you code things the right way)
IMHO, it would be much more sensible to add in a group of machines widely spread across the spectrum of platforms. Corner cases, if you will. If your app can build on three weird platforms, it'll probably build on a fourth. But you can write an app that will compile on every Linux distro in existence, yet fail horribly on any commercial Unix system you can name.And let me emphasis the need to shun gcc and test using vendor-native compilers. If you're on a non-x86 platform, and gcc isn't the only compiler available, you have to cater to people who don't have gcc installed. And more important, the compiler warning messages tend to be a lot more interesting, simply because they're different from gcc's (and gcc's are almost always the same)
But putting an S/390 into the farm is overkill. Far and beyond. Those machines are waaaaay too expensive for something like this. (Seriously; the only thing to top it would be a Cray supercomputer!)
You can't always count on standards (Score:2)
E.g. older libc5 Linux systems don't have strtok_r() -- autoconf can detect this, tell automake to build strtok_r.c, and add strtok_r.o to @LIBOBJS@. Some systems don't quite agree on what the argument types to select() are -- autoconf can determine those, and put them into handy macros you can use for casting. Etc., etc.
The point being: Yes, write your code The Right Way(tm). Anything less is a hack. But use autoconf so that folks with less-than-perfect operating systems can compile it.
Re:Other OS's (Score:2)
As for Microsoft contributing anything: you have got to be kidding. Those boys not only don't have to encourage anyone to write stuff for their platform, they happily charge an arm and a leg for VC++ and all the MSDN program junk....
/. sing along... (Score:3)
E- I/O I/O.
And on this farm they had a cluster..
E- I/O I/O.
With a make make here,
And a make make there..
Make make make make everywhere!
Ol' Sourceforge had a compiler farm..
E- I/O I/O.
---
Re:Shell or Web? (Score:2)
Re:Writing Portable Software (Score:2)
A most excellent suggestion. Myself for one would find such a tool to be invaluable. Most developers who don't work on more than one machine would be surpised at some the assumptions they make. At home I work on a Linux PC. At work I have a Solaris Sparc5. When I compile my code at work I often discover where I have made my bad assumptions.
This is a great way to get rid of "linuxisms", but it requires a range of hardware to test on. The SourceForge idea is great, but it needs to be expanded as opportunity and cash allows. Add some Solaris and IRIX boxes. Include a larger variety of hardware.
Re:Writing Portable Software (Score:4)
The best way to write a program that will work on any Unix is simply not to write one for Linux only
And finally, don't make the same mistake millions of other developers make every day, don't assume that the user has exactly the same system and setup that you do. Don't assume that the user has a large monitor or lots of memory. Don't assume that they have an active connection to the net. Don't assume that home directories are under
Re:VA, top linux company (Score:2)
Dude, You need to quit hitting the crack pipe.
We did not steal any ideas from any group of volunteers. The planning and development of SourceForge was done by 4 people. Those 4 were Tim Perdue, Drew Streib, Uriah Welcome, and myself. When planning and developing SourceForge we received no input from the community until after the site launch on Nov. 17th, 1999. We also received no input from VA. We worked on SourceForge in a "black ops" environment. By this I mean that we were kept separate and shielded from normal channels within VA. I know this because I set up the structure to be this way.
As far as giving back to the community we have tried since we launched the site to listen to the direction the community has wanted to take SourceForge. This is the community at work. Period.
We also released the code under the GPL so that others could help improve our code and if they were so inclined they could use the code to setup their own site.
We are also currently in the process of allowing other developers to work on the "live" code per se to help improve SourceForge.
Re:Security concerns (Score:2)
As for the root thingy... BSD has jail()... As for linux I'm not sure, chroot is not root-resistant.
Re:X Window support? (Score:1)
Of course, there certainly may be security issues. Not sure how to handle that well...
Other Unices? (Score:4)
(Of course, compiling under Windows/Mac/BeOS/etc. too would be even better, for programs which are intended to be used there.)
Re:Telnet Access to Different *x flavors? (Score:3)
BTW, it wasn't posted a few years back, just 1-1.5 year ago...
Enjoy
Re:Sparc / Alpha ? (Score:1)
box happens to compile your app, doesn't mean
that it does on a similar alpha or sparc redhat
box, since the endianess/64bit issues still may
bite you.
Sure, you go a big step towards portability
if you get it to compile cleanly on some linux
and some *BSD, but I still feel that different
platforms would be preferable.
Re:Writing Portable Software (Score:3)
The combination of autoconf and automake is definitely the way to go.
The autoconf and automake docs can be a bit obscure; most people seem to learn how to use them by studying the configure.in and Makefile.am files from other projects.
There is also a brief discussion of autoconf and automake in Havoc Pennington's Gtk+/Gnome Application Development [gnome.org] that might help you get started. This fine book is available on-line: here [gnome.org] is the relevant section.
What architectures? (Score:4)
Many bugs in the Debian Bugtracking system are "fail to compile under foo arch" bugs, so it would be very cool to be able to test the code under various archs to avoid this. That would make SourceForge a unique environment to squash these types of problems, as many programmers can be aware their code does not compile on Sparc but they can't do anything about it as they don't have access to a Sparc machine.
About being able to execute binaries, and X and all... that sounds a bit more difficult here, imagine what hardware you need to allow that number of X sessions?
Re:no, of course not (Score:1)
Only if they've got PPC-compatible CPUs :-)
Re:Long Overdue (Score:1)
I've built (pre 1.0) xmms from unmodified source and used it on FreeBSD without any problems.
I plan on letting SF know that they can send people to the various BSD Ports Collections (Available via FreeBSD's CVSWeb [freebsd.org]) so they can incorporate the various OS specific patches into their source.
(Such changes would exclude items that are specific to the Ports system, like overriding PATHs and make variables, etc. These items would remain in the Ports patches.)
I'll be having articles in the FreeBSDZine [freebsdzine.org] about FreeBSD's Ports system in the near future.
Long Overdue (Score:3)
This is why C was invented in the first place, but too many people forget it.
Many programs already compile on multiple OSs, and kudos to their respective authors for writing good code. Other programs only require minor changes, which is where autoconf makes it easy. Then there's the occasional code that assumes Linux only (kernel or
</RANT>
Then there's the whole "Linux" software phenomenon. It's sad when people don't realize that the programs they use in Linux aren't "Linux" programs, they're Unix programs. I don't just mean grep, etc. I mean programs like XMMS, GNOME, KDE, etc. This stuff compiles, unmodified, on the *BSDs as well. Even some authors call their software "Linux" software when it's portable *as is*. This is misleading, both towards the Media, who pick up on it, but to new free Unix users as well, who might be deciding between Linux and a BSD. This new offering from SourceForge will not only provide a place for authors to test their code, it will help educate people as to the true nature of Open Source as well.
`./configure && make install` shall set you free.
<RANT>
Re:Writing Portable Software (Score:2)
Perhaps a testing enviornment could be written that emulates the various configurations above - ie small and large amounts of memory, small vs large screens, active and innactive net connections, as well as other areas that are often overlooked by programers - millions of colors vs four bit graphics...
Just a thought,
LetterRip
Farms? Hmm... (Score:2)
Re:VA, top linux company (Score:1)
Re:URL? (Score:1)
Re:Shell or Web? (Score:1)
Evidently, there is some web development going on. It's not clear, however, just how extensive the web based support will be.
URL? (Score:2)
anyone have the sourceforge or va linux url for this? I can't seem to locate it on either site.
Cthulhu for President! [cthulhu.org]
Re:Writing Portable Software (Score:3)
What you need is an abstraction layer to allow the same set of code (read: your code) to be compiled without alteration or other system dependance on various platforms -- this means more than ten versions of Linux/FreeBSD. The Cosm [mithral.com] CPU/OS layer is designed to do just that. With a Cosm library for a given platform, your code will compile without modification.
Cosm [mithral.com] is still in development (slow, but what do you want for free.) The Cosm API source [mithral.com] is available via http [mithral.com], ftp [mithral.com], and CVS [mithral.com]. Most of the functionality one would need is there. We [mithral.com] would welcome your input and feedback.
Re:Writing Portable Software (Score:1)
RPM (Score:1)
Re:Shell or Web? (Score:2)
What may be more interesting, and have serious ramifications, is the fact that you're storing your data off site on SourceForge's servers. A coordinated DOS attack to keep you from your source? Crackers going in and modifying your files perhaps?
Re:Nice to hear (Score:1)
The FreeBSD Ports site at first glance looks fantastic. I'll definately email the webmaster for ideas. I'd love to see something along this lines for other Unices as well, and perhaps even for Linux/BSD/Unix porting as well.
---
SourceForge Programmer Type - http://sourceforge.net
Re:Shell or Web? (Score:4)
---
SourceForge Programmer Type - http://sourceforge.net
Features, Current and Future (Score:5)
I see a lot of questions regarding what is available on the Compile Farm, and a lot of great ideas as well. I'd like to dispel a couple of myths and offer some insight as to where this is going as well.
First, this is not a web-only service. We do like to provide web interfaces to as much as possible, but we do realize that for some things, program compliation and testing included, nothing can substitute for shell access.
A lot of people are asking about other hardware architectures and OS's. For now, the Compile Farm is i386 based, and contains several Linux distributions and FreeBSD. This does not mean that we have ruled out other possibilities. This is just another step in what we hope can be an expanding feature set for Open Source developers on SourceForge.
There is a lot of setup involved in something like this Compile Farm, not the least of which is having thousands of skilled Open Source developers with shell accounts on a set of boxes. We're attempting to keep things as secure as possible while also offering enough features to make this thing useful. One reason for the limited number of distributions/architectures/OS's now is the limitation of variables in a very complex system. Hopefully, we can work out the kinks in this system soon so that it can become a valuable resource to developers who might not otherwise have the capability of getting their hands on so many different machines.
We're also working on giving users the advantages ot having a cluster of machines available. Uriah Welcome worked very hard to provide parallel make capability to projects, and this is being tested now. (Parallel makes will allow you to take advantage of multiple dual-processor machines simultaneously in your compiles.)
Please be patient as we test this new system. We're definately open to criticism, but please also be constructive with it so that we can continue to improve these services. Thanks to all of the SourceForge users who have contributed patches, criticism, and helpful suggestions. Every day my confidence in the Open Source model increases...
---
SourceForge Programmer Type - http://sourceforge.net
Re:Shell or Web? -- Both (Score:1)
Telnet Access to Different *x flavors? (Score:2)
If anyone could remind me where that was, I'd be much obliged!
ICQ: 49636524
snowphoton@mindspring.com
X Window support? (Score:1)
Re:Security concerns (Score:2)
Re:Writing Portable Software (Score:1)
As for writing portable code that work on other OS platforms than BSDs, I've found the Glib is an excellent tool for that. It's even Win32 enabled.
Re:Sparc / Alpha ? (Score:1)
Re:Long Overdue (Score:1)
Nah, make install && make distclean does it for me. :-)
Re:Writing Portable Software (Score:1)
Re:Writing Portable Software (Score:2)
Shell or Web? (Score:2)
Re:What architectures? (Score:1)
Sparc / Alpha ? (Score:2)
Other OS's (Score:1)
I understand that the primary target of open source projects these days is open source OS's, but once we get over the 'cool' factor of open source, then maybe we can start developing for mainstream stuff so that the average joe with his win98 machine can take advantage of it.
Now I also realize that it is difficult to provide remote access to windows and macs. Perhaps a better solution to that would be a network of developers who each would be willing to compile other people's source on their machines, and send back bugs, screenshots etc.