The Future of NetBSD

ErisCalmsme writes "In this email Charles Hannum (one of the founders of NetBSD) tells us that 'The NetBSD Project has stagnated to the point of irrelevance. It has gotten to the point that being associated with the project is often more of a liability than an asset. I will attempt to explain how this happened, what the current state of affairs is, and what needs to be done to attempt to fix the situation.' What will happen to NetBSD?"

Very well put...

[chaoskitty]

While there will be those who see this as flamebait, it's high time someone puts into words what many of us are thinking - namely, that something's not quite right, and we should look to those with more experience to give us some clues...

Sounds Interesting

[awss82]

It appears that it is dying, but I think it is not going to affect many. I mean many people use other free operating systems but not many use BSD. And I guess that's the reason why it is dying.

Doesn't seem right

[bobintetley]

I love NetSBD. It's stable, it's fast, the package management is great (and upto date), NetBSD folks don't seem to feel the need to evangalise and beat people over the head with their OS choice. A lot of interesting development is also done in NetBSD (like integrating Xen into NetBSD 3.0, the CCD driver, RAIDframe, etc).

I don't understand what this guy's on about - I use it and love it, so do lots of other people, we have upto date software and a great base system. How exactly is NetBSD irrelevant again? Is he bitching because of a lack of marketshare compared to other BSD/Linux distros? In a world of free software, why exactly does that matter?

It's disingenuous to bitch about the things he does as if they were important - flash file system? So what? Journaled file system? There's a very good reason for the omission of journalling and you can't tell me this guy doesn't know about softdeps.

Just sounds to me like this guy is pissed off with not getting some kind of glory for his work and it's all sour grapes.

Mergers and Acquisition

[delirium of disorder]

OpenBSD was a fork() of netbsd. Is there any chance they could reunite to make a single stronger OS? How difficult would reconciling the politics and the codebase be?

respect

[tezbobobo]

I must say, it is an interesting read but I am struck by the humility and honesty of this guy.

Not surprized

[PrayingWolf]

I've been waiting for this to happen ever since I read how Theo De Raadt was treated in there and how he eventually left the group to work on his own branch. I think you can find an archive of his emails with the NetBSD dev team somewhere...

Now the problem is admitted: FTA:

Partly due to lack of people, and partly due to a more corporate mentality, projects were often "locked". One person would say they were working on a project, and everyone else would be told to refer to them. Often these projects stagnated, or never progressed at all. If they did, the motivators were often very slow. As a result, many important projects have moved at a glacial pace, or never materialized at all.

This is basically what drove Theo out (as far as I understand his great ideas were ignored by the boureaucratic system and he felt frustrated) and now the basic reason why NetBSD is dying.

But NetBSD still lives: in its decendants, like OpenBSD. So let us treat NetBSD with the same respect we would give to a dying grandfather :)

What goes around comes around

[Renegade88]

I'm a big BSD guy, mainly a FreeBSD user, but I intently follow DragonFlyBSD and OpenBSD. Unless I'm mistaken, this is the same Charles Hannum that was directly responsible for kicking fellow NetBSD founder, Theo de Raadt, out of the core group, removed his CVS priviledges, and made Theo twist in the wind for 7 months until he was forced to leave to found OpenBSD. (reading the log I don't see how Theo lasted 7 weeks, he really made an effort to continue with NetBSD despite all of that). So now the evil cabal takes over and kicks Charles out of the core and removes his commit priviledges. It's sad, and I think Charles' points are spot on, but it's a bitter pill to swallow coming from this messenger. You have to shake your head when you think of what NetBSD could have been had they been able to avoid childish political antics in their "cabal".

Re:How many BSDs do we need?

[Draco_es]

NetBSD has features that others don't and in some aspects is innovative(or at least different), so it's valuable(for its own users and for the whole OSS "universe"). What we don't need is the zillionth Linux distro, which just repackages applications in a different way.

Interesting read

[porkThreeWays]

It is a pretty interesting read. I can give you my experience with NetBSD over the past couple of years...

Outside of my regular job we were developing an embedded system. The first thing I thought of was NetBSD. Downloaded it, tested it, critiqued it, and couldn't find enough benefit to use it. The big gotcha was there was no filesystem at the time for running on flash devices. Well, almost every embedded project is going to run on a flash device. Mind you this was a couple of years ago, but according to the post not much has changed. There were a couple of other small gotchas, but in comparing it to Linux, there just wasn't enough reason to use NetBSD.

And therein lies much of the problem. I don't think NetBSD is bad. It's not. However, a lot more people are using Linux for advanced embedded devices than NetBSD and are solving real world problems so you don't have to. NetBSD may run on a plethora of hardware pretty well. But 90% of the embedded world really needs it to run on is i386, arm, and mips. So there is really good linux support for those arches because so many people are developing systems with the linux/uclibc/arm combo. It's the new lamp. NetBSD may have the shock factor of running on things like toasters, but Linux is running on real world things like my phone.

On top of that, the term "embedded" is becoming looser and looser. There was a time when "embedded" meant a 12mhz processor and everything was in assembly and C. Today, I can get a 400mhz gumstix and do all my development in python. I would consider it embedded by today's standards, but in reality that was a normal desktop development machine 5 years ago.

Again, NetBSD isn't bad. If I had to really run something on a 12mhz CPU I doubt I'd be able to use linux/uclibc/arm and NetBSD might be my answer. However, in a world where embedded hardware is the desktop hardware of 5 years ago, there just isn't any benefit to trying to use the same embedded tools of 5 years ago.

Re:What goes around comes around

[macshit]

Charles Hannum is a cool guy (I worked in the same office with him for years), and very, very smart. Theo is also smart, but well-known for being a complete and utter asshole much of the time.

I never followed the theo/netbsd split closely (not being part of netbsd project), but I suspect a great part of the blame for the split lies squarely with Theo...

[I do remember the beginnings of the openbsd project, where the members seemed to have no other goal than to annoy people as much as possible with crap like their idiotic "OLF" format.]

Re:Very well put...

[BrokenHalo]

While there will be those who see this as flamebait,...

Well, it certainly isn't that. The author minces no words when apportioning some of the blame to himself for causes of NetBSD's stagnation.

It's all a bit sad, really. I have a NetBSD server chugging along in a cabinet here that hasn't been rebooted in ~2 years, but that is largely because the updates I have noticed haven't really made it worth the trouble of upgrading.

Re:I hope NetBSD survives

[mistermark]

>I still have a soft spot for NetBSD

Me too, for me it also was the first UNIX I installed (on a SparcStation2) and got to learn the command-line in a proper way (imho) and I could only recommend anyone willing to learn UNIX to start with NetBSD.

At this moment I still have one machine running NetBSD, a 33MHz Mac LC475 (yes, I'm aware it's 2006). The server can be found here: http://mark.is-a-geek.org/ [is-a-geek.org]

>if necessary an active fork is made

Well, don't look any further than OpenBSD :D

Re:How many BSDs do we need?

[FST777]

No they don't, only heritage. But when something relevant is developed for one of the BSD's, the other will soon port it over, since it is easier to cross-port between the BSD's then from Linux to BSD.

The fact that each BSD has it's own kernel AND it's own userland is what makes it so great: each project has a different set of goals, and they are reached by focussing on that one, without having to think about what the other projects would like in the kernel.

Eventually, the projects can grow apart as far as you can think. But since basic functionality is often shared, that won't happen. And thus they abide :)

Re:Sounds bleak

[spauldo]

There's a lot of reasons to have it around.

The BSDs do stuff differently, and there's a lot of cross-pollination among them (and to a lesser extent linux). Someone might have an idea they implement in NetBSD that ends up getting ported to FreeBSD and OpenBSD, and vice-versa.

You also have the fact that the focus of the three major BSDs are different - FreeBSD is a general system, OpenBSD is focused on security, and NetBSD is focused on portability between different architectures.

This also gives more people the chance to contribute to the system in general. If you've got an idea for a new scheduler, you can try to get it implemented on one of the systems. If it works, other systems may copy it for themselves. If there's only one system, though, it's a lot harder to get into development because there's fifty other people with scheduler ideas you have to compete with.

Then, of course, the real reason why there's multiple BSDs around - developers want to work on them. Let them have their fun - just 'cause they make it doesn't mean you have to use it.

Solution?

[Zapman]

He hints around what I propose a lot[1], but he doesn't go there for some reason. His solution is to reform the system that has been broken for a long time, with what sounds like 'entrenched' problems.

My suggestion is to fork. You mention several good people and code. Open a new project (BSDPortable?) tempt the good people over there, and move on.

In my experience, the 'bad elements' very rarely remove themselves...

[1] Dragonfly BSD, Xorg, etc

Re:Sounds bleak

[SnowZero]

FreeBSD is a general system, OpenBSD is focused on security, and NetBSD is focused on portability between different architectures.

NetBSD is clearly the odd-man-out. FreeBSD's 7 arches probably cover 99% of the computers people would want to use. While NetBSD has sacrificed features and speed for portability, FreeBSD has managed most of the portability (from a practical standpoint) while adding new features. OpenBSD has a good niche, as security is a goal for while people are willing to sacrifice some features and speed. Portability alone is a strange goal however, since the only question that really matters is "does it run on all the computers I have".

I'm a Linux user myself, so I don't have much reason for favoritism for a particular flavor of BSD. I try to keep up with OS news in general however. I don't have anything against NetBSD, though I can't say I'd miss it that much (just like slackware as a Linux distro - historically important, but now largely irrelevant). If I had any current concern, it would be that I really hope DragonflyBSD succeeds, as it is pursuing some really interesting ideas in modern OS design.

Re:Sounds bleak

[brunes69]

(possibly a BSD distribution using the BSD kernel with GNU tools?),

Why would BSD the various BSDs switch to using the mostly inferior GNU tools??? The BSD userland is more standard and time-tested.

Sure a few GNU apps have some beels and whistles, like the GNU grep and NU awk, but these are mostly just fluff and could easily be added to the BSD userland if anyone actually cared much about the feature.

Re:Sounds bleak

[jimstapleton]

funny, the GNU tools are the only thing I find advantageous of Linux against BSD in my experience. I have my FreeBSD box running, but installed and use the GNU tools because: (1) more flexibility and options (2) command line arguments tend to be more flexible in their placing... I don't know how many times I've wanted to look at a file, and only after typing the command, considered *how* I wanted to look at it: $ ls somedir -l works great with gnu tools, the BSD tools have a hissyfit. As much as I like (love) the rest of the OS, I stick to GNU for the tools.

Re:one more brick in the wall

[NuclearDog]

"looks to me that the author of this email is just another jaded old coder that got his commit privs revoked. maybe something good will come out of this -- look at what Theo did."

Funny you should mention Theo.

This guy (Charles) was one of the members of the core group around the time they decided to give Theo the boot, and then was the main obstacle for 7 months while Theo tried to get some sort of CVS commit access back before he (Theo) finally just forked NetBSD and took his >10'000 lines of diffs and several drivers (that users were requesting) somewhere else.

ND

Re:Sounds bleak

[be-fan]

NetBSD also has a ridiculously clean codebase. That means if you're porting the OS to a new architecture, or doing some sort of OS research, the NetBSD code is a much better place to start than the FreeBSD or Linux code.

Re:Doesn't seem right

[codemachine]

> It's disingenuous to bitch about the things he does as if they were important - flash file system? So what? Journaled file system?

Flash file system is very important for embedded work. NetBSD could've been a much bigger player in the embedded space had they not fallen behind Linux, especially the uclibc/arm toolchain. Journaling file systems are highly desired by many people, including those in the project itself.

The fact is that NetBSD does run on some of the embedded systems that I'm working on, and might even run faster for that matter, with a cleaner codebase. But given that it doesn't support as many devices, and has no real feature advantages, Linux becomes the default choice. Just as in the desktop space, Linux is better supported, and you gain the advantages of the many other people working with the same toolchain as you (more stuff found on Google, etc).

On the desktop, it is much the same. A few years back, we ran an open source undergrad lab on NetBSD instead of Linux (mostly because we had a NetBSD developer on staff, and we figured why not). While it offered a very similar environment and similar selection of software (though Linux folks ended up installing gnu tools for color ls, etc), it really didn't offer any advantages over the Linux deskops. We found that certain features we were needing (pam, cups, etc) were always going to be ready "in a future release", whereas Linux worked right now. Support for proprietary software was also easier under Linux (though most was possible on NetBSD with Linux emulation).

In the end, even the NetBSD developer decided that we should be running Linux in our labs.

That said, the last few NetBSD releases have really packed in some good features and have shown to have great performance. Integration with Xen is also great move for the future.

From reading through the comments here, I think the biggest challenge NetBSD has going for it is perception. Everyone still thinks of it as the BSD with the main goal of being portable (while FreeBSD is "general" or for performance, OpenBSD for security). While it is true that their clean codebase is very portable, including the pkg-src tree, that is not all they are about. Their performance rivals and sometimes surpasses FreeBSD's. Their security record is quite good, likely due to a clean codebase. It is already a good general purpose OS, but most people don't think of using it as such.

Hard to say where the project should go. If they can get their feature set closer to on par with Linux, they'd be very competitive. But would anyone use it if Linux is "good enough"? Can they get past their stigma of just being "portable"? More competition in the form of an open source Solaris can't help their cause either.

Regardless, I imagine they'll just keep on coding anyhow, regardless of what happens. I don't see it as a dying project at all.

Its true, other projects are experiencing this...

[Halcy0n]

One such example, in my honest opinion, is Gentoo. I was a developer for about a year and a half before I finally called it quits. The major problem that I saw with Gentoo, and is the problem with NetBSD apparently, is that there is no main driving force to give the project direction. One of the great strengths of Gentoo is that there are many people working on things to scratch everyone's itch, but there is no general goal, and that is what leads to all of the flamewars. Everyone has their own idea of what Gentoo should be, and since there is no one to decide it, some people are content with arguing over it until the project dies from stagnation.

The best way to solve this, as I see it, is to adopt the idea of having a permanent "steering committee" for the project. Some major projects already do this, and it provides the central authority/leadership that is needed for any large scale project. Most developers/contributors don't want to deal with the politics that come from not having a central leadership, and there are the vocal few that will make it a living hell for everyone else.

I used to be a firm believer in letting projects govern themselves, but since I've been part of one that operates that way, I see the problems that come from that type of system, and they are crippling.

Re:Sounds bleak

[SillyNickName4me]

Personally, I think the best thing the Linux and BSD world has going for it is that Sun still thinks that it's all about the kernel. Nobody is ever going to switch to Solaris for dtrace, zones and zfs. 64bit filesystem vs. 128bit? Wake us up when it actually matters, if it ever does. I'm not saying kernels don't matter but the differences between various ones the features and lacking feature set is becoming small enough to not really care. The benchmark differences are so small and the stability differences aren't measurable.

Ah, someone else who believes the kernel doesn't matter..

I could run everything that is part of the typical Linux distribution on FreeBSD, and yet, I still would not have decent 5.1 audio and proper usb support for anything beyond a simple mass storage device or mouse.

I could run the FreeBSD userland on top of a Linux kernel however and still get working 5.1 audio and proper usb support

As a matter of fact, the differences between the userland on both are so small as to not matter for most practical purposes, and where there are differences in the userland, it is virtually always possible to port them. This is simply not true for the kernel.

I do agree with your opinion on Apple's changes however, getting rid of init is a very good idea, but then, unlike Linux and the *BSDs, Apple never tried to make a Unix like system, rather, they took the BSD derived system they already had and thought that it would be usefull as a component for building their next OS. This made it easy to get rid of traditional Unixisms like init

As a small sidenote, Apple uses a rather not unix like kernel..

What you are right about is that kernel features in themselves don't matter, what matters is the end-user functionality enabled by those kernel features.

Re:Sounds bleak

[unity]

Probably because a lot of us ran the *BSDs way back when.
I remember installing FreeBSD over ftp on a 14.4 modem back in '95.

This thread is seriously making me reminisce when /. had more quality posts and you were more likely to run into actually informative/insightful posts than everybody trying to make a one line joke. (mod Funny)

But I guess it can't be too bad, since I'm still here.