Debian Locks Out Developers

daria42 wrote in with an update to an earlier story about a Debian server that was compromised. He explains: "The Debian GNU/Linux project has discovered a compromised developer account was used to gain access to a server compromised this week. A local kernel vulnerability was then used to gain root access. Due to this, a number of developers with weak passwords have been locked out of their system accounts." To be fair, they'll most likely be let in once everything's back to normal. Of course, they'll probably need to set safer passwords too.

Ah. balance

[Kid Zero]

That wonderful feeling of making the password hard to guess, but easy to recall.

back to normal

[Coneasfast]

To be fair, they'll most likely be let in once everything's back to normal. Of course, they'll probably need to set safer passwords too.

Um. What exactly does this mean? If everything isn't back to normal, shouldn't they lock out ALL developers? That's what I would do.

password requirements

[PetriBORG]

Hopefully then they will also implement a good set of password rules and enforce them to protect themselves from future problems. Where I work they require 3 out of the 4 rules to be met such as mixed case, numbers and special characters... of course they also make us change our password every 30 days so i've discovered that people have taken to doing things like Asdf1234 and then when the password requires changing changing it to Asdf2345... Doh.

I wonder...

[MSFanBoi2]

Why when this happens on a Windows server is "OMG! Windows is insecure! M$ is evil!!!!"

But with this its "Oh just set more difficult passwords"...

ssh2 keys?

[saleenS281]

Why don't they just have the developers use ssh2 keys? I didn't know anyone actually used passwords on secure systems for authentication...

Re:ssh2 keys?

[Gregg Alan]

Why don't they just have the developers use ssh2 keys? I didn't know anyone actually used passwords on secure systems for authentication...

Agreed. Don't we all know this already? I don't have anywhere near the number of users that Debian does, but it's keys all the way.

On the other hand, if they used a weak password for something as serious as Debian developer access who's to say that their workstation/personal network/whatever wouldn't be compromised and leak the encrypted key and then the key (assuming they have a weak passphrase).

I smell a sleeper.

Re:I wonder...

[The Bungi]

Did you miss the part where a local Linux kernel exploit was used in the attack?

good idea

[smash]

I mean, it's one thing to use an insecure password on your personal machine - but using an insecure password on a common development machine for a fairly high profile project is just irresponsible.

Locking them out is totally fair, and imho it's the responsible thing to do.

STRONG passwords should be enforced (hell, mandatory keyed logins would be better) on machines like this (which are fairly attractive targets for abuse)...

Re:I wonder...

[Anonymous Coward]

There is a huge difference between a local exploit and a remote exploit. Most Windows problems involve remote exploits. This is why Windows is considered so insecure. If you have a personal (single user) machine and don't run servers that allow remote logins then local exploits are of little/no concern to you. Not so for remote exploits. Understand now?

Re:Passwords

[sholden]

There's no way you could be dumb enough to actually think that.

Re:So?

[TCM]

How the hell could this be modded insightful? The whole point of changing passwords is so that the compromise of one password doesn't lead to unlimited access or the compromise of future passwords.

If a password is so secure that it can't be guessed, then why change it? If it's so weak that it gets guessed monthly, changing just one digit doesn't do shit.

And if the system gets compromised, you reinstall and choose a totally different password.

Seriously, this must be the most stupid advice I've seen and it's currently +2, Insightful. Scary.

Re:Ah. balance

[JebusIsLord]

Yeah, instead you have to worry about the locations of their memory keys!

Re:I wonder...

[Anonymous Coward]

Did you fail to understand what a remote exploit is?

Here, let's try an analogy. In this case someone left the door to the building unlocked. A burglar got in. He then methodically cracked the safe, and took the money from within.
  Following this, "MSFanBoi" posts to slashdot making a false equivalency between that and the Win building where the locks were defective and the money was taken from where it was sitting on the floor. (The windows exploits being criticised are remote, the linux exploit was local-only. In the latter, you have to actually break in before they are useful.)
  Do you still fail to see the difference?

Keys need protection as well

[SIGBUS]

However, said keys better be passphrase- (NOT password-) protected! After all, if, let's say, $DEVELOPER's laptop gets stolen and it has a non-passphrase-protected ssh key, then going to the effort of using keys for authentication will be for naught.

FWIW, I recently ditched Debian for a completely unrelated reason (see also, CVE-2006-1173).

Re:Ah. balance

[arivanov]

Ahem.

That was my first reaction as well. If you are using ssh, passwords are "the wrong idea" (TM). A password is guessable while a private key is not. That is the way it is designed.

Not that it would have helped in this case though. The attacker compromised the developer machine first and from there on compromised the server. Once you are in full control of a user machine it is game over as far as any resources it accesses are concerned.

While at it, I can also understand an admin on a large linux shell machine which still allows passwords.

Due to Theo and Co's knee jerk (hello Theo) attitude to non-OpenBSD OSes there is no way to make ssh public key authentication plug cleanly in the OS authentication chain. The reason is that ssh continues to be developed from the perspective "PAM sucks, we here do not do that". That is a purely political OpenBSD position which belongs to the realm of OpenBSD and should have nothing to do with OpenSSH. If OpenSSH is to really support linux without "OpenBSD is better TM)" being involved it will have to support PAM properly, natively, all the way and for all authentication methods including public keys. This is the way of the land.

As this is not the case yet, if you want to limit access to public key only you have to turn off PAM for ssh. Alternatively, if you manage a big machine (or network) and you need PAM you end up turning on passwords and praying.

Re:Weak Passwords ?? If they know that ...well

[Anonymous Coward]

Believe me, the Debian project does not store passwords in the clear.

As administrators they have access to the shadow file that contains hashed versions of all the passwords. What they did was run a cracking utility against the shadow file to pick out weak passwords. Usually these cracking utilities use brute force dictionary attacks to try and randomly guess the password. If the utility was able to guess a password quickly, that password was definitely not secure. It's as simple as that.

I encourage you to read more about the topic, it's a fascinating one. Wikipedia has an interesting article at http://en.wikipedia.org/wiki/Password_cracking/ [wikipedia.org].

Re:Ah. balance

[KiloByte]

Talking about openssh's security, here's a vital patch:
-PermitRootLogin yes
+PermitRootLogin no

Come on, how in the hell the first can be the default?
Is it that hard to log in first to your normal user account?

And don't start on "but having no root and using sudo is better". This is a fallacy, true only if your normal user password and the root one happen to be the same, or you have a root login available from network. Two separate passwords are always better than just one, and forcing people to type "sudo" before every line makes little sense -- quite a lot of servers have nothing to do on them for non-root (firewall, dom0, etc).

Re:Weak Passwords ?? If they know that ...well

[Acc7]

Reply posters,

Interesting comments (except that one anon creature).. Yes, when one has access to the hashed password files, the test is a lot easier than a wholesale crack.

And the net is not exactly a place to send anything that one doesn't want snniffed, is it.

But by leaving us to guess why & how, Debian did leave the door open to speculation on just what they did that opened this vulnerability and what they did to "determine" there were weak passwords. And I was not knocking the Debian code, just the management errors that led to this particular problem.

And the question about the kernekl version is also a valid curiosity, isn't it. btw do they actually know that this was a hack from outside, entirely outside?

As to credibility, would rather see a good open discussion than waste time with name calling any day.

Re:Ah. balance

[arivanov]

With passwords you are absolutely correct. Direct root login is stupid, wrong and insecure and people leaving it should be shot.

With public key authentication for remote access + plain passwords on the machine itself you are not correct. There is no significant difference between giving direct ~/.ssh/authorized_keys based root to all the people entitled to root and giving them access with per user ~./ssh/authorized_keys and su/sudo. The simple password is inherently less secure authentication method compared to PKI and just bolting a password on the back does not do a thing. If you could tightly couple passwords with ssh PKI that will be a different matter. Same if you use one-time passwords after the login. This unfortunately requires tightly coupling all ssh authentication methods with the system ones as well as coupling ssh authentication with system authorisation.

As far as linux is concerned this requires Theo to stop his knee jerk reactions to PAM and integrate ssh and PAM properly.

By the way I agree with him that PAM sucks, but as far as having cross-platform and cross-distribution security framework we got nothing better. And no, I do not want to hear about keynote or COPS. Nobody in his sane mind wants to.

Re:Ah. balance

[Bogtha]

Due to Theo and Co's knee jerk (hello Theo) attitude to non-OpenBSD OSes there is no way to make ssh public key authentication plug cleanly in the OS authentication chain. The reason is that ssh continues to be developed from the perspective "PAM sucks, we here do not do that". That is a purely political OpenBSD position which belongs to the realm of OpenBSD and should have nothing to do with OpenSSH. If OpenSSH is to really support linux without "OpenBSD is better TM)" being involved it will have to support PAM properly, natively, all the way and for all authentication methods including public keys. This is the way of the land.

If this is so important, how come nobody has forked OpenSSH? Debian could do it. Any of the distributions could do it. You are essentially complaining that OpenBSD developers are developing OpenSSH for OpenBSD instead of the way you want them to do it. They aren't under any obligation to make dodgy (in their opinion) security decisions to please you - that would be making politicial decisions.

Re:Ah. balance

[Mark Hood]

Not in this case - if you read the summary, even.

They got in with a developer's password, then used a local exploit to get root permissions.

I.e. they didn't know (or need to know) the root password. So in this case, having two 8 character passwords was exactly as secure as having one 8 character password.

Mark

Re:I wonder...

[TrappedByMyself]

Why when this happens on a Windows server is "OMG! Windows is insecure! M$ is evil!!!!"

Don't forget that people would be making fun of the incompetent MCSE admin for not enforcing a complex password policy

Re:libpam-cracklib

[teslar]

You're laughing, but in practice I have too often seen restrictions on the maximal security of passwords.
Take for instance my online banking system (which in its defense has other security measures alongside the password, but still):

• no more than 10 characters please
  
• upper/or lowercase does not matter, lowercase will automatically be converted to uppercase
  
• alphanumeric characters only

Seriously, what's the point of this?? Why am I forced to use weak passwords just because some developper somewhere can't figure out how to allow a " or a \ in a string?

Overreacting a bit?

[rai4shu2]

"Due to the short window between exploiting the kernel and Debian admins noticing, the attacker hadn't time/inclination to cause much damage," wrote Schulze.

"The only obviously compromised binary was /bin/ping. The compromised account did not have access to any of the restricted Debian hosts. Hence, neither the regular nor the security archive had a chance to be compromised."

It seems like nothing much really happened. I mean, if this is all a hacker is capable of even with root access to a major Debian server, then what's all the fuss about?

howto: strong passwords

[dune73]

If you are in need of a strong password, use the following recipe:

Think of a sentence with 6-10 words with a number in it.
- The number can be inside one of the words.
- If you manage to have multiple Capital words in the sentence, your password gets stronger.

Then take the first letter and write the numbers as digit, include the point,
question mark, exclamation point at the end and you got a strong password.

Today i ate two buns for breakfast! -> Tia2bfb!
I have seen six dups on Slashdot this week. -> Ihs6doStw.
Can you memorize all four new passwords? -> Cyma4np?
And today: A new password for my debian account! -> At:1npfmda!

Works fine for me and is fairly easy to memorize.

Re:Your new Debian password.

[jackbird]

Since they are rightfully in posession of the passwords file and usernames on the system, I imagine they ran a dictionary attack against all the user accounts.

Again?

[wobblie]

As a long time Debian user I am not so surprised by this - it is just the reality when you operate a system with thousands of user & shell accounts all over the world. It isn't that big of a deal if the debian admins respond correctly, which they always do, but it looks bad.

The issue that gets me is this is the second time the Debian system has been compromised, and in the exact same way - a local kernel exploit from a compromised DD account. As good as the Debian security team is, they are frankly terrible with the kernel. The Linux kernel has continual local security exploits (I am not in denial about that); these don't matter so much for most deployments but for the Debian system they are absolutely critical because of all the shell accounts. The Debian kernel team really needs to work out something better (though I know the issue is more complicated than that); this is the one thing Red Hat does very well. I cannot for the life of me understand why debian servers kernels are not upgraded continually. The downtime is immaterial compared to something like this.

Re:Again?

[noahm]

As good as the Debian security team is, they are frankly terrible with the kernel.

The servers compromised, in both cases, were not running kernels supported by the debian security team. The Debian sysadmin people typically roll their own kernels for use on Debian.org machines. In fact, had the default stable kernel been used, the machine would not have been vulnerable to this exploit. See http://lists.debian.org/debian-news/debian-news-20 06/msg00030.html [debian.org] for more info. (In short, 2.6.8 is not vulnerable.) The compomised server was running a custom 2.6.16.18 kernel, and was thus vulnerable. It has since been upgraded to a custom 2.6.17.4 kernel. If you're going to knock anybody in this case, knock the Debian sysadmins for not keeping up with security on their own machines, or for having policies in place that allow weak passwords to lead to any kind of access. But don't knock the Debian security team.

The Debian security team has fixed 98 kernel security bugs since sarge was first released. Refer to changelogs for the kernel-source-2.6.8 versions 4sarge1, 4sarge2, and 4sarge3 for details if you need them. What more do you want?

noah

Passwords? Are you nuts ?

[m.dillon]

There shouldn't be any passworded accounts on a developer machine at all. It should be SSH access via public key only, period end of story. I stopped using remotely accepted passwords over a decade ago. Passwords are only accepted on the console.

Come on, folks. This is UNIX-101. Don't be stupid.

-Matt