Forgot your password?
typodupeerror
Debian Security

Debian Project Servers Compromised 666

Posted by jamie
from the batten-down-hatches dept.
Sean was one of many to pass along the bad news from the debian-announce mailing list: "Some Debian Project machines have been compromised. This is a very unfortunate incident to report about. Some Debian servers were found to have been compromised in the last 24 hours. The archive is not affected by this compromise! In particular the following machines have been affected: 'master' (Bug Tracking System), 'murphy' (mailing lists), 'gluck' (web, cvs), 'klecker' (security, non-us, web search, www-master). Some of these services are currently not available as the machines undergo close inspection. Some services have been moved to other machines (www.debian.org for example). The security archive will be verified from trusted sources before it will become available again." They were going to announce 3.0r2 this morning; they've checked it and it's unaffected but obviously they're still postponing that release.
This discussion has been archived. No new comments can be posted.

Debian Project Servers Compromised

Comments Filter:
  • by Anonymous Coward on Friday November 21, 2003 @09:36AM (#7527505)
    The debian-announce archive [ http://lists.debian.org/debian-announce/debian-ann ounce-2003/threads.html ] doesn't list this message. Of course with the number of machines affected it's possible that the mailing list archive is somehow affected.

    -JohnF
  • by isoga (670113) on Friday November 21, 2003 @09:36AM (#7527507) Journal
    Obviously SCO are trying to break in and steal the source to prove once and for all that Linux has stolen their patents!

    ;)

    dave

    Tech stuff [homelinux.net]

  • That explains (Score:3, Informative)

    by jav1231 (539129) on Friday November 21, 2003 @09:37AM (#7527511)
    Why my apt-get was failing from people.debian.org last nite. Not to mention why debian.org was down. :(
  • apt (Score:4, Interesting)

    by isorox (205688) on Friday November 21, 2003 @09:37AM (#7527514) Homepage Journal
    Of course this raises the whole issue of apt-get. We all rely on apt-get update && apt-get upgrade, all it takes is someone to compromise the servers and insert a backdoor
    • Signatures? (Score:4, Interesting)

      by Sits (117492) on Friday November 21, 2003 @09:40AM (#7527546) Homepage Journal
      Are deb's signed? (I'm not that familiar with debian but I'd imagine they are) If so then just tell apt-get to not install debs that don't match a known signature...
    • Re:apt (Score:3, Informative)

      by tfheen (128718)
      Which is why using something similar to ajt's apt-check-sigs [66.102.11.104]. (google cache, since people.d.o is down.)
    • Re:apt (Score:4, Interesting)

      by Anonymous Coward on Friday November 21, 2003 @09:42AM (#7527566)
      apt-secure [debian.net] uses strong cryptographic methods to verify the authenticity of packages in the archive. It may be the default apt-get for sarge, depending on man-power issues.
    • Re:apt (Score:3, Informative)

      by psamuels (64397)

      Of course this raises the whole issue of apt-get.

      Indeed, that's one of the few areas where the Debian Project has lagged behind other distribution vendors technically - cryptographic signature verification for packages.

      This infrastructure has been kind of long in coming, but as of a few months ago, you can now verify Debian package signatures with debsig-verify [debian.org]. Might I suggest everyone install and use that?

  • by Chris_Jefferson (581445) on Friday November 21, 2003 @09:39AM (#7527528) Homepage
    This is the second time this has happened to a big open-source project (the first being the GNU servers a while ago). All packages by both groups are "md5" signed, which is supposed to protect against malicous hacking. However if the root server is comprimised, this doesn't help. Companies (including at least Microsoft, and the people who make ad-aware) who distribute files over the internet sign them with an RSA (or similar) key, and the computer which does this signing is kept disconnected from the internet. For such large projects which are installed by millions of people, might a similar system not be a good idea?
    • by stevey (64018) on Friday November 21, 2003 @09:40AM (#7527544) Homepage

      MD5 sums are used for the contents of packages, but packages may only be uploaded and processed by the build system if they're correctly signed.

      So yes it's not trivial to backdoor a package - unless you're already a Debian Developer...

      • Consider this. A debian developer's workstation at home is compromised, and the attacker installs a keylogger. What would stop the attacker from creating an approved package and then upload it into the repository?

        Now what's that they say about chains and the weakest link?
        • Let's see...
          1. It's possible that the developer would keep track of his commits and know he most certainly didn't submit that patch at 02:00 while he was out drinking.
          2. The sysadmin keeps noticing that silly log saying Developer X who only has rights to commit to the X11 stuff keeps trying to commit a kernel patch.
          3. The 70 year old neighbor who has nothing better to do than watch the neighborhood dials 911 when somebody starts poking around the developers house.
          4. "Attacker, meet Fluffy my faithful, full-grown m
    • The Packages files includes md5 sums of all the .debs, the Release file contains the md5 sum of all the Packages files, and the Release file itself is signed using GPG. Using apt-check-sigs [66.102.11.104] you can automate the checking of the packages you are installing.
    • by samjam (256347) on Friday November 21, 2003 @09:51AM (#7527630) Homepage Journal
      Don't be certain that digital signing is such a cure.

      The person operating the non-networked signing machine still needs to be sure that what-it-is-that-they-are-signing is what-it-is-supposed-to-be.

      Now how does digitial signing on a non-connected machine help you know the source wasn't tampered with?
    • Then the next point of failure becomes the keyservers. How do you know you imported a good key, and that the keyserver hadn't been compromised when you did it?

      This probably would be no good as a way to sneak backdoors onto more than a few machines, since keys are usually stored once and used often. But it would be good to have some sort of key distribution and verification system. Imagine a key publisher having 7 peers, and where they carry same keys, requiring 5 to 7 matching signatures, and point a nasty
      • by psamuels (64397) on Friday November 21, 2003 @10:26AM (#7527841) Homepage
        Then the next point of failure becomes the keyservers. How do you know you imported a good key, and that the keyserver hadn't been compromised when you did it?

        PGP keyservers (unlike, say, Kerberos KDCs) are completely untrusted. Anyone can upload any key to a keyserver. And downloading a key from a keyserver implies nothing about that key.

        To verify that you have a valid key, you have to rely on the web of trust. Basically, if a key is signed by someone whose key is signed by someone [recurse through however many levels you are comfortable with] whose key you have personally inspected, then the key can be assigned a trust metric based on how reliable you consider that chain of signatures to be. (Basically, how much you trust the integrity and acuity of the people controlling the chain of signatures.)

        PGP and GnuPG have supported this infrastructure from Day 1. Asking people to trust an arbitrary third-party public keyserver was never in the plans.

  • by cgranade (702534) <cgranadeNO@SPAMgmail.com> on Friday November 21, 2003 @09:39AM (#7527532) Homepage Journal
    How long will it take for the few MS fanboys around to say that this why Windows is better? Let me pull a Rumsfield (pre-emptive retaliation, that is...). Everyone gets comprimised once in a while. At least Debian is open about it, and not sitting on an insecure system because it's more profitable to let a bad product go then to risk bad press from releasing a security bulletin.
    • by stevey (64018) on Friday November 21, 2003 @09:42AM (#7527565) Homepage

      Password stealing is pretty OS independent.

      So this compromise, whilst undenyably bad, isn't really going to show much about Debian, or Windows.

    • What do you mean "few"? There are tons of them, even on Slashdot (heck, they're the majority).
    • by G4from128k (686170) on Friday November 21, 2003 @10:18AM (#7527785)
      I doubt that Microsoft (or any commercial software company) would publically annouce that it had been compromised. The source code processes at Microsoft are opaque -- nobody knows exactly who is putting what into the source code. If hackers, goverment officials, RIAA, etc. are modifying Window's source, nobody would be the wiser. In contrast, the openness of open source development creates an audit trail of who did what to the code (assuming the version tracking and submission system is not compromised).

      Transparency is a prerequisite for trust.
  • I don't think woody will be postponed that long. Martin's announcement says, While it has not been announced yet, it has been pushed to our mirrors already.
  • by Anonymous Coward
    Sorry, but I had to say it.... a Microsoft release has never been delayed because one of their servers were compromised.

    Let's just remember that before we extoll the virtues of how great open source is.
    • by jamie (78724) * <jamie@slashdot.org> on Friday November 21, 2003 @10:13AM (#7527762) Journal
      "a Microsoft release has never been delayed because one of their servers were compromised."

      I don't know if this delayed a release, but -- in October 2000, the news broke that Microsoft's internal network had been cracked for three months.

      (Debian made this announcement in 24 hours.)

      Read for yourself:

      Microsoft Cracked [slashdot.org]

      ...the Wall Street Journal article which apparently broke the news - it's the most complete. What's known - the passwords were being sent to St. Petersburg, Russia. They probably had access for about three months.

      "LONDON (CNNfn) - Hackers gained access to some of Microsoft Corp.'s essential product secrets, the world's most powerful technology company said Friday, acknowledging a security breach that is a major embarrassment for the software company..."

      "The Wall Street Journal said security employees had discovered that passwords used to transfer the source code behind Microsoft's software were being sent from the company's computer network in Redmond, Washington, to an e-mail account in St. Petersburg, Russia. Microsoft said it was making sure hackers could not use the stolen source code to change commercial software used by businesses, governments and consumers."

  • by thanjee (263266)
    Errrm, what OS was running on the servers compromised? :)

  • by KoolDude (614134) on Friday November 21, 2003 @09:40AM (#7527542)

    ...thousands of slashdotters flocked to Netcraft website to check whether debian.org was running on IIS.
  • by Alcoyotl (157542)
    Any other company would have sweeped that kind of incident under the rug hoping it had gone unnoticed, or would have cooked up a PR statement to minimize the incident.

    Here we can see the strength of such projects, as in this [slashdot.org] recent kernel story.
  • Makes you wonder (Score:5, Insightful)

    by bigberk (547360) <bigberk@users.pc9.org> on Friday November 21, 2003 @09:42AM (#7527567)
    It really is impressive for me how honest some organizations have been about admitting system compromises (Debian, ProFTP, GNU.org).

    As someone who works with networking security, I know lots of business servers get compromised regularly. Everyone hides it because it's embarassing for a business.

    This makes you wonder how often other 'critical systems' get compromised, and get fixed without any public reports. Government computer systems get regularly compromised after all. But I'm sure so do vital Microsoft, IBM, systems, etc. Windows Update, anyone?
    • Everyone hides it because it's embarassing for a business.

      From my perspective, hiding it is embarassing for business. A major part of the reason I use Debian is exactly this announcement. I could have guaranteed as a fact that the Debian servers would be compromised, it was just a matter of time. What's important to me is that it's easy to detect when it happens, and that everyone is told about it as soon as it happens.

      I have one of my machines which I updated during the compromised period. Now I kno
  • Signed announcement (Score:2, Informative)

    by Anonymous Coward
    here [uni-stuttgart.de].

    To verify it:

    $ wget -O- http://cert.uni-stuttgart.de/files/fw/debian-secur ity-20031121.txt | gpg --verify

    (drop the space, of course)

    Assuming you trust the key it was signed with, of course...
  • by caluml (551744) <slashdot.spamgoeshere@calum@org> on Friday November 21, 2003 @09:51AM (#7527637) Homepage
    .debs should be gpg signed, and should fail to install if the verification fails. In fact, so should all packages from distros. Redhat, +1, Already doing it. -1, not failing to install if the packages don't verify.
    • by jemfinch (94833) on Friday November 21, 2003 @01:12PM (#7529442) Homepage
      Redhat, +1, Already doing it. -1, not failing to install if the packages don't verify.

      Which is exactly the state in Debian, too.

      Jeremy
  • by buddha42 (539539) on Friday November 21, 2003 @09:52AM (#7527648)
    On the one hand stuff like this scare's the hell out of me, but on the other hand I'm very reasurred by how the debian community handles it. Full disclosure, detailed explanations, and very conservative thinking (exibited by the "3.0r2 is fine, but we're not releasing it anyway just to be anally sure").

    At this point I would like to see the debian team develop some written policies and procedures for how they intend to prevent this sort of thing in the future. I checked the site and while there's security info for how to secure your box, there's no policies on 'how does the debian project secure itself'.

    Lastly, one concept you have to keep in mind, we have no idea how often other OS's key servers are cracked because they'd never tell us.

  • by Cthefuture (665326) on Friday November 21, 2003 @09:54AM (#7527660)
    As Linux becomes more popular this is only natural.

    Open-source projects are not immune to attack and they are going to start feeling some of the pain experienced by other big targets like Microsoft. In the beginning it could be really bad because unless you're being attacked seriously all the time then you may not even realize where your vulnerabilities are.

    This is a wake-up call to all "open" projects. Systems that are in use by a large number of people need to be protected better. Sure, this may have been a password compromise but the system should have been secure enough that some low-level user account compromise can't cause serious damage. And the high level accounts should never, ever have a password compromise. This needs to be treated in the same way big business does. Protect the customers, otherwise you may lose them.

    This made me start thinking... Has Redhat ever been compromised? That'd be a reason for going with a commercial distro if the free distros can't get their act together. (I've been a Debian user for many years by the way)
    • by FooBarWidget (556006) on Friday November 21, 2003 @10:05AM (#7527719)
      You're talking as if the Linux community is full of zealots who can't be objective. That's completely wrong.

      People *already* know that OSS is not perfect, and they have known for years. People already know OSS is not immune.
      But, more importantly, those same people know *nothing* is immune. Not MS, not Linux, not BSD, not (even!) MacOS, not DOS. *All* systems can be hacked.

      What *really* matters is the attitude to security.
      - A lot of the larger OSS projects care deeply about security. If a security bug is found, it's usually fixed very fast, and the fix will be peer reviewed.
      - They openly admit all flaws and bugs. Because of this, OSS *appears* to have more bugs.
      Do you see Microsoft admit all their bugs? I don't think so. MS hides a lot of bugs, pretending that they don't exist and that Windows is perfect.

      Too bad all the MS zealots and anti-OSS/anti-Linux zealots use that to "proof" Windows is more secure than Linux/OSS/whatever. The number of bugs is *not* an accurate indication of security.

      Linux zealots are only a small minority of the community. If you think they represent the entire community then you're wrong, just like so many people out there.

      "Has Redhat ever been compromised?"

      Maybe. If they haven't then it's because of pure luck.
      • You talk with *astericks* a *lot* and try to drive the point that people knows OSS is imperfect...do you even visit Slashdot?

        This place is nothing but a haven for anti-Microsoft bias. It's not pro-Linux.

        Linux zealots are only a small minority of the community.

        Yet they are the most vocal.

        If you think they represent the entire community then you're wrong, just like so many people out there.

        Let's face it, Linux and its community of developers will never be accepted professionally beause of their unpr
  • Like Mossberg says, Mac's can't be hacked! [wsj.com]

  • OH NO!!!! (Score:5, Funny)

    by HungWeiLo (250320) on Friday November 21, 2003 @09:58AM (#7527683)
    Was any code stolen? OH wait...
  • by finkployd (12902) on Friday November 21, 2003 @10:04AM (#7527712) Homepage
    First GNU, then Bitkeeper, now this, whatever shall we do?

    Simple, the technology has existed for decades now.

    A little something I like to call "Public Key Cryptography"

    With this "Public Key Cryptography" you could conceivably sign software in such a way that it could not be altered without breaking the signature, AND ensure that nobody else could forge this digital signature (you are keeping your private key private right?)

    MD5 Hashes are a step in the right direction, but by themselves are meaningless. Sort of like improving your home's security by drilling holes in your door to mount a deadbolt but not actually taking the final step and INSTALLING THE DEADBOLT.

    So let's take these MD5 hashes and encrypt them with the package maintainer's private key (or distribution maintainer, whatever). Then dpkg (or rpm, emerge, whatever your favorite package tool is) could be written to decrypt this hash with the corresponding public key. Wait, there is more! Then it could generate it's own MD5 hash of the package in question and COMPARE it to the decrypted hash it just created. If they match, the package is unaltered AND came from a trusted source. This my friends is what we like to call a "digital signature"

    I don't care how you do it, GPG, x.509, whatever. I'm actually leaning toward x.509 since it seems to me to make more sense to have the distro maintainer run his/her own CA and issue certs to package maintainers. This CA could then be included in whatever package tool is used and viola. No mucking about with the web 'o trust (Which rocks for ad hoc trust relationships like between people emailing each other, but sucks for this kind of hierarchal stuff)

    So what do you think everyone? Good idea or should we wait for a few more server compromises before we think about securing software repositories?

    Finkployd
    • With this "Public Key Cryptography" you could conceivably sign software in such a way that it could not be altered without breaking the signature,

      No... the way to alter software is easy to conceive.

      You simply have to hack into the computer holding the private keys used for the signing (very likely the same computer holding the source code as well, and the system which normally uploads new packages to the distribution point). Once there, you can make changes and sign them just as if they were official.
      • No... the way to alter software is easy to conceive.

        You simply have to hack into the computer holding the private keys used for the signing (very likely the same computer holding the source code as well, and the system which normally uploads new packages to the distribution point). Once there, you can make changes and sign them just as if they were official.


        Assuming you knew the password for the private key (private keys really should be encrypted with a password, especially for this).

        Now before you go
  • by S. Baldrick (565691) on Friday November 21, 2003 @10:05AM (#7527717)
    In response to the dastardly assault against the twin (mini-)towers, the President of Debian drew a line in the sand and immediately announced the invasion of Slackware.
  • by O.M.A.C. (181899) on Friday November 21, 2003 @10:22AM (#7527807)
    I ran apt-get and my machine was converted to Windows 2003!
  • Tempered Arrogance (Score:5, Insightful)

    by ChaoticCoyote (195677) on Friday November 21, 2003 @10:52AM (#7528070) Homepage

    All three of my Linux boxes run Debian; this latest security breach will not change that.

    However, I hope this type of incident tempers the often-strident elitism of the free software camp. My faith in Debian continues because they caught this problem and openly announced it; my concern is that the lack of consequences will make people assume that this was a false alarm or unimportant incident.

    Free software suffers from "victory disease" -- an assumption that, based on past success, future success is guaranteed. Because free software has proven reliable and secure, the concensus seems to be that it will always be so.

    Pride comes before the fall, as they say. Attempted infiltrations of the Linux source code control system and breaches of security at Debian suggest that we need to be cautiously optimistic, not naively myopic.

  • by jdifool (678774) on Friday November 21, 2003 @10:58AM (#7528111) Homepage Journal
    Hi,

    218 posts and some rare appropriate reactions.

    • I thought Linux was secure... Guess not. Who told you that Linux was secure ? Your grandma ? Linux is more secure than Windows, of course. But it's not immunized against cracker. The computer world is based on a set of rules that can be broken. The better you are mastering these rules, the more secure your boxes are. But these rules can be broken, which means that, given human nature, they are bound to be broken occasionnaly. Furthermore, you will have noticed that if often relies on human use mistakes (password cracking for instance).
    • Free software sucks, Microsoft rules. Here I can almost physically feel the frustration of advocates of the proprietary world that can do nothing but bash any free software flaw they might encounter. However they deserve a clear, sound, and honest answer. My dears fellows, the free software world never proclamed himself the embodiment of security. We do our best to ensure it. And don't mix things up : our main problem with Redmond handling of security is about post-treatment. We do not appreciate the culture of hiding ; you can see here how coherent we are with ourselves.
    • Gentto is better than Debian ; oh no it's Redhat ; oh no it's Slackware. Hey guys, are you really part of the free software world ? Can you just realize these are the precise sentences that led to proprietary software/world ? And don't you think that you should adopt a more conservative stance ? Don't you think that the moral of this sad story is that nobody is preserved from crackers ? Wake up men, this is the very crucial moment where we must stand united. Keep your ammo for you real foes.
    There are some days when you would think that the free software world is not that 'free as in freedom'...

    Regards,
    JDif

  • by Bob9113 (14996) on Friday November 21, 2003 @11:01AM (#7528139) Homepage
    This news made me realize how much I depend on Debian. At the moment, every one of my machines (four servers, three workstations, and a laptop) runs Debian. I've been running it as my primary OS for... two years? So far I haven't paid a dime for it. It is a nice advantage of Free Software to be able to use it for free, but given the fact that I'm way out of "try-before-you-buy" mode, I'm going to send them a check today. Software in the Public Interest [spi-inc.org] was founded by and is the current funding source for Debian.

    One server compromise in the two years that I've been watching by a company with zero product sales revenue is pretty impressive. An OS that is (IMO) dramatically superior to any commercial offering for free? They've earned my respect, and have clearly earned my cash.
  • by Omega037 (712939) on Friday November 21, 2003 @12:13PM (#7528812) Homepage
    This is much worse than one of Microsoft's normal problems. With Microsoft you expect the problems, and therefore you maintain constant vigilance. This is a perfect example of why linux users and admins need to also be wary at all times. As linux becomes more and more mainstream, the number of security holes shown will increase as well. More people will use linux and more "hackers" will then be attracted to developing viruses and worms that exploit the system. Regardless of what anyone thinks about Windows vs. Linux, everyone must admit that part of the reason more security holes are found in Windows is because there are many more people looking for them. My advice to linux users is to drop any pretense of Linux being infallible and to start using the same caution running a linux-based server as you would running a windows-based server.

    • As linux becomes more and more mainstream, the number of security holes shown will increase as well. More people will use linux and more "hackers" will then be attracted to developing viruses and worms that exploit the system. Regardless of what anyone thinks about Windows vs. Linux, everyone must admit that part of the reason more security holes are found in Windows is because there are many more people looking for them.

      This belief that Linux is some kind of new kid on the block and untested complet

  • GPG already! (Score:3, Interesting)

    by alexandre (53) on Friday November 21, 2003 @12:23PM (#7528910) Homepage Journal
    When are they going to force everyone to sign the package with GPG and have a warning like ssh when a key has changed when you dist-upgrade?

    It's about time will all the server compromised these days...
  • by mediaisthemassage (717633) on Friday November 21, 2003 @03:11PM (#7530725)
    I just based my home cluster on debian because is so sexy...save the soul of your sun boxen and load linux....is fun....

    But security holes exist, there is no getting around this, no matter how paranoid you are...

    trust me..

    I am a sitting in a faraday cage right now...I built it in my apartment to keep those pesky NSA spooks from uplinking with the nano-chips they implanted in my brain....

    most of us are now implanted...you can't dig them out...i've tried....

  • SE Linux (Score:4, Interesting)

    by Tracy Reed (3563) <treed@ultrav i o l e t . org> on Friday November 21, 2003 @04:38PM (#7531497) Homepage
    Steve from Debian Security Audit project says this occurred due to a password goofup so this doesn't necessarily apply here but it easily could have:

    Machine as important as these should be running some sort of Mandatory Access Control system like SE Linux [nsa.gov]. I have done an evaluation of all of the root exploits I could find over the last few years and SE Linux would have prevented every one of them because the MAC system prevents unauthorized priviledge escalations. You can test drive my SE Linux box by telnetting (not ssh) to selinux.copilotconsulting.com with user root and password root.
  • Am I the only one? (Score:3, Interesting)

    by stonecypher (118140) <stonecypher@gm[ ].com ['ail' in gap]> on Saturday November 22, 2003 @04:20AM (#7535301) Homepage Journal
    You know, an enterprising attacker could just pull the trust network down. Someone with sufficient skill could very easily just work on Debian for five or six months, get trusted, and embed a subtle bug into a remote point.

    I mean, we can't find the unintentional ones. What makes you think we could find one chosen for its obscurity?

Nothing succeeds like success. -- Alexandre Dumas

Working...