The Costs of Patching 311
prestidigital writes "vnunet has a brief but interesting article in which Craig Fiebig, general manager of Microsoft's security business unit, is quoted as saying "In dollar terms, patching is the most expensive security measures and keeping your antivirus descriptions up to date is the least." That seems like an important statement coming from a company who's patches are possibly responsible for 45% of traffic on some networks."
Wow...it took them this long... (Score:5, Insightful)
And that is costs more to have a new programmer look at and try to modify code that wasn't written by himself/herself...
Amazing reality breakthrough!
Re:Wow...it took them this long... (Score:3, Funny)
Re:Wow...it took them this long... (Score:5, Insightful)
So you're damned if you do and you're damned if you don't.
Hey, I know, maybe Microsoft could do this new thing called PROPER BETA TESTING, and then maybe the could get it right THE FIRST TIME!
Nah, that'd be too easy.
Re:Wow...it took them this long... (Score:3, Interesting)
What's testing got to do with it? (Score:3, Informative)
The problem is not the testing or even the coders. The problem is often the application designers/architects who often are thinking "features" when they should be thinking "security."
I suspect that $1 of design is worth $10 of coding, $100 of testing, and $1000 of patching for Microsoft, let alone the poor customers.
Re:Wow...it took them this long... (Score:4, Funny)
Hey, I know, maybe Microsoft could do these new things called DESIGN REVIEW and CODE REVIEW, rather than trying to test out bugs.
You must have missed it. After Bill declared a new focus on security, they did a code review -- one month of review for twenty years of code. The next code review is scheduled for 2022. :)
Re:Wow...it took them this long... (Score:2)
Rus
Also known as... (Score:4, Insightful)
Re:Also known as... (Score:2)
Re:Also known as... (Score:2, Funny)
Patching has saved my hundreds of dollars (Score:5, Funny)
Re:Patching has saved my hundreds of dollars (Score:2)
Re:Patching has saved my hundreds of dollars (Score:4, Funny)
Re:Patching has saved my hundreds of dollars (Score:3, Funny)
Patches (Score:2, Funny)
Cost of not patching? (Score:5, Insightful)
However, with security patches usually you have no choice. The only decision for some security patches is how long do you wait before deploying it. Don't wanna be the first ones to put a bad patch on now, do we?
Re:Cost of not patching? (Score:5, Informative)
Fist a high level person would look at the patch (usually using install shield's application repackager), read the documentation, etc. and look for possible conflicts with the production environment. This took between 2-4 hours per patch x $60/h. The regression test took one lower-level tech about 2 days to do. We'd lump a few patches together so say 1 tech x $40/h (at least, w/ benefits, etc.) x 2 days / 3 patches per test = about $213/patch + eval ($180 per patch) = around $400 per patch to test. Deployment took another hour to write the install script (rarely did we rely on MS's installer alone), 1 hour to document and send to the regional offices and each office probably spent an hour implementing the thing. Total cost around $600 per patch for a 1,000 desktop, 11 office environment.
Now you know.
Re:Cost of not patching? (Score:5, Funny)
Re:Cost of not patching? (Score:2)
For the usual "feature" patches ("This patch adds pretty shiny things to the edge of your window"...), you're absolutely right: making any kind of large-scale change (like putting a new patch on 1000 machines) is a big deal. Even if it's all automated via network management tools, you'll need to test, prepare and then support it. Do you really
Re:Cost of not patching? (Score:2, Interesting)
The only decision for some security patches is how long do you wait before deploying it.
That's not quite the only choice--you have two other choices: adopt Linux; adopt Macs. If the cost of patching is really that great, it raises the cost of the machine--until maybe purchasing a Mac isn't all that expensive after all.
Re:Cost of not patching? (Score:2)
From a patching perspective. Why would this cost less? Macs and Linux still require patching, because ALL software has bugs.
Macs especially: Buying a whole new computer is more expensive then using your existing hardware. I don't see where you get the "Mac is cheaper" argument.
Re:Cost of not patching? (Score:3, Redundant)
Considering the ease of use and effectiveness of the latest Samba exploit, anything thinking Linux machines are somehow magically more secure and cheaper than Windows machines is kidding themselves. It's all about how you configure/maintain them.
Re:Cost of not patching? (Score:2)
Re:Cost of not patching? (Score:2)
JACK (V.O.)
I'm a recall coordinator. My job was to apply the formula. It's
simple arithmetic.
TECHNICIAN #1
Here's where the baby went through the window. Three points.
JACK (V.O.)
It's a story problem. A new car built by my company leaves Boston
traveling at 60 miles per hour. The rear differential locks up.
TECHNICIAN #2
The teenager's braces locked around the backseat ashtray. Kind makes a
good "anti-smoking" ad.
JACK (V.O.)
The car crashes and burns with everyone trapp
Re:Cost of not patching? (Score:2, Insightful)
I'm a Solaris admin and have no love for Microsoft. But even I have to admit that all operating systems need patches. Solaris does, Microsoft does, every version of Linux does. Just changing to another OS won't solve all your patching problems. I'll grant you that Microsoft seems to be worse than average in terms of number of patches needed and the hassle involved so changing may definately be a good idea. It just isn't a complete fix.
Re:Cost of not patching? (Score:4, Insightful)
It doesn't have to be all that bad. Packages are relocatable, so unusually sensitive applications can be put into their own root directory hierarchy. Using NFS wisely can allow for one set of applications on a network (patching once and only once is quite nice). Only one or two servers on the whole network should be running Sendmail and BIND in a vulnerable mode. UNIX is also easier to pare down, so there are much fewer things that need to be patched. With a good network design, patches can be rolled out automatically over SCP, and UNIX machines tend to reboot pretty reliably, unless a patch screws up an init script.
It is just a simple fact that UNIX is less complex than Windows. It has fewer lines of source code, more transparent modularization, strict separation between the GUI and the kernel, widely available and thorough documentation, three decades of experience behind it, almost complete scriptability, among other things. Windows, on the other hand, is as opaque as mud--there could be a golden city under there or just more mud, but we'll never know.
Re:Cost of not patching? (Score:3, Insightful)
It's probably more of a nervous chuckle than a laugh. After all, Win2003 developers are paying attention [zdnet.co.uk]. To the point:
interesting debate (Score:5, Funny)
(Google [216.239.51.100] cache version in html.)
Re:interesting debate (Score:5, Funny)
Re:interesting debate (Score:3, Funny)
The number of patches must be the worst possible metric for measuring bugs. A better measure is: (several bugs per 1000 lines of code) X (40,000 thousand lines of code in Windows) = over 100,000 bugs in Windows. Thus, it follows that (100,000 bugs/installation) X (100,000,000 installations) = 10,000,000,000,000 Windows bugs worldwide.....OMG, the plauge of the apocolypse is upon us!
Re:interesting debate (Score:2)
The number I remember hearing is 25 Million lines of code for XP. That's a touch more than 40,000
Patching most expensive (Score:3, Interesting)
IMHO getting hacked is much more expensive.
NEW MATH (Score:5, Insightful)
But spam is responsible for, what was it Taco, 60% of traffic on networks?
I'm at 105% utilization already!
BTW, it's just as costly, if not more, to have to rebuild your linux kernel, SSL, apache webserver, or samba installation when a bug is found there.
Quit pretending that MS has some sort of monopoly on software bugs. "Bad code" is a patentless technique used ubiquitously.
Re:NEW MATH (Score:5, Insightful)
Let's face it. There's no real way to know for sure what is on those wires unless you monitor them. And I don't think anybody here wants to open that can of worms.
Re:NEW MATH (Score:2)
The difference of patching on Linux as I see it is that, kernel patches are rare, and are just about the only update that requires a reboot. All other services can be upgraded without affecting the rest of the system.
Windows seems to give these black box security updates, all of which prompt for a reboot, whether it is technically neccessary or not, I don't know.
Re:NEW MATH (Score:5, Funny)
But spam is responsible for, what was it Taco, 60% of traffic on networks?
I'm at 105% utilization already!
Didn't you see that the article was about Microsoft? I'm sure there is at least SOME overlap in the spam/patch metrics.
Well, as you surely know... (Score:2)
Re:Well, as you surely know... (Score:2)
I think you need to back that up with some facts!
(joke, for those lacking common sense and social skills)
Re:NEW MATH (Score:2)
But spam is responsible for, what was it Taco, 60% of traffic on networks?
I'm at 105% utilization already!
We conclude that at least 5% of network traffic is Windows patching spam. Please don't be so narrow minded.
Windows patching porn spam, actually, incorporating further data from this thread...
(Actually, there's no problem at all - 100% of traffic on some networks may be Quake, while 100% of the traffic of some other networks may be something else...)
Patching vs UnPatching (Score:3, Insightful)
Actually, just the act of patching may roughly equal. But UN-patching a system can be done very easily on a *nix based system. How do you UN-patch a Windows based system?
Also, when I rebuild apache, I know what I am affecting. When I install a Windows patch, I cross my fingers.
Re:Patching vs UnPatching (Score:2)
Re:NEW MATH (Score:3, Funny)
Quick! Get Bezos! We've got to file a patent on Bad Code before anyone else does!
Re:NEW MATH (Score:5, Insightful)
Re:NEW MATH (Score:2)
responsible for 45% of traffic
But spam is responsible for, what was it Taco, 60% of traffic on networks?
Microsoft's patches obviously contain spam! I would consider desktop icons for AOL or MSN count to be spam.
Re: (Score:2)
I prefer Linux, but... (Score:2, Interesting)
Sometimes I wish there was the equivalent of Windows Update for Linux. If it wasn't worth the effort I wouldn't be using it, of course, but the asymmetry between the Windows patches and Linux patches doesn't seem
Re:I prefer Linux, but... (Score:5, Informative)
It works well for me, and all I need to stay on top of are things I build be hand (typically Webserver and its ilk plus kernel), but all my libraries stay nice and fresh.
Re:I prefer Linux, but... (Score:5, Informative)
apt-get update
apt-get upgrade
I don't run Debian's precompiled kernels though so I don't know what the patch/release policy on them is, but for all userland things it's better than WU.
Apt4rpm (Score:3, Informative)
Say it ain't so! (Score:5, Interesting)
apt-get upgrade
That's what I do, and I'm not sure what all the fuss is about. Things get fixed, usually before I ever knew they were broken, deamons get restarted, nothing gets interrupted, life goes on ... If I took the trouble to make it a cron job, I'd never even know.
Is Mr Fiebig telling us that things don't go so smoothly if you use MS products? Or that MS can't keep up with a bunch of amatures? Do MS patches break non-MS apps? Could all this be why so many worms and viruses manage to spread across unpatched MS products? Could it be that MS patches are as bad as the bugs they fix? SAY IT AIN'T SO, CRAIG!
Re:Say it ain't so! (Score:2)
apt-get upgrade
If I took the trouble to make it a cron job, I'd never even know.
I have them in my cron (upgrade -d -qq, so everything gets downloaded) (and another job sends me a mail if need be), but I prefer to be connected to the box and to see that the upgrade goes well (as it has. except for minor details around the one time Debian Woody became Debian stable) Ought to make the cron job run the updates automatically, if they have been laying around for more than a week - probably s
precompiled kernels (Score:2)
Precompiled kernels work just fine and Debian's /etc/modules file makes it easy to change around hardware. Going from 2.2 to 2.4 was easy stuff. I can only imagine that they will use the same kind of upgrade policy for kernels as they do for every other package now, therefore I expect my kernels to be patched if some kind of flaw is discovered.
The po
Re:I prefer Linux, but... (Score:2)
RHN (Score:2)
Redhat network works wonders for me. It catalogues all of the software that shipped with Redhat, and lets me know which of my systems requires what errata (updated software). Third party isn't a term that most open source companies recognize. =)
Best part is, I don't have to be on my actual system to check for available updates. I just log in to the RHN and look at the list of my registered systems. This trounces Windows update IMHO.
YUM (Score:2)
/joeyo
Re:I prefer Linux, but... (Score:3, Insightful)
In essence, there is. Just requires (as always) a little manual setup on your own.
I have one central update box. It runs fmirror every three hours, pulling down the latest Mandrake patches (8.2, 9.0, 9.1) and emails me if there has been a change.
That box has NFS exports (you could use ftp, if you wish, to avoid the NFS problems) to all the other servers.
The other servers have the update box defined as an "update" source in urpmi.
I
Re:I prefer Linux, but... (Score:2)
Not always. Depends on the particular patch being applied.
Random FUD and misinformation does no one any good.
Re:I prefer Linux, but... (Score:3, Informative)
Re:I prefer Linux, but... (Score:2)
Re:I prefer Linux, but... (Score:4, Insightful)
Nothing new there (Score:5, Insightful)
Want security? don't install the kitchen sink! (Score:2, Insightful)
Not suprising (Score:5, Insightful)
Lamers (Score:5, Funny)
Pff.. you lamers with your fancy-pants Windows or your free Linux or *BSDs are all clueless. I haven't patched my Apple ][+'s DOS3.3 for 20 years and it still has yet to be 0wned.
Re:Lamers - Oh Yeah? (Score:4, Funny)
Re:Lamers - Oh Yeah? (Score:5, Funny)
Yeah and? Today is Thursday, May 1 10003.
Re:Lamers - Oh Yeah? (Score:2)
If he's running DOS 3.3, he can't run Appleworks. He'd need ProDOS for that.
Patches (Score:3, Insightful)
-----------
From Ape to Man: Evolution [turnpike.net]
Interesting sidebar. (Score:2, Interesting)
The most expensive security measure is... (Score:2, Funny)
rim-shot
Thank you. I'll be here all week. Tip your waitresses.
MS patches are creepy... (Score:5, Insightful)
I've applied my fair share of patches from MS, but lately I've become really nervous about doing so. I'm always thinking "what kind of DRM will they include in this one? [slashdot.org]". It's gotten to the point where I will NOT apply patches for anything but server products, and only reluctantly so. Call me paranoid if you wish, but I can't really shake that feeling. Hey MS, great way to promote security - making users reluctant to apply patches...
Re: (Score:2)
Re:MS patches are creepy... (Score:2)
No such luck - I work at an all-MS shop, and apart from the all-MS issue it's a great job.
At home I've got an old box running RedHat to play around with, so I get out of the clutches once in a while where *I* decide what gets to run on my machine...
it make sense they would say this.. (Score:4, Informative)
So they will use this 'cost savings' to push the new product. At the launch event, they bagged on there older products pretty damn hard.
It's part of there latest slogan
"do more with less".
personally, I dln't know who this less guy is, or why I would want to do more with him. Ironically I prefer less to more.
I feel the pain (Score:3, Informative)
Guess who gets to come in the office between 8 and 10pm to apply these patches to live servers...who has to wait if someone decides to work late. Who has to cross his fingers with every patch hoping that nothing else breaks...ME! And the only thing I get out of it is to be able to leave an hour or two early that friday...woot.
Sure some things I can and do install from remote, but almost every patch requires a reboot and you just never know when a Win2k system isn't going to boot properly and require you to drive in at 1am wearing your bath robe.
Re:I feel the pain (Score:2)
Work smarter.
Re:I feel the pain (Score:2)
Nope. It still costs the company. Because if you didn't have to do all those things they could probably pay you less or have you do something else that you don't have the time to do today. At the end of the day, it costs your company money.
Re:I feel the pain (Score:2)
Re:I feel the pain (Score:2)
Clearly, you have not read enough BOFH. If someone decides to work late, you ensure that he can never leave the office again!
And what, really, is wrong with kicking everyone off the server at 10:30 AM and not getting the patch installed until 4:30?
Learn more user-control tips here [ntk.net].
Stating the Obvious (Score:2)
In other news, the sky has been discovered to be blue and the planet has been proven round....
System Update Server (Score:5, Informative)
Re:System Update Server (Score:4, Informative)
Works great and my mrtg graphs prove that it works to me
Re:System Update Server (Score:3, Informative)
I am playing with SUS server and its group policy settings, and there is no way for end users to initiate downloads. I can make updates happen over night, and force pc reboots, but I am not thrilled with that solution (i feel that may negatively reinforce user's locking their workstations as a routine behaviour). SUS doesn't interact with windows update at all - disabling windows update via group policy isn't an ideal solution either
Question? (Score:5, Interesting)
Bell labs(now lucent) and various hackers have made string functions that do the same thing but are buffer safe. They are made to create more secure apps.
My question is if gcc or visualc for that matter switched to more buffer safe libraries would it make a difference? Trusted Debian is compiled with buffer safe string functions.
It may be time gnuc did this by default assuming all the apps could be recompiled without a problem.
This would seem to get rid of %90 of holes in user as well as kernel space.
Re:Question? (Score:3, Informative)
The basic string copy functions in C and C++ don't keep a value for the maximum length of a string.
(Actually, they don't even keep a value for the current length of a string, it is calculated by scanning the string and looking for the terminating null.)
The buffer safe string libraries are not designed to be a drop-in replacement for the basic string library, because they demand more information about maximum lengths from the code using them.
Still no liability? (Score:2)
patches hurt people too (Score:3, Funny)
Hmmm... (Score:4, Insightful)
Well... before the knee-jerk MS-bashing starts, let's think about it.
If you patch, you have to recompile the component, and possibly re-boot the machine or re-start the application. This is true for Linux too (unless there's a way to fast-swap kernels that I haven't heard about).
If you update, you don't need to re-start anything.
If you patch, you could have to patch just about anything on the system.
If you update, you are working through one application.
Of course, there's nothing to stop an OSS developer from writing something that just sniffs incoming data for known exploits, like a virus scanner does.
Ahhh... but that would slow the system down.
So I think you have to add "better performance" to the pro-patch argument.
But then, there is probably less effort to updating, especially if it's automated. Is there any OSS system with automated patching that people are willing to trust?
Either way, I think it's an interesting discussion. In practice, I patch.
Re:Hmmm... (Score:3, Interesting)
yes
"that people are willing to trust?"
errr.. no.
change it to
"that corporations are willing to trust?"
yes..Windows 2003.
That's one more reason to like Apple. (Score:3, Interesting)
The download sites are controlled by Apple (and Akamai for all I know) but Apple really serves up the content.
Also they have a better, more secure OS that's conservatively designed and carefullly implemented so viri scouring and bug fixes aren't quite so desperately required by the system owners.
M$ may be too anal-retentive for their client base's own good. The only thing they want to conserve is their cash flow.
Distinctions (Score:2, Insightful)
does licensing allow for non-production installs? (Score:4, Interesting)
LOL the obvious hit M$ in the face (Score:3, Interesting)
Not to let SUN off the hook, their patches take some reviewing as well
CVSUP even my applications get patched (Score:3, Informative)
I've got "cvsup ports-supfile" on a cronjob
Every day I get emailed a list of the applications that have been updated and I can choose when it's worth patching them (they might not be installed - for instance)
to upgrade my *whole* set of port installed software
#portupgrade -ra
& everything stays in regular updated form
I magically keep in step with the mozilla builds
it's great
that's why FreeBSD ain't dying
Re:Well then... (Score:4, Funny)
Re:Well then... (Score:2)
it is an OPTION, so if you like installing a patch on 100 servers by hand,by all means go for it.
Re:Downtime? (Score:3, Insightful)
Re:Downtime? (Score:3, Informative)
Re:Downtime? (Score:5, Insightful)
Yikes. I don't think 'apt-get update && apt-get upgrade' in your crontab is very smart. The probability of breaking something is too high. In fact, that's the message I'm reading between the lines: virus upgrades won't break anything, so they're no problem to automate, but OS/IIS/IE patches pose a much higher probability of risking extended downtime. I don't think the situation is all that different with the Red Hat Network-- look before you leap.
Re:I don't understand... (Score:3, Insightful)
(Some patches break some applications) + (Applications being down means lost productivity, sales, possibly data, depending on the app) + (MS apps won't let you roll back the patch, so you can't recover) = Many companies feel the need to test the patches first.
My computer at work doesn't get patched all that often (luckally it's behind multiple firewalls), because Unigraphics is very touchy (according to our support people).