Recommendations for Third Party Security Audits? 356
palehorse asks: "I am a developer/DBA/etc for a very large State Govt. Agency on the East Coast. We have been subjected to an increasing number of break-ins and website defacements over the past few months. My boss has recently been tasked by our CIO to find a reputable third party (not us or our ISP) to come in and do a complete and independent security assessment/vulnerability analysis for us. Since I'm the guy who usually bugs folks about security, she tasked me to come up w/ a list of firms who could do this for us. and a plan on what to test for and how. I've done the whole Google search/ZD-Net search/etc, which has given me way to many folks who do this kind of stuff, from ISS and IBM on down. Consequently I wanted to get some feedback/suggestions from the Slashdot community on where to go from here."
"Please keep in mind that while we're a large government agency, we have a small and overworked IT staff who have no real experience in internet/web security, and who are just now getting into a serious web presence.
Here are the main questions that I have:
- Who have you used, and were they any good?
- What should we look for in evaluating who to contact and their proposals?
- What would you have done differently?
- What services should we ask for?
- How do we manage the contract to make sure we're not getting a snow-job?
- How do we use the result to get buy-in from management (who will probably need to lay out money for changes) and from the developers (who may be adamantly opposed to changing their systems, either for ego reasons or it's just a lot of work)?
- How often should we re-do these audits?
How about (Score:3, Funny)
Worked for Enron.
http://www.terradoncommunications.com/ (Score:1)
Ya know, I get a lot of their e-mail. (Score:2)
I own Asguard.com
Thanks.
References (Score:1)
Re:References (Score:3, Interesting)
The biggest problem was that the company I worked for didn't want to actually implement the suggestions because it was going to cost some money for things like a real firewall.
I've also had bad auditors come in, usually forced on the admin group by managment and sales staff. I would advise the following to avoid these types:
First, ask them ahead of time what thier requirements are to get started. If they say "root access", show them the door. There is no talent in a company that requires full access to see if you are vulnerable (Note: there is nothing *wrong* with giving them access as part of the audit, but they shouldn't be *starting* there).
Matter of fact, if they start with wanting to login to your servers, you can probably do better.
Make sure they understand trust trees.
Make sure they are familiar with your OSs and critical applications.
Ask for, and check up on, references.
It sounds like you are off to a good start. Having managment ask you to plan something will mean you can get a real audit.. I've been through several where the "audit" started with me handing out root access so they could run "crack" on the shadow files, followed by a find command to look for world writable files, etc..
Well... (Score:2, Offtopic)
and IBM on down
They say nobody ever got fired for choosing IBM. Of course, I find that hard to believe. Surely somebody must have chosen IBM technology when it wasn't appropriate, and gotten fired. Anybody have a story?
Re:Well... (Score:1)
If you can get them, Foundstone (Score:4, Informative)
save yourself money. (Score:1)
Re:save yourself money. (Score:1)
The other thing that would normally come out of security audits are plans to make sure that software that is being used and requires utmost security are updated in a timely fashion. Sure, you can run a Linux/BSD-based firewall, but what if you haven't updated the OS and firewall software for 4+ years? Or how about the DNS servers that are still running either BIND 4 or god-forbid, early versions of BIND 8.
For Apache+PHP web servers, there were a couple of nasty bugs found in PHP 3 and PHP 4 that were quickly fixed... but I still know people and companies that run on Apache 1.2.x and PHP 3.x that don't really keep up with the latest security bulletins.
Microsoft of course! (Score:1, Funny)
Re:Microsoft of course! (Score:2, Funny)
large state government on the East Coast (Score:1, Offtopic)
You mentioned IBM...want to keep the business in-state?
Bet it's NY...
Re:large state government on the East Coast (Score:2)
East Coast? Govt? MITRE? (Score:1)
Audits on the Cheap (Score:5, Funny)
Tell him you'll give him or her a free laptop, and 5 cases of Code Red if they can break in and tell you how they did it.
Re:Audits on the Cheap (Score:1)
(from the original article: We have been subjected to an increasing number of break-ins and website defacements over the past few months.
Holy shit. It doesn't take a fucking Kreskin to secure a web server. What, did they set the root password to 'password' or something?!? Man, Mandrake makes it super simple to do security nowadays (from the sounds of this article I'd be surprised they're running anything geekier). And if you're running IIS, whoo boy, with Microsoft's IIS Lockdown [microsoft.com] tool, it's so exceedingly simple you can get a troupe of trained circus monkeys to secure that web server of yours (discounting the fact that you'll get monkey piss and shit all over the server keyboard, but that's another story).
My other question to the article submitter:
You did reinstall the operating system after the first defacement, and restore from data, not binaries backups from before the defacement, RIGHT? Ok, just checking.
Re:Audits on the Cheap (Score:2)
In managing a college computer lab, I hired many of these guys for lab aides. I learned more from them than I did most of my classes. They were very smart and very creative. Note, these are not the script kiddies, but young, fresh thinking, CS students not with larceny in their hearts, but a desire to outwit 'the system', just for fun. I had very good luck with them not crossing the line.
Who better than them? After all, who are you attempting to be secure from?
These guys might be a good checksum after having the big professional folks come in and do their audit and you performing the fixes they recommend.
Re:Audits on the Cheap (Score:2)
ISS (Score:2, Informative)
If you want to spend large bucks, hire a security firm such as ISS. If your agency doesn't want to spend a lot of money, call a bunch of geeks (like me) to come in and audit the system. IE: replacing wu-ftpd with pure-ftpd, IIS with Apache 2.0. Find the services that are full of holes, and replace them with somthing that has a reputation of security.
Well. (Score:2)
IIS's main selling points are its ease of admin' and it's speed. If you have to have a 'good' admin in order to keep up with the patches, then it's not in fact easier to admin then apache.
Sun Tzu (Score:1)
They were pretty thorough in their research of our systems. We also hosted a security seminar (we're an ISP) and they came in and did a presentation. They seem pretty knowledgable. They're based out of Milwaukee, I believe.
http://www.suntzu.net
We've used ISS (Score:5, Informative)
I don't remember the cost, but I'd use them again.
Is it really what you need? (Score:5, Insightful)
IMHO opinion an audit is not what you need, spend the money employing someone who does know about security to get (and keep) things ship shape. Security is an ongoing issue and can't be solved by a one of check, the audit could be perfect but your still wide open the next time some kiddie finds a hole in your preferred webserver software.
Re:Is it really what you need? (Score:2)
Security is not something that you can buy, or rent from some auditing company. After the report and recomendations are handed back to you, _YOU_ have to implement them, maintain them, and live with them.
You need someone on the inside who loves security (and not just a know nothing, ego tripping, data nazi). Someone who understands the basic tenents of scurity as well as what the implications are when you run software package X on OS Y or use brand X of hardware. They are probably going to have either network admim, and or System admin experience. They will probably not advertise themselves as a security expert since that is not typicaly a position that is well know outside of MegaCorp land.
Good luck with your hunt!
Re:Is it really what you need? (Score:2)
Re:Is it really what you need? (Score:2)
hanzie
Security as a process (Score:5, Insightful)
The first step (really!) is to get a security policy in place. This really doesn't have to be anything special-- but it does need the buy-in of ALL groups affected (sysadmins, developers, marketing, sales, executives, etc.) That's really the only hard part.
Probably the quickest way to get started is to head to the SANS security policy project [sans.org] and adapt their sample policies to your company. This is one of those rare cases where it's more important to get something in than it is to get it right the first time. Policies can be changed fairly easily-- but you don't want to go to all the trouble to implement a secure environment only to have someone on the inside fighting you every step of the way.
Now the fun part-- actually securing your systems. Here are some pointers on places to start:
1) Review the SANS "top 10" security vulnerabilities [sans.org] and make sure they're covered.
2) Review Lance Spitz's excellent collection of host security information [enteract.com] and make sure to follow his recommendations.
3) Make sure your firewall rules are set up with the security best practice of "minimum access to get the job done". Far too many firewalls allow traffic they shouldn't.
4) Get NMAP, a network mapper, port scanner, and OS identifier [insecure.org] and run it from the Internet to your exposed (i.e. DMZ) hosts. Also run it from your exposed hosts to your internal network to validate that only the traffic that should get in can get in. (The traffic allowed back in from your DMZ should be very little, preferably none.) If you find anything that is inconsistent with what you think should be happening, check your firewall rules again.
5) Grab a copy of the Nessus security scanner [nessus.org] and run it against your newly secured systems. If it finds anything, read the description of the problem and see if it's something you can fix. You can bet that everything you find here will also show up on your "security audit" since most "audits" are just someone running a tool like this and then feeding the output to the consultants to make it all pretty for management.
6) You should have most of the obvious, widespread holes plugged by now. This would be a good time to get some sysadmins out to some classes. Verisign [verisign.com] has a number of excellent general Internet security classes. I'm sure there are lots of other good places, too. I was pleased with Verisign because of their Internet focus. Too many security classes only concentrate on host security and neglect network security.
7) Get the application developers at your site to read and follow Dave Wheeler's writing secure programs guidelines [dwheeler.com]. This is a lower priority than OS/network security since these holes are likely to be specific to your site only. Only a determined hacker is likely to find and exploit them-- however exploiting application bugs/holes can severely disrupt your business. What happens when an electronic data interchange transaction gets bogus data inserted? How far will that bogus information make it in before it's detected? In the worst case these bugs could result in people getting free products/subscriptions, stealing credit card info, or destroying data inside your systems.
8) Now it's time to get that audit. They will be able to tell you what you missed in the previous 7 steps. Why wait so long? Most places will keep looking until they find something to report. If you do this too soon, the subtle security problems will be lost in the noise of all the obvious problems the previous 7 steps would have fixed. If you do this last, only the "hard" problems are left for them to find.
Remember above all that this is an ongoing process. Keep current on your patches, and repeat all the above steps regularly to keep all the bad guys away.
Re:Security as a process (Score:2)
I figured that since he was talking about a security audit that he had already done damage control. Clearly the first step is to fix/block the holes that have already been exploited.
I disagree that a third-party penetration test is appropriate for this stage. He *knows* that people can get in.
This would be the perfect time to get the CEO's signature on a security policy. I bet he/she already knows about the problem and is more than willing to do what it takes to get it solved. This signature/buy-in will save the sysadmin/operations staff days or weeks of arguments and petty internal squabbles later on when people balk at the security improvements that were needed to keep the hackers out.
After plugging the already exploited holes and possibly (if you can) slapping in some draconian network security (i.e. block EVERY port but port 80 to your servers and let the other applications suffer for a day or two...) the VERY NEXT STEP should be that security policy.
If people drag their feet, remind them that they won't be 100% operational until it gets done.
Don't skip it-- it's important. Really. In a worst-case scenario you might be forced to REMOVE your security a month or two down the line when the threat seems to have diminished. Instead of spending hours and hours in meetings trying to justify the security each time someone has to learn a new way of doing something that "used to be easy", you can refer people to the policy.
Look at KPMG (Score:3, Interesting)
Definately KPMG. (Score:2, Funny)
There's even a jungle remix! w00t!
:wq
(Personally, tho, I like IBM's "Ever Onward [telocity.com]". Just has that
"1930's cartoon with happy singing cows" feel to it.)
Two thoughts. (Score:5, Informative)
http://www.wealsowalkdogs.com/
I don't know if counterpane.com does audits, but you should definitely consider their managed security service if you don't have a dedicated on-staff security person.
Finally beware these types of audits, they often don't look at your procedures and policies, which are the root cause of most problems. It's always good to have external cross checks from a different point of view, but be very careful about assigning too much importantace to them.
Re:Two thoughts. (Score:2)
Re:Two thoughts. (Score:2)
Well since it doesn't make sense to put an IDS or network monitoring into a network that's already get lots of security holes, I would bet that Counterpane either can conduct security audits and help fix up the network, or they know people who do. Remember Schneier's mantra: security is not a product, it's a process. Also remember his warnings about snake oil, particularly in the post-9/11 world.
--Jim
Look at the bright side. (Score:2, Funny)
Look at the bright side. If they don't do good security, you can have them walk your dog.
Core-SDI (Score:1)
Those people really know what they're doing.
Maybe not the cheepest but.. (Score:1)
Microsoft Security is your best best (Score:1)
Re:Microsoft Security is your best best (Score:1)
Re:Microsoft Security is your best best (Score:2)
if msft rates you 0, it must be harder to properly authenticate against your services than to hack them.
if msft rates you 0, you're probably providing the root passwords for your services in the README
if msft rates you 0, you're probably inconveniently attempting to confirm users intentions before running harmful logic
if msft rates you 0, you probably didn't include _enough_ backdoors in your code
i could go on all day
DISA (Score:1)
This is Funny!! Not offtopic, overated, irrelavant (Score:1, Troll)
Taking bets on moderation totals now
Re:This is Funny!! Not offtopic, overated, irrelav (Score:1)
Hmmm (Score:4, Funny)
Oh but he can't access computers...
Re:Hmmm (Score:2)
What about Mitnick...
Oh but he can't access computers...
doesn't have to. He's so elite nowadays that he just to channel his Geek Powers and a root shell pops out of the air in about a minute. Or so I've heard.
Mitnick didn't have to.... (Score:2)
How about.. (Score:1)
Re:How about.. (Score:2, Informative)
Some advice from the inside (Score:5, Informative)
You should look for:
- resumes of staff performing this activity, for the folks who will actually be conducting the work. How experienced are they? Beware of firms that send their people to a one week training class then turn them loose as experts.
- Breadth of experience in OS, server and middleware products. Don't hire a bunch of UNIX bigots if you have WIn 2K servers. Not only will these folks not be familiar with the technology, they will also have a bias towards bad-mouthing it.
- Do they understand how to rank and prioritize the risks based on the needs of *your* environment? Anyone can generate a cookie-cutter report from a packaged tool. To what extent do they apply some human intelligence to this?
- Following from this, what does the report look like? Do you get a cookie-cutter intro with a zillion pages of ISS output, or do you get something meant for a human being to read?
- Breadth of assessment - do they look at routers and switches? Servers? Applications (is that Oracle financial application wide open)? Desktop machines?
- Are results based solely on a network scan, or do they actually look at host configs that may not be visible from an outside scan? Do they interview staff to get some idea of practices?
Re:Some advice from the inside (Score:2, Insightful)
And vice-versa. Geez.
LEXX
Re:Some advice from the inside (Score:2)
What about looking in the HOWTOs (Score:1)
I would look in those first. They knowledge on certain matters has been approved by the whole OS community which has seen their HOWTO and agree with it.
How much do you have to spend? (Score:2)
I work for a Fortune 500 company and we had one of the Big 5 consultants do a 2 day port scan for us. We wanted a third party as well. They wanted $12k for 2 days work.
Re:I JUST INSTALLED LINUX... (Score:2)
But he had a WinXP sticker on the car so I think he was running LowRide2000...
Know what you ask for (Score:2)
Your IT staff might not have experience but it is unfair to assume they can't do something once you ask them to do it.
Never forget that nobody can read your mind
Why not ask other agencies? (Score:4, Insightful)
Wait.. What am I saying? This is government; agencies don't work together. Nevermind...
The guys that have been around... (Score:2, Insightful)
WARNING: Personal Opinion (Score:2, Insightful)
Find someone who actually gives back to the community, such as packetstorm or the such.
You might also consider Security Focus and places like that.
I'm not sure what your actual goal is, but if it is to actually secure things instead of having a bunch of monkeys come in and take some money from you, then places like that will have the best results.
And try to stay away from those who will require you to buy something, and subscribe to something else in order for you to be secure. ACLs on routers and removing unnecessary services/daemons, and patching those that you need will do a lot more than a firewall from acme security.
---
"Security is a process, not an event". -Some smart person
Re:WARNING: Personal Opinion (Score:2)
Re:WARNING: Personal Opinion (Score:2)
---
Due to the lagging economy, this
Re:WARNING: Personal Opinion (Score:2)
I don't doubt you. I have a lot of respect for them individually, and as a whole. But I think their focus has changed from the old l0pht mentality to the new corporate one. And although they might be corporate now, I am not saying they're any less smarter than they used to be. Just their priorities have changed.
---
A fool must now and then be right by chance. Right?
Poot's Security Shack (Score:2, Funny)
I recommend this great company I found out about, called "Poot's Security Shack".
I... um, I mean, we... I MEAN THEY do a great job, and they cost less than all the big fancy companies with offices and business plans!
Email them at poot@dork.com for more info. Sorry, no refunds.
Netcraft (Score:3, Funny)
http://www.netcraft.com/security/ [netcraft.com]
S
Get the broadest test you can find. (Score:1)
Big-5 Accounting Firms (Score:2)
All of the big accounting firms (KPMG, Deloitte & Touche, Ernst & Young)offer this service. They are fairly reputable and thorough.
I suspect that you might want to pass on Arthur Anderson though, based on their enronic experience...
Re:Big-5 Accounting Firms (Score:5, Funny)
Re:Big-5 Accounting Firms (Score:2)
Personally I wouldn't trust any of the Big-5 for any consulting job. Their accounting branches are only bordering on reputable and that's the best part.
Simple Solution (Score:1)
er...
Wait...
nevermind.
I have heard that these guys are great! (Score:2)
It's not a joke - they actually have such a group (Score:2)
How about... (Score:2, Funny)
WaySecure.net (Score:2)
--Peter
2600 (Score:2)
Price Waterhouse Coopers (Score:1)
Lumeta Corporation (Score:2, Informative)
A few thoughts (Score:5, Informative)
1) you want a complete report, not just a management summary. Make sure there's guidance in the report on how to fix the problems they find, or at least a pointer to where to find the information to fix them.
2) black-box "we can hack anything" audits are sexy, but won't show you the whole picture. Make sure they're looking at both the external settings and any local policy security settings on the machine.
3) Ask to have some of your staff sit in on the audits...you want to learn from this audit as much as possible. If they say "no", ask why. If they're just trying to protect their "script-fu", run...they're probably fake.
4)Get a contract in place that makes it very clear what they are supposed to audit, what they are not supposed to audit, and how they are allowed to do it...get that in place *before* the audit starts. (a "terms of engagement"). This includes what IPs to audit, and what techniques (DoS, social engineering, etc) are allowed.
5) as others have mentioned above, ask for references. If they can't provide them, worry.
I'll stop now. I'm sure there's more, but that's what occurred off the top of my head.
Re:A few thoughts (Score:2)
But, unfortunately, many (less than reputable) companies will refuse to let you see what they're doing at all...usually (in my experience) this is for one of two reasons:
1) they're going to simply run a commercial vuln scanner against you, and then re-package the results. In this case, they don't want you to realize that you can do this yourself. (and for free if you're not alergic to Nessus) A real audit will use a scanner (no reason not to), but then use that as a base point for further exploration.
2) they're actually totally incompetant, and having you watch them flail about will make you realize this. I've watched auditors try to talk their way out of an audit where they audited the wrong machine...it wasn't a typo, it wasn't a nearby range, they just went somewhere else to audit. Those folks really didn't like getting questions...we learned why very quickly.
I'm not saying that your group is doing either one of these. In fact, since you're allowing spectators, you're clearly one of the clueful ones. But, unfortunately, some others are not, and you have to be aware of that when looking for auditors.
Bruce Schneier (Score:1, Informative)
heh (Score:2)
You really should watch how you phrase things around this crowd
Who to go to for an audit (Score:5, Informative)
Another company that you might find useful is Lumeta. This is Bill Cheswick's company, and they take an innovative approach, in particular relating to networking audits. They map your network and create visualizations. See www.lumeta.com. One of their senior folk is Tom Limoncelli, whose book "The Practice of System and Network Administration" was recently reviewed on SlashDot.
How to choose (Score:2, Informative)
I work in this specific industry and you need to be careful how you screen companies. There are a few caveats to watch for:
Ask for references but don't be surprised if they can't give a lot. Why? My company does a lot of work for the Federal Gov't as well as state governments and the work is usually under a NDA. You wouldn't like me to say "sure we audited so and so and found 25 holes" either.
Ask for their methodology and review it. Don't always believe the hype about "custom tools" etc.. Make sure they have some level of redundancy. I worked for one firm that used strobe and ISS and nothing more. Ask what tools they are going to use. Be nervous if they don't want to tell you. You'd be surprised at how many "big players" really are scam artists.
Make sure the resume's you see in the proposal are the people doing the work. You don't want to hire and pay for mudge, only to have Tony the pony come run the scan.
Check the reputation of the finalists. You definitely dont want a fly by night shop doing your work, or a company that might not have good ethics.
dewke
SecureTrendz (Score:2)
Assessments can range from a simple Internet presence audit, to a full-blown enterprise assessment, including policy review and design. All projects are tailored to the customer's needs, goals and expectations. There are no 'cookie-cutter' solutions. Knowledge-transfer is a key component of ST projects. They really endeavor to educate their clients rather than keep them dependent.
ST's engineers are outstanding. Where many assessments stop at simply finding vulnerabilities, the team at ST are often able to leverage access against other systems on a network to provide a very realistic idea of how vulnerable you may be. From both a network/systems and business perspective, they simply have a deep understanding of weakness, vulnerability and risk management.
I know a few people who work there and I highly recommend them.
www.securetrendz.com [securetrendz.com]
sedawkgrep
Several, rotate often (Score:5, Informative)
Security is a mindset and process at least as much as an implimentation. Therefore you don't just need a good aduit, but you need continuing aduits.
Counterpane and Bruce Schneir are the best known names in cyrptography consulting today, but I don't expect them to know much about much about virus attacks.
You probably need several different audits (or maybe an extensive IBM audit) just to get started. However never allow the same auditors in more than two years in a row. (The first year to find problems, then second to find problems in the fixes) People who know what is going on in detail should be working for you, you want an outside, untainted by prior knowledge and and hard work.
Make it a policy that you hire auditors on a two year contract, and make it clear that it is NOT renewable, and they cannot get further buisness in this audit for two years.
Try everyone. Once all the big guys have been through and given you a stamp of approveal you should allow the common theif to see your entire procedures, and get his recomendataions. (Don't nessicarly follow them of course). Try small companies and big ones. Small companies tend to cover one area very well, big ones broad areas not as deep. You need both.
This isn't an overnight fix. It took openBSD several years to become secure. Today they have a well earned reputation as least breakable system. If I remember right they had to go over the same code 3-6 times before they got most of the secuirty problems out. They were not even looking at security, they were looking for things that were wrong.
If you buy closed source code (nothing wrong with it), make sure you vender works for security. You can't fix the holes in a sieve with confidence that the fix will hold. Open source is a little better, but you might have to pay someone to fix those.
Remember that external audits are an assurance. Most of the work is internal. So make sure management is giving everyone enough time to fix the bugs in their own code/implimentation.
These guys were good enough for RCMP (Score:4, Informative)
The guys the RCMP had do it were experienced, knowledgeable, and had ties/backgrounds that included work with the Canadian Security Establishment (Canadian NSA) and the Canadian Military. One of the guys I worked with had just finished some serious security work for CSE. I know enough about crypto and comms protocols myself to know when (as far as security)I meet people who are "the real deal". These guys were it. And they opened the eyes of some of the public wireless providers in a big way.
They can be found via the info at the bottom of this link here. [lgs.com]
CSE & audit trails (Score:2)
you make a good point about security being process related and the usual weakness being human. A $5K crack on the local secretary is more effective than a $50K crack on the network and far cheaper. Not only might you get security info, but you might get important info on where things are stored and what is stored.
Another oft forgotten part of security is auditing - not just knowing that you've been compromised, but knowing how badly and for how long. That can be as important (well, nearly) as defending against the (probably inevitable) crack anyway. At least then you know what was compromised and can take mitigating steps that are targeted. If all you know is you've been hacked, you don't know a lot. If you have to change every aspect of your process, that's a huge expense. Having mechanisms in place to help identify what was accessed in an intrusion is more than slightly useful!
System Vulnerabilities (Score:2)
Seriously, however - if you are having continual troubles with this and an admittedly overworked IT staff unfamiliar with system security issues, get someone who
GRC! (Score:5, Funny)
www.grc.com
J
Details (Score:4, Insightful)
How wide is your network area? Multiple locations? Same cities?
How about your network infrastructure itself? Routers, switches, etc.
A complete audit can take a while and cost a lot of $$, especially if you have a wide range of system types and network spread. It also can depend on how deep you want the audit to go.
I work for Lucent doing large scale audits, so can only comment on what I've experienced. Security is as much policy, training and implementation as it is software/hardware.
E-mail me if you want some detailed information.
Charles Hill
One way to do a cursory audit... (Score:2)
In addition to hiring the pros, you can also do a considerable amount of auditing yourself with the right automated tools. Among these is the program MultiProxy [multiproxy.org] allows you to enter the IP addresses of your machines and quickly see if outsiders can use them to mask their identities during an attack. Its definitly not a substitute for a real audit, but it can help you to get a quick overview of potential problems.
High-tech Contractors (Score:2)
The NSA! (Score:2)
An honest answer (Score:4, Informative)
Here are the main questions that I have:
Who have you used, and were they any good?
I work for a company that does full service security penetration testing, secure network architecture design and implementation, remote monitoring of IDS and other logs. You can email me through my slashdot user name link if you wish or hit our website www.caci-nsg.com [caci-nsg.com]. Therefore I use my own knowledge and that of my co-workers (some of whom work for Attrition.org btw) and yes, we are very good.
What should we look for in evaluating who to contact and their proposals?
You should make sure they have experience with the various OSes you run. People who know how to knock over a UNIX system may not do well against Microsoft and vice versa. Make sure they tell you what needs fixing AND how to fix it.
Some companies I've had to compete with only showed up with one system to run the ISS scanner, generated a _very_ thick report of what was wrong, and left.
No single scanner is perfect and if you don't have human intelligence to interpret the results the test may be meaningless. I've seen the ISS scanner tell people they had a Windows NT system that needed to be fixed. When we checked out the system in question it turned out to be HP-UX!
What would you have done differently?
There are things our team learns at every pen-test we do. Some things I want to do differently would be to standardize our methodology more. One problem is that every network has something about it that makes it unique. This is where you can either go cheap for an "off the rack" solution to your testing or pay for a "tailored suit". Be sure that the team has some real experience behind them though. You don't want the tailor fresh out of tailoring school.
What services should we ask for?
You should ask for a complete report of what the team is able to access on your network. You want to know what can they break into from the internet and what can they break into if they were sitting internally. You need to understand the difference between a theoretical exploit based on how your network is configured and a real vulnerability based on a missing service pack. This tells you about what external attackers can do as well as what disgruntled employees can do. It may also tell you how bad a Sys Admin you have running things. I've gotten one Sys Admin fired because of what I found and his poor reaction to my findings. You'll want a report that explains in detail why you are vulnerable, what to do to fix it, and if possible the impact this may have on your day to day operations.
How do we manage the contract to make sure we're not getting a snow-job?
You can have the team demonstrate for you how they got in. Have them leave a file behind, pull down a password file and crack it, etc. Any team should be willing to discuss things very honestly with you. You may wish to start small. An external test only for a small amount of $$ and time. This lets you evaluate them without being burned too badly.
How do we use the result to get buy-in from management (who will probably need to lay out money for changes) and from the developers (who may be adamantly opposed to changing their systems, either for ego reasons or it's just a lot of work)?
When I broke into a customer that was a credit union and got customer account data, it got their attention. If the test team steals emails or other things from the CEO or other big-wigs, etc. and it doesn't get proper attention with management, I'd look for a new place to work.
How often should we re-do these audits?
Generally twice a year. The main thing is that after the first one you may have a ton of work to do to fix things. You don't want another test until you have had reasonable time to complete your changes. I've had some customers take a year to get fixed up for another test.
Again, I know there's a lot out there by the security firms themselves about this, but I'm really looking for a 'keyboarder-in-the-trenches' view as well. Horror stories are appreciated, since they may give us an idea what to watch out for."
I just hope I was helpful with what I mentioned here. Keep in mind that if you are a government agency you probably have to put the contract out for a bidding process. Write up your expectations as clearly as possible and leave time for a question/response period from the bidding companies. The intelligence of their questioning will tell you a lot. If they don't ask many questions it probably means they don't know what to ask and won't be very good.
What about Social Engineering? (Score:3, Insightful)
Having read a few books here and there on various types of computer crimes, there are a lot of cases where access to a system was gained through a person giving out confidential information to an unauthorized person? In this light, any security audit should include tests of how easy it is to get confidential information from employees and any third party services. For example, there are many small businesses out there in my town that use dialup accounts for internet access and email. Most of these companies will give out the user name and password over tech support if you only supply the account holder's name. This leads to anyone being able to access the company's email. In a big corporation, I'm guessing a few users would give out name/passwords to a call claiming to be from the IT department, if the company has a modem pool, I'm sure its trivial to get that number too ("Hello, Jane Doe? Its John from the IT department. Were doing some work with the phone company, and we're wondering, what number do you use for dialing up? Is it 555-1111? No, you use, 555-1234? Thank you!"
Any good audit should include the social engineering factor.
Just my $.02
Complete solutions (Score:2)
A good security policy would isolate public servers so that if they get hacked it's not a major problem and it's easy to diagnose.
In my opinion you should hire a security consulting firm to come help you design a scurity policy. It doesn't sound as if you have a DMZ set up and that's a good place to start.
Actually the first place to start is identifying what information needs to be protected. A lot of times companies don't protect everything they need to.
But really you need to look at the whole picture: passwords, email clients, wireless, back ups, recoverry after attack, etc.
A good security policy will help you understand what things you need to worry about and what things don't matter. This will help you sleep better and benifit your whole company.
Start from the begining (Score:3, Insightful)
Good security starts with the establishment of a security policy followed by education and regular awareness events. Please be aware that paying someone a ton of money to pen. test and inventory your assets will *not* result in a stronger security posture all on it's own. You must have a policy in place and you must compel your users to abide by it (primarily through education, secondarily through threat of penalty). Consider hiring a CISSP or other certified professional to help you through this process. You might be able to find one in your area by using the ISC2 directory [isc2.org]. SANS is doing some ISO certification as part of the GIAC program [giac.org] now and they may be able to point you towards some appropriate people as well. The ISSA [issa.org] might be able to help as well. As has been mentioned already, you probably don't want to entrust this to someone selling countermeasures [iss.net] or management services [counterpane.com].
Understand, however, that you don't need a firewall engineer right now and you don't need some krad ex-hacker to pen test either. You need someone to help you get your house in order on the administrative side and then you can look into some detailed engineering and assessment. That someone should probably be an independent consultant or at least one working with an infosec specializing firm. If you want a couple bigger names there's @Stake [atstake.com], Booz Allen Hamilton [bah.com], and Predictive [predictive.com], however, I would encourage you to seek out a local independent with good references.
Any knucklehead can run Nessus and patch systems. This alone does not equal information security. If you want a secure environment, start by defining what "secure" means within your environment.
Security is a Process - Not an End-point (Score:3, Informative)
The best thing you can do, if you really need to be online, is to TRAIN YOUR PEOPLE. First in IT, if necessary, then in security.
Doing anything else is a waste of resources that will lead only to a false sense of... well, security.
Yes third party audits are a must!! (Score:2)
Professional security audits for Govt or Big ... (Score:2, Interesting)
Anyway, the original questioner was asking for someone to help his East Coast State Goverment agency. There is one firm that grew out of the government consulting that I've both considered working for when I was consulting and also brought into my own
(-: As a kindness I won't slash-dot the smaller ones that meet the same criteria
The other top consultants to governments, large and small, will be among the presenters and organizers at New Security Paradigms Workshop [nspw.org] (ref coverage [linuxsec.org]).
-- Bill Ricker aka n1vux
Thanks to SUDO, no longer Root@anywhere
Contract Netgraft Corporation.. (Score:2)
We'll provide free initial security auditing just to scare the crap out of you and let you know what you're up against. Then we'll be glad to sit down and discuss the options, etc.
This is more than simply profit motive. Network security is everyone's responsibility and we see it as doing our part.
See the 'ol homepage [netgraft.com] for more info.
I use http://www.mi2g.com (Score:2)
e-mail me [mailto] if you want some more info on them.