From the article:
* Speed of reaction: The faster (the more real-time) a list is updated, the more easier it is to deal with false positives and with false negatives.
* Selection criteria: How are the sources added to the blacklist, based on what criteria? How sure are the blacklist admins that the one they are listing is bad? How sure are you they will not add your partners, customers, suppliers and other business critical peers. Similarly how sure are you they will not list yourself (from experience: this is extremely painful)
* Goal of the blacklist: Does the list have an agenda (hidden or not) that you might not share with them? Do they aim to have 0 false negatives without care for false positives?
* Ease of getting unlisted: How easy is it to contact the list administration for those listed ? Is there 24x7 (remember the Internet is worldwide so thy need to cover all timezones) support on getting back out for those unjustly listed ?
* Working Email contact to get unlisted: This is very tricky for e.g. spam blocking list that are using their own blacklist.
* Try contacting them to get unlisted: if you cannot reach them, remember what your communicating partner that got listed by accident will feel like. And while it might reflect mostly on the blacklist provider, it will also reflect on you and your organization due to your choice and implicit support of their (failing) processes.
* Is there somebody who feels responsible enough behind it to put up out of band contact details such as phone numbers, working snail-mail addresses etc. Of course this means they'll feel exposed to the scam artists they are blocking, but it also means those being blocking without reason have a way to complain.
* Blocking for the right reasons: E.g. some anti-spam lists are blocking with as reason the IP addresses sent unwanted TCP/IP traffic (not just unwanted email). Some might have political reasons or other things you don't want to be associated with.
* Duration of a block: many IP addresses that get infected by bots etc. are home users on a (somewhat) dynamic IP address. Blocking such an IP address for a long time won't help as the IP isn't fixed and the next one to come after it will get blocked unwarranted. Similarly, infected machines do eventually get cleaned up by the rightful owners. So short durations are better.
* Automatic delisting: How automated is the unblocking? Based on what tests is it done? Some listed entities might not know what blacklist they are being listed on. Hence asking them to jump through hoops on their own might not be hard, they might not even know what hoops they need to seek out.
* Granularity of the block: Unless there are clear signs of malice, most regular users will clean up intrusions and malware instead of hopping about the IP address in an address space to avoid blacklists. Hence only very bad neighborhoods should get blocked indiscriminately. Similarly "punishing" an ISP for having a single misbehaving customer will not work as the ISP is hardly punished at all, it's the other (innocent) customers of the ISP that get hit.
While there are people who are going to say they only deal with a specific country/continent and don't need anything outside, think a bit longer: none of the employees of your partners, customers,
* Security of the blacklist provider: Who can submit data to the blacklist, and how is the data authenticated? The bad guys could poison lists by creating fake data and submitting it in order to block even more addresses. Don't forget Availability is part of security: what happens to your processes if the blacklist were to become unavailable or just slow?
* One practice I found to be impossible to deal with from an business point of view: was a blacklist demanding money to get unlisted. Any self-respecting business will feel this is extortion and will not give in. No matter that they send it to their charity of choice, no matter the small amount it actually is, this remains a show stopper. For you this means you'll find contacts who get listed and have no way of getting out again.
* Do the blacklist administrators actually warn those getting listed? Since many of the evil actions a machine does is more often than not done without the knowledge of the rightful owner, a word to the ISP connecting the machine or the business hosting the machine, can in fact be a big step towards detecting the rootkitted botnet and starting the clean-up."