The Most Dangerous Server Rooms 463
Ymerej writes "The Register has an article on dangerous server rooms. Have you seen worse?" Perhaps The Register would like a picture of my desk if they really want to be scared.
This discussion has been archived.
No new comments can be posted.
The Most Dangerous Server Rooms
Related Links Top of the: day, week, month.
Almost anything derogatory you could say about today's software design would be accurate. -- K.E. Iverson
The one we have in the basement. . . (Score:5, Funny)
And they carry card keys.
-bpl
Re:The one we have in the basement. . . (Score:5, Funny)
Re:The one we have in the basement. . . (Score:5, Funny)
And they carry card keys.
Re:The one we have in the basement. . . (Score:4, Funny)
--
Re:The one we have in the basement. . . (Score:4, Funny)
And here I thought it would be for the type of court they'd prefer...
kangaroos are rats? (Score:3, Informative)
I guess they are lawyers though...
Re:The one we have in the basement. . . (Score:5, Funny)
Don't you guys know anything? They're big and friendly, and they bounce up and down the main street of any major city here in Australia. If you give them a carrot, you can ride in their pouch.
The perfect use of a cluttered mess like that (Score:5, Funny)
Re:The perfect use of a cluttered mess like that (Score:5, Funny)
I vaguely recall a similar picture of a suicide setup. I'm serious.
Re:The perfect use of a cluttered mess like that (Score:5, Funny)
In all seriousness though, the backup tapes would probably survive a quick dousing, they look to be in plastic containers.
Re:The perfect use of a cluttered mess like that (Score:4, Funny)
Nah, I'd say putting IIS on every corporate server will better ensure that you're needed every minute of every day of every year for the next couple decades
I mean, just imagine running a Beowulf cluster of WindowsUpdate scripts to keep IIS attack/virus-free!
c'mon (Score:3, Funny)
BOFH would say... (Score:3, Funny)
What's your (l)username?...ok...**clickety click** Why thank you for increasing our IT budget by 1 billion$...now i'm afraid you'll have to go. Do me a favor and check if the pover is on, just stick a piece of metal in the mains... You won't feel a thing
Re:BOFH would say... (Score:3, Funny)
Re:c'mon (Score:3, Insightful)
Server room (Score:4, Funny)
Server rooms can be safe (Score:5, Funny)
Re:Server rooms can be safe -only for techs (Score:3, Funny)
The "photo copy room trysts" follow in a close second place, while the "boss's office boinkings" are becoming more uncommon as more CEOs are of the female gender.
Re:Server rooms can be safe (Score:5, Funny)
For example, statistics show that people who work in server room almost never catch any STDs. I wonder why that is...]
Because the condoms actually work as advertised?
Re:do they really work there or sleep? (Score:3, Informative)
There are more (Score:5, Informative)
Actually, this is the third article about the subject.
I've Seen Server Rooms that were Really Dangerous (Score:5, Interesting)
One of my favourites was actually sitting inside a motor control center, with all sorts of high voltage DC motor starters right behind the main computer terminal. Don't lean back.
ed
Re:I've Seen Server Rooms that were Really Dangero (Score:5, Funny)
Re:I've Seen Server Rooms that were Really Dangero (Score:4, Interesting)
Re:I've Seen Server Rooms that were Really Dangero (Score:3, Interesting)
Re:I've Seen Server Rooms that were Really Dangero (Score:4, Funny)
My boss was telling me about how a guy working in the AC system kicked up some dust and it triggered the Halon system. A voice came on to announce they had 15 seconds to get out of the room before it would be deployed. My boss of course hits the button and stops the countdown. But he lets ago, apperantly you have to hold the button until someone can come by and turn off the system. So he and the network admin go diving out of the room just as the halon is released.
There are other labs on campus that have Halon warning labels on them also, and I wouldn't dare try to check if its true.
Re:I've Seen Server Rooms that were Really Dangero (Score:5, Insightful)
Re:I've Seen Server Rooms that were Really Dangero (Score:5, Informative)
http://www.haifire.com/press/halon_rep.htm
htt
Several of the ozone-friendly replacements (I forget their unfriendly alphanumeric designations) still work by sucking all of the oxygen out ofthe room.
Re:I've Seen Server Rooms that were Really Dangero (Score:5, Informative)
Having thoroughly studied the subject, umm, no, you're wrong.
Halon is no longer used because:
a)it was poisonous
b)it was ozone depleting
The number one cause of death in server room fires is toxic smoke inhalation from burning plastic/paint/wire insulation/circuit board components/etc. Why? People go in to rescue backup tapes, and are rather quickly incapacitated when they inhale the smallest amount of toxic fumes. Folks- NEVER ENTER A BURNING ROOM WITH A FIRE. PERIOD. Backup tapes are NEVER more important than your life. People think "oh, I'll hold my breath." They don't think about how far it may be, or they may get disoriented in the smoke..and they think "oh, just a little breath, i need to breathe." That gets you coughing. Then you die. Smoke inhalation is actually(I believe) the number one cause of death in ALL fires.
Let's not even start about what opening a door into a hot, mostly sealed room letting in a ton of oxygen does.
My "Blue Book" (computer security, mostly focuses on physical) says to NEVER store backup tapes in the same room as server equipment, PERIOD, because the temptation to rescue them is too great.
CO2 is the agent that suffocates you...in that case, yes, the point is to remove as much oxygen from the room, and yes, you will suffocate very quickly.
FM200 is the most common substitute and is fairly harmless, but works on pushing oxygen from the room, and because it is much heavier than air, requires very good sealing of the room in order to work properly. It is next to useless on fires high up in the room unless the system is very well designed.
An alternative is INTERGEN, which is 100% harmless and is mixed specifically to make your body naturally breathe faster, which is a good thing if smoke isn't really a problem and you just need to keep from suffocating. It also works mostly on rapid temperature change and chemically interfering with combustion(not by removing oxygen), and its weight is much closer to natural air, so it doesn't sink.
Thus, the server room doesn't have to be sealed(Halon and FM200 both require it), but the amount of extinguishing gas is greater and piping is slightly different.
In server rooms of sufficient size that use Halon, CO2, etc... SCBA units are usually recommended if not required. Interrupt switches and delays are very common as well, although both are designed mostly with reducing false discharges, which can cost $5-10k per 1,000 square feet.
More and more commonly, dry-pipe water systems are being used, particularly in 24x7x365 managed buildings. The idea is that if the fire is small, an alarm goes off and a countdown starts. On site staff can take appropriate action(cut power, run in with a fire extinguisher, that sort of thing) or, if it's bad, let the system come on. They're usually zoned triggered and fogging nozzles are used to minimize equipment damage.
Unfortunately, for your average 1,000 sq ft room, you're looking at about $30k to $50k, not including modifications to the room to get it sealed. Dry pipe water is less expensive, but one false alarm could cost you 10 times the system's cost in equipment alone.
Re:I've Seen Server Rooms that were Really Dangero (Score:5, Interesting)
Not sure how well they would work in a room full of smoke and halon, but I suppose you could run from station to station, and then somehow pry open the massive firedoor...
Re:I've Seen Server Rooms that were Really Dangero (Score:5, Interesting)
120VAC isn't too bad. Connections soaked in water might survive for a while until the corrosion finally breaks it down, melts the wirenuts, etc. Getting shocked by it isn't enough to blow your fingertips off. 240 volts is usually just 110 volts split into two phases, so it doesn't present any worse of a problem. 480 volts is another story...
480 volts gets interesting when the humidity rises and gets absorbed by the dust surrounding breakers and other switching components. Often, it will flash across the phases, vaporizing the debris, and mysteriously tripping the breaker. No one will figure this out until they happen to take a close look at the wiring, and the humidy from their breath will illuminate the brightest flash they have ever seen in their lives.
Re:I've Seen Server Rooms that were Really Dangero (Score:5, Interesting)
First, a little background info. It's an old building. The electrical is a mess of old and new circuits, some three phase 480 delta, some 240/120 single phase, and one major branch circuit even with a high leg. The former maintenence manager was of the mind to "get things working", rather than "get things right". After he was fired, I started helping out a little with the maintenence staff when emergency things came up, since I know a little about electronics, mostly to help them with computerized and digital control systems.
Anyway, from the street, there are 5000 amp fuses, huge suckers, then 1200 amp fuses on a few main branches. From there to a 1200 amp panel breaker for a major section of the plant (the one that the server room is on), along with most of the manufacturing. In that 1200 amp panel there is a 250 amp 3 phase breaker for the air conditioner. This is all 480v delta 3 phase.
Somehow, that air conditioner breaker failed. It vaporized part of the busbar, tripped the 1200 amp panel breaker, and blew the 1200 amp fuse for one of the phases, leave us down a phase. For the benefit of those who do not know, 3 phase motors running with one dropped phase tend to burn up... quickly! Ideally there is an thermal overload circuit to shut them down before that happens, but that doesn't always work. So bang.. the lights are off, and motors start to burn up in various places around the plant.
Once the maintenence staff figures out what is going on, that we are down a phase, they throw the mains on the service entrance panel for that 1200 amp branch. All seems to be good. We just need to replace that 1200 amp fuse and the faulty breaker right?
Heh. Well, it happened that we didn't have any spare 1200 amp fuses. A 1200 amp fuse isn't something you can run down to your hardware store and get. We send an employee to the next town where a store has exactly three of them in stock. $400 each. We tell him to buy all three. He comes back with one.
We replace the fuse, and the maintenence staff replaces the breaker. Upon reenergizing the circuit everything seems fine... until they go to put the protective metal cover back on.
The panel literally explodes. I wasn't in the room at the time thankfully, but the guys that were there say is was bright, loud, and scary. Apparently what happened was pieces of molten busbar had dripped near the bottom breaker in the panel and were close to shorting out the phases. The slight movement caused by putting the cover back on jarred the chunks of metal and shorted out the phases.
So... the 1200 amp panel breaker trips.... but not fast enough to save the upstream 1200 amp fuses near the service entrance. The ones we didn't have spares for. Again. And now it blew all three of them. And the store only had two in stock.
So we get back on the phone. We find another store that has two in stock, so we send an employee out to get all five, from both stores. He gets it right this time.
We finally replace the three fuses, triple check EVERYTHING, and throw all the breakers back on. We had sent all the employees home hours before... they couldn't do anything without power. But we are finally up... nearly 6 hours later.
Needless to say, some things have changed as a result of this, and it really underscored why the former maintenence manager was fired. We called the electrical engineering firm that had most recently surveyed our power systems, and had them run some more short circuit computer simulations, things like that.
Upon reading their report, I learn that our service panel has a ground fault interrupter, but it was turned all the way up to 1500 amps to prevent nusience trips, after it tripped several times due to our really bad "normal" phase imbalances.
Things are definitely improving, and we are much safer now then we ever were. It goes to show how one bad maintenence manager with a reign of terror, and a long tenure, can really screw things up though. I compare it to a programmer that never comments their code, and uses lots of goto statements, only the stakes are much higher.
Re:I've Seen Server Rooms that were Really Dangero (Score:5, Interesting)
One day I would find out why the 14.4KV fiberglass-epoxy reinforced fuses had mufflers installed on them. Remember that electrical current resists change. If the circuit breaks, the magnetic field surrounding the current collapses and increases the voltage until it goes *somewhere.* Each substation transformer was the magnetic equivalent to a ten foot tall capacitor. Well, if the fuse blows, the the remaining energy in the transformer's magnetic field immediately collapses (the magnetic equivalent of a ten foot tall capacitor) and detonates the fuse filament. This muffler vents this energy harmlessly into the substation as heat without blowing the panels off.
One day, when turning back on the power from vacation, we would find one of these fuses didn't have its muffler installed... And we would learn how things would *seem* to work on two phases.
We would try to install more fuses without the muffler on that phase. The magnetic field was strong enough to pop the fuse out of its holder when the switch was thrown. A wire tie solved the problem while parts were being ordered.
Moral of the story: if you work on high voltage equipment, always leave it as you found it.
Too funny. (Score:5, Funny)
Interesting side note: apparently finding high-priced Cisco gear not connected to anything is not that uncommon. I've also heard horror stories of guy that traced a cable my hand(toner was on the fritz) that looped 4 times around the data center but wasn't hooked up to shit, on either end. Took him an entire afternoon.
Re:Too funny. (Score:5, Funny)
Re:Too funny. (Score:5, Funny)
The Magic Switch [tuxedo.org]
I can't remember the author (GLS) but if you google you'll probably find a more original version.
Re:Too funny. (Score:5, Interesting)
I remember that - it got sealed into a wall. [techweb.com]
BTW, the author of the magic switch story is Guy Steele.
Re:Too funny. (Score:5, Funny)
I was asked to clean up that half of the lab one day, so I unplug one of the boxes. After about 15 minutes, a buzzing alarm goes off. Soon an IT guy comes through a back door that I had never seen opened, and plugs them back in.
I still dont know what those things are.
The original Google storage server (Score:5, Funny)
This is by far to custom case I've ever seen.
Look a bit dangerous though
Re:The original Google storage server (Score:5, Interesting)
Okay, I looked at it. Heck, for the sake of argument, let's say that I'll believe Google started business made out of Legos [lego.com].
Can you tell me why the heck there are always [stanford.edu] blurs of one type [stanford.edu] or another [stanford.edu] shielding the front of the box from view?
So- why have stuff super-neat and tidy? (Score:5, Insightful)
Documenting connections has a real payoff in troubleshooting. But doing stuff on aesthetic grounds is a harder sell. I have a gut sense that a clean layout is important even if you know the destination both ends of a wire whose middle runs through a snarl. Here's what I came up with:
my version of the community policing broken windows theory.
It's psychologically harder to do slipshod, shoddy work if everything around you has been done well. And it's hard to do a proper job if everything else is slipshod. As a matter of housemate politics, it's easier to leave the nth dirty dish in the sink than the first. You are only adding an increment, not changing state.
Doing the Right Thing is contagious. At least, it is among folks I care to work with. Doing the Wrong Thing is catching, too. Morale is higher and people challenge themselves more at a shop that is run well.
That's how I pitched it, and my boss bought it.
Re:So- why have stuff super-neat and tidy? (Score:4, Insightful)
This is not always true!
If you have a company that you are trying to get new customers and investors to come on board - and you need to show them your technology, often times they like to come over and take tours of your datacenter - HQ - IT Dept or office in general.
Believe me - the asthetics of the cables that make your network have a very large impact on how people can view your company.
For example, this picture [theregister.co.uk] shows that this particular setup went up in flames at some point (look at the burn marks on the door and melted cables). now you think that if this closet was setup properly (which would include asthetics) that it would have gone up in flames so easily. And do you think that any client/partner/investor would think you had your priorities straight if they saw a closet like this?
Look out! (Score:4, Funny)
That's nothing. (Score:5, Funny)
Our server room has wobbly floors and tower cases stacked 2 high sitting in the middle of a 16'x16' room, with the more important servers sitting on the top.
We also got a new air conditioner that has an electronic switch and we have problems with brown outs. So in the middle of the summer, the power goes out and the A/C doesn't come back on, usually on a weekend too.
One more thing: brace yourself:
We use Windows servers with IIS!!!!!!!!
Working at an ISP... (Score:3, Funny)
Basement Server Rooms. (Score:5, Interesting)
Re:Basement Server Rooms. (Score:4, Funny)
I'll take a blonde, 6 feet tall, please. Make her lippy, but only to a point. Oh, and she has to know how to cook. Nothing special, just the basics. Thanks.
Suffocation (Score:5, Interesting)
Anyway, one day we were working in our cage, when we heard a warning alarm, and saw all the employees running for their lives. Not knowing that the alarm meant the Halon system was about to be activiate, we joined in anyway and ran for the emergency exits.
It turned out the fire alarms were set off by accident by someone drilling and creating dust, and luckily the people on-site disabled the fire supression system before it went off.
*sigh* It doesn't KILL you, it kills OZONE... (Score:5, Informative)
Look -- geeks sue for the wrong kind of keyboard -- ergonomics = safety after all -- do you really think there is just this dangerous gas ready to be released where you work and you have exactly 20 seconds to escape before you die?
Taken from the first site that came up on Google... [drivewerks.com]
"Three things must come together at the same time to start a fire. The first ingredient is fuel (anything that can burn), the second is oxygen (normal breathing air is ample) and the last is an ignition source (high heat can cause a fire even without a spark or open flame). Traditionally, to stop a fire you need to remove one side of the triangle - the ignition, the fuel or the oxygen. Halon adds a fourth dimension to fire fighting - breaking the chain reaction. It stops the fuel, the ignition and the oxygen from dancing together by chemically reacting with them. Many people believe that Halon displaces the air out of the area it is dispensed in. Wrong! Even for the toughest hazards, less than an 8% concentration by volume is required. There is still plenty of air to use in the evacuation process. (Emphasis added by me)
I'm not an expert, but urban myths bug me...
Photo of my old ISP's server-room-bathroom combo (Score:5, Funny)
Re:Photo of my old ISP's server-room-bathroom comb (Score:5, Funny)
Re:Photo of my old ISP's server-room-bathroom comb (Score:3, Funny)
I just knew there was a punchline there that I was totally missing out on. Damn. Thanks for that. :)
Re:Photo of my old ISP's server-room-bathroom comb (Score:5, Funny)
oh no, the logs are getting backed up!
The mess in that first picture... (Score:4, Funny)
Zip-Ties Were our Enterprise Power Solution (Score:5, Interesting)
One day we had a broom leaning up against the wall next to two of our cabinet's. Someone bumped the broom, which fell in the long arc that brooms do when they fall along a wall when leaning. One the way, it happened to unplug our two cabinets from the wall. So much for uptime. The place when quiet and we all just stared at each other for a few seconds. This is in an envirnment where downtime isn't really tolerated at all.
Our task the next weekend?
We took a whole package of zip ties, loosened up the plug wall plate, zip tied the plugs around the back of the outlet wall plate with an ungodly amount of zip ties, and screwed the wall plate tight again.
Our version of 120volt twist locks.
Was interesting to hear what people would ask after seeing it for the first time.
Not quite the server room from hell, but the story's on topic.
-Pete
Re:Zip-Ties Were our Enterprise Power Solution (Score:4, Funny)
Re:Zip-Ties Were our Enterprise Power Solution (Score:4, Interesting)
The fact is (Score:3, Funny)
I'm messy by nature, but I know that as long as I'm the only person who understands the mess, I'm indispensable
And I got a new job yesterday... I pity my successor
I knew it was dangerous.... (Score:4, Funny)
The Reg's 'Scariest Server Room' ever (Score:4, Funny)
That doesn't surprise me. I have a strange feeling many of you have been caught doing this [havokmon.com] in front of your pc.
So much for my Delta Force keyboard layout ;)
Server room? Luxury! (Score:5, Funny)
Of course, when we did, eventually, move it into a server room, the aircon subsequently broke down and, being an underfunded dotcom, nobody wanted to spring for repairs. We lost at least one server that way (thankfully not a live-facing one).
Problem is... (Score:4, Insightful)
The problem is, once the thing gets into that kind of mess, you rarely have the chance to bring down the entire network to repatch all the cables and cable tie them into some kind of order.
Not only that, but if you have loads of trunks and VLAN's configured, putting it all back in the right order can be a total ballache!
Earthquakes and server rooms (Score:5, Interesting)
Has anyone else been in a non-earthquake-prone place and then had an earthquake? Here at work our server room was completely unprepared for an earthquake. Some of the machines came off the racks, some of the whole racks fell, our T1s got damaged or disconnected somewhere in the process. The whole disaster showed us how stupid some users can be. First, the T1 provider calls from Boston or something, "Duh, we show that your T1 lines are down, blah blah blah..", "Uh
Anyway, we bolted down all the racks, shame on us for not doing it in the first place.
Airconditioning freezing cold? (Score:3, Interesting)
Forget it. We had a problem with the airconditioning at a medium size company in Delft which had it's heat-outlets on the top of the roof.
These outlets were not so well protected to cold as was shown two years ago when, after a freezingly cold weekend we came into the server room and it was really boiling hot. Problem was the huge ventilators on the roof were stuck frozen.
It was the only time we had all windows and doors open in the middle of winter. But this cold could well have started a fire.
Reminds me of... (Score:4, Interesting)
I get a call from the folks on the 5th floor saying there's water coming out from under the door (they don't have access to the room). Being the onsite telephony person, I rush up wondering what the hell is going on. Sure enough, water has soaked the carpet around the door. Opening the door I see as much as 5cm (2in) of water sitting around the base of the switch and various servers connected to the switch. All in all, probably around $300K worth of equipment, and I don't dare go in, because there are power cords lying in the water. Finally get the power turned off, the water out, clean everything up. All in all, costed us a couple hundred dollars for some new cables, one monitor, and various odds and ends.
Apparently what happened was someone on the next floor up was in a bathroom, turned on a faucet and forgot about it. The water managed to move about 5m (15ft) down the hall before deciding to pour out into our telephony room!
My house. (Score:4, Informative)
I put them up there as I got sick of the noise, then ran ethernet cabling down to sockets around the place. My main web server (see sig) has been happily running up there for about 1 1/2 years now - barring the odd upgrade. It's not pleasant to open up a case full of dust and cobwebs, BTW!
Re:My house. (Score:3, Funny)
BAH! (Score:5, Funny)
I've got a set of routers located in a crawlspace where the only way to get to them is to walk across boards spanning small metal beams that were put in to hold a suspended false ceiling. One missed step and you'll drop right through the ceiling, AND IT'S A 2 STORY DROP! Once I dropped a power pack while replacing it and nearly killed a gal working below. Power pack exploded like a bomb when it hit.
We recently had a "security audit" where they recommended we should mount those routers in a locked cabinet for increased security. Not a mention about the boards, lack of handrail, safety net, etc. Heck, who needs a locked cabinet? Just remove one of the boards and NO ONE can get to those routers, not even the people who are supposed to maintain them!
Back when we used thinnet one of the managers didn't like stringing new coax through the building whenever we remodeled or moved people, so he had us cut all the coaxes to length PLUS 25 FEET! He figured if someone moved we could pull back the excess and save time. The cables all terminated in what came to be called the spaghetti room, from the coils of coax all over the floor. We had to step over all the coaxes to get to the routers and hubs. Eventually, the coaxes got damaged from all the abuse and had to be cut off to length anyway, but for several years it was a serious tripping hazard for anyone who entered that room.
Re:BAH! (Score:5, Funny)
Fond memories of chepernet and mud (Score:5, Funny)
Lying on the ground, underneath the leaky tarp, hoping that I did not get electrocuted, or if I did that I would not be held accountable for the damaged equipment [trust me, this was not my idea], I decided that re-enlistment was not a great idea.
[former] USMC geek
Those are severe travesties (Score:3, Informative)
Every seen one in a shower? (Score:5, Funny)
I once visited a client who had his server racks in an old lockerroom shower. This would not have been so bad except when one of my co-workers leaned against the wall and hit the valve we discovered that the pipes hadn't been capped by just had the shower heads removed....that's right three full racks of equipment in a live shower. =)
The security guard from hell... (Score:4, Funny)
An HP technician (yup they have at least one) was restoring the data to a customer's fileserver but the backup software was asking for tape#2. The customer only ever had one backup tape that they recalled, so they were quite perplexed until the security guard entered the server room...
Apparently every morning around 3am when he made his rounds he found the backup server screen blinking "insert next tape" -- The security guard proudly said that he was pushing in the tape for at least six months now...
Some more scenarios... (Score:5, Interesting)
One company didn't order a rack mount kit for their KVM switch (some Belkin model), so they duct taped it to the main monitor. No subtle tape loops under the KVM..... they wrapped the tape three or four times around the KVM and the monitor.
Another company was remodeling their server room but neglected to move the servers somewhere else. There was an inch and a half of drywall and sawdust on top of all the network equipment and servers. The circuit boards looked like it had snowed on them.
I'm doing an audit on some systems. I see a motherboard sitting in a cardboard tray (the kind you get when you purchase a 24-pack of Coke from Costco), along with a hard drive, floppy, power supply and network card. No case. No cooling. Turns out it was their PDC and print server. That's quality craftsmanship.
This isn't about server rooms, per-se, but I did some work for a national pizza chain. They had modems at a central site that were supposed to make a phone call to the stores to print out order tickets. We were sent to figure out why they weren't printing. At one site, the printer was on the floor next to the prep counter where they add the toppings. Someone had spilled a good quart of marinara sauce into the printer. They gave the outside of the printer a good once over, but the inside was just nasty.
We were sent out to troubleshoot a voice-over-IP problem at a garden nursery. We arrive on site and lo and behold, there was a dead rat on top of the router. It didn't have anything to do with the problem, but it sure was unexpected.
I love when people don't properly plan their electrical power consumption in their server rooms. I walked into some company's server room, plugged in my laptop to the rack mounted power strip, turned it on, and blew the breaker for two racks of servers.
I watched a wireless network installer gob Liquid Nails onto the back of an Aironet access point and stick it to the ceiling. I hope they never want to upgrade that particular access point.
Any other good stories?
Re:Some more scenarios... (Score:5, Funny)
I dunno, it sounds like they planned their power consumption PERFECTLY.
Re:Some more scenarios... (Score:5, Funny)
"Um, where's your print server?"
"We don't have one."
"Yeah, you do, all of your machines are talking to it, it's here somewhere."
"I've been here seven years, and we don't have a server."
I traced the cables into a closet. That's blocked off by a workstation/desk. After some convincing, I managed to get them to let me move the desk, and I got into the closet. Where I found a 1987-vintage Mac II, happily munching along as a print server. Hooked into an old phone company-style UPS. Covered in a solid inch of dust and debris. And running without anyone noticing it for at least seven years...
Hmm. (Score:3, Funny)
I remember working at an isp way back when and the server room was so bad that you could basically lean on the rats nest of wiring like it was a makeshift hammock.
Walking around behind the racks meant being completely aware of which line you put tension on, lest it knock some connector not fastened down out of place.
Regular telephone wires... (Score:5, Interesting)
A couple of weeks ago I came accross a couple of photograps of actual telephone exchanges in the streets of Beirut. You just wouldn't believe it, and it took me a few seconds to understand the picture. there were so many wires that you can hardly see the box behind -- kinda like Johnny Mnemonic, except with 10x more wires, and 2 or 3 handsets plugged in seemingly random (or probably not) outlets. I'll post again if I can find it back.
Radar (Score:5, Interesting)
The situation: The client's systems are crashing, on a regular basis, for no understandable reason. No remote diagnostics work, so they send out my friend.
He gets to the server room, and keeps thinking he's seeing things out the corner of his eye. He tells everyone to leave the room, and turns out the lights. The room glows.
The server room at this place was sitting under a huge radar system. (He had some additional explanation -- used to be a physics major -- but I didn't entirely follow it). They moved the equipment (a substantial undertaking!) and the problem went away.
Re:Radar (Score:4, Insightful)
Most likely, when he flipped off the lights... they stayed on; the high-powered, high-frequency RF inducted into the wiring could be enough to keep a fluorescent lit, just like at a Tesla coil demonstration.
CRTs and neon bulbs in equipment would no doubt show the same effect... and of course, if you have enough current in the air to light a fluorescent tube, think of what that does to your system and communications buses!
Glows in the Computer room (Score:5, Funny)
This goes back to the sixties, when computers had twitching reels of mag tape and paper tape was king.
The company had a regular overnight run. A control tape was put into the high speed tape reader, all the relevant mag tapes mounted, and the computer got on with its six hour job (about 20 second job by todays standards). Originally there was an operator on duty, but he blatantly had nothing to do, so they decided they could do without him.
But as soon as the operator disappeared, the job started failing at dead of night.
OK, bring back the operator - he can fix the problem and restart the phase which went wrong.
But, as soon as the operator came back, the problem went away. And this was the pattern - if they watched the system, it worked perfectly. But left alone, it invariably failed.
So an engineer decided to sit there and not touch anything. He told the operator to go away, as if he was't there. Which he did, turning the lights out and leaving our hero in the dark - except for the glow of the high-speed tape reader, which shines a strong light through the holes in the punched tape onto photocells. And as he watches, a moth appears and flies through the pool of light, confusing the tape reader and aborting the job.
Re:Glows in the Computer room (Score:5, Funny)
Sad (Score:5, Insightful)
I know that when i moved into one company in Redwood City - the network wsa a nightmare. We had rooms that looke like that - but over the next 2 years we replaced almost every wire on that network - and demanded budget for proper closet setups - and got it.
We eliminated all those closets that looked like that, and learned one hell of a lot in the process.
I think that if your closets look like that - you are asking for fire - and it shows just how lazy you really are. No arguments of "I'm too busy" allowed - it just means your a lazy slob period.
Re:Sad (Score:4, Insightful)
Just because somebody has wiring from hell and jury rigged systems doesn't always mean they are lazy SOB's
Sooo... (Score:3, Funny)
Yeah - you know you want to.
SW too. (Score:3, Insightful)
Chicks did danger... (Score:3, Funny)
Hot Chick: Yeah, I guess so.
IT nerd in bar: Sometimes I take my pocket protector out of my pocket!
Hot Chick: Uh-huh...
IT nerd in bar: Sometimes I take my mint condition Megatron action figure to LAN parties!
Hot Chick: Uh-huh...
IT nerd in bar: And sometimes I let my server room get really messy so that it's a hazard to my life!
Hot Chick: Wow, that is dangerous. I'm really turned on...hop on me right now...
The Big Red Button... (Score:5, Funny)
At my old university, one of these server rooms was emptied as new, smaller hardware came available and the room was no longer needed. They turned this room into an office for a student organization... leaving the large red button, but taking the "Emergency Shutoff" sign.
This unlabeled button sat neglected on the wall of this little office for about 7 or 8 years until one day a curious student just had to find out what the button did...
The network for all of the engineering schools at this university of 36,000 students went down for most of a day..
The best part is that the button is still in the little office with the students, and it is still unlabeled yet fully functional... They did hide it behind a file cabinet, though
Re:The Big Red Button... (Score:4, Funny)
something you aren't sharing?
Pullin wire is more dangerous than any server room (Score:5, Interesting)
I worked at mothers cookies in oakland CA for a spell. You think malaysia is dangerous?
I would pull off the 880 by the colliseum,mc arthur blvd? It takes you through a very derilict section of oakland factories.
The entire complex is surrounded by bobwire. If you drive down the street, less than a block, you are in one of the worst neighborhoods in oakland. I had a friend who lived in the neighborhood, he wasn't shittin me either about the danger. I went to his house twice to hang out and that was enough for me.
The server room was cool, about 40degrees all the time so you wore a jacket when you went in there. But pullin wire....Ohhh my god!
I had to run a fiber line from the main building to some office in the back of the bakerery. Now before you get the picture of little mothers running around with cookie sheets and kenmoore ovens you have to understand.... That is not what a huge production cookie plant is.
Imaging a HUGE fricken warehouse with conveyer belts of cookies going everywhere, machinery whiiring and cookies going into boxes and filled with creme and those animal cookies with the dots, all in this HUGE room about the size of a football field.
One end was the mixing end, where they had these mixing machines the size of my garage. Into those would go 50 gallon oil drums of butter, lard. Huge bags of flour being loaded by forklift, ect.
Now at the time, wireless hadn't really made it mainstream. So my mangers convienced mothers fiber would be good since it would provide the best ROI. They were sold and I was sent out to work.
Now the factory was built from steel girders covered with that tin roofing, the stuff that looks like a ruffles potato chip. I got up there to where the top girders are and before my eyes was the most treachorous wire run I ever saw.
Remember what I said about 50 gallon oil drums of lard? Well, when the cookies baked, the lard would vaporize and rise to the ceiling then settle on the steel girders. Over the years a 1/4" layer of lard had deposited 70 feet up in the rafters.
I put my finger in the goop to see how slippery it was.
No friction.
I called the office and told my boss. Later he called one of his friends to subcontract the work out too.
*Disclaimer* Despite the lard, mothers cookies makes a great product, and was an awesome place to work. If you ever get the chance to work there, jump on it, you won't regret it (or the 50cents a bag price for employees
Wiring closets (Score:4, Insightful)
Knock-knock... (Score:5, Funny)
I had to remember to knock before rebooting.
Well, back when I was a kid... (Score:4, Funny)
You kids these days just have it too fscking soft, I tell you...
All you need is one of these: (Score:4, Funny)
This happened a few years ago (Score:4, Funny)
At the old company that I used to work for. They had no server room yet so the servers just sat in the corner. One day I was working late and the cleaning staff came in. The first thing that they did was to plug the vaccuum cleaner into the UPS that the main server was hooked up to.
It seems they had always been doing that!
holes in raised floor tiles (Score:5, Funny)
When our old NEC mainframe came out we had a few tiles with 18"x10" holes formerly used for cabling in our raised floor. We still don't have exactly enough tiles, but furniture is arranged better now, a year later, so that these holes are strategically covered by desks, shelves or other equipment.
Better than accidently wheeling your chair over that duct tape patch! I kid you not.
Dog Kibbles and Computer Mice (Score:5, Funny)
Back in '86 I had a top-of-the-line Corona IBM-PC clone (cost me nearly $5,000 then). It had those big full-height floppy drives (two!) and was a very well-built, sturdy unit.
I was working as a computer hardware technician at the time, and I had recently bought a bunch of 256k memory chips. I brought my computer to work to show it off to the guys, and also install the memory where I had a nice anti-static station.
So there I was with all my buddies, showing off my toy. I open the case of my computer, ready to wow them, and at least a pound of dog kibbles spills out of the case. Dog kibbles are strewn all over the computer motherboard. We all kind of stood there for a moment, dumbfounded.
Eventually, I discovered the cause. My house was infested with mice, this I had known. But what I didn't know was that, in the middle of the night, mice will steal dog kibbles from the dog dish, and hide them in little places they can get to later. Apparently, they had been climbing in through the full-height floppy drives and storing the kibbles.
Interestingly, it never seemed to affect the computer!
attn: my cow-orkers (Score:5, Funny)
You have been warned.
(we're in the middle of a rebuild, so it's major chaos before restoration to order)
Phillip Morris (Score:5, Funny)
Re:Phillip Morris (Score:4, Funny)
HP was so happy to have the account that they didn't bitch at all each time they had to replace a disk drive every 3 weeks or so...