Transitioning From Small Shop IT To Enterprise?

Imaginary Friendly asks: "I'm the 3rd guy in a three-person IT firm. We're good and we're expanding. Our clients range from three computers to 30, with our largest client having six servers. We can handle the work but, thanks to my efforts and love (or just luck), I may be signing up two new clients who have 200 networked computers each. We're spread thin as it is, and hiring competent IT staff has been difficult. We're now doing 60-hour work weeks, so re-education has remained passive. How do we transition from manual rebuilds and CD deployments, to full scale (proper) IT administration?"

drink the koolaid

[Gothmolly]

Since 90+% of your clients will be MSFT shops, drink the koolaid. Get an MSDN subscription. Buy a big multicore machine, and you can all learn on VMs. Master all the MSFT (RIS, SMS, etc) and bolt-on tools (NetPro, GPA, etc). Learn to do everything the MSFT way, and you won't starve (at least until Linux takes over, if and when). In your spare time, look at alternative desktop environments for tip and tricks.

First...

[misleb]

I would seriously recommend turning away those bigger clients for now until you first get the staffing to handle it. You can try to pick up the clients later. Maybe they'll hire someone else, but there is a chance that they will be unhappy with that someone else and come to you. You don't want to take the clients on now and screw up and ruin your reputation. If at all possible, hire someone (or multiple people) who already have experience with larger networks and kill two birds with one stone. I don't think it is really worth it to give you advice here on how to manage larger networks. You've got staffing problems. You need to address that first.

-matthew

Re:drink the koolaid

[Anonymous Coward]

Going the Microsoft way isn't a bad thing. Half assing the Microsoft way is what gets people fucked.
For example, installing an SBS server and then wanting two more back up domain controllers (say, one at a data center and one backup locally).
If you go the Microsoft path -- then go the Microsoft path.

If you go the Linux path, expect some initial pain in the learning process. Linux still isn't easy enough to clickety click for all the networking stuff yet -- but after all the tweaky shit, it will work.

You can do a hybrid, just be smart about it. It's VERY hard NOT finding OS fundies around. Whether Microsoft or Linux. These will be a trap for you. Keep as many tools in your toolbox as possible -- otherwise everything with your hammer will look like a nail. Not being biased is hard, but it will save you working more later.

Oh, and get an Action Pak. They are cheap enough.

Now, as for his question on how to transition: VERY SLOWLY. This is the part that can KILL a company. If you grow too fast, then you increase the possiblity of hiring ass holes. Hiring ass holes is bad because it pisses your customers off. While it may save you a few cents here and there (which, make no mistake, those cents add up) -- those customers pay you more than cents. Lose the, lose your business.

Service Management

[_Hellfire_]

On the non-technical side of things, formal Service Management is a must. If you haven't already, I would strongly recommend formalising the management of incidents, changes, requests etc. with something like ITIL [wikipedia.org]. Without formal change management you'll get breakages caused by change and unhappy customers. Recording incidents (every incident) allows you to build up a picture of where your pain points are with each customer, makes it easy for billing, and if you get the same incident again, and you can look at the resolution of the previous incident for a head start in solving the current one.

You mentioned manual rebuilds etc. It would pay to automate this as much as possible (I'm sure you'll get some responses on this). Quality can often be equated with consistency. If you give your customers the same thing over and over they will know what to expect, even if it's only 80% of everything they need. They'll be much happier in the long run than if you give them brilliant service one day and crap service the next.

Plan

[Amouth]

I moved from a small pc shop to a larger company with about 50 people.. i am by my self.. but we are spread out alot.

what you need to is to plann everything. train people that the locations to handel minor things and make them a fire fighting team.. no company is going to complain that you train their people to handel the minor issues so that they don't have to call you. try to make everything in rounds.. if problems can wait let them untill the guys schedualed to come by can get there and have his list and go about his job.

with a good work order system you can plan for the jobs and have job kits for your workers.. a check list ... the simple stuff makes all the diffrence.

and if you can put this in place then hiring people to do the work is alot easier as they don't all have to be experts.

also set up remote admin and monitoring.. companies might fight back alittle but make it fit their policies.. because if you can see a problem and fix it before they notice that is a good thing. also if it is something that could be done remotely you don't have to send people out there..

and for the multi server people a single port KVM over IP connected to a normal KVM rocks.. they arn't cheap but if you are making money from them droping the 500$ for a single port KVM over IP isnt' that bad.. also you can get them with modems so you can dial into them.. makes remote admin easy.

make network maps and keep them where everyone can get to them so that you don't have jsut one guy that can work at a specific place because he is the only one that knows how it is done

just some ideas.. but always plan..

Re:Hiring the Competent

[Anonymous Coward]

Many of the most competent don't have one; change your hiring strategy

Spoken like someone who doesn't have a CS degree. I prefer someone who has a degree (any BA/BS is fine, I knew excellent IT folks with English & History majors) over someone who doesn't, but its not an automatic exclusion. But my experience says on average they guy with the degree is a better employee than the guy without the degree. Self taught experts and paper MCSE's rarely have the depth of knowledge and ability to step outside the problem; its Linux r0x3rs!, replace all you Oracle servers w/ MySQL, and how do I mount an NFS share? I've never worked in a multi-user environment.

But I'm sure you will explain why Linux is Da Bomb, MySQL can do everything Oracle can, and how your limited experience outshines my 30 years of working with computers and 3 advanced degrees

Re:First...

[p0ss]

i agree, your staffing issues are your primary concern, without additional staff you are not going to be able to expand at all. Finding staff is hard, but as discussed in a recent slashdot article, the key is paying well. If there is competition for skilled people, the company who is paying the most will win.

If you can use telecommuters, do, it will broaden your labour market immensly.

One last point, Train your customers as you go. It may seem counter intuitive, but teaching your customers to solve or avoid basic problems themselves will leave you free to handle more complex problems and will increase your value and reputation. one hour spent teaching will save days of repeat visits.

Re:drink the koolaid

[Anonymous Coward]

I'm sure there are but not as easy to use. This is a very big generalization but you can maintain a complete Microsoft system (all servers/all desktops/messaging/file serving/authentication/AD etc) by being a jack of all trades and master of none. It may not be the most efficient or fastest LAN in the world but it will run and meet the business needs require a minimal staff to maintain it. Using messaging for an example. I'd be willing to bet that someone already maintaining Windows servers would be able to setup a working clustered Exchange server with SAN connectivity (they may need help with the SAN disk setup and access depending on experience) with nothing more then a few phone calls to a consultant and some searching of the MSDN site. Take a Linux guy that has never maintained sendmail or equivalent and see how long it takes him/her to built a two server redundant sendmail/Imap/pop system. Take it a step further and add and test a functional and easy to use backup and restore system for that messaging system. To get the same things in Linux typically requires people with more experience in particular areas. For a small or medium business, they do not want to hire more people so the jack of all trades seems to be a good balance. For medium to large businesses, the extra system administrators are needed anyway so having a sendmail god that also does samba and can do basic or semi advanced Apache work is a really good choice. Small businesses do not need that level of expertise and can get by without it. The jack of all trades can call someone more technical when the need comes up. The more experienced sendmail guy can probably fix it himself with little effort as well but he is not fixing the Apache server or Samba with the same ease. An example, I know several 200-300 PC businesses with 5-10 servers that are completely maintained with no more then 2 people. This includes OS upgrades, frequent software deployments, equipment swap outs all in both server and desktop etc.. I'm sure there is a Linux guy maintaining 1000's of desktops around the world as well but those are more then likely something very static like POS systems, remote terminal etc. Not a computer that dumb ass users are sitting in front of and browsing the web, playing music on, and carrying around and giving presentations on LCD panels either. The windows world has many MS and third party tools for rapid deployment and patching that take little effort to get up and running. Again, my comments are VERY general. I am a dual system administrator myself and I deal with both systems equally.

Establish Procedures, hire someone who knows them

[StewedSquirrel]

Hire someone who knows corporate IT.

Please.

I've dealt with too many "three man IT shops" who treat IT work like auto mechanics. "try it, tweak something, try it again, tweak something, try it again, tweak something, try it again, tweak something". All the while, the company is offline. Corporate IT is about establishing procedures BEFORE the issues happen and about having backup plans for WHEN they happen, all of which is designed to minimize downtime.

Working with an office of 2-3 people... if you're diddling with their router for 2 hours, your time is probably worth more than the time the company has lost. But if you diddle with 200 people's connection for 2 hours, you've just cost their company $20,000, possibly more. Imagine what sort of investments could have prevented that downtime, how much cheaper they are than that downtime and why you should have implimented them :-)

FYI, Documentation is more important than you think.

Stew

"good help is so hard to find"

[SuperBanana]

Businesses love to complain about how hard it is to find employees when they're being cheap on labor, or how they can't retain good help.

There's no talent problem; there's a "how the IT industry treats workers" problem. Here's the current IT talent pool "problem", as I see it:

• The IT industry is one of the few industries that seems almost completely unwilling to recognize general skill/talent, and expects to hire someone who they can drop in and have productive in a matter of hours. It doesn't work that way- in the IT industry or any other industry. Every new employee needs training and familiarization, every new hire causes lost productivity. GET OVER IT. There are industries where corporations send workers to a WEEK OR MORE of training before they've "worked" a single hour.
• Loads of IT workers were encouraged to drop out of / skip college because their technical skills were all they supposedly cared about. Now it's "degree or don't apply." So much for technical skill.
• Employers and the industry are doing nothing to make training/certification easy or inexpensive. Redhat certification, for example, costs thousands of dollars- out of reach of most job seekers. Furthermore, loads of employers are refusing to invest in their workerforce (continuing education/training) and/or treat them like shit. They're then shocked when said employee's performance drops and they get fired/"laid off"
• Employers are abusing "temp to perm" and "temp" positions, cheating the unemployment and benefits systems and tricking workers into thinking that, if they're good little drones, they'll get the job at the end of 3 months- when in reality, the company will show them the door with a silly little excuse.
• HR departments use all sorts of fancy technology to effectively dump your resume in the trash can without a single eyeball seeing it, after cheerfully sending you a "thanks for sending in your resume!" letter.
• Employers post insane requirements looking for people with a skillset that goes on for PAGES and have grossly unrealistic expectations for years-of-experience. For even the most mundane schlep, I mean help, desk positions. Candidates respond by simply flooding employers with any position the candidate thinks they might be remotely qualified for.

Is it any wonder that IT staff leave the industry in droves after just a few years?

Re:promotion

[itwerx]

outsource your work and call yourself the VP of marketing and operations.

This is better advice than the poster may have intended. With your current size you really don't want to add FTE's just for two clients. Use that good old "people network" and see who you can shanghai on board temporarily (with an eye towards possibly making them FTE down the road). Otherwise you're investing heavily on what is essentially a gamble at this point.

Read This Book! Live This Book!

[DaGoodBoy]

The Practice of System and Network Administration [amazon.com] by Thomas A. Limoncelli and Christine Hogan is the definitive reference to build, and more importantly, maintain any network and system infrastructure. It is written in an accessible style with plenty of real-world examples that focus on the importance of key infrastructure. It is not a "How To" book exactly, rather it offers advice and specifications for the kind of support infrastructure you have to build to be successful supporting large system and network infrastructure. If you are familiar with this book, please add your comments on it.