Comment Setting the story straight... (Score 1) 476
It's unfortunate that reporters such as this guy would sensationalize a talk by carefully crafting his story from bits and pieces mostly taken out of context. So, in all fairness to my firm and to those who were not present, I feel compelled to set the story straight.
First off, the story is not an interview even though it may come across as such. The title is rather sensational but I certainly wasn't desperate. There were problems and they were fixed and our team was just very resourceful in doing so.
Gedda writes:
> IT managers who want to deploy an open source solution but are worried
> about company politics should go ahead and do it without asking,
> according to PricewaterhouseCoopers (PWC) Japan IT manager Mark Uemura.
No, this is taken out of context. What I said was that we had very big and important changes that we needed to make in order to restore network and application stability. My reference to just going ahead and doing it referred to making the necessary changes behind the scenes. It wasn't about company politics and it wasn't about migrating services from Windows to OpenBSD. My experience was that we did ourselves a disfavour by trying to inform and explain to users and management the technical reasons for the changes that needed to be made. In fact, all of the pushback had nothing to do with OpenBSD. We needed to migrate from an old Domain Controller with a corrupt Active Directory to a new one. We also introduced the concept of working on Application Servers in Terminal Services to take advantage of server power for resource intensive applications that ran very slowly on users' PCs. So, the push back was related to things like "you'll have to login to this new Domain rather than the old one from tomorrow onwards." or getting users to change the way they work and use applications running on a Terminal Servers for speed. In the end, when all was sad and done, users and management realized the difference that we had made; no more downtime or data loss. Furthermore, they've never had everything running so smoothly and as efficiently for as long as they could remember. Their IT problems went away as a result of our efforts and the decisions that we made.
In fact, all of the migrations to OpenBSD were either behind the scenes where the users were oblivious to the changes. Well, almost oblivious. Often times we would get "Hey, the Internet is really fast today, cool!" or "Man, can you guys like spill some coffee in the server room or something? We're not used to this much uptime. It means we can't go home early anymore!"
In those cases where users did have to interact with OpenBSD, it was always well received and positive such as moving off of a very slow VPN for remote access on to a quicker and more user friendly alternative such as port forwarding applications through OpenSSH.
> Faced with an unreliable network, Uemura went ahead and migrated systems
> from Windows to OpenBSD on the premise that management would trust his
> judgement.
Once again, migrating services to OpenBSD was not an issue. So long as we did not compromise security in doing so. Generally, we did so to improve security and that's what OpenBSD is famous for and yet there's so much more.
> "PricewaterhouseCoopers is a Windows shop but we were forced to use open
> source," he said. "I inherited a real nightmare with servers going up
> and down. There were e-mail outages and on top of that there was a bad
> relationship between our users and IT."
Well it's either replace Windows with Window for Internet facing servers or find a more secure alternative that didn't have to be patched and rebooted so often. Bringing back network and application stability was important to the business as much as increasing security wherever it was possible to do so. I feel that stability is a result of good security.
We concentrated on network perimeter security. Hence anything that was public facing was considered so long as it satisfied four main criteria:
1) If security was a concern, then we used a more secure alternative to Windows.
2) If cost was a factor either for software licenses, service/support contracts or hardware, then we considered the Open Source alternatives.
3) If stability and uptime was important, then this was taken into account.
4) If all three points above qualified, then the last question to be answered before replacing any Windows based application or service was the following. Will there be any interoperability issues? That is, will there be any downside to replacing Windows and implementing a more secure, stable, cheaper Open Source alternative? If the answer to the last question was "NO", then we used Open Source when appropriate.
Once again, we were really concerned with any Internet facing servers.
> "My predecessor spent too much [so] I was told not to spend any money."
We could have begged for new hardware but it wasn't necessary. I knew that we needed to make big changes that required applications to be migrated from hardware to hardware. It is true that management told us not to spend any more money than was absolutely needed. This is just good business sense and a good rule of thumb to follow for any company big or small. If we were given the opportunity to spend on hardware, we would have had twice the server power that was really needed for our office in the end.
> When asked what argument he used to convince management to use an open
> source solution, Uemura said: "They didn't have an argument because they
> said don't spend any money." "They trusted me," he said. "The whole
> office was relying on one domain controller which was dying."
Again, we are not talking about the migration from Windows to OpenBSD. The journalist is really good at combining different parts of my talk and the answers to questions following it in order make his story that much more sensational. This news story is a great example proving that you shouldn't believe everything that you read. At face value, it's very misleading.
> Uemura said a lot of work was done "behind the scenes".
My team did a lot of work behind the scenes for which I am grateful. I didn't do it alone.
> "My experience is that if something has to be done, just do it - don't
> ask! They will thank you later," he said.
If you give users a choice, change or no change, they'll tend to favour the status quo.
> commercial lock-in.
There are many companies that have clued into this however many large financial institutions still have big support contracts with Open Source Vendors liken to a kid's security blanket where they just don't want to let go. I'm not against it, I just don't think it adds much value. Rather than hire smart and experienced Admins, they now feel that they can skimp on the higher salary candidate for someone not so qualified because they have a million dollar support contract in hand. My experience has been that hiring inexperienced IT Admins will cost you more in third party vendor support as just about anything that is remotely difficult gets outsourced. Regardless, even with contracts, the savings are still substantial after the decision is made to integrate Open Source.
> "We had a lot of downtime and data loss before we migrated over. After
> five months that was eliminated," he said. "There is a lot about open
> source that people don't know. Many corporations tend to lump open
> source into one basket, which is a shame."
Sadly, this is my biggest gripe when discussing the merits of OpenBSD. It's almost as if having the two words "Open Source" associated with OpenBSD just seems to have a negative effect. I've come to realize that this is mainly due to a misunderstanding or misconception about OpenBSD and Open Source.
> After the five-month migration, PWC's servers are now equally split
> between Windows and OpenBSD.
Yes
> "Microsoft just happens to be one of our clients and Checkpoint is our
> standard firewall," Uemura said. "Checkpoint on Windows was unmanageable
> but after a few months of using OpenBSD we were told to put Checkpoint
> back."
Another glaring example of professional journalism at its best :( After one month, I was informed that OpenBSD was not the firm standard Firewall. No, problem. I just rebuilt the Checkpoint Firewall and put that back into production...
> Then PWC was hit with a virus affecting network traffic and the
> Checkpoint firewall was running at 100 percent CPU capacity which was
> effectively a denial of service.
This was the only case in all of the Windows to OpenBSD migrations that I had to struggle with. As much as this Checkpoint was a new installation, it didn't sit well with me...
> "So we had to put an OpenBSD firewall in front of Checkpoint," he said.
This then satisfied our firm policy and also let me sleep at night knowing OpenBSD was the "Titan" out in front taking the worst of it without breaking a sweat.
> "We saved seven salaries worth over one year. It was so dramatic they
> gave me a big raise and I was promoted from system administrator to IT
> manager. And because of the savings we get more productivity out of old
> hardware."
The savings are all relative but whether you're a small business where every penny counts or a large multi-national with huge IT budgets, the saving are substantial in either case. Any company that doesn't have some sort of Open Source adoption strategy is just throwing money away.
> Despite this Uemura is adamant the move wasn't made because he wanted
> to. "As much as I love OpenBSD, we had no choice," he said.
Just because it can be done, doesn't mean it should be done. We could have done a lot more in terms of our migration of services to OpenBSD. It's not about pushing the envelope, but getting a balance that makes sense while ensuring the utmost in security and interoperability when doing so.
It took about five months with very long days and sacrificing most of our weekends in order to get it all done. However, it was worth it in the end. Not only did we save the firm money, we increased security, stability and restored user confidence in our IT systems and IT team. Our great sense of accomplishment was knowing that we did it, all the while maintaining business as usual.
First off, the story is not an interview even though it may come across as such. The title is rather sensational but I certainly wasn't desperate. There were problems and they were fixed and our team was just very resourceful in doing so.
Gedda writes:
> IT managers who want to deploy an open source solution but are worried
> about company politics should go ahead and do it without asking,
> according to PricewaterhouseCoopers (PWC) Japan IT manager Mark Uemura.
No, this is taken out of context. What I said was that we had very big and important changes that we needed to make in order to restore network and application stability. My reference to just going ahead and doing it referred to making the necessary changes behind the scenes. It wasn't about company politics and it wasn't about migrating services from Windows to OpenBSD. My experience was that we did ourselves a disfavour by trying to inform and explain to users and management the technical reasons for the changes that needed to be made. In fact, all of the pushback had nothing to do with OpenBSD. We needed to migrate from an old Domain Controller with a corrupt Active Directory to a new one. We also introduced the concept of working on Application Servers in Terminal Services to take advantage of server power for resource intensive applications that ran very slowly on users' PCs. So, the push back was related to things like "you'll have to login to this new Domain rather than the old one from tomorrow onwards." or getting users to change the way they work and use applications running on a Terminal Servers for speed. In the end, when all was sad and done, users and management realized the difference that we had made; no more downtime or data loss. Furthermore, they've never had everything running so smoothly and as efficiently for as long as they could remember. Their IT problems went away as a result of our efforts and the decisions that we made.
In fact, all of the migrations to OpenBSD were either behind the scenes where the users were oblivious to the changes. Well, almost oblivious. Often times we would get "Hey, the Internet is really fast today, cool!" or "Man, can you guys like spill some coffee in the server room or something? We're not used to this much uptime. It means we can't go home early anymore!"
In those cases where users did have to interact with OpenBSD, it was always well received and positive such as moving off of a very slow VPN for remote access on to a quicker and more user friendly alternative such as port forwarding applications through OpenSSH.
> Faced with an unreliable network, Uemura went ahead and migrated systems
> from Windows to OpenBSD on the premise that management would trust his
> judgement.
Once again, migrating services to OpenBSD was not an issue. So long as we did not compromise security in doing so. Generally, we did so to improve security and that's what OpenBSD is famous for and yet there's so much more.
> "PricewaterhouseCoopers is a Windows shop but we were forced to use open
> source," he said. "I inherited a real nightmare with servers going up
> and down. There were e-mail outages and on top of that there was a bad
> relationship between our users and IT."
Well it's either replace Windows with Window for Internet facing servers or find a more secure alternative that didn't have to be patched and rebooted so often. Bringing back network and application stability was important to the business as much as increasing security wherever it was possible to do so. I feel that stability is a result of good security.
We concentrated on network perimeter security. Hence anything that was public facing was considered so long as it satisfied four main criteria:
1) If security was a concern, then we used a more secure alternative to Windows.
2) If cost was a factor either for software licenses, service/support contracts or hardware, then we considered the Open Source alternatives.
3) If stability and uptime was important, then this was taken into account.
4) If all three points above qualified, then the last question to be answered before replacing any Windows based application or service was the following. Will there be any interoperability issues? That is, will there be any downside to replacing Windows and implementing a more secure, stable, cheaper Open Source alternative? If the answer to the last question was "NO", then we used Open Source when appropriate.
Once again, we were really concerned with any Internet facing servers.
> "My predecessor spent too much [so] I was told not to spend any money."
We could have begged for new hardware but it wasn't necessary. I knew that we needed to make big changes that required applications to be migrated from hardware to hardware. It is true that management told us not to spend any more money than was absolutely needed. This is just good business sense and a good rule of thumb to follow for any company big or small. If we were given the opportunity to spend on hardware, we would have had twice the server power that was really needed for our office in the end.
> When asked what argument he used to convince management to use an open
> source solution, Uemura said: "They didn't have an argument because they
> said don't spend any money." "They trusted me," he said. "The whole
> office was relying on one domain controller which was dying."
Again, we are not talking about the migration from Windows to OpenBSD. The journalist is really good at combining different parts of my talk and the answers to questions following it in order make his story that much more sensational. This news story is a great example proving that you shouldn't believe everything that you read. At face value, it's very misleading.
> Uemura said a lot of work was done "behind the scenes".
My team did a lot of work behind the scenes for which I am grateful. I didn't do it alone.
> "My experience is that if something has to be done, just do it - don't
> ask! They will thank you later," he said.
If you give users a choice, change or no change, they'll tend to favour the status quo.
> commercial lock-in.
There are many companies that have clued into this however many large financial institutions still have big support contracts with Open Source Vendors liken to a kid's security blanket where they just don't want to let go. I'm not against it, I just don't think it adds much value. Rather than hire smart and experienced Admins, they now feel that they can skimp on the higher salary candidate for someone not so qualified because they have a million dollar support contract in hand. My experience has been that hiring inexperienced IT Admins will cost you more in third party vendor support as just about anything that is remotely difficult gets outsourced. Regardless, even with contracts, the savings are still substantial after the decision is made to integrate Open Source.
> "We had a lot of downtime and data loss before we migrated over. After
> five months that was eliminated," he said. "There is a lot about open
> source that people don't know. Many corporations tend to lump open
> source into one basket, which is a shame."
Sadly, this is my biggest gripe when discussing the merits of OpenBSD. It's almost as if having the two words "Open Source" associated with OpenBSD just seems to have a negative effect. I've come to realize that this is mainly due to a misunderstanding or misconception about OpenBSD and Open Source.
> After the five-month migration, PWC's servers are now equally split
> between Windows and OpenBSD.
Yes
> "Microsoft just happens to be one of our clients and Checkpoint is our
> standard firewall," Uemura said. "Checkpoint on Windows was unmanageable
> but after a few months of using OpenBSD we were told to put Checkpoint
> back."
Another glaring example of professional journalism at its best
> Then PWC was hit with a virus affecting network traffic and the
> Checkpoint firewall was running at 100 percent CPU capacity which was
> effectively a denial of service.
This was the only case in all of the Windows to OpenBSD migrations that I had to struggle with. As much as this Checkpoint was a new installation, it didn't sit well with me...
> "So we had to put an OpenBSD firewall in front of Checkpoint," he said.
This then satisfied our firm policy and also let me sleep at night knowing OpenBSD was the "Titan" out in front taking the worst of it without breaking a sweat.
> "We saved seven salaries worth over one year. It was so dramatic they
> gave me a big raise and I was promoted from system administrator to IT
> manager. And because of the savings we get more productivity out of old
> hardware."
The savings are all relative but whether you're a small business where every penny counts or a large multi-national with huge IT budgets, the saving are substantial in either case. Any company that doesn't have some sort of Open Source adoption strategy is just throwing money away.
> Despite this Uemura is adamant the move wasn't made because he wanted
> to. "As much as I love OpenBSD, we had no choice," he said.
Just because it can be done, doesn't mean it should be done. We could have done a lot more in terms of our migration of services to OpenBSD. It's not about pushing the envelope, but getting a balance that makes sense while ensuring the utmost in security and interoperability when doing so.
It took about five months with very long days and sacrificing most of our weekends in order to get it all done. However, it was worth it in the end. Not only did we save the firm money, we increased security, stability and restored user confidence in our IT systems and IT team. Our great sense of accomplishment was knowing that we did it, all the while maintaining business as usual.
