Kakao Co-CEO Resigns After Its Datacenter Fire Locked 53 Million Users Out (techcrunch.com) 45
Whon Namkoong, the co-chief executive of Kakao, has resigned in a remarkable demonstration of corporate accountability after a fire incident at an SK C&C data center in the south of Seoul caused a mass outage over the weekend and disrupted Kakao's several services, including messenger, ride-hailing, payment, banking and gaming. TechCrunch reports: Namkoong, who joined Kakao in 2015, was elevated to the co-CEO role this March. At a press conference on Wednesday, co-CEOs of Kakao apologized for the mass outage "for such an extended period." Namkoong said that he feels "the heavy burden of responsibility" over the incident, adding that the company will do its best to restore the faith of users. Euntaek Hong, co-CEO of Kakao, who led the firm alongside Namkoong and is currently leading Kakao's emergency taskforce team, will remain the firm as the sole head of Kakao, per a company filing.
Kakao said today morning it had restored additional services like Kakaomail and TalkChannel. Most of its services are in almost full operation, with some remaining partially down. Kakao's slow recovery process was caused by the company's lack of owned server infrastructure and "high dependence" on the SK C&C data center, which caught fire, analysts at Bernstein said in a report this week. Kakao also didn't have a well-distributed backup system, they added. Hong said at a press conference today that the company plans to invest 460 billion KRW (~$325 million) to build its own data center from next year, aiming to complete it in the following year. According to a report in August, the company said its Kakao Talk messenger app has more than 47 million active users in South Korea and 53 million globally.
Kakao said today morning it had restored additional services like Kakaomail and TalkChannel. Most of its services are in almost full operation, with some remaining partially down. Kakao's slow recovery process was caused by the company's lack of owned server infrastructure and "high dependence" on the SK C&C data center, which caught fire, analysts at Bernstein said in a report this week. Kakao also didn't have a well-distributed backup system, they added. Hong said at a press conference today that the company plans to invest 460 billion KRW (~$325 million) to build its own data center from next year, aiming to complete it in the following year. According to a report in August, the company said its Kakao Talk messenger app has more than 47 million active users in South Korea and 53 million globally.
falling on his sword... (Score:4, Interesting)
...or getting the hell out of a company whose tech infrastructure is woefully inadequate, something only learned to the person who's only been there eight months?
There should have been plenty of people who knew how fragile their infrastructure was, for far longer and at depths that should have been in positions to advise senior management over the years for the need for expenditures to correct this.
Re:falling on his sword... (Score:4, Insightful)
Sometimes senior management has to 'take risks' and make bets. That might be for example some 'strategic' level cost cutting decisions around infrastructure design.
Sometimes a risk reward evaluation can be made correctly but still end in catastrophe. Most people though simply can't understand that some mistakes really are only mistakes in hindsight. Organizationally its often better for the person who made the "bad call" to just step aside, even if it was the right call based on all the objective data and risk analysis at the time. The loss of trust in the shot caller is real even if undeserved and organizations can't function without trust.
Re: (Score:2)
Absolutely this. And this is also supposed to be the reason why CxOs get paid the money they do. So they can take that responsibility.
Of course, in most cases that's only theory.
Re: falling on his sword... (Score:2)
Re:falling on his sword... (Score:5, Insightful)
Honor isn't something Westerners understand any more. You only care about plausible deniability to save your own neck.
Re: (Score:3)
English obviously isn't their first language, or they'd have spelled it "Honour".
Re: falling on his sword... (Score:3)
I Doubt I'm Alone (Score:2)
Re: (Score:2)
You question the parent poster's morality, and yet you post anonymously.
You have a slashdot account if you can post here. Own your shit. Or shut the fuck up.
Re: (Score:2)
I'll do what I want
You sound just like my 14 year-old nephew.
Re: (Score:2)
I am a westerner - born and lived in the US, and they're dead right, far too much of the US considers "honor" to be something in stories, not the "real world".
Now, you want to argue if the 2020 election was "stolen"?
Re:falling on his sword... (Score:5, Interesting)
Given this is Korea, you are apparently unaware of the longstanding Confucian influence on Korean and Japanese culture. Resigning due to failure, regardless of how immediately culpable you are, is considered an honorable thing to do.
Working for a consulting company, I once made a coding mistake, a dumb one, that went into a pharmaceutical company's batch processing data logger. The batch processing system looked into various values logged early in the process to make later decisions. During the implementation of the new system, my mistake lead to the loss of a test batch fairly late into its "cooking" process. It was only a test batch, but it would have produced salable product if it had succeeded and we would have gotten a bonus. I offered to resign after the mistake was found, and the VP told me God no, he'd just spent a lot of money training me and intended to get his money's worth. This taught me a lot about the value of life experience. It also taught me good leadership.
The cultural idea that it's honorable to resign after making a mistake leads to the loss of your most valuable training tool - which is real-world experience.
Honor isn't something Westerners understand any more. You only care about plausible deniability to save your own neck.
What you describe isn't honor. One can take responsibility for one's mistakes without resorting to seppuku. The story of the 47 ronin isn't one of honor and sacrifice. It's a sad tale of terrible, terrible waste. A society that wants its pound of flesh so bad that it forces the death of its most valorous.
I'm not suggesting that CYA is a better solution. Instead, I think the solution is somewhere in the middle. Honour is standing up and saying, yes, I made that mistake, not knowing what the final repercussions would be. Honour is not piling on social or societal pressure to fall on one's sword for their mistake.
Re: (Score:2)
I've worked in the financial world, and in a good amount of jobs within that specific market, it's mandatory to have insurance for when you fuck up and your fuckup has severe financial consequences.
Re: (Score:3)
Re: (Score:1)
> and a westerner just is not going to get it.
I guess not. It's hard to just accept a concept you "don't get".
How about a compromise: if something bad happens on your watch, be required wear a chicken suit to work for three months.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
You just don't get it and, it seems, you're just not going to get it.
Re: (Score:2)
What you describe isn't honor.
Yes it is.
One can take responsibility for one's mistakes without resorting to seppuku.
Resigning is not seppuku. Don't be a drama queen.
The cultural idea that it's honorable to resign after making a mistake leads to the loss of your most valuable training tool - which is real-world experience.
Yeah, and you'll notice it's the co-CEO who resigned, and not the actual workers. It's considered honorable for someone in his position to resign, rather than getting rid of the actual skilled workers.
Re: (Score:2)
Yeah, thought so.
What kind of a smug prick answers their own question wrongly, before waiting for the actual answer to decide whether to be a smug prick?
You're posting as an anonymous coward. You literally don't have honour. I, on the other hand, never post anonymously because I'm not a coward who can't stand behind every stupid thing I say.
Re: (Score:2)
It's also loss of face. Japan and Korea both, losing face can be seen as worse than losing money. The more senior you are, the more you are considered to be responsible for the activities that happen below you. It's not the same as "take one for the team", or even "sorry, it's my fault". Sometimes if someone does not resign in such cases it's a bit of a scandal.
Re: (Score:2)
Interesting and productive FP thread, though the humor attempt was rather flat. However I only looked at the story because I tried Kakao a while back and didn't like it enough to keep it on the phone. At that time I was running a bunch of apps of the type, trying to figure out which ones were best, and I'm pretty sure Kakao wasn't even in the top five.
The details of the story remind me of an experience I had working for a company with really poor management. Not such a fancy job title, though I was hired to
Re: (Score:2)
Honor isn't something Westerners understand any more. You only care about plausible deniability to save your own neck.
There are still some that do. Those also understand responsibility. But most c-levels in the west are in it exclusively for personal gain and do not care one bit how much damage they do.
Re: (Score:2)
There are still some that do. Those also understand responsibility. But most c-levels in the west are in it exclusively for personal gain and do not care one bit how much damage they do.
Thank you. This. Exactly this.
Re: (Score:2)
Just a request though. I would very much appreciate it.
Like imagined if I said, "Countries like Canada and Australia have a longstanding American influence of
Re: (Score:2)
Like yeah, it's technically true, but also, it existed before America
But it's not technically true at all, because we can trace historically where democracy comes from in Canada and Australia.
When it comes to Korea and Japan, it is a historical fact that they have been influenced by Confucian philosophy the most. This is simply historical fact.
taking responsibility
But this ISN'T merely taking responsibility. What happened is very foreign to Westerners, and you can judge that by the responses here.
In the US, a presidential candidate can say that you can grab women by the pussy, and be compl
Re: (Score:2)
Re: (Score:2)
Asking myself the same thing. I mean how the hell can you offer banking services on that scale without georedundancy? Did they lie to the banks and credit-card providers or what?
Lack of owned server infrastructure but it save $? (Score:2)
Lack of owned server infrastructure but it saves $? or it may cost more but that funds comes out of an differnt bucket so that the PHB can get an big $$ saving bonus
Re: (Score:2)
Yes, it can actually save money when you do not put a price | value beyond the core functionality. Once you start looking at things like externalities, contingencies, and maintainability then that equation might change. The same is true in reverse-- if you keep everything on cheap old hardware without proper backup and maintenance your costs may appear lower than if you went with a goatee server-less solution.
This has played out often when companies moved to an up-tiered data center; they assumed that most
In Korea its different. Looks like. (Score:4, Informative)
The executives, some six or ten of them, announced and fell on the ground, face down at the feet of the relatives. The relatives were crying, wailing, angry and were throwing their shoes and stuff on the poor executives, spitting on them, ...
I remember another incident. Some business class passenger complained that some flight attendant was rude, and the chief flight attendant made the accused one kneel and beg for forgiveness and pardon. Shocking other (Western) passengers and possibly the complaining passenger too.
Korean executives do not seem to get unlimited deference accorded to US Executives.
I posted this a long time ago in the group alt.pgh or soc.pgh
One guy responded, "all of them are employees. The unlimited deference and freedom from culpability is usually reserved for the owners, not the employees of the companies there. The small number of families that own the ship yards, ships, factories, land and skyscrapers are practically above the law there" he/she said. CEO too is an employee. So he has to behave like one.
corporate accountability? (Score:2)
Maybe this guy was responsible. Or maybe t
Problem solved! (Score:1)
Use multiple availability zones (Score:2)
If you have virtual infrastructure in AWS with high availability requirements, use multiple Availability Zones (AZs), which are distinct locations within an AWS Region that are distant enough from each other to provide isolation from common physical failures (fire, flood, tornado) but have inexpensive, low-latency network connectivity to other AZs in the same AWS Region (for reliable synchronous database replication, for example).
Re: (Score:2)
Is maintaining your systems on multiple AZs going to cost more in time and money on top of the basic service.
Re: (Score:1)
I run multiple full HA (read, multi-AZ, we have data locality requirement, can't do multi-region) and compliant (HIPAA, ISO27k), for less than 1000$ a month each. on AWS. It really isn't as expansive as people make it out to be. By the time you replicated this setup using cheap dedicated or cloud servers, you already spent more money architecting and implementing it all ( and don't forget how software changes every few years...) VS how much AWS costs.
More cynical interpretation (Score:2)
Alternate, more cynical interpretation of the co-CEO's actions: "I have enough money to live comfortably for the rest of my life even if I never work again, and if I don't resign, my life will suck for the foreseeable future as I try to fix the damage while taking daily shit and hatred from everyone on all sides."
The truly honorable thing to do would be to keep working as CEO, but give up his salary for the next year and instead divide it among the employees remaining (dividing the total amount by 365, and
Halon Suppression System??? (Score:2)
Re: (Score:2)
Halon is still around in various forms but it's no longer considered to be the gold standard. From a risk management perspective it doesn't always make sense to play with hazardous, environmentally unfriendly, and expensive chemicals to provide imperfect protection to systems that are not life-critical and don't have hard realtime requirements.
And no form of fire suppression is an adequate replacement for failover, business continuity, and disaster recovery planning plus regular exercises.