Microsoft's Azure Cloud Suffers Major Downtime 210
New submitter dcraid writes with a quote from El Reg: "Microsoft's cloudy platform, Windows Azure, is experiencing a major outage: at the time of writing, its service management system had been down for about seven hours worldwide. A customer described the problem to The Register as an 'admin nightmare' and said they couldn't understand how such an important system could go down. 'This should never happen,' said our source. 'The system should be redundant and outages should be confined to some data centres only.'"
The Azure service dashboard has regular updates on the situation. According to their update feed the situation should have been resolved a few hours ago but has instead gotten worse: "We continue to work through the issues that are blocking the restoration of service management for some customers in North Central US, South Central US and North Europe sub-regions. Further updates will be published to keep you apprised of the situation. We apologize for any inconvenience this causes our customers." To be fair, other cloud providers have had similar issues before.
But Remember - (Score:5, Insightful)
Until it isn't.
Re:But Remember - (Score:5, Funny)
It's very safe though - just so safe no one can get access to it! :)
Re: (Score:2)
Oh their data is safe. They just can't get to it or use it in any way. :)
Re: (Score:3)
Yes, they can. It's service management that's down, not data.
Users can still access data.
data loss (Score:3)
When Amazon had that outage it was just thought to be an outage, but it turned out data was lost.
Disruption is bad enough, but data loss is way worse, since people and businesses likely won't have their own backups, and loss of data, even a low percentage, can easily KILL a business.
Lose an order or a customer's records or a customer's data and you likely lose a customer and get bad reputation.
Lose business records and it might be impossible to exist.
Re:But Remember - (Score:5, Funny)
Nonsense, Microsoft is the name you can trust for security.
Re: (Score:3)
Re:But Remember - (Score:5, Insightful)
Re:But Remember - (Score:5, Insightful)
Re:But Remember - (Score:5, Insightful)
Except those dumb terminals were, well, dumb, while nowadays the "terminals" are essentially the same as the "mainframe" but slower. So you can have hybrid configurations were a dedicated machines handles the base load and spins up remote resources on demand to handle peaks. If those resources are unavailable, the dedicated machine can still do the job, just with some performance degradation.
A good example would be a script on your laptop that started an EC2 instance running distcc to reduce your compilation time from hours to minutes. If the instance can't be loaded, you could still compile, it just takes more time.
Re:But Remember - (Score:4, Insightful)
If only even a single cloud service were actually built this way, it'd be great!
Re: (Score:2)
There are. Search for "hybrid hosting", "hybrid cloud" and similar.
Re: (Score:2)
Re:But Remember - (Score:5, Insightful)
Re:But Remember - (Score:5, Interesting)
Except this time you can add as many mainframes you wanted, dynamically. And access them over the internet. And serve content to millions of people over said internet. That wasn't possible with this clichéd "mainframes!!!!!1" nonsense. Yes, you are using a remote computer. That's the only similarity. The current terminals are far from dumb, and the server being connected to is vastly different to the mainframes of old.
I wonder how old you are? The current "Web 2.0" paradigm reminds me very much of the old 3270 style mainframe environment.
The 3270 terminal (well, the controller) was not exactly "dumb" - it had some base level of intelligence, it knew how to display forms, it could do input validation, etc but it didn't really do much with the data beyond sending it up to the mainframe. The mainframe on the backend took the data and actually did something with it. This is pretty much exactly how "Web 2.0" works, except instead of a 3270 terminal communicating to the mainframe over SNA, you have web browsers calling back to the web server over HTTP using Javascript.
Yes, both the endpoints and servers have become more capable, but there are still many similarities to the old style model.
Re: (Score:3)
Everything old is new again. The only real advance ove rthe mainframe model here is AJAX - and until HTML 5 really matures it's still a half-assed solution, but better than a dumb terminal.
Of course mainframes could add compute power to particular customers dynamically. Of course they could serve content to millions of customers (Visa used mainframes for transaction processing until quite recently). "Normal" HTML pages and forms are *very* similar to terminal-mainframe interaction. The older web server
Re: (Score:3)
The same problems with scaling now existed back in the main frame days, and the same solutions were present. No matter how many CPUs you throw to a developer, the applications must be developed to scale with them.
Do some digging on DMP (Dynamic Multi-Processor) and SMP (Symetric Multi-Processor) architectures and you will probably be amazed at how long ago these methods were being used.
This to me is the hilarity of the people that push "Cloud". They say things like Microsoft did in their "Yeah Cloud" comm
Re: (Score:2)
Re: (Score:3)
Hey! After rain comes sunshine. Now they'll just have to wait for cloud formations again...
Re:But Remember - (Score:4, Interesting)
When you rely on a 3rd party for cloud storage and that 3rd party has a basically nonexistent SLA for an under 30 day outage, it becomes your own fault for making a horrible business decision.
when you take a 3rd party cloud storage solution and implement it yourself for your enterprise, guess what? it works. And if there's issues, you know who's to blame.
https://spideroak.com/diy/ [spideroak.com] - this is one example of but many.
Re:But Remember - (Score:5, Funny)
So they can have virtually unlimited "read only" downtime as long as they turn write back on every
Let me guess. Switched you to read-only right in the middle.
Re: (Score:2)
Or, as I like to phrase it.
If all your data is in the Cloud, what happens when it rains?
Eggs? (Score:5, Insightful)
Basket?
Or how about "Never outsource your core functionality?
Re: (Score:2)
Never outsource your core functionality
Or more specifically, don't cloud your reasons for using it. Know what you are getting before you go there.
Re:Eggs? (Score:5, Insightful)
Ah, so there's the question. How much would it cost for you to run a system with 'no' downtime? I'm at a university, some of our labs (not so much in comp sci but generally) have fairly specific requirements about say not losing power, because it would damage/destroy equipment or running experiments.
But IT is more than just power. In almost 4 years here every year we've had several days of downtime for our main undergraduate server (the one undergrads are supposed to use for various things, and that handles their logins and file storage), and several on the separate but arguably more important staff server, which is supposed does the same thing, but that includes all of our grant applications.
Causes of our server outages (I'm not an IT guy, this is just what they've told us that I can remember): Power failures. Yes we have battery backups, but they're only good for so long, and since none of our equipment suffers permanent damage without power this isn't high priority. Networking. We only have two redundant pipes. That, for home use for example, or most businesses is pretty good. For our pipes one goes to a host to the west, one to the east. I'm not specifically familiar with what failed that took our networking offline for 7 or 8 hours but it affected both pipes. Storage: bad raid controller on the main fileserver. This has a few cascading effects. If you don't realizing it's garbling data it ends up distributing that garble off to the backups or clones. When it crashes (which doesn't take that long after the controller starts getting messy) you may have several backups that need to be repaired. We can't do much to the file system while it's being repaired or rebuilt (which, afaik you should be able to do on most professional grade setups, but for whatever reason our linux guys can't get it to behave). Added fun: When the system comes back up, if you tried to access your e-mail while the file system was garbled you probably still can't. And you get no error message about it. It just spits back nothing, as though you have no new mail. The system is 'up' but doesn't work and you have to go into your directory and delete some files that most people have never heard of. It's not hard to do, but because you have no idea that there's a problem the less technically inclined (or just ESL) people in building full of computer scientists don't always fix it immediately. The net effect is that if the storage controller gets messed up, we're down for 3 or 4 days if not longer.
And that's just one university department. We have a relatively decent amount of money, and several full time staff for these things. But we probably can't match any cloud services uptime, even with 7 or 8 hours of downtime regularly, not even close. It's not a trivial calculation, even a 50 or 60 employee outfit will probably have trouble matching Amazon or Azure uptime with a full time IT guy. There's probably a cross over point where you have enough employees to support big enterprise IT infrastructure and manpower, but only support it badly (there's not enough money for proper replication or whatever), and then eventually you get big enough that you just run everything in house anyway because there's definitely no cost advantage to hiring someone. For us, I think we have 5 or 6 IT staff, if we could toss 3 of them, + all of their equipment, you're looking at somewhere around 350, 400k/year to spend on a support contract. I'm guessing, but don't know, if you can get a cloud service for ~20 TB of reasonably reliable file and e-mail storage for less than 350k/year from these guys.
The big place I see people right now (as a sort of flavour of the month) using cloud service as an augment to burst capacity needs. That's a whole other analysis.
Re: (Score:2, Informative)
What ghetto assed university do you go to where they cannot get their server straight?
-AI
Re: (Score:2)
Community College
Ahhh, well you hit the nail on the head about 100% uptime.
That's something only "God" will allow. Any disagreements
with what he doles out, require 1-2 diesel generators and a
good sized tank of diesel.
Then the tier level of internet pipes you get in, determine
whether they give a crap about you being down for any
period of time.
Essentially, 100% uptime isn't possible just due to the
unknown. Which is why everyone brags about 5x 9's etc.
Almost everything needs to be triple redundant (min) and
you need to be near
Re: (Score:2)
Most reasonable datacenters can do 4 9's in all aspects whether it be networking or physical power.
Re: (Score:3)
Lets not forget that in order to hit that 4 or 5 9s, you have to have built in maintenance times which are down time but don't count toward your SLA.
And before you ask, yes.. I believe that most places will stiff a maintenance window when something bad happens and they have down time. So that 4 hour server crash sits in the 4 hour maintenance window and we get 0 down time for the month.
Re: (Score:2)
Good points. Near 100% uptime is intrinsically hard. And if you think your admins can do it better those at a dedicated cloud hosting provider .. well, maybe they can, but it's a good chance they can't. Get big enough and you can invest in the hardware, network and support resources to do it right, but that's not cheap.
Re: (Score:2)
4 9's (99.99%) uptime is 8759.124 hours.
That means to achieve 4 9's you can only have < 1 hour of downtime per year. This is possible.
Microsoft being out for 7+ hours is a nightmare.
Re: (Score:2)
Re: (Score:2)
Basket?
Or how about "Never outsource your core functionality?
That would be a good engineering practice. A good business practice is to show your initiative by outsourcing to the cloud and then hope to be promoted away before anything bad happens. It really is time for managers to be liable for the mistakes they make in long-term decisions.
Re: (Score:3)
They even may have. It is still a gross mistake, because you need to keep control over core functionality, so you can adjust its parameters to your needs.
Uptime is not all that counts and is not even very important. What is important is what you can do when the system becomes unavailable. In the cloud case, all you can do is wait and hope. If you kept control, you have various options and can have various recovery strategies, depending on your business needs.
Re: (Score:2)
I can see clearly now that the rain is gone...
Gloat gloat gloat. (Score:2)
Re:Gloat gloat gloat. (Score:5, Insightful)
Re: (Score:2)
Cloud services not ready (Score:2)
Re:Cloud services not ready (Score:4, Insightful)
Cluster at the application level and have nodes at different providers. If your volume is too high for that, you are big enough to host your own stuff.
Re:Cloud services not ready (Score:4, Informative)
Re:Cloud services not ready (Score:4, Insightful)
After an outage this long, it takes a LOOONG time to earn your way back to five nines (which works out to 5.5 minutes of downtime per year).
Only 84 years per the article, and growing at a rate of a year every 5 minutes.
Thats probably about how long it would take me to trust MS in an enterprise environment.
Re:Cloud services not ready (Score:5, Funny)
it's a leap year, they can be down a full day and still claim they were up for 365 days this year!
Office 365 (Score:4, Funny)
Maybe it's *because* it's Leap Day? (Score:4, Insightful)
I've always had to laugh at the name "Office 365" -- the fact this happened on Leap Day amuses me to no end.
In light of Excel's horribly buggy code of handling Leap Day, I have to wonder if Microsoft's problems here might not be because it's Leap Day? Whaddaya bet Azure comes back up all fine and dandy once the date rolls over to 1 March instead of 29 February? I'm actually serious about this conjecture, this is not just an attempt at humor.
On a different angle, does anyone else find it amusingly ironic that this service is named Azure, and now it's blue-screened? They've only gone up one letter -- now it's the ASOD.
Cheers,
Re: (Score:3)
I'd be surprised if Microsoft (or anybody) is actually offering five nines for uptime.
The fine print often says "well, we don't actually promise anything, and any outage and loss is your problem".
Re:Cloud services not ready (Score:5, Insightful)
One of the selling points of using cloud services was that it would be more reliable than managing your own hardware/software. But to date, every single big player has suffered major downtime. If I would be hesitant to believe the sales pitch.
But still, for most companies that are good candidates for cloud offerings, even 8 hours of downtime once a year is probably better than they can guarantee using their own infrastructure. Companies in this range tend to not have redundant servers, offsite backups, disaster recovery sites, etc. Larger companies that can build redundant infrastructure (and staff it properly) are probably better off staying away from the cloud since they can guarantee any level of uptime and redunancy they want to pay for.
Of course, when a small company Admin spills a cup of coffee in the Exchange server and they are down for 5 days while building a replacement server, it doesn't make the news so you never hear about it...while when a large cloud provider has a 2 hour outage, it's all over the news.
Re: (Score:2)
Re:Cloud services not ready (Score:5, Informative)
Re: (Score:2)
Amazon had major downtime.
So merely days after announcing the G-Cloud... (Score:2)
...the British Governments Cloud service suffers the inevitable Microsoft kiss of death [v3.co.uk].
Re:So merely days after announcing the G-Cloud... (Score:5, Funny)
The hilarious part of this link is that the article detailing how screwed people are for depending on Microsoft's cloud services is stuffed with rollover ads for...Microsoft's cloud services!
Re: (Score:2)
The article title is
Government's G-Cloud service knocked offline by Microsoft Azure cloud computing outage
And all around the page, an ad that says
Get in the cloud - Microsoft Office 365
Just like hearing a Subway commercial while in the restrooms of a McDonalds... Priceless!
Re: (Score:2)
...And all around the page, an ad that says
Get in the cloud - Microsoft Office 365
Well, clearly they need to release Microsoft Office 366 that works on leap years.
Re: (Score:2)
clearly judging by the name they only intended it to work 365 days a year
Re: (Score:2)
Re: (Score:3)
No... it will still be up for 365 days this year... trouble is... it should have been up for 366.
2/29/2012 (Score:5, Interesting)
Leap year strikes again?
Re: (Score:2)
That was my first thought.
Re:2/29/2012 (Score:5, Informative)
"4:00 AM UTC We have identified the root cause of this incident. It has been traced back to a cert issue triggered on 2/29/2012 GMT."
So yeah, a leap day bug sounds probable.
Re: (Score:2)
My thought, too. You'd think that after the Y2K madness coders would have learned to adopt more robust calendar implementations.
Yeah, like the Mayan Long Date!
Re: (Score:3)
Re: (Score:2)
Some coders are just shit and will never learn if you tell them (we lead them to water but they just don't listen - I've supplied working date-correct code in the past but some folks [usual Visual Basic schmucks] still want to do crappy hacks instead of doing it right).
If you are working with dates you should never work with an internal representation that is not Gregorian (although, of course, you need to display Gregorian, you just don't use that for your internal date representation). For example, the
Re: (Score:2)
So who is it that isn't recognizing 2/29 as a valid date? The platform? The certificate?
To quote the lady in the commercial... (Score:5, Funny)
Yay, cloud!
Now they're slashdotted, too... (Score:5, Funny)
This is not helping, guys!
Wait (Score:2)
Re: (Score:2)
If you are doing a BYO Server thing, or a conventional static-sized hosting package, and buying to fit largely static demand, you may never have touched the power button after you first shoved it in the rack and fired it up. However, if you are doing the cloud thing and not spinning stuff up and down pretty frequently,
Re: (Score:3)
We have a vendor that provides software distribution through Azure. It is completely down; no software and not even the web-based administration panel.
So it isn't just the ability to fire up new VMs, but (from my experience) seems to be a complete platform failure for some customers.
Re: (Score:3)
To be specific Microsoft said about 2.8% of customers lost storage/hosting.
Re: (Score:3)
I concur with what others have said. There are numerous services, being provided by Azure, that are completely unreachable, and have been so for longer than seven hours.
last time (Score:5, Informative)
Re: (Score:2)
it's OK, just explain it in the blog (Score:2)
like google does when something goes wrong. just explain how you're going to change things and why it happened and it will all be OK
Credibility (Score:2, Interesting)
At this point, the best way to keep their credibility from further deteriorating is to provide good reports on what is going on. E.g., not like PSN, more like Amazon [amazon.com]. Currently that Azure dashboard doesn't even load for me... has it been slashdotted or something?
As an aside: whenever a cloud system goes down, people come out to rag on the reliability of the cloud. While I'm also annoyed by the marketing guys throwing around "just put it in the cloud!!" as much as anyone else, and agree some applications
Is real failover redundancy a pipedream? (Score:2)
It seems like even the biggest guys can't make it work reliably, and presumably given the high profile of these services, they're not afraid to throw money and smart people at these problems.
Re: (Score:2)
Feature Suggestion! (Score:5, Funny)
They could have an adorable cartoon chicken that, when the system is working normally, runs around scratching and pecking(speed dependent on load). When downtime occurs, it would begin squawking about how the sky is falling. What could make failure more endearing?
Just to add that Microsoft touch, they could do the entire thing as a Microsoft Agent ActiveX control [wikimedia.org]!
To the cloud! (Score:5, Funny)
BCoD (Score:2)
Re: (Score:3)
It's the Azure CLoud of Death!
FTFY
The thing about clouds... (Score:2)
Ah, the cloud... (Score:5, Insightful)
It's funny how those of us who bring up issues of data security and service resiliency are dismissed as just trying to protect our jobs.
Like so many other things, the actual technical underpinnings of "the cloud" are great, and have been standard fare for years. Virtual machines + flexible networking are a godsend for systems guys tasked with getting capacity for a new project up and going yesterday. I love being able to build and rip down entire test environments just to try something out...that used to mean a rack of physical servers, switchgear, etc. tied up while it was being used. That's why everyone's slowly coming around to the "private/hybrid cloud" model, which is really just code for "VMs + network capacity + something to tie it all together + maybe some external hosting".
The problem is that "the cloud" is very badly misunderstood. As sson as a CIO sees "virtual, on-demand capacity without those pesky physical on-site machines and IT staff, for a fixed cost per compute-hour" everything else takes a back seat. Then, it's "why do we need IT staff on-site, everything's being taken care of in the cloud." Public clouds like Amazon or Azure are great for startups who can't really afford their own data centers, or even bigger businesses to offload some of the nonessential stuff. When you start looking at hosting everything though, the marketing hype of the cloud sometimes distracts people from realities that they have to contend with.
Also, I'm not saying that businesses who go the private cloud or traditional hosting/outsourcing route won't have downtime -- they will. However, having onsite staff and infrastructure means you can work those staff until they fix the problem, and you have control over them. Most sane outsourcing contracts have SLAs in them stating that the vendor will expend X amount of effort to fix your problems. Cloud provider agreements, unless specifically mentioned otherwise, are "as is, where is, best effort restoration with no warranty." OK, maybe some providers will give you an SLA, but all that does is buy you free service at a later date if they violate it...it doesn't bring your application back online. You still have no choice but to sit and wait around for the provider to fix whatever's wrong...just ask Amazon EC2 customers about what happened during their last outage...
Companies need to draw sane boundaries around hosted systems, and decide what is critical and what can be offloaded. Do I care about a set of development/test machines that get used once a month? Probably a lot less than the critical database/application servers that run my core business. Comfort level, cost per minute of downtime vs. cost of dedicated resources and other factors need to be carefully considered before jumping into the cloud with both feet.
Re: (Score:3)
Just so you know, the data is still accessible in Azure, it's the management console that's
down. That's still bad, but lets deal with the actual facts.
A) the cloud doesn't need to mean offsite. It often is, but the philosophy can be brought in house.
B) redundancy.
Companies should completely adopt the cloud philosophy, but keep onsite system redundancy; which is still cheaper and easier then current non cloud solutions.
The desktops should just be cloud machines. Note, I don't say dumb terminals bacau
Advice (Score:5, Informative)
1. Perform virus scan.
2. If that doesn't work, find a different program that will display a reassuring green graphic.
3. If that doesn't work, reboot.
4. If that doesn't work, reformat, reinstall.
5. If that doesn't work, GOTO 1.
Microsoft wouldn't know anything about data center running if it were chase aftering them at full speedo.
Google this: "Microsoft Sidekick / Danger"
http://techcrunch.com/2009/10/10/t-mobile-sidekick-disaster-microsofts-servers-crashed-and-they-dont-have-a-backup/ [techcrunch.com]
https://www.pcworld.com/article/173470/microsoft_redfaced_after_massive_sidekick_data_loss.html [pcworld.com]
http://www.appleinsider.com/articles/09/10/11/microsofts_danger_sidekick_data_loss_casts_dark_on_cloud_computing.html [appleinsider.com]
Is Azure free? (Score:2)
if so, that's the breaks. If not, then there should be contractual SLAs and penalties involved.
Re: (Score:2)
...there should be contractual SLAs and penalties involved
Do you really think Microsoft would put a gun on their own head like that, assuming they learned from their past?
I think they provide the service "As-is and with best-effort service recovery". Read the fine prints, I'm sure you'll find something like that.
Not like Salesforce, yet (Score:2)
I had an outage on Salesforce for 1 week and they did absolutely nothing regarding giving me any free account time or anything except "Sorry".
Their explanation was a massive multiterabyte log file had to processed since what corruption they had extended to their backup.
Shouldn't ever happen.
This was last Autumn.
All boy scouts should take away this: Cloud promises are made to be broken.
Re: (Score:2)
Given how boastful and grand these claims are, this really is not a surprise to anybody competent. Complex systems fail. They fail in complex ways. Redundancy helps in some ways, but makes things worse in others, by increasing complexity.
Also keep in mind that when outsourcing IT, the IT people suddenly have different business goals than you do. As long as they stay afloat, they do not really care whether you go under. In-house IT is different. They are sitting in the same boat. And any sane management wil
Great uptime! (Score:5, Funny)
Re: (Score:2)
Resiliency vs. Control (Score:2)
Cloud ain't so bad (Score:5, Insightful)
I wrote a comment on slashdot a while back which questioned the sensibleness of running services in the cloud. I used to be a sceptic.
Since then I've used Rackspace Cloud and found that it's actually a very good idea, for certain things.
The benefits of using a cloud system are scalability and no commitment- it's not about reliability or higher availability - but you do get a little win in those areas.
To give some examples, I was recently able to play around with mysql clustering. I followed a mysql clustering howto [reliablepenguin.com] and played around with it, setup a mysql cluster with load balancers. Once I was finished geeking about, I saved the VMs to the file storage and deleted the cloud instances. Total cost a £/$2-3 maximum. I hadn't previously been able to do this, I would have had to rent a dedicated server which would serve websites, email etc. I couldn't really use the dedicated server to play with new technology in case it had a negative impact on the live systems. I did have development box for a while, but it essentially doubled my costs without making any more money, just offering some protecting.
Now I have staging/development instances in the cloud - and no commitments to them - I don't have to worry about a £250 monthly bill or sign a 12 month contract to get my own box. I can fire up some resources, use them, and throw it away when I'm done.
The upshot is that I can play around with other peoples cool open source software without risk or buggering something up on my live box, and the costs are insignificant since I'm only renting it per hour. I can try something new, if it works great - it might go/stay in production. If not, delete it and move onto the next cool thing.
If I need high availability, I would use Rackspace, Amazon, Azure, and I'd ensure that I have a plan to deal with a major outage with any of the providers. Each have APIs, so in theory I could create new instances automagically and failover between different cloud providers with a quick DNS change, while keep costs low.
To recap, the cloud isn't all about high availability - no matter what the marketing says. It's about scaling systems and running resources for small amounts of time, and is perfectly suited to services which have peak demand (ticket sales for example).
Re: (Score:2)
Correct about the availability. Cloud services are a cheap way to rent processing/storage resources really cheap.
If I need high availability, I would use Rackspace, Amazon, Azure, and I'd ensure that I have a plan to deal with a major outage with any of the providers. Each have APIs,
Unless the API is proprietary (or just non standard) and the cloud operator introduces some systemic fault* into their services. What then?
Building apps targeted to LAMP services (for example) don't necessarily suffer from these problems. Because not every provider is installing the same patches at the same time (or even running the same configurations). So you gain reliability from a sort of ge
The Daily Show (Score:2)
When Clouds go down... (Score:2)
It's called Fog.
"This should never happen" ... Stupid (Score:3)
People that believe the cloud is not as risk for downtimes are just stupid and deserve exactly what they get. The cloud not only has the normal risks any comparable infrastructure has, but also suffers from additional risks because of complex network connectivity, complex usage patterns and untried system administration patterns.
People that still think this now are not only stupid but unwilling to learn, as the Amazon outage last year clearly showed the risks. In addition, Amazon is very likely more competent than Microsoft at this by any sane metric.
any chance... (Score:2)
I could fix this with a $35 payment to someone?
Re: (Score:3)
Yes, just like flat tires are putting nails in the auto industry coffin.
Re: (Score:3)
If you had "flat tires" that put thousands of cars out of service all at once, then flat tires *would* be putting nails in the auto industry coffin.