Boeing To Use Computer Simulations Instead of Physical Certification Tests For Some Aircraft (theregister.co.uk) 138
New submitter Falconhell writes: In an ironic turn of events, Boeing wants to skip some physical certification tests and use only simulations. Given their current situation, this seems like a rather controversial move. Boeing is "reducing the scope and duration of certain costly physical tests used to certify the planemaker's new aircraft," Reuters first reported over the weekend. The manufacturer wants to switch to software-based trials for things such as wing load testing, "instead of doing things like bending actual, and highly expensive, components until they snap," adds The Register.
Re: (Score:2)
No way (Score:5, Interesting)
I've been a programmer for over 30 years now including lots of "artificial intelligence" and unfortunately a simulation can *never* 100% in all cases model the actual real world.
Re: No way (Score:5, Insightful)
It's people that believe a degree is the sole designator of skills and knowledge that are demonstrating incompetence.
Re: (Score:1)
Neil DeGrasse-Tyson's degree is irrelevant - he is a PR figure and hasn't done any meaningful work in the field he graduated in for over two decades. He is an "expert" in being a token minority with a degree.
Carl Sagan choose poorly in his case.
Re:No way (Score:4, Insightful)
They are specifically talking about doing this for wing tests, which are a well defined problem with a static test frame setup, and a long history of software based design and evaluation with finite-element analysis.
This should be permissible when working with known and validated materials and methods, although experimental or new techniques and materials still should have physical validation IMO. There just isn’t a point in doing it for another autoclave cured composite wing after testing for the 787 and 777 have been done. Out-of-autoclave curing should require physical validation, as well as whatever 3D printed thing comes next.
Re:No way (Score:5, Insightful)
In theory, they _could_ make this work. But it requires an extra careful approach and people very experienced with the software, the simulation and the physical object being simulated. It does cut out the redundancy that real-world tests give you and real-world tests will often find things overlooked in a simulation. As, I am sure, they did some nice simulator tests for their MCAS, but the real-world "tests" later showed a failure condition that is either a lot more probable than expected from the simulation or did not show up at all in the simulation.
So, of course they should simulate. But cutting out the real-world tests is playing fast and loose with their passenger's lives. Which they already have done several times and if you do it often enough, you end up killing a lot of people because in some instance you will find that the real-world tests would have shown you something the simulation either did not or where the simulation results got misinterpreted.
Re:No way (Score:5, Interesting)
Simulations are great as part of the development. They can save a lot of cost and time. However, I have seen simulations come up with decidedly unphysical results (for example, a sprinkler head freezing solid in a raging fire) even while the software is generally reliable. The butterfly effect is very much in force. Wherever possible it really is best to do a final physical test. You really can't build a high rise and then burn it down as a test, but you can certainly test a wing under reasonable flight conditions.
Re: (Score:2)
This should be permissible when working with known and validated materials and methods, although experimental or new techniques and materials ...
You are 100% correct; however, when push comes to shove, the definitions of the bolded words will be stretched to meet whatever pressures the tester is under. That is why this is a bad idea.
Re: (Score:2)
They only test new wing designs. The fact that it is new pretty much invalidates the previous software models.
Re: (Score:2)
That is why a safety factor of 1.5 is built in. (or is it 2.0 for wings and landing gear? I forget.)
Re:No way (Score:5, Insightful)
...a simulation can *never* 100% in all cases model the actual real world.
If it could, Boeing would probably not have killed 350 people recently...
Re: (Score:2)
If it could, Boeing would probably not have killed 350 people recently...
Ironically Boeing's simulator is able to easily reproduce the very problem that killed those 350 people, including the actions of the system and the difficulty of turning off the system. There's a good youtube video of some pilots showing this exact failure mode and how it happens and how it causes pilot confusion *in* the 737MAX8 simulator.
Re: (Score:2)
I don't doubt that. After the fact, if the simulator is worth anything, it can be configured to simulate the specific, now known problem. But obviously, before the fact, this failed. And that is the problem.
Re: (Score:2)
I've been a programmer for over 30 years now including lots of "artificial intelligence" and unfortunately a simulation can *never* 100% in all cases model the actual real world.
If the simulation can be proven to cover all possible inputs to the system then it can verify all behavior appears appropriate (such as no input combination can ever cause the aircraft to command an uncontrollable dive). However I wonder if the cost of covering all possible inputs is actually cheaper than testing with a real aircraft, or if they plan on only testing a subset of inputs that will only ever happen in the "real" world, ignoring that spurious inputs or unexpected situations are not being tested
Re: (Score:2)
Actually, not even then - a simulation by necessity operates in discrete time increments and discrete spatial elements, and can thus only approximate the behavior of real physical systems operating in continuous time with atomic-scale physical elements.
Most simulations in the aviation environment aren't that kind of real time. It is more like frame by frame simulations. So based on what you know about the state of the software, you are testing state transitions more than full simulation. Set inputs a, run the software through x number of frames, verify output A. Then move onto the next state, or reset and set inputs b and run the software through y number of frames. Once you've run through all possible inputs, you may have actually tested more situat
Re: No way (Score:5, Insightful)
Or unless Boeing intends to fly this thing with simulated passengers. If there's an oopsie, just reboot.
Re: No way (Score:1)
Even if that's the case, and you have the code. Your still need to do an implementation of it, which could or could not behave exactly like the original one, or maybe it will need some hardware emulation (which you can't guarantee is completely accurate).
So, be it as it may, it would not be a good idea.
Fun fact (Score:5, Interesting)
The original 737 (or what would become the 737) failed its initial wing loading test. They fixed it pretty fast, but the wings in the first test broke well before they were expected to.
Re: (Score:3)
Re: (Score:2)
The aviation version is called speed tape.
Re:Fun fact (Score:5, Informative)
I'm way more concerned that the A380 also failed its ultimate load test, and that was in an era of FEA software models:
https://www.flightglobal.com/n... [flightglobal.com]
(This article is a darkly comic read: engineers explaining away the fact that the wing did not meet its design spec: 'Garcia says that the failure of the wing below the 1.5 target will require “essentially no modifications” to production aircraft' -> translation: 'This design needs modifications')
Re: (Score:3, Informative)
787 had fucked up wings too, because the composite carbon fiber didn't behave the way they thought it would.
Re:Fun fact (Score:5, Informative)
Shame you didn't quote the reason he gives for it not needing modifications:
"This static test airframe has the first set of wings built, and we have refined the structural design for subsequent aircraft due to increased weights etc. We will use this calibration of the FEM to prove the adequacy of the structure on production aircraft."
In other words they built even more safety margin in to the production versions, and these prototype wings are being used to further calibrate the analysis software.
And of course, the production models did pass the load test in order to get type certification, more than a decade ago, and have been flying just fine ever since.
Re: (Score:2)
He adds that the ultimate load trial is an “extremely severe test during which a wing deflection of 7.4m (24.3ft) was recorded”.
Emphasis mine. That's pretty awesome.
Re: (Score:2)
The first 360 wasn't delivered until 1965, well into the design of the 737. While it was quite impressive for it's day, I think you are over-estimating the computing power of the 360.
Re:Fun fact (Score:5, Insightful)
The original 737 was designed using slide rules and drafting tables. We can do a shit ton better with computer models now.
Not unless you know and understand very well the physics and the mathematics of the problem you're trying to solve. The computer only helps you with the speed of your calculations. You still have to be capable of telling it what to calculate and how to do it.
And the odds are that if you find the slide rule a challenge, you'll suck at computational physics as well.
Re: (Score:2)
The original 737 was designed using slide rules and drafting tables. We can do a shit ton better with computer models now.
Not unless you know and understand very well the physics and the mathematics of the problem you're trying to solve. The computer only helps you with the speed of your calculations. You still have to be capable of telling it what to calculate and how to do it.
And the odds are that if you find the slide rule a challenge, you'll suck at computational physics as well.
Slide rules [wikipedia.org] haven't been taught for a long time. I did study some relevant subjects (numerical analysis, finite element method, (computational) fluid dynamics and high performance computing at university level in the 90s, but I've never touched a slide rule.
Really, Boeing? (Score:2)
Really? *Now* you're going to try that? In public?
I'm not sure "bold" is the word I'd use to describe that.
Re: (Score:2)
They probably think they are invincible. After all, there are 350 people dead which clearly is their fault, but nothing really bad has happened to the people responsible and it looks like nothing will. With all the military contracts they have, they are going to be protected against backlash. I guess it will take passengers and pilots to refuse en-masse to fly with their death-traps before they notice something at Boeing.
Re: (Score:2)
What could possibly go BOOM?
Do current simulations match real world results? (Score:3)
Re: (Score:1)
Well the computer they ran the simulation on Crashed with a Blue screen of death, so yes, tests met real world results.
Re:Do current simulations match real world results (Score:5, Insightful)
Re: (Score:2)
And if there's a difference, the real world wins.
Re: (Score:3)
Well said. Simulation represent a massively simplified version of reality, because they would otherwise become infeasible computationally. That limits their accuracy and predictive power for any effect not anticipated strongly. And simulation software had bugs and can be run with flawed parameters or outside of what its creators anticipated. An example of that is larger tolerances in the real-world object than in the simulated one.
Re: (Score:2)
A favorite old saying: "In theory, there's no difference between theory and practice. In practice, there is."
Re:Do current simulations match real world results (Score:5, Informative)
They never do. Simulations are always simplified and for complex things massively so. The simulation can simulate basically all effects their creators anticipates and all interactions their their creators anticipated. They can rarely find more complex interactions and effects, but they are in no way assured to find all of them. They cannot find effects the creators of the simulation did not anticipate. And, of course, simulations are done by software and software has bugs in its implementation, design and architecture and such bugs can hide even effects that the simulation could have found if implemented correctly.
In the end, simulations are a tool that can reduce the need for real-world measurements and tests, but they cannot replace them whenever success is critical. Over-reliance on simulation is at the root of a lot of industrial accidents.
Re: (Score:2)
I wonder?
Part of it depends on what you want to test. If you want to verify that things run correctly in all real world situations, classifying what the proper outputs are for all possible inputs is very difficult. Instead if you want to test that specific bad outputs never occur for any inputs, say the aircraft commands an unrecoverable or difficult to recover dive, you can throw all possible inputs at the software and make sure none of the outputs are outside of the valid range. Testing when discussed like this
You have to sign a contract before boarding (Score:2)
The safety manual (Score:5, Funny)
The safety manual inside the aircraft will read: "This airplane was fully tested on Windows 10"
Re: (Score:2)
Obligatory IT Crowd scene [youtu.be].
Re: (Score:2)
Re: (Score:2)
"Huh, weird. It didn't crash on my computer." -Every software developer ever.
I don't want this to apply to mechanical engineering as well, and certainly don't want this to apply to airplanes.
Re: (Score:2)
The safety manual inside the aircraft will read: "This airplane was fully tested on Windows 10"
Or that nothing of note has changed, when any change should be of note.
Hopefully they aren’t planning to eliminate all physical tests, otherwise the only Boeing I’d want to fly is virtual too.
Time for a New Competitor (Score:2, Interesting)
Boeing and Airbus are a horrible duopoly that have been propped up by the governments where they reside.
They're both highly flawed businesses building machines that have barely improved over the past 60 years and, when they do improve, magically improve at about the same time because they're not really competing.
A company like SpaceX could almost certainly take a lot of its innovations and apply them to simpler, terrestrial problems and much lower speeds. I'd love to see them create a spin-off that focuses
Re: Time for a New Competitor (Score:4, Insightful)
Well, you have Antonov, Mitsubishi Aircraft Corporation, Comac, Sukhoi. All of them have aircraft that could to some extent compete with the 737. Airbus took over the C Series from Bombardier and Boeing basically taking over embraer so guess they dont count any more.
Re: (Score:3)
Antonov is pretty much dead, but since they're a state-owned corporation, they will linger on for some time still. They have no aircraft that could possibly compete with 737. They really don't have any aircraft that's ready for production IMO. I have read an interview with the director of the company, and he pretty much bluntly stated that there are no customers, no orders, nothing. Even Saudis already walked away from or scaled down the Antonov-132D project.
Sukhoi: Sukhoi Superjet is 100-seat airplane, so
Re: (Score:2)
Well, the main problem with the SSJ is the crippled engine, but since the MS-21 will use the same Pratt&Whitney geared turbofan as most of the other new single aisle airliners, that shouldn't be a problem anymore.
Re: (Score:2)
SSJ's problems extend beyond the engine. Those problems are Crimea, Putin, Syria, sanctions. No one in their right mind outside of Russia will be buying more SSJ or MS-21. Moreover, these aircraft also can't be exported to Iran, Cuba, or Venezuela because they're loaded with western technologies. They're basically for Russia's domestic consumption.
Re: (Score:2)
Comac: With the WH throwing its weight around and twisting every ally's arm to stop doing business with Huawei, the first Chinese company that rose up to challenge the Western electronics supremacy, I think the same thing will happen with Comac. It's a product for China and a few countries that are allied with the Chinese.
The WH trying to strong arm china is going to last as long as Trump does, which wont be long.
Whilst we won't see the COMAC C919 in the west, you can expect to see it in a lot of places outside China, specifically with routes that do not involve Europe or North America (Australia, Singapore, New Zealand, Panama and other notable exceptions). What you've got to remember about the C919 is that it's built with American and European components in China. However COMAC isn't going to be delivering C919s until a
Re: (Score:2)
No. (Score:2)
Also, hell no.
Because they have such proven competence... (Score:3)
... with software. Does this company have a death-wish?
There is a place for simulations (Score:2)
Zero Orders (Score:4, Insightful)
Boeing spent the first couple days with zero orders for new planes at the Paris Air Show. Way to give customers even more reasons to doubt your commitment to safety.
I just don't get corporate thinking sometimes.
Re: (Score:2)
I just don't get corporate thinking sometimes.
That is simple: There is no real thinking done about their products or about engineering. Nobody today has the big picture technology-wise. All the thinking goes to profits, marketing, politics, infighting and fighting the competition. And in large corporation, they deeply believe they are invincible and believe that the corporation will always be there.
Re: (Score:2)
And ended the Paris air show with an order so large that a competitor is pursuing legal avenues to attempt to be part of the tender. IAG announced it will buy 200 737MAX 8s.
If you look at the first couple of days for the Paris Airshow then Boeing would appear to be in trouble. If you look at all of them then Boeing left the show with by far the largest orders.
Now that said I agree with the sentiment. It's tonedeaf to announce just after you killed hundreds of people due to software problems that you will te
Re: Zero Orders (Score:2)
It wasn't really software problems though. The crashes were due to a combination of overly aggressive software, poor hardware design, and pilot error. But even if you laid the blame entirely to the software side of things, it wasn't software testing that was the issue.
I understand why it might seem tone-deaf to people who don't know much about it though.
Which is it, "to use" or "wants to use" simulators (Score:2)
The article keeps changing from 'will' to 'wants' and back. And nowhere in the article does it mention the FAA or its testing requirements.
Real journalism is dead.
Re: (Score:2)
This happens when you let business self regulate (Score:2)
Re: (Score:2)
Re: (Score:2)
If you think that would help, remember how Midgley showed off the apparent safety of TEL, secretly taking a vacation to cure himself of lead poisoning afterwards.
Re: (Score:2)
at MTOW, and through a heavy storm, while doing aerobatic maneuvers.
If the software is right, the only thing the CEO will lose is his lunch.
sometimes a good idea (Score:5, Informative)
So, maybe Boeing wants to replace a few real life tests with simulations. If they've thought it through, and have good technical reasons why they can move from a 60/40 simulation/real-test split to a 65/35 split and save some money while maintaining a thorough analysis, then great. Everyone wins. If it's a manager-driven decision, then all bets are off.
And, yeah, I'd seriously question their judgement, coming right off several crashed planes because they decided to create a single point of failure inside a critical human-involved control loop. That just screams "WE"RE NOT LISTENING TO OUR ENGINEERS".
Re: (Score:2)
Modern simulations are extremely powerful. They can analyze things that are impossible to cleanly test in real life. On the other hand, real-life testing still catches stuff that can't be modeled well yet. For a complex system, both are necessary for a full engineering analysis - anyone who says that either is enough is flat-out wrong. This is something I actually know something about - I'm engineer in a field that uses both mechanical modelling and real-life testing.
.
So, maybe Boeing wants to replace a few real life tests with simulations. If they've thought it through, and have good technical reasons why they can move from a 60/40 simulation/real-test split to a 65/35 split and save some money while maintaining a thorough analysis, then great. Everyone wins. If it's a manager-driven decision, then all bets are off.
Simulations are good for testing and development as they allow you to find simpler issues without the expense of a wind tunnel or in many cases, an actual prototype, drastically cutting both the cost and time of development (although, we seem to be negating both via additional complexity).
However for certifications, these should be the first production models. Always, if you're not ready to put something into mass production, you shouldn't be certifying it. Especially if it's an airliner. In fact I'd go
New management initiative (Score:2)
This is part of Boeing's new management initiative known as "see foot, shoot foot". The simulation approach narrowly beat out the proposal to test on paper after the test group got too many paper cuts while folding the airplanes.
Industry rumor holds that this may be a temporary measure while execs and lobbyists test the waters for "faith based testing".
Certifying a video game? (Score:4, Insightful)
To merely say "these are the parameters we know about" and then to plug them into what is just a computer model - like video games use to simulate physics - cannot test real-life situations, only theoretical conditions within a known envelope that do not define where the limits of knowledge is. Or when they have been exceeded
This effectively turns all new planes into beta tests.
Can't wait for computer simulated drug testing (Score:2)
We need to bring drug prices down!
Maybe Boeing will tell the simulator . . . (Score:1)
Maybe Boeing will tell the simulator about all of the new "features" that were installed.
Hmmmm. (Score:2)
We've already seen what can happen when everything is based on the ability of Boeing's programmers, what with the two 737 Maxes.
But now it seems Boeing trusts its programmers with accurately simulating the even-more-complex physics and engineering of the wings.
I'm really not sure I do...
By all means use the simulations to create the wings. But do an actual god-damn real-world test, if only just to prove your simulations are actually valid. Damn the expensive of it, you're a very rich company, and, well, y
The Result of a Different Simulation (Score:2)
So Boeing made a simulation based on economics, profit, and money related things. It included legal expenses due to things like plane crashes (oops, sorry bout that). How much do you think Indonesian or Ethiopian crash victims will be getting?
Anyway, these financial simulations found that less physical testing would be more profitable so they are increasing aircraft simulations.
This was all explained in Fight Club, regarding car recalls:
A new car built by my company leaves somewhere traveling at 60 mph. T
Great Idea (Score:2)
I use trains. If they do that. I use trains. Or Airbus to cross the pond.
I worked at Boeing on the flight simulators (Score:2)
In the mid 90s, I worked on the flight simulator computers in the lab. I asked one of the engineers how good the simulators were in simulating the performance of the aircraft. He said that they were really quite good and you can easily learn to fly a real plane by doing everything in a simulator.
I then asked if there are things that you can do in a simulator that you CAN'T do in a real plane. He smirked, then proceeded to demonstrate something that you CAN'T do in real life.
1) Start on the runway
2) Tur
the PHB get's an big bonus for the $$$ saved with (Score:2)
the PHB get's an big bonus for the $$$ saved with this idea.