Comment Build failure is just the beginning. (Score 1) 279
Then comes the installation and packaging failures. The dynamic libs get out of synch, wrong dll gets packaged in, etc
Then the build is good, it does not crash on every project. But it fails daily validation suite. After clearing that hurdle the build fails certification (same as validation but more detailed comparison with golden results).
So typically our last daily build is about one day old, last validated build is two or may be three days old. Last certified build could be three to five days old.
And this is a great improvement compared to the past.
Biggest advancement that helped us were dirt cheap prices for storage. So we are able to keep multiple older working builds for the developers. Not all of them need the latest build. Second biggest thing was the multicore machines with enough horsepower to launch all the library builds simultaneously. Third was the drop in network bandwidth prices, we are able to consolidate and synch the source code repo with very small (for a developer I mean, not for a video watcher) latencies.