If you were wondering how I was doing push email tests on iPhone OS 2.0 and Vodafone UK, this is the reason why. The code wizard commandos at the iPhone Dev Team have been working on this non-stop since the early days of beta testing. In fact, I had iPhone OS 2.0 running on my iPhone since last week. That was version 5A345, two below 5A347, but identical in functionality.

[I] work directly with AT&T, T-Mobile, and Sprint / Nextel. I was on the morning conference calls hosted by T-Mobile's president and staff for the American side discussing the issues directly and the solutions to be provided.

What happened: Floods affected the main servers which supported all activation portals, company websites (public and internal), as well as star services (#646#, #225#, etc.) and myFaves updates. ... an attempt was made to migrate the data over to the back-up servers, but that was unable to be completed due to environmental issues.

Existing customers can continue to use their phones as normal, however things such as updating the myFaves settings or any other billing related updates can not be made at this time. Payment can NOT be accepted at T-Mobile corporate stores, however independent vendors (PreCash, etc.) are unaffected by this flood and money will credit to account as normal. FlexPay customers whose bill would be due WHILE the flood situation is being resolved will NOT be disconnected for failure to pay, however the bill will be due immediately after. FlexPay customers whose bill was due BEFORE this incident may be disconnected for failure to pay and must week an alternate independent vendor to pay the bill.

In my experience with these companies, there have been times when whole systems go down. T-Mobile has set up several internal conference numbers that we (employees, dealers, independent retailers, etc) may call in directly to have our questions answered. I have not seen this from any other company. For both our needs and those of the customers, additional operators have been tasked to all working call centers to answer questions and concerns you may have.

The progress: as of 6:00 am this morning [Tue Dec 4, 2007], power was restored and the long process of rebooting the systems has begun again. Proof of the progress can be seen in the main corporate website, t-mobile.com, being live once more. As more applications come online, retailers can expect to see their Samson, Watson, and iCam systems functioning normally. The earliest estimates put these all being corrected around 15:00 PST, however it may be up to 48 hours. Thankfully, though some IT staff did have to relocate during the flood, no one was injured from within the company, and additional staff have already been sent in to facilitate with bringing it all back online.

Please note that what I have mentioned above is NOT to be considered official until T-Mobile directly acknowledges this through a press-release of some sorts.

Bothell has a small concentration of datacenters and callcenters: T-Mobile, AT&T Wireless, OnFiber, Time-Warner Telecom, Worldlink, to name a few. North Creek runs right through Bothell and is prone to minor flooding every decade or so, and a major flood risk is always looming. ... We were at North Creek for 7 years and one of our perennial worries was an autumn storm causing North Creek to overflow its levees.