You hit on the key with "Maybe a couple of numbers under it showing # files done out of # total files". I understand making a predictable progress bar is very difficult. What most people struggle with is knowing whether it's working or not. Using the file example, when downloading a disproportionately large file may stall the progress bar, but providing some indication that it is still receiving data provides the user with enough information to assume that the process is still working and not locked up.
Archie Bunker's solution was to hand everybody a gun as they boarded the plane so that if a terrorist tried something, they would be outgunned.
There are several products on the market that are employed by the Exchanges and their large customers to track all of this.
This is a marketing paper for what appears to be an interesting product.
Existing vendors already capture, log, analyze (in realtime), traffic across multiple probes and provide real-time alerting along with monitoring, measurement, etc. These products are all leading edge and are changing rapidly. They've solved many problems with proprietary schemes of various sorts. Not the least of which was time synchronization at the nanosecond level.
For very simple public information, just look at latencystats.com. Keep in mind, more detailed info and analysis is going on behind the scenes.
While I can't speak to any specific exchange directly, I can say that their designs are fairly similar and fairly simple.
Fast links provide low serialization delay (the time required to put bits on the wire). A 512 byte packet requires 4.256 microsec at 1 Gbps and
Fast switches forward and sometimes filter traffic. the faster equipment requires between
The network design is very simple. Not your traditional vendor-derived three tier architectures, but 1, 1.5, or 2 tier architectures. The largest variations are in the handling on security and market data distribution. Firewalls are slow and typically deprecated. Market data is a religious decision marred by questionable technology implementations
Utilization is typically below 1% for any reasonable measurement period. Bursts may push an interface to line rate fractions of a second. Keep in mind that 10 GigE can carry 14.8 Mpps - so these bursts stress switch buffers and the receiving systems.
The total network latency within an exchange's network is well below 100 microseconds with some offering delays below 50 microseconds and a 1 or 2 that are below 30 microseconds.
The largest contributor to delay is, and will be for some time, the matching engines and other hosts that process the trades.
I think the point is that Willie Mays and Joe DiMaggio would both get the job done, but do it in different ways. Willie Mays would make the catches while running, diving, over-his-head, etc. All of which looked amazing and made the highlight reels. DiMaggio, did a better job of preparing and positioned himself better so that when catching a similar hit, he would be in position to catch the ball standing up. Both were great athletes, but DiMaggio made the job look routine. Even though he was equally capable of making the fantastic plays, his approach was more safe because he didn't have to make them so often. Planning and design vs. fire fighting.
Using real world data from real environments...
the cost of running a server is $3,000-$6,000 / year. The wide variation is due to differing data center resiliency profiles and utility costs (power, water).
That figure includes data center costs (space, power, cooling, utility, etc.) and excludes server operations staff, depreciation of hardware, support systems, backup systems, network, etc.
Firms that understand this and bill back the business units appropriately have markedly different environments.
Short answer: Transmission speed impacts latency when you consider how long it takes to drop all the bits (one at a time) onto the wire. This has not typically been a concern with LANs, but as latency gets measure in 10's of microseconds, it is a very real concern.
Network speed is a bottleneck in that it impacts latency through serialization delay (the time it takes to drop the bits onto the wire). This sounds trivial when were talking about Gigabit, 10 Gigabit and 100 Gigabit speeds, but the Exchanges are striving to meet network and trading latencies measured in microseconds. You can do the math yourself to see how long it takes to drop 512 bytes (1 bit after another) onto a wire. If you're anal you can add the appropriate overhead for framing, headers, etc.
Switching latencies also vary significantly and have a tremendous impact on vendor/product selection. This also drives network architecture as eliminating as many switching hops as possible is key maintaining latency budgets.
Distance from the exchanges is important, but not as important as switching latency and serialization delay. You can do the math to figure out how much time it takes bits traveling at roughly 2/3 the speed of light to reach their destination. Most trading firms pay large premiums to either collocate at the exchanges or very, very close to them. So-called program trading is performed by autonomous server farms operating in these spaces.
I can go on, but refer back to the short answer. Network speed is important.
"Have you ever heard the expression, 'When life gives you lemons, make lemonade, and then toss it in the face of the person who gave you the lemons until they give you the oranges you asked for in the first place!'"
Words to live by.
This worked really well for the banking industry...
Also, perhaps you're too young to remember how bad the taxi experience used to be before tighter regulations were developed AND enforced.
Despite the comments from people that don't live or work in NYC, cabs here are infinitely cleaner, better kept, and safer than cabs in other cities. NYC cabs are limited to 3 years of operation. Cab companies in other cities (Philly, Chicago, etc.) buy these 3 years olds and proceed to drive them into the ground. If you look closely you can usually see a spot or two of NYC yellow where the new paint has chipped off. NYC cabs have mandatory A/C for the passenger section and there's a simple number to call if you're having problems with the cab (311). There is a fixed rate from the JFK to Manhattan. All of this, including the rider's "bill of rights" is displayed clearly inside the cab. As is the driver's license number in case you want to report an issue.
If you're willing to give that up, so that you can save a few dollars and put up with low quality experience that results from cost savings and weak enforcement - please do so in whatever second tier city you inhabit. Finally, NYC cabs are already significantly cheaper than cabs in any other city.