Network monitoring for web traffic (Score 1)

I've spent some of the last few years on network monitoring, primarily for web-focused stuff. This used to be for outward-facing websites, but in the last year a lot of the big corporate apps have migrated their front-ends to HTTP. There are a few shifts that make traditional sniffing less and less useful:

Confidentiality. Most important apps are encrypted, usually with HTTPS, making them hard to diagnose at least for the database-intensive stuff that causes delays. This means you need to decrypt traffic (preferably with a copy of the server's private key) which in turn requires good key management (maybe FIPS) and decent control of data once it leaves the box. For example, you may want to delete any values after the POST parameter "password" before you persist the analysis to disk.

Port 80 convergence: The old analysis of traffic by port you get from a layer 4 sniffer is useful, but when all your traffic sits on one port it's hard to get useful results. You need to get down to a specific domain ("all traffic to example.com"), a specific object type ("all .jsp requests"), a specific page ("index.html") and even a specific parameter ("searches by zipcode").

Volume of traffic means that near-real-time analysis is important, or the buffers have gone by and the fire's somewhere else.

A couple of years ago, this "real user monitoring" stuff got largely ignored in the IT world. Today I get a lot more people who want to look at availability and performance by measuring users (no load, no scripts, more accurate) rather than synth testing.

Anyway, (disclaimer: I do tech strategy for a company that makes stuff in this area) there are a lot of software tools that reassemble the HTTP if it's what you're interested in. Some do it in real time; some mine the data after the fact. Some do it with performance information; others let you replay things. The shortlist of companies I usually pay attention to is:

Coradiant (where I work)

Adlex

Tealeaf

Network Physics

Peakstone

NetQOS

Niksun

NetLogger

Quest Spotlight

Mercury Interactive Real User Analyzer

ClickCadence

I'm curious: how much demand is there for web-specific sniffing (gaining application context at the expense of port breadth and non-http analysis)? Is it after-the-fact troubleshooting, or before-the-fact service level reporting?

Alistair.