seanb's Journal: Managing your application's performance into production

Goal: spent less time fixing old code, more time writing new code.

Why monitor? J2EE is ridiculously complex. Over-engineered solutions can be very slow.

Brief history of tools:
log-based analysis: flexible, fast, human resource intensive, limited application visibility.
code profilers: Great visibility, resource intensive, not designed for production use, deluge of data, no multi-tier visibility.
Synthetic transactions: solid uptime infrastructure, automated test cases, cannot monitor production.
Byte-code instrumentation, flexible, fast, tedious, limited multi-tier.

Goal: monitor production without affecting production. Bytecode instrumentation is the tech of choice, but needs better manageability.

Heisenburg. If you add too much monitoring, you can easily affect production. Visibility vs. Overhead dilemna.

Instrumentation tends to be a manual, iterative process.
Instrument everything and scale back.
Instrument everything and scale up.

Desire: base instrumentation on real usage patterns, minimize developer time, adjust without constantly having to bounce application.

Adaptive instrumentation: collect real, meaningful usage information, adjust as needed without restart.

Instrument slow methods, service requests, frequently used components, methods that do a lot of owrk, things that acces external resources, and frequently-used custom code.

Avoid all methods that will add significant overhead without real analytical value.