
Journal Journal: Performance, performance, performance!
Two performance issues today: for some reason the picking slips being generated from our ERP system just crawl out of the computer, and the time it takes to make backups of our database has increased by 50% in the last week.
The picking slip issue turned out to be an indexing issue. After an hour of digging around in the source code, running the program with timer displays included (for some reason the wonderful profiler included in Progress decided to only show me performance on a couple of procedures - and those were not the ones I was concerned with), I tracked it down to a single statement, verifying whether a pick slip number had been used before by a pick slip, or by an order. The last search was the killer: it took about 3 minutes on our test box to search through all orders, since there is no index on pick slip number on the order table.
Since the check seemed extremely redundant (as long as you don't have two PPS's with the same number, I can explain to a user that the PPS number stored in the order is useless if you've deleted the PPS), I removed the check on pick slip number in the order. Instantly the time it took to generate a single pick slip went from 3 minutes to 0.7 seconds. Worth the change I'd say.
Unfortunately the supplier of the ERP informed me that it might not be a good idea to remove the check, and the newer version has an index on pick slip number in the order table. Oh well, as soon as they send me the exact format of the index, I guess I'll put the check back in and add the index. Even though I don't see the point. I'm flexible (especially on Fridays).
The second performance issue is a lot harder to determine. There doesn't seem to be any changes that have taken place over the last couple of weeks, but nonetheless, the backup on production takes much longer. It even takes longer that on our test machine, which has the same size and information.
One of the things that could have caused it was a fairly full backup disk. However, after cleaning up, it just took as long as it did before. Another reason could be that one of our programmers is running some scripts to create a new Cobol database, but he swears up and down it isn't run during the backup hours. The last suggestion was a runaway process, but nothing jumps out.
To determine the bottleneck, I've requested a copy of SARcheck, which analyzes the output of SAR and tries to compile a shortlist of issues with the systemit has run on. The copy should arrive sometime early next week, and I'm curious to see how screwed up our servers are in their eyes.