Comment: Re:Variety Pack! (Score 1) 228
Yup, that's a good list... some more things:
- I assume GP meant a test plan instead of a test script: a test script is closer to automating (push button x, check value y,
- Investigating (buying of/errors with) test tools (in my field you need some pretty high-end devices to be able to simulate large customer environments and a lot of these devices tend to be limited or broken in very unexpected ways.)
- Testing whether the product will hold under repetitive use (endurance)
- Testing whether the product will hold under high stress (scaling/performance)
- Testing whether the product works well within the environment for which it is intended (lots of software needs to interact with other hardware or software, but you usually don't have control over those other products)
- Testing whether the product is internally consistent and if your feature works well with other features in the same product (on a large project, this aspect of testing tends to explode as the number of features increases. That is when you'll need your testing and product experience to quickly filter out the risky combinations.) If the product is intended to be upgradeable: testing whether for instance data files stay usable between versions.
- For highly available products: check that all aspects of your feature keep working if it has to switch to the backup system
- Testing whether the product is secure
- Discussing (I wouldn't call it arguing) with developers and product management whether issues you found are actually bugs and if the feature as designed (or sometimes even as requested!) will even solve the customer's problem. (You can make a perfectly functioning screwdriver in request for a 'device for mounting fastening materials), but if the customer was looking for something to drive in a nail, chances are it won't sell to that customer.)
- Regularly checking the results of you automated tests and seeing if a failure is due to an intended software change or if it's a regression bug.
- Senior testers where I work are also regularly involved in feature design together with product management and development.
- Pushing the product to its limits (memory exhaustion handling, starvation from lack of other resources, timing issues,
- Working with high-level support to determine the cause of issues coming in from the field. If you have a good support group, whatever comes to the highest level there is something that needs test and development expertise to get solved. (Our highest level of support actually mostly consists of former test engineers.)
In response to the black/grey/white box testing (I actually prefer 'transparent' instead of white; a white box is still opaque): you'll usually work somewhere in between, starting from black box and going deeper if you get a bad feeling about the code you are testing (or if you need to give the developer a very detailed report about what's going wrong.) Full transparent-box testing is more akin to unit testing, which is the developer's responsibility (at least where I work).
If you are at odds with developers over the issues you find, something is wrong, at least when it goes beyond some friendly rivalry between test and development. Test and development are not enemies, they are both working together towards the same goal: a good product that the customer likes to use. In any non-trivial project, the developers *will* make mistakes, that's unavoidable. The testers will also make mistakes and flag issues that aren't really bugs... we are working on the same floor as development and when we find an issue that we can't immediately determine the cause of, we try to find as much information as possible and then ask the developers for help. We (the testers) know what we see, but the developers know exactly how the code works and what causes what we see.