I had an interesting conversation with some fellow testers the other day about “what is it that makes a great tester?” This is an oft-trodden area, but this was a little more specifically angled at knowledge, experience and mental techniques rather than processes and tools.
Here are some of the “knowledge areas” that we discussed were as follows. These areas do lean much more towards the verification (building the product right) rather than validation (building the right product). But maybe that’s (sadly) true of most testers too!
Most applications are written with at least the thought of “this should be intuitive”… even if that thought is not implemented particularly well. Most consumer websites or apps such as eBay, Facebook etc would not go very far if they needed a 1000 page manual to be able to make use of them.
So, someone with no in-depth software or industry knowledge could perform some testing based simply on the fact that they understand the following:
– Basic language: For example Next, Previous, Name, Address, Cancel, Login etc Spelling and grammar is also included here.
– Graphic indications: You see an arrow pointing to the right, you assume this means “progress to the next section” or similar. Something resembling a printer probably means print etc.
– Objectives: I must be operating this software for some purpose. Therefore, I’m looking for some way to get to a place where I see something that indicates success.
– Errors: I understand that seeing a big red flashing error is not good
– Reasoning: If I am filling out an address form for a delivery address, it’s likely that the first line and post code are mandatory. If I don’t fill it in, I should probably see a warning.
– Visuals: I have some idea of what I like as a layout on the screen. I understand that there are some things like overlapping words that no-one would like.
– Learning: If I’ve tested this before and someone has told me I have done something right or wrong, I should learn from that. This learning could help across any of the categories.
– Consistency: I’ve looked at 3 reports with export to PDF buttons, but the fourth doesn’t have one. Why?
This knowledge could be gained from standard training courses such as ISEB but also past experience of what went right/wrong as well as talking to other testers and going to conferences etc. It includes:
– Boundary value analysis
– Equivalence partitioning
– I know that non-numeric characters often mess up an application
– I know that by injecting SQL into a field I might be able to extract some information that I am not meant to see.
This is knowledge which is very useful to “white box” testing, but not exclusively. It can also cover areas such as performance or security testing. Examples include:
– I know that web applications are likely to be limited in the number of users that they can support.
– I know that the software makes use of a certain API that I can test to get some interesting information out of.
– The software is built on MySQL and I know that has a bug that could be exposed.
– This might work on my development environment but I know that when it’s on production it will be load-balanced. Is that going to work?
This knowledge is helpful to a tester in that they may understand certain things such as:
– Common industry language
– Common industry calculations
– Common industry legalities
This knowledge is normally imparted onto team members through welcome sessions, demonstrations and product documentation. This can give testers:
– A complete picture of this specific application (or at least a better one)
– Product specific language
– An understanding of how goal X is accomplished with this product
– Known risky or flaky areas that may need more attention
This knowledge mainly comes into play with regard to the status of the project. Pertinent questions here are:
– What’s just changed? The risk here has probably just gone up.
– Are there any political reasons why something specific simply cannot fail?
– What deadlines are there? Do I have limited time to test?
This is an interestingly long list of knowledge areas that testers possess. It shows the value of having a quality manual tester and why today’s automation capabilities can only be used for certain parts of testing.
I have ordered the list of knowledge areas above as I have, not because I necessarily think one is better than the other, but more because they (generally) go from the general to the specific. Who is to say that the tester with immense technical knowledge is better than the one with impeccable common sense?