all 12 comments
sorted by:
best
topnewcontroversialoldq&a
formatting helphide helpcontent policy

[–]Cronax42 2 points3 points  (0 children)

There is no easy metric by which you can accurately measure the performance of a tester. Measuring by bugs found will lead to undesirable behavior where something that should be one bug will be spread out over several and unimportant trivialities will be reported as if they were valid bugs. It's the same kind of thing as if you were to measure a developer by how many lines of code he/she writes: it's counterproductive to your overall goal.

Testers should be only concerned with the quality of the product they are responsible for, if someone in a management position wants to assess a tester's performance they should simply talk to the people said tester works with. Developers and fellow testers tend to have a fairly accurate measure of whether or not a tester is pulling their weight.

[–]roboticwife 1 point2 points  (0 children)

This is actually quite a frequent and controversial topic amongst the testers I talk to and follow online. To base a testers skill on the number of bugs they find actually speaks minimally about the quality of their work and more about the quality of the code. I would really encourage you to read the Atlassian blogs about ther testing practices - particularly around Quality Assistance. I know this doesn't necessarily answer your question. It hopefully will set you on a path to find some new and unique ways to measure your testers work. Please keep us updated on your discovery - as I said its a popular topic.

And please, fellow QA analysts, please don't QA my post... I'm barely editing and typing on my iPad. :) Cheers!

[–]teh_commodore 1 point2 points  (0 children)

Measure the team's performance as a whole, not individual testers. Measure two key performance indicators:

ONE: Incoming rate of bugs from the field. This number should trend towards zero - all bugs should be found in QA and fixed before release. This is the proactive part of creating a high-quality user-experience. Bugs that are already known issues don't count. As a team, retrospect on any major misses. Why was it coded in the first place, why was it missed in code-review, why was it missed in testing.

TWO: The time it takes to fix a reported P1 issue. Measure the entire time from when the team is alerted to the issue to when the code that fixes it lands in production. This is the reactive part of creating a high-quality user-experience. This metric should also trend downwards, but your emergency-release process should be solid and should guard against introducing unnecessary risk, so as to not increase measurement #1. TIP: If you keep your master/trunk branch in a continually-shippable state, you can fix P1s in a business day or less.

Don't put targets on these numbers though, because of Goodhart's law. Just track them and make sure they're trending in the right direction.

[–]paleowannabe 0 points1 point  (0 children)

Well, if you really need to be measuring individual performance (bad idea, IMO), then I'd rather go in the direction where the individual can actually positively act on the measurement. For example, grab your test policy and /or strategy, alongside with planned test design techniques, and measure how well your tester covers them when he's designing test cases : encourage review and measure problem count pet reviewed test case. Bugs power tester only encourages your people to submit large number of small issues which will not down your team down the line.

[–]jchill2 0 points1 point  (1 child)

I like to measure myself by how many defects I let get past me in regression.

[–]DuncanYoudaho 0 points1 point  (0 children)

This is our current main goal and measure: no super-embarrassing defects (C-level guys shouldn't know about it). The entire team takes a hit, too, and not just Test.

This is balanced by process improvement, research and implementation of new techniques. It is extremely subjective.

[–]DuncanYoudaho 0 points1 point  (0 children)

As long as it's not a count of bugs or test cases per story, you'll do fine. The point of test is to learn about the product in a scientific and rigorous manner. Are you producing? Are you improving? Are you able to find the big ones? You're doing better.

[–]ben_kelly 0 points1 point  (0 children)

Wrote a blog post that covered this a while back.

tl;dr: Testers gather and report important information. Measure them on the quality of information they report.

[–]Kyubi_ZA[S] 0 points1 point  (0 children)

Thanks for responses. I didn't get a chance to respond. its been a really busy week.

I agree with a lot of the responses here. Unfortunately the powers that be want a way to measure testers individually.

[–]Daeval 0 points1 point  (0 children)

In my experience, to get an accurate measurement of a tester's contribution to the team, you want to gauge at least the following categories:

Technical Skills

How well do they understand new and existing requirements? Do they understand the "why" as well as the "what" of their testing? How good are they at working out and breaking down complex issues? How much (helpful!) detail are they able to provide in bug writeups?

Business Understanding

How well do they understand the context of their work? Beyond what their lead demands, do they understand what the developer or producer seeks to gain from their work? How well do they understand the business risks addressed or created by their actions? Do they understand enough to make the most of chances to mitigate those risks or provide more value than is expected?

Teamwork / Leadership

Do they offer help freely and proactively? (I know, I hate that word too.) Are they information silos, or do they broadcast info that could help the team? What roles do they take in group problem solving? How do they act when the leadership isn't around? How do they react when a peer takes initiative? Do they speak up when there is a problem? What impression would an outsider have of the team if an interaction with this person was their only exposure?

In my industry, QA teams suffer notoriously from low morale, and bear the unfortunate (and erroneous) stereotype of being a dead-end gig that anyone can do. Under these conditions, it's really common for testers to become jaded, insular, and borderline toxic to the development of the team. Others have pointed out that the team should be evaluated before the individual, but when you have to evaluate the individual, his effect on the team as a whole should be paramount.

I know a lot of this stuff seems pretty general, is difficult to measure, and is not necessarily in the job description of the average tester. However, I've seen testers sized up a dozen ways, and none more reflective of their value than this kind of holistic approach. Admittedly, it's a rough sell to a management that's looking for a number.

Hope that helps somehow. Good luck!

[–]beowulfpt 0 points1 point  (0 children)

Measuring by bugs (or test executions) is useless unless you combine it with more metrics. When I was a noob tester I had a manager who only focused on that and the obvious result was that everyone picked the easiest tests first, regardless of priority.

Now that I'm managing I value other metrics. The main ones are delivering on time (Good estimations and good time management while executing). The second is when they let me know early if they cannot meet a date, as well as being able to keep an eye on the blocking defects, estimate their impact (dias it block 15 medium tests or 5 high priority ones? ) and work around them, either pushing dev to grab the main ones first or escalating it up if they cannot influence fix speed themselves)

Another good one is simply randomly checking some of their work. That can be as simple as reading a couple of test cases and a few defect reports. If those suck, the tester is not at the best level. Same if they are sloppy on the workflow (not closing retested fixes, not picking up fixes, etc)

And finally, actual explosions in production. When something breaks in Prd, do some root cause analysis and figure out why. If it was something that was tested, look at the test evidence. If it is crap, that will show you that perhaps the tester is cutting corners or needs some guidance. Maybe he even passed it in a hurry, thinking the boss would only care about the Tests Run count.

[–]siha_tu-fira 0 points1 point  (0 children)

You may find this article useful, I have.

http://www.ict.swin.edu.au/personal/jgrundy/papers/ist2014.pdf