February 1, 2017 by SDesign
One of life’s small pleasures is, while staring a list of a hundred or so bugs, to mark an item as Works As Designed (WAD). You may know that as FAE, ‘functioning as expected’, or a similar acronym. (An early note to readers, today’s post applies more to waterfall processes than agile).
During the process of building a new product, there are countless people scrutinizing it as it moves through different developmental milestones. The group typically spending the most time with it of course is QA. It’s QA’s job to find defects. Unfortunately, often QA will end up filing bugs that are not really bugs, they are opinions about functionality. In some instances, QA may even try to claim some ownership or responsibility over the UX.
I have found that typically you need to put some effort into training QA about what is and what is not a bug. If you have specifications, wireframes, or prototypes, you need to reinforce multiple times that these artifacts represent the intended design. If a tester wants to question whether this design is correct, there should be a way to do that. But the tester should reference the existing artifacts, not pretend that a UX Designer has not already considered the behavior.
I have found the best way to get all of this under control is to be at the very front of the bug triage process. Almost all waterfall development methods will have a group meeting with many stakeholders present to determine what to do with incoming bug reports. The UX team should take a front and center role for defects that relate to the user experience. Consider the process below:
The gist is that UX owns vetting of all bugs related to the UI. UX may be able to make a determination right in that meeting, or it may need to study the bug report and make a decision outside the meeting. The designers will essentially have three choices when evaluating the question “Is existing behavior intentional and backed up by existing UX artifacts?”. (Look at the three boxes on the right hand side of the diagram). Let’s look at each option.
Yes. This is my favorite, the one that elicits a small pleasure. Here, the design has been clearly stated, and the tester has not followed the design and decided to make the existing behavior as a bug. The bug can be closed.
No. In this case, the tester has done his or her job. A UX artifact has indicated how the design should work, and the software is not currently working that way. This could be completely by accident, or it could be because the developer didn’t agree with the design. Either way, the bug now gets routed for code remediation.
Indeterminate. This is a pretty big category. Sometimes the bug will not have been specified at all in existing UX documentation, and therefore some consideration will have to go into what is the correct behavior. Sometimes testers catch extremely minor details that you may not care about, but sometimes they will catch edge cases that you have legitimately missed and that you want design for. Also, sometimes you may establish that even though your original UX artifacts said one thing, the tester is making a solid case for why that behavior is incorrect. It’s important to recognize your mistakes when you make them, testers who are allies with UX can make terrific catches on design or specification errors.
What is really great about enshrining a process like this into bug triage is that it trains everybody that the UX design is to primary law of the land when it comes to implementing the UI. Everybody makes mistakes, and there may be mistakes or ambiguity in the UX Design, but when this process is followed rigorously, it teaches all parties how to behave. This process enforces that filing bugs over matters of opinion is not looked upon favorably.
As a coda to this post, I would like to add that this process can work for bugs reported by customers or other internal stakeholders. The main caveat I would add is that when a bug is marked as WAD you are probably going to have to do some real work when it comes to justifying why you have marked it the way you did, and communicating that rationale clearly and politely back to the reporter.