Before reporting, consider the mechanics

Mechanics

This page looks at questions you can ask before reporting, specifically related to the mechanics of reporting. There is an associated blog post as part of the ‘Assisting with inquiries’ series.

Here we address the broad question of HOW you communicate information effectively. Not surprisingly, many of the most important questions on this branch of the mind map are in the HOW sub-branch, but there are some other questions which can also assist.

Below, I have briefly explained each of the questions in this branch of the mind-map and how you might answer those questions. Where possible, I have tried to provide some practical examples, based on my own experiences with reporting.

WHO?

Mechanics who

Who will gather the information?

Who will communicate the information?

These questions help you decide who plays a role in providing information to your audience. Of course, in some circumstances the answer to both questions will be the same. Perhaps you are responsible for all of the testing of a particular product and are also responsible for providing information on that testing. In some larger teams or organisations, testing might involve many people, all of whom are likely to be aware of information which the audience will find useful. Sometimes one person has the job of collating information from many sources, interpreting that information and filtering it in order to present it to the audience.

Depending on the needs of that audience and the answers to some questions later in this blog, some other skills are likely be required, for example an ability to create models and diagrams, to present to others or to use particular tools.

Who will decide on the format and content of any reporting?

Part of the answer to this question is, of course, your audience. Having built up an understanding of their needs, it makes sense to agree some specific details with them on how written reports might look. It is sensible to do this early on. If you have ideas from previous work you have done, explain these ideas and how they have helped you before.

There are good reasons for you to have some say in the format and content. You will need to make sure that any unrealistic expectations from managers are properly addressed. If you are asked to provide metrics which you don’t feel are valuable then it is a good idea to explain why. Once a format for reporting is established it can be difficult to break.

WHAT?

Mechanics what.jpg

What tools might you use in preparing reports?

The answer to this question might well become clear when you agree on the content of the reports. The options available within tools such as HP Quality Center and JIRA are rather restrictive, and tend to result in uninspiring metric based charts. The same can be said of Microsoft Excel which is a powerful tool for collating and manipulating data but offers quite basic ways to present information.

If you are interested in presenting your information as an infographic, there are some excellent tools available, some of which are covered in this article by Randy Krum. Tools for visualisation of data (including some options for developers) are the subject of this article from thenextweb.com.

The best solution may well be a combination of different tools with the ability to capture data, visual elements and of course text, all of which can be combined to tell the story of your testing.

WHEN?

Mechanics when.jpg

When should the information be gathered?

You may need to consider a number of factors in answering this question:

the testing taking place and when information from that testing might be available
who is testing and when they might they be unavailable
any factors which might prevent you getting access to the information you need e.g. environment outages
how long is required to collate and interpret the information and if necessary, to put it into a format for reporting

When is the best time to provide updates?

If you have an understanding of the needs of your audience you will probably provide information according to these needs, but you may also make a judgement based on the findings from the testing that takes place. You may also consider making some information available on a ‘real time’ basis by giving people access to the tools or repositories you use.

HOW?

Mechanics how

How will information be communicated (what methods might you use)?

There are many methods and it is likely you will use a combination of them. Options include informal conversations, regular scheduled updates, formal presentations, written reports, models, infographics and other visual representations. The method can be tailored to the individual needs of your audience – what works well for one person may not work for another.

How can you assist in facilitating consistent interpretation of the information you provide?

If you have important information it is often a good idea to present it to all of the people who need it at the same time. That way you can address questions and clarifications as a group and avoid misunderstandings.

How will the most significant information be emphasised?

In the second blog (Braiding The Stories) in a series on Test Reporting, Michael Bolton discussed the parallels between testing and investigative journalism. His breakdown of the way newspapers are structured is very useful in thinking about how you might emphasise information. Keep in mind that it is very easy to lose important points among large blocks of unstructured text.

How can you keep reports engaging for the audience?

How can you make use of visual elements in reports?

In my first blog of the ‘Assisting with inquiries’ series I suggested taking a look at the work of Edward Tufte and Cole Nussbaumer Knaflic. I’d also recommend searching for examples of really good infographics (National Geographic have produced some spectacular ones). It would be easy to get lost in producing visual elements for reports and I’m not suggesting that you try to produce complex infographics. However, seeing how effective they can be might provide some inspiration. Shrewd use of captivating visual elements will keep your testing reports engaging for your audience.

How might you handle potentially sensitive information differently?

You won’t win too many friends by sharing sensitive information with everybody immediately. Developers might appreciate the opportunity to investigate and consider a response to a serious bug before you report it to decision makers. If during your testing you expose a security vulnerability which exists in a live website, it may be judicious to pass this information on to your operations or security team before circulating it to a wide audience. These may seem like obvious examples, but it pays to consider how sensitive information you uncover might be used.