I said that I’d do a Systematic Review… My Quick Guide to Getting Started (of sorts)


photo_27-4-14_easter_focusStarting at the beginning: I said that I’d do a Systematic Review… Easy right?! Hmm…

For the last two years it’s been my job to find and appraise published, peer-reviewed literature. Over that time I’ve seen my share of great reviews and then there were those that I needed a dictionary to wade my way through reading them. There were reviews that had the ‘insufficient research with robust methodology to support this practice’ statement. Actually there’s quite a few of those to be found on speechBITE, by merely typing ‘insufficient’ into the simple (keyword) search, 39 of them spring up at the time of writing this.

So, after reading and pondering over all these systematic reviews, surely conducting one myself would be a piece of cake (one that hopefully has really good icing)? Particularly when, let’s face it, there’s not a lot of research that has been published in the area of using social media (or specifically the use of Twitter) for enhancing communication after someone has had a traumatic brain injury. As someone wisely once said to me, “What could possibly go wrong?”. Well, the answer is HEAPS! Without a great deal of preparation and planning, conducting a systematic review has the potential to go pear shaped and end up in being a great deal of work for little outcome. By being ‘systematic’ in the approach, my aim is to conduct a robust search and critical review of the literature-to-date in the field and be able to report it in such a way that the search and review could be a) replicated and b) understood with ease (which for me, is just as important).

Working along and getting things started has led me to develop My Quick Guide to Getting Started (of sorts). Essentially, these are the tools that I would point out to anyone thinking about heading down the road to conducting a Systematic Review:

This article by Grant et al. (2009) provides a great outline of the differences between review types and can be used to determine which review type and associated methodology might best suit your purpose in conducting a review. For example, a literature review and a systematic review often differ significantly in the way literature is searched for.

In developing a systematic review the next logical step might be to see if there’s already a review being done in the topic area that you’re keen to investigate. By searching the current published literature you will know whether there are any completed reviews but what about those that are being conducted and yet to be published? Not all researchers publish their review protocol, so my tip – check PROSPERO as a starting point, as this is a prospective register of systematic reviews currently underway internationally.

Next, I like to know what to actually do! By looking through the Cochrane Handbook for Systematic Reviews of Interventions it’s easy to identify and prepare for the different stages of the systematic review process. The Cardiff University’s SysNet website also has some great information and links to resources. There’s even a best practice statement about what the minimum requirements are when reporting a systematic review. The PRISMA statement provides a checklist and flow chart that can help you navigate towards a publication that provides enough detail for the review to be replicated and will also allow readers to easily determine the methodological quality of your review. I know I love it when I get to read a paper that is logical and provides enough detail for me to know the process that has been followed. Having to re-read an article with a fine toothed comb is my least favourite thing to do, particularly if I’m then left frustrated with the lack of clear statements about the methods that have been employed.

A huge amount of my critical appraisal knowledge and skill comes from frequent practice and exposure to research methodology as part of my role in speechBITE. Although speechBITE is not currently able to provide critical appraisal of systematic reviews, if it were able to, the AMSTAR rating scale has been discussed as the tool that it would use. Certainly, alongside the PRISMA checklist, the AMSTAR rating scale is widely used to critically evaluate systematic reviews. Where PRISMA is utilised to evaluate reporting quality, the AMSTAR checklist is a validated and reliable tool that is used to evaluate methodological quality. To this end, in order to comprehensively know what constitutes a ‘good quality’ systematic review, I’m aiming to annotate my ever growing EndNote references by evaluating the systematic reviews using the AMSTAR checklist as well. There are several systematic reviews in the broad arena of investigating social networking sites in health related topics and I was keen to find out what was currently being done and the ‘level of quality’ they would be considered to have using this rating tool. Doing a little more background reading, I also came across this paper by Sharif et al. (2013) which gives a very easy-to-read overview of the AMSTAR rating scale.

So, I’m off to keep rating reviews but the next tip I have would be to look at the search strategies employed by other researchers in the field that you’re investigating. This can be extremely helpful in fine-tuning your own search strategy and double-checking that you haven’t left a crucial term out (or indeed terms)! I’m reading this recent article today that looked at testing search strategies for systematic reviews for some more tips!

I’m hoping to post more about reporting and methodological quality of systematic reviews. In keeping with best practice, I’ll be asking a second, independent rater to review the systematic reviews that I’m reading and evaluating, with the aim of being able to provide some further critical appraisal in upcoming posts. In the meantime, happy reviewing and let me know your tried and true tips on how to survive and shine during the systematic review process 🙂