Systematic reviews

Are you conducting a systematic review? At the library, we often get questions about what exactly characterizes a systematic review, what differentiates it from other kinds of literature reviews, and how one might best go about searching the literature systematically. On this page, we will review a few things worth thinking about when you intend to conduct and report a systematic literature search.

This page is aimed primarily at researchers and doctoral students. If you are a Bachelor's or Master's student, you will find our guide Structured literature reviews – A guide for students useful. As a student you should rather do a structured literature review, including only certain aspects of the methodology described on this page.

What is a systematic review?

A systematic review is a literature review that gathers together all available research within a delimited research area and according to a specific methodology. Systematic reviews usually rate at the top of evidence hierarchy since they analyze and evaluate results from all available, original research articles that answers a specific research question. 

The entire process of conducting a systematic review, from formulating a research question to establishing a protocol, searching the literature and finally collating, examining and analysing the results, needs to proceed according to a carefully planned methodology. All stages of the process should be documented. Systematic reviews sometimes contain so-called meta-analyses, wherein the collected data is combined using statistical methods. 

There is a great deal of literature on method, and textbooks on the topic of conducting a systematic review; these describe the process as well as the steps usually involved in conducting a systematic review. There are also international guidelines that detail how systematic reviews should be reported, the Prisma Guidelines.

Illustration created by Karolinska Institutet University Library.

Tips for literature on methodology

Before you start, consider the following:

What kind of review are you writing? There are many different kinds of reviews, and the kind that will suit you best will depend on a variety of factors, for instance, how much time you have, as well as what kind of research question you’re working with. Cornell University Library has created a decision tree that can help you determine what kind of review might best suit your goals and constraints.

How much time do you have? It can take between 6 months and 2 years to conduct a systematic review. If you have less time than that available, you may want to conduct a rapid review instead. 

Does your team have the expertise required? Apart from field-specific knowledge and awareness of research methodology, you may need people in your team who are experts in statistical analysis and literature searches. To reduce risk for bias, it is also recommended that at least two people, independent of one another, review all found articles and choose which studies to include in the review. 

Do you have access to the necessary tools? Apart from suitable databases in which to conduct your search, you will also want access to a program that can help you manage your references, such as EndNote. You may also want to use a program that can help you screen your references, such as Rayyan. Find additional examples of useful tools at Systematic Review Toolbox

Is a similar review already in the works? Conduct a search in PROSPERO, for example, to ensure that there are no ongoing projects that resemble yours. 

Protocol - Plan on writing a protocol. The protocol can be registered, in PROSPERO for example, or in another open repository, such as Open Science Framework or Figshare

Start with a clear research question 

A key starting point for a systematic review is a clear, carefully delimited research question. There are several different frameworks available that might be able to help you structure and delimit your research question. The University of Plymouth Library has a good website where you can read more about the most commonly used frameworks. In clinical research, it is common to use the PICO-structure: Population, Intervention, Control and Outcome. 

Create search blocks

To ensure that your research question is “searchable,” identify its key elements. You will use these to create search blocks that will then form the basis for the search strings used in the different databases.

Let’s take the following research question as an example: 

•    Does routine use of inhaled oxygen in acute myocardial infarction improve patient-centered outcomes, in particular pain and death

In the example above, we’ve marked some of the key elements in the research question that could potentially create useful search blocks. Note that you would only rarely use all parts of the PICO-question in a search; most of the time, you will focus on population and intervention. In the example above, the search blocks would most likely consist of Population = patients with heart infarction, and Intervention = inhaled oxygen. 

A general principle is that a systematic search should consist of only a few blocks. The more search blocks you have, the narrower your search will be; the narrower your search, the greater your risk of excluding relevant articles.  

Develop a search strategy 

An important aspect of a systematic review is an exhaustive literature search. The search should have high sensitivity, that is, it should be created in such a way that the search will retrieve all relevant articles within a research area, or at least discover as many as possible. This is also called conducting a wide search. A wide search means that not all search results will be relevant. In systematic literature searching, a precision of two-three percent is common. In contrast to a wide search, a narrow (or precise) search will result in a larger share of relevant hits, but you also run the risk of missing relevant studies. In the picture below you can see how the different search strategies compare to one another.

 

Wide search
+ Finds a significant part of the relevant literature
- Will most likely produce a large amount of irrelevant hits

Narrow search
+ Does often have a greater accuracy
- Loss of relevant literature

A systematic search strategy is often constructed using several different search blocks, in which every search block contains both subject headings and free-text terms. The search strategy will usually be translated to several different databases. Sometimes grey literature, such as dissertations or clinical trials, is included in the search as well. The database search can also be supplemented with other search strategies, for instance handsearching in selected journals, reviewing reference lists or undertaking a citation analysis. 

It is recommended that your strategy be reviewed by at least one other person before the final literature searches are conducted. The following checklist may be helpful during this review: Peer Review of Electronic Search Strategies

Get started with your literature search 

It’s often a good idea to start with a slightly less structured search in the various databases, a so-called test search. When you test search, you simultaneously discover what terminology is common in the field, and thus find more search terms.

At this point, it’s a good idea to see if there are any systematic reviews within your research area that have already been published. Sometimes published reviews will include their search strategy in the appendix.

You should also identify a few key articles, that is, the most significant studies within your research field. These should ideally be articles that correspond to your research question as closely as possible. You can use these key articles to both construct your own search strategy and to test it: if your search does not retrieve your key articles, then your strategy needs to be modified. 

You may also want to see if there are any validated search filters you can use. Search filters are sets of search terms chosen to restrict a search to a selection of references, such as articles based on method or study type. 

Websites with validated search filters

Cochranes search filter for RCT-studies can be found in Technical Supplement to Chapter 4: Searching for and selecting studies 3.6 Search filters (p. 58-63)

InterTASC Information Specialists' Sub-group Search Filters Resource is a collaborative resource where you can find many search filters, both published and unpublished. 

Some search filters are also integrated in databases such asPubMed/MEDLINE, PsycInfo och CINAHL. In PubMed, for instance, you can take a look at Clinical Queries. Here, you can use search filters to delimit your search to clinical trials, for instance, or studies about Covid-19. 

Finding search terms

Determining what the relevant search terms are is an important part of the systematic search. It’s often a good idea to start with a test search where you use the subject headings and synonyms you’re already familiar with. By scanning titles, abstracts and subject headings, you may find additional, useful search terms. 

To ensure that your search retrieves as many relevant studies as possible, you should use both subject headings and free-text terms in your search. 

Subject headings are used in databases to tag all articles on a subject. When you include the subject heading in your search you will find articles about a subject even if the authors of the article have chosen other, adjacent words to describe the article themselves. 

It may take a while before articles in a database are tagged with subject headings, so in order to locate newer articles you also need to include free-text terms in your search. Some databases, for instance Web of Science, do not have a list of subject headings at all; to conduct a search there, you have to use free-text terms. 

Other ways to conduct a search 

For some topics and research questions, it might be appropriate to supplement your database search with other kinds of search strategies. This might include handsearching selected journals, or doing a citation analysis. 

If you want to include grey literature in your systematic review, there are particular databases you will want to search. Often you will need to use a modified and simplified search strategy to search for grey material. To find government reports or similar texts, you may even have to go to the organization’s website and search directly among their publication lists. 

In combination with systematic searches, it is also recommended that you do a forwards and backwards citation searching on the studies included in your review. That is, that you review the citations and references for the included publications. A variety of tools and databases allow you to search for citation data, including Web of Science, Google Scholar and SpiderCite

Search examples from different databases 

When conducting a search for a systematic review, it is recommended that you search in several different databases. In biomedicine and health, the most commonly used databases are PubMed/MEDLINE, Embase and Cochrane Library. Depending on the topic, you may want to supplement your search with a multi-disciplinary database like Web of Science, or subject-specific databases, such as CINAHL, PsycInfo or ERIC.

Always use identical search blocks and free-text search terms in all the databases. Subject headings, however, will need to be adjusted to each database’s controlled vocabulary list. You will also need to adapt field tags and other symbols. 

In many databases, the default setting is that the search is conducted in all fields or in a combination of several fields. However, in a systematic literature search, it is recommended that you specify the search field manually instead. This will give you more control over the search and will also make your search strategy more transparent. 

Apart from the tips provided below, KIB’s search team has also developed a database sheet as support while conducting a systematic review. Other tools can help you translate the search syntax used in different databases, for example, Systematic Review Accelerator Polyglot
 

Documentation

PRISMA

As with any type of research, the review process in a systematic review should be transparently documented in all parts, clearly reported in the final publication, and reproducible.

To help you do this, follow established guidelines, such as those found in the PRISMA Guidelines (Preferred Reporting Items for Systematic Reviews and Meta-Analyses).
The guidelines describe how systematic reviews should be reported. PRISMA 2020 consists of a checklist with 27 points and several flowcharts

According to PRISMA’s checklist, all databases, registries and other sources used to find articles should be reported, in addition to the date when each source was last searched. The search strategy should be documented in full for each database, and can be published in an appendix to the published review article. 

There is a particular checklist for how the search itself should be reported: PRISMA-S

Update your search before publication 

According to the Cochrane Handbook, a search should be updated before publication if more than twelve months (ideally, six months) have passed since the original search was conducted. Journals also often demand that a search be updated if several months have passed since the article was submitted. 

A quick way to find new articles is to repeat the search and delimit it to a certain interval of publishing dates. If you do so, however, you risk missing material that may have been added retrospectively to the database. A safer way is to delimit your search to the dates when articles were last added to the database, or use EndNote to deduplicate your new references.

Find out more: 

Reference management 

Systematic literature searches tend to generate a great deal of references. Since you search many databases, you will also likely end up with duplicate references. 

To handle this amount of references, we recommend learning how to use a reference management software such as EndNote. In EndNote, you can both organize your references and get rid of duplicates. 

Screening

During the screening process you may want to get help from specific programs developed to review article abstracts as well as full-text articles. They are often collaborative and tag, for instance, inclusion- and exclusion criteria. At the moment, KI does not have a license for these kinds of programs. A few examples:

At the library we offer workshops on how to get started using Rayyan. McGill Library also has a helpful guide to Rayyan.   

Contact us

If you're a researcher or a doctoral student preparing a systematic review, please contact the search group.

Hand in front of computer

Search requests for researchers

If you are a researcher and need help with literature searches, for example in a systematic review, please contact the search group.

Did you find the information on this page useful?

If you would like us to get back to you, please submit your contact information in the form below along with your feeback.

I agree that my (optional) personal data (name, e-mail) is stored in accordance with the Data Protection Regulation (GDPR). The information is only used to contact the person using the form.
Last updated: 2023-12-11