Structured literature reviews – A guide for students
This is a step-by-step guide aimed at Master's students undertaking a structured literature review as part of their Master's thesis.
There are several different kinds of literature reviews, but any literature review typically includes an extensive literature search. Whenever a systematic approach is used, the literature search features a methodical step-by-step procedure. However, as a Master's student, it might not be possible to fulfill all the criteria of a systematic review when writing a literature review-based thesis; you should rather do a structured literature review, which will include only certain aspects of the systematic review methodology.
In this guide we will go through the different steps of a structured literature review and provide tips on how to make your search strategy more structured and extensive.
Step 1: Formulate and delimit your research question
- It will be much easier for you to perform a structured information search if you first define and delimit your research question in a clear way.
- One way to define and structure your question is to break it down into different parts.
- PICO and PEO are two different frameworks that can be used for breaking down a research question into different parts.
- You also need to define the most important key concepts of your research question.
The formulation of your research question is partly connected to what kind of literature review you are doing. This article by Maria J. Grant and Andrew Booth usefully compares different kinds of reviews. While a systematic literature review is usually grounded in a clearly delimited and structured question, a scoping review may, for instance, feature a wider problem formulation. The wider a research question is, the larger number of search hits it tends to generate.
To be able to perform a literature review, you need to consider a subject area in which there seems to be a sufficiently large number of original research studies. Therefore, it may be a good idea to test search a database for previous research on the subject while you are trying to formulate and delimit your research question.
One way to structure your research question is to break it down into different parts. A well-delimited question often consists of three to four different parts. PICO and PEO are two examples of frameworks that can help you identify and define your research question.
- PICO (Population, Intervention, Comparison, Outcome) is primarily used for quantitative research questions.
- PEO (Population, Exposure, Outcome) is primarily used for qualitative research questions.
Structuring your research question in accordance with a framework, such as PICO or PEO, will also help you decide on the inclusion and exclusion criteria of your literature review.
After you have delimited your research question, you also need to identify the key concepts that make up your question. Based on these key concepts, you will create "search blocks" that you will use to organise your search terms.
Step 2: Find search terms and create search blocks
- Test searching is a good way to investigate the terminology of a subject area and find search terms.
- Reading key articles can help you gather additional search terms for your final search strategy.
- Find subject headings for Pubmed with the help of the US National Library of Medicine's MeSH database.
- Find free-text search terms by investigating what words that occur in the title and abstract of relevant articles.
- A good way of achieving a structured final search query is to arrange your search terms into search blocks; these blocks should arise from the key concepts of your research question.
While working on a literature review-based thesis, you will need to search for articles on several occasions. In the beginning of your project, it is often good to do a couple of unstructured and simple search queries, so-called test searches, in academic databases. This way you are off to a good start, as test searching helps you investigate the terminology of your subject area and find relevant search terms. While the final search strategy is typically reported in full, you don't need to present your test queries in your thesis.
Try to find a couple of key articles, that is, articles that correspond to the type of studies that you are planning to include in your review. Use key articles to gather additional search terms for your final search strategy. Analyse the terminology of your key articles by examining what subject headings (MeSH terms, etc.) that the articles have been tagged with and what words that occur in the titles and abstracts.
To retrieve as many relevant studies as possible, you will need to include free-text search terms as well as subject headings in your final search strategy. Free-text search terms are words that occur in the article's title and abstract – words used by the authors themselves. Subject headings are subject-related words that an article is tagged with when the article is added to the database.
- In Pubmed, articles are tagged with MeSH terms (Medical Subject Headings). You can look up and browse MeSH terms in the US National Library of Medicine's MeSH database.
- Databases such as Cinahl, PsychInfo, ERIC, and Sociological Abstracts have their own subject heading lists; look up subject headings in each database's subject heading list.
- There are also so-called free-text databases, such as Web of Science. These databases lack subject heading lists. Hence, when searching a free-text database, you can only use free-text search terms.
An effective way to increase the structure of your final search strategy is to arrange your search terms in so-called search blocks. Create your search blocks based on the key concepts of your research question.
This search strategy worksheet (.docx) might help you document and organise your search terms.
Step 3: Search in a structured way
- To get a comprehensive search result, you will need to search for articles in several different databases.
- Your search strategy should be as uniform as possible in every database, but you may have to adapt your use of subject headings.
- As you search the databases, combine your search terms and blocks with the help of AND and OR.
- Save time by documenting your search queries.
When doing a literature review-based thesis it is often wise to use at least two different databases. Many databases overlap, but may also contain unique content. At KI it is common for Master's students to use Pubmed and Web of Science when doing a structured literature review as part of their Master's thesis. Depending on your research question, other databases may also be appropriate and useful. Read more about the most frequently used databases at KI.
Your search strategy should be as uniform as possible in every database. However, as mentioned in Step 2, databases may use different subject headings, and some databases only let you use free-text search terms. This means that you need to adapt your use of subject headings depending on the database.
Example: How subject headings may differ between databases
If you want to search for articles about day surgery in Pubmed, you should use the MeSH term Ambulatory Surgical Procedures. However, if you also want to perform your search in a database such as Cinahl, you need to use the corresponding Cinahl Headings term instead: Ambulatory Surgery.
There are many different ways of searching databases. Most databases have one simple, basic Google-like search box and one advanced search form. One advantage of the latter is that combining search terms with AND and OR is usually easier in an advanced search form, especially if you will be using both AND and OR within the same search query. However, you can often combine search terms with AND and OR in a basic search box too, and in that case, you often isolate your different search blocks from each other by enclosing each block in parentheses.
Example: A search query that contains AND, OR, and parentheses
(inflammatory bowel diseases OR ulcerative colitis OR crohn disease) AND (adolescent OR child OR young adult OR teenager) AND (self-management OR self care OR self efficacy)
By choosing the advanced search form you will also be able to exert more control over your search process. The advanced search form lets you specify more closely and decide exactly how you want the database to interpret your search terms; this way you can make your search query more precise.
You should always document your search strategy in order to remember what search terms you have used, how these search terms have been combined, and whether you have applied any limits to your search. The easiest way to do this is to copy and paste your search history from the database into a text document. Also, academic databases often let you create a personal account, so that you may save your searches online.
Step 4: Improve your search strategy
- Briefly examine your search results to see if you need to make any changes or improvements to your search query.
- Investigate whether your key articles are present in the search results.
- By using the advanced search form you can improve your search.
Prepare yourself for having to modify and redo your search query several times, before deciding on your final search strategy. After you have combined all your search terms and made your very first database search, you should examine the search results and analyse whether your search query is able to generate the type of search hits that you are looking for.
Analyse your search results
- Are all your key articles present in the search results, or are there some key articles that your search query is unable to retrieve?
- Are you getting too few search hits? Investigate why. Perhaps you need to remove one of your search blocks, add one or several synonyms within a search block, or search for parts of words by truncating one or several of your free-text search terms, in order to broaden your search?
- Does your search strategy generate too many non-relevant search hits that have nothing to do with your research question? Investigate why. Perhaps you need to add another search block, remove one of the synonyms from one of your search blocks, or search for phrases by enclosing one or several of your free-text terms in quotation marks, in order to narrow your search?
- More tips on how to improve your search strategy.
It is important to remember that there is nothing wrong, per se, if your search query generates irrelevant hits. This is quite normal when performing a structured literature search. What's important is that your search strategy is able to retrieve the type of articles that you are looking for, and that you are not overwhelmed by the total number of hits (given the time frame of your thesis project).
We recommend that you use the advanced search form when improving your search strategy. By using the advanced search form, you will be able to specify which search fields your search terms must be present in; this is often a good way of retrieving more relevant and less irrelevant search hits.
In the box below you will find an example of how one might improve the search strategy that we presented as an example in Step 3.
- We have made our search query somewhat more specific, by searching in Pubmed for MeSH terms in the MeSH terms field and for free-text search terms in the Title/Abstract field. This will give us less hits, because the results will no longer include those articles in which the search terms are present somewhere else in the reference. For example, the new query will not retrieve articles in which the search term is present merely in the name of the publishing journal. This is usually an effective way of reducing the number of non-relevant hits.
- Another reason why our improved search strategy will generate fewer hits is because, when Pubmed interprets the query, the database searches for the free-text terms in the form of phrases in the Title/Abstract field. This means that, if you were to search for key concepts that contain two or more words (for example self care), you would only retrieve the articles in which those words were written next to each other, in that exact order.
- We have also truncated the free-text terms (by adding asterisks), in order to find more variants of the words.
Example: An improved search query in Pubmed
(Inflammatory Bowel Diseases[MeSH Terms] OR Inflammatory Bowel Disease*[Title/Abstract] OR Crohn*[Title/Abstract] OR ulcerative colitis[Title/Abstract] OR IBD[Title/Abstract])
(Adolescent[MeSH Terms] OR Child[MeSH Terms] OR Young adult[MeSH Terms] OR Adolescen*[Title/Abstract] OR Child*[Title/Abstract] OR Young Adult*[Title/Abstract] OR teen*[Title/Abstract] OR youth*[Title/Abstract])
(self-management[MeSH Terms] OR self care[MeSH Terms] OR self efficacy[MeSH Terms] OR Adaptation, Psychological[MeSH Terms] OR Self-Management[Title/Abstract] OR Self Care[Title/Abstract] OR Self Efficacy[Title/Abstract] OR Psychological adaption[Title/Abstract])
Step 5: Select and review articles
- After you have completed your search, you will need to go through all your search hits and select which articles to include in your review.
- When selecting articles, read through the titles and abstracts of each article to decide its relevancy.
- Check the quality of each study that you include in your review.
- When checking the quality of articles, it is common to use critical appraisal worksheets or checklists.
When you have completed and feel satisfied with your search, it is time to go through all the search hits and select which studies to include in your review. All relevant studies, that is, those studies that correspond to your research question and your previously set inclusion criteria, should be included. You decide on the relevancy of a study primarily by reading through the title and abstract. If you feel unsure, go through the whole article. You can describe your selection process with the help of a flow chart, such as the frequently used PRISMA flow diagram.
One of the challenges of systematic literature searches is that the search strategy should be exhaustive, but at the same time the number of search hits also needs to be kept within reasonable boundaries. A search query needs to be broad enough to retrieve all relevant studies, but on the other hand, this also means that a large portion of the search results will be irrelevant. Hence, even though your search strategy may have generated hundreds of hits, it is fine to only include ten to twenty articles in your review in the end.
If you create a personal account in a database it will be easier for you to save any references that you may find there. Another way of saving and organising article references is to use reference management software. There are several different reference management software, for example Endnote Online and Zotero.
When you have made your selection, you should critically examine the quality of all articles included in your review. The assessment is typically performed with the help of a critical appraisal guide or checklist. The purpose is to assess the reliability of the study results and whether there are any methodological flaws that may have impacted the results. Qualitative research articles are often reviewed with a focus on authenticity, credibility, and validity.
There are many different critical appraisal worksheets and checklists. Some examples are the SBU checklists for assessing the quality of randomized studies, observational studies, and qualitative research. In the course book How to do a systematic review in nursing there is a review guide that can be used for assessing different kinds of studies (both qualitative and quantitative); the original source is Caldwell, Henshaw & Taylor, 2011.
Review worksheets and checklists contain criteria and questions that may help you identify flaws, errors, or bias. Sometimes different aspects of the study are scored separately. Later, all scores make up a final score that indicates whether the study is of high, medium, or low quality.
Many programmes and courses provide instructions on which checklists to use when reviewing articles, so check your course guidelines.
Step 6: Report your search strategy
- Describe your search strategy in a manner that makes it possible for your readers to replicate the search and get the same results.
- The search strategy is often presented in the form of a table.
- Look at the search history to see what words and limits that you have used when searching a database.
An important aspect of doing a structured literature review is transparency. It has to be easy for your readers to follow what you did when you searched and selected the articles that you have included in your review. In the method section of your literature review you should describe how you searched different databases. This is also where you describe any manual searches that you did. Search strategies are commonly reported in the form of tables. Present one table for each database.
You can examine your search terms and any limits you have applied when searching a database by visiting its search history.