Why I Stopped Outsourcing My Literature Reviews
Three years ago I was hiring people to do systematic searches for my projects. Cost roughly $400 per review cycle, took two weeks turnaround, and more than once the results were incomplete or missed a key paper. I stopped doing that. I figured out my own process and it cut my time from about two weeks down to roughly four days for most standard reviews. It's not glamorous but it works. The first step is picking your database and getting a handle on search syntax. PubMed and Scopus are the main ones for most fields. Web of Science works too but its export features are annoying. You need to learn how boolean operators work and how to use wildcards. Most people skip this part and end up drowning in 5,000 results or missing half the relevant papers because their search was too narrow. I use a simple framework. Define your research question clearly. Identify the key concepts. Map each concept to synonyms and related terms. Build your search string by combining these with OR within each concept and AND between concepts. Test it in one database first, note the result count, then replicate across others. Adjust for each database's unique field tags.
Once you have your search string, run it and export the results. I use CSV format. Then import everything into a reference manager. Zotero works fine. EndNote is more expensive but handles large libraries better. Mendeley has gotten worse over the years with its forced sync and data issues, so I don't recommend it anymore. The key thing is to keep all results from all databases in one place. People often forget to merge duplicates properly and end up with the same paper appearing five times because it was indexed in multiple journals. After deduplication comes screening. This is where most DIY attempts break down. The best approach is a two-stage process. First, screen titles and abstracts. If a paper doesn't clearly relate to your topic, exclude it. If it might be relevant, include it for full-text review. Second, pull the full texts and decide based on the actual content. I aim for about 90% exclusions in the title/abstract phase. If your exclusion rate is under 60%, your search is probably too broad. If it's over 95%, you're being too aggressive and likely missing relevant work. Data extraction is the next step. Set up a simple spreadsheet with columns for author, year, study design, sample size, key findings, and quality assessment notes. I used to try fancy coding systems for this but they added complexity without adding value. A clean spreadsheet with consistent criteria for each column works better for most people.
Quality assessment matters but people either ignore it or overthink it. For randomized studies use the Cochrane Risk of Bias tool. For observational studies use the Newcastle-Ottawa scale. Don't bother with quality assessment tools if you're doing a scoping review instead of a systematic review. Those serve different purposes and the tools don't apply the same way. One common mistake I see is people applying a RCT quality tool to qualitative studies. It doesn't make sense. Writing the review itself follows a standard structure. Introduction, methods, results, discussion. The methods section is where most DIY reviewers fail because they don't document enough detail. You need to write it so someone else could replicate your exact search. Include the database names, the date you searched, the full search string for at least one database, your inclusion and exclusion criteria, and how you handled disagreements or screening decisions. I ran into a specific problem a while back that illustrates why documentation matters. I was reviewing a topic and my search string kept pulling in a lot of papers about a related but distinct methodology. I thought my search was flawed so I tweaked it multiple times. It took me a week to realize the issue wasn't the search string. The field I was searching had a terminology shift around 2018. Papers before that used different keywords for the same concept. My fix was adding a date range split and searching the older terminology separately. This is the kind of thing you won't learn from a tutorial. You learn it when you spend a week dealing with it.
Get the Full Details
Here's a counter-intuitive thing about literature searches. More databases does not always mean better results. PubMed covers the life sciences well but its coverage of certain regional journals is limited. If your topic involves non-English literature or specific regional research, you might actually get better results by focusing on one well-chosen database and investing more time in refining your search strategy there. I've seen people jump between five databases, run each search poorly, and end up with worse coverage than someone who did one thorough search in a single database. Another thing people miss is that your search string should evolve. You refine it as you screen. When you find a highly relevant paper, look at its references and the papers that cited it. This backward and forward chaining can fill gaps that your initial search missed. The PRISMA guidelines acknowledge this. They don't require you to stick rigidly to one search string from start to finish. Iteration is expected and normal. The biggest bottleneck in DIY literature reviews is consistency. When you're screening hundreds of papers alone, your criteria inevitably drift. I set a rule for myself: if I'm unsure whether to include a paper, I include it. It's cheaper to screen out a few extra papers in the full-text phase than to miss something relevant. Another practical tip is to take breaks between batches. Screen 50 papers, step away for 15 minutes, then come back. Decision fatigue is real and it makes you either too loose or too strict with your criteria.
There are scenarios where DIY doesn't work well. If you need to produce a formal systematic review for a registered protocol, ideally you should work with someone who has done Cochrane or similar training. The risk of missing a bias assessment step or misclassifying a study is too high otherwise. Also, if your review covers more than 20 databases or spans very specialized fields outside your expertise, partnering with a librarian or professional researcher is worth the cost. But for most student projects, thesis work, and general research questions, the DIY approach is completely viable. I still get asked where to find templates and guides. The PRISMA website has the flow diagram templates and the checklist. Cochrane's Methods for Reviewers handbook is free online and covers everything from search strategy through to risk of bias assessment. It's dense but it's the standard reference. JBI also has good guidance documents for different types of reviews. Bookmark those and stop looking for shortcuts. The whole process for a typical undergraduate or graduate level literature review in my experience takes about 8 to 12 hours of focused work spread across a week. That includes building the search, screening, extracting data, and writing it up. People who say it takes months are usually either doing something unnecessarily complicated or not working efficiently. The bottleneck is almost always the screening phase, not the writing. If you're spending most of your time reading papers instead of organizing them, restructure your workflow.
One more practical note on tools. There are programs like Rayyan for collaborative screening and EndNote for reference management. They help but they're not essential. The core of a good literature review is a clear question, a transparent method, and honest reporting of what you found. No software will fix a poorly defined research question. I've seen people spend three weeks setting up elaborate reference management workflows only to write a review that couldn't answer a coherent question. Start with the question. Everything else follows.
