Automated Literature Review System

Based on Sam's ChatGPT literature review workflow.
A tool for analyzing SCOPUS exports with automated ranking and visualization.
ChatGPT is still needed for key findings, but you can now focus on only the top ranked papers.
Tested on a Scopus CSV export with ~5000 papers.

Features

• Publication Analysis: Visualize publication trends by year, document types, and keyword frequencies
• Custom Ranking System: Rank papers based on user-defined keyword weights

Requirements

uv sync

Required packages:

• pandas: Data manipulation and analysis
• matplotlib: Basic plotting functionality
• numpy: Numerical computing

Usage

1. Prepare Your Data

Export your literature search results from SCOPUS as a CSV file. The system expects these columns:

• Year: Publication year
• Title: Paper title
• Abstract: Paper abstract
• Author Keywords: Keywords provided by authors
• Index Keywords: Keywords assigned by database
• Document Type: Type of publication (Article, Review, etc.)
• DOI: Digital Object Identifier

2. Create Ranking Criteria

Create a rank_index.csv file with your ranking preferences. Example format (comma to separate the weight and semicolon to separate keywords):

Weight,Keywords
10,machine learning; artificial intelligence; deep learning
8,neural networks; computer vision; image processing
6,natural language processing; text mining
5,data mining; big data; statistics

Higher weights indicate more important keywords for your research focus.

3. Run the Analysis

python main.py your_scopus_export.csv

4. Results

The system will generate:

• Publication year histogram (saved to plots/publication_year_histogram.png)
• Keywords frequency bar chart (saved to plots/keywords_frequency_bar.png)
• Document type pie chart (saved to plots/document_type_pie.png)
• Ranking comparison scatter plot (saved to plots/rankings_comparison_scatter.png)
• Literature summaries in <original_file_name>_ranked.csv

5. Relevance calculation details

The relevance score in the code is computed from three normalized components derived from the dataset.

• Keyword Match Score

  • Calculated only from the "Index Keywords" and "Author Keywords" fields.
  • The file rank_index.csv supplies criteria rows with Weight and semicolon-separated Keywords. For each publication the code:
    • parses and normalizes publication keywords (lowercased, semicolon-separated),
    • counts matches with each criterion's keywords,
    • multiplies matches by the criterion weight,
    • sums across criteria to produce Rank_Index (index keywords) and Rank_Author_Keywords (author keywords).

• Citation Count Score

  • The raw citation count (Cited by) is transformed with log(cited_by + 1). This produces Rank_Citations.

• Publication Year Score

  • Year-based raw score is computed as max(0, (Year - 1990) * 2) producing Rank_Year (so more recent years get higher raw values).

Relevance Score Calculation

• The code min-max normalizes Rank_Index, Rank_Author_Keywords, Rank_Citations, and Rank_Year to [0,1].
• Combined keyword score = (normalized Rank_Index + normalized Rank_Author_Keywords) / 2.
• Final Relevance_Score = combined_keyword_score + normalized_citations + normalized_year.
• Because components are normalized to [0,1], the final score typically ranges from ~0 to 3 (higher = more relevant).

6. Image examples

a. Publication year histogram

b. Keyword frequency

c. Document type distribution

d. Ranking comparison scatter plot