Score-based Likelihood Ratios For Authorship Verification with Authorship Embeddings

Environment

Set up Anaconda environment:

conda env create -f environment.yml

Activate environment:

conda activate text-forensics

Paper Results

• Experiment results are stored in the results/ directory
• paper-analysis.ipynb reproduces the figures/tables from the paper using these stored results
• Contents of the results/ folder can be reproduced by downloading the datasets into the datasets/ directory, preprocessing the Amazon dataset (others are already formatted), and then running the experiments.py code using the commands in run_experiments.sh

Contents

• datasets/: this directory contains the downloaded datasets
  • DarkReddit Zenodo link (must request access from original authors Manolache et al. 2023)
  • SilkRoad Zenodo link (must request access from original authors Manolache et al. 2023)
  • Agora Zenodo link (must request access from original authors Manolache et al. 2023)
  • Amazon DropBox link (publicly-accessible, original authors Halvani et al. 2017, link retrieved from Ishihara 2021)
  • the preprocessed Amazon review dataset into pair format
• preprocessing/:
  • process_amazon.py: python code for preprocessing the Amazon dataset into AV pairs
• models/:
  • embeddings.py: implements the pre-trained authorship embedding models
    • LUAR HuggingFace link
    • CISR HuggingFace link
  • models.py: defines the SLR classes and implements training/test methods
• utils/lexical.py: implements the lexical/character features used in the manual SLR
• experiments.py: python code for running the experiments
• base_logger.py: sets up logging for status updates on experiments
• run_experiments.sh: script with command line arguments for running experiments
• results: stores experiment results
• paper-analysis.ipynb: notebook for reproducing results from the paper
• figs/: stores figures from analysis