Skip to content

ghextractor CLI not honoring configuration #7

Description

@mminella

I am working with ghextractor and have the following directory structure:

(base) ➜  github-export ls -al
total 8
drwxr-xr-x  4 mminella  staff  128 Apr 22 08:40 .
drwxr-xr-x  6 mminella  staff  192 Apr 22 08:39 ..
-rw-r--r--  1 mminella  staff  165 Apr 22 08:40 .ghextractorrc
drwxr-xr-x  6 mminella  staff  192 Apr 22 08:45 data

The contents of the .ghextractorrc file is as follows:

{
  "format": ["markdown", "json"],
  "output": "./data",
  "backup": false,
  "rateLimit": 6000,
  "cache": true,
  "diffMode": false,
  "forceFullExport": false
}

Even though I have configured the rateLimit value to be 6000, it still stops at 2000 on my run. I have tried using the environment variable as well with no luck. Full output of a sample run can be found below:

┌  GitHub Extractor CLI
│
●  Scanning repositories...
│
◆  Found 589 repositories
│
◇  What would you like to export?
│  Issues
│
◇  Select a repository:
│  micrometer-metrics/micrometer
│
●  Selected: micrometer-metrics/micrometer
│
◇  Select output format:
│  JSON
│
◇  Enter output path:
│  ./data
⠋ Exporting Issues...[INFO] Starting export for micrometer-metrics/micrometer...
⠦ Exporting Issues...[INFO] Fetched 2000 items
⠋ Exporting Issues...[OK] Export completed: 2000 exported, 0 failed
✔ Issues export completed!
[WARN]
[WARN] ⚠️  Partial export detected:
[WARN]    ✗ Pull Requests: missing
[WARN]    ✓ Issues: 2000 files
[WARN]    ✗ Commits: missing
[WARN]    ✗ Branches: missing
[WARN]    ✗ Releases: missing
[WARN]
[WARN] 📊 Analytics options:
[WARN]    1. ⏭️  Skip analytics (recommended)
[WARN]    2. ⚠️  Generate partial report (data quality affected)
[WARN]
[INFO] 💡 To generate comprehensive analytics:
[INFO]    Use "Full Repository Backup" option
[INFO]    This exports all data types needed for accurate metrics
[INFO] Generating analytics report for micrometer-metrics/micrometer...
[INFO] [OFFLINE] Using offline mode: parsing exported markdown files...
[INFO] [PARSED] Loaded 0 PRs, 2000 issues from markdown files
[INFO] [MERGED] Found 0 merged PRs
[INFO] Generating advanced analytics (Review Velocity, Trends, Correlations, Projections)...
[INFO] Validating report for numerical consistency...
[INFO] Validating analytics report for numerical consistency...
[OK] ✅ Report validation passed (25/28 checks)
[INFO] [PASSED] Report validation passed with 3 warnings
[INFO] Comparing metrics against industry benchmarks...
[INFO] Comparing repository metrics to industry benchmarks...
[INFO] Generating executive narrative and insights...
[INFO] Generating executive narrative...
[INFO] Analytics report saved as JSON: data/micrometer-metrics-micrometer-analytics.json
[OK] [OK] Analytics report generated in 0.69s

📊 Export Summary:
──────────────────────────────────────────────────
✔ Status: Success
📦 Total Items: 2000
✔ Exported: 2000
🌐 API Calls: 1
⚡ Rate Limit: 5000/5000 remaining
⏱️  Duration: 21.79s
──────────────────────────────────────────────────

│
◆  Files saved to: data/micrometer-metrics/micrometer/Issues
│
└  Thanks for using GitHub Extractor CLI!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions